War and Peace

If you took any time slice from my career, I could easily tell you if that time period was war time or peace time. Based on the projects, the management, my co-workers, I was either stressing out (war time) or bored out of my mind (peace time).

What I want to do with this post is contrast war and peace in a software creation environment.

War Time

It’s easy to know when you’re at war. There is a constant feeling of subtle panic every time you sit down. You leave the office drained, tired and quite often angry.

Here are some specific symptoms I’ve noticed from my career:

1. You have several outstanding production issues that are non-trivial.
2. You have abandoned quality in favor of quantity. (No unit tests, code review, etc.)
3. Stakeholders are running amok. You are strongly encouraged to say “yes” to everyone.
4. You are missing management to help shield you. You have to make managerial decisions, sit in meetings and develop politics.
5. You are working extra hours to keep up.

There are of course more symptoms. Let’s look at peace time to help emphasize the divide between the two:

Peace Time

1. You have time to read articles about your industry – without feeling guilty.
2. You have time to write unit tests.
3. You have time to re-factor code per (2).
4. You have time to think about names for objects, classes, etc.
5. You have periods of focus that go longer than one hour. It’s normal to have four interrupted hours of time.


Now the point of this post is not to argue that developers should try to get to peace time. The point is not to decry the horrors of war. That is obvious.

The point is that as an organization, you need to know what state you’re in.

If the development team has bloodshot eyes and the stakeholders are playing Nerf basketball, then there is a major disconnect. One team thinks it’s peace time and the other is at war.

This is bad because expectations end up missing each other by wide margins. A peace time customer will ask for 100 features for a system. If he knew the development team was at war, he wouldn’t ask for 100 features. When you’re at war, compromises and sacrifices have to be made.

For any job I’m finding it’s important to know which state I’m in and then communicate that to others. It’s only fair that the stakeholders know that I consider us to be at war. Hopefully it helps them to adjust their expectations.

War and Peace

Battling Burnout

In many of my software jobs I’ve had to battle burnout. Things usually follow this progression:

1. Honeymoon period at job. Tons of time to do quality work up to my own standards.
2. I deliver on a few projects/features. I build a reputation.
3. The dam bursts and I drown in new projects/features.
4. I do my best to keep up and come up with solutions. I want to make people happy!
5. I burn out and quit the job.

Looking back on my career I think I had a huge part in not preventing my own burnout. I’ve tended to blame the jobs, the bosses, the bad management, but in reality I had the responsibility of keeping myself healthy and engaged.

Some things I now try to do to avoid burnout:

1. Keep eating healthy. This tends to go out the window when working too hard.
2. Keep exercising. Again, because I get so tired at work, this tends to suffer in times of burnout.
3. Push back on things that are not necessary. Saying “no” to meetings, projects, features.
4. Allowing things to fail. It’s ok to miss a deadline – usually it’s due to bad management/planning and not from me dropping the ball.
5. Reading articles, writing code not related to work. This is fun for me and it always goes out the window during stressful times at work.

We’ll see if I can learn from past mistakes and keep myself from exploding in flames. I’ve done it twice in the past and am keen to not do it again.

Battling Burnout

Oops x1.6 million

Recently I accidentally sent out 1.6 million emails to customers. I was doing local development that integrated with a third party and didn’t realize my iterative testing was actually firing off emails in production.

The vendor doesn’t have a staging/sandbox environment so extra care is always needed when doing development.

It was a pretty embarrassing situation and I tried to take a few things away from it:

1. If you are integrating with a vendor and they only have production, find a way to mock their environment.
2. Always create safe data structures for testing. Create duplicates with the third party that mimic real data structures, then swap to live objects when going live.
3. Be careful! There should be a weighty, important feeling to doing work with integration. If you’re feeling very casual and loose, then something is wrong.
4. Put in safe guards. In this case I made a whitelist of objects that could be operated on remotely. I also put in place several business rules that would prevent mistakes in the future.

Big mistakes mean opportunities to learn big.

Oops x1.6 million

Responsibility and Authority

At my last job I was the default MongoDB administrator. I lived in the Mongo shell all day for almost a solid year and I definitely used it more than any other developer, so that meant all things Mongo came to my inbox. Someone needs a document (or series of documents) updated? Send it to Ryan. Someone needs a new collection, needs data migrated from one server to another, needs to test connectivity, needs data exported? Send it to Ryan.

As our platform became more stable and more websites were deployed on top of it, Mongo began to wince a bit with the added activity. We started having odd issues with our production replica set and of course part of the investigation landed on my lap.

The point of this post is that even though the responsibility for Mongo was obviously mine, the authority to do real work was not. I could work with production data, but not SSH to the production machines.

How was I supposed to successfully see what might be causing issues in production if I couldn’t get on the box and look at the logs? How could I watch a process or look at the network activity? I couldn’t. I wasn’t given the authority to connect and do my work.

I’ve seen this happen several times in my career. I end up responsible for things, but have no power to do what I need to do in order to fix problems.

I’m happy to say that I’m getting better at identifying situations where I’m stuck being responsible without any authority to affect change.

Responsibility and Authority

Best Practice Exhaustion

I’m sure everyone has days like this – you feel a little worn out physically and exhausted mentally. I always know I’m extra fatigued mentally when I lose all interest in my work. Just like when I was racing mountain bikes, some days your mind refuses to go out and train. It’s too much. The idea of shirking your responsibility and doing anything except what you are supposed to do is irresistable.

While considering how I came to this point of fatigue, a new thought came to me in regards to development.

From time to time I believe I experience best practice exhaustion.

Work is filled with people that know best practices, but any idiot can read and remember rules. Most best practices start with the words “always” and “never”. Some examples:

1. Always check in your code to version control.
2. Always write unit tests.
3. Always do code reviews.
4. Never let an unanswered email fester.
5. Never let someone else drop the ball on a project.
6. Never use XYZ data structure, ABC pattern.

In software, it’s easy to compile a long list of best practices. Things that every good developer will always do.

Here are some implications of best practices:

1. You are responsible for any and all problems with a project.
2. If other people are lazy/non-responsive/not professional, that’s your fault too.
3. If a bug appears, and you could have prevented it, you’re guilty. Why didn’t you follow best practices?
4. If you’re exhausted at your desk and decide to stare aimlessly people wonder, why aren’t you working?
5. Not only are you responsible for coding, but you are responsible for project management, people management, specifications, deployments, etc.
6. Your velocity is always measured by code only and not these other factors.

So I find myself sometimes stuck in a bit of a rut. I take a task at work, I think about it, start developing and this laundry list of do’s and dont’s floods my thought process.

“Have I checked in every piece of code I’ve ever written? Even one off scripts? Even trivial database scripts?”
“Did I release something without a code review?”
“Did I log my time perfectly in XYZ tool?”
“Does everyone always know what I’m doing at all times? Did I send enough emails?”
“Have I thought about every possible boundary/use case no matter how obscure?”

I could go on and on and on.

The bottom line that summarizes this post is the statement: You never have any excuse for not doing something perfectly.

At least that’s how software engineering feels at times. I have found little grace and forgiveness amongst co-workers for simple human error.

We’ll see if there are solutions to the “no excuses” problem going forward.

Best Practice Exhaustion

Broken Windows

In the book Pragmatic Programmer the author outlines the theory of Broken Windows in regards to coding. Jeff Atwood has a nice article about the theory here.

Broken Window Theory: There was a study done that showed that when a building suffered a broken window, other windows were quickly broken and the building fell into disrepair. The theory is that if you leave something in a broken state – or neglect a flaw – the system as a whole will quickly degrade. The Pragmatic Programmer book argues the same thing happens with code.

Broken windows in regards to code look like the following:

1. Comments that have become wrong/incorrect.
2. Conventions being broken – for example inconsistent names for objects.
3. Lazy implementations that could easily be improved upon.
4. Unorganized code and data structures.

The idea is that as you work with a codebase, if you see some of the things above, you are likely to believe no one cares for the code anymore. There are broken windows all over, so why try hard to write good code? Why try to refactor existing systems and classes? Why do extensive testing?

Lately I’ve found myself falling into this mindset when working with legacy code. At the end of this article I’ve listed a few broken windows I’ve found in the past few months. At a low level in my brain I find myself making excuses for taking shortcuts with my code. There are so many broken windows, so why not skip testing everything I touched?

My resolution is that even though I might be working with legacy code, I need to uphold the same standards I would have for new code.

Some examples:

1. A comment in code:


2. A comment in a stored procedure in the database:

// COMMENTS: What is IndividualDates all about ? 

3. We need to be using source control:

// Check cache to see if cache exists

4. Justin needs to test:

//Added true so the menu is always fresh
//You can remove....but justin wants to test the menu

5. Updating!

            if (e.CommandName == "Up")
                if (indx > 0)
                    newCat = lCats[indx - 1];
                    upDateDB = true;
                if (indx < lCats.Count - 1)
                    newCat = lCats[indx + 1];
                    upDateDB = true;

6. Good luck reading that for loop.

            for (int i=0;i<lCats.Count;i++)
                if (lCats[i].MenuCategoryID == mCatID)
                    indx = i;
                    oldCat = lCats[i];

7. Long, long, long. Found a 700 line function to initialize a web forms .aspx page. 850 lines total in the class, so about 82% of the code in this class is dedicated to one method!

Broken Windows

Stubbornness with Tools

For my entire computing life I’ve been slow to adopt new tools. I always end up learning a program or tool and then stick with it no matter what.

Some quick examples:

Purpose Old/Comfortable Huge Advance
Writing Java vim Eclipse (even though I generally hate Eclipse)
Operating System DOS Windows 95
Html/Javascript Notepad Notepad++
Javascript Pure Javascript jQuery
Prototyping Temp .NET Projects LinqPad

The latest entry is using LinqPad to do quick C# prototyping. In the past if I wanted to test an algorithm or some simple function, I would create a new project, call it “Sandbox” and start coding. It would take several minutes to make the console project, make a bunch of static methods and then test. Then I’d have to open a CLI prompt and run it. Bleh.

With LinqPad I can start just banging out C# and run it without all the extra “stuff”. Awesome.

I heard about LinqPad years ago, but refused to look into it. Like all the other entries in my list above, it really comes down to pride.

Deep down in my brain there is this thought, “You’ve wasted a ton of time by not using XYZ tool!” It’s hard to admit that for years I wrote raw Javascript to accomplish tasks that jQuery could have done far better. It’s also hard to admit that I would notice other developers using Notepad++ and thought, “Ha! So weak. Notepad++. I don’t need that.” Now I can’t imagine being without it!

The lesson for me is to be quicker to adjust to new tools that can make me more productive.

Stubbornness with Tools

Dead Ends and Wasted Time

It’s easy to write about success, but not so easy to write about failure. Not just failure either, but unresolved failure.

Some quick specifics:

1. At work I tried to introduce a new library to a .NET project. Specifically the Thinktecture.IdenityModel.dll library.
2. In its assembly manifest it wants version of the Newtonsoft.Json.dll.
3. My project references lots of other projects with really, really, really old versions of Newtonsoft.Json.dll.
4. When old meets new, I get errors that .NET can’t find the version of Newtonsoft. The older version has won and is used across the board.

Unfortunately I lost an entire day trying to resolve this issue. I’m pretty sure some engineer out there would have a clean and clear solution for it, but I couldn’t find that guy online. =)

Things I tried:

1. Merging all DLLs Thinktecture needs into one mega DLL with ilmerge.exe. Didn’t work, but man ilmerge.exe was fun to work with. Good to know it exists.
2. Upgrading the references to the old Newtonsoft.Json.dll. Didn’t work because there are breaking changes between old and new (~2.0 and 4.5).
3. Explicitly referencing the new version in the web.config for my project. I found all sorts of ways to do this online and none of them worked.
4. Turning on assembly loading tracing. This was also really fun and interesting. Super cool to see how .NET is loading assemblies. Ultimately no help though.

When I’m stuck in a maze I tend to only go back 2-3 steps and then try a new approach. After burning through an entire day though, it’s time to take 10 steps back and try a different path.

Dead Ends and Wasted Time