The perfect is the enemy of the good (and the done)

Dear new developer,

When you are facing a problem, it can be very tempting to try to solve it perfectly. Handle all the possible edge cases, make it extensible, have it be configurable without code changes.

As with everything in life, there’s an opportunity cost to this perfection. If you’re spending days perfecting a single page of a webapp, for example, you won’t be able to move on to the next feature.

There are times for perfectionism, without a doubt. When data could be lost or security could be compromised, you want to move slowly and carefully.

For the rest of the time, good and done is better than perfect and undone.

Of course, the definition of ‘good’ is the crux of the above statement. Unfortunately, this is a judgement call, depending on context, team and risk. One thing that is non-negotiable is that the code meets the requirements. If it doesn’t work, then nothing else matters.

Some things to think about when you are considering what ‘good’ is:

  • How often will this be used? Once a second? Once a year? Once?
    • The more often code is used, the more it should be polished.
    • Note that usage often changes over time, so it’s worth keeping an eye on this and spend some time cleaning up code if it moves from the once a year category to the once a week category.
  • Is this a relatively static part of the system, or is it in flux?
    • If you’re touching this once and don’t expect to touch it again for a while, taking some extra time makes sense. Niggling bugs may live for years.
    • On the other hand, if this area of the codebase is churning, polish could be wasted because the code may be trashed.
  • What are the risk factors if this fails? Will someone be killed? Will money be lost? Will someone be inconvenienced?
    • Obviously, the higher the risk, the more care you should take.
  • What is the next thing on the list?
    • If you have other high priority issues, you may want to avoid polish.
  • What is the team quality ethic?
    • You should meet and improve that standard. For example, in the context of testing: if no one writes tests, set things up to write tests. If everyone writes unit tests, think about integration tests. If everyone writes integration tests, think about performance regression tests.
    • This is a great topic of conversation at an interview or team meeting so that everyone can be on the same page.
  • How permanent is this code?
    • The more permanent the code, the more care should be taken.
    • Some code lives on servers and is easy to change. Other code is shipped to mobile apps and is harder to change. Yet more is distributed to IoT devices and really hard to modify (when is the last time you updated your router’s firmware? I thought so.). Some is burned to read only memory and can’t be changed short of great expense.

One question that I’d avoid asking is “how will I feel about this code in 12 months”. Because I know the answer–you’ll cringe. At least, in all my discussions of running code, I’ve never met someone who revisited old code and didn’t find at least one thing to improve.

Two examples, rating the above factors on a scale of 1-5 (where 1 indicates you should just get it done, and 5 indicates you should get as close to perfection as possible).

  • Your website portfolio:
    • Usage: 3. Used maybe once a year or two (if you are an employee) or maybe every month (if a contractor), when someone is checking you out.
    • Static or churning: 4-5. relatively static. You probably build out an example app once.
    • Risk factor: 3. You’re looking for a job, and your portfolio is a powerful signal, but not the only one.
    • Next thing on the list: ?
    • Team ethic: N/A
    • Permanence: 1. This is a website, so you can ship new code whenever you have time.
  • Billing software on your large public site:
    • Usage: 5. Used many times a day.
    • Static or churning?: 4-5. Should be relatively static as payment processing doesn’t change that often.
    • Risk factor: 4-5. Billing mistakes are expensive.
    • Next thing on the list: ? Depends on the company.
    • Team ethic: ?
    • Permanence: 2. You have the code on the server, but you have a lot of history and state to maintain.

When you are working on code and struggling to get to ‘done’, I encourage you to consider the above questions to guide your decision. Finally, don’t forget that (most) code can be revisited. If something is good and done now, ship it. You can typically improve it next week or next year if the code needs it.

Sincerely,

Dan

Learn to look around corners

Dear new developer,

Sometimes you want to play out things two and three steps deep. Kinda like chess players, who think about many moves ahead, if you can consider the ramifications of your decisions, you’ll be well served.

I remember talking to someone about a software position at his company and he referred to the “original sin” of choosing a particular NoSQL database. This had ripple effects throughout the life of the company. Some were good, some were bad.

These kind of downstream effects are worth thinking about. The biggest ones are things like fundamental platform choices (Rails or Django, Java or golang, mysql or postgresql). But even smaller things can have ripple effects. I can’t count the times when I’ve said “I’ll just upgrade this one thing” and then the change just keeps cascading out.

Or if you are working on a feature definition, think through the ramifications. “We want to allow people to change their email address.” But if email address is the primary key of a table, or a primary means of connecting data across accounts, or some other invariant, what will the ripples in your system be.

And these are just the functional repercussions. There are also performance repercussions to changes, some of which are hard to test unless you are in production.

I’m not saying don’t accept changes. That way lies madness (and Fortran). Just be aware of how changes in complicated systems like web applications have a way of rippling out beyond their intended scope, and consider how to test, mitigate or handle such ripples.

Sincerely,

Dan

What if I have to make a technical decision and I don’t know the right answer?

Dear new developer,

Sometimes you are confronted with decisions for which you simply don’t know the correct answer. This has happened to me many times over the years. A recent example is that the client wanted to build an online quiz. They wanted to be able to edit quiz questions and answers. They wanted it to feel like an application (rather than a website). I had never built something like this before, but I was the person best situated to make the architectural decision which would underpin this application.

You may not have time or money to do all the research you feel you need. You may be doing something that has simply never been done before. You may be in an arena where you are being pressed for a decision (like a meeting).

If you are in a situation where you have to make a choice about a fundamental architectural decision (like a platform/library/framework), you probably are in a small team moving fast, or at a company where you are one of a very few number of technical folks. In this case you may feel over your head and incapable of making the correct decision.

And yet, the decision still needs to be made, and you are the one who has to do it. (The unfortunate fact is that the too late perfect decision is worse than the pretty good timely decision.)

Techniques that won’t work out well for large important decisions:

  • avoiding making the decision
  • googling for an answer and take the first option provided
  • deferring to someone else
  • doing a thorough full examination of all the possible solutions, draw up an excel spreadsheet and a powerpoint to make sure you haven’t missed anything
  • avoiding discussion among team members

Here’s how to proceed.

First make sure that you do need to make the decision, and that you have the proper business context. The number of choices that you should consider and amount of research you should do depend on the impact of the decision. Things to think about:

  • how much of the business does this touch? If it is contained or constrained, you can spend less time thinking about this. If you are implementing a small micro service, or if you are choosing a service that is only used by a portion of the organization (like github for your code), then a choice that needs to be unwound will be easier to do. If you are picking the development language that will be used for the core product, rewrites are almost guaranteed to be far in the future.
  • how irreversible is the decision? Most decisions aren’t irreversible given enough time and money, but some are easier than others. Swapping out one memecached provider for another is pretty trivial, but changing from mysql to postgresql is more complex. Changing from GCP to AWS can, depending on the amount of data you have, be person-years.
  • does your company have internal solutions that you might be able to leverage? The bigger the company, the more likely the problem has been solved before, and finding that solution will be quicker and better than rebuilding it.
  • can you defer the decision until later? Is there a manual process or a service you can buy that will avoid a technology investment, even for a few months?

If you can’t defer the decision, then make the best one you can with the information you have. Research based on the impact on the business. I want to acknowledge the uncomfortable tension between knowledge and action that are at the root of any decision made with incomplete knowledge (aka, all of them!). There’s a spectrum based on risk and commitment, and unfortunately, I’m not aware of any great heuristics for determining risk or commitment other than experience and reading.

My approach towards such a decision is:

  • learn what you can. This includes finding out as much as you can about the problem, including current solutions
  • bounce ideas off of others in the company, but make them specific (both in form and in who you ask). This will help avoid analysis paralysis
  • ask for recommendations. Start at your current company. Even if you can’t find someone who has direct experience with the problem, there may be people who have related experience. You can also google to see what other folks have said about the various solutions (make sure to limit the results to “the past year” or you’ll end up reading about old versions). It can also include searching for experts on twitter and linkedin to ask questions (or to look at their blogs/sites to see if they’ve answered it already). And finally, don’t forget to look for communities (slack, old fashioned forums, open source project email lists) and see what they say. It’s far better to read what you can rather than pop into these communities and ask–those kinds of requests can happen often enough to annoy folks. But if you don’t see any answers, do ask.
  • realize that no decision is perfect. In 20 years of development, I can tell you that I’ve never made a perfect decision, they all involved tradeoffs and lack of perfect knowledge.
  • communicate the risks with the business. This includes the risks of you feeling like you are over your head, as well as any other risks that your research has found.
  • come up with a recommendation and implement it.

Again, hopefully you won’t be confronted with a far reaching architectural decision for a number of years, but it happens. Hopefully this framework gives you a bit of guidance toward making that decision.

Oh, and if you wanted to know my quiz conundrum ended, some of the choices I made were implemented, and others got swapped out as the team changed. The app was a success.

Sincerely,

Dan