Fighting the Superstitions of Software Development: Questioning the Assumptions

We fix the problems we know to look for. It’s the assumptions left unexamined that are apt to bite you. Which superstitions are you carrying around that need to be re-examined?

One of the wisest, most brilliant programmers I ever met had a team lead habit I loved. If you made a technical assertion, Mike would ask, “Is that true?” Mike didn’t ask the question to imply you were wrong; he asked, “Is that true?” to challenge your assumptions and to determine upon what data you drew your conclusions. If you said, “Yes, that’s right. Here’s why…” and made it clear your statement came from somewhere, he’d be satisfied, and you’d earn his respect.

If the programmer stuttered, “Um, I guess it’s true,” instead, more often than not the programmer headed back to his office to find out the real story. Or Mike would help the developer re-examine the assumptions being built into the software. If the assumptions made sense, fine. If not, it was an opportunity to write better code.

That wasn’t just a technically interesting exercise. At the time I knew him, Mike and his team were building a compiler to generate the fastest executables possible. Nanoseconds counted. And with Mike at the helm, one particular math routine went from average performance to outstanding to… zero cycles. Because the “Is that right? Is that true?” question process eventually made the team realize that they didn’t need the routine at all. Now that’s code optimization.

At the other extreme, let’s say you want to “uniqueify” an array in JavaScript (that is, find all the unique values in an array). You’ll find a bunch of code samples with a quick online search. Most of them look very similar to one another, and they take a dumb, brute-force approach of walking the array multiple times. Bad performance, no biscuit. But hey, that’s how everyone does it, right?

If you handed that code to an expert programmer like Mike, he’d ask, “Is that the best solution?” Well, no, maybe not. There’s a way to do it that walks the array just once, if you take the time to think about it.

At the Schindler bitranch, we call these “superstitions:” information accepted on faith, without personal knowledge or examination. People pass along “everyone knows” data without questioning it, and others repeat the superstition as though it’s undeniably true. Confidence isn’t knowledge; in fact, confidence can prevent knowledge and innovation from happening, because an unquestioned belief means you never measure, never test, never look at alternatives.

My example above is about application performance, but this questioning-method applies to everything a developer does, from project estimation to designing for end-user usability. And it applies to every field! Stained glass artists accept as superstition that “lead is stronger than copper,” but has anyone tested this assertion? Is it true? Some early experiments imply copper is equally strong. Woodworkers nod along when someone else mentions-in-passing which type of joint “everyone knows” is strongest; it wasn’t until a couple of years ago that one magazine actually tested different kinds of joints for strength (with surprising results).

Superstitions are strongest (and most dangerous) when these “everyone knows” data points are hard to confirm or deny (such as most discussions of search engine optimization, since Google ain’t talkin’) and where “proof” is possible only through data triangulation. It’s worse for developers and people in IT, I think, because things that were true a few years ago no longer are. In addition to questioning our own assumptions (or, as politely as possible, challenging the assumptions of people we work with), we have to test our knowledge against a time-stream. Last week, I flipped through a book on search engine optimization which pointed out that the “right things” to do five years ago are now useless… or worse.

Asking “Is that true?” does tend to upset some people, because they don’t easily distinguish between a sincere question and an assumption that the listener is wrong. They are apt to think you’re accusing them, saying, “This isn’t true!” when you’re asking “Is this true?” However, by asking someone to back up their assertions–as Linus Torvalds does in Linux kernel development—you’re asking them to revisit their assumptions, to identify their confidence in the data and to check whether the speaker is repeating something “everyone knows” but may not be true at all.

I’ll warn you: when people don’t know the answer, they’re going to get pretty mad at you. “Other people do it that way” sure isn’t a good answer to, “Is that true?”—but that’s the first answer they’ll trot out.

Don’t wait for Mike. Turn the question on yourself and your teammates. Instead of accepting others’ (alleged) wisdom, take the time to revisit the question. Sure, write the code. Then look at the assumptions you’re making, and ask yourself, “Is that right? Is there a better way?”