Programming

Are my unit tests taking too long?

Earlier today, one of my friends asked me:

What’s the longest running unit test you ever experienced?

I quickly answered “72 hours”, but then realized that I should probably clarify that answer.

True, jUnit was running for 72 hours at a time, but most of the tests that it was conducting were not unit tests. In fact, now that I think about it: none of the tests were unit tests. Instead, jUnit was being used as a top-level driver to conduct end-to-end system tests (Why? Because pretty reports). So, even though the client was using a unit testing tool, no unit tests were actually being conducted. Therefore, this answer doesn’t count.

My next answer was “3 hours”, but then I realized that I should probably clarify that answer as well.

This client was using Python’s nose framework to conduct 3 hours of actual unit tests. But, there was a catch: the application in question was architected as a monolith. Whenever Jenkins detected a change to any line of code, it would kick off the entire unit test suite. It didn’t matter if a portion of the code was a logically separable unit that hadn’t changed in years. It was going to be retested, and the retest was going to take 10 minutes. So, this answer counts, but only kinda.

Let’s recast the question:

How long should my unit tests take?

My answer is “As little time as possible”.

The goal of unit tests is to verify that a stand-alone component is operating as expected. Ideally, we want to detect failures before the developers begin their next implementation task. To satisfy this objective, the unit tests should be able to complete within a few minutes of each commit.

"No!  That's Impossible!"

Okay, calm down. It’s only impossible if we are trying to retest the entire component each time. Things become much more manageable if we assume that the component’s underlying subcomponents are already operating as expected. Think about it: you don’t need to retest all of the features of your car’s stereo just because you replaced the car’s battery. Once you verify that the stereo is receiving power, you can safely assume that it is still operating as expected.

How can we be confident that the underlying subcomponents are already operating as expected? Easy: we only release a new version of a subcomponent when all of its unit tests are passing. As long as we build our components using these official releases of the subcomponents, we can be confident that the retesting of the subcomponents will be unnecessary.

So, the next time you notice that your unit tests are taking a long time to run, ask yourself: “Can the unit tests for this component be re-envisioned as integration tests for a collection of subcomponents?”. If the answer is “Yes”, then extract the subcomponents and readjust your build process.

Leave a Reply

Your email address will not be published. Required fields are marked *