The Deep Synergy Between Testability and Good Design (Michael Feathers)

Welcome to another Tech Talk Tuesday! This week we’ll be discussing a video from Michael Feathers, where the video centers around the intrinsic link between testability and design. For those who are unfamiliar with that name, Michael Feathers is most famous for his excellent book “Working Effectively with Legacy Code”. The book has been invaluable to me throughout my career, and I can’t recommend it highly enough. It’s a 10/10 book, definitely worth the read.

No need to watch the talk, as the point of Tech Talk Tuesdays is to summarize these long videos, but here’s the video for the interested parties who’d like to spend the time to get the word straight from the horse’s mouth:

In summary, this talk goes over:

• A conjecture that good design must be testable
• An example (an Iceberg class)
• Illustration of how testing pain hints at a design issue
• Talks about why it’s true that good design increases testability

Good design must be testable:

Right off the bat, Michael starts with some funny quips:

If you have a point to make, sometimes annoying people is a way of getting them going, and making them think about something they wouldn’t be thinking about otherwise. So the way that I annoy devs, quite often, is talking to them about testability. And then I go and start saying things like “Well, you know, if you wrote your code better it would be more easily testable”. And then they go and they counter with “Well yeah, but then I would basically be putting the tests in control. And I wouldn’t be able to do exactly what I want to do, because I’m the smart guy. I know how to design classes. I know how to go and create systems that tie together, and I’ve built up design skill over years and years and years.” And when I point at some design they’ve got, and I go “well this isn’t really testable”, they go “okay, so what?  I know I’m a good designer”, and I have to say “No, you’re not a good designer. If your code isn’t testable, then that isn’t a good design”. And that really irritates people…It turns out there’s a deep synergy between testability and good design. And it isn’t just that thing of going and saying “well, sure, if you can’t test it, how could you possibly know if it’s good”. It’s way deeper than that. What it comes down to is that many of the fundamental things we do to make design “better”, also make it more testable!

–Michael Feathers

I find this quite funny, mostly because it’s so true. Most folks thinks they’re so great at design. But no one really is (myself included). Chiefly because I believe perfect design is impossible. No one gets it right up front, because they can’t. You can’t predict the future. Designs need to change over time as priorities/usage/goals change. That’s why iterating on it is so important. It is a huge mistake to just say “welp, I’ve designed this class, and I’m done”. That’s never how it shakes down. However, observe that automated testing allows us to make changes with impunity, including changes to our design.

This is a hint as to why testability is important with respect to design, as it allows us to change that design as we’re going. In particular, automated testing is a must-have to support refactoring. Without automated tests, refactoring and redesigning is truly a dangerous and time consuming operation.

An example where good design increases testability (fixing Iceberg classes):

Next in the talk, Feathers gives some wisdom about breaking up “Iceberg classes”, which is a class that has many private methods with a lone public method poking up (kind of like an Iceberg). Because of this structure, it is often tempting to want to test the private methods. If you’d like to take a deep dive into the Iceberg class issue, you can check out one of my responses on StackOverflow that goes over the problem in depth. In short, breaking up an Iceberg class into smaller components alleviates the temptation to test private methods, because it allows you to test public methods of a different component. Furthermore, this refactoring helps obey the Single Responsibility Principle. Most folks agree that this is a good design principle, as it allows for better locality.

Concrete pain when trying to test poor design:

Now you can see this stuff in code, right, you can look at classes, and get a sense you have too much coupling sometimes. But when you start writing tests, it turns it to concrete pain. It turns into this thing of “Well, I want to write a test, but I can’t really write the test. Okay, I can write the test, it’s just super painful. It’s going to take me a lot of time to hunt through all these dependencies and create all the things that I need to pass them into this particular thing.” So there’s a big difference between mentally knowing about coupling and feeling the pain of coupling. I think this is a thing we can learn from testing in a sense. We can easily dismiss problems mentally, but when we write tests, we feel concrete pain. The concrete pain isn’t because testing is difficult, it’s because we need to change our design.

–Michael Feathers

I’ve always had a strong feeling this was true. Whenever I have trouble testing my own code, I try to take a step back and understand why it was so hard to test. I am (personally) prone to writing difficult-to-setup objects, but I make this mistake often enough that I’ve gotten good at reducing the coupling and breaking my work down further into easier-to-test-components (which naturally reduces coupling) when I see the issue crop up.

Thing is, I probably would never fix the difficult-to-setup issue at all if I didn’t try to write an automated test in the first place. It’s really easy to just gloss over issues, or even think there is no issue. However, how do you think your clients feel when they have to call your API? News flash: they feel just as bad as you do when you’re trying to test your poor design.

Here’s a list of testing pains Feathers goes over, and what their particular design flaws are (sub-bullets are my quick thoughts):

• Long methods where we wish we could assert tests against local variable. Design issue is Single Responsibility Principle violation.
• Difficult Setup. Design issue is too much coupling.
  • This is an issue I run into at least once/week. I’ve gotten good at fixing it at this point (to fix you need to break down into smaller components that are less dependent on each other). It usually happens in an integration test (which I typically write after I have some smaller unit-tested components to tie together), not in a unit test.
• Incomplete shutdown (resource leaks). Works well when using it in short timescales, but funny things happen when application is open for a while. Sometimes test suite will hang because resources are locked or not getting cleaned up. Design issue is poor class encapsulation.
  • Tackled a bug in the sarpy library this week with this exact issue! Note that there is an open function for the reader objects…but no close functions. Seems obvious that this would be prone to leaking a file descriptor. If any tests were written against this lib, though, someone would have probably found the issue because their test suite would be failing (like ours was…). Haven’t pushed my change back yet, but I do plan to do so after the change settles a bit in our own baseline.
• State leaks across tests. Design issue is singletons, or other forms of global mutable state.
• Framework frustration. Design issue is insufficient domain separation.
• Difficult mocking. Design issue is one of Law of Demeter Violations, insufficient abstraction, or Dependency Inversion violation
• Hidden Effects. Can’t test class because you have no access to its ultimate execution. Design issue is insufficient separation of concerns, or an encapsulation violation.
  • This is a common problem for folks who are new to testing. Common complaint or reasoning for why something isn’t testable.
• Hidden inputs, or  no (easy) way to setup tests through the API. Design issue is over encapsulation and/or insufficient separation of concerns.
  • This problem commonly occurs in baselines that don’t have automated testing. I’ve seen it crop up way too many times to count at this point, and the common denominator is that those baselines had no tests or very weak tests. Typical issue is that you need some file or other install on your system (like a heavy config file, or other dependency). If you write a test on a CI system, it’ll fail immediately because the config/dependency is missing. Commonly creates “but it works on my machine” syndrome.
• Unwieldy parameter lists (too many params). Design issue is Single Responsibility violation.
• Insufficient Access (wish we could test something private, iceberg example). Design issue is Single Responsibility violation.
  • As a Python developer I rarely have this issue since there is no public/private. But I do try to keep it in mind. I’ll admit though, sometimes it’s just easiest to test the private methods and move on. It will occasionally hurt in the long run, but if it starts to become a repeated pain point, then I’ll refactor. But a lot of times it’s fine, and I’ll write a test for a “private” function and never look back.
• Test thrash (many tests need to change when code changes). Design issue is Open/closed violations.
  • This seems to be a common problem for folks who are new to testing.

Simply put, complaints about testing really means there’s a problem with design.

Why good design increases testability:

This comes down to why design is “good”.

So why do we want small methods and small class? Assuming that we do. A lot of this in object orientation is about locality. It’s about being able to look at one particular thing, and understand, based on the local context what really happens, without worry about this over here and that over there. What’s interesting is that tests are really another way of understanding your code. It’s a deep thing when you think about it. When we write a test, what we’re doing is we’re basically going ahead and we’re using an automatic construction to check for particular results. Not just by inspection, and reasoning things through, but writing code that actually goes and automates this process for us. So we can run these tests over and over again and find out whether things are broken, or not broken, and stuff like that. Essentially we’re going through the same process. The things that make our code hard to understand mentally often make it hard to produce automatic procedures for checking those things. Because they tend to kind of go/fall in line….good design tends to be good design because it follows cognitive principles, and is kind of the way our brain works. Smaller pieces make it easier for us to reason about them, as a result we tend to go favor smaller pieces when we have a lot of complexity that we need to deal with. So I think it’s this act of trying to go and write tests, and it’s a cognitive act in a way, it’s a way of going and understanding the code, that goes and gives us this extra benefit. It gives us this ability to be able to go and say look, you know, I can have all these interesting thoughts about design, but if I write my test first, before I try to write my code, it’s an instant concrete reminder of whether things are aligned with good design principles or not.

–Michael Feathers

Cognition and understanding are paramount in software. We cannot change or use code that we do not understand, let alone test it. As Feathers points out:

Tests are a way for us to understand our code.

–Michael Feathers

It’s so true. Good design increases understandability through mirroring and adhering to our natural cognition, and in turn, writing the test forces us to obey these design principles. If we don’t, as seen in the previous section, the test is either impossible or extremely difficult to write.

Some empirical evidence that automated testing (TDD in particular) increases good design:

I was at the XP 2010 conference a couple weeks ago, and there was an interesting paper there where they took some large projects they took projects that had done TDD and projects that didn’t do TDD and they started comparing the metrics. One of the things they discovered is, naturally, less complexity, less cyclomatic complex, smaller classes, and smaller methods. And this is without anyone going around to these people and saying “well this is a better style/design, so do it”. They just said “please just use TDD”. One of the side effects was that you end up with smaller pieces. And typically that just ends up being better for us, to be able to compose small things to go and solve big problems.

–Michael Feathers

Some final words of wisdom from Feathers:

It’s true, there are people who have a medical condition where they can’t feel pain. They usually don’t live very long. They usually break arms/legs and they’re not aware of it, things like that. Really serious condition. As much as we hate pain in life, have to remember every pain is a bit of a learning experience. And I think it’s really kind of cool that we have this thing with testing, that it tends to synergize well with design. And that we can learn a lot by doing testing, and those things themselves make our design better.

Here’s the gist of it really, testing isn’t hard, testing is easy in the presence of good design. If you’re having trouble testing things, reconsider your design. You will end up with something better.

–Michael Feathers

Pain in software is one of those things where it’s somewhat counterintuitive. We spend most of our lives avoiding pain (at almost all cost). But in software, what you actually want to do is repeat and iterate over the pain points. Not so that you can go through more pain, but rather so that you remove the pain entirely. You don’t want to ignore or dodge pain in software, because without addressing it, the core issues simply never go away, and nothing will ever improve.

Spot on, per usual, Michael. Thanks for your awesome book, and thanks for the knowledge drop in this awesome talk!