Published 2020-08-12 on The Mister Banal's blog
Unit tests, the curse worse than the cause
Criticizing unit tests is uncommon. Anyway I’ll try to explain why I think we should stop writing them.
Unit testing is a practice that generally follow when you learning about the SOLID principle. Having a single responsibility per subject is commonly narrated to simplify testing.
class FooRepository def get_foos ... end end class GetFoosUseCase def initialize(foo_repo) @foo_repo = foo_repo end def call ... end end
One of the approach to test this code is to write unit tests that will check every class independently. The idea is that if every tests passes, then the whole project should be fine. This structure, with injected dependency truly give the hand to the testes to create dedicated contexts around tested subjects.
class TestGetFoosUseCase mocked_repo = Mock.new.to("get_foos").reply(["a mocked foo"]) subject = GetFoosUseCase.new(mocked_repo) def test_get_foos assert subject.get_foos == ["a mocked foo"] end end
Do you think this test is stupid and tautological ? I think most of contextual
tests actually are. Anyway, to write this kind of tests you got to
know how the
GetFoosUseCase works as you got to know that a call to
@foo_repo.get_foos is done somewhere. That why I think the ideological
“write tests first” is a myth in the unit test world.
So now my point is: You though very hard to write SOLID classes and method, limiting coupling as much as you can, and THEN you wrote a completely coupled test suite, as the tests got to know how the context around the subject is working.
- Refactoring the application
- Changing a method API
- Adding à dependency
- Writing a single line of code somewhere
Will force you to rewrite mocks in every test that use them, most of the time. This kind of test cost lot of time and are insane to maintain.
And this is not the only trade-off of unit tests
As you got to mimic the context, you’ll write lot of code, generally way more than the code you actually want to check. A common “scientific” estimation is that a developer add 3 bugs every 100 lines of codes. Tests are code so if a test do not pass, which one between the code or the test do you trusts more ? The SOLID code or the other one ?
If think we should definitely stop thinking about code coverage but more about code value. What is the value of this code that mocks a dependency ? Is it maintainable ? Does this code will brings robustness to my project ? And at what cost ?
Furthermore, unit tests do not bring more robustness that functional ones.
class TestGetFoosUseCase def test_get_foos assert container("GetFoosUseCase").get_foos == ["a fixtured foo"] end end
I think this test code got a huge value, way more than the previous one. It cover
way more code as it will go through the
GetFoosUseCase and the
FooRepository classes. It will stay the same if you re-factorise both those
classes so it is easier to maintain. You can add features in the
GetFoosUseCase and this test will still be valid and tells you if it
still return the fixtured foo.
Does this functional test is perfect ? Not at all. But it will perfectly serve the automatic test suite purpose :
- is rapid to write and easy to maintain
- detect common errors
- give a rapid feedback about the code, detecting re-factorisation mistakes
Both approach only test a ridiculous portion of the whole code flow possibilities. It is impossible to write tests that cover all cases. Having a 100% code coverage DO NOT MEAN that every cases are covered ! Don’t be delusional about your tests. They’ll fail you because bugs are bugs and you can’t predict them. Write simple code. Evaluate your code value. It can be positive, bringing features, being maintainable, being debuggable. Or it can be negative, having a single almost useless purpose, being hard to understand, requiring regular rewrite.