363
Anon died on the "clang format bad" hill (programming.dev)
submitted 2 weeks ago by to c/programmer_humor@programming.dev
544 comments fedilink hide all child comments
 

Mine is that the useState hook is an actual plague on the entire React ecosystem. Regardless of your opinions of React, if you wanna use something then use it right.

you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 1 week ago (3 children)

And they pin behavior more than strict correctness, in my experience, but that’s a valuable signal. Canaries in the mine.

The problem is that the behavior they pin is not the official API-exposed behavior. The behavior they pin is mostly implementation details. If diffuse(bomb) is supposed to diffuse the bomb, I need to test that given a legal instance of a Bomb it'd end up diffused. A unit test cannot test that - if diffuse is the unit being tested, then bomb is not allowed to be an actual bomb - it has to be a mock. And thus diffuse(bomb) does not test that diffuse diffuses a bomb - it tests that diffuse cuts very specific set of wires in a very specific order.

  • If the bomb change, the unit test is till green because the mock is still replaying the old behavior.
  • If I fix diffuse to handle the new bomb, the unit test becomes red.

Once the bomb change, the unit test shows the opposite of what a test is supposed to show - and now you have to fix the test itself. Because that's what unit tests test - they test themselves. They don't test the system.

An integration test that would bring in an actual bomb, diffuse it, and check if it's diffused in a controlled environment where it can't do damage by blowing up - is going to be more expensive to run, yes, but at least you will be getting a meaningful signal out of it. It'd be testing something of value. The unit test is cheap to run (although writing it was probably more expensive), but it gives you nothing. It's completely pointless.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 2 points 1 week ago* (last edited 1 week ago) (2 children)

    Awesome and well articulated response. Love it.

    There are a few threads to pull here:

    Is an integration test the best way to test the full e2e behavior of an api call? Yep.

    Now, depending on the size (and architecture) of your system, the full test suite of into tests could take like, 10 hours.

    And even then, you're STILL probably mocking something (direct seeding bombs into the db). Full setup of all life cycles of all your entities using their associated entities (bombs, wires, wire clippers, bomb diffusers, their rank, their bosses, their diffusion history, etc...)

    So they're expensive, and if you really look you're probably still cheating some steps anyways.

    Another thread to pull on is what are you mocking in your tests to begin with, and if they're not resulting in valuable test results then that goes back to one of my original points being that you might have suboptimal patterns in your production code.

    Like, what logic governs bomb diffusion and where does it live? Are you mocking the right thing? If you have to mock the wrong thing... WHY? The ritual of writing those tests give you the first whiff of a potential smell.

    I tell my guys not to test implementation: test behaviour. Test contract adherence. If that's hard, no problem, message me let's review it. Maybe a behavior they're trying to test legitimately should be an int test. More often than not, their design is demanding to mock something that should be decomposed further, and then composed with dependency injections that are where the units that should ACTUALLY be mocked are the things that get mocked and injected, so the unit tests are actually testing the BEHAVIORS we want to pin rather than implementations. Composition and single responsibility principal are what enable unit tests to pull their weight.

    Edit: I'm so passionate on the subject that if you scribbled up a scratch example and pinged me a github link I'd genuinely review it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 week ago (1 child)

    Now, depending on the size (and architecture) of your system, the full test suite of into tests could take like, 10 hours.

    You'll eventually have to run these 10 hours no matter what. If you only run the unit tests you won't be able to tell if your code actually works, and if your full e2e suite passes then you won't learn anything of value from running the unit tests - if one of them fails you can be fairly certain that it is that unit test itself that is broken, not the product.

    During development, you don't need to run the full test suite. It's usually quite trivial to identify a single integration test that checks the parts you are touching, and then you only have to run it to see the effects of your changes.

    And even then, you’re STILL probably mocking something (direct seeding bombs into the db). Full setup of all life cycles of all your entities using their associated entities (bombs, wires, wire clippers, bomb diffusers, their rank, their bosses, their diffusion history, etc…)

    Integration tests rarely (if ever) need mocks. If they need Test Doubles at all, they'd typically be _Fake_s rather than _Mock_s. But even that is not always needed - an integration test can often just use non-production versions of the real dependencies.

    So - if your real software typically runs against a database that sits on a powerful server and has tables with billions of rows, the e2e tests can use the same database engine but run it locally and only put a few dozens of rows in these tables. This setup is quick and cheap to set up and tear down while still being faithful to the production setup - maybe not 100%, but orders or magnitude more faithful than a unit test's mock that just compares the SQL commands given to it with the ones preprogrammed into it.

    Another thread to pull on is what are you mocking in your tests to begin with, and if they’re not resulting in valuable test results then that goes back to one of my original points being that you might have suboptimal patterns in your production code.

    Like, what logic governs bomb diffusion and where does it live? Are you mocking the right thing? If you have to mock the wrong thing… WHY? The ritual of writing those tests give you the first whiff of a potential smell.

    My argument is that this is more about the "problem domain" than about the "solution domain". Some field of software work very neatly with unit tests - general libraries are a good example. Other fields are a very poor fit, and trying to shoehorn them into using unit tests - even worse, shoehorning them into using mostly unit tests - will inevitable result in a nightmare of mock maintenance.

    I tell my guys not to test implementation: test behaviour. Test contract adherence. If that’s hard, no problem, message me let’s review it. Maybe a behavior they’re trying to test legitimately should be an int test. More often than not, their design is demanding to mock something that should be decomposed further, and then composed with dependency injections that are where the units that should ACTUALLY be mocked are the things that get mocked and injected, so the unit tests are actually testing the BEHAVIORS we want to pin rather than implementations. Composition and single responsibility principal are what enable unit tests to pull their weight.

    The problem is that unit tests "promote" aspects of implementation into behavior. Classic example - the exact syntax of the SQL query a function uses to obtain the data it needs is by all rights an implementation detail, but a unit test is forbidden from using a real database so it passes a mock that expects a very specific query - now it became a behavior.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 1 week ago* (last edited 1 week ago)

    That's a lot, so I'll be forthright that I'll probably be giving a relatively terse and gappy response, but that's more because of my own limitations as a mobile user.

    Fundamentally every test unit, integration, ui, manual is still the same thing:

    Given X do Y expect Z.

    Mocks, fakes, whatever you want to call them, they're fundamentally the same concept too: "given some idealized version of P"

    All you've done is pushed where that line sits. If I update my company writer to prospend all my company names with "corp.", and my company read API test seeds without it and something at your read service layer strips ".", if I forget to update the "given" in my read test seed data, that bug stays invisible.

    It's the same class of issue. We just kicked the can down the road.

    You could do writes and reads in the same test, but even then you're still kicking the can down the road that your front end is calling the API correctly at all. Maybe that the buttons even render to an end user given their authority.

    One might then conclude, then, that everything should be an automated UI test.

    But would it catch a bug where the bottom collapsed to 1PX? Is that part of the test too? Maybe manual testing is the real way to go.

    A further step back why do we even test anything at all? To provide some ultimate guarantee that everything works perfectly at all times? Is that the gate for a PR? A deployment to prod?

    There are various gates, at different levels of certainty, and there is an economic scaling for the different levels of assurances. AND they scale with the overall complexity of the system.

    And that isn't a bug, that's a feature.

    It's ALL behavioral contracts, up and down the pyramid.

    There is a bang, there is a buck. Get the most bang for your buck. The distribution will vary by complexity and structure of your project.

    That being said, I've said it before and I'll say it again: int and UI tests make it very very easy to write fucking garbage code, the VALUABLE architectural pushbacks on poorly patterned code that unit tests avail is AT LEAST as much value to the overall tech debt accumulation of a project as the results themselves.

    Please don't read this as me shitting on int tests. It's me trying to impress that unit tests are worth writing, and justifications for forgoing them usually are fundamentally driven by an underlying code smell. Sometimes the smell is "our architecture is already so fucked that writing a meaningful unit test would involve re architecting large swathes of our system".

    And in that case I'd say "it is what it is", recalibrate the testing posture towards int tests, and accept that is the reality.

    But I would NOT conclude on the blanket statement that unit tests SHOULDN'T be at the bottom of the testing period generally. Just that I'd inherited or failed to maintain a strong system design, and take the opportunity to reflect on how things came to that state for the purposes of becoming wiser.

    Addendum:

    Even int tests can reveal insane system architecture, just at a different layer and scale. I worked on a system at a fortune 500 company where the architecture was too far gone to even have meaningful integration tests. They were too far gone to even have automated front end tests. All material testing had to be done by hand. They had engineered their system with that staggering level of incompetetence. At the risk of making a slippery slope argument... It's a slippery slope.

  • permalink
  • fedilink
  • source
  • parent