The work·Software teams

Flat Experiment Diagnostician

Was the test too small to see it, or does the change do nothing?

The situation

You ran the test, and the result came back flat. Now you have to say whether the change does nothing, or the test was never big enough to see it. Those two answers look identical and lead opposite ways.

What actually happens

  1. You give it how the test was set up and what came out.
  2. It walks the stages in order — who was included, what they saw, what was counted.
  3. It stops at the first stage that broke, and names that as the cause.
  4. You learn whether to run it bigger or drop the change.

Where it stops

It stops reading at the first stage that failed. Its rules call that the diagnosis site, so what comes back is one cause rather than a list of everything else that also looks wrong.

See it

Open →

Opens in a new tab.

← All the work