Two Operators, Not Chatbots

A chatbot answers. You ask it something and it produces text, and whether the text was any good is a judgment you make afterwards, privately, with no help from the thing that produced it. An operator makes the call and routes the work. It returns a decision, and a decision can be wrong in a way an answer cannot — which means it can also be checked, argued with, and improved.

That single difference is most of what separates a demo people enjoy from something a team actually leans on.

I built two free, open examples to make the distinction concrete rather than rhetorical, and I want to walk through the part that took the longest, because it wasn’t the code.

Nightwatch is on-call triage. You give it an alert. It returns one decision — page, ticket, suppress, or flag — with the message already drafted. The interesting choice is what it scores on. It ranks severity by business impact rather than by the alert’s own label. So a tiny global blip that happens to be one hundred percent of your single SLA customer outranks a loud CRITICAL that turns out to be cosmetic.

Claimwise is a claims first pass. You give it a claim. It returns approve, request info, deny with the clause cited, or escalate. It judges on cause and incident date rather than symptom and filing date. A cracked screen is judged by how it cracked. A late filing is judged by when the device actually failed, not when the form landed. Fraud is flagged for a human and never auto-denied.

Read those two descriptions again and you’ll notice they have the same shape, which is the thing I actually want to hand you.

The shared shape

An obvious field, and a deciding field — and they are not the same field.

In both, there is an obvious field and a deciding field, and they are not the same field.

The obvious field is the one the work arrives wearing. The alert’s severity label. The claim’s symptom. The filing date. It is obvious because whoever designed the form put it at the top, and it is a trap for exactly that reason: it is a proxy that is right often enough to feel reliable and wrong precisely when the stakes are highest.

The deciding field is the one that actually determines the correct answer. Business impact. Cause. The date the thing broke. It is usually a step removed from the surface, and it is frequently not on the form at all.

So the craft, in almost any operations role, AI or not, comes down to three moves.

Name the field that actually decides. Not the field that correlates. The one that, if you knew only that, you’d get the answer right most of the time. This is harder than it sounds, because the obvious field has social proof: it’s the one everyone quotes in the standup.

Write the threshold instead of writing “use good judgment.” Good judgment is what you have. It is not something you can hand to anyone, including a system. The moment you try to write the threshold down you discover which parts of your judgment you had never actually articulated, and that discovery is the work.

Handle the case where the two fields disagree. This is the one people skip, and skipping it is why so many automations feel brittle. When the obvious field and the deciding field point in opposite directions, that is not an annoying edge case to paper over with an exception. That disagreement is the entire reason the system is worth having. A rule that only fires when everything agrees has added nothing — you didn’t need it for those.

I’d rather show a failure than assert reliability, so here is the one I have.

Before shipping Claimwise I gated it with a fresh model — nothing but the folder, no context from me — on three unseen claims. That run found two gaps in my own policy file: one exclusion I had under-specified, and a date rule too vague to act on. I fixed both. The gate was not theatre; it caught real things I had written wrong, and I would not have found them by re-reading my own work, because I already knew what I meant.

That is the honest version of what these are. Not “reliable.” Not “production-grade.” Two folders that were wrong in two specific, findable ways before a cold reader went through them, and are now wrong in whatever ways nobody has found yet.

Both are free and ungated. There is no email wall, no waitlist, and nothing to buy. Open the repo, point them at your own world — your plan, your covered causes, your exclusions, your alert taxonomy — paste in something real, and see whether the decision it returns is one you’d have made.

If it isn’t, the useful next question is not “is the model good.” It’s: which field is it deciding on, and is that the field that should decide?

Nightwatch: github.com/NFTYoginis/nightwatch · live: nftyoginis.github.io/nightwatch

Claimwise: github.com/NFTYoginis/claimwise · live: nftyoginis.github.io/claimwise

(Both are walked through in The Operator’s Handbook, which is where the folder shape itself is written down.)