How a Field Test is run

The task

DIALED defines the task from the nomination and publishes it before filming. One task, one deliverable, one clock.

Real work from the contractor's business, not a demo scenario. The vendor does not pick the task, the contractor, or the incumbent.

Fairness controls

Identical source inputs to both stacks.

Both operators trained equivalently. The vendor may train the contractor's operator in advance, and training time is disclosed on screen.

The vendor rep may not operate their own product during the timed run.

Single take. No resets. Failures are part of the record.

The clock runs continuously and appears on screen.

What gets measured

Time to complete. Output accuracy against the contractor's known-correct answer. The number of manual corrections required. And what the contractor would send to a customer without further edits.

We publish both columns side by side. We do not average them into a score, because a score hides the thing you needed to know.

Disclosed on every episode

That the vendor paid for production. The contractor's configuration, tenure, and skill level. Any training provided in advance. Anything that went wrong with the production.

Both parties speak

The contractor's verdict runs unedited. The vendor rep's closing statement runs unedited, up to 90 seconds. The incumbent's reply, if given, is published on the episode page.

The incumbent

The incumbent never agreed to appear and may lose on camera, so the handling is fixed. We name them factually: version, plan tier, configuration. We never editorialize.

We notify them before publication with the task, the methodology, and the publication date. We offer a standing invitation to their own Field Test on the same terms, and we publish any reply verbatim.

The contractor

Paid $2,500 to $5,000 depending on time and disruption, and they sign a release before any filming.

They approve their own footage only. Their business name is published unless they ask otherwise, their financials never are, and customer data on screen is blurred.

What it costs

Field Tests are produced at cost, and the number depends on the trade, the travel, and the complexity of the task. We work that out with each company directly, and we fund several a year ourselves. Cost has never been the reason a test didn't happen.

The fee buys the test. It has never bought a result. Publication is unconditional. Nobody sees the episode before the audience does.

The board is the record

Nominations, first contact, filming, and publication all appear with their date, and no row is ever removed. The board has four states and none of them characterize a company's behaviour.

We record the fact and never the reason. If a company gives us a reason, we print it verbatim beside their row.

What we will not do

We will not run a test we cannot publish. We will not accept a vendor edit to a result. We will not test a company below the eligibility floor, and we will not test a company a contractor did not ask about.