Exceptions & human review
An exception is the machine saying "I am not sure." Which stages raise them, who sees them, and how much of the corrected work gets checked a second time are all configuration, set in the pipeline and in Reviews.
Where exceptions are switched on
Human review is opt-in per stage. The Split / Classify and Extract stages each carry a Send exceptions to Reviews toggle in the pipeline designer, and both are off by default. With a stage's toggle off, that stage never holds a document for review. Low-confidence values still carry their confidence scores in the results.
Turning either toggle on creates one work queue for the pipeline, named <Pipeline name> · Review. There is one such queue per pipeline, whichever stages are enabled. Its filter matches the pipeline and the exception types of the stages that are switched on, and it stays in step as toggles change. It is an ordinary queue: teams and SLA are edited in Reviews like any other.
What raises an exception
| Exception | Stage | Raised when | Needs the toggle |
|---|---|---|---|
| Low confidence · Classify | Classify | The document's class is below the classification confidence threshold | Split / Classify |
| Split boundary | Split | The Split stage is unsure where one document ends and the next begins | Split / Classify |
| Split & Classify | Split | The class assigned to a split-out document is below the confidence threshold | Split / Classify |
| Low confidence · Extract | Extract | Any field is below its confidence threshold, or a cross-field rule fails | Extract |
| No data extracted · Extract | Extract | Extraction returned nothing at all: no fields, tables, figures or selections | No. It always goes to review |
A field's confidence threshold is resolved from the field's own setting, then the schema's, then a platform default of 0.80; a threshold set on the pipeline's Extract stage overrides all three. A failed cross-field rule does not have an exception type of its own. It raises Low confidence · Extract.
No data extracted usually means the pipeline is misconfigured, for example a wrong schema or a blank scan, not that the model is unsure. That is why it bypasses the toggle. If the pipeline has no Review queue yet, one is created the first time it happens.
A document held for review shows the status In review. Its job waits until every document it is holding for review is cleared, then resumes from where it paused and continues to its destination. If an exception matches no work queue at all, for example because its queue was deleted or its filter narrowed, the document is not left waiting. It is auto-completed with the machine result and shows as Auto-completed.
A stage that fails, such as an OCR crash or a connector timeout, produces a stage error. Stage errors appear under Jobs → Error monitoring, not in Reviews. An exception means the stage succeeded but its output needs a human.
Reviews
Exceptions are worked in Reviews in the app sidebar. Supervisors see a console with six tabs: Work queues, Operations, Operators, Access profiles, Teams, and Verification rules. Operators see My work queues, which lists the queues their team is assigned to.
Work queues
A work queue is a saved live query over the documents in review, not a folder. Documents are never moved; they match a queue or they do not. A queue can filter on pipeline, stage, exception type, document class, schema, confidence, state, dates and extracted field values.
| Queue | Created by | Matches |
|---|---|---|
<Pipeline name> · Review | Turning on Send exceptions to Reviews | That pipeline's exceptions from the enabled stages, plus No data extracted |
| Verifier | The system | Corrected documents sampled for a second pass (state To be verified) |
| Parked | The system | Documents an operator parked for a supervisor |
| Custom | A supervisor | Any filter, including views saved from the Explorer |
The two system queues cannot be renamed, re-filtered or deleted. The Explorer lists every document the viewer can see across all queues. A filtered view there can be saved as a new queue.
Teams
A team is a group of operators that review work is routed to. Assigning a team to a queue can be narrowed to specific stages or exception types, and the team's members see only the documents that assignment matches.
Access profiles
A profile defines what a person may see, independently of which queue they are working. Effective visibility is always queue ∩ profile. A document a profile excludes is invisible, not greyed out. The API returns 404 rather than 403, because the existence of a record is itself information.
Coverage and verification
Not every correction needs checking twice. Each operator has a verification mode on the Operators tab:
| Mode | Effect |
|---|---|
| No review | Corrections are not sampled, and the operator does not verify others |
| Needs review | Corrections are eligible to be sampled into the Verifier queue |
| Can review | The operator verifies others' corrections and still corrects documents |
Only operators marked Needs review are sampled. How much is sampled is set per operator as a list of review coverage rows. There is one Flat row covering every correction, plus optional rows that each name a rule from the Verification rules tab. When a document is submitted, the Flat row and every row whose rule matches the document are compared, and the highest percentage wins. Rule order does not matter. Adding a rule can raise an operator's coverage but never lower it.
| Row | Coverage | Effect |
|---|---|---|
| Flat: every correction | 20% | The floor: about one in five submissions is sampled |
| Rule: class is Invoice | 100% | Every invoice this operator corrects is verified |
With these two rows, invoices are verified every time and everything else 20% of the time. An operator newly set to Needs review starts at Flat 100%, so every correction is verified until a supervisor lowers it. Coverage does not change with an operator's tenure.
Sampling is deterministic: the same document, rule and rate always give the same decision, so any routing can be explained after the fact. Two guards apply:
- No one verifies their own corrections.
- If no one other than the corrector is able to verify, the document is not sampled, so it is never left waiting for a verifier who does not exist.
The operator loop
- Claim: The operator opens a queue and the workstream hands them the next matching document. A claim is exclusive and runs on a 10-minute lease that renews while the document is open.
- Correct: Flagged fields come first. Each correction keeps the original extracted value unchanged beside the corrected value, along with who made it, when, the field's confidence at the time, and an optional note.
- Submit: If the operator's coverage does not select the document, it is Corrected and its job can resume. If it is selected, it moves to the Verifier queue as To be verified. The verifier sees only what changed (Review changes), can switch to Edit all fields, and either confirms or amends. A verified document never returns to the operator.
- Park or skip: Park takes a reason, sets the document to Parked, and routes it to a supervisor through the Parked queue, out of the operator's loop. Skip also takes a reason but changes nothing about the document. It releases the claim and the document stays in the queue for anyone, including the same operator later.
What gets measured
The Operations tab leads with six figures:
- Open exceptions: documents waiting across all work queues, with how many are in progress.
- Cleared today: documents cleared across all queues today.
- SLA at risk: documents overdue or due soon against their queue's SLA.
- Median time-to-touch: how long a document waits from its job to first being opened.
- Verifiers active: people holding a document right now.
- Auto-confirm rate: the share of processed documents cleared without human review.
Inside the workstream, each operator sees their own done today, your pace and left in queue, with the percentage of the queue cleared.