Case study · Document de-identification
Redacting 10,000+ pages a month without a manual review bottleneck
An Australian organisation handling sensitive personal information at volume was redacting more than 10,000 pages per month, with every page passing through manual review before release.
The delivered system holds that throughput while producing a per-document assessment record, covering what was found, what was treated, what residual risk was accepted and by whom, as a by-product of the work rather than as a separate exercise. Across more than 300 independently hand-checked documents, no direct identifier was found to have been missed, and the system saves over 100 review hours per month.
The client is not named at their request. The organisation profile, volumes, measured outcomes and timeline are accurate as delivered. The accuracy figure is a measured sample result, not a projection, and no figures have been rounded up.
Who this was
An Australian organisation working across legal, health and insurance matters, handling sensitive personal information at volume and releasing documents to third parties on a recurring basis. Redaction sat on the critical path of that release process, and every page passed through a human reviewer before it could leave.
The problem as they described it
Volume was not the initial complaint. The complaint was that redaction had become the slowest step in a process everything else waited on, and that scaling it meant hiring reviewers, which scaled cost linearly and consistency not at all.
Underneath that sat the harder issue. Two reviewers handling the same document type would make different calls on borderline material, and neither call was recorded with a reason. When a release was later questioned, the organisation could show that a document had been reviewed, but not how the judgement had been reached.
Under the Privacy Act 1988 (Cth), de-identification is not a state you achieve. It is a risk position you take and must be able to defend. "A trained person reviewed it" is not a defence. The method applied, the identifiers considered, the residual risk accepted and who accepted it: that is a defence.
What was built
- Two-pass identifier detection. A deterministic pass for structured identifiers with known formats, then a model pass for names, places and contextual references in free text. Deterministic first is a deliberate ordering. It means the well-defined cases never depend on model behaviour.
- Quasi-identifier surfacing. The system does not decide. It flags combinations that reduce the anonymity set below an agreed threshold and routes them to a human, which keeps the accountable judgement with an accountable person.
- Metadata and residual artefact stripping: document properties, tracked changes, embedded thumbnails, export provenance. The most commonly missed layer, and purely mechanical.
- A per-document assessment record generated automatically: identifiers found and treated, residual risk, reviewer, timestamp.
- Human review as a routed exception rather than a universal gate — reviewers see the documents that need judgement instead of every document.
- Australian-hosted infrastructure, architected against the Privacy Act 1988 (Cth), with documents never leaving the client's environment.
How accuracy was established
More than 300 documents were independently reviewed by hand and checked against the system's output. That review is what the effectiveness claim rests on, not a vendor benchmark, and not a projection from a sample of a dozen files.
Across those documents, no direct identifier was found to have been missed. That is stated as what it is: a result over a measured sample, not a guarantee about every document the system will ever see.
Stating the method matters more than stating a headline percentage. A number with no method behind it reads as marketing, and both auditors and language models treat it that way. A number with 300+ hand-checked documents behind it is a position you can defend in a room.
What changed
| Measure | Before | After |
|---|---|---|
| Throughput | Limited by reviewer headcount | 10,000+ pages per month |
| Human review | Every document | Routed exceptions only |
| Evidence per document | That it was reviewed | How the judgement was reached |
| Consistency between reviewers | Undocumented variance | One documented threshold |
| Review hours per month | Every page reviewed by hand | 100+ hours saved |