Up to 10,000 Calls a Day, Redacted the Same Day

The County of Orange is the third most populous county in California, home to around 3.2 million people. Its Social Services Agency employs more than 4,700 staff and, by the agency's own reckoning, serves one in three residents of the county each day.

The work runs through programmes most Californians would recognise: CalWORKs, CalFresh, Medi-Cal and General Relief. Every one of them is a programme people call about, and the agency's call centre takes thousands of those calls daily. Benefits questions, case queries, applications, and the full range of contact a resident has with the services that keep them housed and fed.

The agency wanted to understand how well that was going: whether residents were being served properly, where agents were struggling, which issues kept recurring. It had already engaged Qualtrics to do the analysis.

That is where it stopped.

The scale is the reason it mattered. An agency serving a third of a county of three million people every day is not running a call centre it can spot-check by listening. Understanding service quality at that volume needs the recordings analysed systematically, and analysing them systematically meant sending them somewhere.

Social services calls are dense with exactly the information that cannot travel. A resident calling about a CalFresh or Medi-Cal case gives their name, their date of birth, their address, and frequently their card details. The caller is very often in difficulty, which is the point of the agency existing, and is not in a position to weigh what they disclose to get help. Social services work needs that information, because it is how a resident is identified and served. That is also why the recordings could not be handed to a third-party analytics provider as they stood.

Under CCPA and CPRA, recordings full of resident PII could not be shared outside the county's possession. Handing an unredacted archive to any external provider, however reputable, would have created a security exposure the county was not prepared to accept.

So the programme had a hard precondition. Every recording had to be redacted before it left, and redaction had to happen inside the county's own boundary.

This was a new initiative. The county was not replacing an existing redaction process, because there wasn't one. Nothing had been analysed before, precisely because nothing could be shared.

The call centre produces between 7,000 and 10,000 recordings a day, each a few minutes long, and they had to be redacted the same day to be useful to an analysis programme running on current data.

County social services run Monday to Friday and close on weekends and county holidays, which for 2026 is eleven observed days. That works out at roughly 250 operating days, so the annual figure is on the order of two million recordings.

That figure rules out an entire category of solution. Any workflow where an operator opens a file, listens, marks regions and saves is arithmetically impossible at several thousand files a day. Not slow. Impossible. The work had to be unattended by default, with human involvement reserved for exceptions.

A social services call centre is not a constant load. It is busy through the working day, quiet at night, quieter at weekends.

The deployment was built to scale with that: capacity increases as volume climbs through the day and reduces when it falls away, rather than running the daytime peak twenty-four hours a day. For a county paying for its own AWS infrastructure, that is a direct cost question, not an architectural nicety.

At ten thousand files a day, "it appears to be working" is not a report.

The county required reporting on throughput and errors: how many files processed, how many failed, and where. At this volume a small percentage failure is hundreds of files, and it has to be visible rather than inferred.

They also required the ability to spot check and intervene. Most files are never opened by a person. But the county needed to be able to pull a recording, listen to what was redacted, and correct it by hand where the automated pass fell short. In practice this is rarely needed. The capability still had to exist, because a quality programme that cannot audit its own privacy step is not one a county can defend.

Deployment took about two months.

Very little of that was technical. It was government procurement and the coordination of several teams: the business team running the social services call centre, their leadership, and the county's technical team who stood up the infrastructure for the deployment to go into. Access to the call centre system and to the AWS environment each came through its own approval path.

That is worth stating plainly, because it is the realistic shape of a public-sector deployment. The engineering was not the long pole.

The county set an accuracy expectation of at least 98% before deployment. Measured accuracy came in consistently above 99%, averaging close to 99.5%.

That figure was not an impression or a vendor estimate. It was established by building ground truth by hand: listening to audio calls, marking every place where PII was actually spoken, then comparing that list against what the system detected and correctly redacted. The accuracy number is the second set measured against the first.

That method matters as much as the result. It measures the thing that actually creates exposure, which is identifiers that were spoken and never caught. Counting successful detections instead would produce a flattering ratio and tell the county nothing. It is the only way to get a number that means anything.

The gap from the expectation matters more than it looks. At the top of the daily range, 98% leaves about two hundred recordings a day carrying unredacted resident PII to an external provider. At 99.5% it is around fifty. The county set a demanding bar precisely because it understood that arithmetic, and the margin above it is what made the programme viable rather than merely permitted.

The analysis programme went ahead. Qualtrics receives a daily flow of redacted recordings in the structure it expects, and no resident's name, date of birth, address or card number leaves the county's own AWS account to get there.

What the county learned from that analysis is the county's. Our scope began at the recording and ended at the redacted file in the destination bucket; we were never exposed to the analysis itself or its findings. That is the correct boundary for this kind of work, and it is worth stating rather than implying a result we have no visibility of.