Video Redaction Best Practices: Motion, Frame Rates, Tracking and Failing Safe

I led our video redaction project for a county public safety agency where two people handled every disclosure, and video was taking them an average of 62 days to release. In the first six months after deployment, with the same two people doing the work, that average fell to seven days, as the agency's case study records. Most of what I'd now call video redaction best practices I learned on projects like that one, from footage that behaved nothing like a vendor's demo clip.

Before we could redact anything there, we had to integrate with the systems that held the footage. It arrived from the agency's cloud evidence system for body-worn cameras, from its case management system and from systems the county had built for itself. It came in many formats, and each one had to be converted before redaction could start. We deployed VIDIZMO Redactor and AI Intelligence Hub on the agency's own infrastructure, alongside the evidence system it already ran, so nothing had to be migrated. AI Intelligence Hub reached that evidence system through its API so investigators could search what the footage showed, and the redaction itself was Redactor's.

Why a video redaction fails one frame at a time

Release times like that county's are safe only if the redacted file holds on every frame, including the thousands nobody watched. At 30 frames per second, an hour of body camera footage holds 108,000 separate images, and a single unmasked frame is on screen for a thirtieth of a second. That is still long enough to expose someone, because a requester can pause on it and capture the face.

The Scientific Working Group on Digital Evidence (SWGDE) builds this into its definition of selective redaction, which "requires the user to review specific content and track object(s) through sequential frames." Its Video and Audio Redaction Guidelines add that in manual and automated redaction alike, "the completed effect should be reviewed by the practitioner and requestor for quality assurance and accuracy prior to release." I treat that review as the backstop, and everything else we build is meant to leave it less to catch.

Every decision in a video redaction weighs an exposed face, which can't be recalled once the file has left the building, against over-redaction, which withholds more of the record than the law allows. In the US, the federal Freedom of Information Act (FOIA) requires that "any reasonably segregable portion of a record shall be provided to any person requesting such record after deletion of the portions which are exempt." Deadline pressure pushes toward over-redaction, because masking a whole scene is faster than tracking the people in it, and a release built that way can fail the segregability test even when no face leaks.

SWGDE's workflow also starts from a working copy of the source recording, so the original survives whatever happens to the copy, and it says that "all steps in the redaction process should be appropriately documented." On the county project the evidence system stayed the system of record, and every redaction carried the exemption it was made under and a note giving the reason.

Frame rates, conversions and the redaction record

The conversions on the county project taught me to check timing first, because a mask is placed by time and every conversion is a chance for a file's timing to change. The frame rate shown in a file's properties is only a summary stored apart from the frames. It can be wrong by a factor of two while the video plays normally, since a player shows each frame at its own timestamp.

Software that places masks by counting frames at the declared rate does depend on that summary, so a wrong rate moves every mask a little further from its face as the recording runs. A file written out at a wrong rate also drifts away from its own sound, so lips and voice fall out of step by the end of a long recording.

We've measured frame rates misstated by a factor of two in real body camera and closed-circuit television (CCTV) recordings, and no single frame-rate field was right in all of them. Two of those files are taken apart in our write-up on variable frame rate video. Redactor places every mask by its own frame's timestamp and converts a variable frame rate source to a constant rate before masking, so a wrong header can't slide a mask off its frame.

A converted copy no longer matches its source, and whoever presents it later has to be able to say how it changed. For any copy that may be played in court, I'd record whether the source was converted to a constant frame rate, re-encoded or moved into a new container. That won't settle whether a court accepts the copy, but it lets whoever presents it explain every way it differs from the original.

Tracking, gaps and cameras that move

A track is the software's claim that one person sits under a mask from the moment they appear until they leave. When that claim breaks, the result can be a bare frame, a mask handed to the wrong person, or one person's track cut into pieces that a reviewer has to find and join.

The most ordinary break comes when someone steps behind a truck, a doorframe or another person, and the frames on either side of that gap are where a face tends to flash into view. What I ask of any tool is one identity for that person when they reappear and a mask on both edges of the gap. Redactor does both through a short occlusion, and it resets tracking at a hard cut so an identity can't carry over to whoever stands in the same spot in the next shot.

Someone hidden for longer comes back as a new track, which a reviewer merges with the first. Review in Redactor works on whole tracks, so each correction applies to a person's entire appearance, and anyone the detector missed can be drawn in once and tracked like the rest. Crowds make every one of these breaks more likely, and a closer look at how tracking fails through occlusion, crowds and scene cuts follows each failure to what a reviewer does about it.

Masks also have to cover the frames between known positions, where the simplest rule, moving the box in a straight line, lags any subject that turns or speeds up in between. Redactor covers the start and end positions together for the whole interval, so the subject stays covered wherever it went. A worked example of filling the frames between analyzed ones shows how far a straight-line box slips on a single turn, and where extra coverage starts to withhold more than the segregability rule allows.

Body-worn cameras, dash cams and drones add a moving camera to every one of those problems. Faces enter close to the lens and change size within a second, while motion smears features and light swings from headlights to shadow. These recordings therefore put more of the load on review than footage from a fixed camera does. For checks device by device, from plates crossing a windshield to people seen from above, I'd send readers to what a moving camera does to dash cam, body cam and drone redaction.

360 files, mask styles and failing safe

A 360-degree camera records a full sphere around it and stores that sphere as a flat rectangle. The flattening stretches faces near the top and bottom of the frame and splits anyone standing on the seam into two partial people at opposite edges. A mask that stops at one edge leaves half of that person visible, and a re-encoded file can lose the metadata that makes it play back as a sphere at all. If 360 captures are starting to reach your unit, how the seam, the poles and playback trip up 360 redaction is the place to start.

The choice of mask decides how much of a face survives underneath it, and video gives anyone trying to recover that face more to work with than a still photo does. Blur and pixelation both hide a face at normal speed while leaving a degraded copy of it in the frame. Across a recording those copies add up, since every frame shows the face slightly differently.

A solid fill is the one style that leaves nothing of the original under the mask, and it can also carry the exemption code. FOIA asks that, if technically feasible, the exemption be shown "at the place in the record where such deletion is made." Redactor applies blur, pixelation or a solid fill per job and can draw the exemption code onto the mask.

The principle I hold every redaction job to comes from security engineering, where it was worked out long before anyone redacted video. Jerome Saltzer and Michael Schroeder argued for fail-safe defaults in their 1975 paper The Protection of Information in Computer Systems. A mechanism designed that way tends to "fail by refusing permission, a safe situation, since it will be quickly detected." The opposite design tends to fail by allowing access, "a failure which may go unnoticed in normal use."

In redaction, the unnoticed failure is a job that stops partway and still leaves a file that exists, plays and reports success. We built Redactor so that a stage that stops early, an encoder that exits with an error or a run past its time limit fails the whole job. The partial file is then deleted before anyone can release it. A single frame whose masking raises an error is written fully masked instead of passing through as recorded. The research on recovering blurred faces, and the failures behind these rules, are in our engineers' comparison of blur, pixelation and solid fill.

What I tell agencies before a pilot

Video redaction gets hard in exactly the conditions a demo avoids, and I tell every agency to expect all of them in its own footage. Cameras shake and move with the officer, and people pass behind cars, doors and each other. Camera angle and lighting change what a detector can see from one recording to the next. Children in frame are handled differently from adult bystanders when a release is decided, and a lot of footage now comes from phones, held upright, sideways or turned partway through.

That's why I push for a pilot on the agency's own recordings, exported from the systems that will actually send them, since clean sample clips show almost none of those conditions. I don't believe any AI is perfect for every organization and every use case in video, because the conditions that decide how well detection works differ from one agency's cameras to the next. We calibrate detection to each organization's own footage for that reason, and the pilot measures the result before anyone relies on it.

The number I'd measure in that pilot is review minutes per hour of footage, because it shows what the software saves and what it still leaves for a person to do. Release time is the figure people remember from the county project, but review time is what moved it. The median time a reviewer spent on a video fell from two and a half hours to 25 minutes. The median hides the hard cases, and a handful of multi-camera incidents each month still take 12 to 15 hours. A pilot should include incidents like those, because they pull the average well above the median and decide how long the hardest releases take.

The vendor you pilot with shapes the result too, because detection improves most when the vendor understands the conditions your cameras actually record in. Look for one that will work through your footage with you and tailor the solution to it, then trust them and work closely with them. At times we've gone as far as fine-tuning models for a customer's specific conditions, and the results improved on the footage that customer cared about.

Turning video redaction best practices into release checks

Checks run on the released file confirm that what was decided in review actually reached it, and they work without trusting the software that produced it, ours included. I'd keep running them on a sample even after a tool has earned an agency's confidence, because the failures they catch are rare and silent. For a small unit, a workable sample is every release that contains a crowd, a cut or a 360 file, plus a few routine ones each week, with every check noted in the redaction log.

What to license, and where it runs

For this work you'd license VIDIZMO Redactor, which handles detection, tracking, review and masking on the VIDIZMO platform that stores the recordings, keeps the originals and controls who can see what. AI Intelligence Hub is licensed separately, for agencies that also want investigators to search what their footage shows. Both run wherever an agency chooses: hosted by VIDIZMO on shared or dedicated infrastructure, in the agency's own cloud account, on its own servers, or air-gapped with detection running on local models. For criminal justice footage, VIDIZMO offers CJIS-compliant deployments.

Detection, review and output are set out in full on Redactor's video redaction software page.