Redacting 360-Degree Video and Images: Distortion, Seams and Playback

Open a 360-degree photo in an ordinary image viewer and it looks like a warped panorama, with the ceiling smeared along the top edge and the floor along the bottom. A person who stood behind the camera is split in two, half of them at the left edge and half at the right. That flat picture is how a 360 file is usually stored, and it is why 360 video redaction goes wrong in places an ordinary video never does.

Records and evidence staff often meet these files through crime-scene documentation, where investigators capture a room, a vehicle or a roadside in a single sweep, while privacy teams meet them in street-level imagery. Photos and video from the same camera are not redacted the same way. In VIDIZMO Redactor, a 360 photo is examined as undistorted views of the scene around the camera, while a 360 video is redacted on its flattened frame, stretching and seam included. A test of 360 video therefore has to look for the problems below on every frame.

Our guide to video redaction best practices covers the failures every video shares. What the sphere adds is faces stretched near the top and bottom of the frame, people cut in two at the seam, and playback that depends on metadata a re-encode can drop.

How a sphere becomes a rectangle

A common way to store a sphere in a rectangle is the equirectangular layout, in which longitude runs across the image and latitude runs down it. The left and right edges of the rectangle are the same line on the sphere, directly behind the direction the image is centered on. Crime-scene cameras and consumer 360 cameras alike usually hand over a file in this layout, stitched together from two or more lenses.

The stretching follows simple geometry, because every row of the flat frame is the same width while the circles of latitude it represents shrink toward the poles. At 60 degrees above or below the horizon, anything in the flat frame is drawn twice as wide as it should be. The stretch keeps growing from there, until the single point at each pole is spread across a whole row.

A 360 file announces itself through metadata, since the pixels alone look like an ordinary wide picture. Google's Photo Sphere metadata specification lists the projection type among the required properties of a 360 photo and notes that "Google products currently support the value equirectangular only."

For video, the Spherical Video RFC stores the declaration in "a uniquely-identified moov.trak.uuid box" inside the MP4 file. The Spherical Video V2 RFC moves it into dedicated boxes and defines a cubemap projection, six square faces like the inside of a box, alongside the equirectangular one. A cubemap spreads the distortion more evenly, but it adds seams of its own wherever two faces meet.

What breaks when the flat frame is redacted

A 360 camera records everyone around it, including people behind the operator who had no reason to think they were in the shot. A 360 capture therefore tends to hold more bystanders than a conventional recording of the same scene, and every one of them sits somewhere on the distorted frame. At a crime scene that can mean neighbors at a window, officers at the cordon and onlookers across the street, all captured in the one sweep meant to record the room or the road.

Faces near the top and bottom of the flat frame are stretched sideways into shapes that detection models trained on ordinary photos rarely see, so they are easier to miss. A person standing on the seam appears as two partial people at opposite edges, and each half is harder to detect than a whole person would be. A mask drawn on one edge and not the other leaves half of that person visible in the redacted file. For detection on ordinary footage, see our article on object detection for automated redaction.

Playback is the other weak point, because burning masks into a video means re-encoding it into a new file. The spherical declaration the original carried does not always survive that step, and a 360 video that loses it plays as a stretched rectangle that a requester may reasonably take for a damaged file.

Photos have a quieter version of the same problem, and it starts before redaction does. Apps and messaging services can strip a 360 photo's metadata along the way. Software that relies on the metadata then treats the photo as an ordinary wide picture and runs detection on the stretched frame.

How Street View approached it

Google Street View is a useful reference point, because the National Institute of Standards and Technology (NIST) describes it as "one of the largest deployments of photo de-identification technology." In a 2009 paper on privacy protection in Google Street View, Google's researchers explained that they would "only work with images that came from the individual cameras before they are stitched into a panorama." Detection ran on those camera images, where faces were not yet stretched by the projection.

The detector was "tuned for a high recall, low-precision operating point," accepting many false alarms in order to miss fewer faces. A later stage removed false positives using information the detector alone did not have. For redaction that trade is the right one, since a false alarm costs a reviewer a few seconds and a missed face costs the person in it their privacy. The team also ignored the upward-facing camera entirely, "because any faces that may be captured will be too low-resolution to be identifiable."

The lessons carry over to any 360 redaction, starting with detecting where the geometry distorts faces least. Detection should lean toward catching faces and leave review to remove the extras, and someone has to decide explicitly what happens straight above and below the camera.

Checks for a redacted 360 file

One check belongs before redaction rather than after it, because a file that has lost its 360 metadata is harder to redact well. Open the source in a 360 viewer first. If it shows as a flat picture, ask for the original from the camera or its source system, because a copy that passed through a messaging app or editor may have been stripped. Keep the stripped copy out of the release path altogether, so nobody redacts the flat version by mistake.

The remaining checks need nothing more than the redacted file, the original and a 360 viewer that lets you rotate the view, and they apply to any tool's output.

How Redactor handles 360 photos

The 360 photo work in Redactor began with a requirement to find faces, people and vehicles on uploaded 360 photos and show them to a reviewer before redaction. Running detection directly on the flat image would have met distorted faces and people split at the edges. The approach follows the Street View one instead: detect on undistorted views of the scene around the camera, then map every box back onto the panorama.

Mapping every box back matters for review as much as for masking, because a reviewer confirms and corrects detections on the same panorama the requester will receive. A missed face is easiest to spot there, in the context of the whole scene.

The seam needed its own rule, because a person standing on it would otherwise be masked at one edge and left visible at the other. A box that crosses the line where the left and right edges meet is drawn on both edges. Pieces of one object cut by the boundary between two views are joined into one detection, so a reviewer sees one person and one mask.

A reviewer confirms, corrects and adds to the detections before anything is released, and the job burns blur, pixelation or a solid fill into a new copy. Our article on choosing between blur, pixelation and a solid fill compares what each style leaves behind.

360 video in Redactor, and the hard cases that remain

Because a 360 video is redacted on its flattened frame, review carries more of the load for video than it does for photos. A reviewer steps to the seam and to the top and bottom of the frame at intervals through the recording, since those are the places where a missed detection is most likely.

Inside a VIDIZMO deployment, the redacted copy plays back as a sphere when the item is marked as a 360 video, so set that before anyone reviews the copy there. Playback outside it is a separate question, and it is the one the requester actually experiences. A redacted 360 video downloaded and opened in another player may show a stretched rectangle if the re-encoded file lacks the spherical declaration. That is the case the playback check above catches, and restoring the declaration before release fixes it.

The hard cases in 360 material, photos and video alike, are the same ones the whole field still works on, starting with what sits straight above and below the camera. The poles are the most distorted part of every flat frame, and the Street View system simply set aside its upward camera. A crime-scene capture seldom allows that shortcut, because the floor beneath a tripod-mounted camera can hold documents, evidence markers or a victim, so what gets masked straight down needs a deliberate decision for each capture. A person walking across the seam leaves one edge of the frame and enters the other, so the seam is a tracking problem as well as a detection one. Our article on tracking through occlusion and scene cuts describes how and where tracks break.

Panoramas from 360 cameras are handled alongside every other kind of still image in Redactor's image redaction software.