The problem of designing a privacy-preserving camera (PPC) is considered. Previous designs rely on a static point spread function (PSF), optimized to prevent detection of private visual information, such as recognizable facial features. However, the PSF can be easily recovered by measuring the camera response to a point light source, making these cameras vulnerable to PSF inversion attacks. A new dynamic privacypreserving (DyPP) camera design is proposed to prevent such attacks. DyPP cameras rely on dynamic optical elements, such spatial light modulators, to implement a time-varying PSF, which changes from picture to picture. PSFs are drawn randomly with a learned manifold embedding, trained adversarially to simultaneously meet user-specified targets for privacy, such as face recognition accuracy, and task utility. Empirical evaluations on multiple privacy-preserving vision tasks demonstrate that the DyPP design is significantly more robust to PSF inversion attacks than previous PPCs. Furthermore, the hardware feasibility of the approach is validated by a proof-of-concept camera model.

Methodology

---

PPCs that implement a static PSF h enable the solution of tasks like object detection while maintaining privacy (top-left). However, this protection can be easily overridden with PSF inversion attacks. A point light source is used to recover h, allowing the recovery of subject identities by simple deconvolution (bottom-left). The DyPP camera uses a dynamic PSF to prevent these attacks (right). A PSF h𝑡 is randomly sampled from the manifold of privacy preserving camera parameters before picture 𝑡 is taken. This creates a mismatch with a PSF h𝑡′ obtained via inversion attacks, preventing the recovery of private information.

End-to-end training of the DyPP camera. A manifold embedding 𝑔 maps a random code 𝜖 into a vector 𝛼 of PSF parameters on the privacy manifold P. This produces a PSF h(α). Given scene x, the camera then produces measurements x˜. The embedding 𝑔 is trained with task and utility losses that guarantee the balance between privacy (upper bound on face recognition accuracy) and utility (lower bound on target task accuracy) desired for the the privacy manifold P. Double arrows that indicate both forward and backward propagation are performed.

Hardware Proof-of-concept

---

Camera setup and sample images. Left: Real-world privacy camera implementation: 1. linear polarizer, 2. Kowa 5 mm, f/1.8 lens, 3. Iris as a field stop, 4. 50mm achromatic lens, 5. HOLOEYE PLUTO SLM, 6. beam splitter, 7. 50 mm Canon camera lens, 8. Sony IMX178 board level sensor. Right: (from top to bottom) The ground truth image, privacy-preserving measurement, Privacy-preserving pose estimation.

Poster

---

Paper

---

PDF

Supplement

Code

Bibtex

Acknowledgements

We thank Carlos Hinojosa for sharing the PSF of PP-HPE. This work was partially funded by NSF award IIS-2303153, a gift from Qualcomm, and NVIDIA GPU donations. We also acknowledge and thank the use of the Nautilus platform for some of the experiments discussed above.