DPGEM: Differentially Private Generative Model with Exponential Mechanism

The growing use of generative AI models for managing sensitive data presents substantial privacy risks. Differential privacy (DP) methods can mitigate such risks but approaches based on the Gaussian mechanism, such as Differentially Private Stochastic Gradient Descent (DPSGD) and Private Aggregation of Teacher Ensembles (PATE), often produce low-quality synthetic data due to the addition of noise during training. To address this issue, several approaches explored the distribution of private data in the latent space of a publicly-trained generative model, by either building a secondary DP-GAN on the latent space [1], [2] or direct sampling via exponential mechanism (EM) [3]. However, these methods still face challenges in identifying regions in the latent space that yield high-quality private images and in managing the privacy budget effectively. In this paper, we introduce DPGEM (Differentially Private Generative Model with Exponential Mechanism), which enhances the scoring function by employing a classifier trained on synthetic representations of private and public data. We propose a local differential privacy composition rule to achieve tighter privacy bounds and an Uncertainty-Based Spreader Identification Method (UBSIM) to improve privacy accounting. Experiments on MNIST and Fashion-MNIST datasets demonstrate that DPGEM generates high-quality images with competitive performance while maintaining strong privacy guarantees.