News

• Three papers accepted to appear in TPDP 2020, with the work on Characterizing Private Clipped Gradient Descent on Convex Generalized Linear Problems accepted for an oral presentation.
• Our paper titled Guaranteed Validity for Empirical Approaches to Adaptive Data Analysis has been accepted to appear in AISTATS 2020.
• I joined Google as a Research Software Engineer in the team of Françoise Beaufays on September 30, 2019.
• I successfully defended my dissertation titled Advances in Privacy-Preserving Machine Learning on August 1, 2019.
• I was a Visiting Graduate Student in the Data Privacy program at Simons, Berkeley during Spring'19.
• Our paper titled Model-Agnostic Private Learning was accepted for an oral presentation at NeurIPS 2018.
• Our paper titled Towards Practical Differentially Private Convex Optimization was accepted in S&P 2019.

Resume

• My most recent resume (last updated in September, 2020) can be found here.

Manuscripts

• Training Production Language Models without Memorizing User Data.
  Joint work with Swaroop Ramaswamy, Rajiv Mathews, Galen Andrew, Brendan McMahan, and Françoise Beaufays.
• Privacy Amplification via Random Check-Ins.
  Joint work with Borja Balle, Peter Kairouz, Brendan McMahan, and Abhradeep Thakurta.
• Understanding Unintended Memorization in Federated Learning.
  Joint work with Swaroop Ramaswamy, Rajiv Mathews, and Françoise Beaufays.
• Characterizing Private Clipped Gradient Descent on Convex Generalized Linear Problems.
  Joint work with Shuang Song, and Abhradeep Thakurta.
• Differentially Private Learning with Adaptive Clipping.
  Joint work with Galen Andrew, and Brendan McMahan.

Publications

• Guaranteed Validity for Empirical Approaches to Adaptive Data Analysis.
  Joint work with Ryan Rogers, Aaron Roth, Adam Smith, Nathan Srebro, and Blake Woodworth.
  To appear in the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS 2020).
• Advances in Privacy-Preserving Machine Learning.
  Ph.D. Thesis, BU, September 2019.
  Supervisor: Dr. Adam Smith.
• Towards Practical Differentially Private Convex Optimization. Abstract▼

  Building useful predictive models often involves learning from sensitive data. Training models with differential privacy can guarantee the privacy of such sensitive data. For convex optimization tasks, several differentially private algorithms are known, but none has yet been deployed in practice.
  
  In this work, we make two major contributions towards practical differentially private convex optimization. First, we present Approximate Minima Perturbation, a novel algorithm that can leverage any off-the-shelf optimizer. We show that it can be employed without any hyperparameter tuning, thus making it an attractive technique for practical deployment. Second, we perform an extensive empirical evaluation of the state-of-the-art algorithms for differentially private convex optimization, on a range of publicly available benchmark datasets, and real-world datasets obtained through an industrial collaboration. We release open-source implementations of all the differentially private convex optimization algorithms considered, and benchmarks on as many as nine public datasets, four of which are high-dimensional.

  
  Joint work with Roger Iyengar, Joseph P. Near, Dawn Song, Abhradeep Thakurta, and Lun Wang.
  In the 40th IEEE Symposium on Security and Privacy (S&P 2019).
  • Conference Talk Video (Preview)
  • Code Repo
• Model-Agnostic Private Learning. Abstract▼

  We design differentially private learning algorithms that are agnostic to the learning model assuming access to a limited amount of unlabeled public data. First, we provide a new differentially private algorithm for answering a sequence of $m$ online classification queries (given by a sequence of $m$ unlabeled public feature vectors) based on a private training set. Our algorithm follows the paradigm of subsample-and-aggregate, in which any generic non-private learner is trained on disjoint subsets of the private training set, and then for each classification query, the votes of the resulting classifiers ensemble are aggregated in a differentially private fashion. Our private aggregation is based on a novel combination of the distance-to-instability framework, and the sparse-vector technique. We show that our algorithm makes a conservative use of the privacy budget. In particular, if the underlying non-private learner yields a classification error of at most $\alpha\in (0, 1)$, then our construction answers more queries, by at least a factor of $1/\alpha$ in some cases, than what is implied by a straightforward application of the advanced composition theorem for differential privacy. Next, we apply the knowledge transfer technique to construct a private learner that outputs a classifier, which can be used to answer an unlimited number of queries. In the (agnostic) PAC model, we analyze our construction and prove upper bounds on the sample complexity for both the realizable and the non-realizable cases. Similar to non-private sample complexity, our bounds are completely characterized by the VC dimension of the concept class.

  
  Joint work with Raef Bassily, and Abhradeep Thakurta.
  In the 32nd Conference on Neural Information Processing Systems (NeurIPS 2018). Accepted for an oral presentation.
  • Conference Talk Video
  • Poster
• Differentially Private Matrix Completion Revisited. Abstract▼

  We provide the first provably joint differentially private algorithm with formal utility guarantees for the problem of user-level privacy-preserving collaborative filtering. Our algorithm is based on the Frank-Wolfe method, and it consistently estimates the underlying preference matrix as long as the number of users $m$ is $\omega(n^{5/4})$, where $n$ is the number of items, and each user provides her preference for at least $\sqrt{n}$ randomly selected items. Along the way, we provide an optimal differentially private algorithm for singular vector computation, based on the celebrated Oja's method, that provides significant savings in terms of space and time while operating on sparse matrices. We also empirically evaluate our algorithm on a suite of datasets, and show that it provides nearly same accuracy as the state-of-the-art non-private algorithm, and outperforms the state-of-the-art private algorithm by as much as 30%.

  
  Joint work with Prateek Jain, and Abhradeep Thakurta.
  In the 35th International Conference on Machine Learning (ICML 2018). Presented as a long talk.
  • Conference Talk Video
  • Poster
• Max-Information, Differential Privacy, and Post-Selection Hypothesis Testing. Abstract▼

  In this paper, we initiate a principled study of how the generalization properties of approximate differential privacy can be used to perform adaptive hypothesis testing, while giving statistically valid $p$-value corrections. We do this by observing that the guarantees of algorithms with bounded approximate \emph{max-information} are sufficient to correct the $p$-values of adaptively chosen hypotheses, and then by proving that algorithms that satisfy $(\epsilon,\delta)$-differential privacy have bounded approximate max-information when their inputs are drawn from a product distribution.
  
  This substantially extends the existing connection between differential privacy and max-information, which previously was only known to hold for (pure) $(\epsilon,0)$-differential privacy. It also extends our understanding of max-information as a partially unifying measure controlling the generalization properties of adaptive data analyses. We also show a lower bound, proving that (despite the strong composition properties of max-information), when data is drawn from a product distribution, $(\epsilon,\delta)$-differentially private algorithms can come \emph{first} in a composition with other algorithms satisfying max-information bounds, but not necessarily second if the composition is required to itself satisfy a nontrivial max-information bound. This, in particular, implies that the connection between $(\epsilon,\delta)$-differential privacy and max-information holds only for inputs drawn from particular distributions, unlike the connection between $(\epsilon,0)$-differential privacy and max-information.

  
  Joint work with Ryan Rogers, Aaron Roth, and Adam Smith.
  In the 57th Annual IEEE Symposium on Foundations of Computer Science (FOCS 2016).
  • Poster

Workshop Publications

• Privacy Amplification via Random Check-Ins.
  Joint work with Borja Balle, Peter Kairouz, Brendan McMahan, and Abhradeep Thakurta.
  To appear in Theory and Practice of Differential Privacy (TPDP) 2020.
• Understanding Unintended Memorization in Federated Learning.
  Joint work with Swaroop Ramaswamy, Rajiv Mathews, and Françoise Beaufays.
  To appear in Theory and Practice of Differential Privacy (TPDP) 2020.
• Characterizing Private Clipped Gradient Descent on Convex Generalized Linear Problems.
  Joint work with Shuang Song, and Abhradeep Thakurta.
  To appear in Theory and Practice of Differential Privacy (TPDP) 2020. Accepted for an oral presentation.

Internships and Research Visits

• Visiting Graduate Student in the Data Privacy program at the Simons Institute, Berkeley during Spring'19.
• Research Intern at Google Brain, Mountain View, CA during Summer 2018. Mentor: Úlfar Erlingsson.
• Visiting Student Researcher at University of California, Berkeley, CA during Fall 2017. Host: Dr. Dawn Song.
• Research Intern at Google, Seattle, WA during Summer 2017. Mentors: Brendan McMahan, and Martin Pelikan.
• Research Intern in the CoreOS: Machine Learning team at Apple, Cupertino, CA during Summer 2016.

Talks and Poster Presentations

• Towards Practical Differentially Private Convex Optimization
• @ the Future of Privacy Forum booth, Global Privacy Summit 2019, Washington, DC on May 3, 2019. (Coverage)
• @ the Privacy Tools Project meeting, Harvard on March 5, 2018.
• Model-Agnostic Private Learning,
• @ the 2019 IEEE North American School of Information Theory (NASIT), held at BU, on July 3, 2019. (Poster)
• @ the 2018 Open AIR: Industry Open House, BU on October 12, 2018. (Poster)
• Building Tools for Controlling Overfitting in Adaptive Data Analysis, @ the Adaptive Data Analysis workshop, Simons Institute, Berkeley on July 7, 2018.
• Differentially Private Matrix Completion Revisited
• @ the Mathematical Foundations of Data Privacy workshop, BIRS on May 2, 2018. (Talk video)
• @ the BU Data Science (BUDS) Day, Boston University on January 26, 2018. (Poster)
• @ the Privacy Tools Data Sharing workshop, Harvard University on December 12, 2017. (Poster)
• @ the Security Seminar, UC Berkeley on October 9, 2017.
• A brief introduction to Concentrated Differential Privacy, @ CSE Theory Seminar, Penn State on April 14, 2017.
• Max-Information, Differential Privacy, and Post-selection Hypothesis Testing
• @ INSR Industry Day, Penn State on April 24, 2017. (Poster)
• @ SMAC Talks, Penn State on December 2, 2016.
• @ CSE Theory Seminar, UCSD on November 7, 2016.
• @ CSE Theory Seminar, Penn State on October 14, 2016.
• Max-Information and Differential Privacy, @ CSE Theory Seminar, Penn State on May 5, 2016.
• The Stable Roommates Problem with Random Preferences, @ CSE Theory Seminar, Penn State on April 10, 2015.
• The Multiplicative Weights Update Method and an Application to Solving Zero-Sum Games Approximately, @ CSE Theory Seminar, Penn State on November 3, 2014.

Teaching

• Teaching assistant:
• CMPSC 465 Data Structures and Algorithms, Spring 2017 @ Penn State.
• CMPSC 360 Discrete Mathematics for Computer Science, Spring 2015 @ Penn State.
• IT 114 Object Oriented Programming, Spring 2014 @ DA-IICT.
• IT 105 Introduction to Programming, Fall 2013 @ DA-IICT.

Professional Activities

• Program committee member for TPDP 2020.
• Reviewer for JPC 2019, T-IFS 2019, JMLR 2018.
• Reviewer for NIST's The Unlinkable Data Challenge: Advancing Methods in Differential Privacy.
• Reviewer for NeurIPS (2019-2020), PETS (2017-2020), IJCAI 2019, CCS (2018-2019), S&P (2017, 2019), ICML 2018, STOC (2016, 2018), ACSAC 2017, FOCS 2017, WABI 2015.

Recent Awards

• Received a travel award for S&P 2019.
• Received a travel award for NeurIPS 2018.
• Received a travel award for ICML 2018.
• Received a GSO Conference Travel Grant for Summer 2018.
• Received a registration award for FOCS 2014.

Miscellaneous

• Report on Node-differentially Private Algorithms for Graph Statistics. It includes joint work with Ramesh Krishnan.