• Documents
  • Authors
  • Tables
  • Log in
  • Sign up
  • MetaCart
  • DMCA
  • Donate

CiteSeerX logo

Advanced Search Include Citations
Advanced Search Include Citations

Distance makes the types grow stronger: a calculus for differential privacy. (2010)

by J Reed, B C Pierce
Venue:ACM Sigplan Notices,
Add To MetaCart

Tools

Sorted by:
Results 1 - 10 of 54
Next 10 →

Probabilistically accurate program transformations

by Sasa Misailovic, Daniel M. Roy, Martin C. Rinard - In SAS , 2011
"... Abstract. The standard approach to program transformation involves the use of discrete logical reasoning to prove that the transformation does not change the observable semantics of the program. We propose a new approach that, in contrast, uses probabilistic reasoning to justify the application of t ..."
Abstract - Cited by 38 (14 self) - Add to MetaCart
Abstract. The standard approach to program transformation involves the use of discrete logical reasoning to prove that the transformation does not change the observable semantics of the program. We propose a new approach that, in contrast, uses probabilistic reasoning to justify the application of transformations that may change, within probabilistic accuracy bounds, the result that the program produces. Our new approach produces probabilistic guarantees of the form P(|D | ≥ B) ≤ ɛ, ɛ ∈ (0, 1), where D is the difference between the results that the transformed and original programs produce, B is an acceptability bound on the absolute value of D, and ɛ is the maximum acceptable probability of observing large |D|. We show how to use our approach to justify the application of loop perforation (which transforms loops to execute fewer iterations) to a set of computational patterns. 1
(Show Context)

Citation Context

...o provide a justification for applying loop perforation. Reed and Pierce present a type system for capturing function sensitivity, which measures how much a function may magnify changes to its inputs =-=[32]-=-. Although the language contains probabilistic constructs, the type system uses deterministic worst-case reasoning, resulting in a worst-case sensitivity bound. Modeling Uncertainty. Typical approache...

Proving Programs Robust ∗

by Swarat Chaudhuri, Sumit Gulwani, Sara Navidpour, Roberto Lublinerman
"... We present a program analysis for verifying quantitative robustness properties of programs, stated generally as: “If the inputs of a program are perturbed by an arbitrary amount ɛ, then its outputs change at most by Kɛ, where K can depend on the size of the input but not its value. ” Robustness prop ..."
Abstract - Cited by 38 (6 self) - Add to MetaCart
We present a program analysis for verifying quantitative robustness properties of programs, stated generally as: “If the inputs of a program are perturbed by an arbitrary amount ɛ, then its outputs change at most by Kɛ, where K can depend on the size of the input but not its value. ” Robustness properties generalize the analytic notion of continuity—e.g., while the function e x is continuous, it is not robust. Our problem is to verify the robustness of a function P that is coded as an imperative program, and can use diverse data types and features such as branches and loops. Our approach to the problem soundly decomposes it into two subproblems: (a) verifying that the smallest possible perturbations to the inputs of P do not change the corresponding outputs significantly, even if control now flows
(Show Context)

Citation Context

...ess. So far as we know, Hamlet [13] was the first to argue for a testing methodology for Lipschitz-continuity of software. However, he failed to offer new program analysis techniques. Reed and Pierce =-=[25]-=- have since given a type system that can verify the Lipschitz-continuity of functional programs, as a component of a new language for differential privacy [24]. While the system can seamlessly handle ...

Geo-indistinguishability: Differential privacy for location-based systems

by Miguel E. Andrés, École Polytechnique, Nicolás E. Bordenabe, Konstantinos Chatzikokolakis, Catuscia Palamidessi, École Polytechnique , 2012
"... The growing popularity of location-based systems, allowing un-known/untrusted servers to easily collect huge amounts of informa-tion regarding users ’ location, has recently started raising serious privacy concerns. In this paper we introduce geo-indistinguisha-bility, a formal notion of privacy for ..."
Abstract - Cited by 28 (5 self) - Add to MetaCart
The growing popularity of location-based systems, allowing un-known/untrusted servers to easily collect huge amounts of informa-tion regarding users ’ location, has recently started raising serious privacy concerns. In this paper we introduce geo-indistinguisha-bility, a formal notion of privacy for location-based systems that protects the user’s exact location, while allowing approximate in-formation – typically needed to obtain a certain desired service – to be released. This privacy definition formalizes the intuitive notion of protect-ing the user’s location within a radius r with a level of privacy that depends on r, and corresponds to a generalized version of the well-known concept of differential privacy. Furthermore, we present a mechanism for achieving geo-indistinguishability by adding con-trolled random noise to the user’s location. We describe how to use our mechanism to enhance LBS appli-cations with geo-indistinguishability guarantees without compro-mising the quality of the application results. Finally, we compare state-of-the-art mechanisms from the literature with ours. It turns out that, among all mechanisms independent of the prior, our mech-anism offers the best privacy guarantees.
(Show Context)

Citation Context

...distinguishability is an instance of a generalized variant of differential privacy, using an arbitrary metric between secrets. This generalized formulation has been known for some time: for instance, =-=[31]-=- uses it to perform a compositional analysis of standard differential privacy for functional programs, while [16] uses metrics between individuals to define “fairness” in classification. On the other ...

Verifying Quantitative Reliability for Programs That Execute on Unreliable Hardware

by Michael Carbin, Sasa Misailovic, Martin C. Rinard
"... Emerging high-performance architectures are anticipated to contain unreliable components that may exhibit soft errors, which silently corrupt the results of computations. Full detection and masking of soft errors is challenging, expensive, and, for some applications, unnecessary. For example, approx ..."
Abstract - Cited by 27 (3 self) - Add to MetaCart
Emerging high-performance architectures are anticipated to contain unreliable components that may exhibit soft errors, which silently corrupt the results of computations. Full detection and masking of soft errors is challenging, expensive, and, for some applications, unnecessary. For example, approximate computing applications (such as multimedia processing, machine learning, and big data analytics) can often naturally tolerate soft errors. We present Rely, a programming language that enables developers to reason about the quantitative reliability of an application – namely, the probability that it produces the correct result when executed on unreliable hardware. Rely allows developers to specify the reliability requirements for each value that a function produces. We present a static quantitative reliability analysis that verifies quantitative requirements on the reliability of an application, enabling a developer to perform sound and verified reliability engineering. The analysis takes a Rely program with a reliability specification and a hardware specification that characterizes the reliability of the underlying hardware components and verifies that the program satisfies its reliability specification when executed on the underlying unreliable hardware platform. We demonstrate the application of quantitative reliability analysis on six computations implemented
(Show Context)

Citation Context

..., 59, 60, 62, 68], or injected soft errors [20, 37, 38, 64, 70]. Researchers have developed static techniques that use probabilistic reasoning to characterize the accuracy impact of various phenomena =-=[6, 18, 43, 47, 56, 72]-=-. And of course, the accuracy impact of the floating point approximation to real arithmetic has been extensively studied by numerical analysts [17]. 8.4 Probabilistic Program Analysis Kozen’s work [32...

Relational verification using product programs

by Gilles Barthe, Juan Manuel Crespo - In Formal Methods, Lecture Notes in Computer Science , 2011
"... Abstract. Relational program logics are formalisms for specifying and verifying properties about two programs or two runs of the same pro-gram. These properties range from correctness of compiler optimizations or equivalence between two implementations of an abstract data type, to properties like no ..."
Abstract - Cited by 22 (4 self) - Add to MetaCart
Abstract. Relational program logics are formalisms for specifying and verifying properties about two programs or two runs of the same pro-gram. These properties range from correctness of compiler optimizations or equivalence between two implementations of an abstract data type, to properties like non-interference or determinism. Yet the current tech-nology for relational verification remains underdeveloped. We provide a general notion of product program that supports a direct reduction of re-lational verification to standard verification. We illustrate the benefits of our method with selected examples, including non-interference, standard loop optimizations, and a state-of-the-art optimization for incremental computation. All examples have been verified using the Why tool. 1
(Show Context)

Citation Context

...tions on its output. While program continuity is formalized by a formula of the form ∀ >0.∃ δ>0. P , see e.g. [9], continuity can be often derived from the stronger notion of 1-sensitivity, see e.g. =-=[21]-=-. Informally, a program is 1-sensitive if it does not make the distance grow, i.e. the variation of the outputs of two different runs is upper bounded by the variation of the corresponding inputs. Con...

Differential Privacy Under Fire

by Andreas Haeberlen, Benjamin C. Pierce, Arjun Narayan
"... Anonymizing private data before release is not enough to reliably protect privacy, as Netflix and AOL have learned to their cost. Recent research on differential privacy opens a way to obtain robust, provable privacy guarantees, and systems like PINQ and Airavat now offer convenient frameworks for p ..."
Abstract - Cited by 21 (4 self) - Add to MetaCart
Anonymizing private data before release is not enough to reliably protect privacy, as Netflix and AOL have learned to their cost. Recent research on differential privacy opens a way to obtain robust, provable privacy guarantees, and systems like PINQ and Airavat now offer convenient frameworks for processing arbitrary userspecified queries in a differentially private way. However, these systems are vulnerable to a variety of covertchannel attacks that can be exploited by an adversarial querier. We describe several different kinds of attacks, all feasible in PINQ and some in Airavat. We discuss the space of possible countermeasures, and we present a detailed design for one specific solution, based on a new primitive we call predictable transactions and a simple differentially private programming language. Our evaluation, which relies on a proof-of-concept implementation based on the Caml Light runtime, shows that our design is effective against remotely exploitable covert channels, at the expense of a higher query completion time. 1
(Show Context)

Citation Context

...rary queries and can observe all the (remotely measurable) channels that are possible in our model. We present the design of Fuzz, a system that implements this defense. Fuzz uses a novel type system =-=[25]-=- to statically infer the privacy cost of arbitrary queries written in a special programming language, and it uses a novel primitive called predictable transactions to ensure that a potentially adversa...

Measure Transformer Semantics for Bayesian Machine Learning

by Johannes Borgström, Andrew D. Gordon, Michael Greenberg, James Margetson, Jurgen Van Gael
"... Abstract. The Bayesian approach to machine learning amounts to inferring posterior distributions of random variables from a probabilistic model of how the variables are related (that is, a prior distribution) and a set of observations of variables. There is a trend in machine learning towards expres ..."
Abstract - Cited by 16 (4 self) - Add to MetaCart
Abstract. The Bayesian approach to machine learning amounts to inferring posterior distributions of random variables from a probabilistic model of how the variables are related (that is, a prior distribution) and a set of observations of variables. There is a trend in machine learning towards expressing Bayesian models as probabilistic programs. As a foundation for this kind of programming, we propose a core functional calculus with primitives for sampling prior distributions and observing variables. We define combinators for measure transformers, based on theorems in measure theory, and use these to give a rigorous semantics to our core calculus. The original features of our semantics include its support for discrete, continuous, and hybrid measures, and, in particular, for observations of zero-probability events. We compile our core language to a small imperative language that has a straightforward semantics via factor graphs, data structures that enable many efficient inference algorithms. We use an existing inference engine for efficient approximate inference of posterior marginal distributions, treating thousands of observations per second for large instances of realistic models. 1
(Show Context)

Citation Context

...of Probabilistic Languages Probabilistic languages with formal semantics find application in many areas apart from machine learning, including databases [6], model checking [19], differential privacy =-=[24, 34]-=-, information flow [20], and cryptography [1]. A recent monograph on semantics for labelled Markov processes [29] focuses on bisimulation-based equational reasoning. The syntax and semantics of Imp is...

Linear Dependent Types for Differential Privacy

by Marco Gaboardi, Andreas Haeberlen, Justin Hsu, Arjun Narayan, Benjamin C. Pierce
"... Differential privacy offers a way to answer queries about sensitive information while providing strong, provable privacy guarantees, ensuring that the presence or absence of a single individual in the database has a negligible statistical effect on the query’s result. Proving that a given query has ..."
Abstract - Cited by 16 (7 self) - Add to MetaCart
Differential privacy offers a way to answer queries about sensitive information while providing strong, provable privacy guarantees, ensuring that the presence or absence of a single individual in the database has a negligible statistical effect on the query’s result. Proving that a given query has this property involves establishing a bound on the query’s sensitivity—how much its result can change when a single record is added or removed. A variety of tools have been developed for certifying that a given query is differentially private. In one approach, Reed and Pierce [34] proposed a functional programming language, Fuzz, for writing differentially private queries. Fuzz uses linear types to track sensitivity and a probability monad to express randomized computation; it guarantees that any program with a certain type is differentially private. Fuzz can successfully verify many useful queries. However, it fails when the sensitivity analysis depends on values that are not known statically. We present DFuzz, an extension of Fuzz with a combination of linear indexed types and lightweight dependent types. This combination allows a richer sensitivity analysis that is able to certify a larger class of queries as differentially private, including ones whose sensitivity depends on runtime information. As in Fuzz, the differential privacy guarantee follows directly from the soundness theorem of the type system. We demonstrate the enhanced expressivity of DFuzz by certifying differential privacy for a broad class of iterative algorithms that could not be typed previously. Categories and Subject Descriptors D.3.2 [Programming Languages]: Language Classifications—Specialized application languages;
(Show Context)

Citation Context

...lysis results of larger and larger subqueries. This is the basis of previous systems like PINQ [26], which provides a SQL-like language, Airavat [36], which implements a MapReduce framework, and Fuzz =-=[20, 33, 34]-=-, a higher-order functional language. The analysis in Fuzz is based on a type system [33, 34] that certifies queries as differentially private via two components: numeric annotations at the type level...

Broadening the scope of Differential Privacy Using Metrics ⋆

by Konstantinos Chatzikokolakis, Miguel E. Andrés, Nicolás E. Bordenabe, Catuscia Palamidessi , 2013
"... Abstract. Differential Privacy is one of the most prominent frameworks used to deal with disclosure prevention in statistical databases. It provides a formal privacy guarantee, ensuring that sensitive information relative to individuals cannot be easily inferred by disclosing answers to aggregate qu ..."
Abstract - Cited by 15 (6 self) - Add to MetaCart
Abstract. Differential Privacy is one of the most prominent frameworks used to deal with disclosure prevention in statistical databases. It provides a formal privacy guarantee, ensuring that sensitive information relative to individuals cannot be easily inferred by disclosing answers to aggregate queries. If two databases are adjacent, i.e. differ only for an individual, then the query should not allow to tell them apart by more than a certain factor. This induces a bound also on the distinguishability of two generic databases, which is determined by their distance on the Hamming graph of the adjacency relation. In this paper we explore the implications of differential privacy when the indistinguishability requirement depends on an arbitrary notion of distance. We show that we can naturally express, in this way, (protection against) privacy threats that cannot be represented with the standard notion, leading to new applications of the differential privacy framework. We give intuitive characterizations of these threats in terms of Bayesian adversaries, which generalize two interpretations of (standard) differential privacy from the literature. We revisit the well-known results stating that universally optimal mechanisms exist only for counting queries: We show that, in our extended setting, universally optimal mechanisms exist for other queries too, notably sum, average, and percentile queries. We explore various applications of the generalized definition, for statistical databases as well as for other areas, such that geolocation and smart metering. 1
(Show Context)

Citation Context

... Differential privacy [1, 2] is a formal definition of privacy which originated from the area of statistical databases, and it is now applied in many other domains, ranging from programming languages =-=[3]-=- to social networks [4] and geolocation [5]. Statistical databases are queried by analysts to obtain aggregate information about individuals. It is important to protect the privacy of the participants...

doi:10.1145/2240236.2240262 Continuity and Robustness of Programs

by Swarat Chaudhuri, Sumit Gulwani, Roberto Lublinerman
"... Computer scientists have long believed that software is different from physical systems in one fundamental way: while the latter have continuous dynamics, the former do not. In this paper, we argue that notions of continuity from mathematical analysis are relevant and interesting even for software. ..."
Abstract - Cited by 15 (4 self) - Add to MetaCart
Computer scientists have long believed that software is different from physical systems in one fundamental way: while the latter have continuous dynamics, the former do not. In this paper, we argue that notions of continuity from mathematical analysis are relevant and interesting even for software. First, we demonstrate that many everyday programs are continuous (i.e., arbitrarily small changes to their inputs only cause arbitrarily small changes to their outputs) or Lipschitz continuous (i.e., when their inputs change, their outputs change at most proportionally). Second, we give an mostly-automatic framework for verifying that a program is continuous or Lipschitz, showing that traditional, discrete approaches to proving programs correct can be extended to reason about these properties. An immediate application of our analysis is in reasoning about the robustness of programs that execute on uncertain inputs. In the longer run, it raises hopes for a toolkit for reasoning about programs that freely combines logical and analytical mathematics. 1.
Powered by: Apache Solr
  • About CiteSeerX
  • Submit and Index Documents
  • Privacy Policy
  • Help
  • Data
  • Source
  • Contact Us

Developed at and hosted by The College of Information Sciences and Technology

© 2007-2019 The Pennsylvania State University