M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. In Proceedings of the 35th Annual Symposium on Foundations of Computer Science, pages 382--393, 1994.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....negative answers (i.e. validating e ## S) can be handled with a standard method, as discussed in Section 8. 2 Previous Work Authenticated dictionaries are related to research in distributed computing (e.g. data replication in a network [7, 29] data structure design (e.g. program checking [8, 10, 11, 40] and memory checking [9, 19] and cryptography (e.g. incremental cryptography [4, 5, 19, 20] Previous additional work on authenticated dictionaries has been conducted primarily in the context of certificate revocation. The traditional method for certificate revocation (e.g. see [27] is for ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. In Proceedings of the 35th Annual Symposium on Foundations of Computer Science, pages 382--393, 1994.

....the rest of this paper, we denote with n the current number of elements of the set S stored in the authenticated dictionary. Authenticated dictionaries are related to research in distributed computing (e.g. data replication in a network [5, 23] data structure design (e.g. program checking [6, 8, 9, 32] and memory checking [7, 13] and cryptography (e.g. incremental cryptography [2, 3, 13, 14] 2.1. Certificate Revocation Previous work on authenticated dictionaries has been conducted primarily in the context of certificate revocation in public key infrastructure (PKI) The traditional ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. In Proceedings of the 35th Annual Symposium on Foundations of Computer Science, pages 382--393, 1994.

....in [4, 17, 5] are powerful tools for attacking the problem of program correctness. These methods offer both realistic and efficient tools for software verification. Various useful mathematical functions have been shown to have self testers and self correctors; some examples can be found in [5, 3, 17, 9, 14, 18, 1, 19, 21, 6]. The theoretical developments in this area are at the heart of the recent breakthrough results on probabilistically checkable proofs and the subsequent results that show non approximability of hard combinatorial problems. Suppose we are given a program P designed to compute a function f . ....

....where n is the degree of the polynomials being multiplied, and fl is a positive constant) Moreover, this scheme requires the tester to perform polynomial evaluations, whereas ours does not. For Fourier transforms, a different result checker that uses preprocessing has been given independently in [6]. A Useful Fact. The following fact, a variant of the well known Chernoff Hoeffding bounds, is often very useful in obtaining error bounds in sampling 0 1 random variables [15] Fact 1.6. Let Y 1 ; Y 2 ; be independently and identically distributed 0=1 random variables with means . Let ....

M. Blum and H. Wasserman, Program result-checking: A theory of testing meets a test of theory, in Proc. 35th Annual IEEE Symposium on Foundations of Computer Science, 1994, pp. 382--392.

....is interesting to note that this approach to software testing has been used in practice [Cod91, CS91] without any theoretical justification. Formalizing this idea [Blu88, BLR93] has led to efficient and remarkably simple methods of testing programs that are bothprovably goodand practically useful [BW94a, BW94b] Related concepts of result checking [BK89] and self correction [BLR93, Lip91] have also proved themselves to be very useful in ensuring software correctness. Let f be a function and P be a program that purports to compute f . Informally, a self tester for f is a probabilistic program T ....

M. Blum and H. Wasserman. Program resultchecking: a theory of testing meets a test of theory. Proc. 35th FOCS, 382--392, 1994.

....to the second replica of the software a value x # chosen with a large value of #, to decrease the risk of x and (x #) lying in the same failure region. The fault tolerant architecture then takes the form sketched below. Ingenious examples can be found in [Blum Kannan 1989, Blum et al. 1993, Blum Wasserman 1994] although the probabilistic estimate proposed for the reliability increase they achieve may be optimistic, as based on implicit assumptions of independence between the presence of different failure points. Unfortunately, these methods can only be applied when we can first prove that useful ....

M. Blum and H. Wasserman, "Program Result-Checking: A Theory of Testing Meets a Test of Theory", in 35th Annual Symposium on Foundations of Computer Science, (Santa Fe, New Mexico), pp. 382-92, 1994.

....the rest of this paper, we denote with the current number of elements of the set stored in the authenticated dictionary. Authenticated dictionaries are related to research in distributed computing (e.g. data replication in a network [5, 23] data structure design (e.g. program checking [6, 8, 9, 32] and memory checking [7, 13] and cryptography (e.g. incremental cryptography [2, 3, 13, 14] 2.1. Certificate Revocation Previous work on authenticated dictionaries has been conducted primarily in the context of certificate revocation in public key infrastructure (PKI) The traditional ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. In Proceedings of the 35th Annual Symposium on Foundations of Computer Science, pages 382--393, 1994.

....# S) The validation of negative answers can be handled with a standard method, as discussed in Section 6. 2. 1 Previous Work Authenticated dictionaries are related to research in distributed computing (e.g. data replication in a network [5, 21] data structure design (e.g. program checking [6, 8, 9, 29] and memory checking [7, 14] and cryptography (e.g. incremental cryptography [2, 3, 14, 15] The use of one way accumulators (which we review in the next subsection) originates with Benaloh and de Mare [4] They show how to utilize an exponential one way accumulator, which is also known as an ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. In Proceedings of the 35th Annual Symposium on Foundations of Computer Science, pages 382--393, 1994.

....programs [BLR93, Lip91] make runtime checks to certify that the program is giving the right answer. Though efficient, these methods often add small multiplicative factors to the runtime of the programs. Efforts to minimize the overhead due to program checking have been somewhat successful [BW94a, Rub94, BGR96] for linear functions. Can the overhead be minimized further by settling for a weaker, yet nontrivial, guarantee on the correctness of the program s output For example, it could be very useful to know that the program s output is reasonably correct (say, close in Hamming distance ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. Proc. 35th Foundations of Computer Science, pp. 382--392, 1994.

....programs [BLR93, Lip91] make runtime checks to certify that the program is giving the right answer. Though efficient, these methods often add small multiplicative factors to the runtime of the programs. Efforts to minimize the overhead due to program checking have been somewhat successful [BW94a, Rub94, BGR96] for linear functions. Can the overhead be minimized further by settling for a weaker, yet nontrivial, guarantee on the correctness of the program s output For example, it could be very useful to know that the program s output is reasonably correct (say, close in Hamming distance ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. Proc. 35th Foundations of Computer Science, pp. 382--392, 1994. 29

....during a specified exposure period in a specified use environment [16] Hamlet [17] and Littlewood [23] extend existing reliability theory be defining software dependability as a statistical measure of software quality. Hamlet incorporates Blum s idea of self checking self correcting programs [1] into reliability such that the dependability of a program P at input X is defined as the confidence probability that P is correct (with respect to its speci fication) at X. Software reliability models not only demonstrate that uncertainty may be measured and represented explicitly but they can ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. In Proceedings of the 35th Annual Symposium on Foundations of Computer Science, Santa Fe, NM, 1994. IEEE Computer Science Press.

....during a specified exposure period in a specified use environment [Goe85] Hamlet [Ham96] and Littlewood [LS93] extend existing reliability theory by defining software dependability as a statistical measure of software quality. Specifically, Hamlet uses Blum s notion of self checking programs [BW94] to define the dependability of program P at input X as the confidence probability that P is correct (with respect to its specification) at X . Software reliability models demonstrate that software uncertainties may be measured and therefore captured explicitly. Furthermore, reliability modeling ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. In Proceedings of the 35th Annual Symposium on Foundations of Computer Science, pages 382--391, Santa Fe, NM, 1994. IEEE Computer Science Press.

....or may not be acceptable for debugging sessions, but must be turned off in the normal working process for reasons of efficiency. In summary, the robustness of a component must also be flexibly adaptable to allow a context specific compromise between robustness, efficiency, and adaptability. See [BW94] for an introduction to partial and complete output checking for implementations of algorithms. 3.2 Concrete Application As mentioned above, we next discuss a concrete scenario: a traffic information system. Here we are given a traffic network, that is, the nodes are cities and towns, and the ....

Manuel Blum and Hal Wasserman. Program result--checking: a theory of testing meets a test of theory. Proc. 35th Annual Symposium on Foundations of Computer Science (FOCS '94 ), 382--392, 1994.

.... during system testing to uncover temporal faults in test executions [15] Proactive debuggers and run time monitors can use temporal oracles to detect faults, and notify the user when they occur [8,10] If efficient, temporal oracles can also be incorporated into robust self checking programs [1]. Methods for checking that traces conform to temporal specifications are usually designed for specific notations [2, 7, 12] Details of the methods are motivated by the specific semantics of the operators provided for expressing specifications. This makes it difficult to adapt the method for use ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. In Proc. IEEE Symp. Foundations of Computer Science, 1994.

.... needed during system testing to uncover temporal faults in test executions [16] Proactive debuggers and run time monitors can use temporal oracles to detect faults, notifying the user when they occur [8] If efficient, temporal oracles can also be incorporated into robust, self checking programs [1]. Several methods for checking that traces conform to temporal specifications have been designed for specific notations [2, 4, 12, 13] The precise semantics of the operators that a notation provides motivate details of the methods. This practice makes it difficult to adapt a trace checker for use ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. In Proc. IEEE Symp. Foundations of Computer Science, 1994.

....programs [BLR93, Lip91] make runtime checks to certify that the program is giving the right answer. Though efficient, these methods often add small multiplicative factors to the runtime of the programs. Efforts to minimize the overhead due to program checking have been somewhat successful [BW94, Rub94, BGR96] for linear functions. Can this overhead be minimized further by settling for a weaker, yet nontrivial, guarantee on the correctness of the program s output For example, it could be very useful to know that the program s output is reasonably correct (say, close in Hamming distance ....

M. Blum and H. Wasserman. Program resultchecking: A theory of testing meets a test of theory. Proc. 35th Foundations of Computer Science, pp. 382-- 392, 1994.

.... Program checking, which uses the program to verify its own correctness, was first formalized and developed in [BK89] Related notions of self testing and self correcting were further explored in [BLR93, Lip91] These notions have proved to be very powerful from a practical point of view (e.g. BW94] and from a theoretical angle (e.g. AS92, ALM 92] as well. The techniques used usually consist of tests which compare the output of the program either to a predetermined value or to a function of outputs of the same program at different inputs. In order to apply these powerful techniques ....

....are constructed. The domain is assumed to be closed in all of these results. In [ABC 93] approximate checkers for sin, cos, matrix multiplication, matrix inversion, linear system solving, and determinant are given. The domain is assumed to be closed in the results on sin and cos. In [BW94] an approximate checker for floating point division is given. In [Sud91] a clever technique using approximation theory is used to test univariate polynomials of degree 9. It is left open in [GLR 91, ABC 93, RS96, Rub94] whether the properties used to test polynomial, hyperbolic, and ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. Proc. 35th FOCS, pp. 382--392, 1994.

.... needed during system testing to uncover temporal faults in test executions [22] Proactive debuggers and run time monitors can use temporal oracles to detect faults, notifying the user when such occur [10] If efficient, temporal oracles can also be incorporated into robust, self checking programs [1]. Several methods for checking that traces conform to temporal specifications have been designed for specific notations [2,4,16,18] The semantics of the operators provided in specific notations motivate details of the methods. This practice makes it difficult to adapt a trace checker for use with ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. In Proc. IEEE Symp. Foundations of Computer Science, 1994.

....it would not seem that the dependability theory we are seeking should bear on the oracle problem, it does, because confidence in a computed result may make an oracle unnecessary. This point is further discussed in Section 4.4. 2. 2 Blum s Idea of Self checking Programs Manuel Blum has proposed [BK89, BW94] an idea that is almost exactly what we want to mean by dependability. He argues that software users are interested in a particular execution of a particular program only they want assurance that a single result can be trusted. Blum has found a way to sometimes exploit the low failure ....

....that P is correct (as defined by its specification) at X . For this definition to be usefully different from a uniform definition in which the dependability is defined for all inputs, a dependability prediction must be supported by runtime calculations, in the manner of Blum s self checking idea [BW94]. Self checking has been applied to only a few problems, problems whose input data has a decomposition theory, so that random variations on a computation can be performed and their results compared. 3.1 Pointwise Sensitivity Voas s PIE theory of sensitivity (Section 2.4.1) becomes a PI ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. In 35th Annual Symposium on Foundations of Computer Science, pages 382--391, Santa Fe, NM, 1994.

....of program correctness. These methods offer both realistic and efficient tools for software verification. Various useful mathematical functions have been shown to have self testers and self correctors; some examples can be found in [BLR93, BF90, Lip91, CL90, GLR 91, RS92, ABCG93, RS96, Rub94, BW94] The theoretical developments in this area are at the heart of the recent breakthrough results on probabilistically checkable proofs and the subsequent results that show non approximability of hard combinatorial problems. Suppose we are given a program P designed to compute a function f . ....

....where n is the degree of the polynomials being multiplied, and fl is a positive constant) Moreover, this scheme requires the tester to perform polynomial evaluations, whereas ours does not. For Fourier transforms, a different result checker that uses preprocessing has been given independently in [BW94] A Useful Fact. The following fact, a variant of the well known Chernoff Hoeffding bounds, is often very useful in obtaining error bounds in sampling 0 1 random variables [KLM89] Fact 2 Let Y 1 ; Y 2 ; be independently and identically distributed 0 1 random variables with means . Let ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. In Proc. 35th Annual IEEE Symposium on Foundations of Computer Science, pages 382--392, 1994.

....or may not be acceptable for debugging sessions, but must be turned off in the normal working process for reasons of efficiency. In summary, the robustness of a component must also be flexibly adaptable to allow a context specific compromise between robustness, efficiency, and adaptability. See [BW94] for an introduction to partial and complete output checking for implementations of algorithms. 3.2 Concrete Application As mentioned above, we next discuss a concrete scenario: a traffic information system. Here we are given a traffic network, that is, the nodes are cities and towns, and the ....

Manuel Blum and Hal Wasserman. Program result-checking: a theory of testing meets a test of theory. Proc. 35th Annual Symposium on Foundations of Computer Science (FOCS '94 ) pages 382--392, 1994.

....property of interest) In this paper we do not make a distinction between self testing and property testing. We simply refer to both tasks as testing. There is a considerable body of literature dealing with self testing issues. We refer the interested reader to the survey of Blum and Wasserman [BW94] and the thesis of Rubinfeld [Rub90] For pointers to more recent work in the area see [Rub94, EKS96] The work of Goldreich, Goldwasser, and Ron [GGR96] which introduces the concept of property testing also suggests exciting new directions of research. For recent developments concerning issues ....

....its (finite) input space. Self correction is a way of using this same program to find the correct value everywhere (with high probability) Research addressing the self testing problem usually also addresses self correction issues. Thus, the reader interested in this topic is referred to [Rub90, BW94] Low degree Testing: In the low degree testing problem we are given an oracle for a function f : F n F , where F is say a finite field. We are also given a positive integer d. Below we briefly describe some instances of this problem that have been studied in the literature. In the low ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. In Proceedings of the 35th Annual Symposium on Foundations of Computer Science, pages 382-- 392, Santa Fe, New Mexico, November 1994. IEEE.

.... static dataflow analysis to determine which actions the program may or may not perform [8] structural specification based test selection and measurement [3] combining static and dynamic analysis [14] and incorporation of the oracle into a debugging system [10] or a robust self checking program [2]. EXAMPLE To illustrate our algorithm, we choose the GIL axiom DoesPump, as shown in figure 19, from the gas station problem presented in [7] DoesPump requires that if a user pays, she will eventually pump gas and finally get her change back. pay1 pay1 pay1 pump1 Figure 19: GIL axiom does pump ....

M. Blum and H. Wasserman. Program resultchecking: A theory of testing meets a test of theory. In Proc. Symposium of Foundations of Computer Science, 1994.

.... multiplication, integer division, sine cosine, integer multiplication, the mod function, modular multiplication, polynomial multiplication, modular exponentiation, Fast Fourier Transform and determinant, have been shown to have self testers and self correctors [19] 8] 34] 23] 31] 37] 38] 2] 28] [21]. Although many functions can be viewed as linear functions or low degree polynomials over an appropriate group structure, one concern was that these might be the only examples of functions that have self testers and self correctors. Using the new robustness results, we show that self testers and ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. In Proc. 35th FOCS, pp. 382--392, 1994.

....in order to allow one to use a program safely, without having to know apriori that the program is correct on all inputs. Related notions of self testing and self correcting were further explored in [BLR93, Lip91] These notions have proved to be very powerful from a practical point of view (c.f. BW94] and from a theoretical angle (c.f. AS92, ALM 92] as well. The techniques used usually consist of tests performed at run time which compare the output of the program either to a predetermined value or to a function of outputs of the same program at different inputs. In order to apply ....

....For instance, a finite precision rational domain is not closed under addition. In [ABC 93] approximate checkers for sine, cosine, matrix multiplication, matrix inversion, linear system solving, and determinant are given. The domain is assumed to be closed in the results on sine and cosine. In [BW94] an approximate checker for floating point division is given. In [Sud91] a clever technique using approximation theory is used to test univariate polynomials of degree 9. It is left open in [GLR 91, ABC 93, RS96, Rub94] whether the properties used to test polynomial, hyperbolic, and ....

M. Blum and H. Wasserman. Program result-checking: A theory of testing meets a test of theory. Proc. 35th FOCS, pp. 382--392, 1994.

....In collaboration with Hughes Aircraft, we are embedding simple checkers in real world software. These checkers can then serve as extremely reliable testing oracles and may have other uses as well. A central problem in this research has been that of checking the Fourier Transform. A sequel to [3], the current talk will give an informal narrative of our recent work on this problem. The talk may include: 1. A review of the real number Fourier Transform checker we introduced in [3] 2. Comparison with the real number matrix multiplication checker of [1] 3. Our struggle with a seemingly ....

....uses as well. A central problem in this research has been that of checking the Fourier Transform. A sequel to [3] the current talk will give an informal narrative of our recent work on this problem. The talk may include: 1. A review of the real number Fourier Transform checker we introduced in [3]. 2. Comparison with the real number matrix multiplication checker of [1] 3. Our struggle with a seemingly trivial probability question which caused difficulties in both [3] and [1] Including: a solution proposed by Joel Spencer; an alternative approach developed in our revised version of [3] ....

[Article contains additional citation context not shown here]

M. Blum and H. Wasserman, "Program Result-Checking: a Theory of Testing Meets a Test of Theory," in Proc. of the 35th IEEE Symposium on Foundations of Computer Science, 382--392, 1994. Revised version: "Software Reliability via Runtime Result-Checking," http://http.cs.berkeley.edu/ halw/.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC