Citation Context

.... GenProg, TrpAutoRepair, and AE, and most prior techniques, evaluate patches using test cases. GenProg uses the test results to define an objective function that informs an evolutionary search [22], =-=[41]-=- over the space of candidate patches. TrpAutoRepair samples candidate patches randomly, and gains efficiency by prioritizing the test case execution order, and only runs as many test cases as necessar...

Citation Context

...ideally without the defect in question. To find code that matches a specified profile, SearchRepair encodes each input-output example in the profile as SMT constraints and then uses the Z3 SMT solver =-=[17]-=- to find fragments in the database that satisfy them. The first step in searching with an input-output profile is thus to transform input-output information into SMT constraints. Similar to the encodi...

Citation Context

...ructor-written test suite is constructed via careful consideration of the requirements and the input space. IntroClass also includes, for each program, a test suite automatically generated using KLEE =-=[9]-=- to achieve full branch coverage on an instructor-written oracle program. These additional test suites support independent high-coverage validation of program and patch correctness. Our evaluation att...

Citation Context

... queries for concept localization [30]. The search approach we use for program repair depends on a more structured specification. Prior semantic code search has used formal specifications [27], [53], =-=[77]-=- and test cases [55], [58]. Formal specifications allow precise and sound matching but must be written by hand, which is difficult and error-prone. Test cases are more lightweight but, prior to our wo...

Citation Context

...is largely unable to keep up with the volume of defects in extant software [2]. Despite their established detrimental impact on a company’s bottom line, known defects ship in mature software projects =-=[45]-=-, and many defects, including those that are security-critical, remain unaddressed for long periods of time [32]. At the same time, the expansion of the open-source movement has led to many large, pub...

Citation Context

...ffline and produces a database relating code fragments to be searched to sets of SMT constraints over the input-output behavior and the segment’s type signature. For each fragment, symbolic execution =-=[13]-=-, [14], [40] enumerates the feasible paths through it and our approach converts code constructs and predicates controlling each path’s execution into SMT constraints describing variable types and valu...

Citation Context

...o a set of candidate program locations to be potentially changed. Previous techniques, including the three to which we compare in this work, typically use or adapt prior fault localization work [36], =-=[59]-=- to identify promising locations. Second, such a technique must modify the code in an attempt to repair it. This describes the fix space of a particular program repair problem. GenProg, TrpAutoRepair,...

Citation Context

...to guide sound patch synthesis [19], [35], [49], [50], [70], [71]. The other primary class of repair approaches is generate-and-validate repair, which uses searchbased software engineering techniques =-=[31]-=-, [57], [69] or predefined repair templates [15], [39], [54] to generate many candidate patches for a bug, and then validate them using indicative workloads or test suites. In this work, we compare Se...

Citation Context

... is not found, SearchRepair can return a partial repair, as Section IV-B reports. IV. EVALUATION This section describes SearchRepair’s evaluation, including a comparison to three prior tools, GenProg =-=[74]-=-, TrpAutoRepair [56], and AE [73]. Section IV-A describes our experimental set up. Sections IV-B and IV-C evaluate SearchRepair’s effectiveness at producing repairs and the quality of those repairs, r...

Citation Context

...versed. GenProg, TrpAutoRepair, and AE, and most prior techniques, evaluate patches using test cases. GenProg uses the test results to define an objective function that informs an evolutionary search =-=[22]-=-, [41] over the space of candidate patches. TrpAutoRepair samples candidate patches randomly, and gains efficiency by prioritizing the test case execution order, and only runs as many test cases as ne...

Citation Context

...of correct program behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], [18], [19], [21], [28], [33], [35], [39], [42], [48], [49], [50], [51], [52], =-=[54]-=-, [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide and evaluate patching efforts. Modern test-suite guided repair techniques, particularly those following a gene...

Citation Context

...Buggy software costs the global economy billions of dollars annually [8], [60]. One major reason software defects are so expensive is that software companies must dedicate considerable developer time =-=[75]-=- to manually finding and fixing bugs in their software. Unfortunately, manual bug repair, the industry standard, is largely unable to keep up with the volume of defects in extant software [2]. Despite...

Citation Context

...ch, whether human- or toolgenerated, is that it generalizes to the full, desired, often unwritten specification of correct program behavior. This is a challenge for automatic repair techniques (e.g., =-=[3]-=-, [7], [10], [11], [15], [16], [18], [19], [21], [28], [33], [35], [39], [42], [48], [49], [50], [51], [52], [54], [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to gui...

Citation Context

...ental impact on a company’s bottom line, known defects ship in mature software projects [45], and many defects, including those that are security-critical, remain unaddressed for long periods of time =-=[32]-=-. At the same time, the expansion of the open-source movement has led to many large, publicly accessible source code databases, such as GitHub, BitBucket, and SourceForge. Because many programs includ...

Citation Context

... behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], [18], [19], [21], [28], [33], [35], [39], [42], [48], [49], [50], [51], [52], [54], [56], [57], =-=[61]-=-, [69], [70], [73], [74], [76]), many of which use test suites to guide and evaluate patching efforts. Modern test-suite guided repair techniques, particularly those following a generate-and-validate ...

Citation Context

...ndant [24]. Semantically, many methods can be reconstructed by composing other methods in the same project [5], [11], [12]. These findings are consistent with researchers’ observations of code clones =-=[37]-=-. Intuitively, software projects repeatedly reuse the same common building block data structures and methods as other projects, and often reimplement this functionality. This suggests that if a projec...

Citation Context

...any candidate patches for a bug, and then validate them using indicative workloads or test suites. In this work, we compare SearchRepair to three prior generate-and-validate techniques: GenProg [42], =-=[44]-=-, [72], [74], TrpAutoRepair [56], and AE [73]. (TrpAutoRepair was also published under the name RSRepair in “The strength of random search on automated program repair” by Yuhua Qi, Xiaoguang Mao, Yan ...

Citation Context

...ation of correct program behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], [18], [19], [21], [28], [33], [35], [39], [42], [48], [49], [50], [51], =-=[52]-=-, [54], [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide and evaluate patching efforts. Modern test-suite guided repair techniques, particularly those following ...

Citation Context

...ired, often unwritten specification of correct program behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], [18], [19], [21], [28], [33], [35], [39], =-=[42]-=-, [48], [49], [50], [51], [52], [54], [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide and evaluate patching efforts. Modern test-suite guided repair techniques,...

Citation Context

...ization [30]. The search approach we use for program repair depends on a more structured specification. Prior semantic code search has used formal specifications [27], [53], [77] and test cases [55], =-=[58]-=-. Formal specifications allow precise and sound matching but must be written by hand, which is difficult and error-prone. Test cases are more lightweight but, prior to our work, required the code to b...

Citation Context

...that it generalizes to the full, desired, often unwritten specification of correct program behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], [18], =-=[19]-=-, [21], [28], [33], [35], [39], [42], [48], [49], [50], [51], [52], [54], [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide and evaluate patching efforts. Modern ...

Citation Context

...er time [75] to manually finding and fixing bugs in their software. Unfortunately, manual bug repair, the industry standard, is largely unable to keep up with the volume of defects in extant software =-=[2]-=-. Despite their established detrimental impact on a company’s bottom line, known defects ship in mature software projects [45], and many defects, including those that are security-critical, remain una...

Citation Context

...he full, desired, often unwritten specification of correct program behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], [18], [19], [21], [28], [33], =-=[35]-=-, [39], [42], [48], [49], [50], [51], [52], [54], [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide and evaluate patching efforts. Modern test-suite guided repair...

Citation Context

...l, desired, often unwritten specification of correct program behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], [18], [19], [21], [28], [33], [35], =-=[39]-=-, [42], [48], [49], [50], [51], [52], [54], [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide and evaluate patching efforts. Modern test-suite guided repair techn...

Citation Context

...ten specification of correct program behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], [18], [19], [21], [28], [33], [35], [39], [42], [48], [49], =-=[50]-=-, [51], [52], [54], [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide and evaluate patching efforts. Modern test-suite guided repair techniques, particularly thos...

Citation Context

...atabases, such as GitHub, BitBucket, and SourceForge. Because many programs include routines, data structures, and designs that have been previously implemented in other software projects [11], [12], =-=[24]-=-, we posit that, if a method or component of a software system contains a defect, with high probability, there exists a similar but correct version of that component in some publicly accessible softwa...

Citation Context

... and produces a database relating code fragments to be searched to sets of SMT constraints over the input-output behavior and the segment’s type signature. For each fragment, symbolic execution [13], =-=[14]-=-, [40] enumerates the feasible paths through it and our approach converts code constructs and predicates controlling each path’s execution into SMT constraints describing variable types and values. Th...

Citation Context

...ource code databases, such as GitHub, BitBucket, and SourceForge. Because many programs include routines, data structures, and designs that have been previously implemented in other software projects =-=[11]-=-, [12], [24], we posit that, if a method or component of a software system contains a defect, with high probability, there exists a similar but correct version of that component in some publicly acces...

Citation Context

...d, is that it generalizes to the full, desired, often unwritten specification of correct program behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], =-=[18]-=-, [19], [21], [28], [33], [35], [39], [42], [48], [49], [50], [51], [52], [54], [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide and evaluate patching efforts. M...

Citation Context

...hes. Code Synthesis. Code can be synthesized using input-output examples, written in a domain-specific language [29], using predefined components [34], or based on a high-level behavioral description =-=[4]-=-. Recent approaches use context from a debugger to show where in a program synthesis should occur [26]. While effective in that domain, synthesis-based approaches are limited by the solver’s ability t...

Citation Context

...r human- or toolgenerated, is that it generalizes to the full, desired, often unwritten specification of correct program behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], =-=[10]-=-, [11], [15], [16], [18], [19], [21], [28], [33], [35], [39], [42], [48], [49], [50], [51], [52], [54], [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide and eval...

Citation Context

...s to the full, desired, often unwritten specification of correct program behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], [18], [19], [21], [28], =-=[33]-=-, [35], [39], [42], [48], [49], [50], [51], [52], [54], [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide and evaluate patching efforts. Modern test-suite guided ...

Citation Context

...lating queries for concept localization [30]. The search approach we use for program repair depends on a more structured specification. Prior semantic code search has used formal specifications [27], =-=[53]-=-, [77] and test cases [55], [58]. Formal specifications allow precise and sound matching but must be written by hand, which is difficult and error-prone. Test cases are more lightweight but, prior to ...

Citation Context

...tests and tend to generate short patches. It may therefore be difficult for them to repair complex defects. Importantly, such techniques may produce lower quality patches than those written by humans =-=[23]-=-, [39], and recent studies have identified significant concerns with the functional correctness of automatically-repaired programs [20], [57], [65]. One way to measure the quality of a repair is to ob...

Citation Context

...ralizes to the full, desired, often unwritten specification of correct program behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], [18], [19], [21], =-=[28]-=-, [33], [35], [39], [42], [48], [49], [50], [51], [52], [54], [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide and evaluate patching efforts. Modern test-suite g...

Citation Context

...uces higher-quality repairs than GenProg, TrpAutoRepair, and AE, and repairs some defects those tools cannot. I. INTRODUCTION Buggy software costs the global economy billions of dollars annually [8], =-=[60]-=-. One major reason software defects are so expensive is that software companies must dedicate considerable developer time [75] to manually finding and fixing bugs in their software. Unfortunately, man...

Citation Context

... a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], [18], [19], [21], [28], [33], [35], [39], [42], [48], [49], [50], [51], [52], [54], [56], [57], [61], [69], [70], =-=[73]-=-, [74], [76]), many of which use test suites to guide and evaluate patching efforts. Modern test-suite guided repair techniques, particularly those following a generate-and-validate paradigm (i.e., he...

Citation Context

...often unwritten specification of correct program behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], [18], [19], [21], [28], [33], [35], [39], [42], =-=[48]-=-, [49], [50], [51], [52], [54], [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide and evaluate patching efforts. Modern test-suite guided repair techniques, parti...

Citation Context

...it. This describes the fix space of a particular program repair problem. GenProg, TrpAutoRepair, and AE address this challenge using the observation that programs are often internally repetitive [5], =-=[25]-=- and limit changes to deleting constructs and copying constructs from elsewhere in the same program. TrpAutoRepair and AE restrict attention to single-edit patches. Third, such a technique validates c...

Citation Context

...toolgenerated, is that it generalizes to the full, desired, often unwritten specification of correct program behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], =-=[15]-=-, [16], [18], [19], [21], [28], [33], [35], [39], [42], [48], [49], [50], [51], [52], [54], [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide and evaluate patchin...

Citation Context

...ir identifies larger sections of human-written candidate fix code from other projects, which it uses to replace defective regions wholesale. Because code is repetitive and often reimplements routines =-=[5]-=-, [11], [12], [24], it may be possible to find correct alternative versions of a given buggy piece of functionality in other software systems, given a sufficiently large database. Our core assumption ...

Citation Context

...c language [29], using predefined components [34], or based on a high-level behavioral description [4]. Recent approaches use context from a debugger to show where in a program synthesis should occur =-=[26]-=-. While effective in that domain, synthesis-based approaches are limited by the solver’s ability to enumerate and test all possible combinations of program constructs. In our use of solvers for semant...

Citation Context

...ndidate patches for a bug, and then validate them using indicative workloads or test suites. In this work, we compare SearchRepair to three prior generate-and-validate techniques: GenProg [42], [44], =-=[72]-=-, [74], TrpAutoRepair [56], and AE [73]. (TrpAutoRepair was also published under the name RSRepair in “The strength of random search on automated program repair” by Yuhua Qi, Xiaoguang Mao, Yan Lei, Z...

Citation Context

...n testing large numbers of candidate repairs), although typically general and scalable, often produce poor-quality patches that overfit to the specification test suites used to guide patch generation =-=[20]-=-, [57], [65]. By definition, test suites only encode a partial specification of correct behavior. A patch that is correct according to a given test suite may therefore not be fully correct when evalua...

Citation Context

...rect program behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], [18], [19], [21], [28], [33], [35], [39], [42], [48], [49], [50], [51], [52], [54], =-=[56]-=-, [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide and evaluate patching efforts. Modern test-suite guided repair techniques, particularly those following a generate-a...

Citation Context

... produces higher-quality repairs than GenProg, TrpAutoRepair, and AE, and repairs some defects those tools cannot. I. INTRODUCTION Buggy software costs the global economy billions of dollars annually =-=[8]-=-, [60]. One major reason software defects are so expensive is that software companies must dedicate considerable developer time [75] to manually finding and fixing bugs in their software. Unfortunatel...

Citation Context

...eformulating queries for concept localization [30]. The search approach we use for program repair depends on a more structured specification. Prior semantic code search has used formal specifications =-=[27]-=-, [53], [77] and test cases [55], [58]. Formal specifications allow precise and sound matching but must be written by hand, which is difficult and error-prone. Test cases are more lightweight but, pri...

Citation Context

...0] predicated on program synthesis and generate-and-validate techniques that heuristically create and then test large numbers of candidate repairs [1], [11], [12], [15], [18], [39], [47], [54], [57], =-=[63]-=-, [69], [73], [74]. Second, although several previous techniques also reuse human-written code from elsewhere in a program, or instantiate human-written templates to effect local changes, SearchRepair...

Citation Context

...mponent in some publicly accessible software project. The research challenge lies in how to automatically find and use such implementations to repair bugs. Our key idea is to use semantic code search =-=[68]-=- over existing open-source code to find correct implementations of buggy components and methods, and use the results to automatically generate patches for software defects. Semantic search identifies ...

Citation Context

... for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], [18], [19], [21], [28], [33], [35], [39], [42], [48], [49], [50], [51], [52], [54], [56], [57], [61], [69], [70], [73], [74], =-=[76]-=-), many of which use test suites to guide and evaluate patching efforts. Modern test-suite guided repair techniques, particularly those following a generate-and-validate paradigm (i.e., heuristically ...

Citation Context

...by-construction techniques [35], [49], [50], [52], [70] predicated on program synthesis and generate-and-validate techniques that heuristically create and then test large numbers of candidate repairs =-=[1]-=-, [11], [12], [15], [18], [39], [47], [54], [57], [63], [69], [73], [74]. Second, although several previous techniques also reuse human-written code from elsewhere in a program, or instantiate human-w...

Citation Context

...ode to be executed and could not identify partial matches. Code Synthesis. Code can be synthesized using input-output examples, written in a domain-specific language [29], using predefined components =-=[34]-=-, or based on a high-level behavioral description [4]. Recent approaches use context from a debugger to show where in a program synthesis should occur [26]. While effective in that domain, synthesis-b...

Citation Context

...ons of current repair techniques, focusing on output quality, in Section II-C. A. Semantic code search In this paper, we extend our prior work on input-output example-based semantic code search [66], =-=[67]-=-, [68]. We detail our extensions and illustrate with examples in Section III and focus in this subsection on a high-level overview to ground the subsequent material. Code search uses a specification t...

Citation Context

...rge numbers of candidate repairs), although typically general and scalable, often produce poor-quality patches that overfit to the specification test suites used to guide patch generation [20], [57], =-=[65]-=-. By definition, test suites only encode a partial specification of correct behavior. A patch that is correct according to a given test suite may therefore not be fully correct when evaluated with res...

Citation Context

...hether human- or toolgenerated, is that it generalizes to the full, desired, often unwritten specification of correct program behavior. This is a challenge for automatic repair techniques (e.g., [3], =-=[7]-=-, [10], [11], [15], [16], [18], [19], [21], [28], [33], [35], [39], [42], [48], [49], [50], [51], [52], [54], [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide an...

Citation Context

...functionality that may be used to produce a repair. Semantic Code Search. Recently keyword-based searches have begun to incorporate semantic information for finding working code examples from the Web =-=[38]-=- or reformulating queries for concept localization [30]. The search approach we use for program repair depends on a more structured specification. Prior semantic code search has used formal specificat...

Citation Context

...iques. Some approaches target a particular class of bugs, such as buffer overruns [64], unsafe integer use in C programs [15], single-variable atomicity violations [35], deadlock and livelock defects =-=[46]-=-, concurrency errors [47], and data input errors [1]. Other techniques tackle generic bugs. For example, the ARMOR tool replaces buggy library calls with different calls that achieve the same behavior...

Citation Context

...ograms. This relatively simple procedure is sufficient for the IntroClass dataset, but more complex programs will likely require deeper code transfer techniques. Recent advances in code transfer [6], =-=[62]-=-, [63] are complementary to our work and we expect future versions of SearchRepair to use them. Program repair. Automated program repair is concerned with automatically bringing an implementation more...

Citation Context

...9], [50], [52], [70] predicated on program synthesis and generate-and-validate techniques that heuristically create and then test large numbers of candidate repairs [1], [11], [12], [15], [18], [39], =-=[47]-=-, [54], [57], [63], [69], [73], [74]. Second, although several previous techniques also reuse human-written code from elsewhere in a program, or instantiate human-written templates to effect local cha...

Citation Context

...unwritten specification of correct program behavior. This is a challenge for automatic repair techniques (e.g., [3], [7], [10], [11], [15], [16], [18], [19], [21], [28], [33], [35], [39], [42], [48], =-=[49]-=-, [50], [51], [52], [54], [56], [57], [61], [69], [70], [73], [74], [76]), many of which use test suites to guide and evaluate patching efforts. Modern test-suite guided repair techniques, particularl...

Citation Context

...gy programs. This relatively simple procedure is sufficient for the IntroClass dataset, but more complex programs will likely require deeper code transfer techniques. Recent advances in code transfer =-=[6]-=-, [62], [63] are complementary to our work and we expect future versions of SearchRepair to use them. Program repair. Automated program repair is concerned with automatically bringing an implementatio...

Citation Context

... prior to our work, required the code to be executed and could not identify partial matches. Code Synthesis. Code can be synthesized using input-output examples, written in a domain-specific language =-=[29]-=-, using predefined components [34], or based on a high-level behavioral description [4]. Recent approaches use context from a debugger to show where in a program synthesis should occur [26]. While eff...

Citation Context

...antic Code Search. Recently keyword-based searches have begun to incorporate semantic information for finding working code examples from the Web [38] or reformulating queries for concept localization =-=[30]-=-. The search approach we use for program repair depends on a more structured specification. Prior semantic code search has used formal specifications [27], [53], [77] and test cases [55], [58]. Formal...

Citation Context

...er and Hafiz [15], Debroy and Wong [18], Demsky and Rinard [19], DirectFix [49], FINCH [51], GenProg [42], [74], Gopinath et al. [28], Jolt [10], Juzi [21], Kali [57], PACHIKA [16], PAR [39], relifix =-=[69]-=-, SemFix [50], Sidiroglou and Keromytis [61], TrpAutoRepair [56], etc.). Section II-B has discussed the differences between generate-and-validate and correct-by-construction repair. The approach behin...