M. Sullivan and R. Chillarege, "A Comparison of Software Defects in Database Management Systems and Operating Systems," Proc. 1992.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....4.0 BIOS. 2.1 Description of Faults We first describe the types of faults we inject into the operating system. Our primary goal in designing these faults is to generate a wide variety of operating system crashes. Our models are derived from studies of commer cial operating systems and databases [66], 65] 43] and from prior models used in fault injection studies [10] 39] 38] 15] The faults we inject range from low level hardware faults, such as flipping bits in memory, to highlevel software faults, such as memory allocation errors. Table 1 shows examples of how how real world ....

M. Sullivan and R. Chillarege, "A Comparison of Software Defects in Database Management Systems and Operating Systems," Proc. 1992.

....category therefore is not comparable to our operator failure category, despite its being named somewhat similarly. A number of researchers have examined the causes of failures in enterprise class server environments. Sullivan and Chillarege examined software defects in MVS, DB2, and IMS [19]. Tang and Iyer conducted a similar study for the VAX VMS operating system in two VAXclusters [20] Lee and Iyer categorized software faults in the Tandem GUARDIAN operating system [12] Murphy and Gent examined causes of system crashes in VAX systems between 1985 and 1993; in 1993 they found that ....

M. S. Sullivan and R. Chillarege. A comparison of software defects in database management systems and operating systems. Proceedings of the 22nd International Symposium on Fault-Tolerant Computing, 1992.

....devices. This contributes to system reliability by detecting the effects of incorrect installation, spontaneous de programming, radiation damage, etc. expand References, issues. Gertman94] Human Reliability safety Analysis Data Handbook There are many types of software faults, e.g. [Littlewood92, Sullivan92]: computation of addresses that cause (attempted) access violations of memory or other hardware resources . hogging (no timely release) of allocated hardware resources, to the exclusion of access by other software tasks or users . incorrect pointer management or statement logic, causing ....

Sullivan, M.S., Chillarege, R.: "A Comparison of Software Defects in Database Management Systems and Operating Systems", Proc. 22 nd Int'l Symp. On Fault Tolerant Computing (FTCS), Boston/MA, July 1992, pp.475-484

....This study showed that transients follow a Weibull distribution rather than occur at a constant rate as frequently assumed. Gray [3] presented results from a census of Tandem systems. Chillarege [2] presented a study of the impact of failures on customers and the fault lifetimes. Sullivan [18] [19] examined software defects occurring in operating systems and databases (based on field data) An in depth overview of experimental and analytical techniques for analysis of computer systems dependability can be found in [8] 3 Field Data and Analysis Parameters This section presents data ....

M.S. Sullivan and R. Chillarege, "A Comparison of Software Defects in Database Management Systems and Operating Systems," Proc. 22nd Int. Symp. Fault-Tolerant Computing, pp. 475-484, July 1992.

....year period. Their main focus was on measuring errors caused by memory corruption versus everything else. They found that the former generated the highest number of reported system crashes. They also measured how often and why bug xes introduced other bugs. They did similar study for databases [24]. More recently, Xu et al. 26] used reboot logs to measure the dependability of a 503 node Windows NT cluster over 4 months (and 2,127 reboots) They classi ed the causes of failures (hardware, software) time to recover, and availability measurements. An interesting result is that reboots ....

M. Sullivan and R. Chillarege. A Comparison of Software Defects in Database Management Systems and Operating Systems. In 22nd International Symposium on FaultTolerant Computing, July 1992.

....year period. Their main focus was on measuring errors caused by memory corruption versus everything else. They found that the former generated the highest number of reported system crashes. They also measured how often and why bug fixes introduced other bugs. They did similar study for databases [24]. More recently, Xu et al. 26] used reboot logs to measure the dependability of a 503 node Windows NT cluster over 4 months (and 2,127 reboots) They classified the causes of failures (hardware, software) time to recover, and availability measurements. An interesting result is that reboots ....

M. Sullivan and R. Chillarege. A Comparison of Software Defects in Database Management Systems and Operating Systems. In 22nd International Symposium on FaultTolerant Computing, July 1992.

....year period. Their main focus was on measuring errors caused by memory corruption versus everything else. They found that the former generated the highest number of reported system crashes. They also measured how often and why bug fixes introduced other bugs. They did similar study for databases [24]. More recently, Xu et al. 26] used reboot logs to measure the dependability of a 503 node Windows NT cluster over 4 months (and 2,127 reboots) They classified the causes of failures (hardware, software) time to recover, and availability measurements. An interesting result is that reboots ....

M. Sullivan and R. Chillarege. A Comparison of Software Defects in Database Management Systems and Operating Systems. In 22nd International Symposium on FaultTolerant Computing, July 1992.

....(see Chapter 2) However, the failures exhibited by those software faults can be transient, i.e. the failure may not recur if the software is reexecuted on the same input [Gra91, Wan93] this is a frequently used technique in hardware to mask transient hardware failures. Sullivan and Chillarege [Sul92] also showed that a large percentage of software errors are triggered by peak conditions in workload, exception handling and timing. Such errors are likely to disappear when the software is reexecuted after a certain amount of clean up and reinitialization [Ber92] This is because the behavior of ....

M. Sullivan and R. Chillarege. A comparison of software defects in database management systems and operating systems. In Proc. of 22nd International Symposium on Fault-Tolerant Computing (FTCS-22), pages 475--484, June 1992.

....it in the case of a failure. 3.4 Fault Model Faults injected into the applications form the second part of the workload. Our primary goal in designing these faults is to generate a wide variety of software crashes. Our models are derived from studies of commercial databases and operating systems [Sullivan92, Sullivan91, Lee93] and from prior models used in fault injection studies [Barton90, Kao93, Kanawati95, Chen96, Ng97] The faults we inject range from low level faults such as flipping bits in memory to high level software faults such as memory management errors and uninitialized variables. We classify injected ....

.... 50 0 1 1 0 Delete Branch 50 1 3 4 0 Delete Inst 50 5 0 1 0 Change Dest Var 50 3 4 4 0 Total 400 17 (4 ) 9 (2 ) 14 (3 ) 0 (0 ) 49 Mark Sullivan and Ram Chillarege looked at the field failures reported for the MVS operating system [Sullivan91] and the DB2 and IMS database systems [Sullivan92]. They classified the errors into types of programming mistakes that caused them. The fault model we use in this fail stop study is largely derived from their work. A prior paper from the Rio project measures corruption in the Postgres database system by running a pre determined workload, ....

[Article contains additional citation context not shown here]

Mark Sullivan and Ram Chillarege. A Comparison of Software Defects in Database Management Systems an d Operating Systems. In Proceedings of the 1992 International Symposium on Fault-Tolerant Computing, pages 475--484, July 1992. 102

....To answer this question, we studied ninety eight post release software faults on a leading on line transaction processing system. For each fault, we determined whether or not it had been covered in prerelease testing. We assigned each fault to a class, using a fault taxonomy from a similar study [SC92]. Finally, we examined which classes of faults were most prevalent, and which classes most often appear in covered branches. To quantify this, we de ned a branch which was modi ed during a x as an a ected branch , and de ned a ected branch coverage to be the percentage of a ected branches that ....

....the faults into fault classes, and to look at the overall a ected branch coverage for the classes where we would expect branch coverage to do well. For the classes of faults, we chose to use the same taxonomy used in previous studies of database and online transaction processing (OLTP) systems [SC92]. This breakdown focuses on software implementation faults rather than faults earlier in the lifecycle. The fault classes used are listed in 2.3 For each of these categories, we investigate the relationship between a ected branch coverage and fault incidence. We have found that whether or not a ....

[Article contains additional citation context not shown here]

M. Sullivan and R. Chillarege. A comparison of software defects in database management systems and operating systems. In 22nd International Symposium on Fault Tolerant Computing, volume 36, pages 475-484. IEEE, July 1992.

....field, and whether or not their underlying faults had been covered in testing, we have acquired a snapshot of the faults that slip by branch testing. For the classes of faults, we chose to use the same taxonomy used in previous studies of database and on line transaction processing (OLTP) systems [Sullivan and Chillarege, 1992]. This breakdown focuses on software implementation faults rather than faults earlier in the lifecycle. The fault classes used are listed in 2.3 For each of these categories, we investigate the relationship between affected branch coverage and fault incidence. We have found that whether or not a ....

....to a fault class. The descriptions in RETAIN did not include a fault classification system: rather, they favored a textual description of the fault. We chose to use the following fault classification that has been used previously in studies of systems similar to the one discussed in this paper [Sullivan and Chillarege, 1992]. Allocation Management : One module deallocates a region of memory before it has completely finished using the region. After the region is reallocated, the original module continues to use it in its original capacity. Copying Overrun : The program copies bytes past the end of a buffer. Data ....

M. Sullivan and R. Chillarege. A comparison of software defects in database management systems and operating systems. In 22nd International Symposium on Fault Tolerant Computing, volume 36, pages 475--484. IEEE, July 1992.

....checksum. Checksums help to detect accidental corruption taking place on checksummed objects outside the update time window. Such corruption can be caused by malignant software faults, such as, wild pointers. Sullivan and Chillarege have presented the common software faults that causes overlays [14, 15, 3] based on field error reports for operating system IBM MVS and database management system IMS) Errors that occur concurrently during the update time window will not be detected by the checksum mechanism. This class of errors are not covered by the hardware memory protection mechanism adopted in ....

....the software faults causing overlay errors are hard to trace and their impact are much higher than the regular errors. Common software faults causing overlay errors are: assignment faults, initialization faults, wild pointers, copy overflow, type mismatch, memory allocation, and undefined state [14, 15, 11]. Overlay errors may corrupt any part of code or data. The data buffer corruption is a subset of the errors resulting from the software faults causing overlays. We inject errors directly into the data buffer area in order to accelerate the database corruption, instead of injecting faults that ....

M. Sullivan and R. Chillarege. A Comparison of Software Defects in Database Management Systems and Operating Systems. Proc. FTCS-22, pages 475--484, 1992.

....as a whole. Although inadvertent flaws do not usually pose an immediate threat to the security of the system, the weakness resulting from a flaw may be exploited by an intruder (see case D1) There are many possible ways to organize flaws within this category. Recently, Chillarege [6] and Sullivan [28] have published classifications of defects (not necessarily security flaws) found in commercial operating systems and databases. Efforts by Bisbey et al. 16] and Abbott [15] provide classifications specifically for security flaws. After trying these classifications, as well as one we developed, ....

M. Sullivan and R. Chillarege, "A Comparison of Software Defects in Database Management Systems and Operating Systems," Proc. 22nd Int. Symp. on Fault-Tolerant Computer (FTCS-22) (July 1992).

....the faults into fault classes, and to look at the overall affected branch coverage for the classes where we would expect branch coverage to do well. For the classes of faults, we chose to use the same taxonomy used in previous studies of database and on line transaction processing (OLTP) systems [Sullivan and Chillarege, 9 1992]. This breakdown focuses on software implementation faults rather than faults earlier in the lifecycle. The fault classes used are listed in 3.3 For each of these categories, we investigate the relationship between affected branch coverage and fault incidence. We have found that whether or not a ....

....and 85 , with the average being about 55 over 22 products. This supports the finding at IBM that testers are unable to estimate their own coverage that coverage measurement tools must be used. One of the most relevant works for this study was published in 1992 by Sullivan and 27 Chillarege [Sullivan and Chillarege, 1992]. These researchers recognized that in order to achieve greater system reliability, it is important to gain an understanding of the faults which evade internal testing. So, they performed an empirical study on the field errors of three IBM software products the MVS operating system, and the ....

[Article contains additional citation context not shown here]

M. Sullivan and R. Chillarege. A comparison of software defects in database management systems and operating systems. In 22nd International Symposium on Fault Tolerant Computing, volume 36, pages 475--484. IEEE, July 1992.

....with a 200 MHz UltraSPARC processor, running ClustRa v.1.1, as illustrated in Figure 4. 3.2. Fault Model Extensive field error reports and surveys are not available in young products like ClustRa, therefore we have to base our fault model on studies conducted in operating systems and databases [10, 11, 8]. Common software faults causing overlay errors are: assignment faults: e.g. code line PreviousLog = tmp; instead of PreviousTrLog = tmp; corrupts both PreviousLog and PreviousTrLog variables, initialization faults: wrong or forgotten initialization of vari ables, wild pointers: assignment or ....

M. Sullivan and R. Chillarege. A Comparison of Software Defects in Database Management Systems and Operating Systems. Proc. FTCS-22, pages 475--484, 1992.

....2.1 Description of Faults This section describes the types of faults we inject into the operating system. Our primary goal in designing these faults is to generate a wide variety of operating system crashes. Our models are derived from studies of commercial operating systems and databases [Sullivan92, Sullivan91, Lee93] and from prior models used in fault injection studies [Barton90, Kao93, Kanawati95, Chen96] The faults we inject range from low level hardware faults such as flipping bits in memory to high level software faults such as memory allocation errors. We concentrate on software faults as studies have ....

Mark Sullivan and Ram Chillarege. A Comparison of Software Defects in Database Management Systems and Operating Systems. In Proceedings of the 1992 International Symposium on Fault-Tolerant Computing, pages 475--484, July 1992.

....errors on production IBM and Tandem systems. Sullivan and Chillarege classify software faults in the MVS operating system and DB2 Measuring Memory s Resistance to Operating System Crashes 3 and IMS database systems; in particular, they analyze faults that corrupt program memory (overlays) [Sullivan91, Sullivan92]. Their study found that most overlay errors are due to boundary condition and allocation problems, and not timing or synchronization problems. They also found that most overlays are small, and corrupt data near the data that the programmer meant to update. Their research provides valuable ....

Mark Sullivan and Ram Chillarege. A Comparison of Software Defects in Database Management Systems and Operating Systems. In Proceedings of the 1992 International Symposium on Fault-Tolerant Computing, pages 475--484, July 1992.

....the data. Section Four presents the results of the studies, and Section Five summarizes the implications of these results for our system availability techniques. For additional details about the studies themselves, see [73] which compares addressing errors to errors overall in MVS, and [74], which focuses on control errors and discusses differences between operating system and database management system errors. CHAPTER 2. A SURVEY OF SOFTWARE ERRORS IN SYSTEMS PROGRAMS 18 2.2 Previous Work We would have liked to use a survey of data collected and analyzed by other researchers to ....

M. Sullivan and R. Chillarege. A comparison of software defects in database management systems and operating systems. Digest 22nd International Symposium on Fault-Tolerant Computing, July 1992.

No context found.

Sullivan, M., Chillarege, R., "A Comparison of Software Defects in Database Management Systems and Operating Systems," 22nd International Symposium on Fault-Tolerant Computing, July 1992.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC