S. Devadas, A. Ghosh, and K. Keutzer. An observability-based code coverage metric for functional simulation. In International Conference on Computer-Aided Design, pages 418-- 425, November 1996. time distance X 0 1 4 5 6 total # of du/ud pairs 1 6 6 1 1 1 16 norm. MTE fault cov. - 0.33 0.67 0.72 0.78 0.83 - TABLE II MTE FAULT COVERAGE ANALYSIS OF EXAMPLE "FIR".  Home/Search   Document Details and Download   Summary   Related Articles   Check

....such biased distributions for random vector generation are needed, and tag coverage is one such metric for evaluating biased random distributions. 1. 1 Previous Work Devadas et al. proposed a metric for measurement of how well an input vector exercises possible errors of interest in the circuit [3]. A design error could either result in an increase ( Delta tag) or decrease ( Gamma Delta tag) in value at the output of a node in the multi valued circuit. A given tag is stimulated by an input vector if the design error would affect an observable output. The number of tags an input vector may ....

Srinivas Devadas, Abhijit Ghosh, and Kurt Keutzer, "An observability-based code coverage metric for functional simulation," in Proceedings of the 33 rd Design Automation Conference, pages 418-425, June 1996.

....fault model which can be applied to hardware and software descriptions [20] Many control dataflow fault models consider the requirements for fault activation without explicitly considering fault effect observability. Researchers have developed observability based behavioral fault models [21, 22, 23, 24] to alleviate this weakness. The OCCOM fault model has been applied for hardware validation [21, 22] and for software validation [23] The OCCOM approach inserts faults called tags at each variable assignment which represent a positive or negative offset from the correct signal value. The sign of ....

....fault models consider the requirements for fault activation without explicitly considering fault effect observability. Researchers have developed observability based behavioral fault models [21, 22, 23, 24] to alleviate this weakness. The OCCOM fault model has been applied for hardware validation [21, 22] and for software validation [23] The OCCOM approach inserts faults called tags at each variable assignment which represent a positive or negative offset from the correct signal value. The sign of the error is known but the magnitude is not. Observability analysis along a control flow path is ....

[Article contains additional citation context not shown here]

S. Devadas, A. Ghosh, and K. Keutzer, "An observabilitybased code coverage metric for functional simulation", in International Conference on Computer-Aided Design, pp. 418-- 425, November 1996. 5

....a very late stage of design. In [8] a coverage measure to compute the level of design validation obtained is proposed. This paper does not provide an approach to generate functional test pattern targeting the design errors modeled. Furthermore, their design error model works at the gate level. In [9] the authors propose a tag coverage method to evaluate the quality of a set of functional vectors based on HDL coverage and observability information. This information is used to determine if the effects of errors that are activated by the test vectors are observable at the outputs. Observability ....

....In this way, the three problems of design errors identification, testability estimation and faults detection are faced by using an integrated testing strategy. Future work will consider the comparison of the proposed bit coverage metric to coverage criteria based on observability measures (e.g. [9, 12]) Acknowledgments We would like to acknowledge Prof.Sonza Reorda and Dr.Violante of Politecnico di Torino, which provide test vectors for block coverage. ....

S. Devadas, A. Ghosh, and K. Keutzer. An Observability-Based Code Coverage Metric for Functional Simulation. Proc. IEEE ICCAD, pp.418-425, 1996.

....that certain important functions of the controller have not been excited during simulation. The primary drawback of these approaches is that since the inputs to the controller are abstracted, the uncovered edges may be unreachable in the complete design. A related paper is that of Devadas et al. [6], wherein estimation of test suite coverage is derived for high level designs using a generalization of the calculus used for fault simulation. They make the important point that bugs need to be both excited as well as made visible. 2.4 Sequential ATPG There are ....

Devadas, S., A. Ghosh, and K. Keutzer: 1996, `An Observability-Based Code Coverage Metric for Functional Simulation'. In: Proc. Intl. Conf. on Computer-Aided Design.

....cation. This direction, of checking whether the speci cation describes the system exhaustively, has roots in simulation based veri cation techniques, where coverage metrics are used in order to reveal states that were not visited during the testing procedure (i. e, not covered by this procedure) [HMA95, HYHD95, DGK96, HH96, KN96, FDK98, MAH98, BH99, FAD99]. For example, code coverage [CK93] measures the fraction of HDL statements executed during simulation, transition coverage [HYHD95, HMA95] measures the fraction of transitions executed, and tag coverage [DGK96] attributes variables with tags that are used to detect whether assigning a forbidden ....

.... not covered by this procedure) HMA95, HYHD95, DGK96, HH96, KN96, FDK98, MAH98, BH99, FAD99] For example, code coverage [CK93] measures the fraction of HDL statements executed during simulation, transition coverage [HYHD95, HMA95] measures the fraction of transitions executed, and tag coverage [DGK96] attributes variables with tags that are used to detect whether assigning a forbidden value to a variable leads to an erroneous behavior of the system. In [FDK98, FAD99] Fallah et al. compute the tag coverage achieved by simulation and generated simulation sequences that cover a given tagged ....

S. Devadas, A. Ghosh, and K. Keutzer. An observability-based code coverage metric for functional simulation. In Proceedings of the International Conference on Computer-Aided Design, pages 418-425, November 1996.

....a particular statement, but fails to propagate the effect of executing that statement to a part of the machine state that is truly observable (those signals that are also part of the specification) cannot be considered to have covered that statement. To address this shortcoming Devadas et al. [Deva96] propose a code coverage metric based on tag propagation, which was later refined in [Fall98a] and is called OCCOM, which stands for observability based code coverage metric. Errors are associated with assignment statements in the code. The effect of an error is represented by a tag that can ....

....by a tag that can propagate 16 through the circuit according to a set of rules similar to the D calculus [Abra90] The metric measures the fraction of tags that have been propagated to the observable state over the number of tags injected. The major extension of [Fall98a] to the earlier work in [Deva96] is an efficient method for computing OCCOM coverage using a standard logic simulator. The efficiency of computation is closely related to the definition of the tag propagation rules. The propagation rules are defined so that, in essence, the erroneous machine stays on the same execution path as ....

S. Devadas, A. Ghosh, and K. Keutzer. Observability-based code coverage metric for functional simulation. In Proc. Int. Conf. Computer-Aided Design, pages 418--425, 1996.

....of quantifying the effectiveness of a test set. Various coverage metrics have been proposed to address this problem. These include code coverage metrics from software testing [1,4,7] finite state machine coverage [16,17,23] architectural event coverage [17] and observability based metrics [12]. A shortcoming of all these metrics is that the relationship between the metric and detection of classes of design errors is not well understood. A different approach is to use design error models to guide test generation. This exploits the similarity between hardware design verification and ....

S. Devadas, A. Ghosh, and K. Keutzer, "Observabilitybased code coverage metric for functional simulation", Proc. IEEE/ACM International Conference on Computer-Aided Design, 1996, pp. 418--425.

....that certain important functions of the controller have not been excited during simulation. The primary drawback of these approaches is that since the inputs to the controller are abstracted, the uncovered edges may be unreachable in the complete design. A related paper is that of Devadas et al. [6], wherein estimation of test suite coverage is derived for high level designs using a generalization of the f0; 1; D; D;Xg calculus used for fault simulation. They make the important point that bugs need to be both excited as well as made visible. 2.4 Sequential ATPG There are commonalities ....

S. Devadas, A. Ghosh, and K. Keutzer. An Observability-Based Code Coverage Metric for Functional Simulation. In Proc. Intl. Conf. on Computer-Aided Design, November 1996.

....random test pattern generators, or by hand. If for a given test the safety property is violated, the corresponding monitor enters a violation state, agging the failure. E ectiveness of simulation sequences is assessed using several coverage metrics: code coverage [86, 32, 33] tag coverage [35], event coverage [48] and state machine coverage [70, 53, 51, 52] Code based coverage includes statements, branch and path coverage. Tag coverage evaluates the observability of possible incorrect evaluation represented as tags at the circuit outputs. Event coverage is measured by activating the ....

S. Devdas, A. Ghosh, and K. Keuter. An Observability-Based Code Coverage Metric for Functional Simulation. In Proc. Intl. Conf. on Computer-Aided Design, pages 418-425, November 1996.

....that certain important functions of the controller have not been excited during simulation. The primary drawback of these approaches is that since the inputs to the controller are abstracted, the uncovered edges may be unreachable in the complete design. A related paper is that of Devadas et al. [34], wherein estimation of test suite coverage is derived for high level designs using a generalization of the f0; 1; D; D;Xg calculus used for fault simulation. They make the important point that bugs need to be both excited as well as made visible. Conservative approximations Several authors ....

S. Devadas, A. Ghosh, and K. Keutzer. An Observability-Based Code Coverage Metric for Functional Simulation. In Proc. Intl. Conf. on Computer-Aided Design, November 1996. 120

....excite all behaviors, since their observation is guaranteed thanks to the simulation environment, where the designer can inspect all internal values of the design. However, when dealing with production testing, but also for black box verification, the effects of observability can not be neglected [DGKe96]: all activated instructions must be observed at the circuit primary outputs. In the ARTIST framework, the generated sequences must observe a block after having executed it; to lead the GA to this goal we add to the fitness equation the term O, that measures how close the sequence is to observe ....

....the fault free system. A good example of this approach is in [FADe99] where analysis of data dependencies across conditional statements helps estimating the set of signals affected by an assignment. In ARTIST, we implemented an observability strategy more approximate than the one proposed in [DGKe96] and [FADe99] due to the looser integration with the simulator in our case. ARTIST explicitly traces the set of variables and signals to which the target fault has been propagated by analyzing the simulation trace, with the knowledge of the data transfers performed in each basic block. This ....

S. Devadas, A. Ghosh, K. Keutzer, "An Observability-Based Code Coverage Metric for Functional Simulation," Proceedings IEEE/ACM International Conference on Computer Aided Design, 1996 18

.... of modeling faulty behavior observation (since software metrics only consider reachability of conditions, that corresponds to fault controllability) This paper concentrates on the last mentioned issue, namely the necessity of taking into account observability during test pattern generation [DGKe96]. Test generation, other than activating some statements in the HDL description (therefore controlling the resulting logic) requires to observe the effects. This may correspond to observing the value that has been written to a register (observing an operation in the data path) or the fact that ....

.... 40 50 60 70 80 90 100 ARTIST COMMERCIAL 1 10 100 1000 10000 100000 1000000 ARTIST COMMERCIAL Figure 2: ARTIST and a commercial ATPG: fault coverage (above) and CPU time (log, below) However, when dealing with production testing, the effects of observability can not be neglected [DGKe96]: one must guarantee that all activated instructions can be observed at the circuit primary outputs. In ARTIST, this amounts to requiring the generated sequences to observe a block after having executed it; to lead the GA to this goal we add to the fitness equations the term O, that must ....

[Article contains additional citation context not shown here]

S. Devadas, A. Ghosh, K. Keutzer, "An Observability -Based Code Coverage Metric for Functional Simulation," Proceedings IEEE/ACM International Conference on Computer Aided Design, 1996

.... relevance with the advent of high level testability flows and that is proven to yield good coverage with actual defects [SGTT00] In recent years, several research activities contributed to pushing testability related issues to the RT level, including the proposal of several fault models [DGKe96] [TAZa99] MAHo96] RiUc96] the development of fault simulators [FiFu00] FDKe98] or testability analyzers, and of some test pattern generators [FFSc98] RVEC98] CSSq00b] FADe99] The hardest theoretical barrier to the diffusion of test related tools at the RT level is the lack of widely ....

.... description styles that prevents the use of techniques suitable for control oriented or data oriented circuits, only; the complexity of timing schemes (multiple clock circuits are difficult to handle, as they were at the gate level) the difficulty of modeling faulty behavior observation [DGKe96] (since software metrics only consider reachability of conditions, that corresponds to fault controllability) The fault model chosen in this paper is an instantiation of the observability enhanced statement coverage metric proposed in [DGKe96] and [FDKe98] This fault model requires that all ....

[Article contains additional citation context not shown here]

S. Devadas, A. Ghosh, K. Keutzer, "An Observability-Based Code Coverage Metric for Functional Simulation," Proceedings IEEE/ACM International Conference on Computer Aided Design, 1996

....interactions with the environment [FiFu00] and so on) No single fault model is universally accepted, since no comprehensive and general results, valid for all classes of circuits, are known yet. One of the most used fault model is the observability enhanced statement coverage metric proposed in [DGKe96] and [FDKe98] This fault model requires that all statements in the VHDL description are executed at least once, and that their effects are propagated to at least one primary output. Propagation is modeled implicitly, by determining whether the faulty statement may influence the output values but ....

....singlebit stuck at fault is defined as a single bit stuck at in the effect of an RT level assignment operation: when a fault is present, the affected object (signal or variable target of an assignment statement) loads the correct value, except for one bit that remains stuck to 0 or 1. As in [DGKe96], faults are single and permanent: only one fault is inserted at a time and the fault effect is present during the whole simulation. The RT Level single bit stuck at fault model does not explicitly consider control flow faults, such as stuck at true or stuckat false, as [RiUc96] does. Figure 1 ....

S. Devadas, A. Ghosh, K. Keutzer, "An Observability -Based Code Coverage Metric for Functional Simulation," Proceedings IEEE/ACM International Conference on Computer Aided Design, 1996

....specification also escapes the attention of the designer, who is often the one to provide the specification. In simulation based verification techniques, coverage metrics are used in order to reveal states that were not visited during the testing procedure (i. e, not covered by this procedure) [HMA95,HYHD95,DGK96,HH96,KN96,FDK98,MAH98,BH99,FAD99]. These metrics are a useful way of measuring progress of the verification process. However, the same intuition cannot be applied to model checking because the process of model checking visits all states. We can say that in testing, a state is uncovered if it is not essential to the success of ....

S. Devadas, A. Ghosh, and K. Keutzer. An observability-based code coverage metric for functional simulation. In Proc. 8th CAD, pp. 418--425, 1996.

....that certain important functions of the controller have not been excited during simulation. The primary drawback of these approaches is that since the inputs to the controller are abstracted, the uncovered edges may be unreachable in the complete design. A related paper is that of Devadas et al. [5], wherein estimation of test suite coverage is derived for high level designs using a generalization of the f0; 1; D; D; Xg calculus used for fault simulation. They make the important point that bugs need to be both excited as well as made visible. 1.1.4 Conservative approximations Several ....

S. Devadas, A. Ghosh, and K. Keutzer. An Observability-Based Code Coverage Metric for Functional Simulation. In Proc. Intl. Conf. on Computer-Aided Design, November 1996.

....additional restrictions such as single design error, make these metrics even harder to justify. The fourth type of coverage metric is tag coverage, that evaluates error observability at the HDL level. For example, positive or negative mutations in assigned value are chosen as error models in [30]. OCCOM, a related observability based code coverage metric [31] breaks up the coverage computation into two phases: functional simulation of a modified HDL model using a commercial simulator, followed by analysis of a flow graph extracted from the HDL model using concurrent evaluation techniques. ....

S. Devadas, A. Ghosh, and K. Keutzer. An Observability-Based Code Coverage Metric for Functional Simulation. In Proc. Intl. Conf. on ComputerAided Design, pages 418--425, 1996.

....techniques, where coverage metrics are used to improve the quality of test vectors. For example, code coverage [CK93] measures the fraction of HDL statements executed during simulation, transition coverage [HYHD95,HMA95] measures the fraction of transitions executed, and tag coverage [DGK96] attributes variables with tags that are used to detect whether assigning a forbidden value to a variable leads to an erroneous behavior of the system. In [FDK98,FAD99] Fallah et al. compute the tag coverage achieved by simulation and generated simulation sequences that cover a given tagged ....

S. Devadas, A. Ghosh, and K. Keutzer. An observability-based code coverage metric for functional simulation. In Proc. CAD, pp. 418--425, 1996.

.... These techniques include guaranteed coverage of every statement in an HDL description [8] evaluation of transition coverage of a given test set [16] abstraction of models and semantic control over transition coverage [13] and a coverage metric based on observability error propagation [11]. Again, most of these coverage metrics do not provide a measure of the design error coverage. In the more specific context of processor verification, the application of formal verification techniqueshas been limited by the high design complexity. Most automatic methods based on state space ....

S. Devadas, A. Ghosh, and K. Keutzer. An observability-based code coverage metric for functional simulation. In Proc. IEEE Int. Conf. on Comput.-Aided Design, pages 418--425, Nov. 1996.

....that certain important functions of the controller have not been excited during simulation. The primary drawback of these approaches is that since the inputs to the controller are abstracted, the uncovered edges may be unreachable in the complete design. A related paper is that of Devadas et al. [5], wherein estimation of test suite coverage is derived for high level designs using a generalization of the f0; 1; D; D; Xg calculus used for fault simulation. They make the important point that bugs need to be both excited as well as made visible. 1.1.4 Conservative approximations Several ....

S. Devadas, A. Ghosh, and K. Keutzer. An Observability-Based Code Coverage Metric for Functional Simulation. In Proc. Intl. Conf. on Computer-Aided Design, November 1996.

....coverage metric has been chosen because exercising computation paths in a circuit is a tried and trusted way of design debugging. Since the number of paths in the circuit may be too large to enumerate, a subset of paths is selected. Other coverage metrics, such as statement coverage, and OCCOM [10], 11] can also be used in this framework. This will only impact the specifics of the generated satisfiability problem. The remainder of this paper is organized as follows. Section II describes previous work on automatic test generation. Our functional vector generation method is presented in ....

....recently. Design specific coverage analysis and test generation approaches have been proposed for microprocessors (e.g. 1] 17] 18] It is not clear how well these strategies work for application specific designs. Metrics for coverage computation for general HDL models are presented in [10], 13] 15] An observability metric is defined in [10] no test generation method is given. The transition based methods of [13] 15] heuristically select state transitions in the state space to generate functional vectors. The extended finite state machine model is used to automatically ....

[Article contains additional citation context not shown here]

S. Devadas, A. Ghosh, and K. Keutzer. An Observability-Based Code Coverage Metric for Functional Simulation. In Proceedings of the International Conference on Computer-Aided Design, pages 418-- 425, November 1996.

....statement in an HDL model [7] evaluation of transition coverage of a test set [14] and abstraction of models and semantic control over transition coverage [10] These metrics do not directly address observability requirements. Observability requirements are addressed in the metric proposed in [8]. In order to compute observability information, variables are tagged during simulation, and a simulation calculus is used to calculate the coverage provided by an arbitrary set of functional vectors. There are several drawbacks with the tag simulation algorithm and calculus presented in [8] 1. ....

....in [8] In order to compute observability information, variables are tagged during simulation, and a simulation calculus is used to calculate the coverage provided by an arbitrary set of functional vectors. There are several drawbacks with the tag simulation algorithm and calculus presented in [8]. 1. The calculus was developed for only a subset of Verilog In particular, nested if statements cannot be handled, and neither can looping constructs used in popular HDL s. 2. The efficiency of the simulation algorithm leaves much to be desired because the calculus dictated the use of an ....

[Article contains additional citation context not shown here]

S. Devadas, A. Ghosh, and K. Keutzer. An Observability-Based Code Coverage Metric for Functional Simulation. In Proceedings of the International Conference on Computer-Aided Design, pages 418-- 425, November 1996.

....again not very meaningful since it does not address the observability requirement. A recent development in coverage metrics for simulation based validation has been the proposal of an effective observability based statement coverage metric along with a fast companion procedure for evaluating it [3, 4]. This metric is more accurate than just statement coverage since it also incorporates observability criteria. On the other hand, it is also more practical than path or transition coverage since it leads to a much smaller number of vectors. The evaluation procedure proposed in [4] is ....

....at a specific location in the HDL code to some output. This is much harder to do than computing a controllability metric which just determines if a vector results in a statement being visited. A review of previous work on controllability and observability based metrics is provided in [4] In [3], the notion of a tag to model the possibility that an incorrect value is computed at a location was introduced, where a location corresponds to an assigned variable in some statement in the HDL code. As the name suggests, a tag is just a label. It does not have a value, and has nothing to do with ....

S. Devadas, A. Ghosh, and K. Keutzer, "An ObservabilityBased Code Coverage Metric for Functional Simulation," in Proceedings of the International Conference on ComputerAided Design, pp. 418--425, November 1996.

....levels of performance currently achievable using various metrics, normalized to HDL simulation time. Computing simple controllability metrics adds some overhead 2 ; somewhat higher overhead is incurred when computing observability information at the HDL level using a specialized tag simulator [5], or when using gate level fault simulators. A simple example of why observability information is important to the designer is shown in Figure 2. Assume that model A and model B are both exercised thoroughly using a functional vector set. Controllability metrics will report 100 statement coverage ....

....metrics is consistent with our information on commercial tools such as Simulation Technology Summit s Vericov or Advanced Technology Corporation s Covermeter. 5 50 1.0 1.2 3 5 25 1. 5 4 Our approach for OCCOM computation Gate level fault simulation HDL simulation Observability metric of [5] Fig. 1. Coverage Metric Computation Overheads Model A Model B c F 1 0 Fig. 2. Observability Versus Controllability Coverage (can be) directly exploited. There is a loss of simulation efficiency due to the addition of new variables, but this is not large (cf. Section VI) 2. We create a flowgraph ....

[Article contains additional citation context not shown here]

S. Devadas, A. Ghosh, and K. Keutzer. An ObservabilityBased Code Coverage Metric for Functional Simulation. In Proceedings of the International Conference on ComputerAided Design, pages 418--425, November 1996.

No context found.

S. Devadas, A. Ghosh, and K. Keutzer. An observability-based code coverage metric for functional simulation. In International Conference on Computer-Aided Design, pages 418-- 425, November 1996. time distance X 0 1 4 5 6 total # of du/ud pairs 1 6 6 1 1 1 16 norm. MTE fault cov. - 0.33 0.67 0.72 0.78 0.83 - TABLE II MTE FAULT COVERAGE ANALYSIS OF EXAMPLE "FIR".

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC