Expert system validation—that is, testing systems to ascertain whether they achieve acceptable performance levels—has with few exceptions been ad hoc, informal, and of dubious value. Very few efforts have been made in this regard in the transportation area. A discussion of the major issues involved in validating expert systems is provided, as is a review of the work that has been done in this area. The review includes a definition of validation within the context of the overall evaluation process, descriptions and critiques of several approaches to validation, and descriptions of guidelines that have been developed for this purpose. The subject of expert systems—smart computers solving difficult problems—is intriguing, and the transportation engineering area is ripe for their development. Transportation engineering, perhaps more than other areas of civil engineering, is characterized by experiential problems, such as the design of highway noise barriers, traffic incident management, analysis and design of pavement rehabilitation
|
116
|
Computer-Based Medical Consultations: MYCIN
– Shortliffe
- 1976
|
|
106
|
A Guide to Expert System
– Waterman
- 1985
|
|
53
|
Managerial applications of neural networks: the case of bank failure predictions
– Tam, Kiang
- 1992
|
|
46
|
Applications of meta level knowledge to the construction, maintainance and use of large knowledge bases, Stanford AI Laboratory, Memo AIM-283
– Davis
- 1976
|
|
42
|
Validating expert system performance
– O'Keefe, Balci, et al.
- 1987
|
|
30
|
Measuring nominal scale agreement among many raters
– Fleiss
- 1971
|
|
29
|
An Approach to Verifying Completeness and Consistency in a Rule-based Expert System
– Suwa, Scott, et al.
- 1982
|
|
15
|
Validating and verifying knowledge-based systems
– Gupta
- 1991
|
|
15
|
Evaluation of Expert Systems: Issues and Case Studies
– Gaschnig, Klahr, et al.
- 1983
|
|
14
|
A report on the Expert systems Validation Associate (EVA). Expert Systems with Applications
– Chang, Combs, et al.
- 1990
|
|
7
|
The treatment advice of a computer-based cancer chemotherapy protocol advisor
– Hickam, Shortliffe, et al.
- 1985
|
|
7
|
On evaluating AI systems for medical diagnosis
– Chandrasekaran
- 1983
|
|
6
|
A Comprehensive Framework for Knowledge-Base Verification and Validation
– French, Hamilton
- 1994
|
|
6
|
Verifying consistency of production systems
– Nguyen
- 1987
|
|
5
|
Validation, Verification and Testing
– Adrion, Branstad, et al.
- 1982
|
|
5
|
The Evaluation of Decision-Aiding Systems: Guidelines and Methods
– O’Keefe
- 1989
|
|
4
|
Developing a strategy for expert system verification and validation
– Lee, O'Keefe
- 1994
|
|
4
|
Knowledge-Based System Verification, Validation, and Testing: The Evolution of a Discipline
– Nazareth, Kennedy
- 1993
|
|
2
|
A review and survey of expert system verification and validation
– O'Keefe, O'Leary
- 1993
|
|
2
|
Knowledge Based Systems in Engineering
– Dym, Levitt
- 1991
|
|
2
|
Dependency Analysis for Knowledge Validation in Rule-Based Expert Systems
– Wu, Lee, et al.
- 1994
|
|
2
|
Verification and Validation of Expert Systems: Experience with Four Diverse Systems
– Prerau, Papp, et al.
- 1993
|
|
2
|
Antimicrobial Selection by a Computer
– Yu
- 1979
|
|
2
|
Verification of Rule-Based Knowledge Using Directed Graphs
– Nazareth, Kennedy
- 1991
|
|
1
|
NCHRP Synthesis 183: Knowledge-Based Expert Systems
– Cohn, Harris
- 1992
|
|
1
|
Expert Systems in Transportation Engineering: A Critical Review and Application Potential. Oklahoma Department of Transportation
– Zaman, Bozai, et al.
- 1993
|
|
1
|
Survey of Research
– Roddis
- 1992
|
|
1
|
Validating Expert System Prototypes Using the Turing Test
– Spring
- 1993
|
|
1
|
Verification, Validation and Evaluation of Expert Systems
– Wentworth, Knauss, et al.
- 1995
|
|
1
|
Testing, Verification and Validation of Expert Systems: A Case Study
– Spring, Collura, et al.
- 1990
|
|
1
|
Models of Consensus for Validation of Expert Systems
– O’Leary, Pincus
- 1993
|
|
1
|
A Framework for Verification, Validation, and Refinement of Knowledge Bases: The VVR System
– Zlatereva
- 1994
|
|
1
|
KBS Maintenance Validation Using Simulation
– Coenen, Bench-Capon
- 1991
|
|
1
|
Tools for Extracting Knowledge and Validating Expert Systems
– Bahill, Jafar, et al.
- 1987
|
|
1
|
Empirical Methods in Expert System Validation
– Landauer
- 1990
|
|
1
|
An Expert System Development Methodology That Supports Verification and Validation
– Culbert, Riley, et al.
- 1989
|
