P. Cohen. A survey of the eighth national conference on artificial intelligence: Pulling together or pulling apart? AI Magazine, 12(1):16--41, 1991.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....for the academics and the industry. In academics, sometime people seek for the accuracy towards 100 . While in industry a guaranteed 60 accuracy is typically aimed for. In addition, profit is the eventual goal of practitioners, so a profit oriented forecasting model may fit their needs. Cohen [5] surveyed 150 papers in the proceedings of the 8th National Conference on artificial intelligence. He discovered that only 42 of the papers reported that a program had run on more than one example; just 30 demonstrated performance in some way; a mere 21 framed hypotheses or made predictions. He ....

P. R. Cohen, "A Survey of the Eighth National Conference on Artificial Intelligence: Pulling Together or Pulling Apart? ", AI Magazine, Vol. 12, No. 1, 1992, pp17-41.

....patterns. Variations in shade are easily detected by the eye, and provide a much wider range of variation with greater simplicity [Tufte90] 8 Previous Work The problems we discuss here are not limited to computer science systems research. Cohen performed a survey of the 1990 AAAI conference [Cohen91], where he found that 41 of systems centered papers (papers that discussed the behavior of a system that had been built) described only a single illustrative example of the system, without applying the system to any well defined benchmark. His later book on empirical methods for Artificial ....

Cohen, P., "A Survey of the Eighth National Conference on Artificial Intelligence: Pulling Together or Pulling Apart?" AI Magazine, 12, 1, pp. 16--41, 1991.

....had to have a clear path to implementation level systems. The importance of this link has been highlighted by several researchers, some even going so far as to state that AI will not advance as a science until the gap between those who construct models and those who build systems is closed [16]. The formal model of Responsibility aids the application (system) designer by: offering a structured framework for knowledge elicitation, providing a domain independent characterisation of the types of events which can cause problems during cooperative problem solving, specifying the key mental ....

P. R. Cohen, A Survey of the Eighth National Conference on Artificial Intelligence: Pulling 54 Together or Pulling Apart, AI Magazine 12 (1) (1991) 16-41.

....many of the characteristics, both positive and negative, of any particular agent architecture, only become evident when experimental evaluation is performed. Indeed, it could be argued as it has been already by a number of researchers in the agent design and related AI subfields [DK90, LD90, Coh91] that to progress as a science, we must develop more rigorous experimental methods. Unfortunately, AI, according to Cohen [Coh91, page 35] is unlike experimental sciences that provide editorial guidance and university courses in experiment design and analysis. Indeed, at present, there is ....

....evaluation is performed. Indeed, it could be argued as it has been already by a number of researchers in the agent design and related AI subfields [DK90, LD90, Coh91] that to progress as a science, we must develop more rigorous experimental methods. Unfortunately, AI, according to Cohen [Coh91, page 35] is unlike experimental sciences that provide editorial guidance and university courses in experiment design and analysis. Indeed, at present, there is no common 151 Evaluating TouringMachines j 152 language or frame of reference for describing and assessing different agent ....

[Article contains additional citation context not shown here]

Paul R. Cohen. A survey of the Eighth National Conference on Artificial Intelligence: Pulling together or pulling apart. AI Magazine, 12(1):16--41, 1991.

.... task environments are the problem domain for control algorithms like many real time and parallel local scheduling algorithms [1, 17, 23] and distributed coordination algorithms [9, 14] The reason we have created the TMS framework is rooted in the desire to produce general theories in AI [5]. Consider the difficulties facing an experimenter asking under what environmental conditions a particular local scheduler produces acceptable results, or when the overhead associated with a certain coordination algorithm is acceptable given the frequency of particular subtask interrelationships. ....

....processing resources available, and soon, learning coordination algorithm parameters. TMS does not at this time automatically learn models or automatically verify them. While we have taken initial steps at designing a methodology for verification (see [11] this is still an open area of research [5]. Our future work will include building new models of different environments that may include physical resource constraints, such as airport resource scheduling. The existing framework may have to be extended somewhat to handle consumable resources. Other extensions we envision include specifying ....

Paul R. Cohen. A survey of the eighth national conference on artificial intelligence: Pulling together or pulling apart? AI Magazine, 12(1):16--41, Spring 1991.

....academic AI research 3 . Theoretical and empirical analysis is now standard. Mathematics is in widespread use as a theoretical tool. As a result we are now developing a good understanding of the range of mechanisms that have been developed in AI. Methodological issues are by no means resolved, [Cohen, 1991], but they are much discussed and a consensus is emerging on the importance of combining theoretical and empirical investigations. 4 The Fragmentation of the Field AI can be viewed as a forcing ground for new computational techniques. The emulation of intelligence provides a very challenging ....

....which to test parsers, etc. These examples are typical and similar examples can now be found in most areas of AI. As a result today s AI programs are a lot more robust than their predecessors. Of course, there is still lots of room for improvement. A recent survey of AAAI 90 conference papers, [Cohen, 1991], contains a good discussion of the issues and an investigation of the current situation. Among several methodological criticisms of the AAAI 90 papers was the observation: Not only were average case hypotheses and predictions rare, so too were follow up experiments. However, the very ....

Cohen, P.R. (Spring 1991). A survey of the eighth national conference on artificial intelligence: pulling together or pulling apart. AI Magazine, 12(1):16--41.

....relationships, and computational organizational design. T MS does not at this time automatically learn models or automatically verify them. While we have taken initial steps at designing a methodology for verification (see (Decker Lesser 1993b) this is still an open area of research (Cohen 1991). Work now being done includes building new models of different environments that include physical resource constraints, such as hospital patient scheduling and airport resource scheduling. Other extensions we are now working on include more helpful facilities for specifying dynamic objective and ....

Cohen, P. R. (1991), `A survey of the Eighth National Conference on Artificial Intelligence: Pulling together or pulling apart?', AI Magazine 12(1), 16--41.

....behavior. The theory should specify just what class of tasks and range of problems are within its scope, and specify how to evaluate whether or not the theory succeeds. Many researchers fail completely to provide such analyses; indeed, in a survey of papers accepted to the AAAI 90 conference, Cohen (1991) discovered that only 43 of the papers that described implemented systems report any kind of analysis of their contributions. Even of the papers that do describe evaluatory experiments, very few go beyond evaluating the programs to analyzing the scientific claims that the programs were written ....

Cohen, P. (1991). A Survey of the Eighth National Conference on Artificial Intelligence: Pulling Together or Pulling Apart? AI Magazine, 12(1):16--41.

.... methodological questions confronting researchers interested in using simulation techniques for performing AI experimentation [13, 19, 47] As Cohen demonstrated in his analysis of the papers presented at AAAI90, we are, as a discipline, just learning how to perform real, systematic experimentation [12]. One hears a lot of talk about AI as an experimental science, but typically the experiments amount merely to writing a computer program that is supposed to validate some hypothesis by its very existence. As Newell and Simon explained it in their Turing lecture, Each new program that is built is ....

P. R. Cohen. A survey of the Eighth National Conference on Artificial Intelligence: Pulling together or pulling apart? AI Magazine, 12:16--41, 1991.

....different from the DVMT, and also help explain the situations under which it gives poor performance. In order to build predictive models of how to best coordinate in an environment, we need to decide how to characterize the environment with respect to the behaviors in which we are interested [4]. This process includes both deciding what aspects need to be represented, and to what accuracy. We are performing such an environmental assessment for what we contend is a large class of agent architectures and tasks, using a simulation model. This simulation model is more complex than an ....

Paul R. Cohen. A survey of the eighth national conference on artificial intelligence: Pulling together or pulling apart? AI Magazine, 12(1):16--41, Spring 1991.

....simulations, and the control processes for almost any distributed or parallel AI application. We have recently extended the T MS framework to model physical resources as well (see Section 4) The reason we have created the T MS framework is rooted in the desire to produce general theories in AI [6]. Consider the difficulties facing an experimenter asking under what environmental conditions a particular local scheduler produces acceptable results, or when the overhead associated with a certain coordination algorithm is acceptable given the frequency of particular subtask interrelationships. ....

....resources available, and soon, learning coordination algorithm parameters. T MS does not at this time automatically learn models or automatically verify them. While we have taken initial steps at designing a methodology for verification (see [12] this is still an open area of research [6]. Our future work will include building new models of different environments that include physical resource constraints, such as airport resource scheduling. Other extensions we envision include specifying dynamic objective models that change structure as the result of agent actions such models ....

Paul R. Cohen. A survey of the Eighth National Conference on Artificial Intelligence: Pulling together or pulling apart? AI Magazine, 12(1):16--41, Spring 1991.

.... there has been a number of worst case complexity results for planning, showing that the general problem is hard and that several restrictions are needed to guarantee polynomial time [2, 3, 5, 6, 11, 12, 22] A criticism of such worst case analyses is that they do not apply to the average case [8, 18]. Recent work in AI has shown that this criticism has some merit. Several experimental and theoretical results have shown that specific NP complete problems are hard only for narrow ranges [7, 10, 19, 20, 27] and suggests that even the instances within these ranges can usually be efficiently ....

P. R. Cohen. A survey of the eighth national conference on artificial intelligence: Pulling together or pulling apart? AI Magazine, 12(1):17--41, 1991.

....This paper illustrates this by contrasting AI and alternative approaches to speech understanding. By so doing it brings out some key characteristics of AI methodology. This paper is primarily written for a general AI audience interested in methodological issues, complementing previous work (Cohen 1991; Brooks 1991a) It is also written for any AI researchers who are contemplating starting a project in speech understanding it is intended to be the paper that, if available earlier, might have saved me four years of wasted effort. This paper does not provide technical discussion of specific ....

Cohen, Paul R. (1991). A Survey of the Eighth National Conference on Artificial Intelligence.

....have to make decisions about the choice and temporal ordering of their actions. This dissertation will demonstrate a framework that can be used to specify the task structure of any computational environment. It will then instantiate an existing methodology (MAD: Modeling, Analysis and Design [Cohen, 1991]) using this framework to analyze a particular computational environment (Distributed Sensor Networks) and predict and verify the performance of two simple coordination algorithms in that environment. Finally, this dissertation will design a family of generic coordination mechanisms for ....

....software engineering activities. 1.2 Analyzing a Distributed Sensor Network Environment The second major result reported here is a detailed analysis of a simplified DSN environment. The methodology behind this analysis is an instantiation of the MAD (Modeling, Analysis, and Design) methodology [Cohen, 1991] , with T MS providing the modeling and simulation components. This part of the dissertation returns to the work of Durfee, Lesser, and Corkill [Durfee et al. 1987] that showed that no single coordination algorithm uniformly outperformed the others. This dissertation explains this result, and ....

[Article contains additional citation context not shown here]

Paul R. Cohen. A survey of the Eighth National Conference on Artificial Intelligence: Pulling together or pulling apart? AI Magazine, 12(1):16--41, Spring 1991.

....constrain agent design; and the design task, in other words, understanding which agent design or configuration produces the desired behaviors under the expected range of environmental conditions. These two tasks, in fact, are the first two stages of a more complete research methodology which Cohen (Cohen, 1991) refers to as the MAD methodology, for modelling, analysis, and design. This methodology aims to justify system design (and re . The remaining design activities predicting how the system (agent) will behave in particular situations, explaining why the agent behaves as it does, and ....

....re . The remaining design activities predicting how the system (agent) will behave in particular situations, explaining why the agent behaves as it does, and generalising agent designs to different classes of systems, environments, and behaviours are beyond the scope of this work. See Cohen (Cohen, 1991, pages 29 32) for details. BDI Modelling for Coordinated Behavior 25 design) decisions with the use of predictive models of a system s behaviors and of the environmental factors that affect these system behaviors. Like IRMA agents in the Tileworld domain (Pollack and Ringuette, 1990) ....

[Article contains additional citation context not shown here]

Cohen, P.R. 1991. A survey of the Eighth National Conference on Artificial Intelligence: BDI Modelling for Coordinated Behavior 39 Pulling together or pulling apart. AI Magazine, 12(1):16---41.

....agent design; and the design task, in other words, understanding which agent design or configuration produces the desired behaviors under the expected range of environmental conditions. These two tasks, in fact, are the first two stages of a more complete research methodology which Cohen [18] refers to as the MAD methodology, for modelling, analysis, and design. 2 This methodology aims to justify system design (and re design) decisions with the use of predictive models of a system s behaviors and of the environmental factors that affect these system behaviors. Like IRMA agents in ....

....is developed incrementally, at the same time as the agent design. In other words, the agent design (or some part of its design) is implemented as early as possible, in order to provide empirical data (or feedback) which flesh out the model, which then become the basis for subsequent redesign [18]. The implications of adopting such a design method, as well as the roles played in this method by the environmental and behavioral analyses referred to above, are discussed in detail elsewhere [7] The present evaluation of TouringMachines is realized through a series of interesting task ....

Paul R. Cohen. A survey of the Eighth National Conference on Artificial Intelligence: Pulling together or pulling apart. AI Magazine, 12(1):16---41, 1991.

.... Consider Cohen s modeling, analysis and design proposal for AI research in general, where the thrust of any particular piece of work is expected to be a theoretical or (more likely) experimental investigation into the performance of an implemented system as a parameter is varied continuously [4]. Global questions involving the purely theoretical identification of situations or problems for which classes of algorithms are well suited are outside the scope of this MAD methodology (the acronym is Cohen s) Other experimental work has focused almost exclusively on randomly generated ....

P. R. Cohen. A survey of the eighth national conference of artificial intelligence: Pulling together or pulling apart? AI Magazine, 12(1):16--41, 1991.

....3 learning algorithms. After this, we address two broad classes of independent variables aspects of the algorithm and aspects of the environment. Finally, we consider some issues in the design and execution of experiments. Many of our suggestions are similar to the excellent points made by Cohen (1991) in his discussion of artificial intelligence, but they seem worth instantiating for the field of machine learning. 3. Dependent Measures of Learning Most definitions of learning rely on some notion of improved performance. Thus, various performance measures are the natural dependent variables ....

....has a clear model of both the algorithm and the learning environment, particularly when working with simple algorithms and artificial domains. If one is willing to make sufficient assumptions about the distribution of training data, one can make detailed predictions about the system s behavior, as Cohen (1991) has encouraged. For example, Pazzani and Sarrett (1990) present an average case analysis of a conjunctive induction algorithm, which lets them predict detailed learning curves for domains with various characteristics. In a similar vein, Thompson, Langley, and Iba (1991) describe an analysis that ....

Cohen, P. R. (1991). A survey of the Eighth National Conference on Artificial Intelligence: Pulling together or pulling apart? AI Magazine, 12 , 16--41.

....and to other books. The practitioner will find cookbook style procedures that can be easily understood and utilized, but mathematically inclined readers will not find any proofs of correctness nor a concise formal specification of the assumptions required to justify correctness of the procedures. Cohen (1991) surveyed the Eighth National Conference on Artificial Intelligence (AAAI 90) and concluded that the methodologies used are incomplete with respect to the goals of designing and analyzing AI systems. Tichy, Lukowicz, Prechelt Heinz (1995) showed in a very large study of over 400 articles that ....

Cohen, P. R. (1991), "A survey of the eighth national conference on artificial intelligence: Pulling together or pulling apart?", AI Magazine 12(1), 17--41.

....and behaviors. By 1989, the year of my tenure decision, I had built the Phoenix system, in which agents attempt to contain simulated forest fires in Yellowstone National Park [25] and I was confident that experiments with these agents would help me discover general principles of agent design [15, 33]. The Phoenix environment, with its fires, wind and terrain, was my laboratory; and Phoenix agents bulldozers, trucks, and the fireboss were my subjects. In general terms my goal was to find predictive relationships between the behaviors of agents, on the one hand, and the agents ....

....on relatively few aspects of the agents structure, task and environment; at least until we build a corpus of basic laws. Most important, the search for laws involves both empirical and analytical work. 3 Empirical Methods In 1990, I reviewed 150 papers from the Eighth National Conference on AI [15]. By asking a dozen questions about each paper and cross tabulating the answers, I learned that the research in these papers was dominated by two methodologies, which I called model centered and system centered. It is not an oversimplication to say system centered papers reported on real ....

Paul R. Cohen. A survey of the Eighth National Conference on Artificial Intelligence: Pulling together or pulling apart? AI Magazine, 12(1):17--41, 1991.

....and representative of a more complex and sophisticated reality. Unfortunately, the current state of the field often elevates these problems to a new status: they become interesting for their own sake rather than as an aid in understanding a system s behavior on larger, more interesting tasks. Cohen s [1991] survey of papers in the 1990 found that 63 of the papers focused on benchmark problems such as N Queens, the Yale Shooting Problem, and Sussman s Anomaly. Yet very few of these papers made explicit the connection between the benchmark problems and any other task. Without this additional analysis ....

....Research into agent design begins with a theory. Of course, the theory, in whole or in part, may be informed by the theorist s previous experiences building large, interesting systems. An experimental research program on agent design includes the following components (cf. Cohen s MAD methodology [Cohen 1991]) ffl A theory T , describing some aspect(s) of agent design and the purported effect of those design aspects on agent behavior in certain environments, particularly describing the the agent s architecture, the environment, and the agent s behavior. 10 ffl An implemented testbed environment E, ....

Paul R. Cohen. A survey of the Eighth National Conference on Artificial Intelligence: Pulling together or pulling apart? AI Magazine, 12:16--41, 1991.

No context found.

P. Cohen. A survey of the eighth national conference on artificial intelligence: Pulling together or pulling apart? AI Magazine, 12(1):16--41, 1991.

No context found.

P. Cohen. A survey of the eighth national conference on artificial intelligence: Pulling together or pulling apart? AI Magazine, 12(1):16--41, 1991.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC