©1990, 1995 General contents
Chapter 2

Chapter 1: Introduction and study context

1.1 Overview

This inter-disciplinary thesis is broadly concerned with how to investigate and model the cognition behind people's performance of tasks that deal with complex dynamic, or real-time, systems.

The remainder of Chapter 1 (Introduction and study context) introduces the background to this study, details the aspects that are explored in the study, and gives a view of the logical structure of the thesis. The issues of the thesis are not straightforward, and there are no easy solutions reported.

Chapter 2 (Mental models and cognitive task analysis literature) is a literature survey that ends up discovering little of direct positive relevance to the central field of interest. Its main relevance to the rest of the study lies in revealing the way in which the literature as a whole fails to address the relevant areas effectively.

Chapter 3 (Early studies) further defines the area of study by exploring, and ruling out, both a complete study of a naturally occurring complex task (ship navigation), and machine learning approaches that are not based on human performance data.

Chapter 4 (The Simple Unstable Vehicle: a manual control task) reports the more detailed exploration experimentally, of a manual control task (a simulated bicycle-like vehicle) and explains the reasons, chiefly concerning psycho-motor issues, why this too is unsuitable for immediate study here.

Chapter 5 (Non-manual control task selection) is a detailed statement of the necessary features of a task that could be studied in the manner envisaged here. This would be useful for anyone considering undertaking a similar study. It goes on to evaluate options available at the time of the study, concluding that no available systems were suitable, and that a new task had to be constructed specially.

Chapter 6 (The Sea-Searching Simulation task and first experiment) describes the construction of a nautical mine-sweeping simulation task, suitable for the desired experiments, implemented on a Silicon Graphics Iris 3130 workstation, and now being held by YARD Ltd., Glasgow. Rule-induction was found to be a viable analytic tool enabling comparison of representations, and opening up many possibilities for deeper exploration. The methods of data preparation and analysis used, are discussed at length in this chapter.

Chapter 7 (Sea-Searching Simulation task: second experiment), following the findings of Chapter 6, investigates costing information, for a modified version of the same task, as a means of discovering about information use, and about the human structuring of the task. Here, novel analytic techniques are introduced, revealing an individual ‘context’ structure to the task. Ideas both for extensions of these experiments, and for further experiments on the same task, are included in the discussion at the end of this chapter.

Chapter 8 (Overall interpretation of results, conclusions and directions) summarises the main findings of the study, also explaining the nature of the results, along with ideas about the work needed to make the results more general. Suggestions on overcoming the great remaining obstacles to applying this methodology to real tasks are put forward. This application could be to interface redesign, systems design, and training; and for operator support, we introduce the Guardian Angel paradigm, as a vision of what could eventually follow on from this work. Lastly, the implications for further work are explored. These focus around the idea of a new approach to rule induction, using a human-like context structure, which could be based on the principle of minimising the cognitive requirements of executing a task.

1.2 The apparent problem area

1.2.1 The problem in general

For several years, there has been an awareness that tasks involving complex dynamic systems pose particular problems in the field of Human-Computer Interaction (HCI). In 1985, a substantial study was carried out, as part of the Alvey Programme [36], aimed at identifying research needs within this defined area. The common factors of the systems considered in the 1985 study included

The task of controlling such systems was given the title “Information Organisation and Decision-making” (IOD), implying that these tasks formed a natural kind. Quoted as typical examples of such tasks were:

The cited study report tackles three aspects of the problem: applications; technology; and cognitive research. New applications are more complex than older ones, and tolerances are becoming tighter. This demands more sophisticated control, either by more capable people, or by providing aids for the people that there are. Such aids need to go beyond simply collecting the data, to organising it into a higher-level form, perhaps representing goals and sub-goals at varying levels. Ideally a decision aid needs to display “ ‘what the user needs to know’ ” rather than data that are easy to measure (A1 paragraph 3). This brings us to the last of the three aspects of the problem, and it is this that is of most interest here. In an IOD task, how do people organise the information, and how do they make decisions? Without an answer to this question, we risk constructing ‘aids’ that do not in fact aid the people intended. To this end, the 1985 report recommends, (among many other things) that “Research should be performed into the characterisation, representation and evaluation of reasoning strategies during complex command and control interactions...' (7 paragraph 13).

Nor does the problem look like going away soon. A recent paper by Hoc [53] states that still, in many studies, two failings of control rooms are:

And two improvements that are very often suggested are: Presumably these ideas keep on being suggested because no-one knows how to implement them effectively.

The issue of what distinguishes the applications considered in the 1985 study (and this present one) from others, will be taken up below (§1.3.1). But let us note here that there are many areas of information organisation and decision-making that deal, for example, with organisations or businesses, rather than with complex pieces of machinery. Why does the current area of interest focus on machinery? One good reason for this is because it is the mechanical systems where there are already low-level sensors carrying information electronically, and therefore here it is more obvious how in principle one could build a support system that advised the operator about what he or she needed or wanted to know. Another reason is that machinery fails in a more spectacular and immediately dangerous way. Another reason is that in most of the mechanically-based complex systems there is no ‘adversary’—this would add a whole extra layer of difficulty onto the problem (see, e.g., chapters in [41]). To the extent to which one can identify management decision-making at all (doubted even in military circles [62]), those decisions may be based on all kinds of factors, including ones that are not normally electronically measured, and the factors which arise from the likely competition. In mechanically-based systems, at least more of the relevant information would be available, and it is easier and less unreasonable to ignore those sources of information which are not able to be sensed electronically. Thus the mechanically-based tasks are at present more amenable to HCI study and design, whereas the less mechanical tasks, although raising the same issues in principle, cannot currently easily be dealt with from an HCI viewpoint.

1.2.2 Automation

If a task was to be performed by a fully automatic system, which did not need (or did not support) direct human supervision, there would be little motive in designing the system with reference to how a human might perform it. In contrast, many of the systems we are considering are unlikely to be fully automated in the foreseeable future. There are a number of reasons why not. Firstly, we would not generally like to entrust decisions that can affect the lives of people to an automatic system which does not have human accountability, nor the sense of responsibility that comes with that. This may or may not be backed up by legal or licensing requirements. Secondly, there is the need to be able to cope with situations where the control system, for whatever reason, stops working, or malfunctions in a way not explicitly allowed for in the design. This is often termed ‘reversionary control’. Thirdly (a related point), there is a level of complexity beyond which it is impractical to design automatic responses to all possible fault combinations. When something anticipated goes wrong, perhaps an automatic system could have been designed to deal with it, but if something truly unexpected should happen, there will be no automatic system ready programmed for that eventuality. Fourthly, in any system in which people are involved, there are likely to be factors relevant to a decision which are not directly available to automatic sensing or interpreting. In this category would come the personal, the inter-personal, and the social factors, as well as “the apparently unmediated pickup of information from aspects of the system which were never designed for that purpose” [43].

1.2.3 Particular problems

More motivation for improving human-computer interaction in complex system control comes from the results of errors. Often, modern disasters involving high technology do not stem simply from the malfunctioning of a mechanical component, though some such malfunctioning often plays a part. What seems to be generally agreed is firstly that human error often plays a part in accidents, in the sense that if an operator had done something else at a crucial moment, the accident could have been averted; and secondly that the operator's ‘erroneous’ actions have often been performed in the absence of information which indicated a contrary action, despite that information's technical availability. Let us briefly consider some example areas. Maritime collisions

Cahill [19] gives descriptive reports of numerous notable collisions between ships. In every case, there is no doubt that there was some sensible course of action which would have averted the collision. Even in the few cases caused by rudder failure, maintenance of a more cautious separation between the ships could have prevented the collision. Generous (The word ‘strict’ would imply that there was some definite unambiguous interpretation of the rules. This is not the case.) adherence to the Collision Regulations [61] would imply these more cautious miss-distances. The problem is that even if one ship's master adheres generously to the rules, many others will be adhering to rules that are either their own, or very distorted versions of the collision regulations.

Cahill attributes the collisions to a number of causes. First, and most frequently, there is a failure to keep a proper lookout. This means in practice a failure to obtain the information that was obtainable, whether by sight, radar, or VHF communication; and to make the routine inferences from it. Secondly, there is often evidence of low standards of safety: i.e., accepting a lower miss distance than is prudent. And thirdly, there are economic pressures. If saving fuel, or making a deadline at a port, are very highly valued, the prospect of making a large alteration of course for the sake of wider safety margins becomes less attractive.

A belief that other ships are going to behave in an orderly fashion might lead to a combination of the first and second of the attributed causes. Cahill warns that all other ships should be treated with extreme caution, since they may not have a competent person on the bridge, or even in some cases no-one at all! A good example of economic pressure from management leading to low safety margins (more recent than his book) is the Herald of Free Enterprise disaster at Zeebrugge [98].

The availability of the appropriate information, and making necessary inferences from it, are the issues that we are most interested in here. This, and other issues in collision avoidance, will be taken up below (§3.1). Nuclear power plants

The Three Mile Island incident in March 1979 generated considerable literature, but here we need only be concerned with the barest outline of the accident. The technical aspects of the accident are summarised by Jaffe [63], and more of the human factors viewpoint is given by Bignell and Fortune [14], including photographs and diagrams of the working environment. A small number of technical problems provided the background in which the incident developed.

The technical problem thought of as most important by Jaffe, and also mentioned in [14] and by Pheasant [98], was the failure of the “pilot-operated relief valve” to close properly after automatically opening to relieve pressure in the primary coolant system. However, it was indicated closed, because the indicator took its reading from the control signal to the valve, and not from sensing its actual position. One can, with the benefit of hindsight, see this as a design defect, but there are other possibilities that could have overcome this design defect. Probably, what the other relevant instruments read would have been incompatible with the valve actually being closed. Could this not have been detected, by some higher-level sensor? The operators erroneously believed the indicator, but a more thorough understanding of the plant could have led to a correct mistrust. Jaffe lists inadequate training and experience as a factor contributing to the accident. But perhaps another factor is even more significant here, and though given only four words by Jaffe, has been a significant part of hearsay concerning the incident: “A plethora of alarms”. Bignell and Fortune say that two minutes into the incident over 100 alarms were operating.

In effect, the information that would have been most helpful to the operators at the time when the incident was developing was hidden or obscured in several ways. As well as the misleading indicator, and the confusing alarms, a hanging tag obscured an indicator that showed that a different valve that should have been left open was in fact closed. In other words, there were a number of ways in which there was less than helpful provision of accurate information.

It is clear that, as with collision avoidance, there were possible sequences of actions that would have averted the Three Mile Island incident, and presumably other nuclear power plant incidents that have occurred. But where there are such deficiencies in the provision of relevant information to operators, it is hardly reasonable to blame an accident on ‘human error’. The difficulty, in the absence of hindsight, is in knowing what information is relevant to unforeseen circumstances, as well as common ones, and how to provide that information helpfully.

The report of Woods et al. [148], though concerned with nuclear power plant safety, goes beyond description of incidents, to determining whether the current state of models of cognition could help in the prediction of human errors, specifically in the case of emergency operations. They consider the kinds of cognition used in the control of nuclear power plants, using examples from actual incidents, and then consider a system from the artificial intelligence (AI) world that deals with the kind of cognition that they have identified (see the discussion below, §2.1.8.) They see enough overlap to claim that a symbolic processing cognitive model of problem-solving in this domain can be built, as psychologically plausible effective procedure. Whether or not we agree, at least this makes a case for a more detailed consideration of the relevance of this kind of model to costly technological accidents. The study of errors

Although accidents and errors are notable (and newsworthy) aspects of complex systems, the present study is not a study of errors. Human error is studied as a subject in its own right (see, for example, [104], [107]), and some authors believe that the study of errors is the method of choice for the development of improved human-computer interfaces [16]. (For arguments on this point, see below §2.4). The present study will take the view that errors would be elucidated by good models of human task performance, and that one cannot expect to derive good models of human performance from error studies alone. Despite the fact that error information is very useful as an aid to iterative design, there will always remain the problem of designing the first prototype in the best possible way, and for this, models of human performance and cognition could help.

1.3 Placing this study in its general context

The problem area that has just been outlined is clearly extensive. The title of the present work (“Modelling Cognitive Aspects of Complex Control Tasks”) is intended to clarify, in broad terms, the part of the area that is to be dealt with here in greater detail. We shall here take each part of the title, starting at the end, and describe how this helps to define the study.

1.3.1 Control tasks

In the previous section we have mentioned real systems and tasks which have been seen as exemplifying a certain class. How can we define this class other than by example? One adjective that is often applied to such tasks is “dynamic”. This implies that something is happening over time, whether or not a human is interacting with the system at that moment. The system “has a life of its own”, and will not stop and wait for the human to complete lengthy considerations in order to come to the best possible decision.

In contrast, problem-solving has different characteristics. There may be time limits for decisions, but during whatever time is allotted for solving a problem, the problem itself normally does not change. Medical diagnosis is this kind of task, as are mathematical problems, and the kind of puzzles studied by Newell & Simon [91] In that book, humans are shown to have a great diversity of strategy and tactics for problem-solving tasks.

In the literature dealing with human performance of control tasks, particularly complex ones, Jens Rasmussen is a key figure. Much of his work is summarised in one relatively recent book [102]. His ‘stepladder’ diagram (in, e.g., [100]), which is often quoted and reproduced, gives an analysis of the mental activity which can occur between the perception of a signal and the execution of an action. The diagram is arranged in the form of a two-sided stepladder, with the height corresponding to the ‘level of abstraction’. On the left, there are the perceptual steps, from a low level at the bottom to a high level at the top, and on the right there are the steps of decision or action, from high at the top to low at the bottom. Thus, a decision could be initiated by some sensory input; this could be recognised and categorised; and then reasoned about with respect to the overall goals. A decision could be made at a high level, about desired targets, and this could be worked out as a sequence of conscious steps, which would ultimately be effected by unconscious preprogrammed patterns of physical action. Another important aspect of his diagram is that it illustrates possible ‘short cuts’ in mental processing between the legs of the ladder.

In a later paper [101], Rasmussen makes more explicit the concepts of skill-based, rule-based and knowledge-based behaviour. Skilled behaviour, of which the details are largely below conscious awareness, takes direct input from the environment (“signals”) and performs actions at a low level of abstraction. Sporting activities would fall into this category. Rule-based behaviour is where the situation is categorised as requiring a particular response, at a level that is able to be expressed verbally: ‘cookbook recipes’ not involving any reasoning, probably built up from instruction and experience. Information at this level can be thought of as “signs”. Where there are no pre-established appropriate rules governing a situation, knowledge-based reasoning is required, with an explicit consideration of goals, and perhaps planning or search. Here, the corresponding form of information is the “symbol”.

Rasmussen connects the three levels he is distinguishing with three phases of learning a skill: the cognitive, associative and autonomous phases. This implies that although control tasks normally have a substantial skill component when they are established, as they are first learnt they, too, have a knowledge-based character. A corollary is that knowledge-based processing tends to be slow, whereas skill-based processing can be fast enough to form the basis of the real-time control of dynamic systems.

In complex dynamic control tasks, there is no skilled bodily performance to require high-speed processing power, but it is generally agreed that these tasks use the kind of processing that Rasmussen refers to as skill-based. The wealth of information available, and the constraints of time in which to process it and come to decisions, may be the alternative reasons why control tasks need the high-speed processing, characteristic of skilled performance.

In contrast, problem-solving, particularly of the ‘puzzle’ kind (e.g., missionaries and cannibals) met with in AI, often uses a minimum of information defining the problem (typically a short piece of text). If such a problem is not trivial, it must need knowledge-based processing in order to solve it, for if appropriate effective skill-based processes had already been established, we could imagine a very quick solution would be found, since the problem is defined by so little information.

These contrasts between knowledge-based and skill-based processing suggest that in control tasks mental processing has some bias towards the skill end of the continuum, whereas in problem-solving domains, there is the opposite bias towards knowledge-based processing. But to avoid confusion, we must recognise that whenever a process operator goes outside the familiar bounds of experience, their information processing will again be characteristic of the learner, that is, knowledge-based.

Knowledge-based, and problem-solving, tasks, including the problem-solving-like diagnostic tasks that often appear as part of supervisory control, are not here the principal objects of study, because the fact that these situations are less practiced, and the fact that the initially plausible actions have a wider range, together mean that there is less likely to be a uniformly repeated pattern to the skill. If we can first deal with skills that are better learnt, we might then be on the way to dealing with the greater complexity of the knowledge-based area.

1.3.2 Complexity

There has been much study of complexity from the point of view of computer systems or algorithms, which is generally based on some formal representation of the studied system. However, in HCI, some authors take a much more informal view of complexity, which is very reasonable considering how difficult would be the attempt to formalise a real complex industrial process, and how ambiguous the results of such an exercise would be. For example, Woods [145] gives four dimensions of the cognitive demands of a task domain. These are:
  1. dynamism;
  2. the number of parts and the extensiveness of interconnections between the parts or variables;
  3. uncertainty;
  4. risk.
A world where all of these are high would be described as “complex”.

Woods details many ways in which a high value on his dimensions can arise. Specific features that often occur in our typical complex systems include: multiple goals; hidden quantities; long time constants; servo systems embedded within the task; distinct phases to the task with different rules; and a quirkiness that comes from general rules having many exceptions, special cases, etc. However, arriving at a compound measure (even qualitative) based on this multiplicity of factors would be difficult, and might not add much to the common intuitive idea of complexity.

This study proposes a simpler informal operational definition of complexity. A complex task is one for which there are a large number of potential practical strategies. The rationale behind this definition is that if there is only one or a very small number of practically feasible methods of performing a task, then performing it is simply a matter of sticking to explicit rules, such as might be found in a rule book, and there would be the potential for automation, no matter how intricate or involved was the required processing. A typical example of intricate processing that is not complex might be performing arithmetic calculations. Thus, the opposite to ‘complex’ on this definition would not be ‘simple’, but rather ‘straightforward’ or ‘unambiguous’.

Many further examples of tasks that are not complex would be found under the heading ‘clerical’, and indeed, many of these tasks have been automated. When a task has a necessary motor skill component, it is more difficult to be sure of the level of complexity, even under the new definition. How many varying strategies are there for baking a loaf of bread? for sweeping a road? for weeding a garden? The lower the level of analysis, the less clear is the answer. But at least there is one sense in which, say, following a recipe is not complex. That is, that we can describe unambiguously, at some level, what steps should go on—even though the method of effecting those steps (how they should go on) may depend on the person and the situation. The examples of text editing (Chapter 2, passim), and bicycle riding (Chapter 4, come up in the course of this study, and will be discussed further there.

Many tasks are, on the other hand, clearly complex. Programming is an obvious example: for anything but simple programs, people are liable to choose different ways of solving a given problem, and different ways of implementing those solutions, unless constrained by a ‘structured’ methodology. Another good example would be running a business. There may be guidelines, there may be textbooks, but for these tasks, and the control tasks we are considering, there is no fully definitive account of the details of how the task should be performed. This kind of complexity is obviously closely related to the general complexity dimensions given by Woods and by others. In complex tasks, the complexity defies a complete logical analysis, leading to a multiplicity of possible methods. In the doubtful cases, we have not in fact yet performed a complete logical analysis, and therefore we do not know whether there are few or many possible methods.

1.3.3 Cognitive aspects

Many writers in the field of HCI and complex systems (e.g., Rasmussen, Reason) stress the importance of understanding human cognition. The paper by Woods [145] also gives an example of this: “... we need, particularly at this time of advancing machine power, to understand human behavior in complex situations”. Researchers in the field investigate different aspects of cognition, and a number of these will be looked at in Chapter 2.

For tasks that are straightforward rather than complex, with little or no scope for different methods, a logical analysis (including the structures and methods that of necessity arise from the nature of the task itself) might possibly cover the bulk of what is interesting about the task-related cognition. In tasks where the logical structure is salient, it becomes interesting to study the extent to which humans do or do not conform to the logical structure, as Johnson-Laird [64] and others have done. This is not done in the present work.

But the more complex a task is, the more will there be aspects of cognition that are contingent, rather than necessary—‘mental’, rather than purely logical. If the purpose is automation, then arguably how these contingent aspects are implemented is not centrally important; but to understand human action in complex control tasks, we do need to go beyond the necessary logical structure, and investigate some of the contingent cognitive aspects.

These cognitive aspects, that are in the subject area of the present study, include the structures and methods that humans devise to enable them to perform these complex tasks, within the limitations of their ability, and dependent on circumstances. As will be discussed below (§2.6), the aspect of cognition that emerges as central to this study concerns the mental representation underlying the rules describing complex task performance. This goes beyond the logically necessary.

1.3.4 Modelling

Unlike the choices implied by the other parts of the title, we have no option other than modelling, because we cannot directly observe human cognition. Models of some kind have to be built, and if these models are to be validated, they must be tested against the experimental data of recorded human action, which is what we observe.

The choice of what approach to take to modelling is properly explained after the next chapter, which discusses the literature on mental models, cognitive models, and cognitive task analysis. Before starting on that chapter, it may be pertinent to remember that the designers of complex systems form a very important class of end-user for such models. Within the point of view of systems design we can assemble a number of possible purposes for models of operator's mental processes:

  1. to assess what is and what should be in an operator's mental model, and therefore what to present in the course of training;
  2. to enable comparison of different proposed systems, according to formalised measures, for predicting performance or usability of a system or interface (which may include categorising tasks or systems according to their usability by various classes of potential user, predicting likely errors, and demands the system makes on the operator);
  3. to communicate existing wisdom about systems design, based on informal models, providing guidelines on important issues to consider when designing interactive systems and interfaces;
  4. indirectly, to further the understanding of the mental processes in operators, which may lead the designer to more helpful models, whether formal or informal;
  5. to discover what information is actually needed or being used by an operator at a particular time, and therefore how to improve an interface to an existing system;
  6. to predict what information will be wanted by an operator of a system that has not yet been built, to get a good start on designing the interface, and making informed design decisions about the level of support to be provided.
The point raised above (§1.2.1), about “what the user needs to know”, could be taken as referring to either of the last two items in the above list.

If just system designers have so many different possible uses for these models, it is not at all surprising that the literature has a great range. Chapter 2 casts the net wide at first, and there it becomes increasingly apparent which published work relates to which of the purposes, and how relevant it is to our present study. The kind of models and modelling to be studied in this work is best described in conjunction with an overview of the thesis, which now follows.

1.4 Structural outline

Within the context that has just been defined, the thrust of this thesis is to introduce, explore, and progressively to refine, a modelling concept. The idea behind the concept is that we should be able to model something about human cognition by analysing human actions.

In concentrating directly on actions rather than intentions, on what people do rather than what they say, we are moving away from the kind of study based on verbal reports. Verbal reports and protocols (“Protocol” is in this study taken to mean a record made as the task was actually being performed. “Verbal report” is a more general term.) have been the method of choice (or perhaps the default method) for finding out about human concepts and rules for a considerable time. But as Bainbridge points out [6], verbal reports do not directly inform us about what people actually do. We need other methods of study, at least to corroborate verbal reports if not to replace them.

One of the main results that we can hope for from a study of human actions is to discover regularities behind those actions: rules about what action is taken in what situation. Recently, a new paradigm for finding such rules has emerged from the effort to construct “expert systems”, and this is based on machine learning, or, more specifically, rule induction, where rules are induced from examples (see, for example, [87]) Can this paradigm be applied to inducing rules governing human behaviour? One classic study [77] suggests that it can be applied to human diagnosis of soy-bean disease. But in what other situations? Would it be possible to discover rules behind humans' interactions with more complex, less documented and less analysed systems?

If we could discover such rules, they might well form a very significant part of a model of human task performance. One might be able to construct a model predicting the actions that a human would take in a particular task, and it would be natural to ask the question whether these rules could be regarded as an aspect of the mental structures underlying task performance in that human. This could lead on to comparison of these rules with verbal reports, and no doubt many other avenues of investigation.

Looking at these questions leads into the realm of representation, which one can come to see haunting every shadow in artificial intelligence and cognitive science. How do humans internally represent their practical, largely unspoken knowledge? Not only has this question endless fascination, but also answers to it have important practical implications for the design of computer-based information and support systems for the control of complex systems in industry.

It is from a synthesis of such practical and theoretical concerns that this thesis takes its structure, and in outlining the structure of the thesis, we can see more of what the modelling concept is about. At the most abstract level possible, this structure could be given as follows: the practical problems, reflected on, reveal a lack of theoretical underpinning; this is confirmed with reference to related fields of study; an attempt to get to the bottom of the theoretical deficiencies leads to the motive to study human representations, and this motive, tempered by considerations of feasibility and refined by experience, suggests an experimental method; the experimental results give some pointers towards the broader objectives, and show something of the distance that needs to be travelled to get there.

Modelling cognitive aspects of complex control tasks: one possible view of the structure of the thesis

At the next level, some more of the details of the structure appear, illustrated in the Figure. The need to model human cognition, in order to improve human-computer interaction for complex tasks, was given as an initial problem, and is recognised by many writers. Some examples have been given above (§1.2). This leaves us at the uppermost rectangular box in the figure. From the literature on mental models and cognitive task analysis, (Chapter 2), together with early studies, arose the conviction that central to the problem is the issue of representation (the next rectangle). This argument is first made explicit immediately after reviewing the literature (§2.6).

Early studies carried out by the author (Chapter 3) confirmed the importance of representation, and served to explore what research goals might be desirable and feasible. Using maritime collision avoidance (an initially given starting point for the research), as a subject of this study turned out to be fraught with many theoretical and practical problems. A study of machine learning of dynamic control clarified the formal aspect of representation, and confirmed that despite inevitable practical problems, human experimentation was necessary to discover human cognitive structures. A bicycle-like simulation task (the Simple Unstable Vehicle) gave interesting results (in Chapter 4), but it increasingly became clear that this also was off target. What emerged from these studies was a much clearer view that a semi-complex, non-manual task was needed as an experimental vehicle.

The very necessary evaluation of different candidates for this experimental task is not shown on the chart. This is given in Chapter 5. Having chosen the task, implementing it was difficult and very time-consuming; however, again, this stage is a necessary one and as such it adds little to the structure diagram. When originally implemented, it was envisaged that there would be three parallel ways of proceeding from the experiment to evidence about human representations. The first two have already been mentioned: verbal reports; and rule induction. The third depended on discovering different representations for different subjects: not only that, but different representations that could be transformed into different versions of an upgraded interface. In practice, although there was some evidence for differing representations, this was not enough to make different interfaces with any degree of confidence. The experiments that investigated these first two methods are given in Chapters 6 and 7, but the third way remains future work.

It was clear from early on that the originally motivating objectives were out of immediate reach. But it is hoped that a provisional mapping out of how these goals might be achieved could be of some value. In the figure, future paths are shown in the lowest part of the diagram, underneath the box with the heavy outline. These are expounded in Chapter 8.

The justification for this work is not in whether it reaches these further goals: rather, it is that this work explores, and attempts to fill in, some of the necessary foundations on which progress towards those further goals may be built. The further goals have had an important shaping function on the work as a whole, but they remain as ideals. But if one is unaware of them, it is possible to see this research as a collection of disjointed studies, rather than as a coherent whole. The ultimate achievement of the goals required (and still requires) groundwork; the groundwork needed exploration; and that exploratory aim is the point around which this work coheres, and it is reflected in all the sub-studies. The wide range of this exploration is due to the relatively little that had previously been done in this field of study.

A note on terminology used in this study

In this work, the people who interact with complex systems, for whatever reason, in whatever way, are referred to by a number of different terms. Whatever the intention in the literature (see below, §2.1.1), The terms “user”, “operator”, “trainee”, “player”, “subject”, etc., are used here, not to indicate exclusive classes of people, but simply to use a term that fits reasonably into the context. The reader is cautioned against reading too much into this terminology: he or she who is a user could equally be an operator, player, etc.

Next Chapter 2
General Contents Copyright