Design Space Analysis Bridging from Theory to Practice via Design Rationale

Project Nr BRA 3066 AMODEUSDESIGN SPACE ANALYSIS:BRIDGING FROM THEORY TO PRACTICEVIA DESIGN RATIONALEALLAN MACLEAN, RICHARD YOUNG1,VICTORIA BELLOTTI and THOMAS MORAN2Rank Xerox EuroPARC61 Regent StreetCambridge CB2 1ABEngland1 Also at MRC Applied Psychology Unit,15 Chaucer Road, Cambridge, England2 Now at Xerox Palo Alto Research Center,3333 Coyote Hill Road, Palo Alto CA 94304, U.S.A.Email: maclean.europarc@SUMMARYThis paper reports work being carried out under the AMODEUS project (BRA3066). The goal of the project is to develop interdisciplinary approaches tostudying human-computer interaction and to move towards applying the resultsto the practicalities of design. This paper describes one of the approaches theproject is taking to represent design - Design Space Analysis. One of its goals ishelp us bridge from relatively theoretical concerns to the practicalities of design.Design Space Analysis is a central component of a framework for representingthe design rationale for designed artifacts. Our current work focusses morespecifically on the design of user interfaces. A Design Space Analysis isrepresented using the QOC notation, which consists of Questions identifying keydesign issues,Options providing possible answers to the Questions, and Criteriafor assessing and comparing the Options. In this paper we give an overview ofour approach, some examples of the research issues we are currently tackling andan illustration of its role in helping to integrate the work of some of our projectpartners with design considerations.In Proceedings of Esprit'91. Brussels, November 25-29, 1991. pp 720-730AMODEUS CP20MacLean et al. – Design Space Analysis1. DESIGN SPACE ANALYSISAMODEUS (Assimilating Models of Design, Users and Systems) is a project which brings together teams from different disciplines with three general objectives (1) to extend the scope of modelling techniques to provide analytic leverage on the problems of user-system interaction: (2) to bridge the conceptual gaps between behavioural and computing disciplines: and (3) to bridge from theory to the practicalities of designing software artifacts. This paper reports work aimed at supporting the third of these objectives. The work reported has two goals. The first is to develop a technique for representing design decisions which will, even on its own, support and augment design practice. The second goal is to use the framework as a vehicle for communicating and contextualising more analytic approaches to user-system interaction into the practicalities of design. However, we believe that the second of these goals cannot be achieved without also achieving the first. This paper describes some aspects of Design Space Analysis, the basis of our framework for representing design, some prospects for it being the basis of tools for supporting design in the future, and its relationship to the other activities taking place within the AMODEUS project.Design Space Analysis is an approach to representing design rationale (10, 12). It is a central part of a long term project in which we are interested in helping software designers reason about design (individually and in groups) and produce an output which can help others to understand why the resulting design is the way it is. Although our intent is to develop a framework for design applicable to a variety of domains, much of our work to date has been aimed at HCI audiences for two main reasons. First, the design domain in which we are most actively working is user interface design. Secondly, we believe that HCI expertise is a crucial ingredient in developing the approach itself as the usability of the tools and techniques we produce will ultimately be critical in determining the acceptability of our work to its users - i.e. the designers of computer systems.A key characteristic of our approach is that the output of design is conceived of as a design space rather than a single artifact. The approach therefore contrasts with the traditional conception of design which assumes that the eventual output is a specification or artifact. The final artifact, although embodying the designer's decisions, does not preserve any of the thinking and reasoning which went into its creation. We use a semi-formal notation (called QOC, for Questions, Options & Criteria) to represent the design space around an artifact being produced. This design space is an explicit representation of alternative design options, and an explicit representation of reasons for choosing among those options. The main concepts we use for the representation areEsprit Conference 1991. Brussels, November 25-29, 1991.Questions which highlight key issues in the design,Options which are effectively answers to the Questions and Criteria which are the reasons that argue for or against the possible Options. Figure 1 illustrates the relationship between these constituents of the design space.Figure 1.The components of a design space. Options can be thought of as "answers" to Questions. Questions highlight important dimensions in the design space. Criteria argue for or against possible Options. We find that considerable power can be gained from regarding relationships between Options and Criteria as relatively positive (solid line between Option and Criterion) or negative (dotted line) - i.e. arguing for or against the Option. Options may spawn off consequent Questions which allow more detailed aspects of the design to be addressed.This representation of a design space provides a succinct rationale for the final design by placing it in a broader context which highlights how it might be different and why it is the way it is. Such a representation should be able to support communication between people with different backgrounds and goals, for example between membersMacLean et al. – Design Space Analysisof a design team working on an initial design, between the original designers and designers of a later generation system who want to re-use parts of the original design, and even between the designers and users of a computer system. Exploring these claims is a future part of our research strategy. However, Design Space Analysis provides a theoretical framework for design which we have already found useful for helping us better understand design issues (10, 11, 12, 13). This theoretical approach drives such activities as understanding the design process and how it can be improved (11); requirements for tools to support the creation of a QOC representation (and thus the design process) (12); and the integration of other approaches, such as cognitive and system HCI modelling techniques, into software design (1). We believe that our approach has the potential of supporting designers with techniques and ways of working which will be useful even in the absence of computer based tools (11, 12). However, if appropriate tools can be devised more powerful support should clearly be possible.In this paper we briefly discuss three of these perspectives. First we illustrate design reasoning. We then highlight some of the most difficult issues which our experience so far leads us to believe need to be tackled to provide adequate computer based support for Design Space Analysis. Finally we illustrate the relationship between Design Space Analysis and HCI modelling techniques within the context of the AMODEUS project.2. REASONING IN A DESIGN SPACEIn this section we illustrate design reasoning within part of a design space represented using the QOC notation. The example is based on a small issue in the design of a bank ATM (Automated Teller Machine) user interface. The course of reasoning is hypothetical, but the content is based on some of our own analyses of the ATM design space, and on insights from designers we have observed working on the same problem (11).For present purposes, let us consider only one very small part of the ATM design space – when to return the ATM card to a customer. The following example illustrates this Question along with a matrix of possible Options (i.e. possible "answers" to the question), some Criteria which are relevant for evaluating these Options, and an indication of whether each Option is good (+) or bad (–) relative to each Criterion. The precision with which Criteria are related to Options could easily be greater – e.g. by using a five point scale. However, for an initial qualitative exploration of the design space, a simple binary distinction as we use here is often sufficient. For more general purposes, and for larger design spaces, we would normally represent the design spaceEsprit Conference 1991. Brussels, November 25-29, 1991.as set of nodes and links, perhaps using a hypermedia system such as NoteCards (5) –however, the visualisation shown in Figure 2 gets the main points of present interest across, and can in fact be useful for reasoning about local parts of a design space.Figure 2. A piece of design space to help reasoning about the Question "When should the ATM card be returned to the customer?"The two Options being compared initially in Figure 2 are: to return the card to the customer immediately after inserting it, or to return it as the final part of the transaction. The bracket Criterion claims that since the user initiates the interaction by inserting the card, removing the card is an appropriate action for terminating the interaction as the card can then be perceived as a token bracketing the interaction with the machine. The speed Criterion claims that if the user deals with the card only once (inserting it and removing it immediately afterwards), it will reduce the number of steps required to use the machine because inserting and removing the card becomes one step, and so immediate return is the better Option. The security Criterion claims that returning the card at the end is preferable, on the grounds that the bank can retain the card if necessary, and information about the current transaction can be recorded on the card.We would not of course necessarily want to conclude that the card should be returned at the end of the transaction since that option "wins" by two plusses to one. For example, we might want to stress the speed Criterion very strongly, and so go for the faster Option.We may want to question the appropriateness of one of the Criteria being used.MacLean et al. – Design Space AnalysisFor example, for the bracket Criterion we may claim that the card is not the natural focus for the user of an ATM; people normally use such machines to get money; once they get the money they have achieved their goal in using the machine and are likely to walk away forgetting to remove their card. As a result of this argument, we may want to introduce a new Criterion which reflects compatibility with the users' perceptions of the task they are carrying out and which suggests that getting cash, not using an ATM card, is the task focus. Moreover, it would suggest that perhaps the evaluation on the security Criterion should be re-thought as someone else may pick up a card which has been left behind in error. Gaining a richer understanding of the relevant Criteria does not simply mean being better informed about which of the current Options to choose – it may help to generate alternative Options. For example, in this case we could add an Option to retain the card until all transactions have been specified, but to give back the card immediately before giving out the money, or perhaps to give back the card and money together from the same slot in the machine.Note that the very nature of design (certainly the kind of design which we are considering) is that there is no one "best" solution. Using the QOC notation to represent the design illustrates why this is so. The designer has two related tasks. One is to understand the relevant design space (potential options and reasons for choosing among them), and the other is to find an appropriate solution within that space. Since the space of possible options is effectively open ended, and is subject to constant change (e.g. new technology coming along, new ideas on the part of the designers, changing requirements from the users), the potential design space is in a constant state of flux and so a previously good solution within that space may become inappropriate. A discussion of some of the ways in which the representation of the space can be refined and reformulated is given in (3).3. TOOLS FOR SUPPORTING DESIGN SPACE ANALYSISOur emphasis is to understand where current design practice can be improved, and where its characteristics place particular challenges for support. Part of our research strategy has been to produce Design Space Analyses ourselves. We have made use of simple document management tools such as text editors and drawing packages together with ubiquitous aids such as pencil and paper. The most sophisticated system we have employed has been NoteCards (5), used for holding representations of design spaces. A full analysis capturing a range of different kinds of information we have alluded to can become a large and messy object to draw on paper, so that appropriate computer support is essential if Design Space Analysis is to prove practicable for design practice. ThisEsprit Conference 1991. Brussels, November 25-29, 1991.Section discusses the some of the challenges for such tools to be sufficiently usable and useful.One problem with current tools which claim to help users structure ideas is that they force the user into making premature commitments. They insist on content being mapped into structure from the very beginning, before the user understands how best to characterise the ideas in terms of the structure. And once the structure is built, significant redundant effort is needed to change it (4, 14). This issue of premature commitment is widely recognised as a problem within, for example, the hypertext community (6). In the context of design, it causes difficulties. Even in our own analyses, it is not uncommon for our initial conceptualisation of part of the design to prove inappropriate. Thus, in a tool for supporting the manipulation of a QOC representation, it is highly desirable that the designer be allowed gradually to impose structure on initially unstructured material, and later to change it easily.A second requirement of a tool for supporting Design Space Analysis is that it allow multiple views of the representation. In order to use it flexibly for different activities such as building new structure, browsing existing structure to understand the design, finding relevant Criteria, and so on, designers need to see views of the design space which are appropriate for the different tasks. They need to be able to move around, selecting different views as their understanding of the design changes. They may, for instance, want to filter the level of detail shown by specifying which types of objects and links should be displayed. Or it may be that a node-and-link style of presentation is not the most useful for certain purposes. For understanding the local evaluation of design Options, for example, they may want to select a particular Question and be shown an evaluation matrix of Options against Criteria, with each cell containing an indication of the link between them as we illustrated in the previous section. Alternatively, they may require a global view to see how one particular part of the design fits into the larger context.If a tool had knowledge just of the QOC structure (as opposed to content), it could be able to offer low-level assistance to the designer in building the rationale. In (12), for example, we offer a number of heuristics for constructing a design space and improving a design, some of which suggest that each Option should be connected to at least two Criteria, with at least one of the links being positive and one negative. A modestly "unintelligent" tool would be able to check that this heuristic had been followed, and otherwise suggest it as a piece of advice. This role of "helpful assistant" is analogous to that proposed by Young & Harris (15), who describe a structure editor for creating viewdata screens which helps to ensure that the range of necessary tasks areMacLean et al. – Design Space Analysisall undertaken. Similarly, Harp & Neches (8) point out that simply being able to detect the presence or absence of information can be used as the basis for simple automated reasoning, even if the information itself remains informal and its content therefore inaccessible to the machine.4. DESIGN SPACE ANALYSIS: FROM MODELLING TO DESIGNFigure 3 shows a characterisation the AMODEUS project structure which emphasises the role the Design Space Analysis framework for design rationale plays in the project. By having a number of very different perspectives in one project each approach expects to benefit from the strengths of the others, and indeed the project can be viewed in ways which emphasise other relationships among the contributing partners. A brief description of this project will help to highlight how some of the challenges outlined above may be addressed.The project can be characterised in three distinct "layers" covering a spectrum from various approaches to modelling human-computer interaction which are relatively formal and limited in scope, through to studies of design practice where a wide range of relatively informal unstructured activities take place. The system modelling component is centred around work on formal specifications from the computer science tradition (e.g. 9). The user modelling component comes from the psychological tradition and two approaches are represented in the project. One of these – PUMs (Programmable User Models) – is based around an AI problem-solving architecture (17). It is basically a problem-space model of the human cognitive architecture (based on SOAR) and embodies constraints on the representation, processing and acquisition of knowledge. The other is based on a cognitive resources approach, embodied within an expert system (2). The design study activity is geared towards exploring design practice with the aim of eventually incorporating the modelling activities into an applied setting (cf. 7).Mediating between theory and practice, the scenario methodology (16) is aimed at developing a common set of exemplars for the different project activities to focus on, to improve communication between the various approaches. Finally, Design Space Analysis serves a role of providing a representation which helps mediate between the limited scope of the detailed modelling approaches and the relatively ill-structured breadth of design practice.Esprit Conference 1991. Brussels, November 25-29, 1991.Figure 3. Schematic of the AMODEUS project, emphasising the influences on the Design Space Analysis component.The AMODEUS project pushes our Design Space Analysis work in two different directions. First of all, the design study part of the project helps us compare our current conception of Design Space Analysis with design practice. For example the ATM design problem, on which the example described earlier was based, is helping us understand the extent to which examples of our concepts emerge naturally from designers working together. Understanding this relationship is important to ensure that human-human communication mediated by a design space representation will be fruitful. A corollory of this research strategy is that we do not believe that an approach which places excessive emphasis on preserving the richness of the design process willMacLean et al. – Design Space Analysisproduce a useful representation for design – the resulting representation would be excessively complex and unwieldy.Secondly, the relationship with the modelling approaches has a role of developing Design Space Analysis in rather different directions. Our current conception emphasises that the design space itself has to be designed – it provides no assistance in helping constrain possible design options or relevant reasons for choosing among them. We are currently exploring the incorporation of aids to design into Design Space Analysis. Modelling approaches such as the ones being examined in this project are one important direction to pursue. The different modelling techniques incorporate a variety of approaches which assist with reasoning about design. By incorporating appropriate aspects of the modelling approaches into Design Space Analysis, we expect to assist the designer with the design task in the areas where the modelling techniques apply, while also providing an explicit context of the parts of the design where the modelling techniques do not apply.5. AN EXAMPLE: THE UNDO SCENARIOAs a simple example of a relationship between the theoretical modelling approaches and Design Space Analysis, we will finish by briefly considering the analysis of a scenario which has recently been carried out within the AMODEUS project.The scenario concerns the design of an undo facility in a multi-user text editor. The original scenario decription is about one page of text, but the key points are as follows:A team of designers is designing a multi-user text editor. They are considering twoversions of how the undo facility should work:1)Relative to the individual: each Users can UNDO only their own actions.2)Relative to the document: actions can be UNDOne regardless of who carriedout the original action.The designers are also considering two versions of the software:A)The first release will have a single insertion point, with control movingbetween the users by explicit token passing.B)The second release will have multiple insertion points, so that each user canmake changes simultaneously.What can we advise them about the usability of the different designs?A typical Design Space Analysis might start by simply structuring the basic information given in the design problem as shown by the Questions and Options in Figure 4.Figure 4. A Design Space Analysis of the UNDO scenario. Questions and Options are structured directly from the information in the scenario description. Criteria are derived from the analyses carried out by the theoretical modelling. Solid lines linking Options to Criteria represent postive assessments - i.e a claim that the Criterion supports the Option. (Negative assessments are not shown to minimise clutter on the Figure.)The Criteria in Figure 4 represent some considerations from the analyses carried out by the theoretical modelling groups. The solid lines show which Options the Criteria are claimed to support. For example, the user predictability Criterion is derived from principles which the system modelling group can represent in a formal specification of the system behaviour. It basically claims that the user should be able to predict the effect of an undo action and that a system which carried out UNDO relative to the document as whole rather than the actions of an individual user would not meet the Criterion. The modelling argument is of course much more detailed than it ispossible to do justice to here, but it should suffice to add that the further justification rests on considering what aspects of the document would be visible to the user and nature of transitions between system displays resulting from user actions.Similarly, the learnability Criterion is based on the analyses from one of the user modelling groups. It also argues for UNDO being focussed around the actions of an individual user rather than the state of the document, but in this case the argument rests on the claim that learning how UNDO operates relies on immediacy in spotting the relationship between an action and an undesired result, and that this is not likely to be perceived unless the user was himself the originator of the action. These examples illustrate one role of the Design Space Analysis as showing how the different, more rigourous, analyses from the theoretical modelling groups relate to each other by mediating their claims via a representation of the design space. Further discussion of the justification of Option-Criterion links based on theory can be found in (12).The various modelling approaches can clearly play an important role in helping to justify and evaluate candidate design Options. However, an alternative Design Space Analysis of the same scenario can be used to illustrate a more powerful role of user modelling in helping to drive design more directly. Rather than use the information contained in the design problem to impose initial structure on the design space, the user modelling analysis acts as the starting point. The PUMs knowledge analysis of the scenario identified four key pieces of knowledge which the user would need to operate an UNDO facility:1. Know what the relevant stream of activity is.2. Know how the stream is articulated into units.3. Know which unit of activity is affected by UNDO4. Know what the effect of UNDO is on that unit.We have used these key knowledge requirements to suggest QOC Questions to "seed" the design space. Figure 5 shows part of a Design Space Analysis based on these Questions. The aim here is simply to illustrate the rather different emphasis which this orientation gives the design space, so Criteria have not been represented and we do not intend to provide a detailed discussion of the Options represented. A point to note is that the Questions are much more user-oriented than those used in Figure 4. This is particularly important in helping maintain a user-oriented view from the outset by structuring the view of the design space around considerations relevant for the user. In addition, since the Questions are not simply structuring information already given, they serve the role of opening up the design space by helping to suggest a much broader range of possible design solutions, and help get at more detailed aspects of the design which may be necessary to gain a better understanding of the key issues.Figure 5. Part of a Design Space Analysis which starts from Questions generated by the PUMs knowledge analysis. The Options are design possibilities suggested by the Questions. (Criteria to help evaluate the Options are not represented.)6. CONCLUSIONSWe have presented an overview of the Design Space Analysis component of the AMODEUS project and illustrated some of the problems and prospects for developing it into a technique for supporting software design. More specifically, we have also illustrated how the approach relates to the range of activities being explored in the AMODEUS project and some of the mutual benefits which results from these activitiesbeing carried out within a single project. Theoretical modelling activities provide more rigorous analysis than is possible with Design Space Analysis on its own, and Design Space Analysis provides a mechanism for grounding these analyses within the details of design.REFERENCES(1)AMODEUS (1989).Applying Models Of DEsign, Users and Systems(AMODEUS). Technical Annexe. Esprit Basic Research Action 3066, Brussels.(2)BARNARD P., WILSON, M. and MACLEAN, A. (1988). Approximatemodelling of cognitive activity with an expert system: A theory based strategy for developing an interactive design tool.The Computer Journal,31, 445-456. (3)BELLOTTI, V., DOURISH, P. and MACLEAN, A. From Users' Themes toDesigners' DReams: Developing a Design Space for Shared Interactive Technologies. AMODEUS RP6/WP7.(4)GREEN, T. R. G. (1989). Cognitive dimensions of notations. In A. Sutcliffe & L.Macaulay (Eds.),People and Computers V: Designing for Usability,Proceedings of HCI'89, September, Nottingham,443-460. Cambridge: Cambridge University Press.(5)HALASZ, F., MORAN, T. and TRIGG, R. (1987) NoteCards in a Nutshell. InProceedings of CHI+GI'87: Human Factors in Computing Systems, Toronto, Canada. ACM, New York, 45-52.(6)HALASZ, F. (1988) Reflections on NoteCards: Seven Issues for the NextGeneration of Hypermedia munications of the ACM,31, 836-852.(7)HAMMOND, N., JØRGENSEN, A.H. , MACLEAN, A., BARNARD, P. andLONG, J. (1983) Design Practice and Interface Usability: Evidence from Interviews with Designers. In Proceedings of CHI'83: Human Factors in Computer Systems, Boston. ACM, New York, 40-44.(8)HARP, B., and NECHES, R. (1988). Notecards: An everyday tool for aiding incomplex tasks. In Collected Papers of the Workshop on Architectures for Intelligent Interfaces: Elements and Prototypes, Asimolar, California, 287-304.ACM/SIGCHI.(9)HARRISON, M. ROAST, C. and WRIGHT, P. (1989). Complementary Methodsfor the Iterative Design of Interactive Systems. In Proceedings of HCI International, Boston.(10)MACLEAN, A., YOUNG, R., and MORAN, T. (1989). Design Rationale: Theargument behind the artifact. In Proceedings of CHI'89: Human Factors in Computing Systems, April 30 - May 4, Austin, Texas, 247-252. New York: ACM.。