MULTI-ATTRIBUTE VALUE ANALYSIS 1

Multi-attribute value analysis

This chapter examines in detail how decisions involving multiple criteria can be analysed using a formal approach known as multi-attribute value analysis (MAVA). MAVA is one of a whole family of formal approaches which seek to take explicit account of multiple criteria in helping individuals or groups explore decisions that matter, referred to collectively as multicriteria analysis or multicriteria decision analysis (MCDA). One of the central ideas of MAVA is that by splitting the problem into small parts, initially focusing on each part separately and then putting together all the information using a structured framework, the decision maker is likely to acquire a better understanding of a problem than would be achieved by just taking a holistic view. Each stage of the analysis is described in detail and illustrated by reference to a simple application. An important aim of MCDA is to help decision makers cope with difficult issues by helping them through the complexity to a simple representation of the problems which may initially challenge their intuitive thinking but eventually provides a better understanding of the issue and a sounder basis for decision making.

1.

What is multicriteria decision analysis? ................................3

This section introduces multicriteria decision analysis (MCDA) which is a collection of formal approaches which may be used to help individual or groups of decision makers explore decisions which matter. MCDA helps groups and individuals in decision making. It does not seek to replace intuition but to aid thinking. This section also gives an overview of the MCDA process, defines terminology and provides examples of multicriteria problems. In addition it introduces multi-attribute value analysis (MAVA) and the simple multi-attribute rating technique (SMART) which you will study in depth in this chapter.

2.

Problem structuring and model building ............................ 17

In this section we look at problem structuring, the process of making sense of an issue, and from this understanding deriving the framework necessary to proceed with MAVA. This involves identifying alternative courses of action and criteria relevant to the decision and structuring the criteria as a value tree.

3.

Eliciting values scoring and weighting ............................ 36

We now move on to the evaluation phase of MAVA, scoring the performance of alternatives with respect to the criteria and assessing (or weighting) the relative importance of the criteria.

4.

Synthesis of values: aggregating the benefits using the additive model .................................................... 51
Having obtained scores describing the performance of the office sites in the light of each of the benefits, and weights reflecting the importance of the benefits relative to each other, we can now begin to piece this information together to enable us to compare the overall performance of the offices. A simple weighted sum gives an initial, easy comparison, but this may not be enough to give the decision-maker a full understanding and confidence in a decision. Sometimes the results of the model can conflict with the decision-makers intuition forcing further hard thinking

5.

Review exercises ............................................................... 70

2 ANALYTICAL SUPPORT FOR DECISION MAKING

Introduction
As we have seen, many decision problems involve a number of objectives, or criteria, and often these conflict with each other. Earlier we focused on unaided decision making, looking at descriptive models of how individuals make decisions. These models (for example, elimination by aspects, satisficing) reflect the unaided decision makers limited information processing capacity and tendency to use simplifying strategies to cope with large complex problems which present too much information to be handled simultaneously. As we illustrated, empirical studies of how we choose between alternatives have revealed that choice is highly dependent on the specific situation. In this chapter we examine in detail how decisions involving multiple criteria can be analysed using an approach known as multi-attribute value analysis (MAVA). MAVA is one of a whole family of formal approaches which seek to take explicit account of multiple criteria in helping individuals or groups explore decisions that matter, referred to collectively as multicriteria analysis or multicriteria decision analysis (MCDA). One of the central ideas of MAVA is that by splitting the problem into small parts, initially focusing on each part separately and then putting together all the information using a structured framework, the decision maker is likely to acquire a better understanding of a problem than would be achieved by just taking a holistic view. Before looking in detail at the multi-attribute value approach, we will reflect more generally on MCDA and how it would claim to inform the decisionmaking process.

Learning outcomes
At the end of this chapter you should have: an appreciation of the nature of the process of MCDA an appreciation of the potential benefits of MCDA a detailed understanding of MAVA, encompassing initial problem structuring, building a value tree, eliciting values (scores and weights), use of the MAVA model to synthesise values, and the nature and role of sensitivity analysis the ability to apply MAVA to an appropriate problem using the VISA decision support tool

MULTI-ATTRIBUTE VALUE ANALYSIS 3

What is multicriteria decision analysis?

It is not difficult to think of decisions which require us to pay attention to multiple criteria. Even in simple personal choices such as selecting a new house or apartment, relevant criteria may include price, the size and nature of the accommodation, the local area and amenities, accessibility to public transport, personal safety and different parties to the decision may well have different objectives. Management decisions at a corporate level in both public and private sectors will typically involve consideration of a much wider range of criteria, especially when consensus needs to be sought across widely disparate interest groups. For example, a local government body, faced with a decision on the route of a new road, might have to balance objectives such as minimising cost, minimising environmental damage, maximising ease of construction, minimising impact on the community, etc. If the route incurring the lowest construction costs would also lead to the destruction of an important wildlife habitat then judgement about the relative importance of these factors may be unavoidable. To some extent, every decision we ever take requires the balancing of multiple factors, or criteria sometimes explicitly, sometimes without conscious thought so that in one sense we are all well practised in multicriteria decision making. For example, when you decide what to wear each day you probably take into account what you will be doing during the day, what kind of impression you want to create, what you feel comfortable in, what the weather is expected to be, whether you want to risk getting that jacket that has to be dry-cleaned dirty, etc. Sometimes you may even try out a number of different alternatives before deciding. Have you ever stopped to build a formal model to analyse this particular decision? The answer is probably no the issue does not seem to merit it, it is not complex enough, it is easy enough to take account of all the factors in your head, the consequences are not substantial, are generally short term and

4 ANALYTICAL SUPPORT FOR DECISION MAKING

mistakes are easily remedied. In short, the decision does not matter that much. However, in both personal and group decision-making contexts, we are at times confronted with choices that do matter the consequences are substantial, impacts are longer term and may affect many people, and mistakes might not easily be remedied. It is under these circumstances that more formal approaches can help, since it is well known from psychological research that the human brain can only simultaneously consider a limited amount of information, so that all factors cannot be resolved in your head. The very nature of multiple criteria problems is that there is much information of a complex and conflicting nature, often reflecting differing viewpoints and often changing with time. One of the principal aims of MCDA approaches is to help decision makers organise and synthesise such information in a way that leads them to feel comfortable and confident about making a decision, minimising the potential for postdecision regret by being satisfied that all criteria or factors have properly been taken into account. Hence, the definition which we hinted at above: MCDA is a collection of formal approaches which seek to take explicit account of multiple criteria in helping individuals or groups explore decisions that matter. Decisions matter when the level of conflict between criteria, or of conflict between different stakeholders regarding the relevance and importance of the different criteria, assumes such proportions that intuitive gut-feel decision making is no longer sufficient or satisfactory. This can happen even with personal decisions (such as buying a house), but becomes much more of an issue when groups are involved (such as in corporate decision making, or in other situations in which multiple stakeholders are involved). However we do not advocate that analysis should replace intuition, simply that it can aid thinking, an issue to which we will return throughout the course.

What can we expect from MCDA?

In answering this question it is important to begin by dispelling the following myths: Myth 1: MCDA will give the right answer Myth 2: MCDA will provide an objective analysis which will relieve decision makers of the responsibility of making difficult judgements Myth 3: MCDA will take the pain out of decision making.

MULTI-ATTRIBUTE VALUE ANALYSIS 5

Firstly, there is no such thing as the right answer even within the context of the model used. The concept of an optimum a uniquely defined best answer does not exist in a multicriteria framework. MCDA is an aid to decision making, a process which seeks to: integrate objective measurement with value judgement make explicit and manage subjectivity. Subjectivity is inherent in all decision making, in particular in the choice of criteria on which to base the decision, and the relative weight given to those criteria. MCDA does not dispel that subjectivity; it simply seeks to make the need for subjective judgements explicit and the process by which they are taken into account transparent (which is again of particular importance when multiple stakeholders are involved). This is not always an easy process: the fact that trade-offs are difficult to make does not mean that they can always be avoided. MCDA will highlight such instances and will help decision makers think of ways of overcoming the need for difficult trade-offs, perhaps by prompting the creative generation of new options. It can also retain a degree of equivocality by allowing imprecise judgements but cannot take away completely the need for difficult judgements.

Aims of MCDA
The aims and benefits of MCDA should be: to facilitate decision makers learning about the problem faced to increase their understanding of their own priorities, values and objectives and those of other parties and of the organisation to explore these in the context of the problem to guide them in identifying a preferred course of action. This view of MCDA is shared by many others prominent in the field of MCDA, as illustrated by the following quotes: Simply stated, the major role of formal analysis is to promote good decision making. Formal analysis is meant to serve as an aid to the decision maker, not as a substitute for him. As a process, it is intended to force hard thinking about the problem area: generation of alternatives, anticipation of future contingencies, examination of dynamic secondary effects and so forth. Furthermore, a good analysis should illuminate controversy to find out where basic differences exist in values and uncertainties, to facilitate compromise, to increase the level of debate and to undercut rhetoric in short to promote good decision making. Keeney and Raiffa (1972) pp 65-66

6 ANALYTICAL SUPPORT FOR DECISION MAKING

... decision analysis [has been] berated because it supposedly applies simplistic ideas to complex problems, usurping decision makers and prescribing choice! Yet I believe that it does nothing of the sort. I believe that decision analysis is a very delicate, subtle tool that helps decision makers explore and come to understand their beliefs and preferences in the context of a particular problem that faces them. Moreover, the language and formalism of decision analysis facilitates communication between decision makers. Through their greater understanding of the problem and of each others view of the problem, the decision makers are able to make a better informed choice. There is no prescription: only the provision of a framework in which to think and communicate. French (1989) p1 We wish to emphasise that decision making is only remotely related to a search for the truth ... the theories, methodologies, and models that the analyst may call upon ... are designed to help think through the possible changes that a decision process may facilitate so as to make it more consistent with the objectives and value system of the one for whom, or in the name of whom, the decision aiding is being practised. These theories, methodologies, and models are meant to guide actions in complex systems, especially when there are conflicting viewpoints. Roy (1996) p11 The decision unfolds through a process of learning, understanding, information processing, assessing and defining the problem and its circumstances. The emphasis must be on the process, not on the act or the outcome of making a decision. Zeleny (1982) decision analysis helps to provide a structure to thinking, a language for expressing concerns of the group and a way of combining different perspectives. Phillips (1990) p150 Note that the focus is on supporting or aiding decision making; it is not on prescribing how decisions should be made, nor is it about describing how decisions are made in the absence of formal support. There is substantial work in the field of behavioural psychology that is concerned with descriptive decision theory. This work has both informed and fuelled debate amongst those concerned with aiding decision making. The above brief discussion and comments establish the context in which we believe multiple criteria methods in general and MAVA in particular to be practically useful and sets the scene for this chapter. We would like to emphasise a number of points.

MULTI-ATTRIBUTE VALUE ANALYSIS 7

MCDA seeks to take explicit account of multiple, conflicting criteria in aiding decision making. The MCDA process helps to structure the problem. The models used provide a focus and a language for discussion. The principal aim is to help decision makers learn about the problem situation, about their own and others values and judgements, and through organisation, synthesis and appropriate presentation of information to guide them in identifying, often through extensive discussion, a preferred course of action. The analysis serves to complement and to challenge intuition, acting as a sounding-board against which ideas can be tested it does not seek to replace intuitive judgement or experience. The process leads to better considered, justifiable and explainable decisions the analysis provides an audit trail for a decision. The most useful approaches are conceptually simple and transparent. Even so, non-trivial skills are necessary in order to make effective use even of such simple tools in a potentially complex environment.

Multi-attribute value analysis (MAVA) and other MCDA approaches

Multi-attribute value analysis, the approach we will be studying in depth, is one of the more widely applied multicriteria methods. It is underpinned by a set of axioms derived from decision theory: these will be discussed towards the end of this chapter but, for the moment, we can regard them as a set of generally accepted propositions or a formalisation of common sense (Keeney, 1982). If the decision maker accepts the axioms then it follows that the results of the analysis will indicate how to behave. The analysis could thus be seen as normative or prescriptive, showing which alternative should be chosen if a decision maker acts consistently with their stated preferences. There is substantial debate about whether: true preferences exist in the decision makers head and the role of analysis is to elicit or excavate these preferences, or preferences are not predetermined and the role of analysis is to help the decision maker construct their preferences through increasing their understanding of the issue. Although some values will be well formed, our sympathy is nevertheless with the constructivist view.

8 ANALYTICAL SUPPORT FOR DECISION MAKING

Multi-attribute value analysis is normally applied in situations where there is a choice between, or evaluation of, a number of well-defined alternatives, and where the outcome of a particular course of action is regarded as certain, or virtually certain, so that uncertainty is not a major concern of the analysis. Where uncertainty does play a major role, there are a number of ways in which it may be considered in conjunction with the analysis of multiple criteria, for example: through the use of scenarios; by formal modelling using decision trees; by using interval, fuzzy or probabilistic measures to capture judgements; or through extensive sensitivity analyses. You might ask why there is a collection of MCDA methods rather than a single one. One reason is that not all multi-criteria decisions involve choice between well-defined alternatives. In other circumstances the actual options may not be clear initially, or they may be infinitely many, for example: Where should we build a new hospital? What is the best route for a new highway? How best can we encourage the use of public transport? How should I invest this inheritance? What is the best design for an aircraft wing? Such problems are often referred to as design problems the challenge is to design the best way forward rather than to choose between a number of prespecified options and a collection of approaches based on mathematical programming techniques is well suited to this context. A second reason is that several groups interested in the practical use of MCDA felt that early approaches (proposed in the 1970s) were too mathematical, were based on strong assumptions, and made too heavy demands on decision makers. This led to a number of simpler approaches, based on fewer assumptions or working with poorer data. The particular approach to MAVA we will study in this unit is based on a method known as SMART (simple multi-attribute rating technique) which was an outcome of these concerns, which also led to the development of a quite different group of approaches collectively known as outranking methods (detailed exploration of these methods is beyond the scope of this chapter, but an introduction can be found in Chapter 8 of Belton and Stewart (2001).

MULTI-ATTRIBUTE VALUE ANALYSIS 9

An overview of the MCDA process

No matter which particular MCDA approach is used to inform decision making, the analysis needs to be embedded in a larger process of problem structuring and resolution, as illustrated in Figure 1.

Figure 1

MCDA process

This figure, which is deliberately messy in order to convey the nature of MCDA in practice, shows the main stages of the process from the identification of a problem or issue, through problem structuring, model building and using the model to inform and challenge thinking, and ultimately to determine an action plan. This plan may take many forms, for example, to implement a specific choice, to put forward a recommendation, to establish a procedure for monitoring performance, or simply to maintain a watching brief on a situation. We group these stages into three key phases: problem identification and structuring model building and use development of action plans. The initial problem structuring phase is one of divergent thinking, opening up the issue, surfacing and capturing the complexity which undoubtedly exists, and beginning to manage this and to understand how the decision makers might move forward. The phase of model building and use represents a more convergent mode of thinking, a process of extracting the essence of the issue from the complex representation in a way which supports more detailed and precise evaluation of potential ways of moving forward. It is possible that the outcome of this phase is a return to divergent thinking, a need to think creatively about other options or aspects of the issue. A phrase which captures the nature of the overall process very effectively, representing a recurrent theme throughout the unit is

10 ANALYTICAL SUPPORT FOR DECISION MAKING

through complexity to simplicity. Multiple criteria models often appear very simple, and indeed have been criticised as simplistic (see the quote by French on page 6). However, this neglects the nature of the above process. The apparently simple model does not deny the complexity, but has emerged from it as a distillation of the key factors in a form which is transparent, easy to work with, and which can generate further insights and understanding. There are many actors central to the process; these include decision makers, clients, sponsors, other stakeholders, including potential saboteurs, and facilitators or analysts. As indicated in Figure 2.1, one can expect iteration within and between the key phases of the process, each of which is subject to internal and external influences and pressures. It is at the stage of building and using a model that the different MCDA approaches are distinguished from each other; in the nature of the model, in the information required and in how the model is used. Within that they have in common the need to define, somehow, the alternatives to be considered, the criteria or objectives to guide the evaluation, and usually some measure of the relative significance of the different criteria. It is in the detail of how this information is elicited, specified and synthesised to inform decision making that the methods differ. From now we will focus specifically on multi-attribute value analysis and in particular the SMART approach.

Basic terminology
Before proceeding to describe the process of multi-attribute value analysis in detail we pause briefly to clarify the basic terminology we will be using, some of which has been used already in a casual way. Alternatives, options, choices, courses of action, strategies These are all terms used to describe the choice of options open to the decision maker. They will be used interchangeably as befits the situation. Criteria, objectives, attributes These are all terms used to describe factors which the decision maker wishes to take account of in evaluating options. We will use the term criteria to refer in a general sense to factors relevant to the decision, including objectives and attributes, which are used more specifically. An objective has been defined by Keeney and Raiffa (1976) as an indication of the preferred direction of movement with respect to a particular criterion. Thus, when stating objectives, we use

MULTI-ATTRIBUTE VALUE ANALYSIS 11

terms like minimise or maximise. Typical objectives might be to minimise costs or maximise market share. An attribute is a characteristic of the options being evaluated which is measurable against some objective or subjective yardstick. An attribute is used directly to measure performance in relation to an objective. For example, if we have the objective maximise the exposure of a television advertisement, we may use the attribute number of people surveyed who recall seeing the advertisement, in order to measure the degree to which the objective was achieved. Sometimes we may have to use an attribute which is not directly related to the objective. Such an attribute is referred to as a proxy attribute. For example, a company may use the proxy attribute staff turnover to measure how well they are achieving their objective of maximising job satisfaction for their staff. Value and utility For each course of action facing the decision maker, we will derive a numerical score to measure its overall attractiveness as well as evaluations with respect to all the relevant criteria. Where the decision involves no element of risk and uncertainty this score is referred to as the value of the course of action. Alternatively, where the decision involves risk and uncertainty, the scores will be referred to as the utility of the course of action. In such circumstances these scores also incorporate the decision makers attitude to risk.

Example multicriteria problems

Most decisions, no matter how trivial, have a multicriteria element, but what concerns us in this unit are those decisions which matter enough to make it worthwhile engaging in some form of analysis. In the MCDA in practice chapter we will consider a number of published case studies which illustrate a range of situations in which MAVA has been used to inform decision making. For the moment we introduce two illustrative cases which will be used to illustrate the approach; both are concerned with office location and could in practice represent different stages of analysis of the same decision: Office location problem 1: which country? This is about a small company called Decision Aid International wishing to set up a European office. Office location problem 2: which site? This is about a small printing and photocopying concern wishing to relocate to new offices. These two illustrative cases form the basis of several illustrations.

MULTI-ATTRIBUTE VALUE ANALYSIS 12

Office location problem 1: which country? A small company called Decision Aid International, based in Washington DC, USA, provides decision support consultancy to the public and private sector. The company employs five facilitator/analysts who work directly with clients and are supported by a technical team of five software developers and two administrative staff. The managing director believes that there is a new and growing market in Europe for services offered by the company and is looking to open an office there. However, she is unsure of the best location and wishes to explore the multitude of possibilities in a structured manner. Many factors relevant to the decision are highlighted in the following conversation between the managing director (MD) and one of the companys analysts: Analyst: Why do you think it would be advantageous to have a European base? Couldnt the work be handled from Washington? MD: Well .... firstly I think that developments in the European Union and the growth of the market economy in former Eastern European countries mean that there will be substantial new opportunities for companies such as ours in the near future. Although it would be important to involve the US staff in this work and we could get involved to an extent from our current base, I feel it would be essential to have a permanent presence to be close to clients, to be able to react quickly to opportunities and to keep in tune with European cultural issues. Secondly, although occasional trips to Europe would have novelty value for our US employees, too much travel becomes tiring and would undermine morale. Analyst: Given the need to involve US staff in the European work, then presumably you would be looking for a location which is easily accessible? MD: Yes, and one which is reasonably attractive to visit for the US staff. It is also important that it is a location which gives good access to other European countries, particularly as the nature of our work means that we tend to spend one day a week with a client over a period of time. Analyst: Does this restrict you to capital cities, given that they usually have better transport links?

MD: I think we definitely need to be somewhere which has easy access to an international airport, probably with direct flights to the US and a good European network. However, this doesnt necessarily restrict us to capital cities for example, Glasgow or Manchester in the UK, Milan in Italy or Dsseldorf in Germany all have international airports. These cities may be closer to the country's industrial heart, and may have the advantage of attractive financial packages for new companies locating there. Analyst: What type of clients are you most interested in? MD: Currently we have clients in the public sector particularly government and health and the private sector. At the moment our private sector clients are primarily in high-tech manufacturing companies, but we have done work in traditional manufacturing industries and in the service sector. However, our expertise is, of course, equally relevant to all organisations. I think we should be located somewhere which gives good access to a range of potential clients, although I think that the public sector is likely to be most important in the immediate future. Analyst: You mentioned former Eastern European countries earlier would you consider locating in, say, Warsaw? MD: Thats an interesting question. I really do not feel that I have sufficient knowledge to decide. I would have to be sure that suitable office space was available. When we set up here we initially used an office services company which provided secretarial support, photocopying facilities, etc. Ideally I would like to do the same in Europe with the flexibility of moving to employing our own staff and facilities as we expand. This reminds me that the availability of suitably qualified staff both consultants and support staff is, of course, very important. Another issue is the ease of setting up the business. I would

MULTI-ATTRIBUTE VALUE ANALYSIS 13

want to avoid excessive bureaucracy and legalities existing links with the local Chamber of Commerce might be useful. It goes without question that there need to be excellent telecommunications links back to the US. All these comments apply to all potential venues . its just that thinking about somewhere unfamiliar brought them to mind. Analyst: Does thinking about other cities raise concerns in your mind? MD: Funny you should say that, I was just doing a mental check. I have to say that I have some concerns about language. Although I speak French Im not sure which of the consultants do. Might that be a problem if we located in Paris? None of us speaks any other European language. That might suggest that a UK base would be best. Weve hardly touched on financial matters. The high taxes and cost of living in Scandinavian countries is a deterrent, although I have to confess to being attracted to their lifestyle. On the other hand, the food, wine and climate of the southern European countries is rather tempting .... but .... this is meant to be a business decision, isnt it? Where have we got to? At this point, both the analyst and the MD are beginning to develop some common understanding of the problems which have to be addressed, and of some of the issues (criteria) which need to be taken into consideration. They cannot be sure at this stage that all possible locations have been thought of, nor whether all issues of concern have been identified. Nevertheless, there is enough appreciation of what needs to be done, so that the MD and analyst can start the process of structuring and analysing the choices which have to be made. They can probably prepare an initial discussion document, to be tabled at a meeting of the critical stakeholders. We shall return to this structuring phase in the next section.

Office location problem 2: which site? In this example we become involved in the problem at a more advanced stage, when the owner of a small business has identified a number of options available to him, but is unsure how to set about making his choice, given the number of factors involved. The business is a small printing and photocopying concern, which has to move from its existing office because the site has been acquired for redevelopment. The owner of the business is considering seven possible new offices, all of which would be rented. Details of the location of these offices and the annual rent payable are given in the table here. Office location problem 2: site and rent options Location of office Addison Square Bilton Village Carlisle Walk Denver Street Elton Street Filton Village Gorton Square (A) (B) (C) (D) (E) (F) (G) Annual rent ($) 30,000 15,000 5,000 12,000 30,000 15,000 10,000

While the owner would like to keep his costs as low as possible, he would also like to take other factors into account. For example, the Addison Square office is in a prestigious location close to potential customers, but it is expensive to rent. It is also an old, dark building which will not be comfortable for staff to work in. In contrast, the Bilton Village office is a new building which will provide excellent working conditions, but it is several miles from the centre of town where most potential customers are to be found.
2001 Kluwer Academic Publishers

14 ANALYTICAL SUPPORT FOR DECISION MAKING

Outline of the process of multi-attribute value analysis and SMART

As already mentioned, the technique which we will use to analyse the office location problem is based on the simple multi-attribute rating technique (SMART) which was put forward by Edwards in 1971. Because of the simplicity of both the responses required of the decision maker and the manner in which these responses are analysed, SMART has been widely applied. The analysis involved is transparent so that the method is likely to yield an enhanced understanding of the problem and be acceptable to the decision maker who is distrustful of a mathematical black-box approach. This, coupled with the relative ease and speed by which the method can be applied, particularly with supporting software, means that SMART has been found to be a useful vehicle in decision conferences or workshops. These are intensive meetings, often extending over several days, in which a group of decision makers focuses attention on a specific decision problem, usually under the guidance of a facilitator. The cost of this simplicity is that the method may not capture all the detail and complexities of the real problem. Nevertheless, in practice, the approach has been found to be extremely robust (see Watson and Buede, 1987). The overall process was depicted in Figure 1, but for clarity of exposition the main stages in the analysis are listed below. Although each stage is separately identified, in practice, as already illustrated in Figure 1, the process can involve many iterations. In the following sections we will explore in detail the key phases of problem structuring, model building and using the model. Problem identification and structuring Before any analysis can begin, the various stakeholders, including facilitators and technical analysts, need to develop a common understanding of the problem, of the decisions that have to be made, and of the criteria by which such decisions are to be judged and evaluated. The checklist CAUSE criteria, alternatives, uncertainties, stakeholders and environmental factors or constraints can help to ensure that all the important facets of the problem have been considered. Expanding briefly on these aspects in relation to the issue of finding a new location for the photocopying business in the second illustrative case, we have to: Identify the stakeholders. In particular, the decision maker (or decision makers). In the example problem the key stakeholder is the business owner, who we will assume is the decision maker in this case. However, there are other stakeholders, including the staff employed in the business and the customers. The business owner may consider involving his staff in the decision; indeed in

MULTI-ATTRIBUTE VALUE ANALYSIS 15

practice more often than not there are many people involved and a group of decision makers has a shared responsibility. We will discuss different ways of using the approach with groups in the MCDA in practice chapter. In the case of other stakeholders, such as customers, it may be important to consider their perspectives on the decision, for example through the criteria used in the analysis. Identify the alternative courses of action. In our problem these are, of course, represented by the different offices the owner can choose, which were identified by some prior search process. In some cases it is clear what the alternatives are. In others it may be necessary to define the alternatives, a situation in which creative thinking can play an important role. Sometimes the problem is not one of generating alternatives, but one of identifying an appropriate and manageable set of alternatives for evaluation from a much larger set of possibilities a screening process. The appropriate set may be a set of good alternatives, or it may be a set that is representative of the range of possibilities. Identify the criteria relevant to the decision problem. The attributes which distinguish the different offices will be factors like rent, size and quality of working conditions. In the next section we will discuss ways in which we can help the decision maker identify relevant attributes. Identify key uncertainties and external factors. External factors are often a source of uncertainty and so the two are grouped here. Uncertainty may impinge on a decision in many ways, for example, through lack of information about alternatives, not knowing how the future will evolve, not knowing how competitors will react, etc. Factors which the business owner might want to take into account in his decision are expected changes in technology which may change the nature and volume of demand for his services. We will not take explicit account of these issues here, but could extend the modelling in ways mentioned earlier (page 9).

Model building
Structure the criteria as a value tree. The first step in moving from the qualitative phase of problem structuring to the more formal phase of model building is to structure the criteria as a value tree, a process which is discussed in more detail later in the chapter. A value tree is a hierarchical structure that breaks down the decision into progressively greater detail until a level is reached at which it is easy to make comparisons between alternatives. Evaluate the performance of the alternatives with respect to the criteria. For each attribute we need to measure the performance of each of the alternatives with respect to that attribute. For example,

16 ANALYTICAL SUPPORT FOR DECISION MAKING

how well do the offices compare when considering the quality of the working conditions they offer? This part of the process is generally referred to as scoring. There are many possible ways of doing this, as will be discussed later. Assess the relative importance of criteria. This part of the process is generally referred to as weighting. The weight assigned to each criterion should reflect how important it is to the decision maker (though we will discuss the problem of using importance weights later). As with scoring there are many possible ways of weighting criteria, as will be discussed later.

Using the model

Synthesis of values. The first stage of using a model is to determine aggregate performances at each level of the value tree, starting at the bottom and gradually building up to a single overall value, or a few high-level indicators. To do so we consider each family of criteria in turn and for each alternative we take a weighted average of the scores assigned to that alternative on each criterion in the family. This will give us a measure of how well an office performs over all the attributes. In addition to this overall evaluation it is important to reflect on its composition, for example, to look at the profile of an alternatives performance. Does it have certain strong points and other weak ones or is it reasonably strong across all attributes? Are there dominating or dominated alternatives? We look in detail at such analyses in the section on synthesis of values later in this chapter. Sensitivity analysis. It is very important to carry out a thorough sensitivity analysis to see how robust the decision is to changes in the values supplied by the decision maker. As we will see, sensitivity analysis is aided greatly by effective visual display of the relevant information.

Development of an action plan

Analysis does not solve the problem, it informs the decision about what to do next. It may be that the outcome of the analysis is a decision to select or recommend a particular alternative. Or it may be that the outcome of one analysis simply leads on to the next stage, for example, having decided to open an office in Paris, then the next stage may be to consider which area of Paris, or it may be to consider specific locations. However, it may emerge that none of the alternatives considered is felt to be suitable and the next step should be to search for new options.

MULTI-ATTRIBUTE VALUE ANALYSIS 17

Problem structuring and model building

A problem well structured is a problem half solved.

This oft-quoted statement is highly pertinent to the use of any form of modelling you will encounter it in many places on the MBA programme. In the previous section we distinguished three phases of the process of MCDA. In this section we focus on problem structuring, the process of making sense of an issue identifying key concerns, objectives, stakeholders, actions, uncertainties, and so on and the link to model building. The MCDA field has, until relatively recently, paid little attention to the problem structuring phase of the process, focusing more on analytic tools. However, the problem structuring phase is extremely important in the overall process of MCDA and thus deserves serious attention. The process of making sense of the issue may be an informal one. It may be supported by one of a broad range of general managerial tools, such as a SWOT (strengths, weaknesses, opportunities, threats) analysis, or One of the problem structuring methods loosely labelled as soft operational research may be used, for example Journey making (Eden and Ackermann, 1998), Soft systems methodology (Checkland, 1989), and Strategic choice (Friend, 1989). From this understanding it may emerge that more detailed evaluation of options using a formal MCDA approach is appropriate. We give an overview here of aspects of problem structuring in the context of MCDA; this is a summary of the more detailed account which can be found in Chapter 3 of Belton and Stewart (2001). Before going on to consider some ideas for stimulating thinking, let us pause for a moment and consider the context in which this is likely to

18 ANALYTICAL SUPPORT FOR DECISION MAKING

be happening. It may be a group of people working together to explore an issue of common concern, or it may be an individual working alone. They may be helped by one, or more, facilitators; we assume in this discussion that at least one facilitator is involved, although the issues are of equal relevance in the case of unsupported (or DIY that is, do it yourself) work. Facilitators must pay attention not only to ways of stimulating thinking, but also to how to capture the ideas which are generated in a way which will be supportive of the continuing process. Furthermore, when working with a group of people, they need also to be sensitive to group process issues.

Idea generation and capture

By this we mean the process of making explicit and recording notions of relevance to the problem, however defined. In addition to providing a mechanism to allow participants to contribute everything they know and feel about a problem, the idea generation process should encourage the surfacing of tacit knowledge and beliefs, and should foster creative thinking. Facilitating the process of idea generation and capture can be done in many different ways. Different facilitators will prefer different ways of working and different approaches may be required when working with an individual, as opposed to a group. Furthermore, external constraints such as the physical environment, the time available, the availability of computer support etc, may influence the way of working. At one extreme the facilitator may simply engage in a conversation with the decision making group, eliciting ideas, information and values, but not seeking initially to record these in detail. A structured representation of the problem in appropriate form is written down either as the conversation progresses, or to summarise it. This may be a list of key concepts, stakeholders, uncertainties, goals, possible actions or it may be an initial model structure. This way of working captures the essence of the issue, but the detail, although surfaced, is not recorded. It places a lot of emphasis on the skill and memory of the facilitator and is probably not workable for large, complex issues. There is a danger that the representation of the issue is perceived as belonging to the facilitator rather than the group. At the other extreme, the process may be designed to capture as much detail as possible and to fully engage the decision-making group in doing so. One of the most successful and widely used tools to support such a way of working is the humble Post-It (adhesive notelets). More sophisticated variants on this theme include oval mapping (Eden and Ackermann, 1998), or electronic versions of the same.

MULTI-ATTRIBUTE VALUE ANALYSIS 19

Stimulating idea generation

It is usually the case that a large number of ideas emerge from a freethinking process such as a Post-It session, particularly if working with a group. However, it may also be worthwhile for the facilitator to introduce specific activities to focus thinking, in order to ensure that all aspects of the situation have been considered. A number of possible activities are listed below these are described in more detail in Belton and Stewart, (2001). The use of checklists This ensures that key aspects of an issue are not overlooked. Perhaps the most widely known checklist for problem structuring is the one denoted by Kiplings six honest serving men What and Why and When and How and Where and Who. This provides a useful framework for gathering information about a situation and as a basis for idea generation. Checklands CATWOE analysis (customers, actors, transformation, worldview, owners and environment), widely acclaimed amongst systems thinkers, focuses on the nature of the system and the stakeholders involved. Problem structuring for MCDA may also usefully be guided by consideration of CAUSE, as outlined in the previous section criteria, alternatives, uncertainties, stakeholders and environmental factors or constraints. Thinking about specific actions or alternatives Selecting a specific course of action and focusing attention on its strengths, weaknesses and interesting or unusual characteristics (see for example, De Bonos plus minus interesting analysis) can be a useful way of surfacing values. Comparing alternatives can also be helpful. One possibility is to take two randomly selected alternatives, to note how they differ and whether or not those differences would be relevant if you had to choose between them. Along similar lines, the repertory grid approach, as described by Eden and Jones (1984), randomly selects three alternatives, divides them into a pair and a single and asks how the single differs from the pair. Adopting alternative perspectives Participants are asked to imagine themselves in the role of a specific stakeholder in the problem. For example, suppliers might be asked to imagine themselves in the role of purchaser, nurses as patients, teachers as students or parents. They might be asked to think generally about the stakeholders values, or more specifically about

20 ANALYTICAL SUPPORT FOR DECISION MAKING

how they would expect the stakeholder to react if a particular action were pursued, and why they would expect them to behave in that way. Positive and negative reviews Participants are asked to consider, on the one hand, what could go wrong if a particular course of action were followed: What are the potential pitfalls? What are the possible negative consequences? What barriers might be encountered? On the other hand, what are the anticipated benefits? Are there other possible spin-offs? Positive and negative reviews correspond to De Bonos (1990) yellow hat and black hat thinking. He comments that black hat thinking is relatively easy because we are used to behaving cautiously; avoiding danger and mistakes is a natural part of the survival instinct. In contrast yellow hat, or optimistic thinking calls for significant conscious effort. The notion of positive and negative review is also encapsulated in the idea advocate approach, countered by a devils advocate, described by Van Gundy (1988, p232). Barriers and constraints Are there any minimum standards that must be achieved? Are there any limits that must be adhered to? Is there anything which might prevent a particular course of action being implemented? Is opposition anticipated? As mentioned earlier, there are many other aids to thinking. Those discussed above are particularly useful in the context of MCDA, but each intervention is unique and a facilitator should be able to draw on many different ways of prompting thinking, selecting the most appropriate in a given context.

Structuring of ideas
The aim of the structuring process is to identify key areas of concern, to organise ideas in a way which clarifies goals and actions, and to highlight any gaps in the picture. Some structuring can take place as ideas are generated. For example, during a Post-It session as clusters of related ideas begin to form, Post-Its should be moved to a position close to others to which they relate. The process of building clusters

MULTI-ATTRIBUTE VALUE ANALYSIS 21

will stimulate discussion amongst participants, furthering shared understanding. For example: Do similarly worded ideas reflect shared concepts or quite different perspectives? Does a particular idea fit better in one cluster or another? In building clusters it is useful to position the most general concepts at the top of the cluster, cascading down to more specific detail at the bottom. If the focus of the idea generation process was to identify criteria for the evaluation of specified alternatives, then a clustering exercise such as this may be an adequate basis from which to move directly to building a multi-attribute value tree. However it may be worthwhile structuring the ideas in a more formal way, particularly if the initial problem statement was a very general one, or if the issue is a particularly messy one. One way of doing so is to make use of cognitive or cause mapping, which you will encounter elsewhere in the MBA, for example, in the Making Strategy unit. The case study referred to on page 120 (Belton and Ackermann (1997)) describes the integration of cognitive mapping and MCDA in a context of setting strategic priorities.

From problem structuring to model building

At some stage the emphasis must move from problem structuring to model building, the development of a framework for the evaluation of alternatives. Model building should be regarded as a very a dynamic process, informed by and informing the problem structuring process, and interacting with the process of evaluation. It may involve much iteration, search for new alternatives and criteria, discarding, reinstating and redefining of old ones, and further extensive discussion amongst participants. The key elements of the model framework, as highlighted by the CAUSE framework, are recalled here. We will consider in detail the first two elements, comment briefly on the third (as previously mentioned, the modelling of uncertainty is not covered at this point). 1 2 3 The alternatives (options, strategies, action plans) to be evaluated. The model of values (criteria, objectives, goals) against which they will be evaluated. Key stakeholders, their perspective on the decision and how this will be taken into account.

22 ANALYTICAL SUPPORT FOR DECISION MAKING

Key uncertainties, both internal and external, and how these will be modelled.

These elements are, of course, interdependent and cannot be considered in isolation as will become clear in the following discussion.

Identifying alternatives
The alternatives may be relatively few and explicitly defined, as in the evaluation of tenders for a contract, or submissions to a grant awarding body, or they may be infinitely many and implicitly defined, for example, how to best allocate available resources across competing needs. On some occasions the alternatives to be evaluated may appear to be clearly defined, but in other circumstances the definition, or discovery of alternatives may be an integral part of a study. Sometimes the challenge may be to find any suitable alternatives; sometimes it may seem to be impossible to manage the overwhelming complexity of options. Although the rationale for MCDA may appear to be the evaluation of given alternatives, equal emphasis should be given to the potential for creating good alternatives. Pruzan and Bogetoft (1991) describe the process of planning as: ... a search and learning process successively increasing our awareness of our objectives, the alternatives which can be considered, and the relationships between alternatives and objectives. The implication is that alternatives are not givens, but that they evolve. Van der Heijden (1996) talks about option evaluation in the context of scenario planning, saying that: Most work associated with strategic decisions is concerned with redesigning proposals and options such that the upsides are maximised and the downsides are minimised. .... The purpose is not primarily to decide between acceptance or rejection, but to work towards improving the proposal, such that outcomes are as robust as possible over a range of possible futures. If MCDA is to be useful in these contexts design must go hand in hand with evaluation. Combinations of choices Often the alternatives for evaluation are a complex combination of independent choices or actions. For example, an alternative may define a sequence of actions over time, or it may represent a portfolio of investments or activities. Multi-attribute value analysis is designed

MULTI-ATTRIBUTE VALUE ANALYSIS 23

for the evaluation of independently defined alternatives; however, to explicitly enumerate and evaluate all possible combinations of alternatives in such circumstances would be prohibitively time consuming. One way of dealing with this complexity is to identify a small subset of promising alternatives for detailed evaluation using some screening mechanism. There are also more complex approaches to modelling such problems, but the detail of these is beyond the scope of this unit. Framing issues An issue of some concern in defining a set of alternatives for evaluation is the potential influence of phantom alternatives (Farquar and Pratkanis, 1993). A phantom is an alternative which is apparently available, but subsequently turns out not to be. On the one hand, phantom alternatives can help to promote creativity, but research has shown that the inclusion of alternatives with particular characteristics can also significantly affect choice behaviour. For example, the consideration of a phantom which is particularly attractive in some respect can have two effects: the contrast effect, which tends to lead to the lowering of attractiveness of other options on the focal attribute the importance shift, whereby greater weight tends to be given to the focal attribute in deciding between available options. The presence of phantoms can also have disturbing effects if a policy of screening for promising alternatives is adopted. It is unlikely that such effects can be eliminated, but an awareness of their possibility is the first line of defence.

Identifying criteria and building a value tree

The outcome of this stage of the analysis should be an initial family of criteria, or a value tree which captures the decision makers values. An initial candidate set of criteria, together with some sense of hierarchy, should emerge from the problem structuring process as described above. Top-down and bottom-up approaches to building a value tree The approach to problem structuring described above should encompass both top-down and bottom-up thinking, as described by Von Winterfeldt and Edwards (1986) and Buede (1986) in discussion of how to approach the structuring of value trees. The top-down approach tends to be objective led, beginning with a general statement of the overall objectives and expanding these initial values into more detailed concepts which help to

24 ANALYTICAL SUPPORT FOR DECISION MAKING

explain or clarify the former. Expansion should continue until it is felt that the emergent criteria are measurable. In contrast, the bottom-up approach is alternative led, beginning with elicitation of detail stimulated by thinking about the strengths and weaknesses of available alternatives. The top-down and bottom-up approaches reflect Keeneys (1992) value-focused and alternative-focused thinking. It should be noted that there is no one right way of doing things and in practice one perspective should inform the other. We use our two example problems introduced in the previous section to illustrate the different approaches in Examples of top-down and bottom-up approaches (pages 25-26)

MULTI-ATTRIBUTE VALUE ANALYSIS 25

Examples of top-down and bottom-up approaches

Example 1: which country? The problem introduced earlier in this chapter regarding the selection of a European office by Decision Aid International, would involve a relatively small number of stakeholders. The hypothetical discussion between the analyst and managing director already represents a substantial move towards structuring of the problem. It is clear from that discussion that the alternatives are not clearly defined at the outset. In order not to close out options too early, it would be helpful to follow the value-focused thinking approach of Keeney. The analyst might thus facilitate a discussion, or decision workshop involving both staff and management, on the issue of opening a new office. Some of the factors which need to be taken into account have already emerged in the original discussion, but use of any of the various techniques discussed could further enrich this list. For example, the following considerations might emerge from a discussion using the CAUSE framework: Criteria: costs, attractiveness of location, ease of operating, communication links, size of local market, do US staff want to work there. Alternatives: whether or not to open a European office? Why not the Far East? Australia? Which city? Uncertainties: there is some uncertainty about most of the criteria noted above. Stakeholders: company staff (in particular those clear from that discussion that the alternatives are not expected to work at or visit the new office), managers (who need to deal with issues relating to set up-and operation), shareholders (concerned about the financial viability of the company, existing customers concerned about quality of support). External factors: closely linked to the uncertainties noted above nature of the market for services, political environment in different countries, nature of competition.

Relevant factors such as the above and the discussion reported earlier could be captured manually using Post-Its or electronically using software such as Decision Explorer, as illustrated by this spray diagram.

26 ANALYTICAL SUPPORT FOR DECISION MAKING

This would facilitate initial clustering by the analyst, in discussion with the decision makers, leading to the emergence of a more fully structured value tree such as that illustrated here.
Value tree for Decision Aid Internationals location problem

The decision maker in our example problem feels that a good location at a reasonable cost is what he is looking for. If he tries to evaluate the alternatives against the attribute, good location, it may become apparent that there is more than one facet of location. For example, distance from the town centre may be important because of the desire to attract customers and staff, in addition the attractiveness of the immediate surroundings may also have an influence. As a general rule, decomposition of the objectives should continue until a level of detail at which measurement can take place is reached. We start constructing a value tree for our problem by addressing the criteria which represent the general concerns of the decision maker. Initially, the owner identifies two main factors which he decides to call costs and benefits. There is, of course, no restriction on the number of criteria which the decision maker can initially specify (for example, our decision maker might have specified short-term costs, long-term costs, convenience of the move and benefits as his initial attributes). Nor is there any requirement to categorise the main attributes as costs and benefits. In some applications (Wooler and Barclay, 1988) the risk of the options is an initial attribute. Buede and Choisser (1984) describe an engineering design application for the US Defense Communications Agency where the main attributes are the effectiveness of the system (ie factors like quality of performance, survivability in the face of physical attack, etc) and implementation (ie factors like manning, ease of transition from the old system, etc). Having established the main attributes for our business owner, we need to decompose them to a level where they can be assessed: Costs: the owner identifies three main costs that are of concern to him: the annual rent, the cost of electricity (for heating, lighting, operating equipment, etc) and the cost of having the office regularly cleaned.

Once the criteria illustrated in these two figures have been identified, participants engaged in the decision workshop would be encouraged to think creatively about which cities are particularly good as regards each of the criteria in turn. This uses the valuefocused thinking approach. Some of these may eventually need to be rejected, but gradually a shortlist of alternative cities would emerge, agreed by the group to be deserving of further investigation in greater detail. For the purpose of this example to illustrate MAVA, let us suppose that the following seven cities have been identified as being good candidates in terms of some if not all of the criteria: Paris, Brussels, Amsterdam, Berlin, Warsaw, Milan, and London.

Example 2: which site?

In some situations the initial objectives or attributes elicited from the decision maker may be vague (for example, the owner might say that he is looking for the office which will be the best for his business) and time is not available to go through the more detailed problem structuring process described above. A top-down approach gives a relatively fast means of building a value tree, as illustrated here for the second office location example.

MULTI-ATTRIBUTE VALUE ANALYSIS 27

Benefits: similarly, he decides that benefits can be subdivided into potential for improved turnover and staff working conditions. Turnover: the owner thinks that he will have difficulty assessing each offices potential for improving turnover without identifying those attributes which will have an impact on turnover. He considers these attributes to be the closeness of the office to potential customers, the visibility of the site (much business is generated from people who see the office while passing by) and the image of the location (a decaying building in a back street may convey a poor image and lead to loss of business). Working conditions: the owner feels that he will be better able to compare the working conditions of the offices if he decomposes this attribute into size, comfort.

One of the factors that comes to mind is the space available for car parking Filton Village being particularly poor in this regard this is considered to contribute mainly to the assessment of working conditions as there is short-term parking available for use of customers. Taking account of all the factors identified an initial value tree, shown below, is constructed.
Value tree for the office location problem

To ensure that no factors have been overlooked, we encourage the owner to think about the potential sites he has identified, what is good and bad about each of them, and how they differ from each other.

28 ANALYTICAL SUPPORT FOR DECISION MAKING

What makes a good value tree?

Having constructed a value tree, how can we judge whether it is an accurate and useful representation of the decision makers concerns? The following factors need to be considered: Value relevance Understandability Measurability Non-redundancy Judgemental independence/decomposability Balancing completeness and conciseness Operationality Simplicity versus complexity. Value relevance Are the decision makers able to link the concept to their objectives, thereby enabling them to specify preferences which relate directly to the concept? To give a simple example, in comparing models of car, size may have emerged as a criterion. However, it is not clear how size relates to values. It could be that size is important because it relates to the amount of luggage space (which we would like to maximise) or to the number of passengers who can be carried (which should be greater than three) or to the perceived status of the car (the bigger the better). Structuring the problem as a cognitive map would help here by ensuring the linking of the concept to higher level goals. Understandability It is important that decision makers have a shared understanding of concepts to be used in an analysis. The absence of these can lead to confusion and conflict rather than constructive discussion and mutual learning. It is, however, always possible that a misunderstanding does not emerge until later in the process. For example, in evaluating potential house purchases, a couple agree that distance from the station is an important factor, but only when evaluating specific options does it become clear that one person prefers to be close (to minimise the distance walked each day) whilst the other prefers to be more distant (away from the disturbance from noise and commuters cars). This is another illustration of a concept which was not initially linked to values.

MULTI-ATTRIBUTE VALUE ANALYSIS 29

Measurability All MCDA implies some degree of measurement of the performance of alternatives against specified criteria, thus it must be possible to specify this in a consistent manner. It is usual to decompose criteria to a level of detail which allows this. Non-redundancy Is there more than one criterion measuring the same factor? When eliciting ideas often the same concept may arise under different headings. If both are included in the analysis then it is likely that as a consequence the concept will be attributed greater importance. You can check for criteria which appear to be measuring the same thing by calculating a correlation coefficient if appropriate data is available, or carrying out a process of matching as associated with analysis of repertory grids (see Wooler, 1982; Eden and Jones, 1984). Remember that criteria which reflect similar performances over the set of options currently under consideration may not do so in general. As a general rule it is better to combine similar criteria in a single concept. However, there may be good reasons to refer to similar factors, or even the same criterion, in different parts of an analysis as it reflects different values in the different contexts. For instance, Butterworth (1989) describes a multi-attribute value analysis of a banks decision to relocate one of its head office functions. The top level of the value tree was split into staff acceptability and bank acceptability, both of which had as sub-criteria factors relating to unemployment. Not only would the importance of this be likely to be different for the staff and for the bank, but for staff low unemployment is preferable, reflecting concerns about the ease with which other family members could obtain jobs, whilst the opposite is true for the bank, reflecting a desire to be able to recruit new staff without difficulty. Judgemental independence/decomposability Criteria are not judgementally independent if preferences with respect to a single criterion, or trade-offs between two criteria, depend on the level of another. For example, in evaluating alternative job offers someone might feel that ... if I take the job in Greece I would prefer more annual leave to an increased salary, but if I take the job in Alaska I would prefer the salary to the holiday. That is, the tradeoff between salary and leave is dependent on location. It may be possible to overcome judgemental dependency by redefining or regrouping criteria (see, for example, Keeney, 1981). In the example situation, rather than considering salary and leave independently we might try grouping as remuneration package.

30 ANALYTICAL SUPPORT FOR DECISION MAKING

Balancing completeness and conciseness Keeney and Raiffa (1976) note that desirable characteristics of a value tree are that it is complete, ie that all important aspects of the problem are captured, and also that it is concise, keeping the level of detail to the minimum required. Clearly completeness or exhaustiveness and conciseness are potentially conflicting requirements; how can an appropriate balance be achieved? Phillips (1984) concept of a requisite model is a potentially useful one; he defines a model as requisite when no new insights are generated. However, judging when this stage has been reached may not be easy. Operationality Associated with the need to achieve a balance between completeness and conciseness, it is also important that the model is usable with reasonable effort that the information required does not place excessive demands on the decision makers. The context in which the model is being used is clearly important in judging the usability of a model; for example, if the time available is restricted to a one-day workshop then the facilitator must take account of this in guiding the initial specification of a model. Simplicity versus complexity As discussed above, the value tree, or criteria set, is itself a simple representation, capturing the essence of a problem, which has been extracted from a complex problem description. But some representations will be more or less simple than others as a consequence of the degree of detail incorporated and in the nature of the specific structure. The modeller should strive for the simplest tree which adequately captures the problem for the decision maker. However, in practice, the initial representation tends often to be more detailed than either necessary or operationally desirable, and it is only through the process of attempting to use the model that this becomes apparent, leading to further refinement.

Practicalities
Size of tree Sometimes it may be necessary to find compromises between these criteria. For example, to make the tree operational it may be necessary to increase its size, but the cost of this is the increased time required to evaluate alternatives. In situations where it is important to have a well-documented audit trail of a decision, then a detailed tree may be appropriate; where time is limited and the main role of the analysis is to act as a sounding board for intuition, a less detailed tree utilising higher-level concepts may be better suited. We have

MULTI-ATTRIBUTE VALUE ANALYSIS 31

worked successfully with trees with as few as ten criteria and as many as one hundred. However, particularly when working with larger trees, it is important that they are well structured we suggest that if there are more than six criteria in a family (sharing the same parent criterion) then consideration should be given to creating subgroups. Which tree? It is important to be aware that there is probably more than one possible representation of a problem. Often several attempts at formulating a tree may be required before an acceptable structure is arrived at. Brownlow and Watson (1987) discuss this issue with reference to the problem of finding a suitable means of transport for radioactive waste. The tree went through a number of stages of development as new insights were gained into the nature of the problem. The most important factor is that the representation used is one with which the decision making group is comfortable and the process of model structuring has established a sense of ownership and commitment.

Use of generic structures

Whilst it is important that the analyst does not impose a particular structure on a group, the knowledge that there are certain common, recurring structures can be useful in facilitating the process of structuring a hierarchy of criteria. Common generic criteria groupings are: benefits costs (broader than just economic costs) political factors (for example, internal or external pressures) risks particular interest groups. It is also possible to group problems according to type for example, tender evaluation, equipment selection, relocation whereby problems of the same type tend to lead to similar value trees. It may be possible to speed the process of developing the criteria hierarchy for a particular situation by taking one used for a similar problem and modifying it.

Stakeholders
There is a substantial literature on stakeholder identification, analysis and management (see for example, Eden and Ackermann, 1998), which it is impossible to cover here in any depth. However, in a

32 ANALYTICAL SUPPORT FOR DECISION MAKING

multicriteria analysis, as in any form of problem analysis, it is important to recognise the existence and potential impact of both internal and external stakeholders. As a first step the stakeholders must be identified. A dictionary definition (the Chambers Dictionary, 2008 edition) of a stakeholder is someone who has an interest or a stake in something, especially an enterprise, business etc, noting, however, the extension of this definition to the economy or society as a whole. Checklands CATWOE analysis, discussed in the previous section, can act as a useful prompt in identifying stakeholders. However, the extent to which the decision-making group would wish to take account of a particular stakeholders views is likely to differ according to the influence that the stakeholder could exert and, in particular, their power to sabotage a decision. Eden and Ackermann use a two-dimensional grid to classify stakeholders according to their level of interest and power with respect to an issue. They further highlight the importance of not only considering stakeholders in isolation, but also taking into account their propensity to influence other stakeholders as well as the potential for coalitions. Such an analysis is useful in identifying which stakeholders views it is important to consider; there is then the question of how to do so. Another question to be asked is whether it is desirable and/or possible to involve representatives of the relevant stakeholder groups in the decision process. This, of course, will depend on the aim of the process: is it to arrive at a decision which everyone can buy in to? or, is it explicitly to explore different perspectives? Involvement should ensure that all parties views are fully considered and, if the group has a shared sense of purpose, can be the way to develop ownership and commitment. If particular stakeholders are not represented in the decision-making group then role play may be a helpful way of encouraging consideration of an issue from different perspectives. The group, or subgroups, can be asked to think generally about the issue from a perspective other than their own, or they might be asked to anticipate how a particular stakeholder would react to a specific decision. Role play can also be useful in a supportive environment in building understanding of the perspectives of other group members. The enforced consideration of different perspectives serves both to encourage more open thinking about the problem and as a means of anticipating the reactions of stakeholder groups.

MULTI-ATTRIBUTE VALUE ANALYSIS 33

These processes serve to surface issues relating to different stakeholders and to encourage the broader consideration of an issue. Following on from this is the question of how this might then be incorporated in the multicriteria modelling. A number of approaches have been adopted. In the context of multi-attribute value analysis two approaches can be distinguished by whether or not the model structure, the value tree, is shared (whether it is the same for all stakeholders or different for each stakeholder): In the case of a shared model the same criteria and structure are adopted by everyone, but different stakeholders have the opportunity to express their individual preferences through the values entered into the model (Brownlow and Watson, 1987). A different model for each stakeholder may be appropriate if they have very different concerns, manifested in different criteria sets. Case study 3.5 contained in Chapter 3 (Butterworth, 1989) is an example of such an analysis. The decision under consideration was the relocation of a department of the Bank of England; the analysis models separately the interests of the management of the bank and the bank staff, combining these at the top level of the value tree.

Summary of problem structuring and model building

In this section we have explored how to move from an initial concern about an issue, or a loosely specified decision problem, to a wellstructured model of the problem in the form of a value tree, which will form the basis of a multi-attribute value analysis. We have illustrated this process using the two example problems introduced earlier in this chapter. In the next section we progress to a detailed evaluation of the identified alternatives using MAVA. First we explore ways of eliciting and representing the decision makers values and judgements alongside objective information. We will then synthesise this information step by step, making extensive use of visual displays and sensitivity analysis to help the decision maker identify the preferred alternative. Before proceeding to the next section, we suggest you attempt the simple Exercises on problem structuring and model building on which follow.

34 ANALYTICAL SUPPORT FOR DECISION MAKING

Exercises on problem structuring and model building

Buying a car Imagine that you have just inherited 15,000 under the condition that you use it to purchase a new car for yourself or your family (you do not have to spend all the money and are at liberty to supplement it if you wish). (a) Try out for yourself some of the techniques for idea generation described in this section as suggested below. Plus minus interesting Consider each of the following cars in turn and list the plus, minus and interesting points that come to mind for each: Ford Ka Toyota RAV (a sporty four-wheel drive) Porsche a BMW Series 5 (Renault Clio and Ford Focus) vs Rover 25 Now choose another set of three cars and consider each of the pairs against the other car. Note If choosing a car does not interest you, then try to identify a different decision situation with which you feel more comfortable and ask yourself the above questions with respect to appropriate alternatives. The following are possibilities: deciding on a holiday type/location purchasing a house which city/town would you prefer to live in? if you had to make an award for the best film youve seen in the past 12 months (book youve read or play youve seen, etc), which would it be?

(If you are not familiar with these particular cars, think about other models you do know something about.) Pairwise comparisons Consider the following pairs of cars and note any similarities and differences: Ford Ka vs Porsche Renault Clio vs Rover 25 Renault Espace vs Land Rover Vauxhall Omega vs BMW Series 5

(b) Are there any important criteria that are not included in the list of attributes that have been generated using the above approaches? Think carefully about this for a while and if any come to mind add them to your list. Now that you have a comprehensive list of attributes, try structuring these as a value tree. Begin by clustering them in groups of related concepts. Make sure that all the attributes at the bottom level of the tree are concepts on which you are able to assess the relative performance of the alternatives. If not, consider breaking these down to a further level of detail. (c) When you have finished generating ideas and structuring the value tree, spend some time reflecting on whether or not you felt these techniques helped you. Did you find them easy or difficult to use?

Triad approach Consider the following triads and try to identify ways in which the two paired cars are similar but different from the third car: (Renault Clio and Rover 25) vs Ford Focus Renault Clio vs (Rover 25 and Ford Focus)

MULTI-ATTRIBUTE VALUE ANALYSIS 35

The top-down approach

Now try developing a value tree using the top-down approach. Choose another of the examples from the list above, or use any decision which is familiar to you. How does this value tree compare with the one elicited in the previous exercise? Is it more or less detailed?

write them down (as described above you will need to provide each individual with a pen and a supply of cards or Post-It pads), then consider all the ideas together in structuring a value tree. This should lead to a lot of discussion about what was meant by specific concepts, whether what one person described as speed is the same as what someone else described as acceleration (if evaluating cars), and so on.

Idea generation
Now it is time to involve someone else in your learning (if they can be persuaded)! Think of a decision which could involve other people as well as yourself this could be a family decision, a group of friends, or a group of colleagues. Try using the techniques for idea generation in a group context. First let the individuals generate their own ideas and

Benefits of and barriers to idea generation

Reflect on whether or not you feel this process of idea generation might facilitate problem structuring and decision making in your workplace. What benefits might it bring over the way such tasks are approached at the moment? What barriers to use might you expect to encounter?

36 ANALYTICAL SUPPORT FOR DECISION MAKING

Eliciting values scoring and weighting

Introduction to scoring and weighting

In the previous section we looked at problem structuring, the process of making sense of an issue, and from this initial understanding deriving the structure necessary to progress with multi-attribute value analysis. We now have available a set of alternatives for evaluation and a value tree which captures the decision makers objectives in an understandable and useable form. The next stage of the analysis is to bring together objective information and judgement relating to the performance of the alternatives against the criteria captured in the value tree the process referred to as scoring. It is unlikely, however, that the decision makers value all the criteria to the same extent; these judgements are captured in the process known as weighting. We illustrate the processes of scoring and weighting by reference to the example problem of choosing an office location for the photocopying business, the second of the two illustrative cases introduced earlier in this chapter. You can follow the analysis in VISA loading the example OFFICE-V4W. (Before you do this, it may be helpful to work through the Tutorial Example provided). A parallel illustration of the process for the other example, deciding on the location of a European office for Decision Aid International, can be found in Chapter 5 of Belton and Stewart (2001).

Scoring measuring how the options perform on each attribute

Having developed a value tree which captures all the attributes relevant to the decision maker(s), the next step is to find out how each of the alternatives performs on each of the lowest level attributes of the tree. Thus in our example case we need to identify how the seven offices perform on each of the nine lowest level attributes. Before looking in detail at the example, we give a brief overview of the scoring process.

MULTI-ATTRIBUTE VALUE ANALYSIS 37

Overview of the scoring process The multi-attribute value function approach assumes that it is possible to measure preferences on an interval scale.
The Fahrenheit and Celsius temperature scales are the most wellknown examples of interval scales. We cannot, for example, say that water at 80 degrees Celsius is twice the temperature of water at 40 degrees Celsius. You can verify this by converting the temperatures to degrees Fahrenheit to obtain 175 degrees Fahrenheit and 104 degrees Fahrenheit respectively. Clearly the first temperature is no longer twice the second temperature. However, we can say that an increase in temperature from 40 to 80 degrees Celsius is twice that of an increase from 40 to 60 degrees Celsius. You will find that such a comparison does apply even if we convert the temperatures to degrees Fahrenheit.

This implies that it makes sense to talk about the difference in preference between alternative A and alternative B according to a specific criterion, but not about the ratio of preference. It does not make sense to say, for example, that A is three times as preferred as B. To construct an interval scale it is necessary to define two reference points and to allocate numerical values to these points. These are often taken to be the bottom and top of the scale, to which are assigned values such as 0 and 100 (which will be used to illustrate our discussion and are also used in the software which accompanies this unit), but other reference points (and other values) can be used. The minimum and maximum points on the scale can be defined in a number of ways, but it is useful to distinguish between a local scale and a global scale, as described below. A local scale is defined by the set of alternatives under consideration. The alternative which does best on a particular criterion is assigned a score of 100 and the one which does least well is assigned a score of 0. All other alternatives will receive intermediate scores which reflect their performance relative to these two end points. The use of local scales permits a relatively quick assessment of values and can be very useful for an initial roughing out of a problem, or if operating under tight time constraints. A global scale is defined by reference to the wider set of possibilities. The end points may be defined by the ideal and the worst conceivable performance on the particular criterion, or by the best and worst performance which could realistically occur. The definition of a global scale requires more work than a local

38 ANALYTICAL SUPPORT FOR DECISION MAKING

scale. However, it has the advantages that it is more general than a local scale and that it can be defined before consideration of specific alternatives. This means also that it is possible to define criteria weights before consideration of alternatives, as will be discussed later in the chapter. Once the reference points of the scale have been determined consideration must be given to how other scores are to be assessed. This can be done in the following three ways: Direct rating of the alternatives. In this case, no attempt is made to define a scale which characterises performance independently of the alternatives being evaluated. The decision maker simply specifies a number, or identifies the position on a visual analogue scale, which reflects the value of an alternative in relation to the specified reference points. Definition of a partial value function. This relates value to performance in terms of a measurable attribute reflecting the criterion of interest. Construction of a qualitative value scale. In this case, the performance of alternatives can be assessed by reference to descriptive pointers, or word models (to which appropriate values are assigned). In Illustration of scoring methods on pages 39-43, we will illustrate each of these approaches by reference to the benefits branch of the value tree in the example case relating to choice of office site (see page 27). Read this before going on to Assessing costs. A similar discussion relating to the Decision Aid International example can be found in Chapter 5 of Belton and Stewart (2001).

MULTI-ATTRIBUTE VALUE ANALYSIS 39

Illustration of scoring methods

Direct rating, value functions, and qualitative scale Illustration of scoring methods We look first at the direct rating method as this is most easily applied and therefore most often used. Direct rating Let us first consider those attributes which cannot be represented by easily quantifiable variables, starting with the attribute image. The owner is first asked to rank the locations in terms of their image from the most preferred to the least preferred. His rankings are: 1 2 3 4 5 6 7 Addison Square Elton Street Filton Village Denver Street Gorton Square Bilton Village Carlisle Walk

Value scale for office image

The values allocated show that the improvement in image between Carlisle Walk and Gorton Square is perceived by the owner to be twice as preferable as the improvement in image between Carlisle Walk and Bilton Village. Similarly, the improvement in image between Carlisle Walk and Addison Square is seen to be ten times more preferable than the improvement between Carlisle Walk and Bilton Village. Remember it is the interval (or improvement) between the points in the scale which we compare. We cannot say that the image of Gorton Square is twice as preferable as the image of the Bilton Village office. Although no attempt is made to relate performance to a measurable scale, the positioning of alternatives can generate extensive discussion, yielding rich information on the decision makers values. Ideally this information should be recorded for future reference. Having established an initial set of values for image, these should be checked to see if they consistently represent the preferences of the decision maker. We can achieve this by asking him, for example: Is he happy that the improvement in image between Elton Street and Addison Square is roughly as preferable as the improvement in image between Gorton Square and Denver Street? Is he happy that the improvement in image between Carlisle Walk and Denver Street is less preferable than that between Denver Street and Elton Street?

Using a local scale, Addison Square, the best location for image, can be given a value for image of 100 and Carlisle Walk, the location with the least appealing image, can be given a value of 0 (note that this does not mean necessarily that it is bad in an absolute sense). Once the end points of the scale are established we can begin to assess all the other alternatives. The owner is asked to rate the other locations in such a way that the difference between the values he gives to the offices represents his strength of preference for one office over another in terms of image. He might do this by directly assigning numerical values, or using a visual representation of the scale as illustrated above.

40 ANALYTICAL SUPPORT FOR DECISION MAKING

The answers to these questions may lead to a revision of the values. Of course, if the owner finds it very difficult to make this sort of judgement we may need to return to the value tree and see if we can break image down into more measurable attributes. Nevertheless, it should be emphasised that the numbers allocated by the owner to the different offices do not need to be precise. As we will see later, the choice of a course of action is generally fairly robust and it often requires quite substantial changes in the figures supplied by the decision maker before another option is preferred. This procedure for obtaining values can be repeated for the other less easily quantified attributes. The values allocated by the owner for the attributes comfort, and car parking facilities are illustrated in the figure below and shown in the main body of the table on page 43. Ignore for now the column labelled weight and the row labelled benefits.
Value scales for comfort and car-parking

Office location problem 2: floor area

Floor area (square feet) Addison Square (A) Bilton Village (B) Carlisle Walk (C) Denver Street (D) Elton Street (E) Filton Village (F) Gorton Square (G) 1,000 550 400 800 1,500 400 700

The first thing to consider is whether strength of preference with respect to size is increasing, decreasing, or otherwise related to floor area. In this case a larger office is judged to be more attractive than a smaller one across all possible sizes. Using a local scale we would therefore allocate a value of 0 to a floor area of 400 square feet and a value of 100 to a floor area of 1,500 square feet. In mathematical notation we can say that: v(1500) = 100 where v(1500) means the value of 1,500 square feet. Similarly, the smallest offices (Carlisle Walk and Filton Village) both have areas of 400 square feet so we can attach a value of 0 to this area ie v(400) = 0. The next figure shows the relationship between floor area measured in square feet and value if a linear relationship between the two is assumed between these two extreme values.
Linear value function for size

Value functions Let us now consider the benefit attributes which can be represented by easily quantified variables. Thus the first step in defining a value function is to identify a measurable attribute scale which is closely related to the decision makers values. For example, the size of the office is easily measured by floor area. We need then to measure the owners value of, or relative strength of preference for, offices having different floor areas. The floor area of the offices is shown in the following table.

MULTI-ATTRIBUTE VALUE ANALYSIS 41

However, what is the decision makers view about intermediate sizes? It may be that an increase in area from 500 square feet to 1,000 square feet is very attractive to the owner because this would considerably improve working conditions. However, the improvements to be gained from an increase from 1,000 to 1,500 square feet might be marginal and make this increase less attractive. Because of this, we need to translate the floor areas into values. There are several methods which can be used to elicit a value function, but one of the most widely applied is the bisection method. Bisection method This method requires the owner to identify an office area whose value is halfway between the least preferred area (400 square feet) and the most preferred area (1,500 square feet). Note that this area does not necessarily have to correspond with the area of one of the offices under consideration. We are simply trying to elicit the owners preferences for office areas in general and having obtained this information we can then use it to assess his preference for the specific office areas which are available to him. Initially, the owner suggests that the midpoint area would be 1,000 square feet. This implies that an increase in area from 400 to 1,000 square feet is just as attractive as an increase from 1,000 to 1,500 square feet. However, after some thought, he rejects this value. The increases from smaller areas will, he reasons, reduce overcrowding and so be much more attractive than increases from larger areas which would only lead to minor improvements. He is then offered other candidates for the midpoint position, for example 900 square feet and 600 square feet, but rejects these values as well. Finally, he agrees that 700 square feet has the midpoint value, so v(700) = 50, as illustrated. Having identified the midpoint value, the decision maker is now asked to identify the quarter points. The first of these will be the office area which has a value halfway between the least preferred area (400 square feet) and the midpoint area (700 square feet). He decides that this is 500 square feet, so v(500) = 25. Similarly, we ask him to identify an area

Non-linear value function for size stage 1

which has a value halfway between the midpoint area (700 square feet) and the best area (1,500 square feet). He judges this to be 1,000 square feet which implies that v(1000) = 75. We now have the values for five floor areas and this enables us to plot the value function for office size which is shown here.
Non-linear value function for size stage 2

This value function can be used to estimate the values for the actual areas of the offices under consideration. For example, the Bilton Village office has an area of 550 square feet and the curve suggests that the value of this area is about 30. The dashed lines in the figure show the floor areas and corresponding values for each of the seven offices. A similar method can be applied to the attribute closeness to customers. This attribute has been

42 ANALYTICAL SUPPORT FOR DECISION MAKING Non-linear value functions for closeness to customers

alternatives). In such circumstances it is possible to construct an appropriate qualitative scale. As discussed above, it is necessary to define at least two points on the scale (often taken as the end points). Intermediate points may also be defined as few or as many as desired. An example of such a scale in regular use is the well-known Beaufort scale for measuring the strength of wind, shown in the table below. Points on the scale are defined descriptively and draw on multiple concepts in the definition. (Note: the points on the scale are also defined in terms of actual wind speed.) An alternative approach to defining a scale could be to associate specific alternatives, with which the decision makers are familiar, with points on the scale. Beaufort scale for wind strength 1 2 3 4 5 6 7 8 9 Calm, sea like a mirror. Light air, ripples only. Light breeze, small wavelets (0.2m). Crests have a glassy appearance. Gentle breeze, large wavelets (0.6m), crests begin to break. Moderate breeze, small waves (1m), some white horses. Fresh breeze, moderate waves (1.8m), many white horses. Strong breeze, large waves (3m), probably some spray. Near gale, mounting sea (4m) with foam blown in streaks downwind. Gale, moderately high waves (5.5m), crests break into spindrift.

represented by the variable distance from town centre and the value function is shown in the figure above. Note that the greater the distance from the town centre the lower the value will be. The curve also suggests that a move from up to two miles from the town centre is far more damaging to business than a move from six to eight miles. The values identified for the seven offices in terms of office area and closeness to customers are shown in the table on the opposite page. Other ways of assessing value functions, using direct questioning procedures, have been suggested and some are described in Chapter 5 of Belton and Stewart (2001). Also, it may be possible for the decision maker simply to sketch the shape of the value function, but in this case it is advisable to use questions such as those above as cross checks. The definition of a value function can be more time consuming than the direct rating approach described above if evaluating only a few alternatives, but if many alternatives are to be evaluated or the model is one which is going to be used on an ongoing basis then it can be more efficient to define a function. Constructing a qualitative value scale Often it is not possible to find a measurable attribute which captures a criterion and direct rating may be inappropriate or impractical (for example, too many

10 Strong gale, high waves (7m), dense foam, visibility affected. 11 Storm, very high waves (9m), heavy sea roll, visibility impaired. Surface generally white. 12 Violent storm, exceptionally high waves (11m), visibility poor. 13 Hurricane, 14m waves, air filled with foam and spray, visibility bad.

MULTI-ATTRIBUTE VALUE ANALYSIS 43

Qualitative scales characteristics:

should

have

the

following

Score descriptor 0 In a back alley with very few passers-by

Operational: allow the decision makers to rate alternatives not used in the definition of the scale Reliable: two independent ratings of alternative should lead to the same score an

20 On a quiet street and not easily visible to passers-by 40 On a quiet street but with prominent shop front 60 Reasonably busy street and visibility to passers-by 80 On a busy main street, but not at ground level so less visible to passers-by 100 On a busy main street with lots of motor and pedestrian traffic, ground level and open shop front. The values assigned to each of the offices are shown in the table below. This completes the evaluation of the benefits of the seven offices and illustrates three scoring methods.

Value relevant: relates to the decision makers objective Justifiable: an independent observer could be convinced that the scale is reasonable.

In our example case we might define the following qualitative scale to assess visibility, which according to the owner, is a factor which combines the volume of passing traffic and the ease with which the shop is noticed:

Attribute

Weight A B 20 80 10 30 100 30 39.4 C 80 70 0 0 10 100 47.4 32 26 23 10 6 3 100 60 100 75 0 90 80.8

Office D 70 50 30 55 30 90 52.3 E 40 60 90 100 60 70 64.8 F 0 0 70 0 80 0 20.9 G 60 100 20 50 50 80 60.2

Closeness to customers Visibility Image Size Comfort Car parking facilities Benefits

44 ANALYTICAL SUPPORT FOR DECISION MAKING

Assessing costs
Determining the annual costs of operating the offices is relatively straightforward. The owner already knows the annual rent and he is able to obtain estimates of cleaning and electricity costs from companies which supply these services. Details of all of these costs are given below.

Costs associated with seven offices

Office Annual rent ($) 30,000 15,000 5,000 12,000 30,000 15,000 10,000 Annual cleaning costs ($) 3,000 2,000 1,000 1,000 2,500 1,000 1,100 Annual electricity costs ($) 2,000 800 700 1,100 2,300 600 900 Total cost ($) 35,000 17,800 6,700 14,100 34,800 16,600 12,000

Addison Square Bilton Village Carlisle Walk Denver Street Elton Street Filton Village Gorton Square

As all costs are annual and measured in the same units ($) then it is reasonable and sensible to combine these to give a total annual cost, as shown above. Note, however, that if costs are a combination of oneoff capital costs, recurrent costs, and possibly qualitative elements such as hassle, then simply adding them together may not be appropriate and the relative importance of these factors will have to be considered, as will be discussed in the next section. In order to consider costs alongside benefits in the VISA software, we need to translate the monetary value to a value scale using one of the approaches described above. The availability of the quantitative information makes it easy to define a value function. Given that lower costs are preferred the value function will decrease with increasing costs. Using a local scale again, we set v(35,000)=0 and v(6,700)=100 to give the value function shown in this figure. Although it could be the case that monetary costs and preferences are not linearly related, the business owner here is comfortable that, over the range of costs being considered, his preferences are linear.

MULTI-ATTRIBUTE VALUE ANALYSIS 45

Determining criteria weights

In the last section we derived values to measure how well each office performed on each attribute. For example, we found that Addison Square was the best office for image and closeness to customers, but it was the least preferred office for providing comfortable working conditions for staff. Clearly, in order to make a decision, the owner now needs to combine the values for the different attributes in order to gain a view of the overall benefits which each office has to offer. An intuitively appealing way of achieving this is to attach weights to each of the attributes which reflect their importance to the decision maker. However, this aspect of multicriteria analysis has been the focus of extensive debate. It is clear that decision makers are able and willing to respond to questions such as: What is more important to you in choosing a car, safety or comfort? Furthermore, they are able and willing to respond to questions asking them to rate the relative importance of safety and comfort against a numerical or verbal scale. However, it has been argued by many that the responses to such questions are essentially meaningless. The questions are open to many different interpretations, people do not respond to them in any consistent manner and responses do not relate to the way in which weights are used in the synthesis of information. One of the commonest errors in naive scoring models is to assume that weights, or importance, are independent of the measurement scales used; it is clear, however, that the two are intimately connected. To illustrate this, read Problems in determining criteria weights (on page 83) before continuing further, and consider the following problems: consultants performance problem expenditure versus time saving problem.

46 ANALYTICAL SUPPORT FOR DECISION MAKING

Problems in determining criteria weights

Consultants performance problem Firstly consider a simple example, where no scaling of attributes has taken place. Suppose that the performance of a consultant is measured by income earned and new projects over a specified time period. The table below shows the performance of three consultants: Consultant A B C Income ($1,000,000) 0.5 0.7 0.95 New projects 5 6 4

Expenditure versus time saving problem A civil engineering company is anxious to complete a major project for an important client as quickly as possible. By spending more on the project (for example, by hiring extra equipment and labour) time can be saved and the company has to decide how much extra to spend. This extra expenditure can range from $0 to $25 million and the resulting time saved can range from 0 to 200 days. The project leader considers that days saved is four times more important than the extra expenditure this would require. Suppose that subsequently it is found that, for technical reasons, the number of days saved can only range from zero to two, though $25 million would still be required to make the maximum twoday saving. Almost certainly, the project leader should now change his weights otherwise he is implying that a mere two-day saving is still four times more important than expenditure of $25 million. As this example illustrates, it is important to take account of the range between worst and best alternatives in other words, to recognise what 100 points equates to on each scale. However, there is no clear evidence from research that people naturally take the range into account when assigning importance weights. In fact, there is evidence to the contrary (see von Winterfeldt and Edwards, 1986). As we see, multi-attribute value theory uses special techniques of weight elicitation to try to overcome the problem.

The director of the company thinks that new projects are twice as important as income and thus determines an overall performance measure by the formula: Performance = Income + 2 x New projects giving a score of 10.5, 12.7 and 4.95 to A, B and C respectively. That is, the score is dominated by the number of new projects and thus B is preferred. However, suppose income is measured in thousands of $ rather than millions, then the scores become: A = 500 + 2 x 5 = 510 B = 700 + 2 x 6 = 712 C = 950 + 2 x 4 = 958 Now the score is dominated by the income and C is preferred. We might choose to use any other units for income for example, converting dollars to euros and we might get a different preference order again. Clearly, the concept of importance must be related to the units of measurement. In the SMART approach, as we have seen, it is common practice to convert performance measures to a 0 to 100 scale where 100 denotes the best performance on an attribute of the alternatives considered and 0 denotes the worst. So does this get over the problem just described? Unfortunately not, as the following problem illustrates.

MULTI-ATTRIBUTE VALUE ANALYSIS 47

Use of swing weights in MAVA

The weights which are used to reflect the relative importance of criteria in a multi-attribute value function are, however, well defined. The weight assigned to a criterion is essentially a scaling factor which relates scores on that criterion to scores on all other criteria. Thus if criterion A has a weight which is twice that of criterion B then the decision maker is said to value 10 value points on criterion A the same as 20 value points on criterion B. This defines the trade-off which the decision maker is willing to accept between any two criteria. For example, the office owner may be able to specify that he would be willing to accept a reduction of 10 points on the image scale if the closeness to customers could be increased by five points. These weights are often referred to as swing weights (to distinguish them from the less well-defined notion of importance weights). The concept of swing weights captures both the psychological concept of importance and the extent to which the measurement scale adopted in practice discriminates between alternatives (ie how big the range is between the most preferred and least preferred options on each scale this difference being what defines 100 points). One of the best ways to elicit weights, which forces decision makers to directly confront the issue of trade-offs, is to utilise these swing weights. They are derived by asking the decision maker to compare a change (or swing) from the least preferred to the most preferred value on one attribute to a similar change in another attribute. The simplest approach is to proceed as follows. Consider the lowest level attributes on the benefits branch of the value tree (page 27). The owner is asked to imagine a hypothetical office with all of these attributes at their least preferred levels, that is an office which is the greatest distance (eight miles) from the town centre, has the worst position for visibility, the worst image, the smallest size and so on. Then he is asked, if just one of these attributes could be moved to its best level, which would he choose? The owner selects closeness to customers. This attribute should receive the highest weight. After this change has been made, he is asked which attribute he would next choose to move to its best level, and so on until all of the attributes have been ranked. The owners rankings are shown below. 1 2 3 4 5 6 Closeness to customers Visibility Image Size Comfort Car parking facilities.

48 ANALYTICAL SUPPORT FOR DECISION MAKING

We now give closeness to customers a weight of 100 and the other weights are assessed as follows. The owner is asked to compare a swing from the least visible location to the most visible, with a swing from the most distant location from customers to the closest location. After some thought, he decides that the swing in visibility is 80% as valuable, or important, as the swing in closeness to customers so visibility is given a weight of 80. Another way of expressing this is that the increase in overall value arising from a swing from 0 to 100 points on the visibility scale is worth 80% of a swing from 0 to 100 on the closeness scale.

Similarly, a swing from the worst image to the best is considered to be 70% as important as a swing from the worst to the best location for closeness to customers so image is assigned a weight of 70. The procedure is repeated for all of the other lower level attributes and the figure above illustrates the results, which are shown entered into the VISA software in the next figure (Swing weights for benefit attributes). Within the software it is possible to work with or without the numerical values of the weights displayed; given the subjective and imprecise nature of weights it may be easier for decision makers to work with just the visual display.

MULTI-ATTRIBUTE VALUE ANALYSIS 49

As shown in the following table, the six weights obtained sum to 310 and it is conventional to normalise them so that they add up to 100 or to 1 (this will make later stages of the analysis easier to understand). Normalisation to sum to 100 is achieved by simply dividing each weight by the sum of the weights (310) and multiplying by 100. This can be done automatically in the software by clicking on the button = 1 and the display will be updated. Even without renormalising the values can be seen by selecting the option to include cumulative weights from the Settings menu.
Attribute Closeness to customers Visibility Image Size Comfort Car parking facilities Total Original weights 100 80 70 30 20 10 310 Normalised weights (to nearest whole number) 32 26 23 10 6 3 100

The weights for the higher level criteria in the value tree, namely turnover and working conditions, are now found by summing the appropriate lower level weights, so the weight for turnover is 81 (ie 32 + 26 + 23) and the weight for working conditions is 19 (ie 10 + 6 + 3). The equivalent values are reproduced in the table at the bottom of page 43. Note also that on the value tree shown in the value tree above, the weights shown below the branches of the tree are normalised to sum

50 ANALYTICAL SUPPORT FOR DECISION MAKING

to 1 within their families (ie the sum of the weights of all criteria sharing a parent add to 1). For example, within turnover, closeness to customers is allocated 40% of the weight assigned to turnover, visibility 32% and image 28%. With larger value trees it is easier to work within families of criteria when allocating weights rather than across the whole tree as we did here. With only six bottom-level attributes to think about it was not too difficult to determine swing weights for all of them simultaneously but it is more of a problem if working with many factors. Working within families allows the decision maker to focus on closely related attributes. Of course, it is important also to do some across-family consistency checks.

Summary of scoring and weighting

In this section we have focused on the elicitation of the decision makers values which are essentially the data for our multi-attribute value model. Although it may be the case that objectively measured information is available regarding the performance of alternatives with respect to some criteria, it is important to remember that what we are actually concerned about is the extent to which this objective performance impacts on the decision makers values. In order to elicit values we broke the problem down into small manageable chunks, focusing on one criterion at a time. The next step is to synthesise all these judgements to give an overall assessment and comparison of alternatives.

MULTI-ATTRIBUTE VALUE ANALYSIS 51

Synthesis of values: aggregating the benets using the additive model

a measure of how well each office performs on each benefit attribute and weights which enable us to compare the values allocated to one attribute with the values allocated to the others.

We now have:

This means that we are now in position to find out how well each office performs overall with respect to benefits by combining the six value scores allocated to that office. This constitutes a synthesis of values. To do this we will assume that an additive model is appropriate. As we show below, this simply involves adding an offices weighted value scores together to obtain a measure of the overall benefits which that office has to offer. The additive model is by far the most widely used, but it is not suitable for all circumstances. In particular, the model is inappropriate where there is an interaction between the values associated with some of the attributes. For example, when choosing a house, an attractive architecture and a pleasant garden may complement each other, leading to a combined value which is greater than the sum of the individual values. We will examine the limitations of the additive model later. The calculations which the additive model involves for Addison Square are shown in the following table. Each value is multiplied by the weight attached to that attribute. The resulting products are then summed and divided by 100 to obtain the overall value of benefits at that location (on a scale of 0 to 100).

52 ANALYTICAL SUPPORT FOR DECISION MAKING

Attribute for Addison Square Closeness to customers Visibility Image Size Comfort Car parking facilities

Value 100 60 100 75 0 90

Weight 32 26 23 10 6 3

Value x weight 3200 1560 2300 750 0 270 8080

Therefore the aggregate value for benefit for Addison Square is 8080/100 = 80.8. The table on page 43 (in the bottom line) gives a summary of the values obtained for all the offices and their aggregate values. It can be seen that Addison Square has the highest value for benefits and Filton Village the lowest. These results are shown visually in bar-chart and thermometer form in these figures. However, the determination of an overall value should by no means be viewed as the end of the analysis, but simply another step in furthering understanding and promoting discussion about the problem. Although the underlying model is simple and static, that should not be a limitation in its use. It provides a powerful vehicle for reflecting back to decision makers the information they have provided, the judgements they have made, and an initial attempt at synthesising these. The extent to which the model will be a successful catalyst for discussion of the problem and for learning about your own values and those of others depends on the effectiveness with which feedback can be provided. Simple, static visual displays such as those seen in the figures above are an effective means of reflecting back information provided and well designed visual interactive interfaces provide a powerful vehicle for exploring the implications of uncertainty about values. In exploring the model, decision makers should test the overall evaluation and partial aggregations of information against their intuitive judgement. Are the results in keeping with intuition? If not, why not?

MULTI-ATTRIBUTE VALUE ANALYSIS 53

Could some values have been wrongly assessed? Is there an aspect of performance which is not captured in the model? Is the additive model inappropriate? Or does the model cause the decision maker to revise their intuitive judgement? Performance profiles Decision makers should look not only at the overall evaluation of alternatives, but at their profiles, illustrated for the office example in the first figure below. How is an alternatives overall value made up? Is it a good all rounder or does it have certain strengths and weaknesses? Alternatives with similar overall scores can have very different profiles. Are there any dominating, or dominated alternatives? In simple terms, if there is an option which dominates all others it should be preferred, or if an option is dominated by another it should not be a candidate for choice. However, rather than acting as rigid guidelines, these concepts should be used as catalysts for further thought and learning about the problem situation. A quick glance at the figures below (Profiles of performance on benefit attributes) does not reveal any clearly dominating or dominated alternatives:

Although Addison Square has the highest aggregate benefit score and performs very well on most attributes, it does have a significant weakness in its performance on comfort. Elton Street has the next highest aggregate benefit score; it is the preferred alternative on only one attribute, size, but it does not have any very significant weaknesses, scoring 60 or above on all attributes.

54 ANALYTICAL SUPPORT FOR DECISION MAKING

Gorton Square, with the next highest aggregate score is similarly a reasonable all-rounder, but with three scores around 50 or lower. Elton Street and Gorton Square both outperform Addison Square on comfort and one other attribute, an indication that the overall preference ordering may be sensitive to the criteria weights. Profiles of performance at the higher level of the tree are shown to the right. At this level of aggregation Bilton Village, Carlisle Walk, Denver Street and Filton Village are all dominated by one or more of Addison Square, Elton Street and Gorton Square. Of these latter three, Addison Square and Elton Street both look stronger than Gorton Square although neither quite dominates it. At this point we would probably want to carry out some sensitivity and robustness analyses of the evaluations so far, but for clarity of presentation we will postpone this to later in the chapter, after we have looked at bringing costs into our analysis. Trading benefits against costs Until now we have ignored the costs of the offices because of the difficulties which decision makers often have in making judgements about the trade-off between costs and benefits. If our decision maker had not found this to be a problem then we could have treated cost as just another attribute. Having converted the overall values for costs to a 0 to 100 scale we could have determined an appropriate swing weight for costs in comparison with the benefit attributes and determined an overall aggregate score incorporating both benefits and costs. However, because our owner, in common with many decision makers, had difficulty in judging directly the cost-benefit trade-off, we will not try to assess weights for costs and benefits, but proceed as follows. In this figure, the aggregate value of benefits has been plotted against the value relating to annual cost for each of the offices. Clearly, the higher an office appears on the benefits scale and the further to the right on the cost scale then the more attractive it will be. Thus, the ideal office would be located in the north-east corner of the plot as you might expect, there is no office in that position! If we compare Addison

MULTI-ATTRIBUTE VALUE ANALYSIS 55

Square with Elton Street, it can be seen that, while both have similar costs, Addison Square has higher benefits. It would not therefore be worth considering Elton Street and this office is thus dominated by Addison Square in this picture. Similarly, Gorton Square not only has lower costs, but also higher benefits when compared with Bilton Village, Denver Street and Filton Village. Therefore Bilton Village, Denver Street and Filton Village are also dominated offices, this time by Gorton Square. The efficient frontier From the above analysis, you can see that the only locations which are worth considering are Addison Square, Gorton Square and Carlisle Walk. These non-dominated offices are said to lie on the efficient frontier. Do not forget, though, that sensitivity analyses may reveal a different picture. We will explore this later. The choice between the three offices on the efficient frontier will depend on the relative weight the owner attaches to costs and benefits. If he is much more concerned about benefits then Addison Square will be his choice. Alternatively, if he is more concerned to keep his costs low, then he should choose Carlisle Walk. Gorton Square would be an intermediate choice. It costs $5,300 more per year than Carlisle Walk, but offers slightly higher benefits. This information may be sufficient for the owner to make a choice. At the very least it should illuminate his understanding of the decision problem. He may be surprised that Bilton Village has fared so badly or that Carlisle Walk has done so well and he may wish to check back through the data he has supplied to see why this has happened. However, it is possible that the decision maker still feels unable to choose between the three offices on the efficient frontier and feels that a more formal approach would help him. If the decision maker wanted to take the analysis further, he could use the following procedure suggested by Edwards and Newman (1986). Consider first a move from Carlisle Walk to Gorton Square. This would lead to an increase in the value of benefits from 47.4 to 60.2, an increase of 12.8. However, it would also lead to an increase in costs of $5,300. Therefore each one point increase in the value of benefits would cost him $5,300/12.8 which is $414. Similarly, a move from Gorton Square to Addison Square would increase the value of benefits by 20.6 points at an extra cost of $23,000. This would therefore cost $23,000/20.6 which is $1,117 for each extra benefit value point. So if an extra value point is worth less than $414 to the owner, he should choose Carlisle Walk. If it is worth between $414 and $1,117, he should choose Gorton Square and if it is worth paying

56 ANALYTICAL SUPPORT FOR DECISION MAKING

more than $1,117 for each extra value point he should choose Addison Square. Now we need to determine how much each extra value point is worth to the owner. This can be achieved by selecting a lower level attribute from the value tree which the owner will find fairly easy to evaluate in monetary terms. The owner suggests that this is image. He is then asked what extra annual cost he would be prepared to incur for a move from the office with the worst image to one with the best image. He answers that it would be worth paying an extra $15,000. This means that he considers that it would be worth paying $15,000 for a 100 point increase in the value of image. Now the weight of image is 23% of the total weight allocated to the attributes. So an increase of 100 points on the image scale would increase the aggregate value of benefits by 23 points. Therefore the owner is prepared to pay $15,000 to gain 23 points in the value of aggregate benefits. This implies that he is prepared to pay $15,000/23 or $652 per point. On this basis he should choose the Gorton Square office. Of course, the data we have been dealing with is far less precise than the above analysis might have implied and it is unlikely that the owner will be a hundred per cent confident about the figures which he has put forward. Before making a firm recommendation therefore we will explore the effect of changes in these figures through an extensive sensitivity analysis as described in the next section. The investigations described above should all be facilitated by the software tool used to support the analysis, which may be a standard spreadsheet or customised software. In practice, the nature of the analysis is dictated by the software tools available, particularly if working interactively with decision makers. Thus, it is important that the tools are flexible and easy to use as well as providing for appropriate display facilities and analyses.

Sensitivity and robustness analysis

In a sensitivity analysis, changes may be made to investigate the significance of missing information, to explore the effect of a decision makers uncertainty about their values and priorities or to offer a different perspective on the problem. On the other hand, there may be no practical or psychological motivation for changing values; the exploration may be driven simply by a wish to test the robustness of results. Such analysis can be carried out manually by changing the values about which the decision maker feels unsure and reworking the evaluation. However, this process is rather tedious and hardly encourages a thorough investigation, so it is also greatly facilitated by the use of appropriate visual displays and by interactive software.

MULTI-ATTRIBUTE VALUE ANALYSIS 57

Sensitivity analysis can be viewed from the following three perspectives: Technical perspective: From a technical perspective sensitivity analysis is the objective examination of the effect on the output of a model of changes in input parameters of the model. The input parameters are the value functions, scores and weights as determined by the decision makers. The output is any synthesis of this information the overall evaluation of alternatives or the aggregation of values to any intermediate level of the value tree. A technical sensitivity analysis will determine which, if any, of the input parameters have a critical influence on the overall evaluation that is, where a small change in a criterion weight or an alternatives score can affect the overall preference order. Individual perspective: The function of sensitivity analysis from an individuals perspective is to provide the sounding board against which they can test their intuition and understanding of the problem. Do they feel comfortable with the results of the model? If not, why not? Have important criteria been overlooked in the analysis? Group perspective: The function of sensitivity analysis within the group context is to allow the exploration of alternative perspectives on the problem, often captured by different sets of criteria weights. For example, if the problem were one of determining future energy policy one might look at the decision from the perspective of an economist, an environmentalist, different industry representatives. In our example case the business owner is a little worried about the weight of turnover relative to working conditions currently turnover has 81% of the weight in comparison to19% allocated to working conditions. There are two ways in which we can investigate this using the VISA software, as follows: interactive investigation sensitivity graph.

58 ANALYTICAL SUPPORT FOR DECISION MAKING

Interactive investigation VISA allows you interactively to change the value of any input to the model, by dragging the appropriate element in a visual display, and immediately to see the effect of this. Let us look at the impact on overall benefit scores of increasing the weight on working conditions. The two figures on the left show the current allocation of weights between turnover and working conditions and the current evaluation of the seven offices with respect to overall benefit. On the right we see the effect of increasing the weight on working conditions to 50%: the benefit scores of Addison Square and Elton Street are now very close and the order of Bilton Village and Carlisle Walk is reversed.

The impact of this change on the benefits vs costs graph is seen in this figure: Elton Street has now moved to join Addison Square on the efficient frontier.

MULTI-ATTRIBUTE VALUE ANALYSIS 59

Sensitivity graph This figure shows how the value of benefits for the different offices varies with changes in the weight placed on turnover. The horizontal axis is the weight given to turnover. Currently it has a value of 81, indicated by the dashed vertical line. The vertical scale is the aggregated benefits score. Each alternative is represented by a line which illustrates how the benefit score of that alternative changes as the weight applied to turnover is changed. Note that as the weight assigned to turnover is increased, the weight on working conditions is automatically decreased to maintain a sum of 100 and the weights of the subcriteria are adjusted accordingly. For example, if turnover had a weight of zero the benefit scores are given by the positions of the alternatives as the lines cross the Y-axis (ie at the left-hand side of the graph). This would imply that the three subcriteria of turnover would also have zero weights, so the weights for the six lowest level benefit attributes would now be: Closeness to customers: Visibility: Image: Size: Comfort: Car parking: 0 0 0 30 20 10

These normalise to 0, 0, 0, 50, 33.3, and 16.7 respectively which would mean that Elton Street, for example, would have benefits with a value of 81.7 (as seen in the figure above). At the other extreme, if turnover had a weight of 100 (and therefore, working conditions a weight of zero) the value of benefits for Elton Street would have been 60.4. The line joining these points shows the benefit score for Elton Street for turnover weights between 0 and 100. It can be seen that Elton Street gives the highest value of benefits as long as the weight placed on turnover is less than about 50 (52.1 to be precise). If the weight is above this figure then Addison Square has the highest value of benefits. Since the owner assigned a weight of 81

60 ANALYTICAL SUPPORT FOR DECISION MAKING

to turnover, it will take a fairly large change in this weight before Elton Street is worth considering (remember that these two offices have approximately the same costs). The figure above also shows that no change in the weight attached to turnover will make the other offices achieve the highest value for benefits. Filton Village in particular scores badly, for any weight. If we consider the other two offices on the efficient frontier, we see that Gorton Square always has higher valued benefits than Carlisle Walk. There are many other sensitivity analyses we might carry out. Sensitivity graphs such as the one just described can be constructed for each of the bottomlevel attributes: these figures show the impact on benefit scores of the weights assigned to the subcriteria of turnover. We can see that the position of Addison Square is robust, but should Addison Square become unavailable, then whether Elton Street or Gorton Square is preferred is reasonably sensitive to the weights assigned to these criteria. However, referring back once again to the benefits vs costs graph we see that although these two offices might be vying for the highest benefit score, Gorton Square has significantly lower costs. The interactive facility of VISA allows the user to explore the impact of any changes to weights or scores, individually or in combination. This ability to play not only enables sensitivity analysis to be carried out, it also serves as a means of validating the model for the decision maker as they are able to see whether or not it behaves as they would anticipate. More sophisticated sensitivity analyses Further to the interactive and one-dimensional analyses described above there are many more sophisticated analyses which may be carried out. Once again effective software is essential to support these analyses, but equally important is the judgement of the facilitator or analyst as to when it is appropriate to introduce these. Additional analyses should further understanding and learning about the issue; they should not generate unnecessary complexity leading to confusion and rejection of the analysis. A few examples are shown in Additional sensitivity analysis (page 61), illustrated by reference to our illustrative case.

MULTI-ATTRIBUTE VALUE ANALYSIS 61

Additional sensitivity analyses

Could Filton Village be the preferred office? We could investigate this by making interactive changes to the criteria weights as described above. However, unless we do this in a very systematic manner then we may miss a particular combination of weights which would make Filton Village the preferred choice. An alternative is to take an analytic approach to answer the question, which is essentially: is there a combination of weight values which would make the score of Filton Village higher than that of all the other offices? In technical terms, we are asking if Filton Village is potentially optimal. The underlying mathematics is quite complex, but need not concern us here. The outcome of this analysis can be seen in the figure below; Filton Village is now on the efficient frontier. The smallest changes in weights necessary to bring this about are shown in the bar-chart in the figure, the solid bars show the original weights, the diagonally shaded bars the new weights. The allocation of weight between turnover and working conditions is virtually unchanged; within turnover the weight on image is substantially increased and that
Sensitivity analysis potential optimality of Filton Village

on visibility is reduced to zero, and within working conditions the weight on car parking is reduced to zero. The decision maker may feel that these changes would no longer reflect his priorities in particular the substantial increase in importance of image and thus feel comfortable in ruling out Filton Village from further consideration. It turns out that all seven offices are potentially optimal, that is, given a particular combination of weights they would appear on the efficient frontier. It is not, however, always the case that all alternatives are potentially optimal and this may enable the decision maker to rule out some options at an early stage thereby simplifying the analysis. Decision maker is unable to specify high-level weights In the office example we never required the decision maker to specify weights for costs and benefits, but were able to explore the issue through the efficiency plot. If there are more than two top-level criteria this approach does not work, but we can extend the idea to the concept of preference regions which relate to three criteria. This figure on the next page shows the outcome of such an analysis on the basis of costs, turnover and working conditions.

62 ANALYTICAL SUPPORT FOR DECISION MAKING Sensitivity analysis preference regions

Turnover
Addison Square

evaluations. Rather than spending time trying to obtain this information early in the analysis it may be easier to record a score as an interval rather than a single value.

Carlisle Walk

Costs

Gorton Square

The figure below shows the efficiency plot for the office example if some uncertainty is introduced in all scores (generally of the order of 10%, but with at least one factor unknown and recorded as a range of 0-100 for each office). The bottom left corner of Elton Street the rectangle denotes the position of the option if all lowest possible scores were taken, the top right corner denotes the position if all highest possible scores were taken. Thus, all positions in the Working rectangle are possible given the intervals specified Conditions and the size of the rectangle gives an indication of the degree of uncertainty. Perhaps surprisingly, the picture is not substantially different to that seen in the original efficiency plot, indicating how robust the findings are to imprecision in the scores.
Sensitivity analysis impact of interval scores

At the apex of the triangle corresponding to a particular criterion, all weight is allocated to that criterion. The shaded regions show areas of weight space in which a given alternative is preferred there are four such regions in this figure, one corresponding to each of Addison Square, Carlisle Walk, Elton Street and Gorton Square. The picture confirms what we already know, that if we were only concerned about costs Carlisle Walk would be the preferred office, if we were only concerned about turnover then Addison Square is preferred and only about working conditions, Elton Street is preferred. It also shows that the largest and most central region is that corresponding to Gorton Square, indicating that this is perhaps a robust choice. Note that Bilton Village, Denver Street and Filton Village do not appear in this picture, indicating that no matter what weights we allocate to costs, turnover and working conditions none of these three offices is preferred (although as we have seen above, changing lower level weights would have an impact). Precise information is unavailable for scoring In many situations it is not possible to allocate precise scores when evaluating alternatives against criteria. This may be because information is unavailable, or is in itself imprecise, or because decision makers cannot agree on qualitative

MULTI-ATTRIBUTE VALUE ANALYSIS 63

Conflicts between intuitive and analytic results

It may be that, if the decision maker had viewed the problem intuitively, then he would have ranked his preferences for the offices in a very different order from that obtained through our analysis. There are several possible explanations for this. We suggested earlier that when a decision maker is faced with large, complex decision problems then simplifying strategies (such as elimination by aspects, satisficing, conjunctive, disjunctive or lexicographic) are likely to be used. We also drew the analogy with an attempt to answer a mathematics problem by using mental arithmetic rather than a calculator. This view is supported by research which suggests that the correlation of preference rankings derived from holistic judgements with those derived from SMART-type analyses decreases as the number of attributes in the problem gets larger. In other words, the larger the problem then the less reliable holistic judgements may be (see von Winterfeldt and Edwards for a summary of this research). In making a holistic judgement the decision maker may be focusing on just a few key criteria, excluding other factors which when brought into play favour a different choice. On the other hand it may be that the holistic judgement is founded largely on tacit, or subconscious feelings which have not been made explicit in the formal analysis. Another possible reason for discrepancies between holistic and analytic results may be that the axioms on which multi-attribute value analysis are founded are not acceptable to the decision maker (these are discussed in detail in the next section). It is possible that the decision maker could argue the case for a different set of sensible axioms. As long as he or she behaved consistently with these axioms, we could not argue that rejection of the results of our analysis was irrational. A conflict between holistic and analytic rankings is not a reason to reject one or the other. Rather it should serve as a catalyst for furthering understanding of the issue, with the aim of achieving convergence between the two. It may be that the decision maker is persuaded by the analysis to change their intuitive assessment, or it may be that the conflict surfaces new information (for example, an important attribute may have been left out or the interaction between two attributes may not have been taken into account) which can then be incorporated in the analysis. In practice, situations in which such conflicts arise and are resolved are ones which generate the greatest learning and satisfaction in the process. We can, of course, never be certain that a decision model is a faithful representation of the decision makers preferences. Indeed the aim of

64 ANALYTICAL SUPPORT FOR DECISION MAKING

the model is not to be descriptively valid but to inform decision making. In a computer model of a traffic system, for example, the models validity can be assessed by comparing its predictions with the behaviour of the real system, but here we are attempting to model the decision makers beliefs and attitudes for which there is no physical analogue. This begs the question: at what point do we decide that a decision model is adequate so that further refinements and revisions are not worth carrying out? Requisite decision model One response to this question is found in Phillips (1984) concept of a requisite decision model. Briefly, a model is considered to be requisite when it is provides the decision maker with enough guidance and insight to decide upon a course of action. Thus at the point where the decision maker knows what to do next a requisite model has been achieved. Phillips argues that: ... the modelling process uses the sense of unease among the problem owners about the results of the current model as a signal that further modelling may be needed, or that intuition may be wrong. If exploration of the discrepancy between holistic judgement and model results shows the model to be at fault, then the model is not requisite it is not yet sufficient to solve the problem. The model can be considered requisite only when no new intuitions emerge about the problem. Thus the requisite modelling process does not attempt to obtain an exact representation of the decision makers beliefs and preferences, or to prescribe an optimal solution to a problem. However, by exploiting the conflicts between the results of the analysis and intuitive judgements it will help the decision maker to resolve conflicts and inconsistencies in his or her thinking. As a deeper understanding of the problem is obtained the model will be revised and the discrepancy between the analytical and intuitive judgements will be reduced. Eventually, the decision maker will find that the model provides enough guidance to reach a decision.

Theoretical considerations
The approach to decision making described in this unit is one which is widely used in practice and one which is intuitively straightforward. If you were asked to design an ad hoc approach to support decision making, taking into account information about multiple, possibly conflicting objectives, it is quite likely that you would come up with something which is similar. However, the approach is not ad hoc, it is based on a number of well-founded axioms.

MULTI-ATTRIBUTE VALUE ANALYSIS 65

In our analysis of the office location problem we implicitly made a number of assumptions about the decision makers preferences. These assumptions, which are listed below, can be regarded as the axioms of the method. They represent a set of postulates which may be regarded as reasonable. If a decision maker accepts these axioms, and is rational (that is, behaves consistently in relation to the axioms), then he or she should also accept the preference rankings indicated by the method.

The axioms of the method

Decidability We assumed that the owner was able to decide which of two options he preferred. For example, we assumed that he could state whether the improvement in image between Carlisle Walk and Gorton Square was greater than the improvement between Carlisle Walk and Bilton Village. It may have been that the owner was very unsure about making this comparison or he may have refused to make it at all. Transitivity We discussed the idea of transitivity of preferences earlier. To recap, the owner preferred the image of Addison Square to Bilton Village (ie A to B), he also preferred the image of Bilton Village to Carlisle Walk (ie B to C). If transitivity applies then the owner must therefore also prefer the image of Addison Square to Carlisle Walk (ie A to C). Summation This implies that if the owner prefers A to B and B to C, then the strength of preference of A over C must be greater than the strength of preference of A over B (or B over C). Solvability This assumption was necessary for the bisection method of obtaining a value function. Here the owner was asked to identify a distance from the centre of town that had a value halfway between the worst and best distances. It was implicitly assumed that such a distance existed. In some circumstances there may be gaps in the values which an attribute can assume. For example, the existence of a zone of planning restrictions between the centre of the town and certain possible locations might mean that siting an office at a distance that has a value halfway between the worst and best distances is not a possibility the decision maker can envisage.

66 ANALYTICAL SUPPORT FOR DECISION MAKING

Finite upper and lower bounds for value In assessing values we had to assume that the best option was not so wonderful and the worst option was not so awful that values of plus and minus infinity would be assigned to these options.

Assumptions made when aggregating values

In our analysis we used the additive model to aggregate the values for the different attributes. As we pointed out, the use of this model is not appropriate where there is an interaction between the scores on the attributes. In technical terms, in order to apply the model we need to assume that mutual preference independence exists between the attributes. This is illustrated in Example of mutual preference independence on page 67. If mutual preference independence does not exist, it is usually possible to return to the value tree and re-define the attributes so that a set of attributes which are mutually preference independent can be identified. For example, perhaps visibility and image could be replaced by a single attribute ability to attract casual customers. In the occasional problems where this not possible, other models are available which can handle the interaction between the attributes. The most well known of these is the multiplicative model. Consider again the case of the house purchase decision where the quality of the architecture and attractiveness of the garden complemented each other. If we let V(A) = the value of the architecture of a given house and V(G) = a value for the attractiveness of the garden then we might find that the following model represented the overall value of the house: Value = 0.6 V(A) + 0.3 V(G) + 0.1 V(A)V(G) The numbers in the above expression represent the weights (note that they sum to 1) and the last expression, which involves multiplying the values together, represents the interaction between architecture and garden. Because the multiplicative model is not widely used, we will not consider it in detail. Longer discussions can be found in Bodily (1985) and von Winterfeldt and Edwards (1986).

MULTI-ATTRIBUTE VALUE ANALYSIS 67

Example of mutual preference independence

To demonstrate preference independence let us suppose that our office location problem only involves two attributes: distance from customers and office size. Our decision maker is now offered two offices, X and Y. These are both the same size (1,000 square feet), but X is closer to customers, as shown below: Office X Y Distance from customers 3 miles 5 miles Office floor area 1,000 square feet 1,000 square feet visibility and the owner has allocated weights of 40 and 60 to these two attributes. Two new offices, P and Q, are being compared and the values assigned to the offices for each of these attributes are shown below (0 = worst, 100 = best). Office P Q Visibility 0 100 Image 100 0

Using the additive model, the aggregate value of benefits for P will be: (40 x 0) + (60 x 100) = 6000 ie, 60 after dividing by 100 The aggregate value of benefits for Q will be: (40 x 100) + (60 x 0) = 4000 ie, 40 after dividing by 100. This clearly suggests that the decision maker should choose office P. However, it may be that he considers image only to be of value if the office is highly visible. Office Ps good image is, he thinks, virtually worthless because it is not in a highly visible location and he might therefore prefer office Q. Thus, if image is not preference independent of visibility, the additive model will not correctly represent the owners preferences. How can the absence of mutual preference independence be identified? The most obvious way in which this will reveal itself is in the use of phrases like this depends on... when the decision maker responds to questions. For example, when asked to assign a value to the image of an office, our decision maker might well have said that depends on how visible the office is.

Not surprisingly, the decision maker prefers X to Y. Now suppose that we change the size of both offices to 400 square feet. If, as is likely, the decision maker still prefers X to Y his preference for a distance of three miles over a distance of five miles has clearly been unaffected by the change in office size. This might remain true if we change the size of both offices to any other possible floor area. If this is the case, we can say that distance from customers is preference independent of office size because the preference for one distance over another does not depend on the size of the offices. If we also found that size of office is preference independent of distance from customers then we can say that the two attributes are mutually preference independent. Note that mutual preference independence does not automatically follow. When choosing a holiday destination, you may prefer a warmer climate to a cooler one irrespective of whether or not the hotel has an open-air or indoor swimming pool. However, your preference between hotels with open air or indoor swimming pools will probably depend on whether the local climate is warm or cool. To see what can happen when the additive model is applied to a problem where mutual preference independence does not exist, consider the following problem. Suppose now that our office location decision depends only on the attributes, image and

68 ANALYTICAL SUPPORT FOR DECISION MAKING

Summary
In this chapter we have looked in detail at one method which has been suggested for the analysis of decision problems where the decision maker wants to take explicit account of a number of attributes associated with the alternatives. The approach studied is known as multi-attribute value analysis. It requires that the performance of each alternative is measured on each attribute and then the attributes themselves are weighed against each other before a decision can be made. In the example used in this chapter, we saw that the method required the decision maker to quantify strengths of preferences. While this may not have been an easy process, we found that the figures put forward did not need to be exact, though we did try to ensure that they were consistent. The decision problem presented in this chapter was designed to be amenable to hand calculations in order to ensure that the analysis was fully understood. As already mentioned, this would be an extremely tedious way of approaching larger problems and, as we have seen, effective software facilitates even very simple analysis and visual presentations of results as well as permitting interactive investigations and more sophisticated sensitivity analyses.

MULTI-ATTRIBUTE VALUE ANALYSIS 69

A checklist for building and using a MAVT model

1 2 3 4 5 Be clear about who the decision maker or decision makers are. Identify options for evaluation. Identify criteria relevant to the evaluation and structure as a value tree. Confirm the value tree meets requirements specified. For each bottom-level attribute specify the scale to be used to evaluate the performance of alternatives with respect to that attribute: local or global scale? value function, defined qualitative scale or direct rating? Evaluate the performance of alternatives against the specified scales for all bottom-level attributes. Weight criteria, taking into account the measurement scale in use and the intrinsic importance of the criteria. Weights reflect the relative value of 100 points on different criteria and thereby determine trade-offs which are acceptable to the decision maker. Synthesise information about weights and scores to give an overall value for each alternative, or aggregate values on a number of high-level criteria. This gives a first indication of preferred alternatives, but must be supplemented by further analysis. Use the model, supported by the V.I.S.A software, to explore: Profiles of alternatives look out for: dominated alternatives dominating alternatives alternatives which are good all-rounders (alternatives which do reasonably well on all criteria although not necessarily outstandingly in any regard) alternatives which although they score well overall have significant weaknesses.

Sensitivity analyses: one-dimensional sensitivity graphs interactive sensitivity analyses.

Efficiency plots of high-level criteria, for example costs vs benefits costs vs performance risks vs benefits long-term vs short-term benefits different stakeholder perspectives).

70 ANALYTICAL SUPPORT FOR DECISION MAKING

Review exercises

1 You need a word processing package for the personal computer in your office. Because your employer will pay for the package you are not concerned about the cost, but you would like a package which is as easy to use as possible and which also has a wide range of functions such as a thesaurus, spell checker and graphics. After discussing the matter with a friend, who is something of an expert in this field, you identify seven potential packages and allocate values to them to reflect their ease of use and available facilities. These values are shown below (0 = worst, 100 = best).
Package Super Quill Easywrite Wordright Lexico Ultraword Keywrite Fastwrite Ease of use 100 90 50 0 20 40 85 Facilities available 30 70 20 40 100 0 55

(a) Plot each packages value for ease of use and facilities available on a graph and hence determine the packages which lie on the efficient frontier. (b) Suppose that you judge that a switch from a package with the least facilities available to one with the most facilities is only 60% as attractive as a switch from a package which is the least easy to use to one which is the most easy to use. Assuming that mutual preference independence exists between the two attributes, which package should you choose? (c) After some reflection you realise that the extra facilities available on a package will be of little value to you if they are going to be difficult to use. What does this imply about your method of analysis in part (b)?

MULTI-ATTRIBUTE VALUE ANALYSIS 71

2 A chemical company is expanding its operations and a disused woollen mill is to be converted into a processing plant. Four companies have submitted designs for the equipment which will be installed in the mill and a choice has to be made between them. The manager of the chemical company has identified three attributes which he considers to be important in the decision: cost, environmental impact and reliability. He has assessed how well each design performs on each attribute by allocating values on a scale from 0 (the worst design) to 100 (the best). These values are shown below, together with the costs which will be incurred if a design is chosen.
Design Costs ($) Environmental impact A B C D 90,000 110,000 170,000 60,000 20 70 100 0 Benefits Reliability 100 0 90 50

(a) The manager is having difficulty in allocating weights to the two benefit attributes. Assuming that the two weights sum to 100 and that mutual preference independence exists between the attributes, perform a sensitivity analysis to show how the design offering the highest value for aggregate benefits will vary depending upon the weight which has been allocated to environmental impact. (b) Eventually, the manager decides to allocate environmental impact a weight of 30 and reliability a weight of 70. By plotting the benefits and costs of the designs on a graph, identify the designs which lie on the efficient frontier. (c) The manager also decides that if he was offered a hypothetical design which had the lowest reliability and the worst environmental impact he would be prepared to pay $120,000 to convert that design to one which had the best impact on the environment but which still had the lowest level of reliability. Which design should the manager choose? 3 A British company has won an important contract to supply components regularly to Poland. Four methods of transport are being considered: (i) air (ii) sea (iii) road and ferry and (iv) rail and ferry. The companys distribution manager has identified four relevant attributes for the decision: punctuality, safety of cargo, convenience

72 ANALYTICAL SUPPORT FOR DECISION MAKING

and costs. She has also allocated weights of 30 to punctuality, 60 to safety of cargo and 10 to convenience. The manager then rated the performance of each form of transport on the different attributes. The values she assigned are shown below, together with the estimated annual cost of using each form of transport.
Form of transport Punctuality Air Sea Road and Ferry Rail and Ferry 100 0 60 70 Safety 70 60 0 100 Benefits Convenience 60 80 100 0 150,000 90,000 40,000 70,000 Costs ($)

(a) Determine the form of transport which has the highest valued overall benefits, assuming that mutual preference independence exists between the attributes. (b) For each form of transport, plot the value of overall benefits against costs and hence identify the forms of transport which lie on the efficient frontier. (c) If the manager would be prepared to pay $70,000 per year to move from the least safe to the most safe form of transport, determine which alternative she should select. 4 A local authority has to decide on the location of a new waste disposal facility and five sites are currently being considered; Inston Common, Jones Wood, Peterton, Red Beach and Treehome Valley. In order to help them to choose between the sites the managers involved in the decision arranged for a decision analyst to attend one of their meetings. He first got the managers to consider the factors which they thought were relevant to the decision and, after some debate, four factors were identified: (i) the visual impact of the site on the local scenery (for example a site at Treehome valley would be visible from a nearby beauty spot); (ii) the ease with which waste could be transported to the site (for example Red Beach is only two miles from the main town in the area and is close to a main highway while Inston Common is in a remote spot and its use would lead to a major increase in the volume of transport using the minor roads in the area); (iii) the risk that the use of the site would lead to contamination of the local environment because of leakages of chemicals into water courses, for example; (iv) the cost of developing the site.

MULTI-ATTRIBUTE VALUE ANALYSIS 73

The decision analyst then asked the managers to assign scores to the sites to show how well they performed on each of the first three attributes. The scores they eventually agreed are shown below, together with the estimated cost of developing each site. Note that 0 represents the worst and 100 the best score on an attribute. In the case of risk, therefore, a score of 100 means that a site is the least risky.
Site Visual impact Inston Common Jones Wood Peterton Red Beach Treehome Valley 100 20 80 20 0 Ease of transport 0 70 40 100 70 Benefits Risk 60 100 0 30 60 35 25 17 12 20 Costs ($ million)

The decision analyst then asked the managers to imagine a site which had the worst visual impact, the most difficult transport requirements and the highest level of risk. He then asked them, if they had a chance of switching from this site to one which had just one of the benefits at its best value, which would they choose? The managers agreed that they would move to a site offering the least risk of contamination. A move to a site with the best visual impact was considered to be 80% as preferable as this, while a move to a site with the most convenient transport facilities was 70% as preferable. (a) Can we conclude from the values which were assigned to the different sites for visual impact that, in terms of visual impact, the Inston Common site is five times preferable to the Red Beach site? If not, what can we infer from the figures? (b) An alternative way of allocating weight to the three benefit attributes would have involved asking the managers to allocate a score reflecting the importance of each attribute. For example they might have judged that risk was five times more important and visual impact three times more important than ease of transport so that weights of 5, 3 and 1 would have been attached to the attributes. What are the dangers of this approach? (c) Assuming that mutual preference independence exists between the attributes determine the value of aggregate benefits for each site. (d) Plot the aggregate benefits and costs of each site on a graph and hence identify the sites which lie on the efficient frontier. (e) Although a weight of 80 was finally agreed for visual impact, this was only after much debate and some managers still felt that a weight of 65 should have been used while others thought that 95 would have been more appropriate. Perform a sensitivity analysis on the weight assigned to visual impact to examine its effect on the aggregate benefits of the sites and interpret your results.

74 ANALYTICAL SUPPORT FOR DECISION MAKING

References
Belton, V. and Stewart T., Multiple Criteria Decision Analysis: an Integrated Approach, Kluwer, (2001). Bodily, S.E. Modern Decision Making, McGraw-Hill, (1985). Brownlow, S.A. and Watson, S.R. Structuring multi-attribute value hierarchies,Journal of the Operational Research Society 38, 309-17, (1987). Buede, D.M. and Choisser, R.W. An aid for evaluators of system design alternatives, Defense Management Journal 36, 32-8, (1984). Buede, D.M. Structuring value attributes Interfaces, 16, 52-62, (1986). Butterworth, N.J. Giving up "The Smoke": a major institution investigates alternatives to being sited in the City, Journal of the Operational Research Society, 40, 711-18. (1989). Checkland, P. Soft systems methodology, in Rosenhead (1989). Checkland, P. Systems Thinking. Systems Practice, Wiley (1981). De Bono, E. De Bonos Thinking Course, Ariel Books, (1985). De Bono, E. Six Thinking Hats, Penguin Books, (1990). Drake, A.W. Keeney, R.l. and Morse, P.M. (eds) (1972) Analysis of Public Systems, MIT Press. Eden, C. and Ackermann, F. Making Strategy: the Journey of Strategic Management, Sage (1998). Eden C. and Jones, S. Using Repertory Grids for Problem Construction, Journal of the Operational Research Society, (1984). Edwards, W. Social utilities, Engineering Economist Summer Symposium Series, (1971). Edwards, W. and Newman, J.R. Multi-attribute evaluation, in Arkes H.R. and Hammond K.R. (Eds.) Judgment and Decision Making, Cambridge University Press, (1986). Farquhar, P.H. and Pratkanis, A.R. Decision Structuring with Phantom Alternatives, Management Sciences, Vol 39: 10, 1214-1226 (1993). French, S. (1989) Decision Theory: an introduction to the mathematics of rationality, Ellis Horwood. Friend, J. The strategic choice approach, in Rosenhead, pp 120-157 (1989).

MULTI-ATTRIBUTE VALUE ANALYSIS 75

Keeney, R.L. Value-Focused Thinking: A path to creative decision making, Harvard University Press, (1992). Keeney, R.L. Analysis of preference dependencies among objectives, Operations Research, 29: 1105-1120, (1981). Keeney, R.L. and Raiffa, H. Decisions With Multiple Objectives: Preferences and Value Tradeoffs, Wiley, (1976). Keeney R.L. and Raiffa, H. (1972) A critique of formal analysis in pubic sector decision making, in Drake, A.W. Keeney, R.l. and Morse, P.M. (eds) Analysis of Public Systems, MIT Press, pp64-75. Phillips, L.D. (1990) Decision analysis for group decision support, in Eden, C. and Radford, J. (eds) Tackling Strategic Problems: the role of group decision support, Sage, pp142-153. Phillips, L.D. A theory of requisite decision models, Acta Psychologica, 56, (1984). Pruzan, P. and Bogetoft, P. Planning with Multiple Criteria, Copenhagen Business School Press, 2nd edn, (1997). Rosenhead, J. ed. Rational Analysis for a Problematic World, Wiley, (1989). Roy, B. (1996) Multicriteria Methodology for Decision Aiding, Kluwer. Van der Heijden, K. Scenarios: the art of strategic conversation, Wiley (1996). Van Gundy Jr, A.B. (1988) Techniques of Structured Problem Solving, Van Nostrand Reinhold. Von Winterfeldt, D. and Edwards, W. Decision Analysis and Behavioral Research, Cambridge University Press, (1986). Watson, S.R. and Buede, D.M., Decision Synthesis: The Principles and Practice of Decision Analysis, Cambridge University Press, (1987). Wooler, S. A decision aid for structuring and evaluating career choice options, Journal of the Operational Research Society, 33, pp 343-352, (1982). Wooler, S. and Barclay, S. Strategy for reducing dependence on a strike-prone production facility, in P. Humphreys, A. Vari, J Vecsenyi and O. Larichev (eds) Strategic Decision Support Systems, NorthHolland, (1988). Zeleny, M. (1982) Multi-criteria Decision Making, McGraw-Hill. .