Economics Multiple Decision Making Criteria
Methods in Madness: Compensability in Multi-Criteria Decision Analysis
1.0 INTRODUCTION
Do you ever feel overwhelmed with decisions? On average, we make over 220 decisions per day just about food (Wansink & Sobal, 2007). Despite the frequency with which we make
decisions, we often do not process information rationally (Tversky & Kahneman, 1986). MultiCriteria Decision Analysis (MCDA) is widely used for decision making support, particularly in environmental applications, however inappropriate MCDA methods may lead to misleading recommendations. The justifications for such methods focus on pragmatic considerations at the expense of technical suitability to the problem. Literature on both the technical and practical aspects of MCDA suggests a disconnect between the state of theory and practice in environmental MCDA such that commonly used methods allow unrealistic levels of compensation between criteria. The aim of this research is to address the question of how different aggregation methods affect analysis results, in the context of environmental decision making. This study will adopt a comparative design to assess a real-world case study of the 2014 Lower Hunter Water Plan (LHWP) using three different MCDA methods to allow varying degrees of compensation between different criteria. This study will contribute to practice by developing a framework and recommendations for user-friendly non-compensatory MCDA and may assist in the selection of methods for the 2020 LHWP. A theoretical contribution will be to add to research on MCDA methods by combining insights from multiple methods and exploring new ways of calculating suitable weightings. A summary of the relevant literature demonstrating the requirements for environmental MCDA and the shortcomings of widely used methods is presented in section 2. This is followed by the research questions addressed in this study in section 3, and the proposed methodology in section 4. Ethical implications of the use secondary private data are discussed in section 5 and limitations caused by the data and methods to be used are outlined in section 6. The research contribution is discussed in section 7 and the proposed research plan including relevant milestones and completion dates is presented in section 8.
2.0 LITERATURE BACKGROUND
MCDA is used for decisions judged against more than one criterion. These criteria cannot be measured on a common scale, for example in monetary terms as in cost benefit analysis, and often involve trade-off relationships (Baudry, Macharis, & Vallée, 2018). There are a number of approaches to address such problems and each relies on different underlying assumptions. In each of these approaches, a number of stages are included, as shown in figure 1. While each stage requires the previous one to have been completed, the process may be iterative. Prior stages may be repeated as needed (Munda, 2004). Within this structure, different approaches can be taken at each stage, however the most significant differences between methods occur at the stages of choosing weightings, aggregating the information and comparing the options. When comparing options against multiple criteria, a decision regarding how the criteria are weighted must be made. Weightings can either be used as cardinal weightings or as importance coefficients which represent only ordinal information. Cardinal weightings allow for complete compensability between criteria, that is, poor scores for some criteria can be fully offset by positive performance against others (Munda, 2016).
2.1 Value and Preference Theory
In environmental analysis, the extent of compensability between criteria must be carefully considered because situations of complete compensation as implied by the use of cardinal weightings do not exist in environmental systems (Martinez-Alier, Munda, & O’Neill, 1998). Not all natural capital can be substituted with man-made capital as is assumed in classical growth models and, in many situations these are complements not substitutes (Garmendia & Gamboa, 2012; Vatn, 2005). Additionally, as man-made capital is produced using natural capital as an input, such trade-offs are not possible indefinitely, even where the assumption of substitutability is technically appropriate (Vatn, 2005). This means that where cardinal weights are used, trade-offs that do not occur in reality are allowed (Cinelli, Coles, & Kirwan, 2014). Thus, the preferred option given by such analysis may not be preferable if stricter substitution rules were enforced. The process of weighting criteria is further complicated by different conceptions of value. Mainstream economic thought tends to rely solely on exchange values which are monetary values determined in markets or market-like institutions (Vatn, 2005). These values are assumed to represent the utility obtained by individuals from commodities (Douai, 2009). Where markets, and therefore monetary values, do not exist for an element of the analysis, contingent valuation methods are used to create market-like scenarios to elicit monetary values (Spangenberg & Settele, 2010). These theories assume that individuals as consumers hold given, stable and ordinal preferences from which utility or value functions can be determined (Vatn, 2005). Assumptions about the nature of both preferences and value behind monetary valuation methods are challenged empirically and theoretically. Individuals tend not to hold given and stable preferences (Tversky & Kahneman, 1986). Preferences are not fixed but are developed through our interaction with social and cultural institutions (Vatn, 2005). Values demonstrated by individuals may vary depending on the institutional setting in which their decision-making takes place and are affected by internalised social norms causing people to act in ways inconsistent with maximising individual utility (Vatn, 2005). In many instances people make decisions, not in their role as consumers in the market, but as citizens maximizing social rather than personal utility (Vatn, 2005). Insights from behavioural studies indicate people show systematic biases and take decision-making shortcuts (heuristics) that affect their observed values (Tversky & Kahneman, 1974). As a result, methods to elicit values which assume individuals have fixed preferences that need only be revealed, may produce inaccurate results (Douai, 2009). Furthermore, the assumption that exchange values accurately and completely represent value is challenged. Exchange values rest on the assumption that the market price of a good. reveals the marginal utility it provides (Vatn, 2005). In contrast, exchange values do not
necessarily represent the use-value or benefit gained from an object. For items that are nontraded, and therefore lack exchange values, it is not appropriate to elicit quasi-exchange values (as is done in contingent valuation) as plural values exist. These values include concepts such as existence value, aesthetic value, and use value (Bingham et al., 1995). These debates have important methodological consequences for conducting MCDA. Methods that assume stakeholders have fixed values for each criterion that can be revealed may bias the criteria scores and the weights that are elicited and lead to a structure that does not truly represent the stakeholders’ values.
2.2 The Impacts of Stakeholder Participation
Environmental decisions affect a range of stakeholders and the broader community. To address multiple stakeholders, participatory methods at various stages of the analysis are used to incorporate the perspectives of multiple actors into the analysis. The level and type of stakeholder participation have further methodological consequences for the MCDA. Preferred aggregation methods are those that participants easily understand (Baudry et al., 2018; Marttunen, Mustajoki, Dufva, & Karjalainen, 2015; Stauffacher, Flüeler, Krütli, & Scholz, 2008). If complex methods are used, participants may view the procedure as a black-box’ (Cinelli et al., 2014) or as deliberate manipulation of data by analysts (Antunes, Karadzic, Santos, Beça, & Osann, 2011). Over emphasis on this concern, however, may lead to methods of weighting and aggregation which are not sufficiently sophisticated to handle the problem and rely on strong assumptions about the relationships between the criteria and structure of stakeholders’ preferences. Methods that use weights only as importance coefficients are considered most appropriate, however the aggregation procedures associated with these are criticised as being difficult to understand (Cinelli et al., 2014). Although participation increases legitimacy and inclusiveness, combining it with the use
of inappropriate MCDA methods undermines the process and can produce results not truly reflecting the best outcomes for the stakeholders. Therefore, analysis that fulfils the technical requirements for the problem at hand should be integrated with stakeholder participation, but with careful attention paid to the impacts of the stakeholders’ perceptions of the method used. Research combining technically appropriate methods and participatory processes is rare.
3.0 RESEARCH QUESTIONS
The literature is clear on some of the requirements for successful application of MCDA to environmental problems. These include the use of non-compensatory models, stakeholder participation, and methods that deal with complexity of value formation and expression. However, methods that do not satisfy these requirements are widely used. In some cases, these methods are justified for reasons such as ease of application or understanding (Antunes et al., 2011; Mooney, Baldwin, Tan, & Mackenzie, 2012). Others uncritically accept the neoclassical economic assumptions underlying the models and therefore do not recognise these problems. Hence, the field of applied MCDA does not adequately recognise the theoretical issues involved, the reasons behind this, and the potential solutions. To explore the practical implications of the disconnect between theory and practice, this study seeks to examine the impacts of the choice of MCDA method on the results. Previous studies compare different MCDA methods, however they focus on issues such as ease of application, rather than the results (Caterino, Iervolino, Manfredi, & Cosenza, 2009; Özcan, Çelebi, & Esnaf, 2011). The research question addressed in this study is: how do MCDA methodologies affect analysis results? The research will be guided by four sub-questions.
