Multi-criteria Decision Making

The Eastside Community Dialogue held last Wednesday evening was intended to set the direction for decision making on our sewage situation. But where is the starting point and what direction will this process take? Is this truly a fresh start or is the CRD still attempting to pull strings in the background? Will this be a rational science-based process driven by informed residents or will challenges from a misinformed nimby crowd sway council’s mind? Can the Eastside committee avoid the miss-steps of the Westside committee in the short time available or will this train derail before reaching the station?

Many questions but few answers. Perhaps a road map would help to keep the process on the RITE track and help define the process and guide the decision making inputs.  The Triple Bottom Line (TPL) has been promoted and used in past studies as a suitable tool for evaluating and ranking options and alternatives but TPL is but one of many methodologies.

A best practices study conducted by a European university a few years ago reviewed dozens of methodologies and developed an optimized framework specifically for sanitation/wastewater projects. It lists key objectives, relevant criteria to be considered, and a weighted scoring system to rank valid alternatives.

We feel this study was very relevant to what is happening now and have provide the following short summary. If you want to dig right into the report you can find it here: http://www.switchurbanwater.eu/outputs/pdfs/W4-1_CALE_RPT_D4.1.10_Best_practice_decision-support_system_for_ecosan%20systems.pdf

Sustainable Water Management in the City of the Future

Integrated Project Global Change and Ecosystems

D 4.1.10 Best Practice And A Decision-Support System For Ecosan Systems

December 2010

Project co-funded by the European Commission within the Sixth Framework Programme

This document provides a summary of current best practice in urban ecosan (ecological sanitation) systems and presents an approach for technology selection based on a multi-criteria approach.

Abstract

We live in an era of rapid urbanization. Cities are rapidly developing and are in need of good infrastructure and resources like water, energy and food. At the same time they produce large quantities of wastes. Overall, the rapid urbanization requires new approaches in order to remain within the carrying capacity of our global ecosystem.

One of the issues is the implementation of appropriate sanitation systems which also value the ‘resource value’ of wastes and wastewater. The standard approach to manage wastewater is the construction of sewerage and wastewater treatment plants. After treatment, the wastewater is disposed of to surface water.

Alternative options called ecological sanitation, promoting flows separation at household level and wastewater management in decentralized systems, receive increasing attention in the last decades and are experimented with in various pilot projects. A proper evaluation to assess the potential of these new sanitation systems for their potential to improve the sustainability of cities is however lacking. Different methodologies and approaches have been piloted for the selection of sanitation options, some complex, others with high data requirements or complex weighing factors.

A literature review into multi-criteria methodologies and existing frameworks was done to gain insight into decision making process and assessment methodologies regarding sanitation systems. From the review, it was found that 42 different multi-criteria methodologies are applicable for water management; selection of a specific methodology depends on the objective, boundaries, data availability and time allocation of the specific case. Based on some of those methodologies, 18 frameworks for water management assessment have been developed the last 12 years; an inventory of indicators was done finding 77 indicators in total.

Multi-criteria Methodologies for Urban Sanitation Systems Assessment

Introduction

• Multi-criteria methodologies have become a popular aid in decision making processes with involvement of multi-objective approaches and several stakeholders.
• Solving a multi-criteria problem often does not mean finding an optimum solution, but facilitates discussion and understanding of the different alternatives towards the finding of the most suitable solution.
• The range of available methodologies varies from simple ones like brainstorming to complex ones like computer based simulations.
• 42 different multi-criteria methodologies have been developed recently and can be used for the assessment of water management strategies, however none of them can be selected as the most suitable; each one has strengths and weaknesses.
• Those 42 methodologies included 18 different frameworks, 77 different indicators and 4 different aggregation methods.
• All methodologies face a fundamental dilemma: as more criteria are added to broaden the inclusivity of the decision making process and to guide the decision makers, the process becomes more difficult to manage and less reliable. A maximum of 35 indicators is suggested.
• The selection of an appropriate methodology acceptable to all stakeholders often turns into a “problem” by itself.
• In the development of the framework, “there is a trade-off between showing the complexity of a problem in all its facets and its simplification in order to make it understandable for all persons involved in decision processes. Either the process remains on a highly abstract level far away from the real problem, or it reduces the complexity too strongly and thereby loses too much information, so that the results do not adequately reflect the real nature of the decision problem anymore”.
• Based on this review, the main criteria in the methodologies involved in sanitation technology selection were health, social / institutional, environment, costs and system robustness.

Conclusions

• The purpose of the multi-criteria framework is to facilitate discussion during the decision making process; in that sense, transparency of the methodologies is required in order to be trusted for the different stakeholders.
• Definition of some indicators must be clearly stated leaving no room for different interpretations, misunderstanding or inappropriate assessment.
• Where a final indicator value is included, data validation and verification may be difficult, therefore raw data must be available to substantiate all calculations.
• It can be concluded that digitally modeling the systems would be the best approach to measure actual impacts of sanitation systems, however due to high data requirements, this is often not feasible, and the most suitable alternative is to calculate “potential” effects.
• Assessments based on Life Cycle Assessment showed that system boundaries significantly influence the results, and in order to make an appropriate “cradle to grave” assessment, data requirements are high.
• The main drawback of existing framework methodologies is the lack of an aggregation method to assess each criterion; an aggregation method is required to take into account the contribution or relevance of each indicator in the final score, to perform data analysis and provide meaningful comparisons in which a wide spectrum of units of measure are involved.
• There is no consensus about the influence of the methodology in the results. But it is clear that different methods apply different assumptions, weights or normalizations to achieve the expected results so users should be aware of them.
• The methodology should also suit data availability to avoid misleading results. Sophisticated software capable to make complex analyses have been developed, however the quality of the results depends on the quality of the input data. Complex methods have high data requirements and considerable time allocation, while simple methods are less sensitive to uncertainties and a less precise result is achieved in a shorter time.

Recommended Optimized Methodology

• Each methodology has strengths and weakness and may be modified to suit specific applications.
• Each criterion must be evaluated separately.
• Performance must be matched with local conditions.
• Evaluation must avoid aggregation and subjective weighting that affects transparency and traceability.
• The recommended methodology assessment is as follows:

o   Contextual Independence: evaluate to what extent a given technology is independent from external conditions; develop a checklist to measure the degree of sensitivity of the different system components to external factors.

o   Health Risk Assessment: evaluate the public health risk related to a given technology due to the contact of inhabitants with feces, urine, raw wastewater, treated wastewater or sludge. Use the approach of Qualitative Microbial Risk Assessment. (QMRA) in order to assess the potential risk of exposure.  

o   Impact on the Ecosystem: evaluate to what extent a given technology impacts on the ecosystem. Use the approach of Life Cycle Assessment (LCA) to measure the potential impact of the discharges; the two aspects considered were potential eutrophication and potential ecotoxicity.

o   Resources Use: evaluate to what extent a given technology makes an efficient use of the resources (water, energy, nutrients and chemicals). Develop a check list of resources involved in sanitation and normalize each indicator based on the maximum and minimum values of the systems under study.  

o   System Robustness: prepare an evaluation based on failure records, shock load resistance and possible user abuse. Normalize each indicator based on the maximum and minimum values of the systems under study.

o   Invisibility: evaluate to what extent a given technology is invisible for the users and the community. Prepare a check list of the area and space required and assessment of the user perception for nuisances. Again normalize each indicator based on the maximum and minimum values of the systems under study.

o   Total Cost: calculate the total cost of the system by estimating the life cycle annual cost taking into account construction investment, operation and maintenance costs and any revenues received per year.

The recommended methodology is summarized in tables 29 and 30 in the report.

This methodology is a starting point for us; some criteria may be revised and other suitable criteria may be added - but its a leap ahead of where we were only a few months ago.