Governments and big organisations make decisions that shape our society
When powerful people make dumb choices it hurts us all. Here's how to fix it.
There are lot of strategies recommended by experts (see Table 1 of our gap map protocol)
Protocol - Improving Organisational/Institutional Decision-Making (IIDM) EGM
Most of these interventions (Table 1) don’t have systematic reviews already about them.
[Systematic reviews are the best ways of knowing what works and when](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3894019/#:~:text=A systematic review is a,reduce biases and random errors.)
This project would involve conducting reviews on the interventions that might not have existing reviews
| Intervention Category | Sub-categories |
|---|---|
| Improving the quality of decision inputs | Delphi method—“an iterative process used to collect and distil the judgments of experts using a series of questionnaires interspersed with feedback” (Skulmoski et al., n.d.). |
| Prediction markets—“markets where participants trade contracts whose payoffs are tied to a future event, thereby yielding prices that can be interpreted as market-aggregated forecasts” (Wolfers & Zitzewitz, 2006). | |
| Quantitative probability judgements and forecasting1 — eliciting quantified estimates (e.g., probabilities of an event occurring) from either scientific, technical, or forecasting experts, or from a crowd (P. E. Tetlock, 2009) | |
| Use of systematic reviews — any intervention related to the use of systematic reviews (reviews with reproducible methods for finding and synthesising researcher) in decision-making (Langer et al., 2016) | |
| Access to evidence — any intervention related to the access or convenience of access to evidence (Oliver et al., 2014) | |
| Knowledge maps — A visual representation of organisational knowledge, including how each component of that knowledge is connected to other components (Chan & Liebowitz, 2006) | |
| Visualisation — visual representations of information (e.g., figures, multimedia) designed to complement written communication of key messages (Noetel et al., 2021) | |
| Executive or evidence summaries — brief, clear summaries of key, decision-relevant information (e.g., at the beginning of a a longer report; Muehlhauser, 2017) | |
| Increasing motivation for decision-quality | Accountability mechanisms — Management, supervision, or tracking of decision-making practices (e.g., quality of evidence-use) or decision-making outcomes (Kroon et al., 1991) |
| Incentives — performance-based commissions, bonuses, or extrinsic incentives for good decision-making (Tricco et al., 2016) | |
| External advocacy — public pressure campaigns to make a specific decisions (e.g., for greater transparency and ethical controls to combat principal-agent problems; Burstein, 2003) | |
| Organisational norms — establishing and marketing good processes (e.g., use of high quality evidence) as the ‘normal’ or common practice within an organisation (Berkowitz, n.d.) | |
| Group composition | Group diversity — any review focused on linking decision quality and the distribution of personal attributes (e.g., gender, seniority) among interdependent members of a work unit |
| Technical competence — any review focused on linking decision quality to the selection of decision-makers on the basis of some form of technical competence (e.g., superforecasters; P. Tetlock & Gardner, 2016) | |
| Expertise or seniority — The seniority of the decision-makers | |
| Size—Number of people involved in the decision | |
| Identifying threats and opportunities | Risk modelling — any quantitative, qualitative, or hybrid approach that aims to produce risk estimates for the purpose of risk management (Robillard, 2001) |
| Horizon scanning — a technique for detecting early signs of potentially important developments through a systematic examination of potential threats and opportunities (van Rij, 2010) | |
| Scenario planning — structured process of identifying a wide range of possible futures and how the organisation might respond (Schoemaker, 1991) | |
| Backcasting — Working backwards from a desirable future outcome to identify policies and interventions required to get there (Robinson, 1990) | |
| Premortem — Assuming a decision would be a failure, then identifying the most likely reasons for why that decision have performed poorly (Klein, 2007) | |
| Brainstorming — group creativity and problem-solving methods that involve the spontaneous contribution of ideas and solutions (Osborn, 1953; includes other creativity tools like SCAMPER; Serrat, 2017) | |
| Systems thinking tools — a theoretical analysis of the structures underlying a complex situation (UK Government Office for Science, 2022) | |
| Comparing among options | Cost-benefit analysis — a systematic method for quantifying and comparing the total costs and benefits of a decision (Office of Best Practice Regulation, 2020) |
| Multi-criteria decision analysis — a structured process that explicitly defines the criteria relevant to a decision and evaluates options against this criteria (Gebre et al., 2021) | |
| Pros and cons analysis — list the advantages and disadvantages of each solution to facilitate comparison (Baker et al., 2001) | |
| Leaning useful heuristics — training to identify the key information for making specific decisions, and simple rules for deciding using that information (e.g., “fast and frugal trees”; Keller & Katsikopoulos, 2016) | |
| Deep uncertainty tools — methods for decision-making (e.g., robust decision-making, assumption based planning) for situations where decision-makers do not know, or cannot agree upon, the likelihood of different future scenarios (Marchau et al., 2019) | |
| Game theoretic modelling — quantitative analysis of outcomes (usu., in terms of expected utility) where the decision-makers’ actions depend on the actions of others (Mesterton-Gibbons, 2019) | |
| Reducing bias or noise | Red-teaming — the practice of rigorously challenging plans, policies, systems, and assumptions by adopting an adversarial approach (Schwenk, 1990) |
| Debiasing — any intervention designed to reduce bias (e.g., education, adjusting estimates to account for optimism bias, third party referees and reviewers, incentives, nudging; Kenyon & Beaulac, 2014) | |
| Decision hygiene — structured methods to improve reliability of judgements across time or between people (e.g.,; using relative scoring, making independent assessments; Kahneman et al., 2016) | |
| Dialectical inquiry — structured process of challenging assumptions by developing plans using opposing assumptions, then contrasting the strength of both plans (Schwenk, 1990) | |
| Supervision and challenge mechanisms — a review process for decisions in an organisation, where decisions are reviewed by people who were not involved in the original decision (e.g., someone more or less senior; Cox, Strang, Søndergaard, et al., 2017) | |
| Time or effort — allow for increased time, or expend more effort, toward deliberation, analysis, and reflection | |
| Practising and reviewing decision-making | Serious games — a mental contest where through entertaining gameplay the players’ achieve a learning purpose built into the game (Rumeser & Emsley, 2018) |
| Crisis training — the training of decision-makers using experiential learning from practice crisis scenarios (Borodzicz & Van Haperen, 2002) | |
| Decision-review — formal methods for reviewing the outcomes from decisions to identify areas for improvement in the future (a.k.a., retrospectives; postmortem; Nelson, 2008) |