Governments and big organisations make decisions that shape our society
When powerful people make dumb choices it hurts us all. Here's how to fix it.
There are lot of strategies recommended by experts (see Table 1 of our gap map protocol)
Protocol - Improving Organisational/Institutional Decision-Making (IIDM) EGM
Most of these interventions (Table 1) don’t have systematic reviews already about them.
[Systematic reviews are the best ways of knowing what works and when](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3894019/#:~:text=A systematic review is a,reduce biases and random errors.)
This project would involve conducting reviews on the interventions that might not have existing reviews
| Intervention Category | Sub-categories |
|---|---|
| Improving the quality of decision inputs | Delphi method—“an iterative process used to collect and distil the judgments of experts using a series of questionnaires interspersed with feedback” https://paperpile.com/c/FALrBy/osfI. |
| Prediction markets—“markets where participants trade contracts whose payoffs are tied to a future event, thereby yielding prices that can be interpreted as market-aggregated forecasts” https://paperpile.com/c/FALrBy/U8Bk. | |
| Quantitative probability judgements and forecasting1 — eliciting quantified estimates (e.g., probabilities of an event occurring) from either scientific, technical, or forecasting experts, or from a crowd https://paperpile.com/c/FALrBy/3qOi | |
| Use of systematic reviews — any intervention related to the use of systematic reviews (reviews with reproducible methods for finding and synthesising researcher) in decision-making https://paperpile.com/c/FALrBy/60B2 | |
| Access to evidence — any intervention related to the access or convenience of access to evidence https://paperpile.com/c/FALrBy/GaBh | |
| Knowledge maps — A visual representation of organisational knowledge, including how each component of that knowledge is connected to other components https://paperpile.com/c/FALrBy/JTab | |
| Visualisation — visual representations of information (e.g., figures, multimedia) designed to complement written communication of key messages https://paperpile.com/c/FALrBy/a66J | |
| Executive or evidence summaries — brief, clear summaries of key, decision-relevant information https://paperpile.com/c/FALrBy/L6vU/?prefix=e.g.%2C at the beginning of a a longer report%3B | |
| Increasing motivation for decision-quality | Accountability mechanisms — Management, supervision, or tracking of decision-making practices (e.g., quality of evidence-use) or decision-making outcomes https://paperpile.com/c/FALrBy/SQ0g |
| Incentives — performance-based commissions, bonuses, or extrinsic incentives for good decision-making https://paperpile.com/c/FALrBy/FBcx | |
| External advocacy — public pressure campaigns to make a specific decisions https://paperpile.com/c/FALrBy/GZVc/?prefix=e.g.%2C for greater transparency and ethical controls to combat principal-agent problems%3B | |
| Organisational norms — establishing and marketing good processes (e.g., use of high quality evidence) as the ‘normal’ or common practice within an organisation https://paperpile.com/c/FALrBy/Ftx1 | |
| Group composition | Group diversity — any review focused on linking decision quality and the distribution of personal attributes (e.g., gender, seniority) among interdependent members of a work unit |
| Technical competence — any review focused on linking decision quality to the selection of decision-makers on the basis of some form of technical competence https://paperpile.com/c/FALrBy/t2cz/?prefix=e.g.%2C superforecasters%3B | |
| Expertise or seniority — The seniority of the decision-makers | |
| Size—Number of people involved in the decision | |
| Identifying threats and opportunities | Risk modelling — any quantitative, qualitative, or hybrid approach that aims to produce risk estimates for the purpose of risk management https://paperpile.com/c/FALrBy/uEgU |
| Horizon scanning — a technique for detecting early signs of potentially important developments through a systematic examination of potential threats and opportunities https://paperpile.com/c/FALrBy/Cpac | |
| Scenario planning — structured process of identifying a wide range of possible futures and how the organisation might respond https://paperpile.com/c/FALrBy/ODYn | |
| Backcasting — Working backwards from a desirable future outcome to identify policies and interventions required to get there https://paperpile.com/c/FALrBy/2RPD | |
| Premortem — Assuming a decision would be a failure, then identifying the most likely reasons for why that decision have performed poorly https://paperpile.com/c/FALrBy/kxZM | |
| Brainstorming — group creativity and problem-solving methods that involve the spontaneous contribution of ideas and solutions https://paperpile.com/c/FALrBy/SOTR+iqox/?prefix=,includes other creativity tools like SCAMPER%3B | |
| Systems thinking tools — a theoretical analysis of the structures underlying a complex situation https://paperpile.com/c/FALrBy/iQoJ | |
| Comparing among options | Cost-benefit analysis — a systematic method for quantifying and comparing the total costs and benefits of a decision https://paperpile.com/c/FALrBy/sH00 |
| Multi-criteria decision analysis — a structured process that explicitly defines the criteria relevant to a decision and evaluates options against this criteria https://paperpile.com/c/FALrBy/6E8R | |
| Pros and cons analysis — list the advantages and disadvantages of each solution to facilitate comparison https://paperpile.com/c/FALrBy/KFzA | |
| Leaning useful heuristics — training to identify the key information for making specific decisions, and simple rules for deciding using that information https://paperpile.com/c/FALrBy/mbyd/?prefix=e.g.%2C ‘fast and frugal trees’%3B | |
| Deep uncertainty tools — methods for decision-making (e.g., robust decision-making, assumption based planning) for situations where decision-makers do not know, or cannot agree upon, the likelihood of different future scenarios https://paperpile.com/c/FALrBy/01Zc | |
| Game theoretic modelling — quantitative analysis of outcomes (usu., in terms of expected utility) where the decision-makers’ actions depend on the actions of others https://paperpile.com/c/FALrBy/RNhX | |
| Reducing bias or noise | Red-teaming — the practice of rigorously challenging plans, policies, systems, and assumptions by adopting an adversarial approach https://paperpile.com/c/FALrBy/xkdIk |
| Debiasing — any intervention designed to reduce bias https://paperpile.com/c/FALrBy/uEC5/?prefix=e.g.%2C education%2C adjusting estimates to account for optimism bias%2C third party referees and reviewers%2C incentives%2C nudging%3B | |
| Decision hygiene — structured methods to improve reliability of judgements across time or between people https://paperpile.com/c/FALrBy/PJIiQ/?prefix=e.g.%2C%3B using relative scoring%2C making independent assessments%3B | |
| Dialectical inquiry — structured process of challenging assumptions by developing plans using opposing assumptions, then contrasting the strength of both plans https://paperpile.com/c/FALrBy/xkdIk | |
| Supervision and challenge mechanisms — a review process for decisions in an organisation, where decisions are reviewed by people who were not involved in the original decision https://paperpile.com/c/FALrBy/aozO/?prefix=e.g.%2C someone more or less senior%3B | |
| Time or effort — allow for increased time, or expend more effort, toward deliberation, analysis, and reflection | |
| Practising and reviewing decision-making | Serious games — a mental contest where through entertaining gameplay the players’ achieve a learning purpose built into the game https://paperpile.com/c/FALrBy/VAGk |
| Crisis training — the training of decision-makers using experiential learning from practice crisis scenarios https://paperpile.com/c/FALrBy/MGcL | |
| Decision-review — formal methods for reviewing the outcomes from decisions to identify areas for improvement in the future https://paperpile.com/c/FALrBy/Prfw/?prefix=a.k.a.%2C retrospectives%3B postmortem%3B |