Guiding Principles

  1. The first thing you should see on the summary is the 1–3 key options for implementation

  2. Headings should not be questions but answers; they should be spoilers, summaries of what is in the paragraph

  3. Break summaries up into separate summaries for discrete interventions (not "nudges" but "change defaults" and "gain framing") so the titles are more informative and the ratings are easier to do

  4. Add details without mainstream appeal in /toggles

    Some people struggle with toggles, so it's useful to add the instruction [click arrow to expand] the first few times

  5. Remembering that our core goal here is to help people decide (not necessarily implement, at this stage; implementation is a long-term goal)...

    1. First summarise the results and our confidence in those results
    2. THEN spend some time looking up good existing resources about how to implement them
    3. THEN create some ways of self-reflecting and self-assessing your knowledge (MCQ) / implementation (e.g., self-rating likert question)
    4. THEN maybe possibly later create resources of good and poor practice

Style Guide

  1. Meaningful noun project icons for each summary
    1. [email protected] ^z7^B9x5Z5
    2. Colour noun project icons EA dark teal: #0AA9BF
  2. We want to break up slabs of text with headings and a line break '- - -' (no spaces)
  3. Use colour meaningfully to attract attention, especially red for bad effects and teal for good ones. Use EA teal for highlighting something important.
  4. If your parentheses are asymmetrical (➕➕➕), you will be hurdled from the ivory tower. Emoji's do funny things, so add a few spaces until it looks right: (➕➕➕ )

How to code the impact of findings

Our aim

We want to be able to communicate how effective each strategy in our evidence toolkit is so educators can quickly determine what does and doesn't work.

How we do it?

The cutoffs for and interpretations of effect sizes for this toolkit are taken from the works of Funder and Ozer. Information about these cutoffs and interpretations is below. When more than one meta-analysis is used in an evidence summary, the main effect from the meta-analysis that was rated as having the highest quality should be reported.

What about negative effects?

For negative effects, replace the ➕ icons with the equivalent number of ➖ icons (e.g., if d = -0.2, report ➖ ➖ ).

Converting effect sizes

When interpreting a meta-analysis that reports coefficients other than Cohen's d, use the effect size conversion calculators below.

Hedge's g is typically interpreted using the same criteria as Cohen's d and, as such, can be interpreted using the information below.

Cutoffs and interpretations

➕ Indicates a likely inconsequential effect in the short and long run (Cohen's d < 0.1)

➕ ➕ Indicates an effect that is very small for the explanation of single events but potentially consequential in the not-very long run (Cohen's d = 0.1)

➕ ➕ ➕ Indicates an effect that is still small at the level of single events but potentially more ultimately consequential (Cohen's d = 0.2)

➕ ➕ ➕ ➕ Indicates an effect of medium size that is of some explanatory and practical use even in the short run and therefore even more important (Cohen's d = 0.4)

➕ ➕ ➕ ➕ ➕ Indicates an effect that is large and potentially powerful in both the short and the long run (Cohen's d > 0.6)

https://www.escal.site/