Statistical Information Regarding Effect Size

The concept of effect size was developed to facilitate comparisons of efficacy across studies (e.g., meta analysis), but it is also commonly used to express treatment effect in single studies. A commonly used effect size measure calculates the difference between 2 groups (e.g., drug vs. placebo or drug A vs. drug B), adjusted for the variance in response within groups. Required for this calculation are: a) continuous data, b) means, and c) standard deviations. Adjusting for the variance within groups is accomplished by dividing the difference between the groups by a specific measure of variance, the most common one being the pooled standard deviation. The resulting effect size is then unitless. As such, a researcher or clinician can compare different effect sizes that are measures of different outcomes using different measurement scales.

 

Effect sizes can also be thought of as the percentile standing of the control group participant (e.g., treatment group A) who scores similarly to the average experimental group participant (treatment group B). An ES of 0.0 indicates that the mean of the experimental group is at the 50th percentile of the control group's average response. An ES of 0.8 indicates that the mean of the experimental group is at the 79th percentile of the control group. An effect size of 1.7 indicates that the mean of the experimental group is at the 95.5 percentile of the control group. These percentiles come from the 'normal distribution'. An effect size is exactly equivalent to a 'Z-score' of a standard Normal distribution.

 

The following able shows conversions of effect sizes to percentiles (I1) (based on the assumption that scores are normally distributed) and the equivalent change in rank order for a group of 25 (I2).

Effect size1 Percentage in one group (e.g., group X) who are doing less well than the average person in the other group (e.g., group Y)2 Effect size1 Percentage in one group (e.g., group X) who are doing less well than the average person in the other group (e.g., group Y)2
Cohen’s d Hedges’s g Cohen’s d Hedges’s g
0.00 50.0 50.0 0.85 80.0 80.2
0.05 52.0 52.0 0.90 81.3 81.6
0.10 53.9 54.0 0.95 82.6 82.9
0.15 55.9 56.0 1.00 83.9 84.1
0.20 3 57.8 57.9 1.10 86.2 86.4
0.25 59.8 59.9 1.20 88.3 88.5
0.30 61.7 61.8 1.30 90.1 90.3
0.35 63.6 63.7 1.40 91.7 91.9
0.40 65.4 65.5 1.50 93.1 93.3
0.45 67.2 67.4 1.60 94.3 94.5
0.50 3 69.0 69.2 1.70 95.4 95.5
0.55 70.7 70.9 1.80 96.3 96.4
0.60 72.4 72.6 1.90 97.0 97.1
0.65 74.0 74.2 2.00 97.6 97.7
0.70 75.6 75.8 2.25 98.7 98.8
0.75 77.1 77.3 2.50 99.3 99.4
0.80 78.6 78.8 3.00 99.8 99.9

1 Effect size estimates in this table are based on a standardized sample size of 50 people per group. The difference between Cohen’s d and Hedges’ g percentiles increases slightly as the sample size decreases. For example, when the samples are 20 per group the percentiles for an effect size of 0.8 for Cohen’s d and for Hedges’ g are 78.2% and 78.8%. respectively.

 

2 It can also be stated that the effect size indicates the percentage in one group (e.g., group Y) who are doing better than the average person in the other group (group X). Note that the average person represents the 50th percentile of that group and that distributions are assumed to be normal such that the mean and median are essentially equal.

 

3 With significant reservations and pleas not to use them, Cohen offered that 0.2, 0.5, and 0.8 might denote small, medium, and large effect sizes. However, we recommend that these terms not be used. Instead, use the effect size illustrator and its provided interpretation along with your appraisal of the source of data and other evidence to determine the importance of the effect size calculated.