Users' Guide

CONTENTS

Effect Size Calculation tab

Illustration Options tab

Study Details tab

Analysis & Interpretation tab

Illustration tab

Forthcoming features

 

Gardner’s effect size illustrator (ESI) provides users with a tool to calculate and interpret effect size estimates. The interpretations are provided as illustrations and complementary statements. Users require quantitative data that is based on continuous outcomes. Regarding the data, it should be reasonable to assume that the data are normally distributed around the mean. This is a fundamental assumption of any effect size estimate.

 

What you need before you start

 

There are two ways to use the ESI. Using Option 1, you can illustrate an effect size from one that has already been calculated (e.g., from an effect size reported in a study result). With Option 2, you can calculate and illustrate an effect size using the summary data available (e.g., number of participants, means, and any one of standard deviations, standard errors, or 95% confidence intervals). The illustrator allows you to add details to your illustration about the study, participants, and source of information. As such, having the original source available when you use the ESI will enable you to take advantage of this functionality.

 

Getting Started

 

From the Home menu you can go to the effect size illustrator by clicking the button of the same name or by clicking the ‘Effect Size Illustrator” link on the main menu. This activates the Adobe Flash Player, which is installed on 99% of personal computers and is the software behind the ESI. If you are part of the 1% without it go to http://www.adobe.com/products/flashplayer/ to download and install the player.

 

Effect Size Calculation tab

 

Once the ESI has been activated you should see the Effect Size Calculation tab’s details (shown below). You can see the other tabs’ details by clicking each tab.

Entering your data

 

Select either Option 1 (when you are using summary data to calculate, interpret, and illustrate an effect size) or option 2 (to interpret and illustrate an effect size when it has already been calculated for you).

 

OPTION 1

 

 

You have the option to select the specific effect size expression you would like to be calculated. Cohen’s d is the default as it is the most commonly used. Hedges’ g is a slight variation on the same theme in which the Cohen’s d calculation is modified slightly to adjust for biases inherent in using small sample sizes. The difference between the effect sizes calculated using the two formulae is usually quite small. Glass’s delta is appropriate when there are three or more groups being compared. In this case several effect sizes are calculated using a common standard deviation of your choosing, which is usually that of the control group of the study. Use Group 1 for the control group data to calculate Glass’s delta correctly.

 

Next enter your group names and the group sizes, mean scores, and expression of variance values. The default expression of variance is standard deviation (SD). However, you can enter the standard error (SE) if that is what is provided or you can enter a value related to the 95% confidence interval.

 

95% Confidence Interval: When a study provides the mean and its 95% confidence interval (and no SD or SE) the number to be entered is the difference between the upper and lower limit of the confidence interval. For example, if the group’s mean is 9.7 and the 95% confidence interval is 7.7 to 11.7 you would enter 4.0, which is the difference between the upper (11.7) and lower (7.7) bounds of the confidence interval.

 

Once you have entered your group names and data it is time to take a quick look at your Analysis & Interpretation tab as well as your Illustration tab. At this point you will see the calculated effect size and its illustration.

 

OPTION 2

 

Enter the effect size in the box provided and, if known, indicate the type of effect size that was calculated. Add the names of the groups being compared and, if available, the group sizes.

 

Compared with option 1, this option is limited in what additional information can be added to the illustration. For example, the scale cannot be added. However, the illustration does provide an accurate visualization of the overlap in distributions between groups and the percentile ranking in one group that is equivalent to the average person in the other group based on the effect size provided.

 

Other Features

 

To change the colour of your illustrations click the border and fill squares for each group.

 

To add a third or more groups you must use Option 1. This can be accomplished by clicking the “+” button. You will notice when you do this that “Group 2” changes to “Group 3”. Enter the relevant data for the new group. To move among groups 2, 3, 4, and so on use the arrow keys (← →).

 

To delete a group’s data click the " – " key. To completely remove a group click the " – " key a second time.

 

At this point you can review the analysis and interpretation and your illustration by clicking the respective tabs and you can go on to the Illustration Options and Study Details tabs. These tabs can be used to create an appropriately labelled illustration that suits your image preferences and provides details and context regarding the data.

 

Back to Top

Illustration Options tab

 

 

It is recommended that you provide a descriptive or even declarative title to your illustration. Do so using the “Title of figure” box.

 

It is also recommended that you label your axis by filling in the Name of outcome measure box. Do this to indicate what was measured and how it was measured. For example, if a study was measuring pain using a 100 mm visual analogue scale you could label the axis as “Pain severity (100 mm VAS)”, or if the study was measuring a change in depressive symptoms using the Hamilton Depression Rating Scale you could label the axis “HDRS change score”. Indicate the units of measure when appropriate, for example if the axis is showing time you could indicate “Time (hours)” or “Time (days)”.

 

It is very important that you pay attention to the meaning of higher values as this is critical to the accuracy of the interpretation of the effect size and to how it is illustrated. The default of the system is that higher values are indicative of better health (e.g., exercise tolerance test, mini mental status exam scores). However, higher values of many outcome measures actually indicate worse health, for example total cholesterol and Hamilton Depression Rating Scale scores. Be sure to change the meaning of higher values to “worse health” when appropriate.

 

There is flexibility in the effect size illustration that you create. The default image is oriented on the y-axis (vertical) and normal distribution of data is shown using Gardner’s ornament (because it looks like a Christmas tree ornament to many). The two or more groups being compared are shown side by side. However, options are provided to switch the illustration to a typical Gaussian distribution (the normal or bell curve), to show the distribution on the x-axis, and to overlap the images.

 

The image and its interpretation can be further fine tuned with the remaining options:

 

Range of values: the default range of values is the visual representation of 99% of the calculated confidence interval of all groups. This can be changed by specifying a preferred range of values to illustrate. For example, if the mean and standard deviation values lead to negative values when illustrating 99% of the expected responses on a scale that has a minimum score of 0 (such as many symptoms scales) you can change the range of values to begin at zero (0) and to go to a reasonable maximum value. Using this function will limit the use of the “Illustrate” function described below.

 

Image girth: The default is to show the maximum girth. The distributions can be narrowed if preferred. This can be helpful to improve the visual inspection of how the groups compare.

 

Illustrate (99%, 95%, and 90%): The range of the distribution illustrated for the groups can be controlled using this option. The default is an illustration of 99% of distribution. This gives the image a truncated appearance, which it is. However, only the outlying 1% (0.5% per tail) has been eliminated. This has been done to focus the visual comparison. Under certain circumstances it may be useful to further limit the data range illustrated (e.g., 95% or 90%).

 

State percentage of overlap: A measure of how similar the two groups are in terms of the variable measured can be provided by assessing the overlap between groups. This information will be provided in the Analysis & Interpretation tab when “yes” is selected.

 

Show mean lines: You can add the mean lines for all groups in the illustration. The default is to show the mean line of the healthiest group relative to the other groups.

 

You should experiment with these illustration options to create the image you find to be most acceptable.

 

Back to Top

 

Study Details tab

 

 

This tab allows you to add important details about the source of information. What you add to this tab will be included in the Analysis & Interpretation tab. The value of including these details will be more obvious in a future version of the ESI when the “save” and “search” functionality has been added. At this time, you can use the “select all” function to copy and paste the Analysis & Interpretation summary if wishing to save or disseminate this information.

 

Citation: It is recommended you provide useful details about the source of information, for example the study’s citation.

 

What was studied?: Use these boxes to provide details about what was studied. Include what each group was treated with or exposed to. For example, for treatment studies involving medications you could provide the drug name and dosing information (e.g., average dose, dose range, etc.).

 

Who was studied?: Provide a description of who was studied. Provide the most relevant information so that it is clear who the information applies to (e.g., age, diagnosis, concurrent therapies, etc.).

 

Duration of study: Indicate the duration of the study

 

Study design: Indicate if the information comes from an RCT, meta-analysis, or another study methodology.

 

Other study design features: Provide other important design and analysis details (e.g., blinding, cross-over, pre-/post- analysis, intention-to-treat vs. per protocol analysis, etc.).

 

Back to Top

 

Analysis & Interpretation tab

 

 

An example of the analysis and interpretation output is provided above when three groups are being compared. The output has been truncated in this example. However, it shows how the various details included in the previous tabs will be included in the effect size analysis and interpretation. To see the full interpretation scroll down using the bar on the right.

 

Back to Top

Illustration tab

 

Several illustrations are provided showing the various possible formats of the effect size illustrations. Each is based on a single trial involving three groups. The default illustration shows Gardner’s ornaments and has an interactive component. Click on each ornament to see how each relates to the others in terms of the average person in one group compared to the other group. Percentages do not appear if you are using the “overlapping” option.

 

 

 

 

Export: you may wish to use your effect size illustration for various purposes (in a publication or presentation). This can be accomplished by exporting the image as a JPEG. An alternative is to do a screen capture of the image which can then be pasted and cropped in a word processing or presentation file.

 

Back to Top

Forthcoming features

 

Save/Search: We plan to build in save and search features such that you can save your illustrations and later search for it or for illustrations prepared by other uses that are saved to the database. This can be useful if you create an illustration of a highly publicized study or if you want to share your contribution with colleagues. Over time this tool can serve as a repository of effect size illustrations.

 

Print: We plan to build in a print feature in the future.

 

Do you have any other suggested features you would like to see? Don’t hesitate to let us know. See Contact Us.