The online version provides hot links to the sections of the Primer that explain the concepts involved. Clicking on any green term takes the user to a new window to show the Primer section that discusses the term and related concepts in more detail. Holding the cursor over a term in green that also is italicized will cause a definition of the term to appear. The online version also computes the Assessment Guide scores automatically.
Although the Assessment Guide actually can be used to score the usefulness of a research report in developing policy, it is not intended to provide a precise measure. Moreover, policy changes always should be made in the light of the entire body of evidence and not on the basis of a single study. The real value of the Quick Primer is as a heuristic tool and an applied approach to the Primer for policymakers and others who have limited time or learn best “on the job.” Also, keep in mind that although the Quick Primer will enable the user to gain a basic understanding of the concepts involved, it will not give the same depth of understanding as reading the complete Primer. Moreover, the Quick Primer does not introduce the user to all of the material covered in the larger Primer.
Assessment Guide Scoring Directions
Answer the questions in numerical order. For each numbered question, if the answer is yes, score a plus (+) in the right-hand column. If the answer is no, score a minus (-) in the right-hand column. If uncertain about how to answer the question, indicate a question mark (?) in the right-hand column. If the directions say to STOP, it means a fatal error has occurred and the research should not be used for policymaking.
If you haven’t reached a point in the Guide instructing you to STOP or PAUSE – an indication the study you’re reviewing is unhelpful or likely seriously flawed – count the number of pluses, minuses and question marks, and use the following rough guidelines for interpretation:
All Pluses – This study is highly useful for making policy decisions.
Majority Pluses – This study is useful for making policy decisions. Examine the characteristics that received the fewest pluses. Consider how the lack of those characteristics could affect the study’s usefulness.
Equal Pluses and Minuses – This study has limited usefulness for making policy decisions. Examine the characteristics that received minuses. Consider how the lack of those characteristics could affect the study’s usefulness.
Majority Minuses – This study is not useful for making policy decisions. Look for other empirical research related to the topic before using this study.
Majority Question Marks – If the question marks are due to a lack of information provided in the study, consider contacting the researcher for that information. If the question marks are due to a lack of understanding about the study characteristics and/or an inability to judge their presence in the study, consult this Primer, other resources on education research or seek the help of an education researcher.
Click here to launch the online version of the Assessment Guide.
