Standards for NREPP Ratings
CDE's Minimum Standards for NREPP Ratings
All interventions reviewed for the National Registry of Evidence-based Programs and Practices (NREPP), regardless of the ratings, are included in NREPP. To be included on the Science-Based Programs List, programs registered on NREPP must report at least one behavioral outcome related to tobacco, alcohol, or other drug use; disruptive behavior; or violence from experimental or quasi-experimental research that meets the CDE's minimum ratings listed below on all criteria.
Reliability (CDE's Minimum Rating = 2.0)
0 = Absence of evidence of reliability or evidence that some relevant types of reliability did not reach acceptable levels.
2 = All relevant types of reliability have been documented to be at acceptable levels in studies by the applicant.
4 = All relevant types of reliability have been documented to be at acceptable levels in studies by independent investigators.
Validity (CDE's Minimum Rating = 2.0)
0 = Absence of evidence of measure validity, or some evidence that the measure is not valid.
2 = Measure has face validity; absence of evidence that measure is not valid.
4 = Measure has one or more acceptable forms of criterion-related validity OR, for objective measures of response, there are procedural checks to confirm data validity; absence of evidence that measure is not valid.
Intervention Fidelity (CDE's Minimum Rating = 2.0)
0 = Absence of evidence or only narrative evidence that the applicant or provider believes the intervention was implemented with acceptable fidelity.
2 = Evidence of acceptable fidelity, i.e., judgment(s) by experts, systematic data collection or fidelity measure with unspecified/unknown psychometric properties.
4 = There is evidence of acceptable fidelity from a tested fidelity instrument shown to have reliability and validity.
Missing Data and Attrition (CDE's Minimum Rating = 2.0)
0 = Missing data and attrition were taken into account inadequately, OR there was too much to control for bias.
2 = Missing data and attrition were taken into account by simple estimates of data and observations, or by demonstrations of similarity between remaining participants and those lost to attrition.
4 = Missing data and attrition were taken into account by more sophisticated methods that model missing data, observations, or participants, OR there were no attrition or missing data needing adjustment.
Potential Confounding Variables (CDE's Minimum Rating = 2.0)
0 = Confounding variables or factors were as likely to account for the outcome(s) reported as were the hypothesized causes.
2 = One or more potential confounding variables or factors were not completely addressed, but the intervention appears more likely than these confounding factors to account for the outcome(s) reported.
4 = All known potential confounding variables appear to have been completely addressed to allow causal inference between intervention and outcome(s) reported.
Appropriateness of Analyses (CDE's Minimum Rating = 3.0)
0 = Analyses were not appropriate for inferring relationships between intervention and outcome, OR sample size was inadequate.
2 = Some analyses may not have been appropriate for inferring relationships between intervention and outcome, OR sample size may have been inadequate.
4 = Analyses were appropriate for inferring relationships between intervention and outcome. Sample size and power were adequate.
Application of Minimum Standards for NREPP Rated Programs
All programs rated by NREPP will be assessed for the Science-Based Programs List based on two sets of criteria:
1) program requirements and
2) research ratings for behavioral outcomes.
To meet the minimum program requirements to be considered for inclusion on the Science-Based Programs List, programs must
a) be a prevention program,
b) target one or more alcohol, tobacco, other drug, disruptive behavior, and/or violence outcomes, and
c) target students in one or more grades K-12.
Minimum research ratings for behavioral outcomes related to alcohol, tobacco, other drug, disruptive behavior and/or violence from experimental or quasi-experimental research are outlined above.
The following NREPP rated programs have been reviewed for inclusion on the Science-Based Programs List. Summaries (PDF file) of the analysis for inclusion on the Science-Based Programs List can be downloaded from the chart below.
Last updated: 04/23/08
|
