Data and Evaluation Systems
The Data and Evaluation Systems are one of four essential support structures:
Overview
The Data and Evaluation Systems are a collection of measurement tools and procedures used for data-based decision making. They are embedded in every structure and stage of the initiative.
Purpose
The Data and Evaluation Systems are used to document and ensure quality, high-fidelity implementation of practices. The data generated is critical to ongoing planning and decision making throughout the initiative.
Data and Evaluation Systems will:
- show if the State Leadership Team (SLT) is meeting the benchmarks for supporting and guiding statewide implementation of evidence-based practices
- show if practitioner coaches are effective in improving practitioners' fidelity in performing evidence-based practices
- show if a program is meeting the benchmarks established for program and practice implementation fidelity
- show if a program is achieving its desired outcomes
- guide the Program Leadership Team (PLT) and SLT in making decisions
Watch a Learning Lab session on using data for making decisions.
Logistics
SLT, PLT, and PDN collect data on progress and performance across the improvement effort and over time. This data is used to identify and evaluate implementation needs and outcomes. The Tip Sheet Series provides guidance for collecting and analyzing evidence-based practices implementation data. The Look-Think-Act Model is used to examine data for trends and meaningful associations, interpret the data, and determine future actions.
Data collection activities include:
- SLT reviews existing program measurement instruments and selects or creates measures
- SLT establishes data collection and reporting procedures for tracking and analyzing evaluation data
- PDN develops procedures and training in using evaluation tools
- SLT and programs decide: who will collect the data, how and when it will be collected, how the results will be displayed
- SLT, PDN, and PLT collect data on progress and performance across the improvement effort and over time
Evaluating data activities include:
- SLT guides their system improvement efforts specific to implementation infrastructure with Data Decision Making: Benchmarks of Quality: State Leadership Team
- PDN and PLT use Data Decision Making: Coaching Log to examine allocation and impacts of coaching activity and identify action steps
- SLT, PLT, and PDN could use an adaptation of Data Decision Making: Reaching Potentials through Recommended Practices Observation Scale: RP2 OS-C and RP2 OS-HV to determine if practices are being implemented with fidelity by site personnel and identify action steps
- PLT uses Data Decision Making: Benchmarks of Quality: Program-Wide in Centers or Home Visiting Program to examine the site's program implementation fidelity and identify action steps
- PLT and SLT use Data Decision Making: Child Outcomes to examine trends and potential influences on the data and identify action steps
- SLT prepares an annual report of data to share with the public, funders, and policymakers to market the impact of the initiative as well as garner support and resources.
Tools
See: Tools