Early Childhood Implementation Snapshots, No. 5: Data and Evaluation Systems
- June 16, 2021
- by Sherri Britt Williams
The Data and Evaluation Systems structure cuts across the other essential support structures providing and organizing measurement tools and procedures used for data-based decision making within each structure. This structure documents implementation progress and fidelity and the impact of implementation and related support on program outcomes. The data generated is critical to ongoing planning and decision making throughout the initiative.
The Tip Sheet Series: Evaluating the Implementation of Evidence-based Practices is a helpful resource to consider when developing and improving the Data and Evaluation Systems structure. The series provides guidance on what to measure, characteristics of a high-quality measurement tool, establishing a fidelity threshold, and summarizing data for decision making.
Voices from the Field: Wisconsin
Wisconsin collaborated with the Pyramid Model Consortium (PMC) to pilot the Pyramid Model Implementation Data System (PIDS). PIDS was developed by the New York State Council on Children and Families and State University of New York Center for Health and Social Research (SUNY CHSR) with federal Preschool Development Grant (PDG) Birth to Five funding. PIDS holds all Pyramid Model data including social emotional screenings, coaching observations using the Teaching Pyramid Observation Tool (TPOT) and Teaching Pyramid Infant-Toddler Observation Scale (TPITOS), Benchmarks of Quality assessments, and coaching logs. In addition to being used to plan and improve at the program level, program data are shared with Wisconsin Alliance for Infant Mental Health (WI-AIMH) Pyramid Model Coordination in order to report effectiveness, identify professional development gaps at the community, regional and state level, and advise on the development of resources at the state level. Massachusetts, Tennessee and Iowa have all committed to using PIDS based on New York and Wisconsin's efforts. To learn more, watch the PIDS Quick Overview video about Wisconsin's Pyramid Implementation Data System and be sure to review the Pyramid Information Data System (PIDS) Resource List:
"The Pyramid Implementation Data System (PIDS) is able to record all the data, synthesize it, and display it in graph form. It lets you use the data almost instantly."
Considerations for Equity
- Data systems that disaggregate data by race, ethnicity, gender, financial resources, etc. help to identify disparities. Engage stakeholders representing the disaggregated groups in interpreting the data to provide a deeper understanding of the context from lived experience.
- Data analysis that includes investigating the context for and systemic inequities that impact the data helps to provide more accurate impressions, conclusions, and recommendations for improvement. Consider the conditions that allow the identified issue or concern to persist to identify systemic and societal root causes.
- Representativeness gives a more accurate and comprehensive view of the landscape. Consider which voices are missing from survey and feedback data and identify strategies for developing a better understanding of why their perspectives are underrepresented in data. Partner with leaders identifying with underrepresented groups to identify ways to engage missing voices more meaningfully and successfully.
- Statewide Implementation Guide. (2019). ECTA Early Childhood Technical Assistance Center. https://ectacenter.org/sig
The contents of this page were developed under cooperative agreements from the U.S. Department of Education, #H326P17001 (ECTA Center), and #H326B170003 (NCPMI), from the Office of Special Education Programs, U.S. Department of Education. However, these contents do not necessarily represent the policy of the U.S. Department of Education, and you should not assume endorsement by the Federal Government.