ENHANCE: Research on COS Data Quality
ENHANCE, a research project designed to improve the quality of child outcomes data, consisted of four studies examining the quality of the data being collected through the Child Outcomes Summary (COS) process. The Provider Survey, Team Decision-Making Study (Video Study), and the Child Assessments Study were conducted in three local early intervention programs in each of six states, and in three early childhood special education programs in each of six states, for a total of 36 local programs who participated in the project.
The Extant State Data Study, conducted at the state level, examined patterns in statewide data. ENHANCE helped to understand the conditions under which the COS process produces valid and reliable data. Many states use this process to generate data for federal accountability reporting for Part C and Part B Preschool services of the Individuals with Disabilities Education Act (IDEA). The data can also be used for program improvement by identifying program strengths and weaknesses.
Findings
On the basis of evidence collected across four studies, we concluded that when implemented as intended, the COS process produces ratings that are valid for accountability and program improvement purposes. The following are key findings supporting this conclusion:
- Providers understood the types of behaviors included in each of the three child outcomes.
- Providers could accurately apply their knowledge of child development and the COS rating criteria.
- The COS process could be incorporated into existing practice without negative consequences.
- With a few exceptions, children who were rated higher on the COS also scored higher on assessment tools.
- COS ratings were related to the child’s functional abilities and type of disability.
- Children who entered EI and ECSE with higher COS ratings tended to exit the programs with higher COS ratings.
- Most states had stable percentages of children making greater than expected growth or exiting at age expectations over time as measured by the COS.
Another key finding was that some of the programs studied did not always implement the COS process as intended. Details on the findings can be found in the ENHANCE Full Report or ENHANCE Brief.
Considerations and Limitations
- Data for these four studies were collected in 2010 through 2013. Some of the data were collected fairly early in the implementation of the COS process.
- Data were drawn from a limited number of districts and states and were not representative of all places implementing the COS process.
- The programs participating in the study had variable implementation of the COS process. Better implementation, however, most likely would have produced even stronger evidence of validity of the data from the COS process.
Data Collection Tools
See also: Data Quality
Presentations and Reports
- ENHANCE Full Report (2015)
- ENHANCE Brief (2016)