Frequently Asked Questions About Outcomes Measurement

An "outcome" is defined as a benefit experienced as a result of services and supports received. Thus, an outcome is neither the receipt of services nor satisfaction with services, but rather what happens as a result of services provided to children and families.

• Child Outcomes: If an autistic child who receives therapy services to increase communication skills, receiving those services is not an outcome. However, if the child learns words they can use to convey their needs to others, that is a Child Outcome.
• Family Outcomes: If a family receives information about their child's disability, providing this information is a service, but not an outcome. However, if the information enables them to assist in their child's learning and development more effectively, that is a Family Outcome.

In this age of accountability, policymakers are asking questions about the outcomes achieved through participation in programs supported by public funds. Judging the effectiveness of any program requires looking at results, not simply at the process of providing services.

The Office of Special Education Programs (OSEP), U.S. Department of Education requires states to report outcomes data for children and families served through Part C and Part B Preschool of the Individuals with Disabilities Education Act (IDEA) as part of their Annual Performance Report (APR).

Additionally, many states use child outcomes data and family outcomes data to help analyze and improve their programs.

See also: Part C SPP/APR, Part B SPP/APR

States must report the percentage of infants and toddlers with Individualized Family Service Plans (IFSPs) or preschool children with Individualized Education Programs (IEPs) who demonstrate improvement in three child outcomes, which address areas of child functioning necessary for each child to be an active and successful participant at home, in early care and education settings, and in the community:

• Outcome 1: Positive Social-Emotional Skills, which refers to to how a child relatee to and get along with adults and with other children, how a child expresses emotions and feelings and how he or she interacts with and plays with other children. For toddlers and preschoolers, these skills also include how children follow rules related to interacting with others in group situations—for example, a child care center.
• Outcome 2: Acquisition and Use of Knowledge and Skills, which refers to the child's ability to think, reason, remember, solve problems, and use symbols and language. The outcome also encompasses the child's understanding of physical and social worlds. It includes understanding of early concepts (for example, symbols, pictures, numbers, classification, spatial relationships), imitation, object permanence, the acquisition of language and communication skills, and of early literacy and numeracy skills. Outcome 2 also addresses the precursors that are needed so that children will experience success later in elementary school when they are taught academic subject areas (for example, reading or mathematics).
• Outcome 3: Use of Appropriate Behaviors to Meet Needs, which refers to the actions that children employ to take care of their basic needs, including getting from place to place, using tools (for example, fork, toothbrush), and in older children, contributing to their own health and safety. The outcome includes how children take care of themselves (for example, dressing, feeding, hair brushing, toileting), carry out household responsibilities, and act on the world to get what they want. This outcome addresses children's increasing capacity to become independent in interacting with the world and taking care of their needs.

See also: Video: Child Outcomes Step-by-Step

It is true that knowledge and skills are the basis of functioning in each of the three child outcomes.

Outcome 1: Positive Emotional Skills encompasses the knowledge and skills required to get along with others.Outcome 3: Use of Appropriate Behaviors to Meet Needs includes the knowledge and skills needed to move, act, or communicate needs.

However, Outcome 2: Acquisition and Use of Knowledge and Skills refers to the specific set of knowledge and skills that lay the foundation for more formal learning later in school settings (acquiring general knowledge, thinking, reasoning, problem solving, and learning new vocabulary words).

A functional outcomes describes a child's mastery and appropriate application of behaviors, knowledge, and skills in a meaningful way in their everyday lives.

The three child outcomes refer to actions that children need to be able to carry out, and also knowledge that children need to use in order to function successfully across a variety of settings. To be successful in these settings, it is important for children to be able to get along with others, follow the rules in a group, continue to learn new things, and take care of their basic needs in an appropriate way. Ultimately, achieving these outcomes will help children thrive at home, in school, and in many settings throughout their communities.

Many assessment tools examine children's development in domains, which describe children's skills and abilities within areas of development such as "social", "fine motor", "gross motor", "cognitive", or "language". The skills and abilities described by these kinds of domains are a necessary—but not sufficient—component of functioning within the routines and activities of early childhood like toileting, feeding, and playing with peers.

Functioning within routines and activities requires the integration of skills across the various domains. For example, playing with peers requires a social desire to play with peers, and expressive communication to initiate, maintain, and direct the peers' attention. It also requires cognitive skills to sequence actions in play, and fine and gross motor skills to manipulate objects. Functional outcomes look at the integration of behaviors across domains that children need to participate in developmentally appropriate routines and activities.

The three child outcomes focus on functioning rather than traditional domains.

Crosswalks show which items or areas on some of the most commonly used assessment tools in Part C and Part B Preschool address each of the three outcomes. They were developed because most currently available assessment tools are organized around domains, which are not the same as functional outcomes. Crosswalks are not meant to be used as a assessment strument, or a checklist to determine a child's level of functioning in an outcome area.

States are required to compare the functioning of children in Part C and Part B Preschool programs to age expectations, because age expectations provide a common standard for all young children. To merely record that children made progress between entry and exit would not provide strong evidence for the effectiveness of the program. Also, one of the goals of early childhood services is to prepare children to succeed in kindergarten and, in kindergarten, children will be expected to meet grade-level standards.

OSEP recognizes that not all children will be able to function comparable to same age peers at the end of early childhood services, but the system is tracking how many have achieved or moved closer to functioning at an age-expected level.

There are several approaches states use to measure child outcomes. Most use the Child Outcomes Summary (COS) Process.

See also: State Approaches to Child Outcomes Measurement: Part C APR Indicator 3 and State Approaches to Child Outcomes Measurement: Part B APR Indicator 7 (Section 619/Preschool)

The skills children master at different ages can be measured and described so that we can identify children who are developing too slowly. Children who are substantially behind their peers have developmental delays. OSEP has established five progress categories to convey developmental trajectories indicative of developmental delay:

1. Did not improve functioning
2. Improved in functioning, no change in trajectory
3. Moved closer to functioning like same-aged peers
4. Improved functioning to that of same-aged peers
5. Functioning like same-aged peers

To report data in these five categories, states must have information about children's functioning at entry and exit, and have a way to examine the level of improvement or progress over the duration.

Additionally, two "Summary Statements" are calculated for each of the three child outcomes:

• Summary Statement 1 is the percentage of children who made greater than expected growth.
• Summary Statement 2 is the percentage of children who exited at or above age expectations

States must set targets for each Summary Statement, and report on their progress toward that target. The data collected are aggregated to create an overall view of the progress made by all children in the state who received Part C or Part B Preschool services. No identifiable data on any particular child are included in these reports. Although only entry and exit data are reported at the federal level, many states obtain this information at more frequent intervals to support statewide decision-making, program planning, and program improvement.

Yes. The Office of Special Education Programs (OSEP) requires that states provide data on all three outcomes for every child leaving Part C or Part B Preschool services. Data are required for all outcomes even if there are no concerns about a child's development in an outcome, or if the child has delays only in one or two outcome areas. This includes children receiving only a single service such as speech therapy.

For states sampling children from their Part C or Part B Preschool population for outcomes reporting, data on all three outcomes must be provided for every child being sampled.

Children with severe disabilities might never achieve functioning comparable to their same age peers, but they frequently make progress with the assistance of early intervention and preschool special education programs. Children who make progress but have not moved closer to functioning like same aged peers are counted in category "b" for the Office of Special Education Programs (OSEP) reporting purposes. They are counted as children who made progress. Even small steps of individual progress—the very things we strive for and celebrate for children with more severe impairments—count as progress for category "b."

The data reported by states indicates that only a small percentage of children—less than 2% in nearly all states—show no progress between program entry and exit on one of the three outcomes. For their own purposes, it is helpful for states to be able to describe the types of children who are reported in each of the Office of Special Education Programs (OSEP) "a–e" categories, especially those in category "a".

Additionally, many states collect additional information—or link child outcomes data to existing information—so that they can understand more about how child and family outcomes are related to key child, family, and service characteristics.

Using data already reported by states, the percentage of children who left Part C programs because they reached their goals is calculable. However, that doesn't provide any data on children exiting from Part B Preschool, nor any data on the progress of those who transitioned into special education programs. A more complete picture of the effectiveness of the programs requires looking at the progress made by all children who received services.

No. States are expected to enter those data.

States must provide the number of infants and toddlers who exited the Part C program during the reporting period, as reported in their IDEA Section 618 Part C Exiting data.

Yes. Generally, the total number of children assessed should be equal to the total number of children exiting, minus the number of children exiting with less than six months service. A state should provide "additional information" in the indicator if the total number assessed is not what would be expected.

Yes. Generally, the total number of children assessed is expected to be the same as the denominator for Summary Statement 2. A state should provide "additional information" in the indicator if the total number assessed is not what would be expected.

Yes, yes, and yes:

• States with approved policies and procedures to implement the IDEA Part C Extension Option should report children continuing in Part C in the IDEA Section 618 Child Count data and the Part C Exiting data for the year in which they turn 3 years of age.
• Those children are reported under the "continuing in Part C reporting category for the Part C Exiting data.
• States also should report outcomes data on those children in APR indicator C3 during the year in which they turn 3 years old. States are expected to report outcomes data for those children again when they ultimately exit Part C services.

Yes. Children receiving Part C services under the Part C Extension Option should be reported in the 618 Child Count data.

No. Children should only be reported in the Part C Exiting data for the year in which they reach 3 years of age. Children older than age 3 exiting the Part C Extension Option are not reported in the IDEA Section 618 Part C Exiting data.

Yes. Children ages 3–5 who have continued in Part C services under the Part C extension option are reported in the APR indicator C3 (child outcomes) data during the year in which they exit Part C services.

For Part C, states are required to report the percentage of families participating in Part C who report that early intervention services have helped the family:

1. Know their rights;
2. Effectively communicate their children's needs; and
3. Help their children develop and learn.

For Part B Preschool, states are required to report the percent of parents with a child receiving special education services who report that schools facilitated parent involvement as a means of improving services and results for children with disabilities. States also may choose to collect additional information about family outcomes that resulted from the services and supports received in Early Intervention (EI) and Early Childhood Special Education services (ECSE).

There are several approaches states use to measure family outcomes and parent involvement. In some states, surveys completed by families are used to capture information about how much services have helped families and the outcomes families experience as a result of services. Having accurate information about family outcomes and experiences allows programs to improve their services.

See also: Family Outcomes Measurement: Part C APR Indicator 4

States are expected to analyze their data and determine the extent to which the demographics of the infants or toddlers for whom families responded are representative of the demographics of infants and toddlers receiving services in the Part C program. Data analysis is not based on the wider state population or the subset of families surveyed. This includes comparisons across race and ethnicity groups and a second demographic identified by the state (for example, home language or socio-economic status). States must describe the metric used to determine whether the data are representative or not.

If the data are not representative, the state is expected to:

• describe their efforts to understand why a certain subpopulation is not responding (for example, the state might analyze the data collected, conduct focus groups, or collect other data to better understand why a subpopulation is less likely to respond); and then
• report strategies the state is using to increase low response rates from the specific subpopulation that resulted in low representation.

Such strategies should be connected to understanding why those families are not responding as described above, for example, a state might develop an online survey option if feedback from non-responding families indicates that a paper survey is cumbersome, or begin translating their survey into other languages to increase response rate.

Please contact staff at the ECTA Center if you wish to adapt the survey.

Translations of both versions of the Family Outcome Surveys are available in several languages. If you have developed, or are interested in developing translations of these tools in additional languages, contact Thomas McGee.

In addition to the required family data, the ECO Center recommended that states measure family outcomes. As a result of the ECO stakeholder process, the following five outcomes were identified as desired outcomes for all families participating in early intervention:

1. Families understand their child's strengths, abilities, and special needs.
2. Families know their rights and advocate effectively for their child.
3. Families help their child develop and learn.
4. Families have support systems.
5. Families access desired services, programs, and activities in their community.

Consensus was achieved for all five of these outcomes as relevant for families participating in Part C early intervention services. For families with children receiving preschool special education services, there was consensus that outcomes 1, 2, and possibly 3 apply. Consensus was not reached as to whether outcomes 4 and 5 apply to families receiving services through preschool programs.