Data Makes
the Difference
With School Reform

by Victoria L. Bernhardt

Schools that utilize information about
their school communities can better
institutionalize systemic change.

Just about every school in every state throughout the nation is attempting to reform, restructure, reengineer or rethink the business of "school." What separates the schools that will be successful in their reform efforts from the ones that won't is the use of one, often neglected, essential element-data.

Schools that utilize and analyze information about their school communities make better decisions about what to change and how to institutionalize systemic change. Schools that understand the needs of their clientele-the students-are more successful in implementing changes and remain more focused during implementation. Additionally, schools that use data understand the effectiveness of their reform effort, those that don't can only assume.

Schools that are committed to improvement must analyze existing data. They also must collect and analyze additional data in order to understand:

The current and future needs of the school, students, parents, teachers and the community
How well the current processes meet these clients' needs
The ways in which the school and community are changing
The root causes of problems
The types of educational programs and expertise needed in the future

The types of data that assist schools in planning and sustaining systemic reform include demographics; attendance/enrollment; drop-out/graduation rates; assessments of current teaching practices; teachers', students' and parents' perceptions of the learning environment; student-achievement data; problem analyses; and ongoing assessments of progress.

The importance of data

We in education have a history of adopting one innovation after another as they are introduced. Very few of us, however, take the time to understand the needs of the children we serve, the impact that our current processes have on children, root causes of recurring problems and how to measure and analyze the impacts after we implement new approaches.

The use of data can make an enormous difference in school reform efforts, improving school processes and student learning. Data can help us replace hunches and hypotheses with facts concerning what changes are needed; identify the root causes of problems, so we can solve the problem and not the symptom; assess needs to target important issues; know if goals are being accomplished; determine if we are walking our talk; understand the impact of efforts, processes and progress; and continuously improve all aspects of the learning organization.

One small, rural Northern California community learned a valuable lesson regarding the difference between hypotheses and fact while investigating why the majority of their high-school graduates dropped out of college before the end of their first year. The community learned that they could have spent a great deal of money and time "solving a symptom," maybe never getting to the real issue-the quality of their academic programs.

Each year, for several years, the community watched 80 percent of its high-school graduates go off to college in the fall, 40 percent return to the community by Christmas, and almost 95 percent return by the end of spring-for good. This recurring problem was discussed widely among teachers and the community. Their hypothesis was that the problem centered around the students' lack of experience and social skills. Their students simply did not have the social skills to function in other environments. Also, everyone knew that their students did not interact positively with people they knew, so they could not possibly know how to interact positively with strangers.

Based on this "knowledge," the school district began an extensive restructuring effort that centered around working with all K-12 students to develop their social and communication skills. At the request of a consultant, who was hired to " make this vision a shared community vision," the teachers reluctantly conducted a telephone survey of their graduates, asking them why they had dropped out of college. Almost without exception, graduates said the following: "They made me write. I can't write!" Based on this fact-finding survey, the focus of the restructuring effort changed immediately, and the school district began using the data on an ongoing basis.

One large Southern California elementary school learned the value of disaggregating their data to understand their students and to aid their reform efforts.

School personnel stated that their students' test scores were very low because their population included so many "limited-English-speaking children" who were keeping the school's scores at a low level. They also reported that because of these children, they could not become a math-, science- and technology-magnet school.

The standardized achievement tests that had been given during the previous five years were analyzed. It was clear that while the reading and writing scores of the students with limited-English-speaking abilities were lower than the other students, they outscored those students in science and mathematics. With this and other information, the teachers researched strategies that could be used to everyone's advantage.

The school became a mathematics-, science- and technology-magnet school. It utilized hands-on activities in these areas to build language competence in their targeted population, while increasing all students' science and mathematics knowledge. The test scores of the students with limited-English-speaking skills improved significantly in one year.

Another elementary school in California's Central Valley learned how to use empirical data to see that they were not walking their talk. The data provided guidance in establishing a new purpose for the school and in understanding how to reach their goals.

The teachers stated that the purpose of their school was "to prepare students for middle school." As a means of gathering data to understand how well they were accomplishing this goal, a small group of teachers went to the middle schools and high schools, and asked questions of teachers and former elementary-school students. The teachers were mortified to find that the majority of the students who left the elementary school with limited-English-speaking skills were forever tracked in special-education and/or bilingual programs that did not allow them to take college or career-preparation classes. Some had dropped out of school.

The teachers came to the consensus that the purpose of their school, which served a mostly (about 85%) limited-English-speaking community, was to do much more than prepare them for the middle school. The school needed to prepare students by giving them the language skills upon which a career in the United States would be based.

These teachers went back to their school knowing that their highest priority was to get their students speaking, reading and writing English successfully before they left elementary school. They examined the processes used to teach students English. They were teaching English to non-English-speaking children for 20 minutes each day and moving 35 children into English-speaking classrooms each year-a process they had been doing for the last four years. And, because the product of the process was the same every year, a process change was required in order to get different results.

By doubling the time spent teaching English, more than double the number of students became fluent in English. Besides learning how to measure their processes, the teachers learned the importance of tracking student performance on an ongoing basis, ensuring every student's success.

Data barriers

Schools do not ignore data deliberately. They often are just not aware of the wealth of information that can make their jobs easier (through knowing what works and what doesn't) and more satisfying (by learning how to get the results they want).

Some of the reasons schools may not use data regularly include the following:
The work culture does not focus on data.
Gathering data is perceived to be a waste of time.
Few, if any, people at schools are adequately trained to gather and analyze data, or establish and maintain databases.
There's no one to gather and analyze data.
Administrators and teachers think it's not their job to analyze data.
Administrators do not direct office workers to gather and/or analyze data.
Computer systems are outdated and inadequate.
Appropriate, user-friendly software is not available.
District personnel have antiquated job definitions and helping schools with data isn't their priority.
From the state level on down, data is not used frequently.
School personnel have only had negative experiences with data.
School personnel are threatened by data.
There are not enough examples of schools gathering, maintaining and benefiting from the use of data.

Solutions to lack of use

It is crucial to look beyond the obvious when reaching conclusions about solutions to school problems. It is also crucial to reflect on the entire system and understand how change in one part may affect other parts. One of the purposes of gathering so much information is to help schools understand surrounding issues and to find the root causes of problems. If we don't understand the root causes, we will continue to spin our wheels.

Since traditional solutions are not working to improve schools and student achievement, we need to look at new solutions for making data gathering and analysis a critical part of the school culture. Some suggestions are:
Work at the state level to make all state-gathered data available to school districts (that can then make the data readily available to schools).
Help schools learn how to dedicate resources to data-driven decision making, and show them how they can save resources by analyzing data.
Choose someone at the school to gather and analyze data.
Make it everyone's job at the school to utilize data.
Make everyone focus on students and their progress.
While in their preparation programs, train teachers and administrators to use and understand data
Make data-driven self-evaluation the personnel-evaluation process.
Utilize a comprehensive organizer, such as a school portfolio, to help make the data-gathering congruent with the school's vision.

Perhaps it is pure optimism that gets in the way of gathering and analyzing data in schools. Whatever it is that keeps us from assessing our progress and products adequately, we must listen to and gather from all sources, so we know where we are going and how we are doing.

From the examples mentioned previously, you can see that data gathering to benefit children does not require a Ph.D. in statistics. Sometimes it only requires asking the people closest to the action a question.

About the author

Victoria L. Bernhardt is executive director of the Education for the Future Initiative, sponsored by the Pacific Bell Foundation, and professor on leave from the Department of Professional Studies in Education at California State University, Chico.

Bernhardt also directs the systemic reform of schools and districts in the northernmost counties of the WestEd Region XI, federally funded Comprehensive Assistance Center and the implementation of technology throughout the learning organizations of 12 Pacific Bell Education First Demonstration sites. She is the author of The School Portfolio: A Comprehensive Framework for School Improvement, available from Eye on Education, 6 Depot Way W., Ste. 106, Larchmont, NY 10538, (914) 833-0551.