1. WHAT IS THE MET PROJECT?

The MET project is a partnership between 3,000 teacher volunteers and dozens of independent research teams. The project aims to help teachers and schools understand what great teaching looks like. Launched in 2009, the study will identify multiple measures and tools that – taken together – can provide an accurate and reliable picture of teaching effectiveness. By understanding what great teachers do and by improving the ways teachers gain insight into their practice, we can help more teachers achieve success for their students.

Research shows that a teachers' contribution matters more than anything else within a school. More than class size. More than school funding. More than technology. For decades, most initiatives to improve public education have focused on improving poor performing schools. But studies show that there are bigger differences in teaching quality within schools than there are between schools. This means that in the same school, a child taught by a less effective teacher can receive an education of vastly different quality than a student just down the hall who is taught by a more effective teacher. And the way evaluations are currently conducted don't provide the teacher who is struggling with a roadmap to improve.

Because teaching is complex, no single measure can capture the complete picture of a teacher's impact; yet many evaluation systems use tools that provide teachers with very limited, occasional feedback. Multiple measures are needed to help school leaders understand how teaching contributes to student success, because as teachers know, there are no silver bullets in the classroom. Armed with this information, teachers and school leaders can create better professional development programs that promote proven techniques and practices that help students learn, and can make better-informed hiring and tenure decisions.

The project is funded by the Bill & Melinda Gates Foundation as part of its efforts to give teachers the tools they need to be successful and to improve student achievement in public schools across the United States.

2. WHO IS INVOLVED IN THE MET PROJECT?

Truly understanding the skills and techniques that are hallmarks of great teaching requires extraordinary collaboration between educators and researchers. MET is unprecedented: it is a partnership among thousands of teacher volunteers and administrators and union leaders from school districts across the country and dozens of independent researchers and education organizations.

Teachers – more than 3,000 of them – are at the heart of the study. The teacher volunteers were recruited from Charlotte-Mecklenburg Schools, NC; Dallas Independent School District, TX; Denver Public Schools, CO; Hillsborough County Public Schools, FL; Memphis City Schools, TN; New York City Department of Education, NY; and Pittsburgh Public Schools, PA. (Pittsburgh served as the project's pilot district, but no data from this district will be analyzed.)

Lead researchers involved in the project are affiliated with Dartmouth College, Harvard University, the University of Michigan, the University of Virginia and Stanford University. Participating non-profits and education companies include RAND Corporation, Educational Testing Service, Teachscape, The Danielson Group, The New Teacher Center, National Math & Science Initiative, and Westat.

The project is funded by the Bill & Melinda Gates Foundation.

3. WHAT MEASURES OF EFFECTIVE TEACHING IS THE MET PROJECT INCLUDING IN ITS ANALYSIS?

Because great teaching is multi-faceted, it takes multiple measures to give a complete picture of a teacher's effectiveness. The goal of the MET project is to identify which measures give the best and most accurate information about how well a teacher helps his or her students learn – and how these measures should be used together to see the whole picture of a teacher's effectiveness.

Many of the measures being studied by the MET project are well established, though some are new. They include:

• The Tripod Student Perception Survey, developed by Ron Ferguson at Harvard and administered by Cambridge Education [link to Cambridge Education website]. This survey, developed and refined over the past ten years, captures students' perceptions of their classroom experiences, providing teachers with actionable feedback about how to improve teaching practices that lead to student learning.
• The Test of Teaching Knowledge, a new written assessment developed by the Educational Testing Service [link to ETS website] meant to gauge teachers' understanding of teaching strategies.
• Protocols for Classroom Observation designed to help school leaders assess teaching effectiveness by watching teachers in action.
  • The Framework for Teaching, developed by Charlotte Danielson
  • Classroom Assessment Scoring System (CLASS), developed at the University of Virginia
  • Mathematical Quality of Instruction (MQI), developed at the University of Michigan
  • Protocol for Language Arts Teaching Observation (PLATO), developed at Stanford University
  • Quality of Science Teaching (QST), developed at Stanford University
  • UTEACH Observation Protocol (UTOP), developed at the University of Texas-Austin for assessing math and science instruction

For further details about these and other measures being studied by the project, visit the More on MET page.

4. HOW WILL THE MET PROJECT DETERMINE WHICH MEASURES ARE RELIABLE AND VALID?

Teaching evaluations are only valuable if they can be counted on to accurately reflect what a teacher is doing in his or her classroom that is helping students succeed – in research terms, they must be "reliable" and "valid."

A teaching effectiveness measure is considered "reliable" when it shows "inter-rater reliability" – that is, when two people using the same measure to evaluate the same teacher come up with the same results.

"Valid" teaching effectiveness measures are those that are proven to lead to student learning. Many current evaluation systems, by contrast, are subjective observations based on checklists of teaching practices that have not been demonstrably linked to promoting student learning.

Reliability and validity represent the key trustworthy tests that teachers and education leaders are applying as they create new systems of teacher evaluation, so they also represent the key questions the MET project is designed to investigate.

The 3,000 teacher volunteers participating in the project have agreed to be evaluated — for research purposes only — using the teaching effectiveness measures being studied. (Data on individual teachers that is collected for the project is confidential and will never be shared with principals or other school or district personnel.) MET project researchers will see how well each measure identifies effective teaching by studying the relationship between each teacher's results on the measures utilized through the study and his or her students' learning. The researchers will then determine what combination of the measures best predicts teacher and student success.

5. WHY IS IT IMPORTANT FOR A TEACHING EVALUATION SYSTEM TO INCLUDE MULTIPLE MEASURES?

Teachers, school leaders and education researchers agree that teaching is too complex to be fully captured by a single measure like student test scores. Just as importantly, student test scores alone do not give teachers constructive feedback about how they can improve in the classroom and better help their students succeed. Feedback that captures the true range of professional skills and competencies that teachers must employ can be gathered from several sources, including student surveys, classroom observations and student test scores. Together these sources can give teachers the information they need to continue to confirm what they're doing right and where they need to improve. The information can also help districts develop evaluation systems and professional development programs that support great teaching and promote student achievement.

6. WHY IS IT IMPORTANT TO VALIDATE MEASURES OF EFFECTIVE TEACHING AGAINST STUDENT ACHIEVEMENT?

Effective teaching is teaching that leads to student success. The feedback school leaders give to teachers must help them improve in ways that are linked to student learning. Validation is about testing the alignment among measures and outcomes to ensure that they're helping provide teachers with a path forward, and not pulling them in different directions. Validation also keeps teacher evaluation focused on supporting the core business of schools: student learning. By testing different measures of teaching effectiveness to see how well each one links to student learning, MET is helping to ensure that school systems can identify and support great teaching.

Teachers need to know that the feedback they receive from evaluation can help them achieve greater success for their students in terms of learning outcomes.

7. HOW DOES THE MET PROJECT MEASURE STUDENT ACHIEVEMENT?

The MET project validates all its measures against the difference between actual and expected student achievement on standardized tests using a method called "value-added." A teacher's value-added score is determined by comparing the achievement gains made by his or her students to those made by teachers with students with similar characteristics, including similar prior achievement levels. By comparing a teacher's value-added scores with his or her results on the other measures of teaching effectiveness being examined by the study, MET project researchers can investigate which teaching practices, and which teaching effectiveness measures, best predict achievement gains.

8. WHICH STUDENT TESTS IS THE MET PROJECT USING TO MEASURE STUDENT ACHIEVEMENT?

The MET project measures student achievement using two types of standardized tests: the state standardized test and a supplemental test. The supplemental tests used by the MET project are:

• Stanford 9 Open-Ended Reading Assessment, in grades 4-8
• Balanced Assessment of Mathematics (BAM), in grades 4-8
• ACT QualityCare series for Algebra I, English 9, and Biology

While the state tests are designed to measure how well students have learned the in-state standards, the supplemental tests tend to measure more reasoning skills and conceptual understanding. The two types of tests together provide a more complete picture of student achievement than either one alone.

9. WHICH FINDINGS WILL THE MET PROJECT RELEASE, AND WHEN?

Through its close collaboration with teachers, school districts, research organizations and advocacy groups across the country, the MET project is able to share practical insights and tools throughout the course of the study. These insights are helping to support teachers and students in classrooms today.

School districts across the country continue to examine ways to improve their teaching evaluation systems. To help inform this work, the MET project is committed to releasing progress reports as our research and analysis progresses. Through these reports, practitioners and policy makers have quick access to the data developed through the project in order to begin thinking through the practical implications of the work prior to release of the final report in the summer or fall of 2012.

• The MET project's first preliminary findings (released December 2010) showed that surveying students about their perceptions of their classroom environment provides important information about teaching effectiveness, as well as concrete feedback that can help teachers improve.
• The project's second set of preliminary findings, released in January 2012, examines classroom observations and offers take-aways on creating high-quality observations systems.
• In spring 2012, the project will release a report on composite measures of teacher effectiveness.
• Later in 2012, the MET project's final analysis will be complete. The final report will include an analysis of how well the project's measures of effective teaching predict actual student learning. To accomplish this, participating teachers and school leaders agreed that in the second year of the study, MET project teachers would be randomly assigned to appropriate classes after the class rosters of students were set. Random assignment is important to eliminate the possibility that results are skewed by schools putting students who are more likely to make learning gains in the classes of certain teachers. The analysis of results following this randomization will confirm how well and how reliably the measures in the MET study identify great teaching.
• In addition, the MET team and its research partners are part of ongoing national, state and district level conversations among educators about the latest findings and advances in this field.

10. WHAT ARE EXAMPLES OF THE KINDS OF TOOLS THAT THE MET PROJECT IS DEVELOPING?

First and foremost, the final report for the MET project will not set forth a recommended formula or system for the measurement or evaluation of teaching effectiveness. Instead, the project expects the final report to explain the researchers' findings about the way in which each measurement tool being studied yields valuable information about teaching effectiveness, with insights about how districts might think about using that information to help teachers improve and support student success.

In addition, the MET project is developing a set of tools, resources and implementation guides to support schools and districts as they design evaluation systems that accurately and reliably identify great teaching. The tools being developed include:

• Tools for Testing Observation Raters and Instruments. Recognizing the pressing need within districts to ensure that classroom observations are valid and reliable, the MET project is using its extensive data and analytical resources to develop two tools a rater "certification" tool districts can use to assess whether the individuals they train can make accurate judgments. Beta-testing has begun, with product release expected in summer 2012.
• "Validation engine" for testing observation instruments. States and districts can use this online software to determine for themselves if the tools they use for classroom observations are valid and reliable. Trained raters in a school district can score videotaped lessons from the validation engine's online library. School leaders can then see whether their observation tool accurately identified the effective teachers, and whether there is consistency among the school's raters. The validation engine began beta testing with selected districts in spring 2011 and is expected to be made widely available in summer 2012.
• Panoramic camera for taping classroom practice. To capture videotaped lessons for the classroom observations portion of the study, the MET project developed a small, easy-to-use camera that records a nearly 360-degree view of a teacher's classroom. The camera does not interfere with classroom activity and can be set up easily by teachers who then upload their videos to a secure website. The camera allows teachers and observers to see how the entire class was reacting during the lesson and provides greater perspective and insight into the teaching dynamic. Designed specifically for this study by MET project partner Teachscape, the camera is available for purchase by schools and districts and is already being used by schools across the country for teacher professional development.

In addition, in the 2011-12 school year, the MET project is embarking on a new round of data collection aimed at creating an online video Library of Practice.