Measure for Measure: The Relationship Between Measures of Instructional Practice in Middle School English Language Arts and Teachers' Value-Added Scores
Issue/Topic: Teaching Quality--Evaluation and Effectiveness
Author(s): Boyd, Donald; Cohen, Julia; Grossman, Pam; Hammerness, Karen; Lankford, Hamilton; Loeb, Susanna; Wyckoff, James
Organization(s): Stanford University; Stanford University School of Education; University of Albany, Center for Policy Research; University of Albany, School of Education; University of Virginia Curry School of Education
Publication: National Bureau of Economic Research Working Paper
Published On: 1/1/2010
While there is a growing consensus on effective approaches to early literacy instruction, there is much less agreement about effective literacy instruction for secondary school students, including middle grades.
To determine what classroom practices, if any, differentiate teachers with high impact on student achievement in middle school English Language Arts (ELA) from those with lower impact.
The findings reveal consistent evidence that high value-added teachers have a different profile of instructional practices than do low value-added teachers.
1. Explicit Strategy Instruction appears to distinguish the more effective (i.e., high value-added) teachers in the sample. Explicit Strategy Instruction provides students with very structured and specific ways to approach ELA activities. Some examples of Explicit Strategy Instruction are: (1) systematically breaking down a newspaper article by instructing students on how to compose a list of "who, when, where, and what," and how to use that list to create a focused lead and incorporate supporting details; (2) carefully breaking down the steps involved in using context clues to determine the meaning of unknown words (i.e., reading the sentence before and after the sentence at hand, substituting multiple words, etc.); and (3) teaching students how to identify the passive voice in their own writing and then change it to the active voice.
2. Unfortunately, instances of Effective Strategy Instruction are rare. The goal of many lessons was completion of the specific task rather than mastering a more broadly applicable skill or strategy.
3. High value-added teachers in the sample focused less on reading and more on writing and research skills than low value-added teachers. Moreover, low value-added teachers' lack of strategy instruction was particularly pronounced during writing instruction. The majority of what was observed were lessons in which students spent class time writing but were given little to no direction about how to structure their writing or strategies to improve their writing. For example, often teachers told students to "work on" or revise a draft for an entire class period with no guidance about how to structure their efforts. Thus many students simply typed or copied earlier drafts in neater handwriting in an attempt to revise. In contrast, literature lessons tended to be more structured around comprehension questions or reading texts aloud and analyzing as they went. Additionally, high value-added teachers used small groups far more than low value-added teachers (36% vs. 16%) and they used large groups far less (26% vs. 44%).
4. Teachers who provide models or rubrics to illustrate "good" work and define what constitutes quality in a specific lesson or domain are more likely to also give students feedback on how to improve their work. Further, modeling was positively correlated with students' opportunity to construct knowledge, whether or not the assignment required basic recall of fact or asked students to provide rationales for their analyses and interpretations.
5. Teachers' self-reports of content coverage in the teacher logs differed significantly from the structured observations, indicating that teachers perceive what they are doing differently from the outside observers (e.g., on days when the observer noted one teacher teaching grammar over 30% of the time, the teacher reported teaching it only 7% of the time).
- Value-added measures do more than measure the characteristics of students that arrive in a teachers' classroom. They also seem to be capturing important differences in the quality of instruction.
- Teachers who provide a clearer sense of their expectations may be better able to push students to higher levels of analysis and interpretation because the instruction is better structured and goal-directed.
- It is possible that the impact of some aspects of instruction, such as the quality of classroom discourse that may be important in developing students reasoning abilities and conceptual understanding of literature and writing, may not be measured well by the tests used to construct teachers' value-added scores.
Structured classroom observation
24 NYC teachers teaching 6th, 7th, or 8th grade classes that take the English Language Arts exam (83% female; approximately 21% African American and 4% Hispanic; 50% entered teaching through traditional teacher education programs)
Year data is from:
Data Collection and Analysis:
Analysis of teacher instruction, as measured by the Protocol for Language Arts Teaching Observation (PLATO) and Classroom Assessment Scoring System (CLASS), teacher logs and student assignments and work samples
Reference in this Web site to any specific commercial products, processes or services, or the use of any trade, firm or corporation name is for the information and convenience of the public, and does not constitute endorsement or recommendation by the Education Commission of the States. Please contact Jennifer Thomsen at 303-299-3633 or email@example.com for further information regarding the information posting standards and user policies of the Education Commission of the States.