Nancy Lewis, Evaluation Director

The Lockheed Martin/UCF Academy for Mathematics and Science (LMA) offers a master’s degree program dedicated to systemic improvement of mathematics and science teaching and learning in the K-8 schools of central Florida. The master’s degree curriculum design is consistent with national curriculum, instruction, assessment and teacher preparation standards. Moreover, within the program, LMA is committed to the creation of a network of school-based leadership in mathematics and science education. Therefore, it is committed to the support of all participants and graduates in their efforts to enhance the quality of mathematics and science education.

Due to the fact that the LMA is an ongoing teacher enhancement program and because there is a national need for improvement in mathematics and science teaching, the National Science Foundation (NSF) awarded the LMA a four-year grant to evaluate the fidelity of the program, its effectiveness and impact, and to examine program sustainability. The evaluation used a multi-wave approach that spanned four years. The evaluation design was shaped somewhat like a funnel; it began very broadly with surveys, focus groups, and over fifty classroom observations of scholars teaching a mathematics or science lesson. Each year the evaluation design called for a more in depth data collection and analysis.

By year three the narrowing of the evaluation design resulted in 13 mini-case studies. That is, Wave 3 of the evaluation consisted of extended observations of thirteen teachers, all graduates of the K-8 Mathematics and Science Education Master of Arts degree program. Originally fifteen teachers were selected for observations, but due to major scheduling issues and unforeseen circumstances, only thirteen were able to be included in this wave of the evaluation. Each teacher was observed on three to five different occasions for a minimum of two hours each session. Approximately 260 hours were spent observing teachers during Wave 3. Observers spent an additional 270 hours expanding and analyzing field notes. A total of nearly 530 hours were spent on the qualitative component of Wave 3 of the LMA evaluation.

A number of themes emerged while during the field note analyses. One of the most recurrent themes was that of positive classroom culture. Every teacher observed enjoyed a wonderful rapport with his/her students. An atmosphere of mutual respect and a safe, risk free learning environment was evident in each classroom. For many of the teachers, humor played a large role in their relationships with their students. Several teachers, at a variety of grade levels, were observed joking and having fun with their students.

Similarly, the philosophy that permeates the Lockheed Martin Academy was evident, to varying degrees, in each teacher’s instruction. All teachers used a variety of questioning techniques that emphasized critical and higher-level thinking. Various forms of mathematics manipulatives and science “laboratory materials were used in the lessons to promote hand-on, minds-on learning. All teachers embedded connections during instruction. The connections in this project were made between mathematics and how it is used in the real world. Students were required to “collect data on several different stairways, find the proportion of the rise and step, decide which proportion [was] best, defend their decision in writing, and design a stairway of their own and provide a drawing of it.” Connections were also made within mathematics as well as between subject areas. Many of the lessons observed connected mathematics or science to literacy. For example, students participated in shared reading about science concepts in a first grade classroom and eighth grade students wrote thoughts about a science lab in their journals.

Working together, or in collaboration, was seen in various forms in all but one of the classrooms. Most commonly observed was collaboration among students. At times this collaboration took the form of structured cooperative groupings. One first grade teacher used cooperative groups extensively in her class. She kept a list of the various group structures on the front board and all of her students were familiar with the various group structures. It was obvious they enjoyed working with different classmates in a variety of situations. Most times the collaboration between students occurred in less formal structures. The students worked in partners, possibly discussing the solution to a math problem or in small groups of three, four, or five students. Several times small groups were observed, from kindergarteners to eighth graders, working to understand science concepts such as how things sink or float or how to construct electrical circuits.
Collaboration between students and teacher was also observed during Wave 3, though not as often as collaboration among students. Although only a small number of teachers were observed actually collaborating side by side with their students in mathematics or science, a larger number exhibited a collaborative relationship with their students. This collaborative relationship was fostered by the teachers’ use of inquiry and questioning strategies in an effort to facilitate the students’ efforts to make meaning of the concepts presented in the lessons.

While questioning strategies, use of mathematics manipulatives, connections between mathematics and science, and collaboration were tenets of the LMA that all teachers exhibited, the more accomplished teachers’ instruction further illustrated the overall philosophy of LMA. These accomplished teachers relied heavily on informal, on-going assessments of their students understanding to guide their instruction. They taught for understanding and encouraged meta-cognition. They celebrated divergent thinking and had high expectations for all of their students. As a result of these practices, the more accomplished teachers tended to have more empowered students. The students were observed easily expressing their thoughts and feelings on a variety of subjects as well as taking responsibility for their own learning. Observers were in a classroom when one student calmly walked to the door and turned off the lights in the classroom. He explained to his classmates that it was too loud in the room for him to think. The teacher thanked the student “for reminding his classmates that there is a certain level of noise that is acceptable when people are trying to work.” The class resumed with the noise level at a more appropriate level.

Reflection also played a major role in the classrooms of the more accomplished teachers. In a first grade classroom the students came together each day before going to lunch to reflect on and discuss the morning’s happenings and learning. The teacher encouraged students to express their successes as well as what they felt they could improve upon. This reflective session was repeated at the end of the day to discuss the afternoon’s activities. The students looked forward to these times and were eager to share their reflections. In the upper grades reflections often took the form of journal writing. In an eighth grade science class, the students were asked each day to reflect on the concepts presented the previous day. After about ten minutes of writing the students were asked to share their reflections. The teacher always read the students’ journals and used them as a way to gauge her students’ level of understanding of the concepts presented.
All teachers exhibited in their instruction, to varying degrees, aspects of the LMA philosophy and goals. However, during the study of the field note analyses, another category emerged; practices opposite of LMA philosophy. The most common practice that illustrates this point is the amount of teacher talk during instruction. In some classrooms it was obvious the teacher owned the knowledge due to the fact that he/she did most of the talking. Conversely, in the classrooms of the more accomplished teachers where the students were encouraged to talk and share ideas, the students appeared to own the knowledge. The teacher was there to facilitate the process of learning, not direct it.

Finally, one of the most significant themes that surfaced in the Wave 3 field note analyses was that of leadership. Almost every teacher observed shared examples of his or her high degree of leadership. This leadership took many forms. Some teachers led in obvious ways while others chose to lead quietly from their classrooms. Most of the teachers conducted professional development sessions. Some presented sessions in their schools or districts, while others presented at national conferences. Many were involved in writing curriculum or preparing curriculum documents to assist their colleagues. One middle school math teacher helped write the 2000 NCTM Standards. Several are head of their departments in middle schools and likewise in the elementary schools many are team leaders. A number of teachers received high praise from their principals and/or colleagues. One middle school principal expressed how pleased he was that he “stole” the particular teacher the observers had visited and how he felt she was “an exceptional math teacher.” In another case a teammate of a first grade teacher actually sought out the observers to tell them her colleague was a great teacher and team leader.

The Wave 3 analysis showed a great deal of fidelity between the program tenets and the practices of classroom teachers who experienced the program. The fidelity was observed in a majority of the teachers observed which gives credence to the effectiveness and impact of the program. The teachers observed included recent graduates and graduates from as long as ten years ago. The high level of effectiveness across the sample shows the effects of the program have a reasonable expectation of sustainability.