MeTA musings

In response to Evan Abbey's thoughts on assessment for learning and student information systems as it related to the Iowa Core Curriculum, I felt compelled to share my current progress towards standards-based reporting. A week ago, we started a new reporting period so I decided to make a drastic change by going (nearly) 100% standards-based reporting in two of my math classes. I had been assessing my students based on "learning targets" (standards) for a few weeks at the end of the last grading period, but had still been reporting their assessment scores as total points in our student information system.

"Old" way of reporting
Here is what the grade book looked like from my perspective in the "old" way. Notice the typical categories along the top: test, homework, quiz, etc.


Standards-based reporting
Here is a screen shot of what my grade book looks like this quarter so far. Notice the different categories along the top. The learning targets are the main (only?) area of focus.
I give students a list of the learning targets - the "big ideas" I hope they'll learn from our unit of study. Here's an example from the assessment reported above.

1. Define and classify special types of quadrilaterals.
2. Use relationships among sides, angles and diagonals of parallelograms.
3. Use properties of rhombuses and rectangles.
4. Write ratios and solve proportions
5. Identify and apply similar polygons.
6. Use and apply AA, SAS, and SS similarity statements.
7. Find and use relationships in similar right triangles.
8. Use the Side-Splitter Theorem and Triangle-Angle Bisector Theorem

Each learning target is assessed on a sliding four point scale. Multiple measures would be ideal for determining where each student's understanding is at for each target, but I'm still working on how this looks in a math class. For right now, I've come up with the following scale:

4 – demonstrates thorough understanding
3.5 – high level of understanding, but with small errors
3 – demonstrates understanding, but with significant gaps
2 – shows some understanding, but insufficient for a passing grade
1 – Attempts the problem

Student response to standards-based reporting
Students in my class are now accustomed to detailed feedback in the form of 1-4 scores on each learning target, however a few were a bit hesitant when I told them that beginning the fourth quarter all tests would be recorded by learning target ONLY in the grade book. Their parents would now see (via the student information system) a break down of their current level of knowledge by learning target rather than seeing only a single score.

One student chimed in, "I don't want my parents to see that I got an F on one learning target."

He was making reference to the fact that he had 4's on all learning targets with the exception of one. I explained that previously using the "old" system, his score would have been recorded as 30/32 or 94%. Now he would see seven 4's and one 2 in the grade book - still 94% overall if calculated by points, but the standards-based reporting approach makes his learning more transparent to me, him and his parents. Parents now see that their son has a solid understanding of seven key ideas, but has failed to demonstrate a sufficient knowledge for a passing grade for one learning target. The student wasn't convinced that this was better for him, because he did not want to face the possible repercussions at home for a "failing" grade!

Looking ahead:
You'll notice that my grade book only includes scores for learning targets and no longer homework or quiz scores. Here is my thought process on adding homework and quiz scores:

Homework and quizzes are both "practice" or "formative assessment" opportunities and should not negatively impact a students' overall "grade."

Consider Student A who bombs the homework, bombs the quiz, but "gets it" between the formative assessments and the test (assessment that "goes in the grade book"). Now consider Student B who "gets it" on the homework, aces the quiz and aces the test. In the old system, Student A is penalized for not "getting it" early, so his/her grade suffers. Student B's grade looks outstanding because he/she "got it" right away and earned all of the points along the way. If Students A & B both have the same level of understanding, I believe that what is reported out about these two students should be consistent as well.

At the current time, I am keeping track of students' homework completion and learning target scores on quizzes (formative assessment that drives my instruction), but am not sure if/how I will add these to the grade book. As I figure out how best to play the "grading game," I think I may need to somehow record the homework scores in the grade book to give the students an external incentive to do it. Many, but not all, students see the connection between completing the practice (homework) and formative assessments (quizzes) and the positive impact it has on understanding/learning. On the flip side, I don't want to simultaneously report out "understanding" via learning targets and "effort" via homework completion, because I fear it is a step backwards towards the "old" system and it also confuses stakeholders along the way. I have heard of other schools using citizenship grades, which I think lends itself nicely to separating the reporting of understanding and responsibility.

This new way of reporting learning has been a time-consuming, but rewarding process. It is also a work in progress that will surely need to be tweaked along the way. I'm open to any suggestions/comments from those who may also be stepping up to the challenge of standards-based reporting.

I had my biannual trip to the dentist yesterday. Aside from a half day off from teaching, it gave me some extra time to think about MeTA. This may be a stretch, but I think there are lots of connections between assessment practices and going to the dentist. Let me explain:

As I sat in the chair listening to the hygienist lecture me on flossing techniques (she was right!), I was reminded of what makes assessment useful - meaningful feedback. If I went to the dentist and all I got was a good cleaning and few shots of lead injected into my teeth, I wouldn't be bettering myself. My problems would seem "fixed" temporarily, but my habits would inevitably stay the same. The plaque on my teeth, which I believe beautifully represents students' misconceptions, may go away with a few shots of red ink, but by the next visit (assessment?!), it will surely find its way back. We're doing the same thing to our students when we simply "check" their work, note the "correct" way and add another score to the grade book.

A worthwhile dental visit "educates."
My hygienist gave me flossing tips because she genuinely wanted to help me take better care of my teeth. She wanted me to come back next time with as little plaque as possible. As teachers, we must help our students debug their own work. Spewing out correct answers isn't enough. Understanding where a student's thinking is coming from can be the first step towards helping him/her to get to the next level along the continuum of learning.

Look in the mirror.
Our goal should be to help students look in the mirror (a la self-assessment) and see where their own mistakes lie. Yes, a dental visit typically reveals much plaque, but it doesn't have to be the only time/place where plaque is identified. Continuing with the same theme, educators should continually be looking in the mirror to identify poor assessment tools and revise them accordingly.

Brushing often?
I go to the dentist two times per year, but I brush my teeth twice daily. How often are we assessing our students? How meaningful is our "brushing" and "flossing"...is it getting rid of the plaque? After all, it is possible to brush and floss without removing much plaque. When we get our new toothpaste, toothbrush and floss at the end of the visit, have we been inspired to use it? I wonder if our assessment often hurts, rather than hinders our students to make meaningful changes in the way they think not only about the important concepts, but about school in general.

Clean teeth expectation.
Patients who leave the dentist's office expect to have clean teeth when they leave. What do our students expect to gain from their assessments? What do we as educators expect to learn from our assessments? In the midst of rolling out a more "standards-based" reporting system, I'm realizing that my students are shocked to receive feedback in the form of multiple scores rather than a single number or grade describing their performance. This form of reporting makes the academic expectations of students much more transparent. I look forward to the day when students expect detailed feedback on their work - more than just a single score or letter grade. I admit that breaking this "tradition" of schooling can be a difficult task.

Have you or your students been to the assessment dentist lately? What are your personal "assessment cavities?"

I've been thinking about personal learning networks as an ideal model for differentiated professional development. A complaint I often hear regarding in-service time is "how can I use THAT in my classroom?" I brought this up at our building leadership team this week and responded with a question of my own:

What if EVERY professional development session was catered to your interests?

After all, if we're supposed to be differentiating content, process and product in our classrooms, why shouldn't we expect the same from our in-service time each year? I used this Educational Leadership article by Bill Ferriter as an example of how it might be done through the use of personal learning networks (PLNs). I'm happy to report that my building principal thought it was a great idea and asked me to present it at the next morning's staff meeting. I knew that my five minute allotment at the meeting wasn't going to be enough to "sell" the idea and definitely not enough time to thoroughly explain the complexities of what a PLN might look like. Instead, I decided to give a "teaser" to the staff to see if I could recruit five staff members who might be interested in giving the idea a try. Twenty-four hours later, I had the fifth volunteer approach me as I walked in to work and started to make a few copies. I'm in the midst of meeting with each of the five educators and helping them setup their own quasi-PLN through the use of Google Reader and various edublogger RSS feeds. Other social networking tools such as Twitter, the Ning I setup earlier this year, and subscribing to journals via Ebsco's RSS feed will be "phase two" for these individuals if all goes well. Furthermore, I have the green light to use our May in-service time to get all of the staff members in my building setup with their own personal learning network. I'm hoping the five volunteers will not only be able to help with their newly acquired technical expertise, but that their enthusiasm towards the idea will spill over between now and then as well. In my previous attempt to diffuse innovations, my interpersonal communication has been lackluster at best. My hope is that by interacting with these five over the next month that the differentiated professional development model might catch on and be something that we can spend even more time developing next academic year. Look for more posts in the future as this model develops over the next month or so.


Note: This is not my original idea. Others, including Evan Abbey at Heartland AEA have suggested using PLNs as professional development. This is my humble attempt at documenting and sharing the idea as it unfolds so that others, too, may hopefully benefit someday.

From an article that recently ran in T.H.E. Journal:

"One of the biggest reasons we face resistance is because so many times we give instruments to the teacher with no follow-up or no training...I don't think fear is the right word. Some teachers have been burned by technology in the past. They used it and found it was either not great or incomplete, or whatever, and so they're not interested in trying again."

-Tom Nolan, curriculum support specialist for the Albuquerque Public Schools

I've seen plenty of educators with this mindset. They may have tried out a cool new software application or website once upon a time, but the technology just didn't "work" for them. Maybe the laptops were low on battery power. Maybe the internet was slow or down that day. Maybe the software itself had glitches or didn't work like it was advertised. Perhaps the comfort level just wasn't initially present with the tool to begin with and this lack of confidence spewed over in to the instruction causing students to be turned off. Regardless of the issue, it left a sour taste in his/her mouth. The first impression was so sour to the point of no return.

I've seen another angle to this problem. Marc Prensky's 2005 Edutopia article sums up a four step process to technology integration that leads to this point:

1. Dabbling.
2. Doing old things in old ways.
3. Doing old things in new ways.
4. Doing new things in new ways

When an educator sees technology as simply "play" on the student's end (#1) or not improving their instruction (#s 2 & 3), why would they care about trying out Moodle, a Flip camera or The Geometer's Sketchpad? Their response to "technology integration" is the same as it's been to any other professional development experience: It's more of the same "stuff" I don't need in my classroom.

An academic recently posed a challenging scenario to me. It went something like this:

"Imagine I am a teacher down the hall from you at your building. I think my teaching strategies are pretty good. In fact, I've been teaching for a while and my students seem to be learning a lot, too. Why should I change?"

In light of teachers being burned by technology in the past and seeing it as more of the same, what answer would you give to the "teacher down the hall?"

(This post is the third in a series based on metacognition as a way of improving classroom assessment and instruction)

Debugging concepts vs. procedures

"Outstanding teaching requires teachers to have a deep understanding of the subject matter and its structure, as well as an equally through understanding of the kinds of teaching activities that help students understand the subject matter in order to be capable of asking probing questions." (Bransford, 2000, p. 188)

When educators are focused on conceptual understanding, debugging can become an effective means of incorporating metacognition in the classroom. With a shallow level of content and pedagogical knowledge in my earlier years of teaching, I found debugging to only happen from a procedural perspective. I was more inclined to point out a step that a student missed in his/her computation and/or algebra rather then stepping back and analyzing where the misunderstanding might be that leads to the "bigger picture" causing students' misconceptions. With these thoughts in mind, I embarked several days ago on changing the way I handle students revise their formative assessments.

Debugging Opportunities
In my quest towards standards-based grading, I have been conducting formal formative assessments every few days (and without much resistance from my students either, I might add!) and reporting progress by standard:One of the many reasons I believe in standards-based grading is that it gives feedback to students on the specific standards ("learning targets") they "get" as well as the learning targets that they may still need more work on. Students took the quiz one day and the following day I paired up students according to their relative strengths and weaknesses on the learning targets and set them loose for about five minutes to ask questions of each other. I'm starting to make this a regular part of my formative assessments. Compare this with my previous practice of marking them up heavily with red ink and verbally regurgitating the preferred solution to commonly missed problems - an exercise that not only bored the students, but never seemed to produce positive results either. Several minutes into this diagnostic session, a really neat conversation was initiated. Two groups were eager to get my attention with similar inquiries:

"Mr. T...we don't understand how to do this problem - we did the same thing! Can you help us?"

This is the type of question that gets a math teacher chomping at the bit! Two typical responses come to mind...
Scenario #1: That's a great question. Let me show you how to get the right answer...
This scenario is what I call a return to the teacher-as-answer-holder system. In my previous post, I laid out several problems with this approach. It takes the responsibility of "thinking" away from the student and places it in the hands of the teacher. Hopes for metacognition and student-initiated debugging are quickly squashed.

Scenario #2: Wow. I'm so glad you asked. (Insert probing question)
This scenario is the one we should be aiming for, but as Bransford suggests this process requires a deep understanding of the subject matter in order to ask questions that will help students overcome their misconceptions - a practice I admit improves daily with practice. This is when the debugging begins on the student's part! Colleagues lament that this is ideal of behavior we wish all of our students would exhibit, but seldom do. I believe the keys to this practice of encouraging students' debugging are wrapped up in connecting collaboration with explicit opportunities for revision. We must teach and value revision through a debugging process. Not only must students be exposed to and practice debugging, the educator must also have a general idea of the common misconceptions students will face for any given standard/lesson.

"Expert teachers know the kinds of difficulties that students are likely to face, they know how to tap into students' existing knowledge in order to make new information meaningful, and the know how to assess their students' progress." (Bransford, 2000, p. 45)

Just as parents remind their children not to touch a hot oven, they must also be taught how to properly use a hot pad when navigating the same steamy temperatures. The quote above reminds me the importance of modeling to students "how not to do" something as well so that they are better able to debug their own academic work. I will never forget my fifth grade teacher who would often purposefully write sentences with mistakes on the board or solve math problems in an incorrect way and wait for a student in the class to notice. He modeled (and expected) debugging in his classroom! This was a one-of-a-kind ordeal during my K-12 years. I only hope that I can provide this same memorable (and academically stimulating) experience for my students.

A virtual colleage of mine deserves credit with a recent post in which he asked students two thought-provoking questions:

1. What do you want to be able to do a week from today that you can't do now?
2. What can I do to help you get there?

Kudos to Mr. G over at TAGmirror for putting metacognition into practice! Perhaps some debugging with follow as the students come back with their plans that potentially need revision.

Is "debugging" a natural part of your classroom practice? What type of debugging events are you planning for the remainder of the year?