MeTA musings

As I entered the city limits on my way to work today, I saw a radar speed sign presumably placed by the county sheriff's department. This is not the first, nor probably the last time, I will see this sign. Rather than paying a deputy to sit at the city limit sign, they are using a piece of technology with the intent of slowing down traffic as it enters town.

No, I didn't get picked up for speeding, but the idea of a "warning" in this form reminded me of how we should be assessing our students. How often do we wait until the "test" to communicate to students' their misconceptions? The radar sign seems to be a nice metaphor for formative assessment. It gives feedback to the driver (student) in a way that is meaningful. Rather than an officer saying, "Sorry, you get a ticket today," (penalty-driven, summative assessment?) the radar sign kindly reminds us to slow down. We must help our students to "slow down" via better feedback in the form of formative assessment techniques. Waiting until the mighty officer turns on his/her sirens is simply too late!

Does this metaphor resonate with you? Are you more often the deputy or the radar sign? Leave your comments below.

I have been reading a PDK article by Paul Black and Dylan Wiliam entitled, "Inside the Black Box: Raising Standards Through Classroom Assessment." A colleague and I have been talking quite a bit about formative assessment and she recommended this article. A few excerpts from the article hit me as I was going about my usual teaching responsibilities. The first one helped solidify all of the thought I've been giving to self-assessment and metacognition on this blog.

"Thus self-assessment by pupils, far from being a luxury, is in fact an essential component of formative assessment...if formative assessment is to be productive, pupils should be trained in self-assessment so that they can understand the main purposes fo their learning adn thereby grasp what they need to do to achieve."

I started to pat myself on the back a bit, but then quickly tasted a slice of humility pie as I read on...

"...the choice of tasks for classroom work and homework is important. Tasks have to be justified in terms of the learning aims they serve, and they can work well only if opportunities for pupils to communicate their evolving understanding are built into the planning. Discussion, observation of activities, and marking of written work can all be used to provide these opportunities, but it is then important to look at or listen carefully to the talk, the writing, and the actions through which pupils develop and display the state of their understanding..." (Emphasis mine)

In my effort to be more "data-driven" in my formative assessments, I feel like I've been a bit too analytical and quantitative. Rather than simply touting standards-based reporting as the end-all solution, I realize I need to listen to the students discuss their misconceptions rather than just seeing the level of understanding as some sort of number on paper. I decided it was worth giving a try.

"Listening" in action
I gave a brief quiz yesterday in one of my math classes. From a standards perspective, it assessed two of this chapter's learning targets. One was a "new" learning target and one was a learning target that was assessed via a quiz once already last week. Today, I handed back the quiz and matched up students according to their relative strengths and weaknesses on their current level of understanding on the learning targets. This was my previous practice. Today I took it a step farther. I circled the room and helped groups who were struggling and listened in with the intent of getting a better grasp of their current misconceptions. This led to a more informed and meaningful whole-class discussion after the groups had time to share with each other.

I believe that listening is allowing me to take a step forward towards answering a key question the article posed:

"Do I really know enough about the understanding of my pupils to be able to help each of them?"

Formative assessment guides instruction
In addition, it led me to reaffirm my belief that an assessment is not "formative" unless it truly guides instruction.

I need to use this quantitative and qualitative "data" (by taking a detailed look at the mistakes students made on the quiz and listening to their conversations as the students talk about their mistakes) to decide how to proceed with instruction even more in the future. Am I transparent in the way I communicate this with students or do they think a "quiz" is merely a "mini-test?" I wonder if we too often separate "instruction" from "assessment" in our classrooms. "Today is a quiz day..." when in fact, the two entities should not be mutually exclusive. I came up with a new motto. It's not really earth shattering and pretty simple:

Everyday is "formative assessment day"

It is more than a change in semantics. Our assessment and instruction should be a cyclical process if we're truly using "formative assessment" techniques. In computer programming lingo, it looks something like this:

START
IF STUDENTS < "GET IT" THEN INSTRUCT
ASSESS
IF ASSESS >= "GET IT" THEN "DONE" ELSE GOTO START

I'm not 100% satisfied with this syntax, probably because there's no one easy way of figuring out the "GET IT" part. Standards-based grading and listening are two possible components, but the puzzle doesn't seem to be solved yet. What other components or key characteristics do you see as essential in effective formative assessment?

Dr. Mark Stock asked me to guest blog for him recently at the Hope Foundation. The topic was doing "old things in new ways OR new things in new ways?" in the context of technology integration. Many of my thoughts have been influenced by the TPACK framework, an idea derived by Mishra and Koehler at Michigan State University. If you haven't read the landmark article (pdf), I highly recommend it.

Head on over and check out the post by clicking here.

Dr. Mark Stock at the Hope Foundation was asked a thought-provoking question by one of his "pre-administrator" students.

"How are cut scores for proficiency determined?”

I am assuming that he is making reference to cut-off scores such as the 40th percentile cut-off mark for "proficient" on standardized tests for the purposes of reporting AYP.

Mark's response follows.

"....But…..the dirty little secret to setting proficiency levels is still rooted in the question, “How many do we want to fail?'"

I think Mark would agree that we (educators) don't want any students to fail. His point is obviously to answer with a question that continues the conversation in his classroom. I'd like to take it one step further as it relates to standards-based grading/reporting. As I mentioned in an earlier post, I am in the midst of transitioning to a 1-4 point scale for the purpose of reporting out student understanding of learning targets. My current scale looks like this:

4 – demonstrates thorough understanding
3.5 – high level of understanding, but with small errors
3 – demonstrates understanding, but with significant gaps
2 – shows some understanding, but insufficient for a passing grade
1 – Attempts the problem

Scale meets letter grade
Until now, I had not thought about the implications of this quasi-rubric matching up with my overall grading scale of A: 90 and up; B: 80 and up; C: 70 and up; D: 60 and up, F 59 and below. Breaking the 1-4 scale down into percentages and then assigning a letter grade makes for an interesting discussion.Let's assume a student is being assessed on two learning targets. On one they demonstrate a high level of understanding (4) and on the second they show some understanding, but insufficient for a passing grade (2). Assuming all learning targets are weighted equally, as they are now in my grade book, this student has earned a "C" or 6/8 learning target points. I subsequently report out that they have a "75%" level of understanding. By mastering one learning target and failing another, should a student receive a "C"?

I wonder if a better way of reporting out student learning might be by the number of learning targets they've mastered. I'm thinking about a hybrid system that might require students to have a "4" level of understanding on at least 90% of the learning targets in order to get an "A." The percentage here is up for discussion, but if that system is a step in the right direction, what would an appropriate break down for various letter grades look like? Should there also be a required number of "3" or "3.5" learning targets in order to earn a given grade? In other words, should a student who has demonstrated a high level of understanding (4) on 90% of the learning targets, but has significant gaps on the rest (3) earn a different letter grade than a student who has also mastered 90% of the learning targets, but has some small errors (most likely computation in my math course; 3.5) on the remainder earn the same overall letter grade?

I can think of many more scenarios similar to the one I just described. A quick fix is eliminating letter grades, but unfortunately that's not an option for me as public school teacher who is required to use a student information system. The bigger question worth discussing is what makes an "A"?

An excellent comment was made on my last post. I started to reply in the comments section, but got a bit long-winded and realized it would be better illustrated in a full-fledged post.








My initial thoughts are below, but they are in no way "best practice" as I'm constantly tweaking this new way of reporting learning.

How do you grade late work?
This could be rolled into a citizenship grade or as you suggested into some sort of new learning target. A virtual colleague suggested to me that 5-10% of a student's grade be something along the lines of "responsibility" (just to play the 'grading game' and keep students accountable), perhaps giving students one point per day for coming to class prepared and ready to ask questions about the homework. Right now, I'm just recording who turns it in, how much they did and if they turned it in late. I should have also mentioned in my initial commentary that I post the answers to all "practice" assignments on the board, so there's no incentive to copy from a friend. Check out some of my previous posts on feedback for more insight on this philosophy. If/when it is entered into the grade book, I want to do it in a way (numerically, I admit this may be difficult) that clearly makes it stand apart from the learning targets. Since understanding and responsibility are not easy to pull out from the "old system," I don't want to fall into this trap again.

Another aspect of this system is allowing students to be re-assessed on a learning target if they're not happy with how they did. For example, I would give the student who got all 4's and one 2 the opportunity to come in for re-teaching, do extra problems/activities, and then eventually a mini-assessment on that learning target. An improved score replaces the old score in the grade book so that students are not penalized for "getting it" later than others. I explained to students that re-takes must take place outside of class, are only available to students how have put forth a solid effort on the homework, arranged ahead of time, and may only be done once it is evident that new learning has taken place. It puts the burden on the shoulders of the student, eliminates the "anyone can re-take at anytime, so why do the homework?" mentality, and helps students understand it takes more time on my end to create an alternative assessment, so I expect additional effort from them, too.

I hope this helps and am open to any critiques on this current practice of reporting learning.