Resource Book Review: Rethinking Rubrics

Rethinking Rubrics in Writing Assessment by Maja Wilson

My rating: 4 of 5 stars


came to know about Maja Wilson's Rethinking Rubrics during an exchange on the English Companion Ning. Wilson replied to some of my discussion topics about assessment and the use of portfolios. Her thoughtful and balanced replies made a lot of sense and I made myself a note to find her book and read it.

I'm heading into my 4th week of a summer workshop with the National Writing Project. My reading over the past few weeks has brought me back to the exchange I encountered with many teachers on the Ning. More specifically it brought me back to rubrics and grading--so much so, that it is now the subject of my inquiry project as I move forward into the upcoming school year.

I read Wilson's book this morning and again found that I appreciated her well-supported candor. Most interesting to me was Wilson's research into the history of assessment in our country. American colleges needed a ranking system for high school students, standard modes such as multiple-choice testing and rubrics were born out of that need.

Wilson questions if something like a rubric, which actually brought writing back into our schools, serves the purpose it promises. She explores what rubrics offer and what rubrics exclude and deny.

Read this book and you won't look at rubrics the same way again.

View all my reviews

Southeastern European Food for Thought

am currently reading Maja Wilson's Rethinking Rubrics which (so far) is essentially a soup to nuts examination of how we assess writing in the United States.  I pause to share an excerpt from the second chapter, There is a Cow in Our Classroom, because of the enormity of its implications:

Robert Yerkes, professor at Harvard and president of the American Psychological Association, was hired by the military around 1920 to develop and administer an intelligence test to all of its recruits.  Yerkes used a multiple-choice format, created by Arthur Otis in 1915.  Yerke's creation, the Army Alpha test, was the first large-scale multiple-choice test to be administered, and it publicity and popularity would pace the way for  the same technology to be used by writing assessments.

The multiple -choice format served the demands of standardization and ranking beautifully, allowing anyone anywhere to score the same test in exactly the same way in a short amount of time.  Another bonus of the multiple-choice format was that the results of the test could be quickly grouped, categorized, and analyzed in a number of  ways, individual test-takes and groups of people could be ranked, sorted, and analyzed.  In association with The National Academy of Sciences, Yerkes analyzed the results of the Army Alpha test, announcing that native-born American whites had scored the highest on the test and that of all the immigrant groups, those born in southeastern Europe scored the lowest.  Carl Brigham, who had worked as Yerkes' assistant to develop the Army Alpha test, did his own analysis, concluding that "'selective breeding' would purify and preserve the intelligence of Americans" (Caruano 1999, 12).  Added to the growing anti-immigrant backlash, Yerkes' announcement and Brigham's conclusions led to the National Origins Act of 1924, which established quotas for all immigrants to the United States.  The most severe restrictions applied to no other group than eastern and southern Europeans--those who had scored lowest on the Army Alpha test.  Multiple choice testing had become a powerful tool of discrimination.

The chapter goes on to study the history of assessment in our country and how Harvard specifically had its sticky crimson paws all over it--all in the name of ranking students.  Wilson continues to trace the development of this demand by institutions of higher education for a better and more efficient way of evaluating writing...and rubrics were born.

In the end though, I had to pause before I moved on to read the third chapter to think aloud a bit and simply share the fact that discrimination is somehow tangled within the history of writing assessment, and I can't help but wonder if it still is as rubrics are still firmly rooted in place and in many schools the use of rubrics drives the curriculum.

Resource Book Review: Clearing the Way

Clearing the Way: Working with Teenage Writers by Tom Romano

My rating: 3 of 5 stars


As I read more about the teaching of writing, a common message has arisen: encourage the writer.

Published over twenty years ago, Romano proposes finding the good in student writing, having students write more than a teacher could possibly assess, write and read all kinds of writing, talk about writing, talk about writing, and, oh yeah, talk about writing.

The same message is still be trumpeted today by members of the National Writing Project and many well-versed teachers of writing. Yet, the stereotypical red-pen wielding, mistake maven still dominates English classrooms. Shackled by any number of things, Romano asserts that teachers fall into the habit of becoming college professors. Our downfall is we critique the mechanics of writing so much that we kill the writer's voice. We do more damage by damning things like the comma-splice...

I got the most out of Chapter 8 where Romano addresses assessing student writing. He admits that there have been occasions where he has failed miserably at his job in this capacity. Yet, I am encouraged by his own revelation:

Distinguishing between the student and the writing is a fool's distinction. Writing is the writer. It embodies her voice, her passion, her thinking, her intellect, her labor, and, on some occasions, her very soul.

You are a fine person, but this piece of writing is a D.

Don't kid yourself--when that happens the writer is stung.

Such a method is directly opposed to our purposes as teachers.

I'm quickly becoming fascinated with how many teachers of writing are disgusted by the damage we do to young writers by grading their work. This is different than assessing. Romano and the rest lay out a variety of powerful and effective assessing models. But this issue of grading is sitting with me like a ghost this summer.

Romano's book resurrected the conversation for me today.

View all my reviews

Resource Book Review: Hidden Gems

Hidden Gems: Naming and Teaching from the Brilliance in Every Student's Writing by Katherine Bomer

My rating: 5 of 5 stars


I can't say that I ever anticipated reading an emotionally moving book about writing instruction. Over the last 24 hours I have read my first.

The intent of Katherine Bomer's cogent book on teaching writing, Hidden Gems, is to transform the way the rest of us approach student writing. Admittedly, it is awfully difficult to see a piece of student writing and not leap to what is wrong with it. It is how I was taught when I was 10, 11, 12, 13, 14, 15... And I allowed myself a brief moment of silent shame in a class this week with the Pennsylvania Writing and Literature Project (NWP) when a colleague projected a piece of student writing for all of us to discuss. Immediately, I saw the errors.

We didn't once discuss the errors--we discussed what was strong in it. We discussed what the writer could build upon. I listened and I have to admit, my perspective of what I do and how I do was altered.

Bomer addresses the fact that many teachers share in my experience:

Constant attention to and judgment on mechanics and organization becomes a systematic means for placing restraints on kids without listening to what they have to say.

Especially helpful is the fact that Bomer includes many student samples in her book. Yet, she not only covers what is strong, but she also models how she would confer with the student and what a plan might be moving forward. These are not perfect texts by our traditional definition. She selected a wide variety of samples.

Bomer also takes into consideration that what may be making teachers begin and end with negative comments on student writing is traced to the writing rubric and grading itself. She quotes from Maja Wilsons Rethinking Rubrics in Writing Assessment:

However, writing may not be a simple system like billiards, subject to the laws of determinism.  Writing may more closely resemble complex, chaotic systems like global weather, economic systems, or political unrest.

Bomer is trying to balance the reality of our profession (or vocation) with the best case scenario for all involved. I liked that she focused on teaching us how to find things to compliment and build upon in student writing--she is trying to change the way you see yourself and how you see your writers.

Bomer writes that teachers need to write for good cause:

Teaching writing without doing it ourselves is like trying to teach a four-year-old how to tie shoes when we have only worn flip-flops our entire life.

I've come to a crossroads myself as a teacher of writing. It is only through discussion with peers, reflections on past practice, and reading books such as this that I can come to an educated decision about how I will move forward next year. I do not have a perfect answer yet, and I do not know that any of us ever can:

Most of us have a long way to go toward creating assessments that teach writing rather than sort, humiliate, and confuse...

Kids try hard to make writing look and sound like the #4...and if they miss the cues and fail to conform, this becomes one more notch in their belt of failure...

The beauty in this book, is not just teaching us to see the beauty and possibilities in all student writing, but it also very clearly holds our "schizophrenic" roles of supportive facilitator to critical judge right up to our faces.

You may not look at your role in the classroom the same way again.

View all my reviews

National Writing Project Day 7: Domain on the Brain

I have a few thoughts elbowing their way around my brain after today's workshop with Dr. Andy Fishman:

Thought(s) 1:

Have you found that you/your colleagues have mashed together the concept assessing/conferring about domains and grading essays with rubrics?  The more I learn the more I think we've come to that--the word "rubric" has found a way to replace the correct terms of more than just "domain."

Conferring about a specific domain is quite different than using a domain chart to stamp a grade on an essay, isn't it?

Is it easy for a teacher to confuse the two...use each incorrectly...and confuse the student writer?

I find that the use of the Pennsylvania Writing Assessment Domain Scoring Guide makes sense when  dealing with formative assessment and conferences.

I'm struggling with the distinction between that guide and a rubric.  Can that guide be a rubric?  Should it be?

I don't like the thought of a number on a guide...or on something which is best promoted (in my opinion) as a formative assessment guide.

It gets gummy when I start thinking about numbers and letters generated from that guide.

Thought(s) 2:

I can't recall if I read it in Study Driven or Teaching Adolescent Writers...or someplace else...but the concept that writing in the real world doesn't supply a rubric before writing kept creeping back into me today (even though today's discussion was domains).

If we are teaching domains (Focus, Content, Organization, Style, and Conventions) and these are best reinforced through rehearsal and conferring and writing and conferring and revision and conferring over and over, then why are we really using rubrics?  To teach to the essay?  To have a piece of paper with a stamp of approval upon it?  To cover our own rear-ends so that we can say "See, I used a rubric, my expectations were clear!"

Can we ever really separate and distinguish the difference?

To Counter the Culture of Numbers

I've been engaged in some meaningful dialogue with English teachers (from middle school through college) across the country about moving to a numberless/letter-less grading system for student writing.

Some of my reading on current and past research has suggested that placing a number/letter grade on writing actually inhibits growth.  However, we are all held to the expectation of giving a grade.  A number.  A letter.  Something tangible must end up in a grade book--it is the reality of public education, isn't it?  The state even hands us a rubric...we've had to display large laminated posters of the state numerical rubric for years.

As I read and asked questions, a colleague on the English Ning suggested Linda Christensen's article My Dirty Little Secret.  She addressed my common concern:

In too many classrooms, grades are the "wages" students earn for their labor. Teachers assign work, students create products, and grades exchange hands. There are problems with this scenario. Students who enter class with skills—especially reading and writing skills—are rewarded with higher grades. They already know how to write the paper; they just need to figure out what the teacher wants in it. 

At the time I'm reading Christensen's article I'm discussing with my students that they are making the same mistakes in their writing and I'm writing the same comments on the essays week after week.  We've been having conferences, we've had lessons, notes, examples...yet, most students seem to want to focus on, "Why did I get a B?"

My feedback needed to improve.

As I read in Linda Christensen's article My Dirty Little Secret, "I see responding to student papers as part of the teaching process rather than the evaluation process."

I thought I was evaluating.

So, I've been reading and asking.  The feedback and research I've been handed back has truly been informative and from a wide variety of sources across the country: middle school and high school teachers, college professors, retired teachers, and everything in between.  I've read the thoughts of these colleagues as well as those from the writers and scholars they turned me on to: Pat Schneider, Peter Elbow, Linda Christensen (Director of the Oregon Writing Project), to articles from the National Writing Project, and the Pennsylvania Writing Project.

Maybe more importantly I pulled my chair to the front of the class and shared the research I had been reading.  I shared the feedback I have received from colleagues across the country,  and then I asked my students for their written feedback--what would be the (most challenging, interesting, helpful, positive, worrisome,______) aspect of our moving to an assessed portfolio of your work (the positive and negative) as opposed to continuing with single grades on individual essays?  How are you handling all of this?

What I learned is that it hasn't been my feedback.

There is clearly a percentage of students who rank the grade above the feedback.  If the grade disappears they can't see how they will understand what is lacking or positive in their work.  This is a real eye-opener for me--the grade dictates whether they feel their work is "good" or "bad."  Then, they think little about it again.

They view the written feedback as something they have to decipher...there is no deciphering a "C" evidently.  Yet I am also hearing that if there is a C on their paper and they feel they worked hard on it, they don't like pulling it out and looking at it to try and figure out my feedback.  They don't like looking at the grade, so they ignore the rest of the feedback.  Amazing.  I share in this.  Yet, why does it surprise me.  Shame on me for being surprised, I guess.

Tom Thompson, professor at The Citadel, replied to my open discussion on the English Ning:

I've used portfolios (in college) for which I gave comments-only feedback on individual assignments during the semester, but still had to assign a grade for the term. I finally gave up (gave in?) because my students were simply too stressed. They're too used to grades. Maybe if we started them in middle school with non-graded assessment, we could carry it over into high school and college. Or maybe if I gave better feedback, they might be less stressed.

I'm still sold on focusing on feedback rather than grades, but we have lots of un-training to do to counter the culture of numbers.

After a day of discussion and a Q&A about the shift in ideology, my classes overwhelmingly want to give this a try.  We will reassess it together after one portfolio of work: four essays/stories in total.

They like the idea that for three papers they'll continue to work on areas of weakness and further develop the positives.  They also liked that their writing grade will evaluate their growth in addition to their product.  While we will conference after each paper is completed, I see the final portfolio conference at the end as an opportunity for them to self-reflect on their growth for the past three papers (the fourth will be a self-reflection/persuasive piece).  They will walk away knowing they had a voice--not a debate--about their growth.  I want to teach them how to speak about their work beyond "Why did I get a B?"

I realize this is a leap I can't just make without some careful thought.  Maja Wilson, a high school teacher on sabbatical and currently teaching 1st Year Composition at the University of New Hampshire, wrote to me:

In my mind, it doesn't make any sense to un-grade unless several critical conditions are in place, among them:  

~students experience the work as meaningful

~students and teacher trust each other

~the teacher is committed to descriptive, specific feedback (as opposed to circling someone else's descriptors on a rubric) and conversation about the work throughout

Kim McCollum-Clark, college professor, commented several times to my questions and concerns:

My students are anxious about this grading strategy until the first paper gets back.  Then they love it.  It relieves their anxiety, helps them focus on learning their own highly personal "process" and "bugbears" as writers.  It keeps the focus on revision, conferring and feedback, and sharing your work to gain more perspective on it.

Finally, I want to share some of the honest feedback from my students--the encouraged and the worried:

Often, I only try to write to the grade which ends in sub-par writing.

The most useful part is the fact that we will know how to improve.

I like the new system because I feel like now I can write what I want without worrying about the grade I get.

It will probably freak me out a bit because I am one of those people who always knows what I have to get on everything.  It will be a challenge for me to make sure that I improve every single paper, though.

When I get a grade, I am unfocused on other points.  I can concentrate on mistakes and correcting them instead of the grade on my paper.

Based on yesterday's discussion I'm feeling more confident about our writing assignments.  I like that our grading system is based on improvement that we make paper to paper rather than a raw rubric.

I think that the portfolio grading system will benefit the class.  To begin with, the new system will shift the focus to improvement instead of a focus on meeting preset guidelines.  There is nothing wrong with preset guidelines, but occasionally writers can get caught up in following them strictly.

I believe the best aspect of change that changing grades into a portfolio will be getting out of the mindset that everything has to have a grade.  I think so many people today in school just complete the work to get a good grade instead of enjoying it.  Most students look at a grade and usually don't know how they got it or what to change to get a better grade.  In completing the portfolio we can improve and look how well we achieved our goals.

I feel that conferencing with you will greatly improve my writing as I am not as worried about meeting requirements to get a good grade, but trying to improve my writing.

I think the most revolutionary aspect of the change will be how there aren't specific grades for each one.  I've always been so focused on getting the "A" or following a rubric that it may have been stifling my ability.  

I don't think the switch from grading the individual essays to grading the portfolio will affect me.  I mean, I'll have to show growth, but that's about it.  I don't think it's that big of a difference.  I would prefer to have each paper graded individually.

I have always been one of those people who needs to see the letter, the percent, and the point value at the top of my paper.  It's rewarding to see that, after I took the time to write an essay that met all requirements, I got a grade that reflects that effort.  The fact that I won't be getting that grade is, in a way, frustrating.  I also feel that this new system of grading is a way to reward the kids who don't put so much effort into their papers.

One thing that will take some time to get used to is not having a written grade on each paper.  Knowing my grade right away is something I've grown used to.

Wonderful. That is what I think this new idea is.

I think the most interesting aspect will be trying to adjust from just looking at a grade to looking at the feedback.  I might end us stressing about my grade instead of looking/reviewing/applying feedback and suggestions.  I really don't know what will happen.

In the past I would take grades to heart, yet not completely understand what I had done to receive those grades.  Now without the confusion of a rubric and the introduction to the more frequent one-on-one conference system, I can change specifically what needs improvement.  Also, I do not mind the number/grade on my paper as long as I can pin point how I did.

Conferencing will make me more confident about what I am writing and therefore make it better.

I like the idea of conferencing after every paper (or even before).  I think that this will help to get a stronger understanding how to improve my work and this will help me focus on how to improve my writing.

I believe this new procedure will definitely help me.  At my old private school there were no such things as grades.  The teachers offered criticism on paper, not numbers.  When I came to this school I became focused on numbers.

It will push me to give 100%.  I think it will also aid my ability to conference over my work.  It will also help me acknowledge my faults and bad habits.

Another good thing about this is that conferencing will help me start planning ahead of time.

I like the change.  It may be challenging but I never ever understood the 4, 3, 2, 1 rubrics anyway.  Isn't that all just opinion?  At least now I can talk about my work instead of trying to understand why it is a "3".

I have mixed feelings, but one good aspect is that we are now required to conference a lot.  When we'd have opportunities to conference on our own I never really took them seriously.

I like the change.  If I get a bad grade now I think "Oh wow I'm a terrible writer," but if we don't have a number written on it then I can truly just look at the feedback to help me.  Also, I won't be as afraid to take my paper back out of folder to look at it again.  I hate looking at Cs.

Usually when I get the grade I enjoy getting the percent and seeing what I got on the rubric.  However, talking it out is also good for what I have stated earlier: growth.

In most classes the only communication is handing in your paper and it coming back with a grade on it.  Conferencing frequently with a focus allows us to get help and actually talk about our writing instead of a grade.  I think a lot of people will benefit from this.

All in all, I respect and love this idea and I wish other classes would adopt some form of this idea.

I don't like the fact that we are grading everything at once.  It doesn't feel right to me.  A grade should come as a grade for one assignment, not for multiple assignments.  However, I do like that I get to meet often with now.

I think this will be a challenge for me because I like see a rubric or grade so I can see where I lost points.  I'm not sure if I like the portfolio yet, but I will most likely have my opinion after I write my first paper and have my conference.

I think it is better to talk in person and to talk about what I can do better next time, than try to decipher the notes on our story.  I like that we are also being graded on our growth through writing rather than in the content of a story or essay.

I think that the best part of the new grading technique will be the fact that you really can't compare the outcomes of a single paper to someone else.  For me, I think it will help me think more about how I can personally improve rather than stressing about how I compare to my peers.  These changes will me not be so competitive and bring off a lot of stress from my shoulders.  I am really looking forward to it.

I think the new grading system will be an overall beneficial change for me.  My first reason for this is because when I get a graded paper back I do tend to just look at the grade, then I put it away.  I will read the comments but I generally will not go back and try to improve my story again.

I though about it and I usually depend on a grade.  You said we have been trained that way and I completely agree.  I usually walk home and go straight to SAM looking for my grade.  Right now I am starting to think that this could help me focus more on improving my writing.  Not that I don't already do that, but instead of doing it for the grade, I can do it for myself.

I've become dependent on letter grades to tell me if I did what was asked and how well I did it.  I don't think this would necessarily be a bad change it would just be different.  I think I will get used to it eventually but it will take some time because ever since we've really started school we've been give a grade on every paper.  It is going to take some time.

The class discussion really opened a whole new door for me.  I began to think about how I am going to be able to cope without the reassuring illusion of a solid grade.  I am sure I will be fine but this will definitely affect my writing directly.  It will be interesting to go through this experience.

I think it will be challenging for me to not get a letter grade.  To me a letter grade tells me how I did, really the only thing I look for on a paper.  Though, I do like the idea of the portfolio and talking more.  I think, actually, that this new way may make me think about becoming a better writer.

The best aspect of this is conferencing more meaningfully.  Before I wouldn't be sure what you thought I did right or wrong.  The comments helped, but some things were still left unclear and I always didn't meet with you.  Now I have way to talk about what was right or wrong.  Usually the grade tells me if my paper is good or bad.  Now maybe that will just happen in a conference.

If I see a grade I just care about that.  If I do not see a grade I do study the paper more.

I have always depended on individual number or letter grades as a source of comfort, even praise, or as a signal that I'm doing something wrong.  At the moment, I am not open to this change.  I would go as far to say that I am defensive of the classic system. 

However, I know myself well enough to know that I will eventually accept the new way of grading or even prefer it. I kind of like the change in the way we will be receiving grades. 

Almost everyone immediately looks for a number/letter grade.  They may read the comments, but do they apply the comments to their future work?

Now I am going to have to really go over my stories and work a lot longer on them to make sure that I am actually getting better as a writer.  It sounds like I need to work harder.  I'm kind of nervous about the conferences because I am not the kind of person who stands up to the teacher.

No offense but I like to know whether I did good or bad so I think we should keep things how they are.

The new way makes it sound like I will be less likely to repeat the same mistakes.

As it will be tough for me at first, I think it will focus my attention more toward improving my writing instead of writing just to get a good grade.  In the end, I am looking forward to improving my writing, and think that I will have a lot of opportunities to do it now.

I'm really just used to looking at the grade and being done with the paper.  I like that you are now making me fix my mistakes .

I think the most difficult and frustrating part of the portfolio will be fixing my weaknesses.  They are, after all, my weak points.  I think it will be fun to adjust to the idea of no letter grades.  But I also think that turning my weaknesses into strengths will help my writing immensely.  I look forward to becoming a better writer.