« About fractals and mountain-climbing | Main | Data in the classroom? »
August 04, 2007
About data and how to use it
It’s amazing that after over a month at Akshara I continue to learn more about how the organization works. In the past two weeks, I’ve met more cluster volunteers, two BEOs, some BRPs and CRPs, P, the academic who designed the math curriculum, a few teachers, a group of third-standard students, and two representatives from H, who provided library content and training.
Two large questions have become a recurring theme, both related to assessment. The first is this: What does assessment actually assess? And the second is: What should assessment be used for? In fact, the questions are actually almost the same.
When I visited the school in Jeevanbhima Nagar with research students from Columbia, the first thing I noticed was how nice it was to see kids again – it seemed that over the years, the more I learned about development and community work, I spent less and less time with kids and more and more time burrowing in my laptop, discussing kids with people who were not kids, and reading what other people had written about kids.
I also started to think more about the question about what assessment should assess. Assessment is a powerful tool in curriculum, and not just in its role at the end of the process: it shapes the focus of the curriculum and makes a final statement about what was important and what was not. An assessment is almost a philosophy of education, in that it identifies the key elements that educators seek, ultimately, to impart to their students in a lasting way. Are you being marked on your writing skills? Your ability to question? The quickness of your memory? The strength of your foundations?
So what was the pearl of wisdom that the Columbia economics team had to offer on what matters in education? Ostensibly, the mantra that permeates through all of Akshara’s activities: basic skills so that children have something to build upon. As I was watching the children take the pilot test, though, I couldn’t help thinking that it actually stifled the things I thought mattered in education. As proctors of the pilot test, we were responsible for telling the children not to copy, not to talk. As the CV explained to the children what they needed to do, the burst out with a chorus of questions, directed at her, at us, and at each other. There was lots of collaborating, lots of copying. It seemed like these children thought of the test as a group exercise rather than a chance to demonstrate their individual progress.
Watching the children interact with the teacher and each other made me question the values testing imparts. After all our talk of interactive learning, of bringing the teacher and the child closer together, the endpoint is the message that what’s really important is for you to sit at your desk, not look to your left or right, not help anyone, not ask for help, not receive support from the teacher, and get the right answer. It seems like a mismatch. Wouldn’t it be interesting to test students and reward them for their ability to work together to solve a problem, instead of their ability to struggle with things in silence alone? What is the boundary between individual success and group success?
I think, in the end, this is an ideological question, but it seems, ironically, like the culture of testing has had enough power to offer support to things like cooperative learning. Slavin’s research, for example, argues for cooperative learning because it improves test scores – a cooperative process to achieve an individualistic outcome. Interestingly, though, Slavin also concludes that cooperative learning works best when students know that their individual assessments depend on those of others in the group: their successes are interdependent. So cooperative learning is both an end in itself – because, obviously, it teaches cooperation -- and a means by which to improve students’ individual performance. Testing students individually also has the advantage of being much more apppealing to institutional culture than allowing students to “cheat” – it’s able to show the results that can be used for good advocacy for good causes.
The H meeting I attended danced around a similar question – what is important to assess? – though this time, it related to the relationship between a child and a book. The main conflict was between the importance of processing a book as a whole and the importance of being able to read words on a page. H, leaning on the premise that books with less text on each page are easier for children to understand, cares about whether a child can access and understand, generally, a book. Akshara cares about whether the words on the page are made up of letters that children can read. Complicating the whole discussion was the reality of Kannada publishing – there simply aren’t many books for low-level readers with simple words. Then there’s the question of development: is an older child who can’t read simple words interested in the same book as a younger child? Probably not.
Here too, there was the concern that what you assess defines what you value. For H, what is important is that kids love books, even if they aren’t literate enough to read all the words in them – thus, assessment hinges on the book layout and accessibility. For KLP, what is important is that kids can read, and assessment focuses on letters much more than on complexity of meaning. I wonder how these two axes of reading skill could be incorporated into an assessment system. For example, what if books were assessed by word complexity, with an additional rating for book complexity? Then we’d assume that a young, poor reader would read a book with low complexity and low word complexity, but an older poor reader would read a book with low word complexity and high book complexity – a more mature story, fewer graphics, and perhaps a more challenging vocabulary.
At the end of all this assessment is the question that I began working through with A this week – what should assessment be used for? In an a large meeting, P nsisted that there was a difference between diagnostic testing and assessment. Diagnostics are used to analyze problems and design solutions for them; assessments are used to make a final judgment on something’s success or failure. In program evaluation, this is the distinction between formative and summative assessment.
It seems to me that KLP’s foundation, at least in theory, is the idea of diagnostics – remedial programmes should be setting off a long-term cycle of improvement, with each test offering the information that can inspire the next round of improvement. This was my work with A on policy models for KLP. Yet my evaluation professor’s distinction between formative and summative evaluation is key here: the final deciding factor in whether an evaluation is formative or summative is the way it is used. In the case of the math programme, the tests are not diagnostic: teachers likely will not use them to shape their teaching. Even the reading programme’s tests were not diagnostic – the data mostly served to affirm success rather than to identify problems and solutions.
This is the question that a BRP from the South 4 district raised when we went to visit the BEO office there. Sitting in the back of the room, she began to contribute her thoughts on KLP. She had a lot to say, but the very first thing she said was that KLP made teachers feel “down” – by assessing students so regularly, the programme only managed to affirm that teachers were doing a bad job and someone else could do a better job than they could. The worst part, to her, was that the teachers were actually doing all the work in both cases.
I think that underlying her problems with KLP was the same distinction between diagnostics and assessment that I’ve been talking about. To her, the reading programme had been assessed, not diagnosed: KLP came in, imposed a programme, declared success, and left. This had left teachers, according to this BRP, feeling attacked rather than supported. The last part of the cycle was missing: the feedback and planning for the next strategy for improvement. The BEO told us that she hadn’t seen much impact of the reading programme in her block – perhaps an indication that one turn of the wheel is not enough to make a lasting change. What defines KLP’s testing as diagnostic will be the way it is used, and so far it has not been. This space is where I am hoping to insert myself over the course of the year.
Posted by gowriv at August 4, 2007 12:09 PM