Main | About fractals and mountain-climbing »
June 27, 2007
From the macro to the micro
In a few hours I’ll have spent a week at the Akshara Foundation, or almost a week, excluding this weekend’s somewhat undeserved vacation. Looking back on it, the week went by quickly. When I left Harvard, I had the impression that I had left behind with it the world of substantive theoretical conversations about nonprofit work. Not that there wouldn’t be theory outside of academia – I had at least a broad enough idea of theory and practice to know that they were difficult to separate – but I believed that nonprofit organizations were, on the explicit end of things, preoccupied with details. Yet the week has gone by full of conversations about the big picture. At the management level, Akshara, unlike many nonprofit organizations I’ve been involved with, has a straightforward mission and a strong commitment to assessing the match between its mission and its outcomes. And rather than causing it to lose sight of the details, this orientation seems to make its approach to faithful implementation somewhat relentless. In many ways it’s a student’s dream come true.
What this means is that I’m already seeing in front of me many of the questions we grappled with in my master’s program, questions many smaller organizations are not even close to being ready to worry about. For example, my third day here, I accompanied A, the head of the library program to a meeting of block coordinators and ERC managers – responsible for each of the nine designated blocks of urban Bangalore – and had a conversation with him on the bus about the ethics and the usefulness of randomized controlled trials. That same afternoon, I attended a meeting back at the Banaswadi office, at which the heads of various programmes reported on their progress. There was an atmosphere of critical friendship. People were lighthearted when necessary, but also seemed to demand that their peers be articulate about justifying their decisions and understanding their terrain. Each person seemed to have a deep knowledge of the other programs, both from a mission standpoint and an operational one. What amazed me, too, was the huge scale at which these leaders think and operate on a day-to-day basis.
Finally, at the end of the week, I joined the block coordinators – many of whom I had seen previously at the library meeting – A, the head of the maths program, and C, the Chief Operating Officer, for a meeting at which A asked block coordinators to provide summary data on the results of a maths baseline assessment. A was concerned about the data because it seemed that something had gone wrong with the testing – much fewer students had low enough scores to qualify for the remedial program than expected. Some teachers reported that multiple students had scored 100% on the baseline test. Besides, the data collection was delayed – it was difficult to get to certain parts of the district; there were power outages; students were absent; teachers were absent. Many schools were missing.
In this first week, I’ve noticed Akshara’s complicated relationship with data. On the one hand, there’s the idea that numbers can drive educational reform by drawing attention to service gaps, identifying problem areas, and making maximum impact. A has already asked me two or three times whether I’m “good with statistics and numbers.” On the other hand, there’s the constant struggle to achieve accurate numbers. At this large scale of operations, in the long process of consolidation that places a single child in the context of a citywide school system, there are many places where a number can fall through the cracks, where data can get distorted and lost. And there’s the constant question of whether the numbers really tell you much at all. What does it mean to read at the sentence level, anyway? How might different teachers define it as they gather information? Is it really fair to deny thousands of children access to a library, even when you have enough funds to provide it, in order to prove that the libraries achieved so many standard deviations of literacy skill? Some of these questions are part of the painful process of honing and streamlining and eventually producing a reasonable dataset. Others are inherent limitations of research.
There also seem to be challenges in using the discourse of statistics itself, because that discourse highlights the differences between Akshara’s staff at different levels of operation. Almost everyone I’ve talked to at a management level at Akshara has spoken of the challenge of communicating the importance of data to people without data experience. “They just don’t understand,” is one common refrain, another is “Education levels are very low here.” Of course, the irony is that this problem is the exact one Akshara is trying to combat in the first place. And after this week, something A mentioned to me my first day has become increasingly clear: the significant role of building skills within the organization. The same systemic shortcomings that exist outside the organization exist within it as well – in this case, a lack of quantitative and, even more broadly, independent critical thinking skills – and must be faced directly to be overcome. It is the same philosophy that drives the idea of the Karnataka Learning Partnership. Work with a flawed system in order to improve it, rather than creating an isolated, perfect programmatic specimen that’s impossible to reproduce.
This philosophy tends to operate at the level of “dream big and then make small adjustments” rather than “dream small and expand the vision slowly.” In multiple conversations, I’ve heard assertions of this kind of optimism. We can’t afford to compromise, Akshara seems to believe. If we make the ambitious claim that we can eliminate illiteracy in 5 years, we might eliminate it in 8; if not, we may never eliminate it at all. In fact, when I first came across the Akshara Foundation last summer while I was working with the Deshpande Foundation, this attitude struck me right away. It was unlike anything I had ever seen, and I wasn’t sure whether to be incredulous or impressed (now I’m a little of both). I’ve noticed this attitude manifested in the interactions I’ve seen between block coordinators, A, and A. It seems like A and A are always asking block coordinators to do things quickly, and block coordinators are explaining why they can’t. The power is out, or the internet is down, or the rains are making the roads bad. It’s the same kind of request the very notion of the Karnataka Learning Partnership makes of the government. It asks schools to make children literate, a demand that most people, I imagine, have probably given up on. The underlying idea seems to be that only by asking the impossible question can you begin to make answering it possible.
At the top couple of levels, this seems to work, as far as I can tell. The data might not arrive tomorrow, but it arrives maybe a week later. And Akshara’s large-scale successes show that, eventually, its impressive aims are often met. That said, I’m very curious now about what the process of gathering statistics looks like on the ground in the intervening time. What corners are cut? What principles are followed? At the meeting of block coordinators I attended with A, I was surprised at how little of the time they spent talking. They referred to A as “sir” and reported to him the way a student would to a teacher. There were things about Indian work culture I had yet to understand. In later conversations, however, I was amazed with the amount of work they did. Each responsible for several clusters, they coordinate the library program as well as the maths program. Their work reminds me of student life not only in the way they report to higher-ups, but also in the way they are responsible for various demanding programs – just like students juggling classes with multiple professors, each enough to fill more than a regular work schedule.
Despite what might look like a very trickle-down organizational structure, the block coordinators seem very committed and engaged in their work on an individual level. One block coordinator, S (who tolerated my Kannada and smiled encouragingly after I nearly threw up my second time on a Bangalore bus) told me again about the importance of statistics. She told me that Akshara’s successes with funders were largely due to its ability to demonstrate with data the impact of its work, and bemoaned the fact that some staff did not value data collection. Some teachers, she said, didn’t know how to calculate percentages – considering that this came through while collecting data for the baseline assessment for the maths program, this is a particularly jarring fact.
With characteristic insistence on improving the system, Akshara works with teachers both to assess and improve the performance of students who are falling behind. What I would like to hear more about, though, is what teachers might be doing that is especially positive for children. So far, I’ve only had the chance to learn about teachers as a piece of the complex machinery of Akshara’s work – often as a challenge, but sometimes as neutral implementers. But teachers are the heart of the Karnataka Learning Partnership. After all, Akshara’s “product” is not a piece of hardware: it is, at least in its most utopian vision, simply a demand for better teaching. And as such, it is messy and nebulous. What is good teaching, after all? At the top level, teachers look like tiny specks in a huge constellation. I’m interested to know what the variations between those tiny specks might be. It would mean coming to understand what exactly, at a curricular level, is working best within Akshara to produce those coveted numerical results. Otherwise, the demand for quality education lacks a definition of exactly what quality is.
At the end of this week, my curiosity is wandering away from a friend’s recommendation to “think big” as I explored development work here in Bangalore. I find myself increasingly curious about the small pieces of Akshara’s work. I’m wondering how teachers actually teach this material, what from the training they implement and what they ignore. I’m wondering what the teaching process actually looks like, what the implementation turns out to be between the baseline and the midpoint, and between the midpoint and the posttest. I’m wondering what they do well (and what they do badly) that Akshara’s training hasn’t thought of. I’m wondering how cluster volunteers go about collecting data, how block coordinators divide their time. And at the end of it all, I’m wondering how the collection of data will actually serve as the push for effective service delivery. How will the data be disseminated? Who will use it? What elements of the data will they ignore, and what will they use? What will they use it for? What will bridge the gap between information and its use for change, the same gap that plagues even the most mundane of Akshara’s daily tasks?
Posted by gowriv at June 27, 2007 12:05 PM