ARTICLES

 

Identifying mechanisms of change in the Early Grade Reading Study in South Africa

 

 

Brahm Fleisch; Kerryn Dixon

School of Education, Wits University, Johannesburg, South Africa. brahm.fleisch@wits.ac.za

 

 

---

ABSTRACT

The Grade 2 learner results of the Early Grade Reading Study (EGRS) randomised control trial (RCT) in the North West province showed that the structured pedagogic intervention model that included instructional coaching, lesson plans, and quality learning materials was more cost-effective than a similar intervention model that excluded coaching. These results are read against the growing body of literature that questions the value of RCTs in identifying mechanisms of change. For this reason, two small-scale qualitative case studies were undertaken to identify the difficulty in detecting workings of classroom practice that RCTs cannot show. Three mechanisms of change: the organisation of space, time, and implementation of routines drawn from the work of Foucault, were analysed to interpret observed changes to teacher practice and some of the weaknesses in the EGRS programme.

Keywords: early grade reading; Foucault; qualitative case study; RCTs

---

 

 

Introduction

Angus Deaton (2010) has questioned the value of randomised control trialsⁱ and the promise of the experimental method to improve social policy. He and his colleagues argue that the fundamental problem with RCTs is that they do not uncover mechanisms of change:

I also believe that RCTs of 'what works,' even when done without error or contamination, are unlikely to be helpful for policy, or to move beyond the local, unless they tell us something about why the program worked, something to which they are often neither targeted nor well-suited […] For an RCT to produce 'useful knowledge' beyond its local context, it must illustrate some general tendency, some effect that is the result of a mechanism that is likely to apply more broadly. (Deaton, 2010:448)

More recently, Deaton and Cartwright (2018:18) expanded on this theme:

Yet without the structure that allows us to place RCT results in context, or to understand the mechanisms behind those results, not only can we not transport whether 'it works' elsewhere, but we cannot do the standard stuff of economics, which is to say whether or not the intervention is actually welfare improving … Without knowing why things happen and why people do things, we run the risk of worthless casual ('fairy story') causal theorizing and have essentially given up on one of the central tasks of economics.

It is precisely to move beyond fairy stories to explore why things happen that in-depth case studies were undertaken in the Early Grade Reading Study (EGRS).ⁱⁱ We use Deaton and Cartwright's formulation of a mechanism. From a philosophical perspective, a mechanism refers to causal processes located in systems of interacting parts which make some processes possible and others not. "'Mechanism' in this sense is not restricted to physical parts and their interactions and constraints but includes social, cultural, and economic arrangements, institutions, norms, habits, and individual psychology" (Deaton & Cartwright, 2018:12).

The article begins by locating the current EGRS study in the emerging literature on effective intervention models that improve early grade learning outcomes. Taking a cue from Deaton's critique of individual RCT studies and systematic reviews, we recognise the absence of substantive engagement with contexts within them. But, most importantly, we recognise that little of the current research pays attention to understanding how change occurs. We briefly outline the state of reading research in South Africa before discussing our research methodology, framing how qualitative in-depth case studies complement EGRS data collected through multi-wave learner achievement testing and large-sample structured classroom observations.

Literature Review

RCTs in educational research

The dramatic increase of RCTs in education, specifically those that aim to contribute robust evidence to what works to improve learning outcomes, has given rise to meta-analyses and systematic reviews. McEwan's (2015) review found that combined programmes were the most effective models, although teacher training and other related programme types work to some extent. Since the visit of researchers from the International Initiative for Impact Evaluation (3ie) to South Africa in 2015, we have been familiar with the systematic review 3ie completed on learning and school participation. Like McEwan (2015), Snilstveit, Stevenson, Menon, Phillips, Gallagher, Geleen, Jobse, Schmidt and Jimenez (2016) argue that programmes using structured pedagogy work best to improve learning outcomes. In contrast, Masino and Niño-Zarazúa's (2016) systematic review found that the most effective intervention models are those that combine accountability with supply-side interventions. Glewwe and Muralidharan (2015), in a conceptual rather than systematic review, argue that interventions that focus on improved pedagogy (especially supplemental instruction to learners lagging behind grade-level competencies) are particularly effective. (But they also point to interventions that improve governance and teacher accountability.) Their insights are heavily influenced by research from Pratham in India.

With all these and other meta analyses (e.g. Conn, 2017) of what works to improve learning outcomes, it has emerged that although they analyse the findings of the same robust studies, the conclusions drawn are different. Evans and Popova (2016) argue that while there are unique insights, there is basic common ground across all the meta-analyses and systematic reviews. The common ground is the insight that "two classes of programs are recommended with some consistency. Pedagogical interventions that tailor teaching to student learning levels―either teacher-led or facilitated by adaptive learning software―are effective at improving student test scores, as are individualized, repeated teacher training interventions often associated with a specific task or tool" (Evans & Popova, 2016:242).

Evans and Popova (2016), and others have been criticised by Haddaway, Land and Macura (2017) for using methods at wide variance from accepted criteria for systematic review/synthesis. Deaton and Cartwright's (2018) most useful critique is that these studies seldom explore the mechanisms that make the interventions work. Without understanding these mechanisms, Deaton and Cartwright argue that RCT findings have limited policy utility as it is difficult to assess what can and what cannot be transferred or scaled up.

Our review of the research on what works to improve learning outcomes suggests the need to better understand whether an intervention model is impactful, and equally how it works, particularly when implemented at a large scale. We are persuaded by the Deaton-Cartwright argument about the importance of understanding mechanisms. We believe that in-depth qualitative case studies are one way of beginning to understand complex systems of interacting parts. To do this, context is crucial, as are theoretical lenses to interpret the data. Contextualised reading in the early years and Foucault's work is thus contextualised as one way to identify and understand mechanisms of change.

Early reading research

The crucial role of literacy in learning success is generally accepted. What is now commonly referred to as the literacy crisis in the country has not diminished despite the implementation of a number of programmes (e.g. Foundations for Learning Campaign [Department of Education, 2008a]; Gauteng Primary Literacy and Mathematics Strategy [Republic of South Africa, 2012]; National Reading Strategy [Department of Education, Republic of South Africa, 2008b]). A decade of Progress in International Reading Literacy Study (PIRLS) assessments (2006, 2011, 2016) reveals that South Africa continues to be the lowest performing country. Data from the latest assessment shows that 78% of Grade 4 learners do not meet the lowest international benchmark, which means that they cannot read for meaning, or answer simple questions by retrieving basic information from a text (Howie, Combrinck, Tshele, Roux, McLeod Palane & Mokoena, 2017).

Many of the reasons for the reading crisis have been identified. General factors like the legacy of apartheid and ongoing systemic inequity, poverty, inadequate access to resources, and a lack of sustained funding for national programmes are compounded by insufficient teacher content and pedagogical knowledge about literacy, language, and lack of a literate culture (Fleisch, 2008; Hoadley, 2017, 2018; National Education Evaluation and Development Unit [NEEDU], 2014; Spaull & Hoadley, 2017; Taylor, 2009). In addition, due to limited research of literacy in the Foundation Phase it is not possible to determine what works. This is particularly true for the teaching of African languages, which is evident from a recent review undertaken by Pretorius (2018).

Theoretical Framework

In order to begin to better understand "why things happen and why people do things" (Deaton & Cartwright 2018:12), we used the work of social theorist, Michel Foucault, to read qualitative case studies. Although the language he uses is different from that used by Deaton and Cartwright, he provides an analytical framework for identifying and understanding the interacting parts that constitute mechanisms of change. Foucault's work focuses on the ways in which mundane practices that take the form of norms, shape who we are, creating what Deaton and Cartwright (2018) refer to as habits. He is also aware of the ways in which socio-cultural institutional arrangements make processes possible and constrain others.

Foucault (1977) was interested in how people learn to behave in expected ways that are considered to be normal. This happens through bodily training, the standardisation of actions over time, and spatial control (Rabinow, 1991). We consider these to be mechanisms of change. People are subjected to external controls that organise and classify them in various ways. These ways are informed by bodies of knowledge, which are expressed discursively as particular views of the world. People come to be when, what Foucault calls "disciplinary technologies," knowledge, and discourse work together.

This notion is useful because disciplinary technologies work at the level of the mundane, in small acts. Methodologically, this is important to us as the reasons for change are not always large and obvious. Foucault provides a way to talk about how teachers' practices may be more productively reconstituted beyond merely implementing alternative practices from a programme like the EGRS. We know from failed interventions that take-up is low because change needs to be embodied and habituated. Teachers have been trained to teach in certain ways and these regimes of truth are associated with what it means to be a teacher, and about what teachers know and do. The EGRS programme challenges some of these long-held practices as a new form of truth. It is underpinned by a body of knowledge/view of the world on what literacy is, and how reading should be taught. This view requires of teachers to take on a set of (bodily) practices that may be different from their understanding and practice of literacy. Foucault does not discount people's agency or capacity for resistance. Teachers do not simply take on new practices. Old ways of being and old discourses have been internalised, entrenched, and embodied. Take-up is, by necessity, uneven, as new practices can be in opposition to old practices.

 

Method

The Early Grade Reading Study

The EGRS I (the second early grade reading study is currently underway) focuses on measuring the cost-effectiveness of three early grade reading intervention models. The first two interventions with teachers comprise an "education triple cocktail" (Fleisch, 2018), which consists of tightly aligned scripted lesson plans, quality learning materials, and capacity building. Intervention 3 is a parent intervention to support early grade reading at home. The key difference between Intervention 1 and Intervention 2 is the type of capacity building. Intervention 1 uses conventional training in two-day out-of-school training sessions, while Intervention 2 uses instructional coaches who work on-site in schools. Using baseline, midline, and endline learner testing, the RCT provides robust evidence of the marginal impact of the three intervention models relative to the control group (Cilliers, Fleisch, Prinsloo & Taylor, 2019). The comparative learning testing results, however, do not provide insight into the mechanisms of change.

Design

Understanding the mechanisms of change requires a longitudinal multiple case study design. As this is costly, we used multiple in-depth snapshot case studies. However, the disadvantage thereof is that these case studies do not provide evidence of change over time, but they do provide insight into the process associated with the enactment of new practices that form key intermediary factors between the intervention and the improved learning outcomes.

This article provides evidence from two stand-alone sets of case studies. The first was undertaken by one of the EGRS principal investigators and a university colleague who specialises in early grade reading. The second, commissioned by the EGRS, was undertaken by a leading independent consultant (see Reeves, 2017).

Sampling

The two study teams used distinct approaches to case study sampling. The Principal Investigator (PI) team (Case Study Team 1), who undertook the research a few weeks prior to the external consultant, focused on Intervention 2 schools and used extreme sampling criteria (McMillan & Schumacher, 2006) to select four schools - two showing strong and two showing weak performance (the top and bottom 10% of the study population on the midline results). The external consultant (Case Study Team 2) selected schools of which performance was in the middle of the study school population distribution. The two independent sets of case studies enabled the study team to gain insight into school context and teachers' views and enactments from three very different types of schools, albeit, in the poorest three quintiles.

Data Collection within Schools

The two case study teams took similar approaches to data collection. Case Study Team 1 observed 12 lessons in Grade 1 and Grade 2 Home Language (Setswana) classrooms. Principals, Heads of Departments (HoDs), and teachers whose lessons were observed were interviewed about their views of and attitudes towards the EGRS in general, components of the EGRS, (in particular about the lesson plans and coaching), and contextual factors. Eighteen interviews were conducted. Photographic and video evidence of the classrooms and wider school environment were collected. The researchers took field notes of the classrooms and during the interviews. After each day of field work the team discussed the site visits, observations, and interviews, after which summaries of the collected data were recorded.

The case study 2 consultant worked with a fieldworker who was a Setswana speaker. She did lesson observations, interviews, and examined classroom documents and records. She recorded information on Grade 2 teachers' timetables, lesson plans, work schedules, and year plans, and she scanned learners' workbooks, exercise books, and assessment records.

The focus of her review was on

· curriculum planning, instructional alignment, and macro pacing,

· curriculum coverage,

· opportunities to write (the number of pages of work in learners' workbooks, the number of days' work in workbooks, type and frequency of writing in learners' workbooks),

· frequency and quality of teachers' assessment of written work in assessment records.

A feature of the consultant's data-collection approach was her focus on rich time-based records describing and tracking teacher and learner activities. Her interpretation of the case studies was framed against her earlier observational studies of early grade reading in Limpopo and the Western Cape (see Reeves, 2010; Reeves, Heugh, Prinsloo, Macdonald, Netshitangani, Alidou, Diedericks & Herbst, 2008).

Data Analysis

Three steps were followed in the data analysis. Firstly, the data sets were compared with the EGRS's daily lesson plans to establish levels of compliance and gaps. Secondly, a Foucauldian lens was applied to the data to analyse the ways in which normalised (habitual) practices were taken up by individuals and within the institution by considering how shifts in the operation of time, space, and mundane, routine practices operate as mechanisms of change. Time-based records from classroom observations were constructed. References to spatial movements from these records, field notes, photographs and interviews were triangulated. Patterns of routine practices were identified from observations and cross-referenced to the EGRS lesson plans and interview transcripts. Lastly, incidents of resistance/disruption were identified. These were interpreted against literature reporting on entrenched practices in South African schools.

 

Results

It is important to note that mechanisms of change are interrelated, but in this paper it is not possible to identify all the causal processes of the interrelated parts. The operation of time and space are mutually constitutive (Foucault, 2000) but are, for purposes of analysis, separated here.

The Use of Time

Data from the classrooms indicates time operating at a number of levels: the overall use of curriculum time for the teaching of literacy, the time taken to cover a range of literacy activities (time for activities), and transitional time (time between activities). Observations revealed that teachers taught longer than the minimum time of 7 hours per week for Home Language (HL) lessons prescribed by the Curriculum Assessment and Policy Statement (CAPS) (DBE, Republic of South Africa, 2011). This revealed a shift in practice as previous studies showed that South African teachers did not spend enough time on teaching, and that time spent on reading was below the PIRLS mean (NEEDU, 2014; Taylor, 2009). Table 1 summarises the duration of lessons observed for the four schools in Case Study 2. Table 2 narrows the focus summarising the lesson duration within one Case Study 1 school.

While teachers, for the most part, appeared to be consistently teaching HL each day for the scheduled 90 minutes, they complained about not being able to cover the curriculum. Teachers in Case Study 1 schools talked about feeling pressure when they fell behind, and then took time from Numeracy and Life Skills lessons to catch up. Case Study 2 teachers mentioned that they implemented "extra time" classes before school to help learners catch up.

Prescribed literacy activities outlined in the lesson plans were observed in both case studies, which was an indication that teachers were following the structured programme. An analysis of learners' books supported this as they showed evidence of prescribed writing tasks. While not all writing tasks were systematically completed or completed for every lesson by all classes, these tasks were routinely done and the quantity of writing surpassed that reported on in schools in the NEEDU report (Taylor, Draper, Muller & Sithole, 2013). The following literacy tasks were observed across both case studies:ⁱⁱⁱ

· Phonics instruction,

· Handwriting instruction,

· Creative writing instruction,

· Group guided reading,

· Shared reading,

· Listening and speaking,

· Individual work, and

· Independent seatwork at desks.

Tables 3 and 4 provide detailed breakdowns of the times and types of literacy activities that took place at Case Study 2 schools. An analysis of the tables reveals two aspects where time impacts on literacy teaching. The first is that time is not constructed linearly (Dixon, 2011; Foucault, 1977). Teachers and learners participate in both independent and small-group tasks scheduled simultaneously, thus increasing the number of literacy tasks that can be done. This requires a greater level of control and organisation on the part of teachers, and a level of reinforcement, so that learners can work independently. It also indicates a shift in the norm of whole-class teaching in many Foundation Phase classes.

The second aspect is the amount of time in which transitions between tasks and other lessons are made. While not all observed transitions were managed smoothly, there were indications that teaching time was utilised more efficiently. The learners had been trained to hand out books between tasks and appeared comfortable with this routine. There did seem to be issues when the content of the scripted lessons was not conceptually demanding enough. Fieldnotes from Case Study 1 are a case in point,

Time seems to collapse towards the end of the lesson. Once they [the learners] get to shared reading, there is a half hour gap where nothing really happens. Teacher marks the books, but the kids sit quietly and wait for what is happening [next].

Overall, the operation of time as a mechanism of change resulted in better pacing and sequencing of lessons, better coverage of curriculum content, and the imposition of routines, long identified as challenges to quality teaching and learning (Hoadley, 2017).

Organisation of Space

Productive shifts in spatial reorganisation indicate how space operates as a mechanism of change. Although many of these shifts are invisible and mundane in highly-functioning classrooms, their presence cannot be taken for granted in South African classrooms. Foundation Phase classrooms can be characterised by four spaces: the teacher's desk, the learners' desks, a reading corner, and a carpet (Dixon, 2011). The placement of learners' desks is important for learning, and it gives insight into teaching practices. Although the placement of desks differed slightly in the classrooms, learners in all eight classrooms were able to see the teacher when she did whole-class teaching and demonstrations. Some classrooms had desks that accommodated two learners in rows; others had groups of connected desks arranged in rows lengthwise across the classroom.

The use of the carpet was another important use of space. Teachers used the carpet to do whole-class as well as group guided reading tasks. Two of the eight Case Study 2 classrooms did not have space to accommodate a carpet or a reading corner.

Teachers used the resources from the programme to decorate their classrooms. Teachers made flashcards, which were displayed on the classroom walls for specific lessons (see Figure 1).

A commonly held perception is that many Foundation Phase classrooms in underperforming schools appear to be resource-poor. Resources are often available, but are not accessible, as they are locked in storerooms or cupboards, or teachers do not know how to use them adequately. The same may be said of learners' exercise books. It was evident in both case studies that teachers were using the resources from the programme, Department of Basic Education (DBE) workbooks,^v and learners' exercise books. The Vula Bula series of graded readers for group guided reading was clearly accessible, and, for the most part, neatly stacked or stored in boxes.

The increase in instructional time, access to material resources, and the reorganisation of space to accommodate whole-class and small-group teaching appears to have increased teachers' confidence and motivation. Teachers talked about learners' improved performance. Grade 2 teachers reported that Grade 1 learners who had been exposed to the EGRS programme were better prepared. They also reported feeling more able to follow the proper method for group guided reading using resources.

Even though their positive attitudes were evident, teachers had to deal with the discomfort of change. Reflecting on this process and the value of scripted lessons teachers said:

Ms S: Yoh, that uploads your work. Firstly, it was difficult doing something that you didn't plan, but as time went on and we went to workshop, it becomes so easy [...] I wish it cannot come to an end.

Ms D: Firstly, it was difficult […] even if we were trained [...] I didn't know where do I look for group and individual teaching? It is nice that everything is written there, that everything is written step by step.

The impact of routinised practices that arise from lesson plans as a mechanism of change that affects confidence and psychological well-being should not be discounted.

 

Discussion

Despite the evidence from RCTs that structured programmes work, as mentioned earlier, there is little exploration of why they work. We are interested in understanding what it is about the enactment of the EGRS that changes outcomes for learners and teachers. Descriptive accounts are useful because they illustrate patterns and may identify some mechanisms, but without a theoretical lens, it is not possible to understand how they operate as interacting parts in a causal system. This is because social practices are often subtle and not immediately discernible.

In a schooling system where this is not always the case, the EGRS's scripted lessons result in the establishment of rhythms directed onto teachers' and learners' bodies. Along with the indications that teachers were mostly keeping to time (see Tables 1, 2, 3, 4), observations indicated that learners knew which activities followed on others and what occupations they would be involved in next. This was evident in the ways designated groups of learners purposefully moved to the carpet for group guided reading. Mrs D commented that the learners complained when she reordered the groups.

The imposition of occupations and cycles of repetition (Foucault, 1977) results in increased instructional time, with a consistent focus on content. Inconsistent pacing and sequencing have long characterised South African education (Hoadley, 2012). The EGRS programme requires different daily rhythms to be internalised. The step-by-step timed nature of the lesson plans is enabling for teachers. Breaking tasks down into timed steps has the benefit of breaking time into manageable pieces that work to increase overall teaching time (Dixon, 2011). When learners and teachers internalise routines and transitions are smooth, several operations can take place simultaneously. Two examples of this can be provided. The first is learners' ability to complete handwriting tasks. This is an essential skill in Foundation Phase classrooms and requires intense bodily training. Grade 2 teachers at Ratlou reported gaining time because learners from Grade 1 showed a level of mastery that was not previously evident. The second is the impact of the number of literacy tasks allocated to daily teaching time. Teachers needed to manage their time carefully to complete prescribed activities. Pacing and sequencing were handled far better during the Setswana lessons than during the Numeracy and English First Additional Language lessons.

While the use of time and quality of instruction improved, take-up was not even across classrooms. Efficient use of time is affected by older practices, comfort with new practices, and design flaws in the CAPS curriculum and the EGRS programme, which refers to the relational nature of mechanisms of change. Practices that teachers were comfortable with had a fluidity that new practices did not have. This was evident in the way flashcards were used: teachers used them in phonics lessons, they comfortably displayed them, and they reused them in group guided reading and appeared to take pride in having made reusable resources. This is in contrast with more complex practices, which one teacher erroneously referred to as "group and individual teaching." While the teachers followed the lesson plan steps, it was clear that they did not all have full mastery of tasks. The researchers noted that eight months into the year teachers still relied on lesson plans and referred to them often. At one level the teachers should have mastered routine and repetitive tasks. On another level, the well-used plans showed that teachers were following the programme. This raises the question of how long something takes to become habitual. We argue that this in itself is an indication of change, as teachers are reading curriculum documents in a country where many teachers seldom read these. One of the design flaws of the EGRS lesson plans is that independent tasks designed for learners take place while the teacher is doing group guided reading, remain the same for the whole year. Learners' growing proficiency and need for more challenging work is not factored in, and disruptions in class increase.

Changes to classroom practice are also evident in the spatial relations. Foucault (1977:141) observes that "discipline proceeds from the distribution of individuals in space." The structured programme requires very different bodily movements from both teachers and learners. Literacy as constructed in these scripted lessons calls for a learner who can learn as a member of the class, as an independent individual, and in a small group. This means that teachers' and learners' bodies need to be reconstituted in space. They need easy access to material resources in order to manage temporal transitions. A better overall organisation and placement of resources was evident (readers were near teachers' desks, and exercise books were stacked and accessible). Teachers knew where their teaching resources were, and activities were not delayed by looking for them.

Classroom organisation for this programme needs to allow for movement flows. Although the configuration of desks in traditional rows is in no way innovative, it enabled teachers to move to help individual learners, with the exception of the teacher at Anna, whose classroom was severely overcrowded. The classroom configurations enabled small groups of learners to move to the carpet to work with the teacher. The carpet space is often under-utilised in Foundation Phase classrooms, where whole-class pedagogy is a dominant practice. The rule of functional sites (Foucault, 1977) was evident in the classrooms, where a space like the carpet has more than one function. Creating more spaces to learn is linked to the simultaneous use of time for more than one task.

The demands of the scripted lessons supported by new materials and coaching require of teachers to use time and inhabit space in different ways. The structured nature of the programmes results in routinised behaviours, which we argue began to be internalised and embodied. The signs of new embodied practices were evident in classroom displays and classroom organisation. But taking on new practices is always partial and not always successful for all teachers. Teachers are more likely to make shifts when parts of old practices remain or can be easily integrated into new ones. (The use of flashcards is evidence of this.) When practices and the knowledge associated with them are alien, partial uptake or resistance is likely.

This was most obvious when one considers the ways in which group guided reading was enacted. Teachers took learners to the carpet to read and attempted to follow the lesson steps. The benefit of small groups is that they enable the process of individualisation (Foucault, 1977) to operate: each learner in the group becomes an individual whose abilities are seen and measured. Another benefit of small groups working in a different space was the creation of a safer classroom environment. In this space teachers and learners laughed and talked more than is often allowed in the space of the whole class. But implementing group guided reading requires a level of knowledge and view of literacy not contained in a scripted lesson plan. It requires a move from decoding to inferential meaning-making by the use of carefully phrased questions. Although the processes of group guided reading are mostly followed, teachers' gaps in content knowledge indicate a lack of embodied/habituated understandings of this pedagogy.

Spatial reorganisation required by accessible lesson plans and supported by coaches has resulted in changes in practice. The increase of quality of teaching time combined with more efficient spatial organisation impacts on classroom relations. As teachers see an improvement in learners' performance, there appears to be growth in their confidence and motivation and the beginnings of a more agentic internalisation of new practices.

 

Conclusion

In this paper we attempted to understand how the operation of space, time, and implementation of new routines function as mechanisms of change in the EGRS programme. Using a Foucauldian lens to analyse qualitative case studies has enabled us to identify how the EGRS's approach to literacy teaching is embedded in the operation of space, time, and mundane routines that have begun to be institutionalised, normalised, and habituated. The data shows how, as causal processes of interacting systems, these mechanisms enable productive changes, but at the same time constrain change. Identifying these weaknesses in the EGRS programme is useful for programme redesign so that teachers can be further supported.

While this work is clearly a key part of transforming the culture or system of reading instruction, it is not the whole story. Four further issues need to be addressed. Firstly, we need to determine what a programme or intervention would look like that allows teachers to make the second transition from greater formalism (structured and systematic instruction) to instruction that includes greater emphasis on comprehension and independent reading for meaning. Secondly, we need to understand the unique challenges of taking a proven change model from 50 schools to 20,000 schools. Thirdly, we need to anticipate the need to reform the upper levels of schooling, so that the improvement is not squandered in classrooms unprepared for better readers and writers from the Foundation Phase. Finally, the binding constraints require attention. These are policy and organisational impediments that militate against the effectiveness of new instructional practices.

 

Acknowledgements

Support for the wider EGRS I study has been provided by the United Nations International Children's Emergency Fund (UNICEF), the Zenex Foundation, the Anglo-American Chairman's Fund, the Department of Planning, Monitoring and Evaluation, and the North West Department of Education. UNICEF, in particular, provided financial support for Case Study 1. However, any flaws and errors are the authors' alone. Ethics clearance was approved by the Non-Medical Ethics Committee of the Human Sciences Research Council.

 

Authors' Contributions

BF was responsible for the overall conceptualisation of the paper and led the write up of the paper. KD was responsible for the theorising and write up of the data analysis.

 

Notes

i. Fleisch, Taylor, Schöer and Mabogoane (2017) describe features and the rationale for randomised control trial (RCT) as a research methodology. A randomised control trial is a research method designed to evaluate the impact of a social or educational intervention by comparing intervention and control group baseline and endline outcomes. RCTs use a rigorous sampling process and multi-level randomised assignment. By minimising bias and establishing a measurement of impact, the RCT method prevents researchers from reaching potentially harmful false positive findings.

ii. The EGRS is a multi-year randomised control trial study lead by the Department of Basic Education in collaboration with the Human Sciences Research Council and a number of universities. The purpose of the EGRS is to provide rigorous evidence of cost-effective models to improve early grade reading outcomes in African languages. Undertaken with the North West Department of Education, the study was conducted in 230 randomly selected schools in two districts, comparing three different interventions to schools in which business carried on as usual (control). The results from the comparison show that combined training/on-site coaching, quality materials, and scripted lesson plans are the most effective to improve learning outcomes. See Cilliers et al. (2019) for the endline results of the North West EGRS study.

iii. Not all of these tasks were observed in every class because of the EGRS's programme design, but data from the learners' books is evidence that these tasks had been completed.

iv. We are aware of the methodological issues arising from pre-arranged classroom observations. The teachers were aware that the researchers were coming and had prepared a sanitised version of their teaching. We know this because the lessons were prescriptive, and we knew what we expected to see. It is probably more accurate to say that what we observed was teachers' best performance of what they interpreted the new practice to be.

v. The Department of Basic Education provided all learners with workbooks, which contained a range of tasks and activities aligned with the requirements of the curriculum. While these workbooks, in themselves, do not provide adequate curriculum coverage, the expectation is that learners must use them.

vi. Published under a Creative Commons Attribution Licence.

vii. DATES: Received: 11 June 2018; Revised: 7 November 2018; Accepted: 27 April 2019; Published: 31 August 2019.

 

References

Cilliers J, Fleisch B, Prinsloo C & Taylor S 2019. How to improve teaching practice? An experimental comparison of centralized training and in-classroom coaching. Journal of Human Resources:0618-9538R1. https://doi.org/10.3368/jhr.55.3.0618-9538R1        [ Links ]

Conn KM 2017. Identifying effective education interventions in Sub-Saharan Africa: A meta-analysis of impact evaluations. Review of Educational Research, 87(5):863-898. https://doi.org/10.3102%2F0034654317712025        [ Links ]

Deaton A 2010. Instruments, randomization, and learning about development. Journal of Economic Literature, 48(2):424-455. https://doi.org/10.1257/jel.48.2.424        [ Links ]

Deaton A & Cartwright N 2018. Understanding and misunderstanding randomized controlled trials. Social Science & Medicine, 210:2-21. https://doi.org/10.1016/j.socscimed.2017.12.005        [ Links ]

Department of Basic Education, Republic of South Africa 2011. Curriculum and Assessment Policy Statement Grades R-3: English Home Language. Pretoria: Author. Available at https://www.education.gov.za/Portals/0/CD/National%20Curriculum%20Statements%20and%20Vocational/CAPS%20English%20HL%20GRADES%20R-3%20FS.pdf?ver=2015-01-27-154201-167. Accessed 3 August 2019.         [ Links ]

Department of Education 2008a. Foundations for learning campaign. Government Gazette, 513(30880):1-23, March 14. Available at http://www.eccurriculum.co.za/FoundationPhase/FFL-Gazette30880-2008.pdf. Accessed 3 August 2019.         [ Links ]

Department of Education, Republic of South Africa 2008b. National reading strategy. Pretoria: Author. Available at https://www.education.gov.za/Portals/0/DoE%20Branches/GET/GET%20Schools/National_Reading.pdf?ver=2009-09-09-110716-507. Accessed 3 August 2019.         [ Links ]

Dixon K 2011. Literacy, power, and the school body: Learning in time and space. New York, NY: Routledge. https://doi.org/10.4324/9780203851487        [ Links ]

Evans DK & Popova A 2016. What really works to improve learning in developing countries? An analysis of divergent findings in systematic reviews. The World Bank Research Observer, 31:242-270. https://doi.org/10.1093/wbro/lkw004        [ Links ]

Fleisch B 2008. Primary education in crisis: Why South African schoolchildren underachieve in reading and mathematics. Cape Town, South Africa: Juta.         [ Links ]

Fleisch B 2018. The education triple cocktail: System-wide instructional reform in South Africa. Cape Town, South Africa: University of Cape Town Press.         [ Links ]

Fleisch B, Taylor S, Schöer V & Mabogoane T 2017. The value of large-scale randomised control trials in system-wide improvement: The case of the Reading Catch-Up Programme. South African Journal of Education, 37(1):Art. #1248, 13 pages. https://doi.org/10.15700/saje.v37n1a1248        [ Links ]

Foucault M 1977. Discipline and punish: The birth of the prison. Translated by A Sheridan. London, England: Penguin Books Ltd.         [ Links ]

Foucault M 2000. Different spaces. In D Faubion (ed). Essential works of Foucault 1954-1984: Ethics, subjectivity and truth (Vol. 2). London, England: Penguin Books.         [ Links ]

Glewwe P & Muralidharan K 2015. Improving school education outcomes in developing countries: Evidence, knowledge gaps, and policy implications (Research on Improving Systems of Education [RISE] Working Paper 15/001). Available at https://punarjitroyc.weebly.com/uploads/4/6/3/3/46337267/glewwe_and_muralidharan_2015.pdf. Accessed 16 July 2019.         [ Links ]

Haddaway NR, Land M & Macura B 2017. "A little learning is a dangerous thing": A call for better understanding of the term 'systematic review'. Environment International, 99:356-360. https://doi.org/10.1016/j.envint.2016.12.020        [ Links ]

Hoadley U 2012. What do we know about teaching and learning in South African primary schools? Education as Change, 16(2):187-202. https://doi.org/10.1080/16823206.2012.745725        [ Links ]

Hoadley U 2017. Learning to fly: Pedagogy in the Foundation Phase in the context of the CAPS reform. Journal of Education, 67:13-37. Available at http://joe.ukzn.ac.za/Libraries/No_67_2017/Learning_to_fly_Pedagogy_in_the_Foundation_Phase_in_the_context_of_the_CAPS_reform.sflb.ashx. Accessed 8 July 2019.         [ Links ]

Hoadley U 2018. Leading for literacy: A review of the research (Stellenbosch Working Paper No. WP19/2018). Stellenbosch, South Africa: Research on Socio-Economic Policy (RESEP), Department of Economics, Stellenbosch University. Available at https://resep.sun.ac.za/wp-content/uploads/2019/01/2018-12-12-Hoadley-Leadership-for-literacy-review-1.pdf. Accessed 16 July 2019.         [ Links ]

Howie S, Combrinck C, Tshele M, Roux K, McLeod Palane N & Mokoena G 2017. PIRLS 2016. Progress in International Reading Literacy Study 2016. Grade 5 benchmarking participation: South African children's reading literacy achievement. Pretoria, South Africa: Centre for Evaluation and Assessment (CEA), Faculty of Education, University of Pretoria. Available at https://www.up.ac.za/media/shared/164/ZP_Files/grade_5_pirls_2016_final_country_report.zp147996.pdf. Accessed 16 July 2019.         [ Links ]

Masino S & Niño-Zarazúa M 2016. What works to improve the quality of student learning in developing countries? International Journal of Educational Development, 48:53-65. https://doi.org/10.1016/j.ijedudev.2015.11.012        [ Links ]

McEwan PJ 2015. Improving learning in primary schools of developing countries: A meta-analysis of randomized experiments. Review of Educational Research, 85(3):353-394. https://doi.org/10.3102%2F0034654314553127        [ Links ]

McMillan JH & Schumacher S 2006. Research in education: Evidence-based inquiry (6th ed). Boston, MA: Allyn and Bacon.         [ Links ]

National Education Evaluation and Development Unit (NEEDU) 2014. NEEDU National Report 2013: Teaching and learning in rural primary schools. Pretoria, South Africa: Author. Available at http://www.purpletod.co.za/docs/Unpublished%20%20Controversial%20Report%202014.pdf. Accessed 22 July 2019.         [ Links ]

Pretorius EJ 2018. Reading in the African languages: An annotated bibliography 2004-2017. Pretoria, South Africa: University of South Africa.         [ Links ]

Rabinow P (ed.) 1991. The Foucault reader. London, England: Penguin Books.         [ Links ]

Reeves C 2010. Appendix A: Analysis of survey data (A report for the Grade 3 Improvement Project). Stellenbosch, South Africa: University of Stellenbosch.         [ Links ]

Reeves C 2017. The Early Grade Reading Study (EGRS): In depth case studies of home language literacy practices in four Grade 2 classrooms in treatment 1 and 2 schools (October 2016-January 2017). Johannesburg, South Africa: EGRS.         [ Links ]

Reeves C, Heugh K, Prinsloo CH, Macdonald C, Netshitangani T, Alidou H, Diedericks G & Herbst D 2008. Evaluation of literacy teaching in primary schools of Limpopo Province. Pretoria, South Africa: Human Sciences Research Council. Available at https://www.researchgate.net/profile/Kathleen_Heugh/publication/264520788_Evaluation_of_literacy_teaching_in_the_primary_schools_of_Limpopo_Province/links/58230b1608ae61258e3c704e/Evaluation-of-literacy-teaching-in-the-primary-schools-of-Limpopo-Province.pdf?origin=publication_detail. Accessed 21 July 2019.         [ Links ]

Republic of South Africa 2012. Gauteng Department of Education launches plan to improve language and maths skills for grades 4-7. Available at https://www.gov.za/gauteng-department-education-launches-plan-improve-language-and-maths-skills-grades-4-7. Accessed 29 July 2019.         [ Links ]

Snilstveit B, Stevenson J, Menon R, Phillips D, Gallagher E, Geleen M, Jobse H, Schmidt T & Jimenez E 2016. The impact of education programmes on learning and school participation in low-and middle-income countries: A systematic review summary report, 3ie Systematic Review Summary 7. London, England: International Initiative for Impact Evaluation (3ie). Available at http://repositorio.minedu.gob.pe/bitstream/handle/123456789/4892/The%20impact%20of%20educatio n%20programmes%20on%20learning%20and%20school%20participation%20in%20low-%20and%20middle-income%20countries.pdf?sequence=1&isAllowed=y. Accessed 6 July 2019.         [ Links ]

Spaull N & Hoadley U 2017. Getting reading right: Building firm foundations. In L Jamieson, L Berry & L Lake (eds). South African Child Gauge 2017. Cape Town, South Africa: Children's Institute, University of Cape Town. Available at http://www.ci.uct.ac.za/sites/default/files/image_tool/images/367/Child_Gauge/South_African_Child_Gauge_2017/Child_Gauge_2017_lowres.pdf. Accessed 21 July 2019.         [ Links ]

Taylor N 2009. The state of South African schools Part 1: Time and the regulation of consciousness. Journal of Education, 46:9-32. Available at http://joe.ukzn.ac.za/Libraries/No_46_June_2009/Completeissue_No_46.sflb.ashx#page=13. Accessed 6 July 2019.         [ Links ]

Taylor N, Draper K, Muller J & Sithole S 2013. National Education Evaluation and Development Unit (NEEDU) National Report 2012: The state of literacy teaching and learning in the foundation phase. Pretoria, South Africa: NEEDU. Available at http://www.shineliteracy.org.za/wp-content/uploads/2015/11/NEEDU-Report.pdf. Accessed 3 August 2019.         [ Links ]