RESEARCH ARTICLE

 

Evaluating innovation in transdisciplinary sustainability education: TRANSECTS' international learning labs

 

 

Eureta Rosenberg^I; Jessica Cockburn^II; Maureen Reed^III; Wendy James^IV; Jana Gengelbgch^V; Heike Walk^VI

^IEnvironmental Learning Research Centre, Rhodes University, Makhanda, South Africa
^IIDepartment of Environmental Science, Rhodes University, Makhanda, South Africa
^IIISchool of Environment and Sustainability, University of Saskatchewan, Saskatoon, Canada
^IVGwenna Moss Centre for Teaching and Learning, University of Saskatchewan, Saskatoon, Canada
^VBiosphere Reserves Institute, Eberswalde University for Sustainable Development, Eberswalde, Germany
^VIFaculty of Sustainable Business, Eberswalde University for Sustainable Development, Eberswalde, Germany

Correspondence

 

 

---

ABSTRACT

Evaluative research can advance sustainability education through the learning it can enable, at micro and systems levels. This proposition is explored by examining evaluation practice in a 6-year international programme entitled Transdisciplinary Education Collaboration for Transformations in Sustainability involving universities and biosphere reserves/regions in Germany, South Africa and Canada. A Transdisciplinary International Learning Lab (TILL) was evaluated using a theory-based evaluation approach and interviews, focus groups and questionnaires that yielded qualitative data. Through meta-reflection, we concluded that our TILL had elements of a Field School, rather than a Learning Lab, and that our curriculum required more explicit deliberation among programme developers and implementers towards a deeper and shared understanding of pedagogical assumptions and more congruent practice of transdisciplinary and transformative sustainability education. The reflective, theory-based approach enabled learning from evaluation and was captured in a shared refinement of the theory of change, which makes it explicit that learning from pedagogical innovations is not only for students but also for academics. The paper is an invitation to other innovators in sustainability science, education and evaluation in higher education, to share related findings.
SIGNIFICANCE:
Through evaluative research, educators gained insight into how transformative sustainability education and transdisciplinarity play out in practice, and how theory-based evaluation can inform more transformative programme design. As higher education practitioners collaborating across continents and disciplines for systemic change, we noted that transformative concepts do not immediately translate into transformative practices, unless we critically and collectively reflect on practice and outcomes. Such (meta) reflection requires data and purposefully designed evaluation frameworks-in-use. This idea is not new, but its manifestation in practice was illuminative and could also be of scholarly interest to other curriculum and evaluation designers.

Keywords: evaluation for learning, theory of change, transdisciplinarity, transformative sustainability education, higher education curriculum innovation

---

 

 

Introduction

TRANSECTS^1,2 is a multi-year, international programme, entitled Transdisciplinary Education Collaboration for Transformations in Sustainability, at the intersection between universities and UNESCO biosphere reserves. In sustainability education, there is a quest for innovative curricula that engage participants in learning not only how to analyse complex sustainability challenges but also to work with others to seek solutions.^3,4 It is for this reason that TRANSECTS offers Transdisciplinary International Learning Laboratories (TILLs) on three continents, involving students, mentors and practitioners from diverse disciplinary backgrounds.

The TRANSECTS TILLs are held in biosphere reserves (regions in Canada; hereafter BRs), these being characterised by UNESCO as 'sites of excellence' for sustainability⁵, as governance, practice and learning spaces in complex social-ecological landscapes.

TRANSECTS invites graduate students to join BR managers in exploring issues experienced in these landscapes, and consider solutions, with the aim of developing transdisciplinary competencies for sustainability practices.

The TILLs themselves, though interesting as curriculum innovation in sustainability education, are not the main focus of this paper; rather, we reflect here on the use of the framework that the authors co-designed to evaluate the TRANSECTS programme⁶, including the TILLs. Analysing the use of the evaluation framework to deepen innovative practices is interesting - and a research paper rather than simply an account of practice - because of the manner in which theoretical concepts of sustainability science, transformative higher education^7,8 and transdisciplinarity⁹ (TD) are encoded in the evaluation framework.

Furthermore, these concepts have, in the first two years of implementation, been informed and deepened by the application of the evaluation framework.

 

Context and literature

The need for pedagogical innovation in higher education

The need for higher education innovation in response to sustainability challenges is explored more fully elsewhere^3,4,7,8, but one consideration is pertinent here: that universities' responses to sustainability challenges must include pedagogical innovation³. The need for reorienting pedagogical practices is repeatedly emphasised in the quest for impactful learning outcomes for both individuals and society.⁷ If higher education is to catalyse and enable new ways of thinking, valuing and doing (i.e. to be transformative), it needs to provide learners with opportunities to critically reflect on existing frames of reference and beliefs and transform them into new ways of understanding and problem-solving, inter alia through a reframing of issues.^10,11 Education becomes transformative when it seeks - contrary to more instrumental approaches - to encourage participants to critique status-quo values and norms and to empower them to become change agents in complex systems^10,11, applying sustainability principles and ethics to address unsustainable practices⁷.

Responding to sustainability concerns requires multiple actors to collaborate.³ In complex sustainability contexts, the role-players are many and have diverse professional and cultural backgrounds, holding often conflicting interests. Educators have thus been proposing concepts like agency¹², action competence¹³, interpersonal and sustainability competencies¹⁴, intercultural competencies^14,15, technical and transformational leadership skills¹⁶, relational and transformational³ competence and reflexive competence¹⁷. Various curriculum and pedagogical innovations that encourage 'active learning'¹⁸ have been proposed, including project-based learning¹⁹, multi-step social learning processes²⁰, more generally creating transformative interdisciplinary and intercultural learning environments²¹, and the Learning Lab, the pedagogical innovation of choice for TRANSECTS. A Learning Lab (similar to Challenge Lab or Living Lab) is an educational opportunity created for students to engage with a sustainability challenge outside the academy, which is usually multi-faceted, requiring analysis from different disciplinary and non-disciplinary perspectives.²² In the Learning Lab, the problem is probed through research and stakeholder engagement, and solutions are developed and/or explored, and even tried out to start a further cycle of reflection and development.²²

The TRANSECTS programme

TRANSECTS was initiated by collaborating universities in Canada, South Africa and Germany, with the lead partner and main funder in Canada. Implementation activities commenced in 2022. These include, among others, new short courses, TILLs and Programme Institutes. In the latter, partners (academics, practitioners and students, from universities, BRs and elsewhere) come together to network, share, reflect, learn and plan.

The team conceptualising TRANSECTS (which includes the authors) produced an evaluation framework to track, reflect and report on all programme processes, outcomes and impacts over its envisaged 6-year lifespan.⁶ As TRANSECTS is about exploring innovation and transformations in sustainability education, we aimed to design an evaluation framework that aligned with the transformative intent of the programme and to optimise ongoing learning, as explained below.

Evaluation approaches

When resources are invested in a programme of interventions, evaluation is essential - not just at the end, to satisfy funders, but also along the way, to respond to emerging issues, to improve the programme and its chances of achieving desirable outcomes.

Furthermore, as we show in this paper, evaluation can support reflective practice and learning, among programme participants and potentially across a field as a whole.²³

Evaluation theory has evolved in tandem with broader research paradigm debates.²³ Over time, there have been various responses to the observation that social change is complex, non-linear and seldom easy to capture with pre-test, post-test measurements.²⁴ Much has been written about the limitations and negative consequences of imposing an 'experimental versus control group' evaluation design onto non-linear social interventions in complex systems.^24-26 Alternative approaches have been proposed to evaluate programme processes and development^24,26, values and narratives²⁷, principles²⁸ or identifying the underlying mechanisms that give rise to change²³.

Associated with the latter approach is theory-based evaluation.²⁹ An early proponent was Weiss³⁰, who proposed that in order to evaluate a programme of interventions, it is necessary for programme designers to articulate their programme theory, thus surfacing their assumptions of how change is likely to come about (theory of change (ToC)) and their theory of action, explaining why the intervention actions might effect that change. The goal is to evaluate the programme according to this explicit theory, in such a way that the evaluation findings indicate not only whether a desired change has taken place but also why this change happened, or not.²³ Such insights furthermore create an opportunity to interrogate the programme theory itself, and inform potential scaling.²³

All programmatic interventions are typically based on a theory of some kind, and most evaluations proceed from a ToC. These theories are, however, seldom explicit.^23,25,30 For example, the commonly used 'logical framework' embodies a programme theory or logic: If these activities are undertaken with these inputs, then these outcomes will eventually lead to this desired impact. How X is going to lead to Y is seldom explained.

Thus, the recommendations^24,26,30 are to start an evaluation with the articulation of an explicit ToC from which indicators are derived to guide what data should be collected and how it should be analysed. This 'theory' should be open to review, with evaluation creating a feedback loop from which programme designers and implementers can not only make implementation adjustments but also re-think their ToC. Where necessary, implementers can revise the ToC and associated indicators, accordingly.^25,29

Evaluation framework and tools for TRANSECTS

For the grant application³¹, TRANSECTS' programme designers produced a standard tabular log-frame about the relationship between programme inputs, outcomes and impacts (Figure 1). In addition, we produced a non-linear graphic version (Figure 2) that mapped the three change domains that were of interest to us: how institutions support transdisciplinarity, participants' learning and engagement practices in the BR landscapes.

 

 

 

 

Contrary to the logic presented in Figure 1, we did not assume that change will only take place in a predictable and linear way; therefore, Figure 2 has three concentric circles, with higher education innovations in the centre. While not linear, the general direction of change was nonetheless implied as starting from the academy, rippling out through to the learners, and then to the field of research and engagement practices in BRs, represented by the broadest sphere on the outside of the graphic. Unlike in Figure 1, Figure 2 is explicit that 'learners' include students as well as practitioners and academics.

A cross-section of TRANSECTS partners, academics and practitioners provided support for the ToC representation. Implementers also agreed that from time to time, it should be reviewed and the selected evaluation indicators, instruments and processes adjusted if necessary. This is standard practice, at least according to theoretical descriptions of theory-based evaluations.²⁶ An evaluation process based on an explicit, non-linear and evolving ToC is, however, a departure from the norm in programme evaluation.²⁶ The MEL team thus undertook to monitor the evaluation framework itself, as it unfolded, using the ToC to guide data collection, and periodic meta-reflections on emerging findings.

 

Methodology on which this study is based

The research methodology for this paper is underpinned by the mentioned theory-based evaluation approaches^25,29,30, drawing primarily on qualitative data. Referring to the ToC diagram (Figure 2), planned programme activities were aligned with the three domains of change, and associated evaluation questions created, along with instruments to gather data about those activities. The broad evaluation questions were:

1. Were activities executed as planned and according to TRANSECTS' principles? How, or why not?

2. Were desired learning outcomes, derived from literature in the sustainability sciences and education^3,10 achieved? How, or why not?

3. What other outcomes emerged, relevant to TRANSECTS' transformative intent?

The first two TILLs were offered in Germany, starting with a 2-week pilot in 2022. Both TILLs were evaluated, but the 6-week 2023 TILL was evaluated more comprehensively, by both internal and external evaluators. Hence, in this paper, we focus on results from the 2023 TILL.

The evaluation processes consisted of questionnaires, focus groups and individual interviews, which were conducted either in person or online, recorded and transcribed. Both internal and external evaluators gathered extensive confidential qualitative data on the experiences and insights from the various TILL participant groups. Ethical approval was provided by the University of Saskatchewan.

In addition to on-site data collection in Germany, some students who participated in either of the TILLs shared feedback with mentors and programme staff during a Programme Institute in South Africa. This meeting was held some months after the 2023 TILL. Members of the programme design team who were present then engaged in informal meta-reflections on this feedback and other data that had been collected and analysed. During reflections, we applied inter-subjectivity as a means to bring objectivity to our process³², that is, we challenged each other's interpretations, and when found to be sound, built on them. This included a subsequent online discussion of findings with TILL hosts. The full set of findings as well as methods and instruments are detailed in the primary evaluation report, which is available upon request.³³

In Tables 1-3, we share only selected findings followed by meta-reflections. While all three evaluation questions apply to this paper, the main focus for this paper is question 3: What other outcomes emerged and seemed relevant to TRANSECTS' transformational intent?

 

 

 

 

 

 

Findings

Overview of the 2023 TILL

The 2023 TILL took place in mid-winter in a BR in a rural region of Germany. Its focus was on different forest ownership types, with different management objectives (optimum forestry yield vs. biodiversity, for example). The 17 participating students were graduates, most with master's degrees, representing eight nationalities, and a range of universities, disciplinary and cultural backgrounds. They were selected on the basis of their academic and leadership abilities and their motivation to learn more about transdisciplinary sustainability practices. They stayed in shared accommodation and, for part of the TILL, had to plan and shop for shared meals. The hosts arranged outdoor excursions and meetings where forest scientists shared their expertise.

Configuring the TILL involved many more role-players than the BR hosts. Six weeks before the in situ TILL, students attended a Foundational Course, a series of online orientation sessions and seminars. These included academic presentations on transdisciplinarity, and on Constellation Analysis³⁴, a transdisciplinary method for analysing sustainability challenges and identifying entry points towards solutions. Besides local instructors, four international mentors were appointed for the first 2 weeks of the TILL, each with four or five students.

Despite concerted efforts to involve them, some mentors were not able to attend all the orientation sessions, and the BR managers found it particularly difficult to attend, possibly due to connectivity and work load. Another key development was that COVID struck during the first week of the TILL, leading to two mentors leaving the site of the Learning Lab early and offering to continue to mentor online; some other mentors and students experienced this as a notable gap in support.

Selected evaluation data

Programme developers' meta-reflections at the Programme Institute

During the 2023 Programme Institute in South Africa, the authors considered the above data. We concluded that the TILL was a highly rated and worthwhile learning experience for students. However, it did not provide as innovative a transdisciplinary learning experience as we had intended. As a collective, we may have conceptualised and approached the TILL more as a Field School, than as a Learning Lab. In sharing with each other what we understood to be the differences between these two curriculum offerings, we found this conclusion to be a sound and powerful explanation for what transpired, that resonated with all of us, and with TILL mentors, when we later engaged them.

In the discussion below, we reflect on why this conclusion is warranted and pertinent for transdisciplinary and transformative approaches to sustainability education - in relation to TRANSECTS (micro-level) but also to wider theory and system building. We also explain the relationship between the evaluation processes and our learning.

 

Discussion and conclusions: Reflection and elaboration

Our meta-reflection revealed that transdisciplinary curriculum development for transdisciplinary Learning Labs across different contexts is:

• more complex than we had anticipated, particularly in relation to intercultural and relational competencies;

• requires concerted communication between curriculum developers and between developers and implementers (such as Learning Lab hosts and mentors); and

• requires shared and ongoing clarification of transdisciplinary and pedagogical approaches.

What is a Field School, and what is a Learning Lab? What are the differences between them, and why did we think that we have in some ways approached the TILL more as one, rather than the intended other?

Drawing on the literature on Learning Labs and Challenge Labs^22,35,36 and our experience as higher education practitioners in the Geographical and Sustainability Sciences, Higher Education Scholarship of Teaching and Learning, and Environmental Education, of Field Schools (the term used in Canada) or field trips (the term used in South Africa), we identified key differences related to purpose, process and end-points (Figure 3).

 

 

We realised that the purpose of a Learning Lab, to collaboratively work towards a solution for a problem that has also been jointly identified and explored, and share that solution with each other and possibly a broader range of community partners, should have been made clearer to BR hosts, mentors and students. Throughout the TILL, learning should have been mediated with references back to the Foundational Course and the theoretical discussions on transdisciplinarity. The gap left by the early departure of two mentors signalled just how important learning support was, not only during the first 2 weeks, but throughout -something that was not fully anticipated when the Learning Labs were conceptualised.

Similarly, the value of the diversity of the contributions of students with backgrounds in Politics, Economics, Education, Governance, Forestry, Agricultural Sciences and Ecology, should have been more apparent to all. Students were not attending to simply collect field data as free research assistants. The relevance of inputs from a top ecologist in relation to the sustainability issue under investigation, should have been discussed before and during the TILL, and not assumed.

Learning Labs (and a transdisciplinary process like Constellation Analysis³⁴) start with the identification or elaboration of a sustainability issue through community partner engagement because the process of formulating the central problem and associated research question(s) with community partners (in this case, BR practitioners, other forestry owners and neighbours) is paramount and not simply a precursor to the research. Thus, Learning Labs require ample time and opportunity for community partner engagement.

Learning Lab participants should agree that the key question(s) to research might not be clear at the start, or at least somewhat fluid; however, there should also be an agreed-upon process for concluding what would be the most relevant question to research. This is a fundamental aspect of transdisciplinary work - not just a preliminary step to quickly get out of the way, or to be handed down before the start of the Learning Lab. In the 2023 TILL students, mentors and BR practitioners were either unclear as to what the key research question was, or unclear about how it and when was to be derived, and by whom.

In some ways, we approached the TILL like a Field School where the focus is usually on collecting bio-physical data, for example, by not fully anticipating the requirements for engagement with community partners. An example is that the majority of BR community partners spoke only German, which only a few students could speak, leaving the majority of students unable to directly engage with community partners.

We also realised that students needed to hear explicitly that challenges experienced around living together (e.g. deciding between meat or vegetarian meals) were part of the intended learning outcomes. Relational³ or interpersonal¹⁴ competencies are prerequisites for solving sustainability challenges with others²⁰. TRANSECTS proposed to develop such competencies by selecting graduate students from different nationalities, disciplinary and cultural backgrounds to participate in the TILLs. We did not anticipate just how steep this learning curve would be for some TILL participants who needed ongoing and expert facilitated learning mediation in this regard.

Mentors were uncertain about whether or how to address the challenges that emerged. On a field trip, social conflict and taking time to resolve it is simply a by-product of the primary focus on co-habitating in a remote area in order to (learn how to) collect separate pieces of bio-physical information. In the case of a Learning Lab, however, 'finding' each other (across disciplinary and cultural boundaries) is a key success factor for working together to address a complex problem. Resolving the problem requires participants not only to communicate and work together but also to appreciate and use diverse contributions. Mentors felt ill-prepared to facilitate conflict resolution; it did not feature in the 'job description' and requires skills they either felt they lacked or were not primed to draw on. While mentors and students alike reported that students eventually found peace and even joy in their differences, we collectively missed the opportunity to make the importance of relational competencies³ explicit and to provide scaffolding to strengthen learning.

In a Field School, mentors have particular roles: providing instruction about data collection, assisting with technical aspects, perhaps socialising after-hours with students to help induct them in the field and assessment of tasks completed. Roles were less clear in this Learning Lab. What were mentors' role in relation to the sett(l)ing of the research question, engaging partners and addressing interpersonal conflicts? The evaluation suggests that there was a need for more explicit learning mediation along the way - that the TILL could not be left to unfold without regular feedback to the students, with reference to the intended learning outcomes, and a recommended suite of transdisciplinary engagement methods from which to choose.

At this point, it should be noted that the TILL supported key learning outcomes and had many positive features and outcomes for students, mentors and community partners. For example, the students' final assignments were of good quality and well received by academics and BR practitioners alike; several students asked to attend the next TILLs as mentors; mentors offered to participate in future TILLs; and new relationships between BRs and universities were forged as a result of the shared endeavour. Given such successes, it would in fact have been easy for us to overlook the fact that the curriculum offering was in some ways simply a more ambitious version of what we would have offered in the past (a Field School) rather than the fundamentally different intervention (a transdisciplinary Learning Lab) we had theorised it to be.

Why, despite significant efforts by all parties, did we conclude that the curriculum design was not as innovative or transformative as we had sought? On reflection, we realised that our approach to the TILL was predominantly resource-based (asking what resources we have and how best to use them) with less attention to designing curriculum to achieve learning outcomes. We also noted that while students and mentors had been briefed about the ways in which the TILL was to be transdisciplinary, this phrase had different meanings and applications. We did not clarify what forms of transdisciplinarity we were seeking to promote. Ironically, we did not consider just what it requires for us as the institutional partners to collaborate as a multidisciplinary team spread across three continents. Online meetings were not ideal for developing a shared conceptualisation of the TILL. Even team members who regularly interact with each other and had, on the surface, shared understandings of the nature of the innovation, also approached it quite differently. Disciplinary differences might have had a role in this, and thus, it is an instructive example of the situations that transdisciplinary practitioners (including our graduates) find themselves in, in the complex social-ecological landscapes of practice. We can only conclude, retroductively³², that transcending years of excellent disciplinary training were not going to happen in a single event - unless one applies these ideas in practice, and reflects on them, as we attempt to do here, on an ongoing basis.

In response to the evaluation findings, the TRANSECTS programme designers subsequently took several steps to strengthen the planning of future TILLs, including 'backwards' curriculum planning (from desired outcomes to required practices), adjustments to the timing and content of the Foundational Course and increased mentoring support. Changes included more emphasis on communication and inter-cultural competencies, assigning and clarifying pedagogical roles for TILL mentors and hosts, carefully considering the ways in which the proposed TILL focus and research question(s) lend themselves to transdisciplinary and engaged research and innovation.

The findings also informed an adjustment to TRANSECTS' ToC (Figure 4). As noted earlier, theory-based evaluations^23,25,29,30 start with surfacing a shared programme theory from which to derive indicators that guide what data should be collected and what should be evaluated²⁵. This ToC must be open to review, and evaluative practices should create a feedback loop from which implementers not only refine implementation but also, where necessary, re-think their ToC and revise it, and the associated indicators. In this case, we have added evaluation of the process of TILL development to our MEL framework in order to track the extent to which we are designing for transdisciplinarity. Additionally, we added recursive arrows to make it explicit that change does not only take place among learners in the second domain of change; change also has to take place in the central domain where higher education institutions need to change the way in which we conceptualise, design and deliver our curriculum offerings, based on reflective practice informed by feedback from the field. We reiterated that, contrary to the original logical framework (Figure 1), the network of learners consists not only of students but also BR practitioners and academics. We had initially indicated this when conceptualising TRANSECTS (Figure 2) but are now clearer on how this learning can happen. This feedback loop and learning would not have been possible, without evaluation, specifically by applying the theory-based evaluation process we followed.

 

 

The use of a non-linear ToC^25,29 encouraged us to be reflective practitioners who look across the data of practitioners, mentors and graduate students, given that our ToC presents the relationships among these domains as important, if as yet under-theorised. The ToC afforded deeper thinking than if we had simply counted numbers of participants, or checked whether learning outcomes had been achieved. A simple, but significant, flow of the key elements for students, practitioners and mentors made it easier to engage in deep conversations around what the evidence indicated, without the limitation of a narrower focus on specific outputs or structures of a standard logic model.

The results of evaluation-in-use include deeper iterations of the programme theory, notably the distinctions between a more standard Field School and what a transdisciplinary and intercultural Learning Lab was intended to do. Working reflexively with a ToC proved even more significant given the number of people involved in the evaluation, and communicating across continents, time zones and disciplines, and complex TILL experiences.

Ultimately, some of the best evidence of strong evaluation is the capacity to use it in situ²⁵ to make changes iteratively. Thus, we confirm the value of theory-based evaluation and working iteratively with a programme theory. As Oberlack et al.³⁷ argued:

ToCs trigger debate among the stakeholders and evaluators of an initiative regarding the hypothesized and observed effects of actions as well as regarding underlying assumptions about how change happens. Therefore, they can strengthen the effectiveness of research, practice, and education in sustainability science.

Our study shows that a ToC approach to evaluation can catalyse not only a more rigorous evaluation focused on the change process, but it can also frame and catalyse the kinds of relational and deliberative processes needed to collaboratively make sense of evaluation data and insights, to make improvements to an ongoing programme, and perhaps also to contribute to theory development in an emerging field.

 

Coda - The role of evaluation in developing transformative higher education curricula

When one of us shared some of these findings at a conference that invited delegates to explore 'bridging theory and practice', the moderator congratulated TRANSECTS on being prepared to share and learn from our 'mistake'. The term 'mistake' was surprising and served as a reminder that reflective practices - learning from reflecting on doing -are not common practice in higher education. The drive for sustainability transformations should surely be characterised by experimentation where the term 'mistake' might not be the best way to describe an exploration of innovation attempts that require refinement; sustainability challenges necessitate critically evaluative ways of working. The paper provides one example of an evaluation framework and process that yielded both data and insights, and thus also the evidence that evaluation, if approached as a form of theory-driven and data-informed feedback, can assist higher education practitioners to deepen insights into practice.

The paper illuminates how concepts of transformative sustainability education play out in practice, how challenging it is to develop a common strategy for transdisciplinary work and how evaluation can inform more transformative programme design, implementation and learning for all participants. As higher education practitioners collaborating across continents and disciplines for transformations in sustainability education and practice, we learned that transformative concepts do not automatically turn into transformative practices and require collective and critical reflection-in-practice. Such (meta) reflection requires congruent evaluation frameworks-in-use. While this idea is not new, its manifestation in practice was illuminative, and we have already seen that other curriculum and evaluation designers also find it insightful.

 

Acknowledgements

We thank the external evaluator, Tillmann Konrad of Konrad Applied Science Consulting; the TILL participants (students, mentors, biosphere reserve practitioners) and the TRANSECTS Programme Managers and Coordinators, MEL team, Teaching and Learning Team and Steering Committee members.

 

Funding

We acknowledge the German Academic Exchange Service (DAAD), Germany; the National Research Foundation Community of Practice, South Africa; and the Social Sciences and Humanities Research Council of Canada (SSHRC), for financial support.

 

Data availability

The data supporting the results of this study are available upon request to the corresponding author, provided our research ethics protocols are adhered to.

 

Declarations

We have no competing interests to declare. We did not make use of artificial intelligence for data collection, analysis or writing. Ethical approval was provided by the University of Saskatchewan (Application ID: 3808).

 

Authors' contributions

E.R.: Conceptualisation, methodology, research tools, data collection, analysis, writing - the initial draft, writing - revisions. J.C.: Conceptualisation, methodology, research tools, data collection, analysis, data curation, validation, writing - revisions. M.R.: Analysis, validation, writing - revisions, project leadership, funding acquisition, acquiring ethics clearance certificate. W.J.: Conceptualisation, methodology, research tools, data collection, analysis, data curation, validation, writing -revisions. J.G.: Data collection, analysis, validation, project management, funding acquisition. H.W.: Conceptualisation, data collection, validation, writing - revisions. All authors read and approved the final manuscript.

 

References

1. TRANSECTS. Programme design [webpage on the Internet]. No date [cited 2024 Feb 19]. Available from: https://sens.usask.ca/transects/about/structure/design.php https://sens.usask.ca/transects/        [ Links ]

2. TRANSECTS. Operating principles [webpage on the Internet]. No date [cited 2024 Feb 19]. Available from: https://sens.usask.ca/transects/about/approach/operating-principles.php        [ Links ]

3. Rosenberg E, Lotz-Sisitka H, Ramsarup P. The green economy learning assessment South Africa. Higher education, skills and work-based learning. High Educ Ski Work-based Learn. 2018;8(3):243-258. https://doi.org/10.1108/HESWBL-03-2018-0041        [ Links ]

4. Holmén J, Adawi T, Holmberg J. Student-led sustainability transformations: Employing realist evaluation to open the black box of learning in a Challenge Lab curriculum. Int J Sust Higher Ed. 2021;22(8):1-24. https://doi.org/10.1108/IJSHE-06-2020-0230        [ Links ]

5. UNESCO. Man and the Biosphere Programme (MAB) [webpage on the Internet]. No date [cited 2024 Feb 19]. Available from: https://www.unesco.org/en/mab        [ Links ]

6. Rosenberg E, Cockburn J, James W, Walker R, Egunyu F, Reed M. TRANSECTS Monitoring, Evaluation and Learning (MEL) framework and implementation plan: Working draft. Makhanda: Rhodes University; 2022.         [ Links ]

7. Rodriguez Aboytes JG, Barth M. Transformative learning in the field of sustainability: A systematic literature review (1999-2019). Int J Sust Higher Ed. 2020;21(5):993-1013. https://doi.org/10.1108/IJSHE-05-2019-0168        [ Links ]

8. Barth M, Michelsen G. Learning for change: An educational contribution to sustainability science. Sustain Sci. 2013;8(1):103-119. https://doi.org/10.1007/s11625-012-0181-5        [ Links ]

9. Reed M, Robson JP, Campos Rivera M, Chapela F, Davidson-Hunt I, Friedrichsen P et al. Guiding principles for transdisciplinary sustainability research and practice. People Nat. 2023;5:1094-1109. https://doi.org/10.1002/pan3.10496        [ Links ]

10. Cranton P, Taylor EW. Transformative learning theory: Seeking a more unified theory. In: Taylor EW, Cranton P, editors. The handbook of transformative learning: Theory, research, and practice. San Francisco, CA: Jossey-Bass/ Wiley; 2012. p. 3-20.         [ Links ]

11. Mezirow J. Transformative learning: Theory to practice. New Direct Adult Contin Educ. 1997;74:5-12. https://doi.org/10.1002/ace.7401        [ Links ]

12. Jalasi EM. An integrated analytical framework for analysing expansive learning in improved cook stove practice. Learn Cult Soc Inter. 2020;26, Art. #100414. https://doi.org/10.1016/j.lcsi.2020.100414        [ Links ]

13. Mogensen F, Schnack K. The action competence approach and the 'new' discourses of education for sustainable development, competence and quality criteria. Environ Educ Res. 2010;16(1):59-74. https://doi.org/10.1080/13504620903504032        [ Links ]

14. Wiek A, Withycombe L, Redman CL. Key competencies in sustainability: a reference framework for academic program development. Sustain Sci. 2018;6(2)):203-218. https://doi.org/10.1007/s11625-011-0132-6        [ Links ]

15. Nikiforova M, Skvortsova I. Intercultural competence and intercultural communication in the context of education for sustainable development, E3S Web Conf. 2021;296, Art. #08026. https://doi.org/10.1051/e3sconf/202129608026        [ Links ]

16. Schârmer O. Ten propositions on transforming the current leadership development paradigm. World Bank round table on leadership for development impact. Washington, DC: World Bank Institute; 2009.         [ Links ]

17. Holden P Cockburn J, Shackleton S, Rosenberg E. Supporting and developing competencies for transdisciplinary postgraduate research: A PhD scholar perspective. In: Kremers KL, Liepins AS, York AM, editors. Developing change agents: Innovative practices for sustainability leadership. Minneapolis, MN: University of Minnesota; 2019. Available from: https://open.lib.umn.edu/changeagents/chapter/supporting-and-developing-competencies/        [ Links ]

18. O'Donoghue R. Environment and active learning in OBE: NEEP guidelines for facilitating and assessing active learning in OBE. Howick: ShareNet; 2001.         [ Links ]

19. Brundiers K, Wiek A. Do we teach what we preach? An international comparison of problem- and project-based learning courses in sustainability. Sustainability. 2013;5(4):1725-1746. https://doi.org/10.3390/su5041725        [ Links ]

20. Wals A, editor. Social learning: Towards a sustainable world. Wageningen: Wageningen Academic; 2007. https://doi.org/10.3920/978-90-8686-594-9        [ Links ]

21. van Dam-Mieras R, Lansu A, Rieckmann M, Michelsen G. Development of an interdisciplinary, intercultural master's program on sustainability: Learning from the richness of diversity. Innov High Educ. 2008;32(5):251-264. https://doi.org/10.1007/s10755-007-9055-7        [ Links ]

22. McCrory G, Holmén J, Schâpke N, Holmberg J. Sustainability-oriented labs in transitions: An empirically grounded typology. Environ Innov Soc Transit. 2022;43:99-117. https://doi.org/10.1016/j.eist.2022.03.004        [ Links ]

23. Pawson R, Tilley N. Realistic evaluation. London: Sage; 1997.         [ Links ]

24. Patton M. Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York: Guilford Press; 2010. Available from: http://www.guilford.com/cgi-bin/cartscript.cgi?page=pr/patton.htm&dir=re        [ Links ]

25. Funnell SC, Rogers PJ. Purposeful program theory: Effective use of theories of change and logic models. San Francisco, CA: Jossey-Bass/Wiley; 2011.         [ Links ]

26. Rosenberg E, Kotschy K. Monitoring and evaluation in a changing world: A southern African perspective on the skills needed for a new approach. Afr Eval J. 2020;8(1), a472. https://doi.org/10.4102/aej.v8i1.472        [ Links ]

27. Davies R, Dart J. The Most Significant Change (MSC) technique: A guide to its use [document on the Internet]. c2005 [cited 2024 Feb 19]. Available from: https://www.mande.co.uk/wp-content/uploads/2005/MSCGuide.pdf        [ Links ]

28. Patton MQ. Principles-focused evaluation. New York: Guilford Press; 2018.         [ Links ]

29. Brousselle A, Buregeya J. Theory-based evaluations: Framing the existence of a new theory in evaluation and the rise of the 5th generation. Evaluation. 2018;4:153-168. https://doi.org/10.1177/1356389018765487        [ Links ]

30. Weiss CH. Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. In: Connell J, Kubisch A, Schorr L, Weiss CH, editors. New approaches to evaluating community initiatives: Concepts, methods and contexts. New York; Aspen Institute; 1995. p. 65-92.         [ Links ]

31. USASK (University of Saskatchewan) School of Environment and Sustainability (SENS). Grant application to the Social Sciences and Humanities Research Council of Canada (SSHRC). Unpublished; 2021.         [ Links ]

32. Sayer A. Method in social science: A realist approach. London: Routledge; 2010. Available from: https://doi.org/10.4324/9780203850374        [ Links ]

33. Cockburn J, Rosenberg E, James W, Gengelbach J, Merry M, Konrad T, et al. TRANSECTS Germany Winter TILL (2022-2023) synthesis evaluation report. Makhanda: Rhodes University; 2023.         [ Links ]

34. Ohlhorst D, Schon S. Constellation analysis as a means of interdisciplinary innovation research - theory formation from the bottom up. Hist Soc Res. 2015;40(3):258-278. Available from: http://www.jstor.org/stable/24583155        [ Links ]

35. Larsson J, Holmberg J. Learning while creating value for sustainability transitions: The case of Challenge Lab at Chalmers University of Technology. J Clean Prod. 2018;172:4411-4420. https://doi.org/10.1016/j.jclepro.2017.03.072        [ Links ]

36. Knickel M, Caniglia G, Knickel K, Sumane S, Maye D, Arcuri S, et al. Lost in a haze or playing to partners' strengths? Learning to collaborate in three transdisciplinary European Living Labs. Futures. 2023;152:103-219. https://doi.org/10.1016/j.futures.2023.103219        [ Links ]

37. Oberlack C, Breu T, Giger M, Harari N, Herweg K, Mathez-Stiefel S, et al. Theories of change in sustainability science: Understanding how change happens. GAIA Ecol Perspect Sci Soc. 2019;28(2):106-111. https://doi.org/10.14512/gaia.28.2.8        [ Links ]

 

 

Correspondence:
Eureta Rosenberg
Email: E.Rosenberg@ru.ac.za

Received: 15 Feb. 2024
Revised: 04 July 2024
Accepted: 23 July 2024
Published: 26 Sep. 2024

 

 

Editors: Chrissie Boughey, Nkosinathi Madondo
Funding: German Academic Exchange Service (DAAD), National Research Foundation of South Africa, Social Sciences and Humanities Research Council of Canada

 

 

Supplementary Data

The supplementary data is available in pdf: [Supplementary data]