RESEARCH ARTICLE

 

Exploring the relationship between entry requirements and throughput rates for honours students

 

 

Mike Murray

School of Mathematics, Statistics and Computer Science, University of KwaZulu-Natal, Durban, South Africa

Correspondence

 

 

---

ABSTRACT

In order for a student to enrol in an honours programme at the University of KwaZulu-Natal (UKZN), a weighted average mark for their final year of undergraduate study must exceed a particular threshold value. Students are then ranked according to this weighted average mark, with entry into the honours programme offered on a top-down basis, within the constraints of teaching resources and space. A proposal has been made at UKZN to remove existing barriers for entry into an honours programme, i.e. to allow entry to any student who has completed a 3-year undergraduate degree with a major in that discipline. The impact of such a decision was investigated. By lowering the requirement for entry into an honours programme, one is expected to predict how a new cohort of students will perform. Apart from obviously having a lower weighted average mark for their final year of undergraduate study, these new students may also differ in other unobservable ways which need to be accounted for. In a regression modelling context, one is asked to predict outside the range of a collected data set. A Heckman selection model was used to account for a possible self-selection bias that may arise because the subpopulation for which a prediction is required (namely those new students who will now be able to enter an honours programme), may be significantly different from the population of UKZN undergraduate students who are currently permitted entry to an honours programme.
SIGNIFICANCE:
• A modelling technique that accounts for a possible sample selection bias was used to determine the impact of lowering the entry requirements into the honours programme at UKZN to allow entry to any student who has completed a 3-year undergraduate degree.

Keywords: Heckman model; sample selection bias; regression model

---

 

 

Introduction

In order to prepare students for entry into a job market that is rapidly evolving, 3-year undergraduate degrees that were once common in North America and parts of East Asia and South America have been replaced with 4-year undergraduate degrees. In Australia, the University of Sydney recently announced a radical reorganisation of their course offerings with an emphasis on the restructuring of their 3-year degrees into 4-year degrees. Noting that only a 'relatively small fraction' of students left their university after completing a 3-year degree, by embedding an honours degree within a 4-year undergraduate degree not only would the university be able to better prepare students for postgraduate study but they would also be able to substantially reduce the number of degrees that they have to offer. In South Africa, a similar debate is taking place at the University of KwaZulu-Natal (UKZN). In some disciplines, such as Engineering, a 4-year qualification has for many years been the norm. In the College of Management Sciences, students have the option to complete a 3-year undergraduate degree (which is then followed by an additional year of honours study) or a 4-year degree. When faced with this choice, students in that college seem to prefer enrolling for a 4-year degree.

This paper is concerned primarily with the identification of the impact of the decision to lower the requirement for entry into honours on the throughput rate within that programme. The positive (or negative) consequences of allowing these (possibly weaker) students to enrol for an honours degree need to be weighed against the 'massification' of undergraduate programmes that is taking place, in which an increasing number of undergraduate students are wanting to improve their qualifications by enrolling in an honours programme.

Given that there is a financial cost incurred in training these students, by relaxing the entry requirement one may simply be increasing the number of students who are being set up to fail. A White Paper outlining a National Plan for Higher Education¹ made mention of this problem, highlighting at an undergraduate level the need for a planned expansion of the system to be 'responsibly managed and balanced in terms of the demand for access, the need for redress and diversification, the human resource requirements...and the limits of affordability and sustainability'. The document further states¹:

While it is possible to achieve rapid enrolment growth without extra expenditure, the penalties for doing so are harsh. Experience...shows that expansion without new investment results in overcrowded facilities, low morale of academic staff, poor quality programmes, a fall in research output and quality, and...devalued products of higher education.

More recently² the debate around curriculum change has grown; the Council on Higher Education released a document in 2013 proposing an overhaul of the undergraduate curriculum in South Africa. However, the focus of this paper is on the consequences of streamlining the pipeline for entry into honours.

Undeniably, being able to increase the number and knowledge base of students in our higher education institutions has many benefits. Research into academic performance at an undergraduate level has traditionally identified prior performance as being an important predictor variable. However, more recent studies^3-9 have found that academic success also depends significantly on the perception of a student regarding their learning environment and how successful they have been in coping with difficult situations they have encountered at university^10-12. Economic, social and cultural factors also have an important role to play in determining academic performance at an undergraduate level. However, the socio-economic factors that affected one's performance at an undergraduate level (in the above studies^3-12) may not be as strongly pronounced for an honours degree. Nevertheless, including these factors as covariates in our prediction model will be important for the analysis that follows.

Currently, entry into an honours programme at UKZN is not automatic. A weighted average mark in the final year of undergraduate study must exceed a threshold value of 55% before a student is allowed to enrol for an honours degree. Will the lowering of this entry requirement to a weighted average mark of 50% or more have a serious impact on the throughput rate that will then be recorded by students entering a fourth year (honours) study at UKZN?

International research on this matter is usually restricted to cases in which a group of students for which the relaxation of entry requirements does not apply are compared with another group for which a relaxation of standards has been applied.^13-16 One can then look for a difference in performance between these two cohorts - controlling, where necessary, for background variables that may also impact on their performance in their honours year of study. In this study, however, it was not possible to observe an honours-based performance for students who, under the current criteria, do not qualify for entry into honours. Consequently, the focus in this paper is different: to determine how this new cohort of students would perform if they had been given the opportunity to enrol for an honours degree at UKZN. A Heckman model needs to be used to adjust for a possible self-selection bias that may arise because the outcome of interest (in our case a suitably chosen measure of performance in the honours programme) can be observed for only a subset of students who were previously eligible for entry into honours.

The Heckman model and its use in a sociological and economic setting has been well documented in the literature.^17-21 In an educational setting, however, its application has been restricted mainly to identifying what sort of causal effect a particular level of education has on a given socioeconomic response variable (such as earnings). In fact, a detailed review of the literature²² has found that only 14% of 386 articles discussing the problem of selection bias have done so in an educational context with a Heckman selection model then fitted to the collected data. The application of a Heckman model to our problem - for which 'entry into honours' replaces 'level of education' as the treatment variable and a weighted average mark in honours is used as a response variable -could well be novel, but is most certainly well supported by a number of other applications in the literature.

 

An analysis based on a weighted average mark in honours

This study focuses on a very specific cohort of students: those who completed an undergraduate degree at UKZN during the period 2011-2013. This cohort comprises 9398 students who obtained a weighted average mark in their final year of study that exceeded a threshold value of 50%. Under the prevailing set of rules allowing for a progression into honours, 3233 students from this sample were allowed to enrol for an honours degree because their weighted average mark exceeded a threshold value of 55%. An additional 2021 students from other universities were also allowed to enrol. Apart from noting very briefly that these other students generally did not perform as well in the honours programme as their counterparts who completed undergraduate studies at UKZN, the focus of this paper will rest entirely on comparing the observed performance (in honours) of the 3233 students who completed undergraduate studies at UKZN and were able to enter honours, with the performance that one would expect from a hypothetical cohort of 9398 UKZN graduates if the other 6165 students who achieved a weighted average mark in their final year of study of 50-55% joined the 3233 students currently in the honours programme.

To measure their performance in honours, each course was assigned an appropriate credit point value; an honours degree is considered to be completed when a certain number of credit points have been acquired. Using these credit point values as appropriate weights, a weighted average mark for all the compulsory courses was used as response variable Y_j. For example, if eight courses must be passed in order to complete an honours programme and each course has been given a credit weighting of 16 points, then

will be the observed weighted average mark in honours for that student. If a course had to be repeated then both marks together with their appropriate weights were included in the response variable calculation for that student. Permission to use this data set was given by the Teaching and Learning Office at UKZN.

The results in Table 1 suggest that students who completed an undergraduate degree at UKZN performed significantly better in the UKZN honours programme than those who completed undergraduate studies at another university (f=12.185). Perhaps a significant number of the students who transferred from another university were weaker students who were refused entry into an honours programme at that university. There may also be other reasons for such an anomaly. For example, the demographic breakdown shown in Table 2 indicates that the students from other universities were older than those who entered from a UKZN undergraduate programme.

 

 

 

 

Because the goal was to assess the impact of lowering the entry requirement on the throughput rate in the honours programme, the actual performance of the 3233 UKZN graduates enrolled in honours was compared with the expected performance of all 9398 graduates who would have been allowed to enrol if the entry requirement had been a weighted average mark of 50% or more.

Table 3 shows the results of regressing the weighted average mark in honours of each of the 3233 students against a set of appropriately chosen predictor variables. Ordinary least squares methods were used to produce the parameter estimates that appear in Table 3.

 

 

For the interpretation of Table 3, 'female' refers to a 0/1 indicator variable that is set equal to 1 for a female student. 'Black' refers to a 0/1 indicator variable that is set equal to 1 for a black student. Similarly, 'honours funding' refers to a 0/1 indicator variable that is set equal to 1 for a student who has managed to obtain some form of external funding for honours study. Because UKZN is composed of four colleges (College of Agriculture, Engineering and Science, College of Health Sciences, College of Humanities and College of Law and Management Studies), indicator variables for the first three colleges were included as covariates in Table 3 with an effect for the College of Law and Management Studies forming part of the intercept term in the above model structure. These covariates were chosen based on previous research^23-26 at UKZN that has identified them as being important predictor variables for determining performance at UKZN.

As one would expect, the weighted average mark that a UKZN undergraduate student obtains in their final year of undergraduate study correlates strongly and positively with the weighted average mark that they record for their honours year of study. Older students tend to not perform as well as their younger counterparts and female students perform better than their male counterparts. Having some form of honours funding also has a positive effect on performance in honours. Significant college effects are also present, with students from the College of Health Sciences scoring 3.139% lower than that of a baseline student from the College of Law and Management Studies. Students in the College of Agriculture, Engineering and Science score on average 0.649% more than a baseline student from the College of Law and Management Studies. The most significant effect, however, appears to be associated with race, with black students scoring on average 4.733% lower than their counterparts from other race groups.

In the above regression model setting, a set of parameter estimates is identified that optimally links a set of demographic variables to the weighted average mark Y_i that a student i obtains in their honours year with e_i forming an error term accounting for a lack of fit in the following model structure:

The results given in Table 3, however, relate to a very specific data set -namely those 3233 students with an undergraduate degree from UKZN who were allowed to enrol for an honours degree under the current set of rules which the university is now wanting to relax. With these results in hand, a student with a demographic profile of, for example:

• Weighted average mark in final year of undergraduate study = 65%;
• Age (years) = 25;
• Matric point score = 43;
• Female = 1;
• Black = 1;
• Honours funding = 0;
• College:
  • Agriculture, Engineering and Science = 1
  • Health Sciences = 0
  • Humanities = 0

  could be expected to record (on average) the following weighted mark for their honours degree:

E(Y)=40.9+0.52(65)-0.17(25)-0.016(43) + 1.56(1)-4.73(1) + 0.65=67.24.

Because these results are estimated using a sample of 3233 UKZN graduates who were able to enrol in an honours programme, one cannot simply assume that the same sort of effects would occur if UKZN were to lower their entry requirement to include UKZN graduates with a weighted average value for their final year of undergraduate study of 50-55%. Apart from obtaining a lower weighted average score, these students may also differ in other important respects from the 3233 UKZN graduates who are currently eligible for entry into honours. To correct for a possible (sample selection) bias that may arise if we were to simply assume that the same set of results that appear in Table 3 would apply to this new cohort, a Heckman selection model was fitted to a now extended set of data comprising the 6165 UKZN graduates with a weighted average score for their final year of undergraduate study of 50-55% and the 3233 UKZN graduates currently in the honours programme for which we have a weighted average score for honours.

A Heckman selection model attempts to solve the problem associated with a potential sample selection bias by linking the actual selection process for entry into an honours programme to a set of appropriately chosen predictor variables . Let R_idenote a 0/1 indicator variable that models the selection process that is actually taking place. More specifically, assume entry into an honours programme (i.e. setting R =1) is determined by the following decision rule:

where u_f~N(0,1) denotes a random error term used to reflect the fact that once the threshold value for admission into an honours programme has been passed, admission into an honours programme is not necessarily automatic. For example, UKZN applicants are ranked according to the weighted average mark that they record in their final year of undergraduate study with 'top-slicing' then occurring based on other factors relating to classroom size and staff supervisory constraints. Parameter estimates for Equation 2 are derived from the extended set of data that include the 6165 UKZN graduates with a weighted average score for their final year of undergraduate study of 50-55% and the 3233 UKZN based undergraduate students who were allowed to enrol in honours.

As a second stage in the Heckman model, a regression model is then fitted to the weighted average marks Y_i of those 3233 UKZN undergraduate students who were able to enrol for an honours degree at UKZN:

To complete the model formulation, u_i and e_i are assumed to have a bivariate normal distribution with

In the context of this paper, it is important to note that one is only observing an honours performance based outcome for someone who has actually been selected for entry into that programme. For the self-selected subpopulation of random variables Y_jthat we are able to observe, Heckman has shown that^5,7,8:

and

where

denotes an inverse Mills ratio, φ a standard normal density and Φ a standard normal cumulative distribution function.

One could estimate the unknown model parameters in Equations 2 and 3 simultaneously using a maximum likelihood method based on the joint normality assumption of u_i;and e_iAs an alternative, however, if λ_i were observable, then ordinary least squares could be used on

with w_jdenoting a zero mean error term with variance σ²(1-ρ²δ_!) that is now distributed independently of {χ_i Z_i}. As λ_ίis unknown, Heckman proposed the following two-step procedure to estimate β in the outcome Equation 3:

• Step 1: Use all 9398 observations on Z_ithat are available, namely those associated with students who were eligible for entry into honours (i.e. those with R_i= 1 and also those who were refused entry into honours (i.e. those with R_i= 0 to generate a maximum likelihood estimator δ for δ based on the selection Equation 2. This estimate can then be used to compute the following inverse Mills ratio term for each observation; viz.

• Step 2: Now use the self-selected sample only (i.e. all 3233 observations for which we have R_i= 1) to run an ordinary least squares regression on the following equation that is implied by the formulation given by Equation 4:

Because

an appropriate correction needs to be made to the variance covariance matrix that is associated with (β, β_λ).

Tables 4 and 5 present the parameter estimates that result from fitting a Heckman model to the UKZN undergraduate based data set that is available. Table 6 contains a parameter estimate for ρ with a likelihood ratio based Wald test for ρ=0 rejected at all of the major levels of significance (chi-square=23.55, ρ=0). This result confirms that the results in Table 3 do in fact need to be corrected for a sample selection bias because unobservable factors making up the error term u_i in the selection equation are correlating positively with the error term e_i that forms part of the response model determining the weighted average mark for honours. One cannot simply extrapolate the results shown in Table 3 to the new cohort of students and assume that these exact same results will be true for this new cohort of students.

 

 

 

 

 

 

Replacing the '3-year plus honours' degree structure with a single 4-year structure is synonymous with lowering the honours entry requirement for UKZN undergraduates to 50% and excluding all foreign (non-UKZN applicants) who want to do an honours degree at UKZN. For example, for a 25-year-old, black, female UKZN undergraduate (with 43 matric points) who obtained a weighted average mark of 50% for her final year of undergraduate study and wants to enrol for an honours degree in the Faculty of Humanities, for which she did not receive any extra funding, the parameter estimates that appear in Table 4 indicate that she would on average record the following weighted average mark for honours:

E(Y)=34.46 +0.59( 50)-0.28(25)-0.03(43)+1.60(1)-4.76(1)+1.45(0)-1.07(1) = 51.44.

If one were to make use of the results that appear in Table 3 which do not correct for a sample selection bias, then a much higher expected mark would erroneously be associated with the performance of this student in honours:

E(Y)=40.98+0.52(50)-0.17(25)-0.016(43)-4.73(1)+1.56(1)-1.72(1) = 57.15.

One cannot therefore use a regression model to naively extrapolate beyond the range of the data and expect to obtain appropriate results when attempting to answer the question posed in this paper. Furthermore, significant college based effects are recorded in Table 4 which suggest that any decision to relax the entry requirement for a specific college should not be applied as a blanket rule for all colleges. For example, if a weighted average mark of 50% is all that is required to complete an honours year of study, then the results in Table 4 suggest that students in the College of Health Sciences, black students, older students and male students do not perform as well as their counterparts in their honours year of study. Having access to some form of funding seems to improve results, but it is important to note that we are dealing with an associative rather than causative effect in this analysis. For example, funding may be associated with better results because higher achievers are more likely to receive funding; thus they perform better because they are higher achievers rather than because of the funding they received.

In conclusion, the purpose of this paper was twofold. Firstly, to contribute to a debate on the restructuring of undergraduate degrees in which there is a danger associated with naively extrapolating a regression model beyond the range of the data. Secondly, to provide a modelling technique that accounts for a possible sample selection bias that may arise because the profile of the proposed students may be very different from the profile of the students in the sample from which these inferences were drawn. To answer more specifically the question posed: lowering the requirement for entry into honours for UKZN graduates will result in this new cohort not performing as well in honours as their counterparts who currently are allowed into honours. However, the results in Table 1 indicate that students from other universities who currently are allowed into honours do not perform as well as their UKZN graduate counterparts. Therefore, replacing some of these students with a new cohort of UKZN graduate applicants may in fact improve throughput rates in some of the Colleges at UKZN.

 

Possible limitations

Because this study is primarily exploratory in nature, there are several issues or limitations. Because the focus was on UKZN data only, results from this study cannot necessarily be extrapolated to other universities in South Africa. However, showing how a self-selection bias can be corrected for using a Heckman model can easily be extended to data from another university, once the need to correct for a possible self-selection bias has been fully appreciated.

Because UKZN wants to relax the entry requirement for honours study from a weighted average mark of at least 55% to that of 50% for the final year of undergraduate study, this study focused on a very specific cohort of new students, namely those in the 50-55% band who would then be eligible to join students in the 55+% bracket who currently are allowed to enrol in an honours programme. If UKZN were to further relax the requirements, for example, to include students in the 45-55% band, then a very different set of results may arise.

Given that the aim was primarily to relate the weighted average mark recorded in honours to the mark obtained in the third year of undergraduate study, the inclusion of additional factors related to race, gender and matric point score may be viewed as complicating the analysis unnecessarily. The estimates that appear in Tables 3 and 4, however, will be appropriate only if the error term e_i that is referenced in Equations 1 and 3 is no longer correlated with any of the variables {X1_; ... X_pi} that are included as predictors in these respective regression model structures. By including enough of these variables in the model building process, the hope is to reduce the omitted variable bias problem that might arise if these confounders in the analysis are not controlled for.

 

References

1. South African Department of Education (DoE). National plan for higher education. White Paper 3 section 2.1.1. Pretoria: DoE; 1997.         [ Links ]

2. South African Council on Higher Education (CHE). Report of the task team on undergraduate curriculum structure: A proposal for undergraduate curriculum reform in South Africa. Pretoria: CHE; 2013.         [ Links ]

3. Astin AW. Student involvement: A developmental theory for higher education. J Coll Student Dev.1999;40:518-529.         [ Links ]

4. Burton LJ, Dowling D. In search of the key factors that influence student success at university. Paper presented at: 28th HERDSA Annual Conference: Higher Education in a Changing World (HERDSA 2005); 2005 July 3-6; Sydney, Australia. Available from: https://eprints.usq.edu.au/753/17Burton_Dowling_HERDSA2005_PV.pdf        [ Links ]

5. Entwistle NJ, Wilson JD. Degrees of excellence: The academic achievement game. London: Hazell Watson and Viney Ltd; 1977.         [ Links ]

6. Dayioglu M, Serap TA. Gender differences in academic performance in a large public university in Turkey. High Educ. 2007;53:255-277.         [ Links ]

7. Bowden J, Marton F. The university of learning: Beyond quality and competence. Abingdon-on-Thames: Routledge Falmer, 1998.         [ Links ]

8. Jonassen DH, Grabowski BL. Handbook on individual differences, learning and instruction. Mahwah, NJ: Lawrence Erlbaum Associates;1993.         [ Links ]

9. Pustjens H, Van De Gaer E, Van Damme J, Onghena P. Effects of secondary school on academic choice and on success in higher education. Sch Eff Sch Improv. 2004;15:281-311. https://doi.org/10.1080/09243450512331383222        [ Links ]

10. Lizzo A, Wilson K, Simons R. University students perceptions of the learning environment and academic outcomes: Implications for theory and practice. Stud High Educ. 2002;27:27-52. https://doi.org/10.1080/03075070120099359        [ Links ]

11. Drew PY, Watkins D. Affective variables, learning approaches and academic achievement: A causal modelling investigation with Hong Kong tertiary students. Brit J Psychol. 1998;68:173-188. https://doi.org/10.1111/j.2044-8279.1998.tb01282.x        [ Links ]

12. Zajacova A, Lynch SM, Espenshade TJ. Self-efficacy, stress and success in college. Res High Educ. 2005;46(6):677-706. https://doi.org/10.1007/s11162-004-4139-z        [ Links ]

13. Sternberg RJ, Bonney C, Gabora L, Merrifield M. WICS: A model for college and university admissions. Educ Psychol. 2012;47(1):30-41. https://doi.org/10.1080/00461520.2011.638882        [ Links ]

14. Atkinson R, Geiser S. Reflections on a century of college admissions tests. In: Soares JA, editor. SAT wars: The case for test-optional college admissions. New York: Teachers College Press; 2012. p. 23-49.         [ Links ]

15. Soares JA. The future of college admissions: Discussion. Educ Psychol. 2012;47(1):66-70. https://doi.org/10.1080/00461520.2011.638902        [ Links ]

16. Niu SX. Minority student academic performance under the uniform admission law: Evidence from the University of Texas at Austin. Educ Eval Pol Anal. 2010;32(1):44-69. https://doi.org/10.3102/0162373709360063        [ Links ]

17. Heckman JJ, Lalonde R, Smith J. The economics and econometrics of active labor markets programs. In: Ashenfelter A, Card D, editors. Handbook of labor economics. Vol. 3. New York: Elsevier; 2000.         [ Links ]

18. Schwiebert J. Estimation and interpretation of a Heckman selection model with endogenous covariates. Emp Econ. 2015;49:675-703. https://doi.org/10.1007/s00181-014-0881-z        [ Links ]

19. Heckman JJ. Dummy endogenous variables in a simultaneous equation system. Econometrica.1978;46:931-959. https://doi.org/10.2307/1909757        [ Links ]

20. Heckman JJ. Sample selection bias as a specification error. Econometrica. 1979;47:153-161. https://doi.org/10.2307/1912352        [ Links ]

21. Briggs D. Causal inference and the Heckman model. J Educ Behav Stat. 2004;29(4):397-420. https://doi.org/10.3102/10769986029004397        [ Links ]

22. Deschacht N, Goeman K. Selection bias in educational issues and the use of Heckman's sample selection model. In: Contemporary Economic Perspectives in Education. Leuven: Leuven University Press; 2015. p. 35-53.         [ Links ]

23. Murray M. Does poor quality schooling and/or teaching hurt black South African students enrolling for a degree at the University of Kwa-Zulu Natal? PLoS ONE. 2016;11(4), e0153091, 11 pages. https://doi.org/10.1371/journal.pone.0153091        [ Links ]

24. Zewotir Z, North D, Murray M. The time to degree or dropout amongst full-time master's students at University of KwaZulu-Natal. S Afr J Sci. 2015;111(9-10), Art. #2014-0298, 6 pages. http://dx.doi.org/10.17159/sajs.2015/20140298        [ Links ]

25. Zewotir T, North D, Murray M. Student success in entry level modules at the University of KwaZulu-Natal. S Afr J High Educ. 2011;25(6):1233-1244.         [ Links ]

26. Murray M. Determining the efficacy of bridging programmes in the Faculty of Science at the University of Kwa-Zulu Natal. S Afr Stat J. 2015;49:241-257.         [ Links ]

 

 

Correspondence:
Mike Murray
murraym@ukzn.ac.za

Received: 15 Sep. 2016
Revised: 18 Jan. 2017
Accepted: 09 June 2017