SciELO - Scientific Electronic Library Online

vol.22 issue1 author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand



Related links

  • On index processCited by Google
  • On index processSimilars in Google


South African Journal of Psychiatry

On-line version ISSN 2078-6786
Print version ISSN 1608-9685

S. Afr. j. psyc. vol.22 n.1 Pretoria  2016 



Reliability of the GAIN-SS, CRAFTT and PESQ screening instruments for substance use among South Africanadolescents



Tara CarneyI; Bronwyn MyersI, II; Johann LouwIII

IAlcohol, Tobacco and Other Drug Use Research Unit, South African Medical Research Council, South Africa
IIDepartment of Psychiatry and Mental Health, University of Cape Town, South Africa
IIIDepartment of Psychology, University of Cape Town, South Africa





INTRODUCTION: Screening for adolescent substance use can assist with the early identification of substance-related problems and guide the provision of appropriate services. As such, psychometrically sound screening tools are needed. The aim of this study was to compare the reliability of the CRAFFT, Global Appraisal of Individual Needs-Short Screener (GAIN-SS) substance use subscale and Personal Experience Screening Questionnaire (PESQ) among adolescents from disadvantaged communities in Cape Town, South Africa.
METHODS: Adolescents aged 12-19 years (n = 231) completed the three screeners at two points in time.
RESULTS: Findings show that all three of the screeners had adequate internal consistency (Cronbach
α > 0.8). Test-retest reliability was similar for all three screeners, with intraclass correlation coefficient values slightly higher for the PESQ (0.82, 95% CI: 0.77-0.86) than for the GAIN-SS substance use subscale (0.79, 95% CI: 0.73-0.84) and CRAFFT (0.76; 95% CI: 0.660.83). Kappa values indicated that the GAIN-SS substance use subscale and CRAFFT had moderate levels of agreement, while the PESQ had substantial levels of agreement for identifying those who had moderate or higher substance use risks at Time 1 and Time 2.
CONCLUSION: The findings indicate that all of these short screeners seem to have acceptable reliability when used in this population. All of the three screeners are appropriately reliable when used with adolescents from disadvantaged communities in Cape Town, but the PESQ performed slightly better. Future studies should also include the assessment of validity of these screeners in this context.



There is a high prevalence of substance use among adolescents in the Western Cape, with up to two-thirds of adolescents in this province reporting a lifetime use of at least one substance.1,2 Adolescents in this province report using a broad range of substances, with alcohol and tobacco being the most commonly reported, followed by cannabis and methamphetamine.3 The high prevalence of substance use among adolescents in this region is cause for concern as adolescence is a sensitive developmental period4,5 and early initiation of substance use could negatively impact on neurocognitive development.6 In addition, adolescent substance use has been associated with poor physical health,7 mental health,8,9 and academic outcomes10 as well as engagement in other risk behaviours,2,9,11 all of which may negatively impact on future well-being.12

Consequently, it is vital to identify adolescents who may be using substances so that those at risk for adverse consequences can be provided with preventative interventions that reduce the risk of harms associated with continued substance use involvement.13 There is accumulating evidence that brief interventions are effective options for adolescents who have mild-to-moderate substance use problems but may not yet require intensive treatment.14,15,16,17 However, to realise the promise of these brief interventions, adolescents who may benefit from these interventions first need to be detected. Universal screening of adolescents in healthcare settings has been recommended as a method for identifying adolescents with potential substance use problems.18

For universal screening of adolescents to be effective, self-report screening tools need to be reliable for this subpopulation.13,14,15,16,17,18 Using a reliable screener means that it should be relatively free from error and consistent in its measurement. Two types of reliability are usually measured for screening tools: internal consistency and test-retest reliability. A screener with acceptable levels of internal consistency implies that responses to the different items in the instrument are similar to each other,19 while for test-retest reliability, the results of the screener administered at two different times are the same unless there has been an actual change in their behaviour.19,20 If a screener is reliable over time, one could advise service providers that the tool is an accurate reflection of problem behaviours.

Screening tools should also be effective in detecting those at risk of and actively using substances,18 so that they can differentiate between adolescents who are and are not at risk for substance use at various time periods. In the United States, several screening tools developed to detect potential substance use problems among adolescents have been shown to be psychometrically sound. For example, the Global Appraisal of Individual Needs-Short Screener (GAIN-SS) is a brief self-report screening tool that was developed from a standardised clinical interview.21 US studies have shown that the GAIN-SS performs well in comparison with standardised clinical interviews, demonstrating acceptable internal consistency, and is able to accurately identify adolescents who use substances.22

The CRAFFT is another brief screening tool that has acceptable reliability and validity among US adolescent populations.23,24,25,26 It comprises six questions examining substance use and risk behaviours associated with substance use,5 which are simple to score and easy to remember.6 In US adolescent populations, the CRAFFT appears to have adequate levels of test-retest reliability and internal consistency,27 but internal consistency findings in other countries have been equivocal.28,29

Another screener that has been developed from a more comprehensive clinical assessment is the Personal Experience Screening Questionnaire (PESQ). Items in the PESQ derive from the Personal Experience Inventory.13,30 The PESQ is not as well-studied as other screeners. Two studies using diverse US adolescent populations have demonstrated that the PESQ has acceptable levels of reliability and validity.31,32

While all three of these screeners seem appropriate to use among US populations, the extent to which these screeners are helpful for identifying adolescents in developing country settings, such as South Africa, who may be at risk for substance use disorders remains unknown. For example, although the GAIN-SS and CRAFFT are increasingly utilised as screeners for substance use in South Africa,31,32 their psychometric properties have not been established for local populations, and it is unclear whether findings from US studies can be extrapolated to South Africa.33,34 South African adolescents have lower levels of literacy35 and educational attainment36 relative to their counterparts in developed countries. This may impact on how South African adolescents respond to the items contained in these screeners and consequently their sensitivity and specificity for this population. Before these screening tools can be recommended for use among South African adolescents, more research is needed to establish their psychometric properties for this population.



The aim of this study was to examine the relative performance of the GAIN-SS, CRAFFT and PESQ among a sample of South African adolescents. Specifically, we set out to establish the internal consistency reliability and test-retest reliability of these screeners.


This study used a repeated measures design with two data collection time points.

Sample characteristics

Participants were recruited using convenience sampling. Adolescents were recruited from 15 community-based organisations providing programmes for at-risk adolescents from several economically disadvantaged communities in Cape Town. To be included in the study, participants had to be aged between 12 and 19 years, attending school (or the equivalent of school) and able to understand English. EpiCalc 2000 was utilised to calculate the sample size that would be needed to have least 80% power, allowing for a 5% error margin rate. Post-hoc power calculations were consequently done with the following results. In a test for agreement between two raters using the Kappa statistic, a sample size of 260 subjects achieves 90% power to detect a true Kappa value of 0.65 in a test of H0: Kappa = κ0 versus H1: Kappa Φ κ0 when there are three categories with frequencies equal to 0.50, 0.30 and 0.20. This power calculation is based on a significance level of 0.05 (Table 3).






The set of screeners included the GAIN-SS, CRAFFT and PESQ

GAIN-SS: In addition to demographic questions, the GAIN-SS comprises 20 questions grouped into four subsections, which address internalising disorders (mental health issues such as depression and anxiety), externalising disorders (mental health issues related to attention and behaviour), substance use disorders (abuse, dependence and problems resulting from any kind of substance use) and engagement in crime or violent behaviours (interpersonal violence and criminal behaviour). Each subscale consisted of five questions.21,22 The current study, however, only examined the substance use subscale in order to compare the psychometric properties of this subscale relative to the other screeners. Respondents are asked to rate the frequency of symptom/behaviour occurrence, with responses rated on a five-point Likert scale from 'never' (score: 0), 'more than a year ago' (score: 1), '7-12 months ago' (score: 2), '1-6 months ago' (score: 3) to 'in the past month' (score: 4) to provide a clear idea of how recently behaviours occurred in this study. The recommended cut-off point for the substance use subscale is 1, which indicates moderate use and a need for brief intervention or outpatient intervention. A score of three or higher indicates a probable diagnosis but with a requirement for a formal assessment and intervention.22

CRAFFT: The CRAFFT includes questions about past year alcohol use, cannabis use and other drug use (Section A) and six questions on the consequences of substance use (Section B).26 Responses are 'yes' or 'no' for all items. Each 'yes' answer in Section B is scored as 1, with a total score of two indicating that further assessment of substance use involvement is warranted.25

PESQ: A modified version of this screener was developed to include demographic questions on race, gender, current grade and age. The screener includes 18 items that ask about substances used and the consequences of substance use. Responses to these items range from 'never' (score: 1), 'once or twice' (score: 2), 'sometimes' (score: 3) or 'often' (score: 4). Cut-off scores of 23 and 24 are used to identify younger (aged 12-15 years) and older (aged 16-18 years) adolescents, respectively, who would benefit from a brief substance use intervention. However, younger adolescents who score at least 30, older adolescent females who score at least 34 and older adolescent males who score at least 35 should receive a comprehensive assessment as they may have a substance abuse or dependence problem.30



Ethics approval for this study was obtained from the Faculty of Health Sciences at the University of Cape Town. Community organisations were approached, and informed about the study. At those organisations that agreed to participate in the study sessions, appointments were arranged for administering the screeners to eligible participants at Time 1 (T1) and 2 days later, at Time 2 (T2). This short time period was selected to avoid measurement over the weekend, which is the most likely time that adolescents used substances. This ensured that the screeners were not measuring any actual behavioural differences. Before the screeners were administered, we obtained informed consent to participate in the study from parents and (separately) from the adolescents. After the informed consent process was completed, a project staff member read the screeners aloud to the adolescents, item by item. Participants then completed the screeners themselves in a private setting approximating examination conditions. Each participant was given a unique identifying number which was placed on the screener that they completed. This enabled questionnaires at T1 and T2 to be linked to a single participant. Screeners were provided in English, with an Afrikaans and isiXhosa project staff member present to translate if necessary. The participants were provided with small incentives for their participation in the study. The screeners were administered in the same order at T1 and T2.

Data analysis

Data were entered into Excel spreadsheets and checked for inconsistencies. It was then imported into SPSS (Version 22). Frequencies and descriptive statistics were calculated at T1 and T2. The psychometric evaluation of the instruments included the following:

1. Test-retest reliability: Intraclass correlation coefficients (ICCs) were computed to compare the results from T1 and T2. ICC values of 0.40-0.59 are considered fair, 0.60-0.74 considered to be good and 0.75 or more considered as excellent.37 Kappa coefficients were then used to compare the performance of the overall scale to the dichotomised scoring of the scale (those who scored above and those who scored below the cut-off scores). Moderate agreement is considered to apply to kappa values of 0.41-0.6, substantial agreement to values between 0.61 and 0.80 while values over 0.81 are considered near perfect agreement.37

2. Internal Consistency: Cronbach's alpha for the total list of items and for the subscales to measure internal consistency. Acceptable alpha values range from 0.70 to 0.95 for optimal levels of internal consistency.38



The sample size was 266 at T1, and 231 at T2, with 35 participants dropping out because of academic or sports commitments (n = 27) and a small number not providing a reason for drop out (n = 8). Participants were 56.3% female, 48.9% Black African and 48.5% 'mixed race'. The majority of participants were in Grade 9 (22.6%), followed by Grade 8 (18.7%). Over a quarter of the participants were still attending primary school (27.4%) and the mean age of participants was 15 (SD = 1.67; Table 1).

The mean score on the GAIN-SS substance use subscale was slightly lower at T1 (M = 2.73, SD = 3.65) than at T2 (M = 3.42, SD = 4.97). The difference was statistically significant (t = - 2.94, df = 230, p < 0.01). In terms of test-retest reliability and internal consistency, respectively, the ICC values (ICC: 0.79, 95% CI: 0.73-0.84) and the alpha values (α = 0.80) were high and very similar (Table 2). For the overall GAIN-SS scale (not shown in the table), the internal consistency at T1 was higher (α = 0.88) and the ICC was 0.87 (95% CI: 0.84-0.90). The mean score for the PESQ was slightly higher at T1 (M = 26.03, SD = 8.76) than at T2 (M= 24.43). This difference was statistically significant (t = 3.91, df = 230, p < 0.01). Again, the ICC (ICC: 0.82, 95% CI: 0.77-0.86) and alpha values (α = 0.83) were high. The mean score for the CRAFFT was also significantly higher at T1 (M = 1.96, SD = 1.55) than at T2 (M = 1.37, SD = 1.32; t = 6.27, df = 120, p < 0.01). The ICC value (ICC: 0.76, 95% CI: 0.660.83) and alpha values (α = 0.82) were similar.

The GAIN-SS identified 40.7% of participants with at least mild-to-moderate substance-related problems at both T1 and T2. The Kappa (K) values indicated that there was a moderate level of agreement between the two time periods (K = 0.59). For the CRAFTT, close to a third of the participants scored above the cut-off point for substance use problems at T1 (30.7%) but only a quarter scored above the cut-off point at T2 (24.7%). The Kappa (K) values indicated that there was a moderate level of agreement between the two time periods (K = 0.56). The PESQ also had a higher number of participants who scored above the cut-off at T1 (53.7%) in comparison with T2 (41.1%). The Kappa value was the highest of the three screeners (K = 0.65).



This study is the first to explore the psychometric properties of the CRAFFT, GAIN-SS and PESQ for use among South African adolescent populations. Findings from this study are potentially important, as these screeners are increasingly being used to detect adolescents who may benefit from substance use interventions,35,36 yet little is known about their reliability sc in adolescent populations from low- and middle-income countries. Findings from this study therefore provide preliminary insights into the cross-cultural applicability of these screeners.

Our findings suggest that while all three screeners have acceptable levels of temporal stability and internal consistency among South African populations. However, there were some differences among the screeners, and while the GAIN-SS and PESQ scored similarly, the CRAFFT seemed to have better internal consistency than test-retest reliability. The PESQ also seemed to have slightly higher test-retest reliability in the current study than the GAIN-SS and CRAFFT.

These findings are in agreement with findings from US studies which show that the GAIN-SS and PESQ have adequate psychometric properties. In terms of internal consistency, for example, findings from this study are consistent with studies examining psychometric properties of the CRAFFT,23,26 PESQ30 and GAIN-SS22,38 in terms of internal consistency in the United States. Findings from countries outside of the United States regarding the CRAFFT in comparison with other short screeners indicate that its reliability is lower than others,28,29 which was the case in the current study as well, albeit only very slightly. There only seems to be evidence on test-retest reliability on the CRAFFT,27 so this study is one of the first studies to look at temporal stability of a number of short screeners for adolescents.

In addition, even in this small sample of school-going adolescents in Cape Town, the average scores at both time points were above the cut-off scores for moderate risk of substance use problems. These findings are in line with previous South African studies that used the GAIN-SS32 and CRAFFT,33 although there have been no studies using the PESQ. This indicates the need for short screeners that show consistent performances with South African adolescents, in order to detect early problematic substance use and then provide timeous services. However, while the reliability values were acceptable, there were still differences between the proportions that scored above the cut-off points at the two time periods that were unaccounted for. Adolescents who score above the cut-off should first be referred for a comprehensive assessment that allows for potential substance use problems to be more thoroughly explored before being referred to intervention services after screening.

In summary, findings on the screeners' performances suggest that GAIN-SS and CRAFFT are potentially suitable for use in this resource-poor context, because both have adequate psychometric properties, are brief, and are easy to administer and score.

Nonetheless, these findings should be considered in the light of several methodological limitations. Firstly, it was not possible to compare the three screeners with a standardised diagnostic interview which is the gold standard for determining the diagnostic ability of short screeners. This is costly, and is beyond the resources of the current study. However, the GAIN-SS is based on the full GAIN interview and has acceptable psychometric properties that may be closest to a diagnostic interview, but this could not be tested in the current study.40 Secondly, all three of the screeners were originally developed in English. Despite the training provided to research staff and the thorough instructions provided to adolescents in their indigenous language, it is possible that some of the adolescents would have had a better comprehension of the screening questions if they were provided in their home language.

Future studies should consider translating these screeners into the other two indigenous languages most commonly utilised in the Western Cape and assessing the psychometric properties of these language versions. Finally, it is possible that some of the adolescents simply recalled their responses given at Time 1 and again at Time 2, since it was only 2 days later. Further research that extends the interval between Time 1 and Time 2 should therefore be conducted, to compare if there are differences between the results with a longer time period (Figure 1).




In conclusion, our findings suggest that the GAIN-SS, PESQ and CRAFFT are appropriate for use with South African adolescents, although the CRAFFT seemed slightly less reliable than the other two screeners. While it is important to measure reliability, future studies should also consider measuring these screeners against a standard diagnostic tool in order to measure validity, and obtain a comprehensive picture of the instruments' psychometric properties. However, this is an important step in identifying appropriate tools for the early detection and intervention of substance-using adolescents.



This study was funded by an International Congress of Psychology Change fellowship (Myers) and Hendrik Vrouwes scholarship (Carney). These funding sources had no further role in study design, in the analysis or interpretation of data, in the writing of the report or in the decision to submit the paper for publication.

Competing interests

The authors declare that they have no financial or personal relationships which may have inappropriately influenced them in writing this article.

Authors' contributions

T.C., B.M. and J.L. equally contributed to the research and writing of this article.



1. Reddy SP, James S, Sewpaul R, et al. Umthenthe Uhlaba Usamila - The South African youth risk behaviour survey 2002. Cape Town: South African Medical Research Council; 2003.         [ Links ]

2. Reddy SP, James S, Sewpaul R, et al. Umthente Uhlaba Usamila - The South African Youth Risk Behaviour Survey 2008. Cape Town: South African Medical Research Council; 2010.         [ Links ]

3. Morojele NK, Myers B, Townsend L, et al. Survey on substance use, risk behaviour and mental health among grade 8-10 learners in Western Cape provincial schools, 2011. Cape Town: South African Medical Research Council; 2013.         [ Links ]

4. Meeus W. The study of adolescent identity formation 2000-2010: A review of longitudinal research. J Res Adolesc. 2011;21(1):75-94.        [ Links ]

5. Casey BJ, Getz S, Galvan A. The adolescent brain. Dev Rev. 2008;28:62-77.        [ Links ]

6. Crews FT, Boettiger CA. Impulsivity, frontal lobes and risk for addiction. Pharmacol Biochem Behav. 2009;93:237-247.        [ Links ]

7. Morojele NK, Brook JS, Kachieng'a MA. Perceptions of sexual risk behaviours and substance abuse among adolescents in South Africa: A qualitative investigation. AIDS Care. 2006;18:215-219.        [ Links ]

8. Saban A, Flisher AJ, Distiller G. Association between psychopathology and substance use among school-going adolescents in Cape Town, South Africa. J Psychoactive Drugs. 2010;42(4):467-476.         [ Links ]

9. Plüddemann A, Flisher AJ, Mathews C, Parry CDH, Lombard C. Methamphetamine use, aggressive behaviour and other mental health issues among high-school students in Cape Town, South Africa. Drug Alcohol Depen. 2010;109:14-19.        [ Links ]

10. Plüddemann A, Flisher AJ, Mathews C, Carney T, Lombard C. A prospective study of methamphetamine use as a predictor of high school non-attendance in Cape Town, South Africa. Subst Abuse Treat Prev Policy. 2010;5(25):5-25.        [ Links ]

11. Brook JS, Morojele NK, Pahl K, Brook DW. Predictors of drug use among South African adolescents. J Adolescent Health. 2006;38:26-34.        [ Links ]

12. Lubman DI, Hides L, Yücel M, Toumbourou JW. Intervening early to reduce developmentally harmful substance use among youth populations. M J Australia. 2007;187(7):S22-S25.         [ Links ]

13. Winters KC, Kaminer Y. Screening and assessing adolescent substance use disorders in clinical populations. J Am Acad Child Psychiatry. 2008;47:740-744.        [ Links ]

14. Erickson SJ, Gerstle M, Feldstein SW. Brief interventions and motivational interviewing with children, adolescents, and their parents in pediatric health care settings: A review. Arch Pediatr Adolesc Med. 2005;159(12):1173-1180.        [ Links ]

15. Grenard JL, Ames SL, Wiers RW, Thush C, Stacy AW, Sussman S. Brief intervention for substance use among at-risk adolescents: A pilot study. J Adolesc Health. 2007;4(2):188-191.        [ Links ]

16. Tait RJ, Hulse GK. A systematic review of the effectiveness of brief interventions with substance using adolescents by type of drug. Drug Alcohol Rev. 2003;22:337-346.        [ Links ]

17. Wachtel T, Staniford S. The effectiveness of brief interventions in the clinical setting in reducing alcohol misuse and binge drinking in adolescents: A critical review of the literature. J Clin Nurs. 2010;19:605-620.        [ Links ]

18. Babor TF, McRee BG, Kassebaum PA, Grimaldi PL, Ahmed K, Bray J. Screening, brief intervention, and referral to treatment (SBIRT): Toward a public health approach to the management of substance abuse. J Lifelong Learn Psychiat. 2011;9(1):130-148.        [ Links ]

19. Center for Substance Abuse Treatment. Screening and assessing adolescents for substance use disorders: Treatment Improvement Protocol (TIP) series. Rockville, MD: Substance Abuse and Mental Health Services Administration; 1998.         [ Links ]

20. Levy S, Knight JR. Screening, brief intervention, and referral to treatment for adolescents. J Addict Med. 2008;2:215-221.        [ Links ]

21. Dennis M, White M, Titus JC, Unsicker J. Global appraisal of individual needs-short screener (GAIN-SS): Administration and scoring manual version 2.0.3. IL: Chestnut Health Systems; 2008.         [ Links ]

22. Dennis ML, Chan YF, Funk RR. Development and validation of the GAIN Short Screener (GSS) for internalizing, externalizing and substance use disorders and crime/violence problems among adolescents and adults. Am J Addict. 2006; 15(S1):80-91.        [ Links ]

23. Dhalla S, Zumbo BD, Poole G. A review of the psychometric properties of the CRAFFT instrument: 1999-2010. Curr Drug Abuse Rev. 2011;4:57-64.        [ Links ]

24. Knight JR, Sherritt L, Harris SK, Gates EC, Chang G. Validity of brief alcohol screening tests among adolescents: A comparison of the AUDIT, POSIT, CAGE, and CRAFFT. Alcohol Clin Exp Res. 2003;27(1):67-73.        [ Links ]

25. Knight JR, Sherritt L, Harris SK, Gates EC, Chang G. Validity of the CRAFFT substance abuse screening test among adolescent clinic patients. Arch Pediat Adol Med. 2002;156:607-614.        [ Links ]

26. Knight JR, Shrier LA, Bravender TD, Farrell M, Vander Bilt J, Shaffer HJ. A new brief screen for adolescent substance abuse. Arch Pediat Adol Med. 1999;153:591-592.        [ Links ]

27. Levy S, Sherritt L, Harris SK, et al. Test-retest reliability of adolescents' self-report of substance use. Alcohol Clin Exp Res. 2004;28(8):1236-1241.        [ Links ]

28. Rumpf H, Wohlert T, Freyer-Adam J, Grothues J, Bischof G. Screening questionnaires for problem drinking in adolescents: Performance of AUDIT, AUDIT-C, CRAFFT and POSIT. Eur Addict Res. 2013;19:121-127.        [ Links ]

29. Subramaniam M, Cheok C, Verma S, Wong J, Chong SA. Validity of a brief screening instrument-CRAFFT in a multiethnic Asian population. Addict Behav. 2010;35: 1102-1104.         [ Links ]

30. Winters KC, DeWolfe J, Graham D, St. Cyr W. Screening American Indian youth for referral to drug abuse prevention and intervention Services. J Child Adolest Subst. 2006;106(1):39-52.        [ Links ]

31. Winters KC. Development of an adolescent alcohol and other drug abuse screening scale: Personal experience screening questionnaire. Addic Behav. 1992;17:479-490.        [ Links ]

32. Carstens A. Analysis of GAIN-SS data: 2012. Cape Town: The Mutida Foundation: Outpatient Treatment Programmes for Secondary Schools; 2012.         [ Links ]

33. Moodley SV, Matjila MJ, Moosa MYH. Epidemiology of substance use among secondary school learners in Atteridgeville, Gauteng. S Afr J Psychiatry. 2012;18(1):2-9.         [ Links ]

34. Mitchell SG, Gryczynski J, O'Grady KE, Schwartz RP. SBIRT for adolescent drug and alcohol use: Current status and future directions. J Subst abuse Treat. 2013;44:463-472.        [ Links ]

35. Flisher AJ, Townsend L, Chikobvu P, Lombard CF, King G. Substance use and psychosocial predictors of high school dropout in Cape Town, South Africa. J Res Adolesc. 2010;20(1):237-255.        [ Links ]

36. Statistics South Africa. Census 2011. Pretoria: Statistics South Africa; 2012.         [ Links ]

37. Hallgren KA. Computing inter-rater reliability for observational data: An overview and tutorial. Tutor Quant Methods Psychol. 2012;8(1):23-34.         [ Links ]

38. McDonell MG, Comtois KA, Voss WD, Morgan AH, Ries RK. Global appraisal of individual needs short screener (GSS): Psychometric properties and performance as a screening measure in adolescents. Am J Drug Alcohol Abuse. 2009;35(3):157-160.        [ Links ]

39. Dennis ML, White M, Titus JC, Unsicker J. Global appraisal of individual needs: Administration guide for the GAIN and related measures. IL: C.H. Systems: Normal; 2008.         [ Links ]



Tara Carney

Received: 17 Dec. 2015
Accepted: 23 May 2016
Published: 15 July 2016

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License