International Education Studies; Vol. 9, No. 5; 2016 ISSN 1913-9020 E-ISSN 1913-9039 Published by Canadian Center of Science and Education Does Problem-Based Learning Improve Problem Solving Skills?—A Study among Business Undergraduates at Malaysian Premier Technical University Z. Abdul Kadir1, N. H. Abdullah2, E. Anthony1, B. Mohd Salleh1 & R. Kamarulzaman1 1 Department of English Language & Linguistics, Centre for Language Studies, Universiti Tun Hussein Onn Malaysia, Malaysia 2 Faculty of Technology Management, Business and Entrepreneurship, Universiti Tun Hussein Onn Malaysia, Malaysia Correspondence: Z. Abdul Kadir, Department of English Language & Linguistics, Centre for Language Studies, Universiti Tun Hussein Onn Malaysia, Malaysia. Tel: 60-19-777-6276. E-mail: [email protected] Received: October 9, 2015 Accepted: November 24, 2015 Online Published: April 26, 2016 doi:10.5539/ies.v9n5p166 URL: http://dx.doi.org/10.5539/ies.v9n5p166 Abstract Problem-based Learning (PBL) approach has been widely used in various disciplines since it is claimed to improve students’ soft skills. However, empirical supports on the effect of PBL on problem solving skills have been lacking and anecdotal in nature. This study aimed to determine the effect of PBL approach on students’ problem solving skills using a quasi-experimental non-equivalent group pretest–posttest design. Fifty management students from a premier Technical University in Malaysia were assigned to experimental and control groups. In the experimental group, students were given four problems to be solved and their solutions of the problems given were assessed in terms of their accuracy and quality. Students in the control group received conventional classroom instructional design. Results indicate that students in the experimental group have better problem solving skills (z: -4.220, p: 0.001 for accuracy and z: -2.594, p: 0.009 for quality) compared to those who were not exposed to the PBL approach. This finding substantiates the use of PBL as an effective instructional tool to improve students’ problem solving abilities. Keywords: problem solving skills, problem-based learning 1. Introduction A survey conducted by ManpowerGroup (2012) found that many employers are not satisfied with their current employees’ problem-solving skills. Similarly, fifty-five percent of employers participated in the survey by Grant Thornton LLP (AccountingWeb, 2010) claimed that recruiting accounting executives with necessary soft skills such as communication, critical thinking, and problem solving abilities poses the most significant challenge. The centrality of problem-solving skill is accentuated when it is found to enhance creativity, enjoyment, interest and learning (Yu et al., 2010). This issue has led to further scrutiny on instructional delivery at tertiary education level which mostly uses conventional classroom approach. A meta-analyses comparing PBL to conventional classrooms indicate that PBL approach is superior in terms of long-term retention, skill development and students satisfaction (Strobel & Barneveld, 2009) while Yadav et al. (2011) found that PBL leads to higher problem-solving skills and increase students’ factual knowledge (Yadav et al., 2011). Moreover, various studies in diverse disciplines such as Mathematics (Rohani & Sahar, 2012), Communication (Chapman, 2002), Medical (Nan, 2011), and Social Sciences (Nair & Alkiyumi, 2011) have shown that PBL have positive effects on students in building up their problem solving skills, critical and creative thinking skills, cooperative and communication skills, and adaptability (Albanese & Mitchell, 1993; Hmelo-Silver, 1998; Koray et al., 2008; and Nur Izzati et al., 2010). However, most of these studies are cross sectional in nature which leads to heated debates on its effectiveness. Anecdotal evidence in forms of inconsistent and contradicting findings also indicates the need to further investigate the effectiveness of PBL on problem solving skills. Thus, the main purpose of this study is to identify whether the students exposed to PBL treatments improve their problem solving skills. All students in PBL and control groups were given similar test at the beginning of the semester to establish the baseline score. The PBL group experienced the treatments while the control group was 166 www.ccsenet.org/ies International Education Studies Vol. 9, No. 5; 2016 having non-PBL approach in their learning process. At the end of the semester, another test was administered to all the students in both groups to gauge their problem solving skills. 2. Method 2.1 Design This was a quasi-experimental study with a non-equivalent control group, using pre and post analysis on measuring students’ problem solving skills in PBL learning environment and non-PBL learning environment. This design is appropriate for groups that are naturally assembled, such as intact classrooms (Campbell & Stanley, 1963). The design included the pre-test and post- test for both groups. The effect of instructional approach of learning (PBL) on students’ problem solving skills was measured in terms of the quality and accuracy of solutions provided by the students. 2.2 Sample Both experimental and control groups had 25 participants in each group. All of these participants were first year undergraduate management students at Universiti Tun Hussein Onn Malaysia, enrolling in “Effective Communication” subject. The study was conducted for 14 weeks or one semester. Prior to exposure, all participants were given a test measuring their problem solving skills based on accuracy and quality of solutions. Later on, participants from PBL group were exposed to the PBL approach such as how to deal with the problems, preparing the “FILA” table–a four-column table (Facts, Ideas, Learning Issues, Action), and how to organize the information in solving the problems. Similar test was given at the end of the semester to all participants in both groups. Marks for pre and post measures tests were compared in indentifying whether PBL helps students in fostering students’ problem solving skills. 2.3 Instruments The tests were prepared by the researchers and were verified by experts in content area, teaching and learning and PBL. The experts evaluated the content of the tests based on literature on crafting a good trigger, especially characteristics highlighted by Tan (2007). In getting the scores, the students’ answers were given to two raters. They assessed the students’ work on the accuracy and quality of the solutions. The accuracy and quality of the solutions were evaluated using the rubric provided by the researcher. The rubric was the modified version of the standardized MUET band description used in MUET examination. The scores were recorded and the data were analysed using Mann-Whitney U test. This test was selected in comparing the samples’ performances in assessing whether there is any significance difference in students’ problem solving skills from these two different groups at the beginning and at the end of the course. Furthermore, the number of respondent is 25 for each experimental and control groups, thus it can be concluded that this test is the most appropriate test to be used. Two facilitators gave their ratings on the students’ work. Marks were given and the average marks were recorded. Inter-raters of this study were chosen based on several criteria. They were experts in the course, shared the same interest, and had been teaching the same subject for several semesters. Initially three lecturers were chosen and they were given 10 scripts of the students’ work to be assessed. Based on the inter-rater agreement (kappa) result, two lecturers were chosen to be the inter-raters of this study. Table 1. Inter-rater agreement (Kappa) Kappa value p IR 1 and IR 2 0.375 0.03 IR 1 and IR 3 0.035 0.775 IR 2 and IR 3 0.205 0.159 Kappa for the agreement between the ratings of inter-rater 1 and inter-rater 2 was 0.375 with p = 0.03. However, the agreement between the ratings of inter-rater 1 and inter-rater 3, followed by inter-rater 2 and inter-rater 3, were not significant with p-values 0.775 and 0.159, respectively. Therefore, inter-rater 1 and inter-rater 2 were chosen to assess the students’ problem solving skills. 2.4 Data Collection and Procedure At the beginning of the semester, the pre-test picture was conducted at the same time among the two classes identified. The subjects were given a problem in the form of a picture to measure their ability in solving the 167 www.ccsenet.org/ies International Education Studies Vol. 9, No. 5; 2016 problem. The picture portrayed a clogged drain. 15 minutes were allocated for them to give solutions for the problem given. The test was conducted at the same time for PBL and non-PBL classes. A post-test picture was conducted at the end of the semester, after the subjects were exposed to PBL treatments. The post-test was a picture, which was similar to the one given at the beginning of the semester. The PBL and non-PBL classes sat for the test at the same time. The reason for having the post-test was to see if there was any difference in their problem solving skills before and after the exposure of the treatments. The scores on problem solving skills were processed using Statistical Package for Social Science (SPSS). All hypotheses were tested using non-parametric of Mann-Whitney U test. The non-parametric tests were conducted because the number of respondent was small (N = 25 for each PBL and non-PBL class). According to Siegel (1956), if the sample sizes are small, as small as N=6, non-parametric tests should be used unless the nature of the population distribution is known exactly. In addition, the adequate sample size for parametric tests is N>30 (Tomkins, 2006). Non-parametric tests are also available for all common experimental designs (Siegel, 1956). In addition, non-parametric tests are suitable for treating samples made up of observations from several different populations (Siegel & Castellan, 1988). 3. Results The objective of this study was to determine if there was any difference in students’ problem solving skills at the beginning and at the end of the semester after attending PBL treatments. Table 2. Participants’ demographic background (n=50) Characteristics PBL (n=25) Non-PBL (n=25) Gender: Male 10 (20%) 11 (22%) Female 15 (30%) 14 (28%) z: -.284, p = 0.777 Age: 20 4 6 21 18 12 22 3 7 z: -1.409, p = 0.159 Race: Malay 9 (18%) 11 (22%) Chinese 13 (26%) 10 (20%) Indian 2 (4%) 1 (2%) Others 1 (2%) 3 (6%) z: -.244, p = 0.807 168 www.ccsenet.org/ies International Education Studies Vol. 9, No. 5; 2016 Table 3. Pre and post measures on students’ problem solving skills between PBL and non-PBL (n=50) Non-PBL (n=25) PBL (n=25) Mean rank (accuracy post) 15.50 35.50 Mean rank (quality post) 20.88 30.12 Mann-Whitney U (accuracy) 62.50 (quality) 197.00 z (accuracy) -4.96 z (quality) -2.913 p (accuracy) 0.001 p (quality) 0.004 Table 1 displays the mean ranks of accuracy and quality on problem solving skills between PBL and non-PBL groups. It can be seen that there was a significant difference on the mean ranks between PBL and non-PBL in terms of accuracy and quality of the solution. For the PBL group, the mean rank was 35.50 compared to non-PBL group, 15.50. These results support the contention that there was improvement on problem solving skills in terms of accuracy after the students attended PBL treatments. From the aspect of quality, the mean rank value presented was 30.12 for PBL group and for the non-PBL group, the value was 20.88. Thus, it can be concluded that students in PBL group had better quality in their solutions of the problems. Furthermore, the z value displayed was -4.96 for accuracy and -2.913 for quality. The p values for accuracy and quality were 0.001 and 0.004 respectively (p<0.05). Therefore, the Mann-Whitney U test of the experiment and control groups revealed that there was a statistically significant difference between these two groups on the accuracy and quality with respect to problem solving skills after the treatments. The finding of this study was consistent with the studies done by Hmelo-Silver (1998), Koray et al. (2008), Neo and Neo (2005), and Nur Izzati et al. (2010) despite the different types of measurements used. This indicates the suitability of PBL approach to be used in various disciplines. 4. Discussion This study adds to existing literatures on the benefits of PBL approach compared to the traditional classroom approach. The problem-solving process, which is central within a PBL environment, was guided by a framework where students learn to explore and analyse problems systematically (Dochy et al., 2005). This enables students to be proactive and expand their existing knowledge bases. Students also use various sources of information from their textbooks and others in informal discussions with peers (Major & Palmer, 2001) which enable them to integrate knowledge across disciplines and approaches. Other than that, PBL provides channels for students to experience authentic problems (Savery & Duffy, 1995). By having authentic problems to be solved, students are more involved in the learning process. Therefore, students are usually more innovative and creative in their problem solving approach. In contrast, in a non-PBL approach of learning, students tend to have the knowledge rather than to create the knowledge and emphasis is on the broad coverage of content areas through lectures (Beachey, 2007). Traditional learning leads students to be passive in class, to memorize, repeat, and rely on notes given by the lecturers. When students are given a trigger by the lecturer (known as facilitator), they need to discuss and find solutions for the trigger. In the process of finding the solutions, they go through the process of information gathering (such as doing library research and search information from the Internet), conduct meetings, interact with people, and prepare the presentation (O’Grady & Alwis, 2002). Therefore, with enough information and ideas being discussed, the possibility to reach accurate and with quality solutions are there (Jonassen, 1997). 5. Recommendations Future research might need to focus on the implementation of PBL at different levels of students, such as primary school, secondary school and part-time students. In addition, a larger sample should be considered and the assessments should be done not only for one semester but it could be throughout the whole course. This study, however, has several limitations inherent to quasi experimental research design. The findings of the research cannot be generalized to other settings, since it was employed with small samples of students at a premier 169 www.ccsenet.org/ies International Education Studies Vol. 9, No. 5; 2016 Malaysian Technical University for only one semester. Furthermore, this study measured problem solving using two dimensions only, namely accuracy and quality. Thus, future research might include more comprehensive conceptualization and measurement of problem solving. References AccountingWeb. (2010). Finance execs: Many candidates lack critical thinking, problem solving abilities. Albanese, M. A., & Mitchell, S. (1993). Problem-based learning: A Review of Literature on Its Outcomes and Implementation Issues. Academic Medicine, 68, 52-81. http://dx.doi.org/10.1097/00001888-199301000-00012. Anthony, E. M., & Zulida, A. K. (2012). A road not taken: A breakthrough in english for specific purposes via problem-based learning. Journal of Technical Education and Training (JTET), 4(1), 51-71. Barrows, H. S. (1986). A taxonomy of problem-based learning methods. Medical Education, 20, 481-486. http://dx.doi.org/10.1111/j.1365-2923.1986.tb01386.x. Barrows, H. S., & Wee, K. N. L. (2007). Principles and Practice of a PBL. Singapore: Pearson Prentice Hall. Beachey, W. D. (2007). A comparison of problem-based learning and traditional curricula in Baccalaureate respiratory therapy education. Respiratory Care, 52(11), 1497-1506 Belland, B. R., French, B. F., & Ertmer, P. A. (2009). Validity and Problem-based Learning Research: A Review of Instruments Used to Assess Intended Learning Outcomes. The Interdisciplinary Journal of Problem-based Learning, 3(1). http://dx.doi.org/10.7771/1541-5015.1059 Bernama. (2010). 5 reasons why Malaysian graduates are unemployed. Retrieved from http://tlc-kdu.blogspot.com/2010/02/5-reasons-why-malaysian-graduates-are.html Bigelow, J. D. (2004). Using problem-based learning to develop skills in solving unstructured problems. Journal of Management Education, 28(5), 591-609. http://dx.doi.org/10.1177/1052562903257310 Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally & Company. Chapman, D. W. (2002). Words That Make a Difference: Problem-based Learning in Communication Arts Courses. The Journal of General Education, 51(4). http://dx.doi.org/10.1353/jge.2003.0009 Ersoy, E., & Baser, H. (2014). The Effects of Problem-based Learning Method in Higher Education on Creative Thinking. Procedia-Social and Behavioral Science, 116, 3494-3498. http://dx.doi.org/10.1016/j.sbspro.2014.01.790 Ferreira, M. M., & Trudel, A. R. (2012). The Impact of Problem Based Learning (PBL) on Student Attitudes toward Science, Problem-Solving Skills, and Sense of Community in the Classroom. Journal of Classroom Interaction, 47(1), 23-30. Gijbels, D., Dochy, F., Bossche, P. V., & Segers, M. (2005). Effects of problem-based learning: A meta-analysis from the angle of assessment. Review of Educational Research, 75(1), 27-51. http://dx.doi.org/10.3102/00346543075001027 Goodnough, K. C., & Woei, H. (2008). Engaging Teacher’s Pedagogical Content Knowledge: Adopting a Nine-Step of Problem-Based Learning Model. The Interdisciplinary Journal of Problem-Based Learning, 12(2). Heppner, P. P., & Baker, C. E. (1997). Application of problem solving inventory. Measurement & Evaluation in Counseling & Development (American Counseling Association), 29(4), 229-242. Hmelo-Silver, C. E. (1998). Cognitive consequences of problem-based learning for the early development of medical expertise. Teaching and Learning in Medicine, 10(2), 92-100. http://dx.doi.org/10.1207/S15328015TLM1002_7 Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16(3), 235-266. http://dx.doi.org/10.1023/B:EDPR.0000034022.16470.f3 Jonassen, D. H. (1997). Instructional Design Models for Well-Structured and Ill-Structured Problem-Solving Learning Outcomes. Educational Technology Research and Development, 45(1), 65-94. http://dx.doi.org/10.1007/BF02299613 Kelson, A. C. M. (2000). Assessment of students for proactive lifelong learning. In D. H. Evensen, & 170 www.ccsenet.org/ies International Education Studies Vol. 9, No. 5; 2016 Hmelo-Silver, C. (Eds.), Problem-based learning: A research perspectives on learning interactions (pp. 315-345). NJ: Lawrence Erlbaum Associates. Khairiyah, M. Y., Zaidatun, T., Syed, A. H., & Jamalludin, H. (2005). Promoting problem-based learning in engineering courses at the Universiti Teknologi Malaysia. Global Journal of Engineering Education, 9(2), 175-184. Klegeris, A., & Hurren, H. (2011). Problem-based learning in a large classroom setting: methodology, student perception, and problem solving skills. Proceedings of EDULEARN 11 Conference. Barcelona, Spain. http://dx.doi.org/10.1152/advan.00046.2011 Koray, O., Presley, A., Koskal, M. S., & Ozdemir, M. (2008). Enhancing Problem Solving Skills of Pre-service Elementary School Teachers through Problem Based Learning. Asia-Pacific Forum on Science and Teaching, 9(2), 1. Lee, M., & Kim, D. S. (2005). The effects of the collaborative representation supporting tool on problem-solving processes and outcomes in web-based collaborative problem-based learning (PBL) environments. Journal of Interactive Learning Research, 16(3), 273-293. Loyens, S. M. M, Rikers, R. M. J. P., & Schmidt, H. G. (2006). Students’ conceptions of constructivist learning: A comparison between a traditional and a PBL curriculum. Advances in Health Science Education, 11, 365-379. http://dx.doi.org/10.1007/s10459-006-9015-5 Loyens, S. M. M., Magda, J., & Rikers, R. M. J. P. (2008). Self-directed learning in problem-based learning and its relationships with self-regulated learning. Education Psychology Review, 20, 411-427. http://dx.doi.org/10.1007/s10648-008-9082-7 ManpowerGroup. (2012). 2012 Talent Shortage Survey. Retrieved June 4, 2013, from http://www.manpowergroup.us/campaigns/talent-shortage-2012/pdf/2012_Talent_Shortage_Survey_Results _US_FINALFINAL.pdf Mohd, N. I., Nor, A. N. & Irfan, N. U. (2010). The effects of mind mapping with cooperative learning on programming performance, problem solving skills and metacognitive knowledge among computer science students. J. Educational Computing Research, 42(1), 35-61. http://dx.doi.org/10.2190/EC.42.1.b Morreale, S. P., Spitzberg, B. H., & Barge, J. K. (2001). Human Communication: Motivation, Knowledge, and Skills. Belmont, CA: Wadsworth. Nair, S., & Alkiyumi, M. T. (2011). Investigation the relationship between intrinsic motivation and creative production on solving real problems. Journal Sosiohumanika, 4(2). Nan, O. (2011). Perception of first and second year medical students on problem-based learning in Universiti Malaysia Sarawak. World Applied Sciences Journal, 14(11), 1628-1634. Neo, M., & Neo, T. (2005). A multimedia-enhanced problem-based learning experience in the Malaysian classroom. Learning, Media and Technology, 30(1), 41-53. http://dx.doi.org/10.1080/13581650500075553 Nur Izzati, A., Rohani. A. T., & Rosini, A. (2010). The Effect of Problem based Learning on Mathematics Performance and Affective Attributes in Learning Statistics at Form Four Secondary Level. Procedia Social and Behavioural Sciences, 8, 370-376. http://dx.doi.org/10.1016/j.sbspro.2010.12.052. O’Bannon, F., & McFaden, C. (2008). Model of experiential andragogy: Development of a non-traditional experiential learning program model. Journal of Unconventional Parks, Tourism & Recreation Research, 1(1), 23-28. O’Grady, G., & Alwis, W. A. M. (2002). One-Day, One-Problem: PBL at Republic Polytechnics. Paper presented at 4th Asia Pacific Conference on PBL, Thailand. Pease, M. A., & Kuhn, D. (2011). Experimental analysis of the effective components of problem-based learning. Science Education, 95(1), 57-86. http://dx.doi.org/10.1002/sce.20412 Recker, M. (2010). Learning and communication theories in instructional technology. Retrieved from http://itls.usu.edu/~mimi/courses/6260/construct.html Rohani, A. T., & Sahar, B. (2012). Collaborative problem-based learning in mathematics: A cognitive load perspective. Procedia Social and Behavioural Sciences, 32, 344-350. http://dx.doi.org/10.1016/j.sbspro.2012.01.051 Savery, J. R., & Duffy, T. M. (1995). Problem-based learning: An instructional model and its constructivist 171 www.ccsenet.org/ies International Education Studies Vol. 9, No. 5; 2016 framework. Educational Technology, 35(5), 31-38. Savin-Baden, M., & Major, C. H. (2004). Foundations of problem-based learning. Maidenhead: The Society for Research into Higher Education and Open University Press. Schecter, C. (2011). Switching cognitive gears: Problem-based learning and success-based learning as instructional frameworks in leadership education. Journal of Educational Administration, 49(2), 143-165. http://dx.doi.org/10.1108/09578231111116707 Selcuk, G. S., & Caliskan, S. (2010). A Small-scale Study Comparing the Impacts of Problem-based Learning and Traditional Methods on Student Satisfaction in the Introductory Physics Course. Procedia Social and Behavioral Sciences, 2, 809-813. http://dx.doi.org/10.1016/j.sbspro.2010.03.108 Senocak, E. (2009). Development of an instrument for assessing undergraduate science students’ perceptions: The problem-based learning environment inventory. J. Science Education Technology, 18, 560-569. http://dx.doi.org/10.1007/s10956-009-9173-3 Shaizatul, A. R., & Maisarah, M. S. (2015). The Perception of Critical Thinking and Problem Solving Skill among Malaysian Undergraduate Students. Procedia-Social and Behavioral Sciences, 172, 725-732. http://dx.doi.org/10.1016/j.sbspro.2015.01.425 Siegel, S. (1956). Nonparametric statistics for the behavioral sciences. New York: McGraw-Hill Book Company, Inc. Siegel, S. (1957). Nonparametric statistics. The American Statistician, 11(3), 13-19. http://dx.doi.org/10.1080/00031305.1957.10501091 Siegel, S., & Castellan, N. J. (1988). Nonparametric statistics for the behavioral sciences (2nd ed.). New York: McGraw-Hill. Sinprakob, S., & Songkram, N. (2015). A Proposed Model of Problem-based Learning on Social Media in Cooperation with Searching Technique to Enhance Critical Thinking of Undergraduate Students. Procedia-Social and Behavioral Science, 174, 2027-2030. http://dx.doi.org/10.1016/j.sbspro.2015.01.871 Strobel, J., & van Barneveld, A. (2009). When is PBL More Effective ? A Meta-synthesis of Meta-analyses Comparing PBL to Conventional Classrooms. Interdisciplinary Journal of Problem-Based Learning, 3(4), 44-58. http://dx.doi.org/10.7771/1541-5015.1046 Subadrah, N., & Mohammed, T. A. (2011). Investigation the relationship between intrinsic motivation and creative production on solving real problems. SOSIOHUMANIKA, 4(2), 185-196. Tan, O. S. (2007). Problem-based Learning in eLearning Breakthroughs. Singapore: Thomson Learning. Tomkins, C. C. (2006). An introduction to non-parametric statistic for health scientists. University of Alberta Health Sciences Journal, 3(1), 20-26. Wijnia, L., Loyens, S. M. M., & Derous, E. (2010). Investigating effects of problem-based versus lecture-based learning environments on student motivation. Contemporary Educational Psychology, 36(2), 101-113. http://dx.doi.org/10.1016/j.cedpsych.2010.11.003 Wijnia, L., Loyens, S. M. M., Gog, T. V., Derous, E., & Schmidt, H. G. (2014). Is there a role for direct instruction in problem-based learning? Comparing student-constructed versus integrated model answers. Learning and Instruction, 34, 22-31. http://dx.doi.org/10.1016/j.learninstruc.2014.07.006 Yadav, A., Subedi, D., Lundeberg, M. A., & Bunting, C. F. (2011). Problem-based learning: Influence in students’ learning in an electrical engineering course. Journal of Engineering Education, 100(2), 253-280. http://dx.doi.org/10.1002/j.2168-9830.2011.tb00013.x Yu, W. F., She, H. C., & Lee, Y. M. (2010). The Effects of Web-based/non-web-based Problem Solving Instruction and High/Low Achievement on Students’ Problem Solving Ability and Biology Achievement. Innovations in Education and Teaching International, 47(2), 187-199. Copyrights Copyright for this article is retained by the author(s), with first publication rights granted to the journal. This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/). 172