Vol. 40 (Number 15) Year 2019. Page 21
SANTHUENKEAW, Thanarak 1; TONTIWONGWANICH, Somkiat 2 & PIMDEE, Paitoon 3
Received: 17/01/2019 • Approved: 05/04/2019 • Published 06/05/2019
ABSTRACT: Digital media, technology, and the Internet have transformed how we teach, learn and present, with flipped learning opening new frontiers for both teachers and students. Therefore, a study was undertaken of two groups of 30 Thai grade nine students to evaluate and assess how they performed in a flipped-classroom environment (experimental group) and a traditional classroom environment (control group). From this, four main steps in a flipped-classroom environment, with the experimental group having a significantly higher score compared to the control group. |
RESUMEN: Los medios digitales, la tecnología e Internet han transformado la forma en que enseñamos, aprendemos y presentamos, con el aprendizaje invertido abriendo nuevas fronteras tanto para profesores como para estudiantes. Por lo tanto, se realizó un estudio de dos grupos de 30 estudiantes tailandeses de noveno grado para evaluar y evaluar cómo se desempeñaron en un entorno de aula invertida (grupo experimental) y un entorno de aula tradicional (grupo de control). Se determinó que hay cuatro pasos principales en un entorno de aula invertida, y el grupo experimental tiene una puntuación significativamente mayor en comparación con el grupo de control. |
Rapid scientific and technological development brings about many changes in all aspects of life and work. In such cases, digital technology in the form of smartphones, computers, and the Internet have become a crucial part of all aspects of life, including business, education, and social communications (Zvezdan & Boško, 2016).
In Thailand, the vision for the future has been labelled ‘Thailand 4.0’, which finds in beginnings in Industry 4.0 and the Internet of Things (IoT). Furthermore, Thailand 4.0 is currently envisioned as a new international, digitally based, $65 billion e-commerce economy, which will be at the core of the Kingdom’s projected growth across 10 targeted industries (Leesa-nguansuk, 2017). As such, how education plays a role in these new 21st-century knowledge workers, is becoming a topic of great discussion, not only in Thailand but across the world (Lara, 2018; Reeve, 2016).
From the 2018 BETT educational technology show in London, the editors of The Economist Intelligence Unit (EIU) compiled Figure 1’s results from thousands of educators concerning their importance ranking on the question of “what are the most important teaching strategies, and how well does technology support these approaches?” (Lara, 2018).
Figure 1
What are the essential teaching strategies, and how well does
technology support these approaches? (Source: Lara, 2018)
The results also show there is agreement among the 1,200 educators surveyed, with 82% agreeing that technology is a valuable tool for modern workplace skill development, while 77% agreed that technology helps in making technology more engaging. Technology use in the classroom can also help free up time for lesson planning and encourage collaboration. Also, 81% of the surveyed educators indicated that cognitive activation (with students focusing on methods instead of solutions), and personalized learning (addressing the needs and interests of individual students) were felt to be equally important.
One use of technology in education today has appeared in the form of the ‘flipped classroom.' Origination stories differ, but most credit US chemistry teachers Jonathan Bergmann and Aaron Sams as the originators of the inverted, or flipped classroom (Arnold-Garza, 2014). The ‘flip' evolved out of a history of experimentation with the concept of hybrid, or blended learning and problem-based learning (PBL), using active learning techniques and new technologies to engage students. The flipped classroom has two defining components. The first is moving the lecture outside of the classroom, usually delivered through some electronic means, and second, moving the practical application assignments, formerly homework, into the classroom (Educause, 2012).
Although flipped learning is not explicitly classified as a technology, the tools to implement it usually are. These include the Internet, media such as YouTube, a learning management system (LMS) software such as Moodle or Schoology, computers, and smartphones. This is consistent with Pumahapinyo and Suwannatthachote (2014), which also indicated that various technologies are used to facilitate e-learning, with most e-learning applications using a combination of techniques such as blogs, collaborative software, e-Portfolios and virtual classrooms (Dorninger & Schrack, 2008). Virtual learning environments (VLEs) are also appearing more frequently in higher education as courses are consistent and standard user interfaces throughout the institution (e.g., Moodle, Schoology, Edmodo, etc.).
According to the Oxford University Press (2015), a VLE is a system for delivering learning materials to students via the web. These systems include assessment, student tracking, and collaboration and communication tools. These systems can also be accessed both in the classroom and at home, potentially supporting students' learning outside the classroom 24 hours a day, seven days a week. This is consistent with Muñoz and Towner (2009), which stated that students are heavily immersed in Web 2.0 technologies (i.e., blogs, twitter, podcasts, wikis, social network sites, virtual worlds, video sharing, and photo sharing), with educators now turning to Web 2.0 tools that assist with the creation and collaboration of content.
In Thailand, Phanich (2012) suggested that the Thai young generation has characteristics which demand that they are given the freedom to select freely what they want, expressing their personal opinions and individuality, considering play and enjoyment as all part of working, learning, and social life. This entails demanding rapid communication, searching for information and answering questions, and creating innovation for everything in life. The link therefore between social network learning and 21st-century skills has fortunately been proven (Greenhow, Robelia, & Hughes, 2009), and VLE’s now offer great potential in resolving current educational problems.
It has been further suggested that the implementation of innovative models in teaching is the critical factor that determines the ability to raise teaching quality within schools (Biljana & Dragana, 2017). This is also connected to the educator's ability at lifelong learning, with Reeve (2016) indicating the importance of teacher competency by stating it is the key to vocational and educational training (VET).
1. To develop a digital media model based on classroom-based management techniques.
2. To compare the performance and learning achievement of the students in an experimental group and a control group.
The experimental group has a higher performance and learning ability than the control group.
The study consisted of two phases with Phase 1 concerned with input from the instructors, while Phase 2 was concerned with input from the students.
Phase 1 involved the study of the issues concerning the use of flipped classrooms combined with digital technologies. Information providers included seven experts in information technology education, measurement, and evaluation in learning management, and technology innovation in education.
The data providers were seven qualified university members from Thailand, who had experience in teaching courses in industrial education and technology, teaching and learning methods, classroom management, and information and communications technology (ICT).
A focus group session was convened for the seven focus group members on the 29th of August 2016, from 9.00 am - 11.30 pm, and from 12:30 pm to 4:30 pm in the faculty’s library. All seven experts shared their views on the integration of digital media and the use of classroom-based management techniques. Furthermore, insights were gathered as to students' computer skills, and how digital tools could be used to enhance their capabilities in relation to education.
The tools used to collect data in this research consists of a structured interview as well as the analysis and synthesis of research from the theoretical and conceptual framework.
Phase 2 was concerned with the opinions and behavior of students.
The population of the research included Mathayom Suksa 3 (Grade 9) students under the Phrae Provincial Administrative Organization, Phayathai Chaiyaphum, during the 2017 academic year. The sample of 60 students was accomplished by the use of two-stage random sampling. From the sample of 60 students, 30 were selected to participate in an experimental group (Group 1). Another 30 students were selected for participation in the control group (Group 2).
Group 1 was identified as the experimental group for the duration of the study. From Group 1, the teachers clarified the learning objectives and provided a description of the course. Students also took a pre-test to assess the basic knowledge of each learner before learning the course's content. Finally, the experimental group learners were asked to follow the learning model's six steps. These included:
1. Analyze and prepare
This is the stage where the instructor analyzes the curriculum and plans to prepare digital media for the Internet. At this stage, teachers have prepared beforehand. The learning process begins in Step 2 onwards.
2. Preparation
The teacher defines the conceptual framework of the practical skills that the learner needs to learn in the classroom and prepares the learner for an understanding of how to practice (e.g., studying the details.).
3. Experimental procedure
The teacher demonstrates and learns the steps slowly after the teacher examines and evaluates the knowledge gained from the learner's experiments by observing from the classroom work that comes from understanding.
4. Practical experience
The students have repeatedly practiced. The content or the problem that the teacher prepared for the maneuverability and confidence in the assignment as skilled as a skill.
5. Application
Step 5 consists of the creation of a new assignment, where learners design new pieces of knowledge that they have learned. In Step 5 students can prepare work individually, as a team, or a larger group, according to the activities in the assignment.
6. Write, report and present the assignment
This is the stage where students present the assignments they have created and share their knowledge with other members of the group. Additionally, during Step 6 students take tests during class, practice their exercises to measure their ability to perform tasks, and accomplish a posttest.
Group 2 was identified as the control group for the duration of the study. From Group 2, the teachers clarified the learning objectives and provided a description of the course. Students also took part in a pre-test to assess the basic knowledge of each learner before learning the course's content, as well as a post-test to assess the course's outcome.
The educational experts’ opinions and areas were analyzed by content analysis. Furthermore, the appropriateness of the digital media model based on classroom learning techniques was analyzed by both mean and standard deviation (S.D.). Finally, the experts' opinions from their discussion session were taped and analyzed to further review and revise the model.
Results of the comparison between Group 1 and Group 2 student performance and achievement was conducted by use of MANOVA (multivariate analysis of variance). Additionally, an analysis was conducted by use of descriptive statistics, including frequency, percentage, mean and standard deviation (σ) (Keengwe & Onchwari, 2015). Additionally, Bartlett's Test of Sphericity measured the sampling adequacy and tested the null hypothesis that the correlation matrix is an identity matrix.
Achievement measurement was accomplished by the use of 50 quizzes which had an acceptable IOC value of 0.60 or higher, a difficulty value of 0.2-0.8, and a discriminant rating of 0.2.
Development of a rubrics assessment process was chosen to evaluate each student's performance as they can increase the consistency and reliability of scoring (Jönsson & Svingby, 2007). It consisted of a fixed measurement scale and detailed description of the characteristics for each level of performance (Table 1). These descriptions focused on the quality of the product or performance and not the quantity. Rubrics are recognized as an effective way to assess student learning while communicating expectations directly, clearly and concisely to students. Rubrics includsion as a teaching resource provides opportunities to describe stages in the development and growth of knowledge, understandings, and skills (Panadero & Jönsson, 2013). To be most effective, rubrics should allow students to see the progression of mastery in the development of understandings and skills.
Table 1
Rubrics score for assessing student performance
Question |
5 points |
4 points |
3 points |
2 points |
1 point |
1. The work meets the intended purpose. |
The work is consistent with the purpose and covers all required content. |
The work is consistent with the purpose and covers most of the content. |
The work is consistent with the purpose and covers some of the content. |
The work is consistent with the purpose but does not cover the content. |
The work is not consistent with the purpose and content. |
2. The work is entirely accurate. |
The content of the work is correct and complete. |
The content of the work is correct and complete. |
The content of the work is partially correct and complete. |
The content of the work is not correct |
The content of the work is not accurate, and not complete. |
3. Creative work. |
The work expresses the idea in a beautiful way. |
The work expresses the idea, with creativity and novelty. |
The work expresses the idea with creativity, but it is not novel and beautiful. |
The work expresses the idea with creativity, but it is not novel and has its drawbacks. |
The work does not express the idea, is not novel and not beautiful. |
4. The work is organized. |
The work is beautifully organized. |
The work is well organized, beautiful, and exquisite, with few mistakes. |
The work is well organized, beautiful, and exquisite, with some mistakes. |
The work is not well organized, beautiful, or exquisite, with many mistakes. |
The work is not organized and shows little attention to detail. |
5. The work is completed on time. |
The work was submitted on-time. |
The work was submitted 1 day late. |
The work was submitted 2 days late. |
The work was submitted 3 days late. |
The work was submitted 4 days late. |
The interpretation of the rubric's scores was defined as follows: 4.50 - 5.00 indicates that the student has an excellent level of performance: 3.50 - 4.49 indicates that the student has a good level of performance: 2.50 - 3.49 indicates that the student has a moderate level of performance: 1.50 - 2.49 indicates that the student has a low level of performance: 1.00 - 1.49 indicates that the student has the lowest level of performance. The student performance measure had an acceptable IOC of 0.60. Furthermore, the eight areas for the formal assessment of fit for the digital media model based on classroom learning management techniques which promotes the student's computer skills is presented in Table 2.
Table 2
Formal assessment of fit
Issues/Questions
|
Criteria for consideration |
Suggestion |
||
1 |
0 Uncertain |
-1 Doesn’t Fit |
||
The concepts of the teaching system, the practical skills, and the appropriateness of the theory. |
|
|
|
|
The instructional system of skills has been clearly defined. |
|
|
|
|
5. The system of teaching practical skills is appropriate for use with digital media and helping to teach. |
|
|
|
|
6. The instructional practice system is appropriate for classroom management techniques. |
|
|
|
|
7. The system of teaching skills is suitable for use in teaching and learning computer courses. |
|
|
|
|
8. The instructional systems can assess the learning achievement of learners. |
|
|
|
|
There are four main steps in the cycle of teaching in a flipped-classroom environment (Figure 2) (Gerstein, 2011).
The learner-generated steps are:
In the “now what” phase, learners get to demonstrate what they have learned and apply the material in a way that makes sense to them. This is in line with the highest level of learning within Bloom’s Revised Taxonomy of Learning, creating, whereby the learner creates a new product or point of view.
The “So what” step is involved in the creation of ideas through media such as blogging, videos, audio-visual projects, and tests.
The educator-suggested steps are:
This is the ‘Experience’ component of the flipped-learning classroom in which knowledge is gained through experiential engagement. This can include activities such as science experiments, community projects, simulations, games and the use of the Arts.
This is the ‘What’ of the learning process and is the time in the learning cycle when the learners view content-rich videos, such as Khan Academy, Neo K-12, and Veritasium.
Figure 2
The flipped-classroom cycle of learning
Source: Gerstein, 2011
The experts’ comments and opinions towards virtual learning classroom implementation of a problem-based learning model were calculated by using a formula to find mean ( ) and σ (S.D./standard deviation). The definitions of the five possible responses applied to the questionnaires were as follow:
Average scores ranging from 4.51 – 5.00 meant completely appropriate
Average scores ranging from 3.51 – 4.50 meant very appropriate
Average scores ranging from 2.51 – 3.50 meant moderately appropriate
Average scores ranging from 1.51 – 2.50 meant somewhat appropriate
Average scores ranging from 1.00 – 1.50 meant inappropriate
To test the efficiency of the flipped classroom learning management system (LMS), Promwong’s (1978) level of effectiveness index (E1/E2)was selected since it has been often accepted as a tool for assessing the quality of an instructional program in Thailand (Kannarik, 2014; Meksophawannagul & Hiranburana, 2013; Komanee, Hoxsuwan, & Phusee-on, 2013).
Promwong (1978) suggests that the acceptable quality of the effectiveness index (E1/E2) should be at 85/85 if the learning content is identified as learning by memorizing. If the learning content is considered to be a process of development or a process of changing learners’ behaviors and attitudes, (which takes time), the acceptable quality of E1/E2 should be set at 75/75 or even as low as 70/70. The learning content of the developed module was not considered to be learning by memorizing. Thus, the acceptable quality of E1/E2 should be set at 75/75, because it is equal to 75% (A ‘B' grade in Thailand.).
E1 for this study referred to the efficiency of established classroom processes. It was measured by calculating the percentage of student’s performance in classroom exercises and/or learning activities.
E2 referred to the efficiency of students’ performance development which could be calculated from post-test scores and/or completion of learning activities. If students could not reach the benchmark score, the virtual classroom needed to be changed and improved before retesting.
The steps outlined above illustrate the teacher's role and relationship with the student in the learning process as well as the teacher's activities. This is consistent with Xhemajli (2016), which presented a view on the role of teachers in interactive teaching, and suggested that teachers act as an advocate as a means of interactive teaching. This is an exciting challenge because it replaces traditional teaching that focuses on verbal teaching and content retention.
The results from this process are presented in Table 3, which shows the results of the efficiency of the Internet network based on classroom management techniques. Learners took quizzes (E1) in each lesson. The average score was 40.67, or 81.34 percent, and the (E2) score was 41.47, or 82.94 percent.
Table 3
Results of the efficiency of the Internet network based on
classroom management techniques
Score |
Students (n) |
Total Score |
Average |
Performance |
Five-subject exercise test (E1) (classroom processes) |
30 |
1220 |
40.67 |
81.34 |
Five-subject posttest (E2) (performance development) |
30 |
1244 |
41.47 |
82.94 |
Based on classroom management techniques, the total score of 30 learners was higher than the pre-test scores, and the mean scores were 21.23 (out of 50) and 41.47 (after full marks 50 points) (Table 4). The t-value is calculated as 14.58, and the t-test is 1.699. It can be concluded that the t-test is greater than the t-table. The table shows that the achievement of Adrian's' higher learning with teaching practical skills with digital media.
Table 4
The results of learning management with the
teaching of computer skills in digital media
Item |
|
Total Score |
|
SD |
df |
t-test |
t-table |
Sig. (1-tailed) |
Pre-test |
30 |
50 |
21.23 |
5.84 |
29 |
14.58 |
1.699 |
.00 |
Posttest |
30 |
50 |
41.47 |
4.59 |
Table 5 shows the expert opinions on the appropriateness of digital media formats when used with classroom-based management techniques.
Table 5
Mean and standard deviation (σ) of the expert opinions on the appropriateness of
digital media formats when used with classroom-based management techniques
Item |
|
σ |
1) The synthesis of the teaching system, the practical skills, and related theory are accurate. |
|
|
2) Application or modification in some steps is appropriate. |
0.86 |
0.41 |
3) The synthesis of teaching skills has a clear procedure and shows each step in detail. |
|
|
0.71 |
0.82 |
|
4) The sequence of training steps is appropriate. |
1.00 |
0.00 |
5) The synthesis of appropriate instructional systems for use with digital media is helping in teaching. |
0.57 |
0.84 |
6) The synthesis of instructional practices is appropriate for classroom management techniques. |
1.00 |
0.00 |
7) The instructional system of synthesized skills is suitable for use in the teaching and learning of computer courses. |
0.57 |
0.82 |
8) The synthesis of instructional practices can assess student achievement. |
1.00 |
0.00 |
Table 6 shows the results from the analysis of students’ learning achievement and digital competency between the experimental group and the control group. Results indicate a significant difference between the experimental group and the control group for both the practice ability score (PAS) and the achievement score (AS). The difference between the groups was statistically significant (Box's M = 52.64, p = .00) and Bartlett's Test found that the PAS variance and the AS have a relationship, and show that the polynomial variance can be analyzed.
Table 6
Results from the analysis of students’ learning achievement and digital
competency between the experimental group and the control group
Group |
Practice Ability Score (PAS) |
Achievement Score (AS) |
||
Mean |
SD |
Mean |
SD |
|
Experimental Group (n = 30) |
90.96 |
2.20 |
82.93 |
9.20 |
Control Group (n = 30) |
69.44 |
5.12 |
62.50 |
11.68 |
Note. Box's M = 52.64, df = 605520.00, p = .00: Bartlett's Test: Likelihood = .00, Approx. Chi-Square = 74.72, df = 2, p = .00
Table 7 shows the results from the multivariate tests (Olson, 1974). Results are judged by the following:
Each multivariate statistic is also transformed into a test statistic with an approximate or exact F distribution (Table 8). Scheffe's Test was also used in the statistical testing, which is commonly used to make unplanned comparisons among group means in an analysis of variance (ANOVA) experiment. When tested with Scheffe's 'average' test, it was found that the mean scores for both performance measures and the learning achievement of the experimental group were higher than that of the control group (Tabachnick & Fidell, 2013).
Table 7
Multivariate coefficient variance (MANOVA) scores for student achievement
Multivariate Tests |
Value |
F |
Hypothesis df |
Error df |
Sig. |
Pillai's Trace |
0.89 |
242.88** |
2.00 |
57.00 |
.00 |
Wilks' Lambda |
0.11 |
242.88** |
2.00 |
57.00 |
.00 |
Hotelling's Trace |
8.52 |
242.88** |
2.00 |
57.00 |
.00 |
Roy's Largest Root |
8.52 |
242.88** |
2.00 |
57.00 |
.00 |
-----
Table 8
Tests of between-subjects effects
Source |
Dependent Variable |
Type III Sum of Squares |
df |
Mean Square |
F |
Sig. |
Couple comparison |
Group |
skill |
6946.66 |
1 |
6946.66 |
446.77** |
.00 |
Experiment> Controls |
|
achievement |
6262.82 |
1 |
6262.82 |
56.66** |
.00 |
Experiment> Controls |
Error |
skill |
901.82 |
58 |
15.55 |
|
|
|
|
achievement |
6411.37 |
58 |
110.54 |
|
|
|
Total |
skill |
393770.88 |
60 |
|
|
|
|
|
achievement |
329937.00 |
60 |
|
|
|
|
Note. **p < .01
In conclusion, from the digital model based on classroom learning management techniques, the following six steps were determined to be appropriate by the experts (0.57-1.00). These included: 1. Analyze and prepare, 2. Preparation, 3. The experimental procedure, 4. Practical experience, 5. Application, and 6. Write, report and present the assignment.
The students in the experimental group validated the experts' assessment of the model as their average scores, ability, practice, and learning achievement was determined to be higher than that of the control group, which had a statistical significance at the .01 level. Flipped learning when combined with practical, hands-on applications such as Dreamweaver was verified as a successful process by the experimental group in teaching and learning in a digital and Internet medium, as the five-subject pre-test (E1) score was 81.34 and the five-subject posttest (E2) score was 82.94.
Concerning the development of the learners' practical skills in teaching and learning activities by digital media using the Internet and digital media instructional systems, the average score for the web pages generated by the Dreamweaver program was 22.95 (from the average of 25 points), with a standard deviation of 0.54, which indicated that the learners had improved their skills significantly.
Arnold-Garza, S. (2014). The flipped classroom teaching model and its use for information literacy instruction. Communications in Information Literacy, 8(1), 7 – 22. doi: https://doi.org/10.15760/comminfolit.2014.8.1.161
Berry, K. J., Mielke, P. W., & Johnston, J. E. (2016). Permutation Statistical Methods: An Integrated Approach. Berlin, Germany: Springer Books.
doi: https://doi.org/10.1007/978-3-319-28770-6
Biljana, N. C., & Dragana, S. (2017). Education need of teacher for introduction and application of innovative models in education work to improve teaching. International Journal of Cognitive Research in Science, Engineering and Education, 5(1). doi: https://doi.org/10.5937/IJCRSEE1701049N
Dorninger, C., & Schrack, C. (2008). Future learning strategy and ePortfolios in education. International Journal of Emerging Technologies in Learning, 3(1), 11-14.
doi: https://doi.org/10.1007/978-0-387-09729-9_35
Educause. (2012). 7 things you should know about flipped classrooms. Retrieved from: http://net.educause.edu/ir/library/pdf/ELI7081.pdf.
Gerstein, J. (2011). The flipped classroom model: A full picture. User-generated education. Retrieved from: https://tinyurl.com/gor2e3t.
Greenhow, C., Robelia, B., & Hughes, J. E. (2009). Learning, teaching, and scholarship in a digital age: Web 2.0 and classroom research--What path should we take "now"? Educational Researcher, 38(4), 246–259. Retrieved from: http://tinyurl.com/hdwdq3t.
Hutchings, M., & Quinney, A. (2015). The flipped classroom, disruptive pedagogies, enabling technologies and wicked Problems: Responding to ‘the bomb in the basement’”. The Electronic Journal of e-Learning, 13(2), 106-119. Retrieved from: https://tinyurl.com/yd4dy57o.
Keengwe, J., & Onchwari, G. (2015). Handbook of Research on Active Learning and the Flipped Classroom Model in the Digital Age. IGI Global Publishers.
Lara, V. 2018. Tailoring education for the 21st century: perspectives from educators. Economist Intelligence Unit. Retrieved from: https://tinyurl.com/y8cwdbte.
Leesa-nguansuk, S. (2017). Lazada to provide online training to local SMEs. Bangkok Post,
Retrieved from: http://tinyurl.com/l4ohxb2.
Jönsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130 – 144. doi: https://doi.org/10.1016/j.edurev.2007.05.002
Kannarik, P. (2014). Integrating google apps for education to business English student trainees’ on-the-job training English reports. In: The Proceedings of International e-Learning Conference 2014, August 5-6, 2014. Retrieved from: http://tinyurl.com/qgnoct3.
Komanee, S., Hoxsuwan, S., & Phusee-on, S. (2013). The contemplative education learning environment management model for undergraduate students. The Journal of Global Business Management, 9(3). Retrieved from: http://tinyurl.com/ppx39w7.
Meksophawannagul, M., & Hiranburana, K. (2013). The effectiveness of an online case-based collaborative learning (CBCL) module for a business English communication course. International Journal of Business and Social Science, 4(12). Retrieved from:
http://tinyurl.com/qctvlpl.
Muñoz, C. L., & Towner, T. L. (2009). Opening Facebook: How to use Facebook in the college classroom. In I. Gibson, R. Weber, K. McFerrin, R. Carlsen & D. Willis (Eds.). Proceedings of Society for Information Technology & Teacher Education International Conference 200, 2623-2627. Chesapeake, VA: Association for the Advancement of Computing in Education (AACE). Retrieved from: http://tinyurl.com/ygtjsvp.
Olson, C. L. (1974). Comparative robustness of six tests in multivariate analysis of variance. Journal of the American Statistical Association, 69(348), 894-908.
doi: https://doi.org/10.2307/2286159
Oxford University Press. (2015). Learn about Virtual Learning Environment/Course Management System content. Retrieved from: http://tinyurl.com/o54enla.
Panadero, E., & Jönsson, A. (2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9, 129 – 144. doi: https://doi.org/10.1016/j.edurev.2013.01.002
Phanich, W. (2012). Ways to build learning for students in 21st century. Bangkok, Thailand: Sodsri – Saritwong Foundation. Retrieved from: http://tinyurl.com/zcc233j.
Promwong, C. (1978). Introduction to social research: Quantitative and qualitative approaches, Psychology, 18, 79-92. Retrieved from: http://tinyurl.com/jdnpuvm.
Pumahapinyo, S., & Suwannatthachote, S. (2014). Factors affecting the innovation-decision process to adopt online graduate degree program in Thailand. In: The Proceedings of International e-Learning Conference 2014, August 5-6, Bangkok, Thailand.Retrieved from: http://tinyurl.com/qgnoct3 .
Reeve, E. M. (2016). 21st-century skills needed by students in technical and vocational education and training (TVET). Asian International Journal of Social Sciences, 16(4), 65 - 82. doi: https://doi.org/10.29139/aijss.20160404
Tabachnick, B. G., & Fidell, L. S. (2013). Using Multivariate Statistics. Pearson New International Edition.
Xhemajli, A. (2016). The role of the teacher in interactive teaching. International Journal of Cognitive Research in Science, Engineering and Education,4(1), 31-38. doi: Retrieved from: https://zenodo.org/record/59415#.Wy82DCBQjIU.
Zvezdan, A., & Boško M. (2016).Importance of computer technology in realization of cultural and educational tasks of preschool institutions. International Journal of Cognitive Research in Science, Engineering and Education,4(1). doi: Retrieved from: https://doi.org/10.5937/ijcrsee1601009a.
1. Faculty of Industrial Education and Technology, King Mongkut’s Institute of Technology Ladkrabang (KMITL), Bangkok, Thailand, e-mail: 58603003@kmitl.ac.th
2. Faculty of Industrial Education and Technology, King Mongkut’s Institute of Technology Ladkrabang (KMITL), Bangkok, Thailand, e-mail: somkiat.tu@kmitl.ac.th
3. Faculty of Industrial Education and Technology, King Mongkut’s Institute of Technology Ladkrabang (KMITL), Bangkok, Thailand, e-mail: paitoon.p@kmitl.ac.th