Evaluating the Impact of Formative Assessment Intervention and Experiences of Secondary Teachers in Teaching High-Order - Thinking Skills in Physics

Introduction

Formative assessment (FA) is an active and intentional learning process, where a teacher and the students continuously and systematically gather evidence of learning with an intention of improving students learning outcomes. In addition, FA is a planned, ongoing process used by all students and teachers during learning and teaching to elicit and use evidence of student learning to improve student understanding of intended disciplinary learning outcomes and support students to become self-directed learners. The teaching/learning process is systematic and dependent on the type of learning tasks and thinking involved in the process. In the teaching learning process a teacher is a stakeholder, a facilitator of learning and consciously initiates the student into collective learning achievement of every learner in the classroom setting [1].

In the 21st century, students are required to acquire numerous skills including critical thinking in order to deal with many situations that arise in the real world. Such thinking skills more often are imagined to be Higher Order Thinking Skills (HOTS). HOTS taken to mean use of the potential of the mind to cope with new challenges in an environment. HOTS assist teachers to enhance learners’ thinking skills by construction of knowledge in order to think more creative, critical and innovative. Furthermore, in a classroom setting, HOTS are examined in the light of three perspectives identified by some researchers: as transfer, critical thinking and problem solving. The reimaging of students HOTS is no longer national but an international priority for education, where students can train themselves in order to face the challenges in the modern world. Thus, educating the students of the 21st century who face a complex of multiple challenges, with real-life problems often needs similar complex solutions [2].

Objectives of the Study

To establish the extent the formative assessment strategies intervention enhances S. 4 students’ HOTS academic achievement in physics when comparing pre- and post- intervention. To find out the extent formative assessment strategies intervention enhances S. 4 students’ teaching HOTS in physics when comparing the pre- and post- observation.

To find out the teacher’s experiences and reflections following the formative assessment strategies intervention and physics teaching on the students’ learning outcome.

Literature Review

3.1. Theoretical Perspective

The social Constructivist theory propounded by Vygotsky in 1978 provided the theoretical underpinnings for this study. Social constructivist theory posits that knowledge is constructed through interaction with others. The core construct of this theory is zone of proximal development (ZPD), which emphasises the role of the teacher (instructor) in an individual’s learning. The ZPD suggests that, with the help of an instructor, students are able to understand and master knowledge and skills that they would not be able to do on their own. This implies that there are activities that a student can do without help, and other activities the students cannot do without the help of the instructor. The Formative assessment’s learning theory is firmly established in who came up with the idea of a Zone of Proximal Development (ZPD) between what a student is able to do alone and what a student can do with others. Therefore, teachers and their students are actively and intentionally using formative assessment process when there is cooperative teaching and learning. That is, there is focus on learning goals; then taking note of where current work is concerning the goal; and take action to come closer to the goal. Formative assessment (FA) is a pedagogical approach to increase student learning and amplify teacher quality. Formative assessment is rare in most classrooms largely simply because teachers themselves do not know it and do not possess the necessary skills to implement it. Basically, FA process is meant to collect evidence of learning to inform instruction., every now and again during the course of study (National curriculum Development Centre, NCDC, 2021) and harboured with three major guiding questions: Where am I going? Where am I now? What strategy or strategies can help me get to where I need to go? These central questions guide both teachers and students together. According to, FA is a teacher- student learning team where evidence is paramount in making informed decisions about what to do next and choose the appropriate chance to close the learning gap and improve on student achievement [1,3-5].

3.2. Teaching of HOTS in Physics

The teaching of High Order Thinking Skills (HOTS) and assessment are informed by taxonomy as the foundation for teaching and learning. At the beginning of the 21st century, Anderson and Krathwohl introduced a revised Bloom’s taxonomy, which replaced the original three-layer hierarchical model with two dimensions: cognitive processes and knowledge. Precisely the cognitive process has six hierarchical categories: remembering, understanding, applying, analysing, evaluating, and creating. The knowledge dimension contains four types: factual, conceptual, procedural, and metacognitive knowledge. Moreover, the difference between HOT S and Low Order Thinking Skills (LOTS) has been provided. LOTS items just require recall of simple information or a simple application of known theory or knowledge of familiar situations and context. However, for HOTS items require quantitative problems or qualitative conceptual questions which not familiar to the students, that require solutions of high-level knowledge and application of known formula, they require analysis, synthesis and problem-solving and critical HOTS refer to complex thinking processes that involve conscious activities and the ability to think critically and creatively. The skills emphasised by HOTS include critical thinking, creative and innovative thinking, problem solving, and decision making. Therefore, for students, learning HOTS will strengthen their minds, guiding them in producing more alternatives, actions, and ideas. Furthermore, learning HOTS maintains the students’ critical thinking, developing many ideas, and problem solving-skills for their everyday life8 [6-11].

In physics, teachers have to be knowledgeable and skilful to select an appropriate approach to cause and assess students’ HOTS to prepare them for future success. Researchers have it that a student who has acquired a higher level of thinking can do things like analysing the facts, categorising them, manipulating them, putting them together, and finally applying them in real-life situations [13-14].

Methodology

This study used a mixed-method, sequential embedded approach with single-group, pre-test, interventions (training), post-test design. A sample of 14 physics teachers, 280 secondary students from 14 secondary schools in Kabale District, was selected and exposed to the intervention. Before the training, 2-days professional training on using formative assessment and teaching of HOTS in physics, each participating teacher was observed teaching physics as per the programme. Physics pre-tests containing HOTS were also administered to students. Scenario items (HOT items) which were valid and reliable were used. Upon training, the physics teachers were given time to plan and trial the Formative Assessment High Order Thinking Skills (FAHOTS) intervention in the classroom, and the researcher visited teachers, to coach on the job so that they can be independent when implementing FAHOTS by themselves. After two months of applying the intervention, students again were subjected to a physics post-test, with HOTS items inclusive. The two tests were administered according to the standardised testing procedures. The post-observation and interview were also done among the participating teachers.

Validity and Reliability

Validity takes different forms. It may be face, internal content or construct. In this study face and content validity were ensured. For example, measurement procedure’s possible effect on the manipulated variable’s construct validity was very important consideration. affirmed that there are several factors apart from manipulated variable that may contribute to the changes in scores. Others noted that, observed changes may be due to reactive effects, but not the manipulated variable. According to this study, the intervention in focus FAHOTS as a manipulated variable was appropriately defined, guidelines highlighted based on meaningful constructs that represent the intervention efforts. The validity of the HOTS instruments was verified by 3 experts in the field of physics. The quality of the HOTS physics questions developed was seen from the validity and reliability and item difficult index. Reliability of the HOTS instrument is also quite considered with a reliability index of 0.74 which is calculated using Cronbach Alpha. The index of difficulty level of HOTS items is in the range of -1.03 to 1.04. This indicates that all questions are included in HOTS [15].

Data Collection Instruments

6.1. Student Pre- and Post-Tests

The students’ pre –and –post assessment was used to compare the attainments realised in physics achievement after the intervention was handled. The pre- and post –Higher –order- Thinking Skills items were adapted from the Uganda National Examinations Board (UNEB) assessment modalities and sample papers. So, considering the pre-test results as a baseline, a post –test was also done after the intervention period of two months, and students were expected to answer more questions correctly based on an improvement in knowledge and understanding. Five well-functioning common items (fully moderated) from the pre-test were retained in the post- test to ensure the similarity of the two tests, the common scenario items linked the two tests and hence sets a comparison of the performance on the common scale. Each of the scenario items from each of the elements of the construct consists of open ended questions that will attract multiple perspectives from the students or problems that merited partial credit when scoring following UNEB’s proposed scoring guidelines.

Table 1: Students Test-Item and Person Reliability for Pre- and Post –Test Analysis

Table 1 summarises the pre- and post –results which imply that the assessment instruments had excellent item reliability based on the Winsteps 3.75 programme. Pre-test and post-test items could be separated into nearly the appropriate groups according to the responses of students. Accordingly, because of grouping of people between -3.00 and 0.00 logits, the scale of measure is generally low and there is spread of items from -1.6 to 2.6. Therefore, the persons reliability was marginally acceptable and the students could be categorised in two groups by the items in both assessment instruments (Ref. Table 1).

6.2. Interview and Classroom Observation for Teachers

The customised classroom observation to use for physics teachers while implementing formative assessment was adapted. In this study researcher was a non-participant and made sure that the presence of the researcher in the classroom should not interfere with the teaching and learning process. Hence the data collected could be actually from the classroom by not coming late to class, not interacting with the students, not leaving class in the middle of the lesson and maintaining a normal facial expression. The appropriate observation tool was developed by experts and adapted. This tool was organised in four features in order to measure all the necessary components of formative assessment. The tool also measures the whole process from the beginning of a lesson, what a teacher is expected to do and provide as a short introduction, in lesson development, fulfilment of the objectives, success criteria. In addition, the tool checks on the task, whether it is focused on enhancing students’ participation, also teachers questioning techniques and interaction. Also checking whether questions are calling for students’ HOTS in their work during the lesson as individuals, groups, pairs before giving them feedback. The tool also looks at how a teacher concludes the lesson. It focuses on whether a teacher completes the lesson and determines whether learning objective has been completed and checks whether assessment criteria have been met. The observation tool with all the relevant characteristics was based on the evidence collected in the lesson observation. The behaviour scores captured using the tool were later translated into an appropriate point element scales that quantify formative assessment practices and teaching of HOTS in physics. The series of lesson observation were captured at least twice per teacher in a 40- min lesson. The classroom observation was also done pre- and post. After 2 –days workshop for teachers facilitated by the researcher, teachers were also expected to implement their classroom lessons instruction within FAHOTS when teaching physics, was also post- observed.

Therefore, a comparison of both findings for pre- and post classroom observations and students HOTS achievement at different times was of vital importance to determine the level of gain and corroboration among the different findings. The comparison among the different findings. The comparison between the baseline and post data looks relevant in measuring the same person and to establish whether there is some indication of growth benefited. Furthermore, the observation tool was also aligned with the interview variables systematically in capturing the formative assessment strategies in the classroom. Therefore, S 4 Physics teachers under focus were followed purposively to get their view points concerning intervention through interviews.

Data Analysis

Data analysis involved Rasch analysis which is one of the educational techniques used. It is the best approach to analyse the quality characteristics of the student data responses. Quantitatively, Rasch analysis is ideal for determining the extent to which items belong to a single dimension and where items sit within that dimension. With this technique having established the common scale through Rasch using Win steps software, students’ physics achievements in HOT from Time 1 and Time 2 were evaluated. Qualitatively, interview data were analysed using descriptions to capture respondent views and organise them into themes. In this case questions were focused on formative assessment practices, strategies and their impacts on students’ learning and understanding of HOTS physics, techniques for involving learners in the classroom, strategies used to improve questioning, and higher- order thinking skills. All interviews transcription was done.

Ethical Consideration

Looks at ethics in research as what is or is not legitimate to do, or what moral research procedure involves. It is acknowledged that the need to obtain valid and reliable data shall oblige the researcher to seek and access information that participants may rather keep under wraps. Permission in selected schools and departments was secured from heads of the respective organisation. All physics teachers and their students were informed about the study rationale and were subjected to sign consent forms. Therefore, all ethical issues like informed consent, privacy of all respondents and confidentiality were all observed in pre- and post-tests.

Results

Comparison of Pre- and Post –Testing of the Achievement Findings of Students in Physics

Table 2 shows participants’ counts and average logits physics achievement at different times. According to Table 2, overall the students in the pre-intervention test (n = 280) obtained a lower mean value (-1.61 logit) than when participated in the post- intervention (n = 280 test (-0.61 logit). According to investigation, Table 2 reveals that all schools had slightly performed better during post- test than pre-test. Further analysis shows that students’ physics achievement findings recorded a significantly higher gain of average achievement values from the pre-test (1.61 logit, SD =1.59) to post-test (- 0.61 logit, SD =1.21), z =11.40, p = 0.000, with the large effect size (d = 3.0). All the schools significantly shown a great improvement from pre-test to post-test results, except for school (525), which was not significantly gained (p = 0.150). Further, the effect size for all statistically significant schools show great improvements in students’ achievement similar even to the overall improvement.

Table 2: Students’ Physics Achievement Scores Analysis for The Pre – and Post Test

Classroom Observation in Pre and Post- Findings

10.1. Teachers’ Use of Learning Goals, Success Criteria, Questioning and Learning Tasks

The number of times for teachers in using learning goal and success criteria, questioning and interaction, leaning (HOTS) tasks and feedback is taken to be effective in enhancing students, performance. According to this study, after the professional development, (FAHOTS) teachers were expected to apply these elements more frequently during their lessons than before the intervention. The researcher noted that, the proportion of using learning goals and success criteria has to some extent changed when comparing the pre-observation to post-observation. Evidence have it that some teachers were seen discussing the learning goals and success criteria that were either presented orally or written at chalk board which is a positive practice in the formative assessment. This practice properly followed enhances students’ understanding of lesson direction from the beginning of a lesson up to the end. However, because of some limitations of some resources like charts or handouts, observed teachers were seen neither during the pre-observation nor during post-observation using learning goals and success criteria.

Regarding their use of questioning, the results of the pre-observation indicate that many teachers were classified under ineffective use of questions an indication that teacher still use traditional approaches where teachers basically focus on the students who raise their hands in an attempt to respond to a question. As for learning tasks for pre and post scores, results reveal that none of the teachers applied the strategy in pre-observation while only one teacher applied learning tasks during post-observation. However, participating teachers did not differ significantly in their use of learning tasks (HOTS), indicating insufficient evidence for its beneficial in the classroom.

10.2. Peer and Self-Assessment Pre and Post- Observation

The frequency and use of proper feedback within the FAHOTS during pre-and post-observation was carried out. Therefore, it is concluded that with oral feedback on improvement was realised in a post –observation. Furthermore, after giving the learners feedback some classwork to do, a number of teachers participated in confirming how students were doing it compared to the pre observation handling. A number of teachers were seen participating in checking students’ responses, providing some comment when compared with the pre-observation. Peer assessment strategy was carried out however challenges cropped up in the process of assessment. However, some students were allowed to check for their friends work and also getting guidance on how they conduct peer assessment implemented during post-assessment. Self assessment for pre- and post-assessment was also handled. It was observed that teachers were seen; reminding students on how to use self-assessment processes, rules, how students are supposed to use success criteria, while checking their work and finally confirming how they conduct the self-assessment with ease.

10.3. Physics Exercises and The Type of Feed Back for Pre- and Post Observation

Books of learners were sampled for lower and higher-performing students after lesson observation. It was later found out that teachers increase of HOTS exercises was a desirable change for classroom practice and exposed students to different skills like problem –solving activities to activate their thinking. In addition, it was also noted that a shift of fewer LOTS and more HOTS was also observed. It is concluded that with the acquisition of multiple skills will result in explanation, decisions, and products that are valid in view of promoting continued growth in high-order thinking and other intellectual skill.

Discussion

According to objective 1 seeking to determine the extent to which the intervention has enhanced senior 4 students’ HOTS academic achievement in physics when comparing the pre- and post-intervention. The physics achievement findings revealed significantly different scores, which showed how much students gained as far as learning was concerned. It is noted that learners gained in the field of HOTS skills. The greater improvement in students learning can be associated with the intervention. The major positive gains in students learning can be associated with the invention. The magnitude of the expected growth, an effect size greater than hinge-point in the case of the intervention. This evidence corroborates the observational findings that teachers indeed did something different, and it was a sign of change in the classroom teaching and learning of physics.

It is noted that the results of the study showed a statistically significant gain that was observed for student’s achievement in physics in the tests that constituted of HOTs scenario items. The f indings of this study were in line with Babincakova, which reveals that the introduction formative assessment classroom techniques (FACTS) in the classroom to have significantly increased both students lower- and higher-order cognitive skills and students showed a positive attitude towards the introduced method. The results of the study got, corroborate the study by, who found positive impact of the five key strategies on students, achievement in physics instructions. According to classroom observation, secondary school teachers’ exposure to the FAHOTs intervention, has enhanced their teaching of HOTS in physics when comparing the pre- and post-observation. The classroom observation have it that teachers rarely employed FAHOTS. However, that commonly used strategies were, learning goals, effective classroom discussions on questioning, learning is an important experience to the working /active teachers, ignoring the initial approach of teaching to the test. The record books also disclosed that there are significant HOTS tasks compared from pre- observation to post – observation. The investigations indicate that all teachers who participated fully may have some HOTS learning tasks ready always as taught during the work shop. It is important to note that, it was also realised that a good development is when a change in students’ work output were assisted with the appropriate feedback to improve learning. In addition, the study’s findings demonstrated some limited evidence that teachers who participated had on FA, but the observed significant change in students work shows an impact on the transformation of the teaching and learning of physics, particularly the HOTS.

Experiences and Reflections of Physics Teachers

Objective three looks at the participating teachers’ experiences following the FAHOTS intervention and physics teaching on the students’ learning outcomes. Teachers revealed some positive experiences with the FAHOTS intervention during interview f indings. The participants asserts that they had implemented two to three of the formative assessment strategies. The interview f indings corroborate the classroom observations and confirmed that teachers have employed FAHOTS especially in using learning objectives, effective classroom discussions using some questions, and learning tasks to bring out evidence of learning HOTS concepts. With all the positive developments observed in teaching and learning is an important experience for the participating teachers to suggest to others that teaching to test should be abandoned. The findings are consistent with Li, et al., (2030) who identified teacher training in professional development learning communities, where schools and government are support the implementation of formative assessment as a means of improving students’ performance. The teachers who attended the training said that they were in favour of the FAHOTS intervention, as revealed in the interview findings. The teachers support for in-service training of the FAHOTS fits well with Secondary Science and Mathematics Teachers (SESEMAT) programme, a government programme proposed teacher training development in the school based assessment. Results from teachers’ reflections following the FAHOTS intervention after interviews and physics teaching of students’ HOTS were positive. The participating teachers suggested that such professional development should be practiced even in other classes or levels. Across different secondary levels. This result indicates that FAHOTS intervention could be used as a strategy to enhance teaching and learning of HOTS. The results are in-line with findings where an in-depth handling is exercised, when provided with specific information about FA through staff development, teachers become more empowered and positive towards such assessment, and their implementation skills improved greatly. The findings in this study were similar to the positive factors highlighted by. These positive factors include teacher’s workload is lessened, improved critical and problem solving, student values, attitudes and interests is raised and both teachers and students love using formative assessment strategies [16].

Conclusion

The findings showed that intervention could improve teaching and learning, as teachers’ teaching and attitudes improved. There was evidence of patterns of change as the teachers started to use some new strategies in teaching physics, though written feedback remained a challenge and it was just evaluative. However, student’s achievement, pre- and post-achievement gains in physical HOTS items reflected to some extent the quality of instructional strategies that the teachers employed in the classroom. Most importantly, it has to be considered that physics emphasises students’ ability to develop and apply scientific thinking to solve a range of problems in everyday situations, which has improved after the intervention. The participating teachers’ reflections supported findings that rolling out the formative assessment is a real sign of reimaging educational assessment in this era of multiple dimensions of learning and professional development [17-23].

References

1. Moss, C. M., & Brookhart, S. M. (2019). Advancing formative assessment in every classroom: A guide for instructional leaders. ASCD.

2. Sewell, a. (2023). Higher-order thinking skills. Retrieved from https://www.structural-learning.com/post/higher-order thinking-skills.

3. Vygotsky, l. (1978). Interaction between learning and development. Readings dev. Child. 23 (3), 34

4. Lonka international, k. (2018). Phenomenal learning from f inland. Edita, finland: edita publishing.

5. Moss, C. M., & Brookhart, S. M. (2019). Advancing formative assessment in every classroom: A guide for instructional leaders. ASCD.

6. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of educational research, 77(1), 81-112..

7. Heritage, m. (2010). Formative assessment: making it happen in the classroom. Thousand oaks, ca: corwin press.

8. Brown, G. T., & Hattie, J. (2012). The benefits of regular standardized assessment in childhood education: Guiding improved instruction and learning. In Contemporary debates in childhood education and development (pp. 287-292). Routledge.

9. Qasrawi, R., & BeniAbdelrahman, A. (2020). The Higher and Lower-Order Thinking Skills (HOTS and LOTS) in Unlock English Textbooks (1st and 2nd Editions) Based on Bloom's Taxonomy: An Analysis Study. International Online Journal of Education and Teaching, 7(3), 744-758.

10. Babincáková, M., Ganajová, M., Sotáková, I., & Bernard, P. (2020). Influence of Formative Assessment Classroom Techniques (FACTs) on Students' Outcomes in Chemistry at Secondary School. Journal of Baltic science education, 19(1), 36-49.

11. Mohamed, R., & Lebar, O. (2017). Authentic assessment in assessing higher order thinking skills. International Journal of Academic Research in Business and Social Sciences, 7(2), 466-476.

12. Ganapathy, M., Singh, M., Kaur, M., Kaur, S., & Kit, L. W. (2017). Promoting Higher Order Thinking Skills via Teaching Practices. 3L: Southeast Asian Journal of English Language Studies, 23(1).

13. Yee, m. H., jailani, m. Y., widad, o., razali, h., and tee, t. K. (2011). “recent advances in educational technologies.” In the effectiveness of higher order thinking skills for generating idea among technical students.

14. Balakrishnan, M., Nadarajah, G. M., Vellasamy, S., & George, E. G. W. (2016). Enhancement of higher order thinking skills among teacher trainers by fun game learning approach. International Journal of Educational and Pedagogical Sciences, 10(12), 3954-58.

15. Kerlinger, f. N. And lee, h. B. (2000). Foundations of behavioral research (orlando, fl: harcourt college publishers).

16. Magaji, A., & Ade-Ojo, G. (2023). Trainee teachers' classroom assessment practices: Towards evaluating trainee teachers’ learning experience in a teacher education program. Social Sciences & Humanities Open, 7(1), 100467.

17. Brookhart, s. M. (2006). Formative assessment strategies for every classroom: an ascd action tool. Alexandria, va: ascd.

18. Brookhart, S. M. (2010). How to assess higher-order thinking skills in your classroom. Ascd.

19. Li, Z., Yan, Z., Chan, K. K. Y., Zhan, Y., & Guo, W. Y. (2023). The role of a professional development program in improving primary teachers’ formative assessment literacy. Teacher Development, 27(4), 447-467.

20. Ministry of Education and Sports (2006). Teaching guidelines for science and mathematics teachers in secondary schools. ISBN 9970 575-00-7. SESEMAT project, Uganda

21. Ncdc. (2021). Assessment framework for the new lower secondary education curriculum (nlsc).

22. Neuman, w. L. (6th ed.). (2006). Social research methods qualitative and quantitative approaches. Boston: pearson education.

23. Pambuko, Z. B., Setiyo, M., Praja, C. B. E., Setiawan, A., Yuliastuti, F., Muliawanti, L., & Dewi, V. S. (Eds.). (2023). Proceedings of the 4th Borobudur International Symposium on Humanities and Social Science 2022 (BIS-HSS 2022) (Vol. 778). Springer Nature.