Improving short answer written responses in the Science classroom using targeted literacy strategies

Nonie Taylor Physics, Earth and Environmental Science, Mathematics, and iSTEAM teacher

A Abstract

Five literacy strategies were embedded into a Year 9 mixed-ability science classroom and the impact was evaluated through a variety of observations and student artifacts. Evidence included the analysis of pre- and post-Inquiry student samples of ‘describe’ and ‘explain’ questions and conclusions for similar practical exams, as well as overall performance scores, teacher observations of student engagement and student feedback. While the Inquiry did not demonstrate a statistically significant improvement in performance of short answer responses, it did show some positives from the intervention – both for the students, and for the teacher. The benefits did not seem to be the same for all groupings within the classroom – for example, the foundation group of students showed the greatest improvement in overall content knowledge, whereas the extension group showed the greatest improvement in literacy skills. Overall, the students reported increased confidence in the subject, increased ability to understand and increased understanding of the terminology. Students also felt that they were receiving feedback which could help them to improve.

This research was completed as part of the Inquiry pathway submission for accreditation as Experienced Teacher with the AISNSW and ISTAA.

Literature Review

Literacy is a key component of the Science 7 - 10 Syllabus and has also been identified as a Cross Curriculum General Capability - meaning that there is an expectation that it will be taught across all subjects. ‘Scientific Literacy’ is a slightly different concept – is not limited to language skills, but rather to making meaning from scientific information and processes. This inquiry is focuses more on literacy in the general sense as it relates to the amount of new terminology students encounter in the science classroom and the requirement to be able express their understanding in the written form.

Observations of Year 9 mixed ability students’ work revealed that many of the students' responses to short answer questions - especially 'describe' and 'explain' questions, and in writing conclusions - lacked the required terminology, level of detail, links to the science

learned in class, conventions, and relationships between ideas, which is why these types of responses became the key focus of this inquiry

Five literacy strategies were selected to be implemented in the 2022 Science classroom.

V Vocabulary Training Henry (2021) presented on ‘High Yield Oral Strategies for Improving Literacy’, where she outlined the benefits of prioritising oral language interactions in the classroom. Oral language interactions give students the opportunity to rehearse new vocabulary and sentence structure and receive feedback before they need to write anything down. This ‘low stakes’ use of new terminology builds confidence in students.

The first strategy that Henry presented was explicit vocabulary instruction. This involved the teacher clearly introducing new terminology, along with a visual presentation, including definition, use in a sentence, synonyms, antonyms and etymology to give students an understanding what the term means. During this time, the teacher repeated the new vocabulary multiple times, including variations of the words also used in science. The teacher then showed an image with some other key words and modelled how to use the image and the words as prompts to use the new terminology in a sentence. She then called on students to try with a couple of other examples, supporting the students with further verbal prompts if required.

A Annotating the Passage and Expanding Kernel Sentences Another strategy Henry presented to build on the vocabulary training was modelling annotating a passage related to the new terminology. The teacher called on different students to read a sentence at a time, and modelled on the board underlining key terms. She encouraged students to do the same, and used questioning to check for understanding from the students. For any unfamiliar words, she encouraged students to write definitions in the margins. Following this, she provided students with some very short sentences related to the content in the passage. She then asked students to add detail to the sentences, based on the information provided in the passage.

C CUBE the Question CUBES is an acronym for ‘Circle the numbers, Underline the question, Box the key words, Evaluate and Solve’ (Lempke, 2020) and is typically used for solving mathematics word problems. It gives students a tool to help them decode what a question is asking. A version of this is suitable for science short answer questions – CUBE (Circle the verb, Underline the key/topic words, Box the important information e.g. plurals and additions, and Elaborate /Evidence/ Examples). This strategy helps students to determine what a question is asking for and how much detail it requires. The teacher’s version of this strategy also included prompts for students to plan their answer; writing down the key terminology they needed to mention or define; and then to write their answer by first rephrasing the question. As students become used to the process, scaffolds and prompts are removed.

When students are not confident what the question is asking, they tend to write down anything that they think is remotely relevant and miss the point of including the required detail of the question. This strategy was selected in order to give students a framework with

which to approach short answer questions, and to teach them the conventions involved. It also helped students to identify key terminology required and whether they have to make connections between ideas.

S Spaced Retrieval In Morkunas’ (2020) article, learning is defined as a ‘change in long-term memory’. Spaced retrieval is one method of helping students retain their learning by building stronger connections in their brains.

Spaced practice is the technique of giving students a number of opportunities on different days to review information, helping them to commit it to long-term memory. Retrieval practice is bringing the information from long-term memory into working memory. This strengthens the memory, making it less likely to be forgotten.

Spaced retrieval was also described by Cooney Horvath (2021) as being key to committing information to long-term memory. Cooney Horvath also stressed the importance of laying a solid foundation of content before students can build on their concepts. Through spaced retrieval, students are given opportunities to strengthen their understanding of the science specific terminology in class, laying the foundation to be able to build the scientific concepts in their minds.

This strategy was selected in order to strengthen students’ memory of key scientific terminology so that they could form appropriate concepts in their minds and then express them in their short answer written responses. It also gave them opportunities to practice using the terminology orally in a low-stakes scenario (in pairs rather than in front of the class) to make them more familiar with it. This was typically implemented in the form of starter routines such as flashcards and questioning, as well as separating the theory and practical lessons for a particular knowledge outcome.

P Peer Modelling and Evaluation Geithner and Pollastro (2016) referred specific study done on the impact on scientific literacy and writing skills of engaging in peer review and receiving peer feedback. Students involved in their study had their work reviewed by their peers and had the opportunity to resubmit their revisions. Through the study it was demonstrated that peer-evaluation made students better able to assess and analyse their own work, and also gave them exposure to a range of samples of work so that they could learn to identify what the key features of the ‘good’ samples of work were. It also made the students in the study more open to receiving feedback and seeing the value of it in moving them forward in their understanding.

This strategy was selected as it gave students exposure to ‘good’ samples of work, and also modelled the criteria being looked for in various types of short answer questions. It was also chosen for its potential to improve the writing of all students in the class – those who struggled with written responses would learn to identify the key components which make a good response, while the stronger students would benefit from the positive reinforcement of having their work modelled.

Methodology Subjects A mixed ability Year 9 Science class was selected for this investigation, made up of 22 students. These consisted of 2 ‘extension’ students requiring more challenging work, 17 as ‘core’ or standard level ability, and 3 as ‘foundation’ requiring additional assistance and adjusted learning activities. In addition, a number of the students were classified as “disruptive” for analysis in this paper. Analysis across these classifications can help teachers predict the effect of these literacy strategies in their own class context.

T The Intervention Throughout the module on Energy (in all, a period of 8 weeks including approximately 24 hours of lessons), all five of the literacy strategies described in the literature review were explicitly used to support student growth. While some lessons implemented one particular strategy, most involved multiple strategies combined using the teacher’s discretion to best present the lesson.

D Data Collection

A range of academic data, engagement and participation notes, and student feedback was collected for the purposes of this inquiry.

A. Content Knowledge

Similar to typical academic data, overall performance was assessed with a pre-test (before the intervention) on Chemistry content and a post-test (after the intervention) on Energy content. While the content was different, and therefore this isn’t testing whether literacy strategies improve content knowledge, it was seeing if a targeted focus on literacy strategies would have noticeable effects on overall performance, especially when compared to a class without the targeted focus on literacy strategies.

To compare pre- and post-intervention overall performance, the normalised gain (the amount students improved divided by the amount they could have improved) was calculated for each student and then the gain of all students in the class were averaged.

B. Literacy

One of the working scientifically skills identified in the science curriculum is the ability to communicate scientifically. An important part of this is being able to communicate ideas in short answer responses. The following conventions are expected and were measured before the intervention (Chemistry) and after the intervention (Energy):

a. Students need to have an understanding of what the verb in the question means. For consistency, the definitions of these verbs used in the NSW Education Standard

Authority (NESA)’s ‘Glossary of Key Words’ (2019) was used. Here ‘describe’ means ‘to provide characteristics and features’, and ‘explain’ means ‘to relate cause and effect; make the relationships between things evident; provide why and/or how’, features of something.

b. Scientific conclusions in an investigation must always relate to the aim or the hypothesis and they must always include the data that the student has collected from their experiment.

c. In short answer responses, students need to be able to correctly recall and use the specific scientific terminology which they have learned in class. When marking short answer responses, it is often the terminology for which marks are allocated.

d. Another skill for short answer responses is being able to make connections between scientific ideas and observations that the student makes. Often students can recall some of the scientific reasoning but not apply it to the specific context of the question.

To compare pre- and post-intervention literacy performance, samples of student responses to ‘describe’ and ‘explain’ questions, as well as scientific conclusions were evaluated using rubrics developed to assess each of the expected conventions. A generalised form of the rubric for ‘describe’ and ‘explain’ questions is shown in Figure 1. The rubric for scientific conclusions is shown in Figure 2.

Each response was marked for each standard, with 1 mark awarded for emerging, 2 for approaching, 3 for meeting and 4 for exceeding, giving a maximum score of 16 for ‘describe’ questions and 20 for ‘explain’ questions and conclusions. The normalised gain was calculated for each student and then the gain of all students in the class were averaged.

Figure 1: Generalised Rubric for marking ‘Describe’ and ‘Explain’ Questions.

Figure 2: Generalised Rubric for marking Scientific Conclusions.

C. Student Engagement During lessons when the specific literacy teaching strategies were implemented, notes were compiled by the teacher in relation to engagement and completion of the task. Engagement and completion were rated for each student as ‘below’, ‘meeting’ or ‘exceeding’ expectations in both areas, and these grades were converted to a mark of 1, 2, or 3 to quantify student engagement with various teaching activities.

D. Student feedback

Two types of feedback were collected - specific feedback on lessons/activities via surveys, and general feedback on teaching and learning in our class pre- and post-implementation of the Inquiry program. Surveys were collected for both classes allowing an understanding of the student perspective on the instruction.

Results & Analysis A. Content Knowledge The students’ overall exam test scores for the Energy practical exam (post Inquiry program) were similar to those for the Chemistry prac exam (pre-Inquiry program), with an average normalised gain of -0.18 (which means that on average, students’ marks were lower in the Energy exam). This value was the lowest in the foundation group of students (-0.07).

Initially, this result is unimpressive. However, the Inquiry unit of work contained far more unfamiliar content, new terminology and more difficult concepts than the pre-Inquiry unit of work. The smaller drop in scores in the foundation group of students suggests that the targeted literacy strategies were most effective in promoting the learning of content knowledge in this group compared with the Core and Extension groups. If the disruptive students’ scores (average -0.62) were removed from the core group, then the core group had a normalised gain of 0.02, the only group to have improved their marks in the final assessment. This is noteworthy, as I mentioned above, because the post-Inquiry assessment was harder in terms of content. It is likely that the literacy strategies didn’t increase understanding of the content of the disruptive students as they were less likely engage with them.

B. Literacy All groups within the test class showed a positive average normalised gain in their short answer detailed scores between the pre- and post- assessments. The greatest gain was in the extension group of students (0.13), followed by the core students minus the disruptive students (0.04). The disruptive students and foundation groups only made small gains (0.02 and 0.01).

The results demonstrated that all groups had improved their short answer responses, despite the fact that the content was more difficult than in the pre-inquiry assessment. For the students in the extension group, they had been able to apply the structures and conventions they had been explicitly taught to improve the quality of their answers. While the core and foundation groups did not show such a marked improvement, recall that the foundation group did improve their content knowledge as shown in the section above. This may reflect the that for those students, there is still a discernable gap between being able to understand the content and then being able to express that it the written form. It also may suggest that the eight-week timeframe of the study was insufficient for these students to acquire this skill.

C. Student Engagement Students on average scored the highest in engagement in the Resistance lesson (average engagement score 2.11 out of 3) where the strategies of vocabulary training, annotating the passage, and expanding kernel sentences were used; followed by the Heat Transfer lesson where CUBE your question and peer evaluation were used (average engagement score 2.10 out of 3). In these lessons there was a high level of teacher interaction and support, students were handwriting responses on paper, and the teacher circulated the room and gave

immediate feedback. These lessons also involved a range of activities to break the lessons up. The additional scaffolding made it possible for even the lower ability students to engage.

Students also engaged well with the spaced retrieval starter activities using flashcards. This strategy engaged students because it meant that they did not need to wait for others or for the lesson to start, and it gave them the opportunity for social interactions as they quizzed each other in a low-stakes scenario. Students also had the expectation that the teacher could call on them for a response at the end of the activity, and so there was an impetus for them to engage.

The way that the unit was sequenced to promote student engagement – it started by activating prior learning and structuring the content in the context of models, moving from somewhat familiar content to new ideas and concepts. Selected literacy strategies were used to build content knowledge as well as to improve short answer responses, and also as a way to provide spaced retrieval at a later date. Unfortunately, student engagement was somewhat negatively impacted on by the timing of the Inquiry - it was broken up by holidays, and camps of varying durations for different students. This did however create opportunities to return to previous content and including spaced retrieval as a way of refreshing students' understanding.

D. Student Feedback

The feedback provided from students after a range of the targeted literacy activities was useful as it identified some areas for improvement. They found the heat transfer lesson which incorporated the targeted literacy strategies 'CUBE your question' and 'peer evaluation' the most effective of the lessons surveyed, with students rating it 3.75 out of 5 starts for how useful they thought it was. Within that lesson, they felt that peer evaluation of other student work samples in group was more helpful to improving their responses than the CUBE strategy. Students also found expanding kernel sentences - where they had to intentionally look to add detail to their responses - valuable in the Convection and Resistance lessons. They also found the spaced retrieval through the flashcards beneficial in helping them to remember terminology and then be able to use it in their short answer responses.

The peer evaluation activity involved students writing, and then reading and critiquing, which students found more helpful than ‘listening to other students’ responses’ (as they did in the Convection lesson). This feedback suggests that reading and discussing other students' responses in groups may have a greater impact on improving short answer responses than merely listening to other students’ responses.

The Teaching and Learning Surveys for the test class showed gains pre- and post-inquiry in the areas of student confidence in the subject, organisation of learning, understanding of terminology, skills, feeling like they were getting feedback that helped them to improve, and that there were high expectations of them and expectations that they will reflect and act on feedback.

It was encouraging to see students reporting this increase in confidence through something as simple as targeting literacy. The types of activities implemented included a lot of verbal feedback opportunities in them. This feedback has affirmed that giving students explicit

literacy training gives them more confidence in the learning, as they know what is expected of them (in their short answer responses) and they can think more critically about their own responses after practicing on others in peer evaluation.

Discussion

The Inquiry Question was: ‘What impact does using targeted literacy strategies have on a Year 9 mixed ability students’ performance in short answer responses in the Science classroom?’. This was evaluated through the analysis of pre- and post-Inquiry student samples of ‘describe’ and ‘explain’ questions and conclusions for similar practical exams, as well as overall performance scores, student observations of engagement and student feedback.

While the Inquiry did not demonstrate a statistically significant improvement in performance of short answer responses, it did show some positives from the intervention – both for the students, and for the teacher. The benefits did not seem to be the same for all groupings within the classroom – for example, the foundation group of students showed the greatest improvement in overall content knowledge, whereas the extension group showed the greatest improvement in literacy skills. Overall, students reported increased confidence in the subject, increased ability to understand and increased understanding of the terminology. Students also felt that they were receiving feedback which could help them to improve.

The inquiry showed the teacher that teaching literacy does not need to be at the expense of time spent covering content – but in fact literacy activities could be one of the tools by which content can be taught, revisited, and consolidated. The inquiry also fostered improved teacher routines in monitoring student engagement and providing immediate feedback in the classroom.

The student data did not show improvements in student writing of conclusions. This could be because of the difference in level of complexity of the conclusion question in the postInquiry task, but possibly also due to the types of literacy strategies used and the focus the teacher placed on 'describe' and 'explain' questions by comparison. As students embraced peer evaluation activities well, this may be an effective strategy to use when working on improving conclusions.

Future Research

This project was conducted over a short period of time and with a small group of students, so it is difficult to make generalisations about how foundation, core and extension groups of students benefitted overall. To improve the rigor of the study, it could be extended over a longer period of time and across a larger group of students. It would be interesting to see if a longer timeframe would help the foundation students not just improve their content knowledge, but also be able to improve their written communication.

This Inquiry project opens up the question of what other strategies would be useful, and the value of one strategy over another or different combinations of strategies. It would also be interesting to see if different strategies were more effective for different ability groups.

The inquiry has raised discussion within the Science department about how teachers can be more intentional about embedding literacy activities into programming, and using Progressive Reporting as a way of capturing student progress in short answer written responses across the middle school. The targeted question types could follow a progression from lower order ‘describe’ and ‘outline’ in Year 7 through to higher order verbs such as ‘assess’, ‘evaluate’, ‘justify’ in the later years.

References

Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman. Cooney Horvath, J 2021, ‘The Science of Learning’, Barker College Professional Learning, Barker College, Waitara, NSW, 19 April. Cooney Horvath, J 2022, ‘The Cognitive Science of Routines and Habits’, Barker College Professional Learning, Barker College, Waitara, 26 April. Geithner, C.A. and Pollastro, A.N. (2016). Doing peer review and receiving feedback: impact on scientific literacy and writing skills. Advances in Physiology Education, 40(1), pp.38–46. Henry, M (2021). ‘High Yield Oral Language Strategies for Improving Literacy’, Learning Disabilities Australia High Impact Strategies for Supporting Secondary Students, Online Conference Sydney, 30 May. Kesty, S 2018, Supporting Literacy in the Science Classroom, Edutopia, George Lucas Educational Foundation. Lempke, I. (2020). Strategies to Develop Effective Problem Solving Habits for English Strategies to Develop Effective Problem Solving Habits for English Learners in a Problem-Based Learning Classroom Learners in a Problem-Based Learning Classroom. [online] Available at: https://digitalcommons.hamline.edu/cgi/viewcontent.cgi?article=5491&context=hse_all [Accessed 6 May 2021]. Morkunas, D. (2020). Spaced, interleaved and retrieval practice: The principles underlying the Daily Review. Learning Disabilities Australia Bulletin, [online] 52(3), pp.20–22. Available at: https://www.ldaustralia.org/app/uploads/2021/03/Morkunas-Spaced-interleaved-and-retrievalpractice.pdf [Accessed 6 May 2021]. NSW Education Standards Authority (NESA) (2019). Glossary of Key Words. [online] Nsw.edu.au. Available at: https://educationstandards.nsw.edu.au/wps/portal/nesa/11-12/hsc/hsc-studentguide/glossary-keywords [Accessed 6 May 2021].