Assessing Greek pre-service primary teachers ’ scientific literacy

ABSTRACT

Specifically, Fives et al. (2014, p. 550), define SL as the "ability to understand scientific processes and to engage meaningfully with scientific information available in daily life," giving emphasis on processes that allow "individuals to engage with science in practical and meaningful ways within daily life" (McKeown, 2017, p. 13). Moreover, Fives et al. (2014, p. 555) suggest that being scientifically literate requires more than knowledge;

Research Article
"one must also have the motivation and beliefs necessary to engage that knowledge when needed as part of one's daily life." Therefore, motivational beliefs about science are equally important pre-requisites for SL.
To achieve the goal of a scientifically literate society significantly depends on teachers who "are the most important element in implementing educational reform" (Sultan at el., 2021, p. 2). In fact, a key feature in literacy development is scientific knowledge (Roberts, 2007) and seeing that scientific knowledge starts to develop from childhood, it is needed to cultivate SL from the first years of primary education (Fragkiadaki et al., 2022;Kähler et al., 2020;Sargioti & Emvalotis, 2020). Therefore, it becomes imperative that primary school teachers are equipped with skills and knowledge that will allow them to powerfully engage in their students' SL skills development (Sargioti & Emvalotis, 2020). However, it has been suggested that teachers are usually unwilling to "assume the responsibility for teaching literacy within their science classes" (Drew & Thomas, 2018, p. 2).
From an empirical perspective, research on pre-service primary teachers is limited (Sultan at el., 2021), which adds to the challenge of measuring their extent of science knowledge and skills. For instance, Sultan at el. (2021) assessed the USA-based pre-service primary teachers' conceptual understanding of terms such as SL, nature of science (NOS), and science-technology-society (STS) employing a qualitative interview design with semi-structured interviews. Their findings revealed that SL and NOS need improvement in contrast to STS where pre-service teachers indicated sufficient understanding. In Indonesia, findings from administering tests, researchers' observations and interviews showed that science teachers' levels of SL were not satisfactory (Rubini et al., 2017) In Turkey, Bacanak and Gokdere (2009) administered a multiple-choice test of 35 items to assess preservice primary teachers' levels of SL. Questions tested knowledge of physical science, life and earth science, scientist properties, NOS, science and technology, and the social perspective of science. The results demonstrated moderate SL levels among pre-service teachers with higher scores on NOS and scientist properties and lower scores on science and technology. No gender differences were found. Altun-Yalcin et al. (2011) investigated whether a pre-service science teacher's undergraduate year or gender significantly affected their SL levels. The results only revealed an effect of students' undergraduate year of study on SL levels. Similarly, Karamustafaoglu et al. (2013) investigated the SL levels of pre-service teachers in their final two years of undergraduate studies. Despite the added academic experience, the results showed that preservice teachers' levels of SL were low. Likewise, Ozdemir's (2010) results highlighted low levels of SL among pre-service science teachers.
In Taiwan, Chin (2005) used the test of basic scientific literacy (TBSL) instrument to investigate the level of SL of first-year pre-service teachers in colleges. The selected items of SL were science content, the interaction between STS, NOS, and attitudes toward science. The statistical analysis revealed that the SL level of the Taiwanese sample was overall satisfactory. However, female pre-service science teachers scored lower than their male counterparts in earth and life science, science content and the TBSL in general. Cavas et al. (2013) also administered the TBSL to pre-service primary science teachers to investigate their SL levels taking into consideration variables such as gender and number of undergraduate study years in Turkey. The results demonstrated that pre-service teachers possess a sufficient level of SL. In terms of gender and undergraduate year, females and 4 th year students performed better than males and 1 st year students in the TBSL respectively. From a cross-cultural perspective, in the USA for example, it seems that the SL levels of preservice primary teachers are satisfactory (Sultan et al., 2018) and, in the Philippines, 4 th year pre-service secondary science teachers showed satisfactory SL levels (Flores, 2019).
Finally, in Greece, Sargioti and Emvalotis (2020) investigated pre-service teachers' attitudes towards science and examined the influence of beliefs about science on SL. The research tool was a revised version of the questionnaire used in PISA (OECD, 2016) which highlighted five indices of SL (enjoyment of science, the engagement in scientific activities, epistemic beliefs, views on general value of science, views on the activities that contribute to changing ideas). The results showed that females and students with a social sciences/humanities background in high school were more likely to enjoy participation in science than males and students with a science and technology background. Overall males were more scientific literate than females.
In this context, this paper aims to validate the scientific literacy assessment (SLA) tool to investigate the levels of SL of pre-service primary teachers in Greece.

Purpose of Research: Research Questions
The purpose of this research study is to validate SLA in Greek and investigate the levels of SL of pre-service primary teachers in Greece. More specifically, the study's objective is five-fold and sets out to investigate: (1) the multidimensionality of the measurement model for SL, (2) the levels of demonstrated SL, (3) differences in scores on the two components of the SLA according to gender, undergraduate year, and high school course specialization, (4) associations between the two components of the SLA and gender, undergraduate year, and high school academic specialization, and (5) associations between the two components of the SLA.

Research Design
In this research design, a questionnaire was used to collect data and draw conclusions on the SL of preservice primary teachers. The first step was to translate the tool into Greek. All questions and items were translated into Greek for content and conceptual equivalence, according to the International Test Commission guidelines for trial adaptation (Hambleton, 2001). The original SLA version was translated into Greek by two bilingual speakers and then one other bilingual speaker back translated the target SLA into English. Minor vocabulary adjustments were made following translation differences. Also, in order to ensure the degree to which the questionnaire accurately measures what it intends to measure, each item was examined by two postdoc researchers and a professor well acquainted with the literature and experienced in the research field in the Department of Primary Education, to determine the validity of the content and cultural relevance of the questionnaire. The translated questionnaire was tested on 30 students of the Department of Primary Education to verify the following elements: participants' interest, question deficiencies or errors, time required to complete the questionnaire, difficulties in understanding its content, the wording of questions, terms or concepts that were either unknown or misunderstood by the participants. For both components of the questionnaire (SLA-D1 and SLA-MB) there were no indications of deficiencies or errors or concepts that were difficult to understand. The time needed to complete the instrument was approximately thirty to forty minutes.

Participants
A random sampling method selected 362 students (303 females and 59 males) enrolled in the Department of Primary Education at the University of Ioannina. 83.7% were women, and 72.9% had selected the humanities/social sciences specialization course in high school, 18.8% had focused on the natural sciences, and 7.5% chose the technology course route. Regarding undergraduate year, 31.2% of participants were enrolled in their first year of undergraduate studies, 24.6% were in their second year, 22.7% in their third year and 21.5% in their fourth year. All questions were mandatory, and students had to submit a response to every question. There was no time limit assigned to completing the SLA in the main study.

Research Instrument
Participants were asked to complete the questionnaire developed by Fives et al. (2014), to draw conclusions on their SL.
This questionnaire was specifically selected because it does not "tend to be field/discipline specific," assesses "students' motivation for and beliefs about science" (Fives et al., 2014, p. 557), and uses mathematics as "working knowledge" in science (Fives et al., 2014, p. 555). Additionally, this tool was developed to assess middle school students' (ages 11-14) SL, not university students per se. Finally, it has not been used in Greece to date. It consists of two components. The first component (SLA-D) assesses SL through 26 multiple-choice questions based on specific examples from everyday situations. In the present study, the SLA-D1 version was used (Fives et al., 2014). The second component (SLA-MB) evaluates students' motivational beliefs about science. It consists of 25 items scored on a five-point Likert scale and is divided into three categories: value of science, SL self-efficacy, and personal epistemology.

Data Analysis
Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) confirmed the goodness of fit of the instrument. The internal consistency reliability was checked by Cronbach's alpha coefficient. A descriptive analysis of responses to the multiple-choice questions of the SLA-D1 and the Likert scale questions of the SLA-MB revealed the performance of participants. Specific statistical indicators were calculated (average, frequency, standard deviation, percentages, etc.) and the appropriate diagrams and tables were created for visual representation. In addition, a statistical significance means test was performed to examine differences among overall scores and responses, taking into consideration the demographic characteristics of participants. Finally, a correlation analysis investigated the relationships between the SLA's components and demographic characteristics. Statistical analyses were generated using statistical package IBM SPSS statistics 26.0 and Microsoft Office Excel spread sheets.

Exploratory Factor Analysis on the SLA-MB
EFA was performed to confirm the sound application of the SLA questionnaire. Initially, wrongly formulated questions were recorded by reversing the polarity (11 items of the "personal epistemology" scale).
A principal component analysis was then performed on the 25 item SLA-MB using the orthogonal rotation method (varimax rotation). Criteria for determining the number of factors are factor structure coefficients of 0.30 or greater, inspection of the scree plot, eigenvalues above one, correlations with other factors and the conceptual meaningfulness of factors (Benishek & Lopez, 2001;Pett et al., 2003;Stevens, 1992).
Moreover, the Kaiser-Meyer-Olkin Test (KMO) for sampling adequacy was .890 and Bartlett's test of sphericity was statistically significant (3772.451, p<.05) verifying the appropriateness of the EFA and CFA and supporting the factorability of the correlation matrices (Bartlett, 1950;Kaiser, 1970).
One item had relatively low loading and hence was deleted (item: In general, I find working on science assignments). The first three factors explain 50.372% of the total sample variance and the scree plot confirmed the retention of the first three factors. From the third factor onwards, there was a sharp change to the slope of the line. The first factor concerned "personal epistemology" and interpreted 26.14% of the total variance. The second factor, "SL self-efficacy" explained 17.68% of the variance. The third factor, which consisted of "value of science" items interpreted 6.53% of the variance ( Table 1). .752 7.PE Scientists pretty much know everything about science; there is not much more to know.
.735 9.PE Once scientists have a result from an experiment, this will be the only answer.
.712 11.PE Only scientists know for sure what's true in science.
.708 8.PE If you read something in a science book, you can be sure it is true.
.707 6.PE What the professor says in the class is true.
.679 3.PE Scientific knowledge is always true.
.661 10.PE Scientists always agree with what is true in science.
.647 2.PE All questions in science have one right answer.
.595 1.PE Everybody has to believe what scientists say.
.477 5.PE The most important thing in doing science is discovering the right answer.
.400 3.SE I know how to use the scientific method to solve problems.
.752 6.SE I can use math to answer in scientific questions.
.737 4.SE It's easy for me to see the difference between scientific findings and ads.
.708 8.SE It is easy for me to make a graph with my data. .696

Confirmatory Factor Analysis on the SLA-MB
A CFA using AMOS was conducted to test the fit of the proposed model (Figure 1). As part of the CFA, factor loadings were assessed for each item. Model-fit measures were used to evaluate the model's overall goodness of fit (CMIN/df, CFI, TLI, SRMR, and RMSEA) and all values were within acceptance levels (Hu & Bentler, 1999;Stylos et al., 2022;Ullman, 2001). The factor model yielded a satisfactory fit for the data: CMIN/df=2.02, CFI=0.93, TLI=0.93, SRMR=0.05, and RMSEA=0.05.

Exploratory Factor Analysis on the SLA-D1
A principle-component factor analysis was performed on the 26 item SLA-D1 to determine the factor structure. The Shapiro-Wilk and Kolmogorov-Smirnov test showed a non-normal distribution (p<0.001). Skewness and kurtosis of each item demonstrated that the variance was acceptable (West et al., 1996). Most of the correlations between items were lower than 0.3 indicating that factoring may not be useful (Beavers et

Reliability and Internal Consistency Testing of the SLA-D1 and SLA-MB
The Kuder-Richardson 20 and a-Cronbach reliability tests verified the internal consistency of the SLA-D1 and SLA-MB. The reliability factor for all SLA-D1 factors was α=.730, indicating valid responses. The discrimination indices for the 26 items of SLA-D1 ranged from 0.30 to 0.72. Item 19 showed the highest rate of correct responses at 86% and item 1 the lowest at 23%. Overall, 19 of the 26 items were answered correctly by more than 50% of students. As for the SLA-MB factors, the coefficient was α=.870. The category "value of science" consisted of five factors and had a reliability coefficient α=.816. The category "self-efficacy for SL" consisted of eight factors with a reliability coefficient α=.861. Finally, the category "personal epistemology" consisted of 11 factors with a reliability coefficient α=.860 ( Table 2).

Scores in SLA-D1 and SLA-MB
Four new variables were created when calculating the scores on the SLA-D1 (sum of the correct and wrong responses) and SLA-MB components (sum of the responses). Higher means "indicate better performance, stronger value, self-efficacy, and more sophisticated beliefs about science" (Fives et al., 2014, p. 569). The descriptive statistics are presented in Table 3.

Differences in Scientific Literacy According to Gender
Data for both men and women deviated significantly from the normal range on both components. Statistically significant differences were observed for self-efficacy and personal epistemology (Table 4). Specifically, men's self-efficacy beliefs and personal epistemology beliefs differed significantly to women (U=7,238.0, z=-2.317, p=.021). The women in the sample held more sophisticated personal epistemology beliefs about knowledge than men (U=6,452.5, z=-3.384, p=.001).

Differences in Scientific Literacy According to University Years of Studies
The data on undergraduate year deviated significantly from the normal range. No significant difference was found ( Table 5).

Differences in Scientific Literacy According to High School Course Specialization
Data on senior high school course specialization deviated significantly from the normal range. Students with a natural sciences and technology background in senior high school performed better than those with a social sciences and humanities background (U=7,599.0, z=-5.710, p=.000). Also, students with a natural   sciences or technology senior high school background expressed more positive views on value of science (U=7,606.0, z=-5.705, p=.000) and held higher self-efficacy for SL (U=6,140.5, z=-7.391, p=.000) ( Table 6). Table 7 presents the correlations among the SLA-D1 and SLA-MB components and the three high school course specialization, gender, and years of study. The analysis showed that performance on the SLA-D1 and the value of science category were negatively correlated with gender and undergraduate year. On the contrary, performance on the SLA-D1, categories value of science and self-efficacy for SL were positively correlated with pre-service teachers' high school course specialization.

Correlation Analysis
Additionally, the correlation analysis among components of the SLA-D1 and SLA-MB revealed correlations that ranged from -.211 to .319, establishing the independence of these constructs, excluding any significant correlation between value of science and self-efficacy beliefs for science literacy (Table 8).

DISCUSSION
The purpose of this study was first to validate the Greek version of SLA, i.e., a measure of students' ability to think scientifically and assess their motivational beliefs about science and second to investigate the level of SL of pre-service teachers.
Initially, EFA and CFA examined the factor structure of each of the two SLA components. The results confirmed the one-dimensional and three-dimensional construct of the SLA-D1 and SLA-MB components, respectively. In terms of reliability, the results suggested acceptable α-coefficients. The statistical analyses showed that the Greek version of the SLA has the appropriate psychometric properties to assess pre-service primary teachers' ability to think scientifically and their motivational beliefs about science.
Greek pre-service primary teachers had moderate to high scores on both components. SL (SLA-D1) was moderate (55.61%) indicating that pre-service teachers are unable to respond effectively to everyday situations and examples. These results are in accordance with other studies (Bacanak & Gokdere, 2009; Karamustafaoglu et al., 2013). Gender and undergraduate year differences were statistically insignificant aside from high school specialization. Students with a high school natural sciences and technology background showed better evidenced knowledge than those with a social sciences/humanities background (Sargioti & Emvalotis, 2020).
Such a finding is to be expected, as pre-service teachers with a high school background in the natural sciences had already been exposed throughout their three years in senior high school to subjects similar to those in the survey, contrary to students with a social sciences/humanities background who, although the majority had been in contact with the subject of biology, were not as oriented toward the natural sciences. Science courses in high school seem to increase students' ability to apply their knowledge to everyday contexts (Stylos et al., 2021).
With regards to the SLA-MB component, responses were satisfactory for value of science, self-efficacy, and personal epistemology with corresponding percentages 76.67, 75, and 74.55. The latter is very important as, aside from knowledge, a scientifically literate person must have science motivation and beliefs to apply said knowledge in real life situations (Fives et al., 2014).
The results demonstrated gender differences on self-efficacy and epistemic beliefs. Specifically, male students' levels of self-efficacy beliefs were stronger than their female counterparts which is contrary to the results of Stylos et al. (2022) where no gender differences were detected. The opposite however applies for epistemic beliefs where female students scored higher (Hacieminoglu et al., 2015;OECD, 2016). Similarly, Hofer (2000), in a study among university students, came to the same conclusion, that men are more stable in their view that scientific knowledge is not subject to change. With regard to high school experiences, preservice teachers with a high school background in the natural science score higher on value of science and hold stronger self-efficacy beliefs. It is worth noting that individuals' experiences play an important role in the development of their beliefs (Schommer, 1994). This difference in epistemic beliefs is consistent with other studies (Sargioti & Emvalotis, 2020).
Moreover, the non-statistically significant differences between undergraduate year and the SLA-MB component may be due to students' attendance of approximately the same number of science courses regardless of academic year. As a result, experiences or motivational beliefs about science do not differ according to years of academic enrolment. It is worth noting that individuals' experiences play an important role in the development of their beliefs (Schommer, 1994). The positive impact of high school course specialization on the score of the SLA components was confirmed empirically, providing clarity on the process by which undergraduates develop SL.

CONCLUSION
The present study shows the results of a quantitative research that measured the SL of 362 pre-service primary teachers. For this purpose, a specific tool, SLA, developed by Fives et al. (2014) was used. EFA and CFA confirmed the construct of the SLA in the Greek context. Internal consistency reliability for each component had acceptable values. Pre-service teachers' levels on the SLA-D1 and SLA-D1 scales were moderate and high respectively. The statistical analysis demonstrated a significant effect of high school course specialization on pre-service teachers' science motivation and beliefs. Low but significant correlations were found between science motivation and beliefs.

Implications
The implications deriving from the study concern educators. researchers, and curriculum developers of pre-service teacher education programs. Teacher preparation programs should consider that pre-service teacher populations consist of students with various high school learning status. In a follow-up experimental context, the SLA can also be distributed as a pre-and post-test to compare diverse teaching methods or integrated in longitudinal analyses to report modification in demonstrated knowledge, beliefs, and values. Inquiry-based teaching, group dialogue and STEM-based teaching, game-based teaching can improve