A Study of Learning and Motivation in a New Media Enriched Environment for Middle School Science
Liu, M., Horton, L., Olmanson, J., & Toprac, P. (2011). A study of learning and motivation in a new media enriched environment for middle school science. Educational Technology Research and Development, 59(2), 249-265.
- What kind of study is this?
This study is a mixed-method design with both quantitative and qualitative data collected but a heavier emphasis upon quantitative data. The research is correlational in nature in that the researchers attempt to analyze the relationship between student participation in a new media enriched project-based learning experience and their learning of science concepts and levels of motivation, but do not attempt to establish causation. While constructed responses provide qualitative data in which students describe their learning experiences and motivation levels, quantitative data from pre- and post-tests and student surveys are the crux of the study and provide an avenue for analyzing the relationship between the learning experience itself and these outcomes.
- What was the purpose of the study? (what questions)
The purpose of this study was to utilize a new media enriched project-based learning experience called Alien Rescue (developed previously by the researchers) to arrive at conclusions based on three research questions established by the authors:
- What is the effect of the new media PBL (project-based learning) environment on 6th graders’ science learning?
- Are 6th graders motivated to use this new media enriched PBL environment and in what way?
- What is the relationship between students’ motivation and their science learning?
The authors also indicate that due to existing research showing evidence of gender bias in computer-based instruction, gender differences were also carefully observed.
- What is already known about the subject and what does this study add to the literature?
Over the last two decades, there has been a slowly growing movement to incorporate increasing amounts of technology in science classrooms and to implement new and innovative methods of technology-based instruction for students. As pointed out by the authors, this comes as no surprise since modern youth have a broad variety of experiences with the use of technology in their day to day lives. Nowhere are students more connected to their personal technology than through new media applications. Carrington (2004) indicates that the interactions students have with technology and these applications provide opportunities for the creation of and interaction with dynamic texts like the ones utilized in this study. In fact, Judson (2010) goes as far as stating that a refusal to exploit the relationship students have with technology and these types of applications can result in teachers missing opportunities to maximize the potential of modern students. In terms of motivating students and engaging them in their learning experiences, research has also shown that the cooperative group-based nature of project-based learning can have a similar impact (Shernoff, et. al., 2003). As a result of the corresponding benefits present in existing research, Liu, Horton, Olmanson, & Toprac are attempting to explore curricular avenues for bringing new media and technology into the science curriculum in a project-based learning environment.
Alien Rescue is a new media enriched PBL environment developed for 6th grade space science students (Liu, Williams & Pedersen, 2002). The ultimate goals of Alien Rescue are for students to gain and utilize knowledge about space and planetary science, the scientific method, and inquiry to solve a complex problem involving finding a planetary settlement that would meet the needs of a group of aliens. New media applications such as an alien database, a communication center (chat room), probe design room, and mission status center are utilized to provide students with information about characteristics and needs of the aliens (http://alienrescue.edb.utexas.edu). Students must then utilize existing knowledge about planetary characteristics to find a suitable habitat for the various alien species. Previous research (Liu, Toprac, & Yuen, 2009) focused on factors that students found to be motivating in a new media enriched PBL environment. The current study builds upon the knowledge gained in previous research by focusing on the impact new media enriched PBL environments have on student achievement, the correlation between motivation and student achievement, and possible gender differences. The positive results of this study also support other recent research on the benefits of infusing learning environments with new media (Barab, et.al., 2005; Ketelhut, 2007).
- What (teachers, classes, groups) did the author(s) study?
The study focused on 220 6th grade students from a middle school located in the southwestern United States. 119 of the students were female, while 101 were male. The school demographics in terms of race are approximately 46% White, 30% Hispanic, 11% Asian, 9% African-American, and 4% other. Students were members of general/”regular” education science classes and participated in the Alien Rescue experience for 3 weeks during 45 minute science class sessions. Two science teachers at the school facilitated the Alien Rescue experience. Both had previous experience with Alien Rescue and engaged in professional learning on its use prior to the study.
| Teachers engage in training and professional development in use of the new media enriched learning environment, Alien Rescue |
In sequence, what were the major steps in doing the study?
| Researchers analyze data from pre-and post-tests, IMI surveys, and open ended responses. | |||||||
- What methods were used to collect the data that were used for analysis?
The methods used to collect the data for analysis in this study were both quantitative and qualitative in nature, with a significantly higher emphasis placed on quantitative data. Qualitative data was utilized for student explanations and detail gathering from participants. The quantitative data on student learning was collected using a 20 question multiple choice test on science concept knowledge before and after student participation in the Alien Rescue curriculum. Questions had four possible answers with one being “Unsure.” Changes in student knowledge from pre- to post-curriculum were assessed. Quantitative data on student motivation was collected utilizing 15 questionnaire items from the Intrinsic Motivation Index (IMI), a 7 point Likert scale. Four subscales in particular were selected due to their relevance to the defined research questions: interest/enjoyment, perceived competence, effort/importance, and value/usefulness. Open ended response questions asked students to describe their experience to a friend and compare it to other science activities they have done in the past.
- What kind(s) of analysis of data was used?
When analyzing the data collected through science knowledge tests and IMI questionnaires, means and standard deviations of pre- and post-scores were observed with two ANOVAs conducted on test and questionnaire data. To address the first research question, ANOVAs used gender as a between-subject independent variable (since, as previously mentioned, gender bias was a concern) and testing time (pre- and post-) as the within-subject variable. Dependent variables were science knowledge test scores and number of “unsure” answers respectively. The second research question was assessed by conducting ANOVA with the same gender between-subject independent variable and total and subscale IMI motivation scores as the dependent variable. The third research question was analyzed using multiple regression analyses utilizing total IMI motivation scores, subscale scores, and post-test science knowledge scores.
- What were the results?
Science Knowledge Test Results:
There were significant increases in student performance on the science knowledge test among both genders with male students having slightly higher post-test scores (M=83.53) than females (79.36). However male pre-test scores were higher than female pre-test scores so female students actually made greater gains (M=31.85 points) than their male counterparts (M=28.02 points). Overall, student knowledge was significantly increased (M=30.31 point gain). The percent of questions to which students responded that they were “unsure” also dramatically decreased from 26.15% of questions on the pre-test to on 8.02% of questions on the post-test. Qualitative responses from participants provided details and elaboration on the reasons for these results.
Motivation Results:
ANOVA analyses of the IMI questionnaire and corresponding subscales demonstrated that there were no significant differences in male and female motivation regarding this experience (Mmale= 3.60, Mfemale=3.75). Regression data indicated that student motivation was highly positively correlated to science knowledge post-test scores. Of the subscale focuses, perceived competence demonstrated the highest reliability as a predictor of student success on the content assessment with effort/importance having a significant but less highly correlated impact. Value/usefulness and interest/enjoyment subscales did not show a correlation with content knowledge post-test performance. Qualitative responses from participants provided details and elaboration on the reasons for these correlations.
- What does the author conclude?
Simply stated, the authors conclude that the new media enriched PBL environment had a significant positive impact on student learning and motivation. Students of both genders significantly increased their scientific concept knowledge from pre-test to post-test and decreased their number of “unsure” responses (although female students showed greater decreases in this area). Students were also able to specifically identify what they learned through their open response questions. Since the curriculum was self-paced and required minimal teacher intervention, the authors also conclude that such PBL environments can allow student attainment of content knowledge through self-directed learning and peer interaction. Additionally, the researchers were able to conclude that the majority of students were motivated to learn using the Alien Rescue new media enriched PBL environment. There was a significant positive relationship between student motivation levels as measured on the IMI questionnaire and student post-test scores as well. Furthermore, high scores in the perceived competency subscale proved to be moderately related to high post-test scores indicating that perceptions of competency in scientific tasks somewhat led to better learning experiences and higher knowledge attainment. Ultimately, the incorporation of new media and the use of student-designed projects within learning experiences like Alien Rescue appear to have a positive impact on student learning and motivation.
- What cautions (if any) does the author raise about interpreting the study?
Though the authors do not explicitly state any cautions about interpretations of the study other than the limitations listed below, there is an implicit concern which is apparent. Generally speaking, the authors caution about drawing a strong linkage between new media enriched learning experiences and student learning and motivation based upon this study alone. On multiple occasions, the authors acknowledge that this is simply initial research into a possible correlation and emphasize the necessity of further study. This research provides promising possibilities, but like so much of the research into incorporating technology in the science classroom it is still in its early stages.
- What do you think are important limitations of the study?
There are two major limitations of this study which should raise some concern when analyzing the results of the research and extrapolating its utility to other scenarios. The first of these is the naturalist setting of the study. All 6th grade courses participated in the Alien Rescue learning experience and, as a result, comparative analysis cannot be done. The impact of the learning experience on student performance in pre- and post-tests and surveys can be assessed, but not against the backdrop of students who did not participate in the learning experience because such students were not studied. The other important limitation is the fact that this learning experience took place at the end of the school year. In addition to the fact that this means it was not integrated or incorporated into the standard science curriculum, this means that many students were unable to complete post-tests due to many end of the year activities that pulled them from their science classes. Since data of this type was not collected, the sample size of the study was substantially impacted. Though neither of these limitations inhibit analysis of the impact of the new media enriched learning experience, the study and its conclusions could be significantly strengthened with further research that takes these issues into account.
- What particularly interesting or valuable things did you learn from the study?
In my current classroom, I attempt to incorporate technology as much as possible into the science curriculum. I am currently running a pilot blended learning program at my school and hope to expand it next school year to include incorporation of project-based learning. As such, this study is key to my areas of interest. First and foremost, the study provides valuable information about the validity of utilizing technology in a PBL environment. Achievement data from this study demonstrates that students will still acquire the requisite content knowledge if such a method is used. The study’s findings regarding student motivation are also encouraging because they indicate that students find the experience to be worthwhile. Furthermore, the demographics of the school utilized for the study are fairly similar to mine so there is no reason for colleagues to discount the findings of this research.
While I have previously used numerous technology-based applications for both traditional instruction and project-based learning, I have never utilized a platform as fully formed and comprehensive as Alien Rescue. This new media enhanced PBL environment is very appealing and based on previous experiences with my own students I believe this would be extremely beneficial in terms of both learning and motivation. Typically technology-based instruction of the past has involved the standard curriculum and the incorporation of technology-based applications for product creation where applicable. However, Alien Rescue is highly integrated into the curriculum and seems to be the type of application there needs to be more of in the future if true technology integration in the science curriculum is the goal. Though I am not currently teaching 6th grade science, I will be investigating other technology-based PBL environments for use with my 7th grade curriculum and will utilize Alien Rescue the next time I have the opportunity to teach space science. This research is yet another example of the expansion of technology into middle school science and helps to significantly inform aspects of my future research focus.
References
Barab, S., Thomas, M., Dodge, T., Carteaux, R., & Tuzun, H. (2005). Making learning fun: Quest Atlantis, a game without guns. Educational Technology Research and Development, 53(1), 86-107.
Carrington, V. (2004). Texts and literacies of the Shi Jinrue. British Journal of Sociology of Education, 25(2), 215-228.
Judson, E. (2010). Improving technology literacy: Does it open doors to traditional content? Educational Technology Research and Development, 58(3), 271-284.
Ketelhut, J.D. (2007). The impact of student self-efficacy on scientific literacy skills: An exploratory investigation in River City, a multi-user virtual environment. Journal of Science Education and Technology, 16(1), 99-111.
Liu, M., Williams, D., & Pedersen, S. (2002). Alien Rescue: A problem-based hypermedia learning environment for middle school science. Journal of Educational Technology Systems, 30, 255-270.
Liu, M., Toprac, P., & Yuen, T. (2009). What factors make a multimedia learning environment engaging: A case study. In R. Zheng (Ed.), Cognitive effects of multimedia learning (pp. 173-192). Hershey, PA: Idea Group, Inc.
Liu, M., Horton, L., Olmanson, J., & Toprac, P. (2011). A study of learning and motivation in a new media enriched environment for middle school science. Educational Technology Research and Development, 59(2), 249-265.
Shernoff, D.J., Csikszentmihalyi, M., Schneider, B, & Shernoff, E.S. (2003). Student engagement in high school classrooms from the perspective of flow theory. School Psychology Quarterly, 18(2), 158-176.
