My Literary Infographic! (Zoom in to see all the details)

Loading...

Mini-Research Project: An Analysis of Critical Thinking in STEM Classes

Link to presentation: http://prezi.com/hpziypt9owqn/?utm_campaign=share&utm_medium=copy

An Analysis of Critical Thinking in STEM Classes

Ted Watson

Stevenson University

Abstract

 This study is an analysis of critical thinking skills in 9th grade science and recommendations for improvement regarding enthusiasm and skills in critical thinking. Thirty students’ tests regarding STEM-themed knowledge, using a mixture of selected response and open-ended questions, allowed for analysis of Critical Thinking skills.  A formative assessment was administered to students to gain insight in the topic of critical thinking. A description of critical thinking questions was included with the assessment to bring clarity to the subject in question. The results of both the test results and the formative assessment show no strong opinions either for or against critical thinking problems. Upon analysis, recommendations have been made to implement a section of curriculum devoted to critical thinking and analysis skills to encourage use and enthusiasm for real-life scenarios in the classroom. 

Introduction

Critical thinking can be defined as using intellect to synthesize and extrapolate into a concepts that are of “a higher order and abstract nature.” (Shaw 2014). Critical thinking in the classroom removes the single correct answer and demands that students use their knowledge to create a solution on their own that best suits a scenario. Dr. John Barell writes that it is a 21^st century skill, because problems that arise in everyday life and in careers hardly ever have a single, concrete answer (2010). By teaching students to look for an answer in an unconventional manner, teachers are preparing students to be creative and effective when a challenge arises. It is a life skill as much as an academic one. Students can do this by raising questions, gathering information, analyzing their research and then communicating that research to others (Scott, 2008). This can be related to the scientific process, but also applied to producing factual reports and solutions.

During my last three years of teaching, there has been one stand-out problem that I have encountered over and over again: lack of critical thinking. When I present a class with open-ended questions, more often than not I am met with groans and blank stares, regardless of what grade it is. The students get flustered and say things like "Why can't there just be one answer?" or "Can't you just tell me what it is?" They continue a downward spiral when I explain to them that there is no single answer, and that it is a matter of research and opinion.

Problem solving in open-ended situations is something that my 9^th and 11^th graders often face as new things in my class. An example of this is a recent project I gave my seniors. The scenario was as follows:

You have been contacted by a lawyer from northwest Vermont stating that a long-lost relative has willed you their dairy farm. Regardless of your desire to keep the farm, the lawyer has stated that there is a dangerously eutrophic pond on the property. The pond is at risk of flooding and running into Lake Champlain via a nearby river, and will be subject to enormous fines and penalties if it does. Your task is the following: using the chart below, develop a plan to contain, clean up and prevent future eutrophication to avoid penalties. You may use your text and research methods of clean up and testing for lake health.

This was a "group quiz" where the students were asked to work in pairs to use their knowledge of our last unit on pollution and pollution management to come up with a strategy to fix their farm. After the students had read the directions, the blank stares and push-back started with statements like "but this wasn't in the book!" and "I can't find anything online that can help with this assignment!"

Research question: What is students’ level of enthusiasm for critical thinking and how can that enthusiasm increase in the classroom and in work?

Variables Measured: The study will also analyze student’s scores in the critical thinking (open-ended and ill-defined problems on the test) as a gauge of critical thinking skills in the 9th grade science class. The study will also analyze student’s enthusiasm for critical thinking problems, as gathered from a Likert scale self-assessment.

Controls: Interviews and self-assessment questionnaire with the instructor(s) to gauge the amount of open-ended questions and critical thinking skills that are promoted in class. Analysis of the exact same test in two different classes (same 9th grade science course taught by 2 different teachers).

No control group will be used.

Sample size: 30 students

Data Collection

Thirty (30) tests were collected from two different classes. The test was a summative assessment focused on lab design in the “Doing Science” section of a 9^th grade Introduction to STEM course. It contained mostly short answer questions with a few multiple response in the middle of the test. The short answer were ordered in a manner that asked for a single answer for the first question, and ended with open questions at the end of the section, asking students to extrapolate on the information they had learned previously.

            The open-ended questions were analyzed for accuracy in the students’ ability to correctly answer what was asked of them. Out of a percentage scale, it was found that the average score was a 9/15, or a 60%.

Immediately following the test, students were asked to fill out this self-assessment:

Directions: Place a check mark in the box that best describes YOUR opinion. There is no right or wrong answer, so please answer truthfully!

	Strongly Disagree	Disagree	Agree	Strongly Agree
1. I enjoy questions that do not have a single, concrete answer
2. I feel frustrated when I encounter a question on a test that I have not seen in my book or in class
3. I feel challenged by questions that do not have a single correct answer
4. I find an open-ended question helpful when they have related questions preceding it

Figure 1. Self-Assessment given to the students after their test.

Results and Discussion:

The data was answered as follows:

	Strongly Disagree	Disagree	Agree	Strongly Agree
Question 1	4	18	7	1
Question 2	2	4	9	15
Question 3	0	0	19	11
Question 4	2	9	11	8

            Through the responses to the self-assessment, it is clear that a majority of the students do not enjoy open-ended questions. Common responses when asked why were: “I just want to study a definition and write it down” and “it makes me think too much during the test”. Interviews with the instructors indicated that there is not curriculum in place to promote or explain why it is important to use critical thinking in the classroom, on tests, or while learning the required concepts.

Figure 2. Bar graph depicting responses to the self-assessment

There are a few different reasons that could explain the outcome of this data. First, I realize that at this age, cognitive development can differ from student to student, which means that some students will have to work very hard to take an abstract scenario on paper and apply concepts from earlier in the week to solve an issue. This is especially true of 9^th graders. The next issue that could be conflicting with this type of question could be a lack in practice in this type of question. Second, it became very apparent from the interviews with the two teachers that there has been almost no preparation in regards to answering critical thinking problems, be it in formal instruction or graded assignments.

Suggestions for Improvement

            There are a number of solutions that stem from the data and information collected. First, it is important to implement a lesson or lessons devoted to instructing students on how to solve critical thinking problems. Possible methods for instruction can be carried out by providing a flowchart (Barell, 2010) or framework (Scott, 2008) to guide students through the somewhat daunting idea of addressing an open-ended question. Along with this, the same Likert scale assessment given previously to the Introduction to STEM class could be issued by the teacher to re-assess the class in order to gauge possible changes in the students’ attitudes towards critical thinking. In a model based off of SUNY Binghamton, feedback from the self-assessments could be analyzed by the department and assignments can be adjusted to prepare students more effectively for critical thinking scenarios using a flowchart to efficiently go through steps for improvement (McKitrick and Barnes, 2012). Also possible is a project for the students to work on where they address a current issue in today’s world that can be solved using concepts learned in their STEM class.  Ideally, students can pick their own subjects to ensure enthusiasm for the project (Barell 2010).

Conclusion

Critical thinking is an essential part of success in modern society. However, students are disheartened by the amount of work involved in successfully answering critical thinking problems. My survey found that students are not very successful, nor enthusiastic about the concept of open-ended, critical thinking problems in the classroom. To counteract this lack of enthusiasm, it is important to formally teach these students the both the proper way and importance of answering critical thinking problems so that they are prepared for real-world scenarios and their future careers.

References

Anonymous and Anonymous (per interview request), personal communication. March 3, 2016

Barrel, J. (2010). Problem-based learning: The foundation for 21^s century skills. In Bellanca, J. & Brandt, R. (Eds.), 21^st century skills: Rethinking how students learn. Bloomingon, IN: Solution Tree Press.

Mckitrick, S.A. & Barnes., S.M. (2012). Assessment of Critical Thinking: An Evolutionary Approach. Journal of Assessment and Institutional Effectiveness, 2(1), 1–29. http://doi.org/10.5325/jasseinsteffe.2.1.0001

Scott, S. (2008). Perceptions of Students' Learning Critical Thinking through Debate in a Technology Classroom: A Case Study. The Journal of Technology Studies, 34(1), 39–44. Retrieved from http://www.jstor.org/stable/43604224

Shaw, R. D. (2014). How Critical Is Critical Thinking? Music Educators Journal, 101(2), 65-70 DOI: 10.1177/0027432114544376

Final Personal Assessment Philosophy

My Final Personal Philosophy

At the beginning of this course, I was strongly opposed to standardized testing. As we finish the fifth week of this course, my views have shifted significantly. Before this course, I was under the impression that standardized testing was only bubble tests that gave a score to rank schools and students; basically, the only type of test were the ACT and SAT style tests. However, I have now seen the usefulness and diversity of standardized assessments.

First, I was unaware that the word “assessment” covers tests other than a bubble assessment. I have been using self-assessments and rubrics since my first quarter of teaching without knowing that they were a form of standardized assessment. I also did not know of the usefulness of assessments in quality control and achieving teaching goals in a classroom. Using tools such as self-assessments can boost morale, effectiveness and enthusiasm in the classroom, and I now have good ideas on how to implement them more effectively (Andrade 2008). For example, it is useful to give multiple self-assessments throughout the year, so that students and their teachers can see their progress as they progress through course.

Second, I was not aware of the help that assessments could bring to evaluating a teacher’s own assessment. There is always a nagging voice in the back of my head when a student gets a poor grade on a test or project that says “maybe I did not teach this correctly to this student”. Using different types of assessments and questions can help to nip that voice in the bud and ensure that 1) students are being assessed fairly and 2) that I have taught the subject adequately to the students.

As with all things, I strongly believe that there is no silver bullet to standardized testing. As I have mentioned before, I feel that the basic bubble test only tests rote memorization. Using authentic methods has shown that students are more likely to be able to use their academic strengths to express their classroom knowledge (Sternberg 2008). However, I have experienced first-hand the disadvantages of the authentic testing grading system. Being an IB school, our standardized tests are all essay and short answer tests coupled with a separate evaluation of a written lab report. These assessments are collected and sent to an assigned grader somewhere in the world. Each year, our school is frustrated in at least a few subjects by the inconsistencies provided by the graders. One year a grader may focus in minute detail on a certain grading criteria while in the next it could be something totally different from another. IB grading is sent back with comments both on the student’s work and on the thoroughness of the teacher’s assignments, but each of those are taken with a grain of salt in the science department because of the lack of quality assurance of the graders. While the teachers are frustrated, the students are the ones who suffer, because some continue to college without an IB Diploma. This is rare, but it does happen, and then there is a repeal process for the grades and the papers and more time is used.

I am left in a middle ground. I despise the industrial-esque utilization and impracticality of the multiple choice test, but the human error in grading of assessments can also be frustrating. I am not the only one who has seen this. In his paper: Crossed Random-Effect Modeling: Examining the Effects of Teacher Experience and Rubric Use in Performance Assessments, Adnan Kan notes that “One of the shortcomings of performance assessments is the subjectivity and inconsistency of raters in scoring” (2014). He goes on to explain that a possible solution to this shortcoming is a much more detailed rubric for grading that can allow for a wide range of responses to be graded into each category. I am in agreement. I think that it is entirely possible that students now are able to express themselves in a way that promote their strengths. One could argue that this is how we tackle life. Humans are naturally drawn towards what their strengths are and are most interested in a expressing ourselves in that manner. If we are preparing students for life, shouldn’t we allow them to do the same in school?

Authentic testing has been displayed in many test classrooms in the last few years and recently it has taken a rise in STEM classrooms across the country. In a recent study in North Carolina, Jeremy Ernst and Elizabeth Glennie analyzed the results of a pilot program for a performance-based assessment. The data showed that there were improvements overall from 48% of the students, but the overall results cannot yet be confirmed due to small sample size. This is a new field of study, and will continue to provide results as the years go on.

Bridging from previous classes, I am now very interested in creating assessments that cater to 21st-century skill sets. I have progressed through this course with a overhanging theme of “How can the students learn this material in a way that can help them outside of the academic world?” One of our books from an earlier class, 21st Century Skills: Rethinking How Students Learn, helps shed light on that question (Bellanca and Brandt 2010). First, 21st century skills are defined as the skills that will help students in today’s modern society. They include innovation, life and career skills, and working with information, media and technology (Kay 2010). The idea behind these skills are to prepare a student to be effective in the working world, dealing with everyday problems, and teamwork. Dr. Douglas Reeves writes in his chapter, titled A Framework for Assessing 21st Century Skills, develops a framework that allows for assessment of students while they are able to gain life skills in the process (2010). Reeves states that a good 21st-century assessment should maintain 5 parts: Learning, Understanding, Exploration, Sharing and Creativity. Learning entails the basis of any assessment, to display a student’s knowledge of the content in focus. Understanding is the critical thinking aspect of assessments, an ability to bring that knowledge to abstract contexts, rather than simply reciting a definition. Exploration asks students to delve deeper into the knowledge, asking them to back up their knowledge with research and examples from reliable sources. And finally, creativity and sharing ask students to understand the material to a point where they are able to express their knowledge in a way they are comfortable with (Reeves 2010).

I will be taking this material into account when I create my final exam this year. Rather than give a written exam, I will be asking students to create a portfolio on an endangered animal of their choice. They will be analyzing the threats to the species, designing a conservation plan, and looking at the possible impacts of the ecosystems and the local populations. This portfolio “action plan” will enable students to research, develop, and present a formal action plan much like one would present a business proposal, while asking the students to demonstrate a deeper knowledge of the material they have learned over my course.

Overall, I am surprised at how my opinions have reversed over this course. I have most certainly let my guard down when it comes to the term “standardized” and now realize the importance of assessments, both for teachers and students.  

References

Andrade, H. (2007/2008, December/January). Self assessment through rubrics. Educational Leadership, 65(4), 60-62.

Ernst, J. V., & Glennie, E. J. (2015). Redesigned High Schools for Transformed STEM Learning: Performance Assessment Pilot Outcome. Journal of STEM Education: Innovations and Research, 16(4), 27-35. Retrieved from http://eric.ed.gov/?q=performance assessments&pr=on&id=EJ1086343

Kan, A., & Bulut, O. (2014). Crossed Random-Effect Modeling: Examining the Effects of Teacher Experience and Rubric Use in Performance Assessments.Eurasian Journal of Educational Research EJER, 14(57), 1-28. Retrieved from http://eric.ed.gov/?q=performance assessments&pr=on&id=EJ1056261

Kay K. (2010). Foreword: 21st century skills: Why they matter, what they are, and how we get there. In J. A. Bellanca & R. S. Brandt (Eds.), 21st century skills: Rethinking how students learn. Bloomington, IN: Solution Tree Press. 

 Reeves, D. (2010). A framework for assessing 21st century skills. In J. A. Bellanca & R. S. Brandt (Eds.), 21st century skills: Rethinking how students learn. Bloomington, IN: Solution Tree Press. 

Sternberg, R. J. (2007/2008, December/January). Assessing what matters. Educational Leadership, 65(4), 20-26.