Designing Next Generation Assessments to Support the Teaching and Learning of Life Science

Project Overview

The NGSA Life Science project is a 3-year, $1.46 million project funded by the Gordon and Betty Moore Foundation. Based on the work underway in another project in the larger Next Generation Science Assessment endeavor, the team is designing, developing, testing, and validating a set of 3-dimensional assessment tasks and scoring rubrics aligned with the Next Generation Science Standards Performance Expectations for middle school science classrooms.

 

Big Ideas


NGSS and the Framework

The Next Generation Science Standards (NGSS) are very different from the previous generation of science standards. They require learners to use scientific practices (such as asking questions and defining problems) to develop and apply core ideas (such as those concerning forces and motion) and crosscutting concepts (such as cause and effect). This new idea of 3-dimensional learning is significantly different from traditional ways of learning.

While NGSS marks a new era for science teaching, there are currently very few supports for teachers to put NGSS ideas into practice. Assessment is an essential piece of the puzzle, but the challenge is how to replace the traditional test items that don’t align with NGSS. What is tested often influences what teachers decide to teach and what they hold their students accountable for knowing. Even if standards are closely aligned with a desired vision of teaching and learning, achieving a coherent system is undermined when assessments are not aligned with that vision.

This project helps meet the critical need for high-quality assessments that align with the performance expectations in the new standards and are instructionally suitable for teachers to use in classrooms. Teachers, administrators, school boards, and others need high quality, research-based examples of innovative assessments developed specifically to support their transition to the new standards.


Addressing Multiple Needs

The goal of this project is to address this urgent need for examples of research-based assessment. The team will develop next-generation assessment items that address specific core ideas in life science—structures and processes of organisms, and interactions, energy, and dynamics of ecosystems—and two important crosscutting concepts: patterns, and energy and matter. They will also integrate the content of the tasks with two important scientific practices: constructing explanations, and developing and using models. The team selected these core ideas because they help to explain and predict a wide variety of phenomena that occur in the everyday lives of students, and because these practices are central to the work of scientists but are too often not seen in science classrooms.

The new assessments are being iteratively designed and tested in middle school science classrooms to ensure their usability. They will also be administered across grade levels to allow the research team to explore how students progress over time in their understanding of science practices and their knowledge of content. Researchers will work in both NGSS-focused classes and classes with a more traditional curriculum. The Investigating and Questioning our World through Science and Technology (IQWST) curriculum was an earlier CREATE for STEM Institute project that developed curriculum aligned with NGSS for middle school science classes. By testing the assessments in both environments, researchers hope to create assessments that are broadly usable across all types of science classrooms.

Outcomes of the project include: (1) a framework for the principled design and analysis of assessments that measure core ideas in integration with science practices, (2) operational tasks and rubrics that can be used formatively in middle school life science instruction, and (3) a practice guide to support teachers’ formative use of these assessments.

The design framework and exemplar assessments will serve as timely models for the science education and assessment communities about what the next-generation science assessments should look like. The development of an ECD-based assessment design approach (see below) for NGSS has already garnered a lot of attention from three key stakeholder groups: assessment design and research, science education leadership and research, and science educators.

The science teacher community will especially benefit from the assessment items and rubrics, as well as the guidelines for developing instructional resources that support the formative use of the assessments.

The assessment community will be interested in our design patterns, task templates, assessment items, and rubrics, as well as our application of technology and advanced psychometric models to the measurement of student performance targeted at the NGSS.


Evidence-Centered Design (ECD) Approach

The project will incorporate principles of evidence-centered design (ECD) and state-of the art psychometric methods. Psychometrics is the field of study concerned with the theory and technique of psychological measurement, which includes the measurement of knowledge, abilities, attitudes, and personality traits. The team will use these methods and ECD to develop, test, and analyze their technology-supported assessments for middle school life science classrooms.

The ECD approach proposed here will provide an optimal framework for integrating the substantive aspects of valid design and the technical infrastructure required to convert evidence gathered from student performances into reliable inferences about student thinking and understanding.


Equity/Fairness and Technology

There is a big push in the science community to be more inclusive. This means that many researchers are working hard to make sure that traditionally underrepresented groups have an equal chance at receiving a quality science education. In the same way, this project team is looking to make sure its assessments are fair.

Researchers are applying an equity/fairness framework in developing all of the Learning Performances (LPs) and tasks. Eventually, all of the LPs and tasks will undergo reviews by an outside panel. The reviews will explore the comprehensibility, equity, and fairness of all of the assessment tasks in order to ensure that they are accessible and fair to students of diverse cultural, linguistic, and socioeconomic backgrounds.

Another important commitment to equity is that all of the tasks are offered over the internet on a free site hosted by the Concord Consortium. The fact that they are online means that more districts will have access to them. This also means that developers can implement innovative technological components to help students represent their ideas. The Concord site also hosts the assessment tasks developed for the NGSA Physical Science project, the project on which this NGSA Life Science project is based.

Videos and simulations are incorporated into some of the tasks to create authentic and engaging scenarios and also to reduce literacy demands. Stamps and drawing tools are included to help students more easily express ideas and formulate responses. Guidance, such as prompts to support students in constructing explanations and building models, is included to help students who are making early steps toward a set of performance expectations.

The team is also developing paper-and-pencil task alternatives in order to be inclusive of diverse school settings. We realize that some schools lack the necessary resources to use digital tasks formatively. Researchers will compare the results across testing modes.

Although the team hasn’t posted any life science-oriented assessments on the website yet, improvements to the site will facilitate usage and dissemination in the future.


Formative Assessment

Formative assessments help teachers gauge how their students are relating to a certain topic and whether they understand content. The project team’s prior research on formative assessment development demonstrated how including teachers as co-designers of formative assessment resources can make the products more useful and usable. Therefore, they are working with teachers to understand what kind of rubrics, curriculum guidelines, and other resources they need to use the assessments in this way.

The assessments being developed for this project will allow students to demonstrate an increasingly sophisticated understanding related to a core idea and set of practices in the NGSS. Researchers can track the interaction of this core idea with the practices within and across middle school grades 6 to 8. In other words, they will use the assessments to track students’ knowledge of a topic over time.

The team will develop multiple, small rubrics that are focused on a specific focal knowledge, skill, and ability. These focal knowledge, skills, and abilities are integrated statements of students’ knowledge-in-use and therefore fit with the 3-dimensional nature of NGSS. Scores on these small rubrics are focused on components of the learning performance, and this should help teachers use the assessments formatively. Additionally, scores on these small rubrics can be easily combined to give an overall score on the assessment task, and therefore serve as indicator of students’ proficiency with the entirety of the learning performance.


Dissemination

The team has allotted time at the end of each of the four major activities for synthesis of results, writing papers and conference presentations, and disseminating findings and associated products.

For more information and the team’s resources, please visit the sites below:

 

The Learning Performances

To meet the goal of developing assessment tasks that can support NGSS-aligned instruction, the team must unpack some NGSS performance expectations. This involves consideration of the core components of the science practices, intersections with other science practices, and the evidence required to demonstrate the practice. They become statements called learning performances, which guide assessment task development for classroom use. The team completed preliminary unpacking of the practices as part of the NGSA Physical Science project, so this project will focus on refining and extending the unpacking of science practices.

Learning performances are akin to learning goals that take on the structure of the performance expectations. They articulate and integrate assessable aspects of performance that build toward the more comprehensive NGSS performance expectations. A single learning performance describes an essential part of a performance expectation. Together, a set of learning performances provides the detail needed to create a coherent set of assessment tasks covering the full scope of a performance expectation bundle.

The performance expectations have been grouped into two bundles based on topic. Bundle 1 involves how photosynthetic organisms create their own food, and how both photosynthetic and non-photosynthetic organisms use food to provide energy and matter necessary for life and growth. Bundle 2 focuses on entire ecosystems, especially the interrelationships between organisms, the flow of energy, and the cycling of matter within ecosystems.


For the MS-LS1-6 performance expectation (Bundle 1):

LP P01 - Students analyze and interpret data to determine whether plants and other photosynthetic organisms grow with the input of energy from sunlight.

LP P02 - Students analyze and interpret data to determine whether plants and other photosynthetic organisms require water, CO2, and energy (e.g., sunlight) and then produce food (sugar), and O2.

LP P03 - Students develop a model that shows that plants (or other photosynthetic organisms) take in water and carbon dioxide and then form food (sugar) and oxygen.

LP P04 - Students construct a scientific explanation for how plants are able to use the food (sugar), which was produced by plants and other photosynthetic organisms, to grow and support other necessary (life-supporting) functions.

LP P05 - Students evaluate how well a model shows that plants and other photosynthetic organisms use energy from the Sun to drive the production of food (sugar) and O2.


For the MS-LS1-7 performance expectation (Bundle 1):

LP R01 - Students construct a scientific explanation to describe how animals must consume other organisms that have carbon-containing molecules in order to obtain food.

LP R02 - Students evaluate whether a model shows that when an organism takes in food and oxygen, energy is transferred to other systems within the organism, and to systems outside of the organism.

LP R03 - Students construct a scientific explanation for how consumed molecules of food and oxygen contain atoms that rearrange and form H2O, CO2, and new molecules for animal growth.

LP R04 - Students analyze and interpret data to determine whether the molecules formed during cellular respiration and other chemical reactions are used in organisms for growth, structure, and storage.

Researchers are currently working on drafting learning performances for the performance expectations in Bundle 2.

 

Partners

This project is a collaborative effort between CREATE for STEM Institute at Michigan State University, the University of Illinois in Chicago (UIC), the Concord Consortium in Massachusetts, and SRI International in California.

Project Leaders

  • Dr. Jim Pellegrino – PI, UIC
  • Dr. Louis V. DiBello – Co-PI, UIC
  • Dr. Christopher Harris – Co-PI, SRI
  • Dr. Kevin McElhaney – SRI
  • Dr. Joe Krajcik – Co-PI, CREATE for STEM, MSU
  • Daniel Damelin – Concord Consortium

School District Partnerships

  • Los Angeles Unified School District (CA)
  • Saginaw Public Schools (MI)
  • Waukegan School District 60 (IL)

Expert Review Panel for LPs

  • Ted Willard – NSTA
  • Katherine McNeil – Boston College
  • Misty Richmond – Department of Science, Chicago Public Schools

External Advisory Board

  • Bruce Alberts – University of California, San Francisco
  • Tammy Long – MSU
  • William Penuel – University of Colorado, Boulder
  • Peter McLaren – ACHIEVE, Inc.

 

Milestones

The project began in summer of 2015 and is expected to continue through summer of 2018. The development target is at least 5 validated tasks for each of the 12-16 learning performances as well as 5 validated tasks that directly assess each performance expectation (total of 80-100 tasks).


Year 1

During the first year of the project, the team performed the following activities:

  • Completed a framework from which to create assessment tasks
  • Developed learning performances that constitute knowledge-in-use statements that incorporate 3-dimensional learning
  • Identified place-based phenomena that are culturally relevant
  • Provided compelling examples of the disciplinary core ideas to ensure that tasks are equitable and inclusive to student populations
  • Identified intersecting practices to enhance student opportunities to engage with and learn the content
  • Expanded and edited the unpacking of science practices.

 

Year 2

In year 2, the team will finish writing the learning performances for Bundle 2 and then convene an expert panel that will review what they have created. They are then looking to develop about 3 assessment tasks per learning performance. The tasks will be 3-dimensional, will require students to demonstrate knowledge-in-use, and will be designed so that students can complete a task in about 5-10 minutes.

During this period, the team will be conducting both cognitive lab studies and classroom field study activities. They will also be developing support materials for teachers to use; these materials will include an explanation of the design process, assessment task rubrics, and suggestions for how to use the tasks in the classroom. All of the teachers involved in the Year 2 studies will attend a 2-day professional development workshop that covers the science Framework and the NGSS.

By the end of the first 2 years, the team will have accomplished all the major goals and objectives set out for this NGSA Life Science project. They will have:

  1. completed a thorough analysis of the disciplinary core ideas, practices, and crosscutting concepts associated with four important middle school performance expectations in life science,
  2. generated integrated domain maps for the associated knowledge and skills,
  3. generated a set of learning performances that are aligned with and encompass the major knowledge and skills associated with the target performance expectations,
  4. generated sets of assessment tasks aligned to those learning performances,
  5. made those tasks available for student and teacher use in a flexible and instructionally supportive technology delivery and evaluation platform, and
  6. done initial work with students and teachers related to the validity and usability of those assessment tasks as part of ongoing NGSS-aligned instruction in middle school life sciences.


Moving Forward

The main study will occur in Year 3 to determine the validity and reliability characteristics of the assessments. This main study will incorporate approximately 30 teachers and their students (about 600 students total). As in Year 2, all participating teachers will participate in a 2-day professional development workshop. The results of the study will provide the team with valuable information about the assessment tasks that will help them with revisions.

Given the perceived interest in the project and the value of the work underway, the team is considering applying for more funding to continue work after the end of the current grant period.

 

Design Principles

The activities that have guided the development of the materials include:

  • Develop tasks and rubrics
  • Conduct a cognitive analysis study
  • Conduct 3 student performance studies (early-stage testing, a pilot study, and a main study)
  • Generate guidelines for classroom use

The team created a design pattern for each learning performance. These patterns specify the focal knowledge, skills, and abilities (KSAs) that students need in order for them to demonstrate the learning performance. There are also multiple tasks within a “family” that vary in difficulty level, allowing teachers to assess their students’ understanding of concepts over time or even across grade levels.

Another important consideration for the task design process is the application of the equity/fairness framework to help ensure that the tasks are accessible and fair to student of diverse cultural, linguistic, and socioeconomic backgrounds.

The team iteratively refines the design of tasks using several steps:

  1. think-aloud sessions that examine whether tasks are understandable to students and whether they elicit 3-dimensional proficiency,
  2. judgments by independent experts on the alignment of tasks with learning performances and of learning performances with performance expectations,
  3. an equity/fairness review to ensure tasks reflect fair assessment design principles, and
  4. classroom studies with teachers, who provide design feedback on tasks for formative use.


Links to the Next Generation Science Standards

The materials will help students build competency in the performance expectations defined in the Next Generation Science Standards (NGSS). The assessment tasks target the following performance expectations from NGSS:


Bundle 1

  • MS-LS1-6. Construct a scientific explanation based on evidence for the role of photosynthesis in the cycling of matter and flow of energy into and out of organisms.
  • MS-LS1-7. Develop a model to describe how food is rearranged through chemical reactions forming new molecules that support growth and/or release energy as this matter moves through an organism.


Bundle 2

  • MS-LS2-2. Construct an explanation that predicts patterns of interactions among organisms across multiple ecosystems.
  • MS-LS2-3. Develop a model to describe the cycling of matter and flow of energy among living and nonliving parts of an Ecosystem.

 

Connections to other CREATE for STEM projects

The NGSA Life Science project is one of several CREATE projects working to create innovative assessments. Another endeavor, the NGSA Physical Science project, preceded this grant. For that project, researchers and assessment developers worked to design assessment tasks and rubrics for middle school physical science classrooms.

Much of the work done for that project, including the unpacking of performance expectations and 3-dimensional learning concepts, has been adopted for use in this NGSA Life Science project. This work is an asset to the science education community, as the need for Next Generation Science Assessments is great.

To read more about the NGSA Physical Science project, please click here: http://create4stem.msu.edu/node/3433