Automated Analysis of Student Writing

Diagram

The Automated Analysis of Constructed Response (AACR) Research Group is a collaboration of researchers from seven universities with backgrounds in various STEM disciplines, linguistics, and educational research.  The project is exploring a computerized analysis of students’ writing in large enrollment undergraduate STEM courses. We are pursuing multiple approaches to using linguistic analysis software for identifying conceptual categories in students’ written responses and using that data to build statistical models of student thinking in biology, chemistry, and statistics.  These techniques have the potential for improving assessment practices across STEM disciplines.  

Constructed response questions – in which students must explain phenomena in their own words – create more meaningful opportunities for instructors to identify their students’ learning obstacles than multiple-choice questions.  However, the realities of typical large-enrollment undergraduate classes restrict the options faculty have for evaluating students’ writing.  

In the AACR group, we have created computer models that can predict expert ratings of student responses with accuracy approaching inter-rater reliability among expert raters. These techniques also provide insight into students’ use of analogical thinking, a fundamental part of scientific modeling.  These techniques have the potential for improving assessment practices across STEM disciplines.  

Our collaborators are:  

State University of New York – Stony Brook

University of Colorado – Boulder 

University of Maine 

University of Georgia 

University of South Florida 

Western Michigan University 

Collaborators on other AACR-related projects:

University of Washington (LeaP UP)

Stanford (Argulex)

BSCS (PCKLex)

Significant contributors to AACR work (in addition to all of the above):

Middlebury

California  State Stanislaus

South Dakota State University

 

Associated Projects