Helping students learn by sketching

“Sketches are difficult and time-consuming to grade,” said Northwestern University’s Ken Forbus. “Intelligent tutoring systems, which enable students to receive feedback on their work anywhere and anytime, rarely are capable of understanding sketches.”

Forbus, Walter P. Murphy Professor of Computer Science in Northwestern’s McCormick School of Engineering, and his team have developed a new solution called “Sketch Worksheets,” a software equivalent of pencil and paper worksheets commonly found in classrooms. The difference? The software can also provide on-the-spot feedback by analyzing student sketches and then comparing them to the instructor’s sketches.

An instructor might ask students to draw the chambers of the heart, for example. If a student misplaces an atrium, then he or she is immediately alerted to the mistake by the Sketch Worksheet.

Supported by Northwestern’s Spatial Intelligence and Learning Center (SILC), the research was recently published in the journal Topics in Cognitive Science. Forbus was the paper’s first and corresponding author.

Sketch Worksheets software is based on CogSketch, an artificial intelligence platform previously developed in Forbus’ laboratory. A sketch-understanding system and high-level model of human vision, CogSketch uses visual processing algorithms to automatically reproduce and understand human-drawn sketches.

Sketch Worksheets’ comparisons of student and instructor sketches is carried out by an analogy model, developed in collaboration with Northwestern psychology professor Dedre Gentner. Students and instructors apply conceptual labels to their sketches to express relationships among the drawings’ different parts. (For example, Earth’s core is inside the mantle, or the heart’s aorta is above the left atrium.) Without needing a deep understanding of the sketch’s subject matter, CogSketch uses analogy to compare the labels and provide feedback.

One of Forbus’ main goals is for Sketch Worksheets to be accessible to instructors in any field — not just computer science. To ensure this, his team and SILC collaborators tested the software on more than 500 students in biology, geoscience, and engineering, ranging from the fifth grade through college. A team of geoscientists at the University of Wisconsin-Madison have already used the software to develop a set of 26 sketches that cover topics in introductory classes. These worksheets are publicly available and have been used in classes at the University of Wisconsin and Northwestern.

“We hope that others will follow the lead of the geoscientists and create Sketch Worksheets to help their students learn,” Forbus said. “This is a step in creating software that can communicate with people as flexibly as we communicate with each other.”