- Lea's Blog

Universities departing standardized tests, use rubric to assess students
Illustration by Olivia Falcigno/Daily Free Press

An assessment method tested by 59 colleges and universities could be used in K-12 schools as well

Extracts from an article by Katrina Schwartz

Right now, some universities require a small sample of their students to take a standardized test before graduating, but many administrators and faculty find this method problematic. Students have no personal investment in the test, and it is divorced from the coursework that they see as their primary objective while in college. Also, not everyone does well on tests, but they may shine in their coursework.

Concerns over the effectiveness of standardized tests prompted the Association of American Colleges and Universities to begin working on a rubric-based alternative that is consistent and valid.

First, they set out to define the essential learning outcomes that faculty, employers and accreditors saw as important. They settled on 16 qualities, some of which are: critical thinking, writing, quantitative literacy, oral communication, ethics, teamwork, intercultural understanding, and integrating learning from one area to another.

For the first-year pilot study they focused only on three of those outcomes: written communication, critical thinking and quantitative literacy. The faculty worked together to write rubrics (called Valid Assessment of Learning in Undergraduate Education or VALUE rubrics) that laid out what a progression of these skills looks like. The rubrics were tested on campuses and rewritten three times before reaching a final version.

In a pilot study of the rubrics, 127 trained scorers evaluated 7,000 samples of student work across a variety of disciplines. Because they were grading the cross-cutting skills of written communication, critical thinking and quantitative literacy, faculty evaluated work from disciplines that were not their own.

“These rubrics are designed to be cross-disciplinary,” explained Bonnie Orcutt, associate professor of economics at Worcester State University and temporarily the director of Learning Outcomes Assessment for the Massachusetts Department of Higher Education.“ I can look at something and have no idea if the content was correct, but that’s not what I’m looking for. Independent of whether the content is correct, they may have used a body of evidence really well, have good organization, good syntax, good citations.”

In other words the facts might be all wrong, but the person is a good writer, which is what the scorer is trying to evaluate with this rubric.

Bridging the gap to K-12

So far, this rubric work has been happening only at two- and four-year universities. But the conversation happening in higher education isn’t so different from that going on in K-12 schools. Parents and teachers are pushing back against blunt assessment instruments like standardized tests, and are looking for a way to hold schools accountable that doesn’t mean taking time away from class work.

Many K-12 educators and parents would like to see a similar type of system in their schools. Many welcome assessment and see the need to make sure kids are learning, but they’d like to see those evaluations happening based on the work students produce for class in context that they care about.

No feedback yet

Lea's Learning Analytics Blog

Learning analytics, educational data mining, formative assessment - all recent buzz words in educational research. In principle, the idea is to find theoretical frameworks, models, procedures, and smart tools to collect, aggregate, analyze, reason on and visualize large scale educational data. LEA’s BOX is a research and development project funded by the European Commission. The project aims at (a) making educational assessment and appraisal more goal-oriented, proactive, and beneficial for students, and (b) at enabling formative support of teachers and other educational stakeholders on a solid basis of a wide range of information about learners. That means, LEA’s BOX is a learning analytics toolbox that is intended to enable educators to perform competence-centered, multi-source learning analytics. More info at http://www.leas-box.eu!


powered by b2evolution