LibGuides: Open Education Resources: Measuring Success

Measuring Success of the Use of OER

According to David Wiley, the key metrics for measuring the impact of open educational materials o student outcomes are SUCCESS, SCALE, and SAVINGS. In other words, how much can you improve student outcomes, for how many students, and how much money can you save students while doing so.

Guidebook to Research on OER Adoption
The purpose of this guidebook is to provide ideas for how individual faculty members and those who support them (e.g., librarians, instructional designers, etc.) can research the effect of their adoption of open educational resources (OER).
OER Research Toolkit (including Surveys) The OER Research Toolkit is comprised of the OER Research Guidebook and several additional resources, including surveys for both students and instructors who are using OER in the classroom. The purpose of this guidebook is to provide ideas for how individual faculty members and those who support them (e.g., librarians, instructional designers, etc.) can research the effect of their adoption of open educational resources (OER). Clearly educational research is a challenging enterprise; this guidebook is not meant to replace the substantive courses and experiences that a PhD in educational research would provide. Rather, our hope is to provide some straightforward suggestions that could be implemented by OER adopters so as to help them identify what has happened as a result of their OER adoption.

Defining Success

Success: Completing a course with a final grade that allows the course to count toward graduation.

Measured use a grade C or better from before OER were used. Using an average of multiple previous terms is a more stable measure than a single prior term. C or better grade after the OER began being used (using an average of multiple terms rather than a single term.
The max value for success is 1, and this occurs only when the C or better grade is 1 for OER users. This encompasses both the idea and the goal that each and every student should succeed in the course. The value is undefined when the C or better rate is 1 for control students, as it is not possible to improve success in this case.
How much does this innovation improve student success?

David Wiley (n.d.). Licensed under the Creative Commons Attribution 4.0 International license.

Defining & Measuring the Scale of Success

Measuring Success: The proportion of students being reached.

Scale = Number of Students in OER Sections / Number of Students in All Sections

How many of our students are benefitting from this innovation?

David Wiley (n.d.). Licensed under the Creative Commons Attribution 4.0 International license.

Defining and Measure Savings

Savings: The amount of money spent on course materials by the average student using OER compared to the amount of money spent on course materials by the average control student.

Savings = Average amount spent control - Average amount spent on OER / Average amount spent control

How much money does this innovation save students?

David Wiley (n.d.). Licensed under the Creative Commons Attribution 4.0 International license.

Impact = 4 x Success + 2(Scale) + 1(Savings)

How much does this innovation improve student success?
How many of our students are benefitting from this innovation?
How much Money does this innovation save students?

David Wiley (n.d.). Licensed under the Creative Commons Attribution 4.0 International license.