10 Metrics

Metrics play an important role in monitoring OER’s growth and impact on campus. Metrics may be used for basic reporting and monitoring purposes; they also can provide additional context useful in storytelling that highlights OER’s impacts on students and the campuses. Selecting the appropriate metrics depends on the information required and the purpose it will serve:

  • What do institution and system leaders want to know about OER?
  • What information will encourage faculty and students to transition to openly licensed materials?
  • What can OER advocates share that will help build ongoing support from institution, System, and state leaders?

Since student savings are often the primary motivation for developing OER courses, the savings from eliminating or reducing commercial textbook purchases is a key metric. Traditional measures of textbook savings assume all students would have purchased a new textbook and therefore reflect “maximum” student savings. More realistic estimates adjust for student purchasing patterns, including utilization of used or rental texts, or forgoing textbook purchases altogether, as well as shopping at lower-cost online retailers. These adjusted estimates should also capture any additional costs students may incur from the optional purchase of printed OER textbooks.

Student success-related metrics can be used to show OER’s impact beyond just student savings. For example, comparisons of student performance in OER and non-OER sections can help departments diagnose any imbalances across OER and non-OER course sections. Differences in DFW rates can also be analyzed to determine if OER courses are generating any departmental efficiencies by reducing the number of students that need to repeat courses. However, the basic calculations comparing student outcomes should not be used to make causal inference about the efficacy of OER without accounting for other factors, such as student characteristics and preparation.

Institutions may also choose to administer surveys to collect additional contextual information about students’ and faculty members’ perceptions and experiences with OER.

WTCS colleges are required to report (Client Reporting) adoption of the OpenRN Open Nursing Textbooks as part of the Department of Education grant reporting. Colleges can also submit the OER adoption code for other course sections. Related to this requirement, campuses first need to ensure the platform and procedures are in place to identify and report basic OER course and enrollment information. For more information, see the WTCS OER Client Code Guide.

The WTCS OER Network also developed a scale of adoption assessment for open and affordable educational resources. This assessment tool can help colleges identify areas of strength and need for developing OER work across the district.

OER Metrics: Basic and Advanced Calculations 

Metric Basic Calculation Advanced Calculation
Implementation Number of OER courses and sections, and percent of total courses and sections. Number of OER enrollments and percent of total enrollments Basic metrics calculated for: Department (or college), Gatekeeper courses, High-enrollment courses, Student characteristics (e.g., Pell students)
Student Savings Maximum student savings (OER enrollments x new textbook cost) Adjust to reflect textbook purchasing patterns and additional costs: Use blended textbook price (total new, used and rental sales ÷ total units sold).  Exclude the proportion of students who do not purchase textbooks (estimate using information from own or other student surveys. Adjust for proportion of students purchasing lower-cost used textbooks online (own or other student surveys; literature on used textbook prices). Add offset for printed copies of OER materials purchased
Student Success Number of DFWs and DFW rates in OER/non-OER sections. Average grade in OER/non-OER sections (Use for diagnostic purposes only; differences may be attributed to student/faculty/course characteristics) Adjust for student/faculty/course characteristics (e.g., grade point average; socio-economic indicators; type of OER)
Perceptions Student survey (e.g., OER awareness, quality, textbook use, financial savings, etc.). Faculty survey (e.g., availability and quality of resources; development time; professional development and support, etc.) Additional OER-related topics of interest


Icon for the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License

WTCS OER Field Guide for Sustainability Planning Copyright © by WTCS OER Network is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, except where otherwise noted.

Share This Book