Team taking an innovative, data-driven approach to course design


A Penn State team is looking at some of its graduate courses by studying data that shows how students learn and behave. They hope to use their findings to rethink courses in ways that help students become more successful online learners.

The team of instructional designers, faculty, and teaching support specialists analyzed these learning analyzes from two years of coursework in Penn State’s online MBA program, which is offered through Pennsylvania State Global Campus and led by the Smeal College of Business.

The data consists of a wide variety of activities, from the links students click to the depth and quality of the posts they write on discussion boards. The data will be used to guide an innovative approach to course design, aligning the main learning objectives of the program with how the team wants students to act and engage with the courses.

The work in online master’s of business administration (MBA) courses is the first of its kind at Penn State and among the first in online higher education in the country.

“The data represents a study of student behaviors based on the unique graduate teaching and learning experience and tells us a story about what is happening in our courses,” said Janet Duck, President from the online MBA faculty that championed the project.

“We can use this story, through deliberate course design, to inspire student behavior that reinforces course skills and provides an engaged experience for our professional students.” “

This screenshot from the MBA Learning Analysis shows an analysis of the data.

A new look at the courses

With nearly 1,000 students, the Penn State Online MBA is the most registered graduate program offered by the Penn State World Campus.

The program is led by Smeal College and includes a consortium of faculty from Penn State Behrend, Penn State Harrisburg, and Penn State Great Valley School of Graduate Professional Studies. It has consistently been ranked among the top 10 online MBA programs nationwide by US News and World Report.

Over the past two years, the team has collected data on the top 13 online MBA courses in Canvas, the university’s learning management system, and Zoom. Data is visualized on a dashboard using charts, tables and charts.

Jessica Resig, senior instructional designer for business programs offered by Penn State World Campus, said this approach allows the team to see student behaviors in new ways.

For example, the data can show trends, such as whether students are struggling with a concept, as evidenced by low scores on the same assessment each semester. Or they could see if students choose to watch videos that are included in their weekly classes but are not required.

“These kinds of things can give us clues as to where students are spending their time or where they engage with our material, to help us decide if the content meets their needs,” Resig said. “It’s a full snapshot that’s presented in a way we didn’t have before. “

Resig said student privacy is paramount in data collection.

“Our goal is to understand the effectiveness of each course and the design decisions we make, not to focus on individual course participants,” Resig said. With this in mind, we aggregate all data used as part of the course analysis process and have a strong set of ethical and procedural standards in place to ensure student privacy and data security are strictly maintained. “

This screenshot shows data on how students interact with various content in online MBA courses.

This screenshot shows data on how students interact with various content in online MBA courses.

Rethinking course improvement

The designers hope that learning analytics will allow them to make continuous improvements to online MBA courses as and when needed instead of waiting for courses to be updated every few years.

Duck said that while the team has just started analyzing the data, the learning analyzes are already indicating improvements and a more holistic understanding of teaching and learning.

For example, studying the depth of responses on discussion boards shows that when students write long messages on discussion boards, those messages tend to attract fewer responses. The data shows which pages of content students visit the most and where student engagement tends to drop, Resig said.

“Are there ways to redirect them to the content of this course?” Said Resig. “Can we rethink the way content is presented to make it more valuable to them? “

Duck said the data is used to guide behavior in other aspects of daily life, and she advocates for the approach to be applied in higher education.

“We know that our professional graduate students want to participate in their learning experience,” said Duck. “We know they want to be able to immediately apply what they’ve learned, and they want to understand the big picture, both in terms of themselves, the company and the industry. We want engaged, applied and integrated learning, and data can help us do that.

A bar graph shows student collaboration on a discussion post in the online MBA program

This screenshot shows the data on the discussion messages of the students of the online MBA program.

Spreading the word outside Penn State

Resig and Duck are sharing their knowledge with their peers in the e-learning arena across the country, and Duck said she was receiving inquiries about these efforts.

Resig and Chris Millet, senior director of learning design for Penn State World Campus, led a virtual presentation in September on learning analytics at the Online Learning Consortium’s OLC Accelerate conference, one of the largest meetings of online education practitioners. Resig and Millet have written two book chapters in scholarly publications on this use of learning analysis.

Duck and Resig will host a virtual presentation on the subject in November at the Quality Matters Connect LX conference.

Resig said that another group of online MBA courses will be revised next fall, and the last group of the program will be revised in fall 2023. After that, the team plans to use the table of edge to spot trends or warnings in learning analysis and make continuous improvement of course content.

This is why the data-driven approach to learning design has the potential to be so transformational.

“It allows us to be smarter and more intentional in our design decisions going forward,” Resig said.

Visit the Penn State World Campus website for more information on online learning.


About Stuart M. McFarland

Check Also

Free smartphone filmmaking course comes with Zacuto Smart Z-Finder

In partnership with Rob Kaczmark and MZed, Zacuto expresses their gratitude to all supporters of …