Author Archives: Rahim S. Rajan

A Major Study of Online Learning at Public Universities – The ITHAKA Study

Earlier this week, ITHAKA released its findings from a randomized study at 6 public universities of Carnegie Mellon University’s Open Learning Initiative’s Statistics course, as delivered in a blended format. Their research found no credible differences in course completion or student learning outcomes between the students who took the face to face or blended course. A link to the full report is here.  While the Gates Foundation didn’t sponsor or fund the research, the results of this study have important implications for our the technology enabled learning and innovations work that we fund broadly via our Next Generation Models team or the Next Generation Learning Challenges partnership.  While the results of this study aren’t necessarily novel – they do emphasize once again that the development of intelligent learning technologies or digital cognitive tutors are at an early stage. While there are many caveats that even the researchers point out (i.e., this is only a study about one OLI course), this research is of tremendous importance to those of us who care about evidence based policy and change.

Some highlights:

  • The cost and level of implementation difficulty in executing this type of a randomized research experiment in higher education are high. While this experiment involved six campuses, because of complex institutional research requirements and the voluntary nature of participation in the study by students, ITHAKA was only able to put together a sample size of 605 students (total student enrollment in the course across all six campuses was over 3,000 students).  This sample size is nevertheless statistically significant.  In comparison, only 5 studies cited in the oft-cited DOE meta-analysis on online learning had sample sizes over 400 students.
  • ITHAKA worked with six public universities to create “control” and “treatment” groups at each institution to compare and measure student learning outcomes between a intro statistics course taught in the traditional face to face method vs. a course taught with OLI (coupled with a once a week face to face meeting).  The institutions that participated in the study were: CUNY Baruch College and CUNY City College, SUNY Albany, SUNY Institute of Technology, UM Baltimore County, and Towson University (also part of the University of Maryland system). While these institutions don’t represent an effective sample of all American higher ed, they do represent populations and demographics have both large commuting and residential populations, large numbers of minority students and students from low socioeconomic status families.  No community colleges were able to participate in the study despite ITHAKA’s attempts to include them.
  • Of great importance: the study found “no statistically significant differences in learning outcomes between students in the traditional  – and hybrid format sections…we can be quite confident that the ‘average’ effects were in fact close to zero.” Even when a variety of other potentially significant factors amongst various subgroups were evaluated by the researchers (i.e. student data vis a vis race/ethnicity, gender, parental education level, college GPA, number of hours worked for pay), they still found that “on average, students learned just as much in the hybrid formats as they would have had they instead taken the course in the traditional format.” Even across the various institutions they didn’t see large differences in outcomes. Instead, the research did reveal that students who took the blended course spent an average of 25% less time completing the course (so similar learning outcomes as the face to face course in 25% less time).

A conversation about a conversation….

Last week I attended a symposium at MIT on the topic of quality in online learning.  In addition to visiting this amazing local museum and an opportunity to hear the talents of MIT’s oldest vocal ensemble, I had a chance to participate in a whirlwind 48 hour idea and design fest devoted to quality in online learning. Jointly organized by our colleagues in the College Ready team and  MIT’s Office of Educational Technology and Innovation, the meeting brought together about 80 individuals (grantees and non-grantees that included researchers and faculty, representatives from online learning solutions and tools, and representatives from grantee school districts and CMOs). The stated goal of the symposium was to “launch a national conversation on the quality of online learning in K-12 education.”

The meeting agenda, list of participants, and presentations have been posted online. In addition (if you’re interested) you can also view the twitter stream from the entire 2 days.The meeting was organized into a half day of presentations. Panels included a number of CRW grantees including (Arizona State University, Creative Commons) as well as representatives from industry and the broader online learning sector (Bror Saxberg from Kaplan and Judy Codding from the Pearson Foundation). The panels focused on three broad topics:

  • Immersive Gaming / Focused Tutoring
  • Quality Assessment: Assessing Course Quality
  • Scaling Quality

The symposium also brought together a student panel consisting of four high school students, two teachers from Highline School District and Bellevue School District, as well as a team of researchers led by Dr. Phillip Bell from the University of Washington’s College of Education. The student/faculty panel discussed in detail their use of Educurious in the classroom. A core highlight from this session was learning first-hand how the resource enables students to tap into a network of accomplished and seasoned professionals (as thought partners and information resources) over the course of completing particular assignments and research problems. It was also really interesting to hear how teachers felt “relieved” by having a resource like Educurious to depend on because it reduced the pressure that they felt “relieved for not having to know everything” and more importantly, were now empowered to connect students to other external resources that could provide students with the expertise and information that they needed. The students’ excitement and engagement in their learning and the process was also palpable and very cool.

In a talk titled the “Opportunity and Challenge of Pervasive Quality”,  Jim Shelton pushed us all to create more durable and lasting learning innovations that benefit our students, faster. Mr. Shelton first spoke about two recurring challenges regarding innovation in education:

  • A tendency to think learning will happen because of innovations in hardware (i.e., magical device + student = improved student learning and mastery)
  • A tendency to focus on the perfect rather than the good –  we tend to focus on “what would be great”) and swirl in circles rather than focusing on “what can be great”

While he acknowledge  the existence of weighty academic debates regarding the attributes and characteristics of a high-quality online learning experience, Mr. Shelton (ever the pragmatist) asked us to focus on the most important factor. Find, test, build, and scale learning experiences and environments that first and foremost deliver better student learning outcomes and mastery. Online learning should be about learning – not bells and whistles.

Learning. Plain and simple.

P.S. I expect in a couple of months that we will actually see the outcomes and learning from the workshop delivered back to the public for further discussion.

P.S.S. A special shout out to colleagues in our College Ready team for organizing such a great symposium.

Tagged , , , , , , , , , , , ,

Breakthrough educational models that will lead us to a new normal

EDUCAUSE, via the Next Generation Learning Challenges (NGLC), recently announced a third wave of funding for higher education, to fund breakthrough models for college completion. The goal of this RFP is to seed the development and creation of new post-secondary models that can answer the question of whether learning science, technology, policy, and organizational processes have advanced far enough to enable quality Associate’s and Bachelor’s degree programs to be delivered at a cost of $5,000 – our target goal –  per student per year.

This wave of funding will award six grants of up to $1,000,000, disbursed over 2 years to educational models that dramatically increase personalization, engagement, and efficiency using the best of what is
exists in learning science and innovative technology. Funding is available to public, private not-for-profit, and for-profit institutions of higher education accredited to offer associate’s degrees, bachelor’s degrees, or both. A variety of partnerships may also be considered by challenge panel including but not limited to public private partnerships, 2-year and 4-year partnerships, or even state or regional consortia.

So why this and why now?

Educators and policy makers have known for decades that dramatic learning gains are possible if we
can consistently scale the best of what we know about student learning and success. For example, one size fits all learning has been shown to be effective only for a minority of students. Benjamin Bloom’s research has demonstrated that there is great potential in the application of technology to realize the potential “mastery learning” outcomes of 1:1 tutoring. The seemingly endless rise in costs of higher education combined with the dilemmas posed by what has been described as an “Iron Triangle” poses
a myriad of challenges for our nation. There is credible research that points to how dramatic gains in college completion would have significant positive benefits for our economy and on our nation’s unemployment. Direct benefits – not ambiguous or unclear benefits. We have complicated
problems that require tremendous creativity and bold thinking to be solved – and that is why we’re funding EDUCAUSE and the broader NGLC partnership to NGLC’s work is so valuable.

One of our unique roles as a private foundation is to help the post-secondary community test and refine practices and solutions that can meaningfully increase both affordability and the quality of today’s post-secondary educational programs. We know that the answers to many of these problems remain elusive and seem too difficult to solve. We also know however that there exists a growing community of educators and innovators who can and will rise to this challenge, guided by a shared set of aspirations:

  • Education should be learner-centered
  • Active, situated, and experiential learning, best measured by mastery, improves engagement, problem solving, and achievement
  • Information technology can be an enabling force for learning and for learners
  • Evidence and data should drive innovation and decision-making
  • Change requires that scale be built into innovation design

For more on the Next Generation Learning Challenges, please visit the NGLC website.

Tagged , , , , , ,