How Upswing Reduced No-Shows for Online Tutoring
As colleges around the country move their tutoring services online in response to the COVID-19 pandemic, different challenges have surfaced as a result of these new processes and technologies.
One challenge that has long existed occurs when students schedule tutoring sessions, but do not show up for the session (at Upswing we call these “abandoned sessions”). Students are busy and they might schedule an online tutoring session with full intentions of attending, but then something comes up at the last minute. Rather than canceling or rescheduling, it’s often easier for them to just not attend.
To the student, it may not seem like a big deal — they don’t lose any money and can set up a new appointment when they have time again. For tutors and the academic support department, abandoned sessions can be a frustrating and costly experience.
We’re going to cover how we tackled this issue including:
- Why addressing abandoned sessions matters
- Initial hypothesis & research
- Potential solutions
- Outcomes
- Conclusions
Upswing’s Response to Abandoned Sessions
It’s not unusual for tutors to plan their day around a scheduled session. They often hurry home from another job to make it in time, and they may have even adjusted their availability to meet the student’s scheduling needs. It can be disheartening when a student simply “no shows”.
Beyond the frustration that abandoned sessions can have on tutors, is the financial impact on the organization. At Upswing, we still compensate tutors for abandoned sessions and, depending on an institution’s policies, this can add up financially. In the year 2019 alone, abandoned sessions represented more than $25,000 in costs.
Initial Hypothesis & Research:
When we first decided to address the issue of abandoned sessions, our initial hypothesis was that this problem was driven by a small group of students who abused the system repeatedly. One idea was to create a system that would allow us to easily identify these repeat offenders so we could send communications, implement session limits and, if needed, enforce consequences to decrease the volume of abandoned sessions. We sought to verify this theory and make sure we were building the right solution, so we looked at the data.
We looked at the probability of a session being abandoned, given the student’s prior session history (Figure 1). For example, if you had completed your first two sessions and abandoned zero, your third session would fall at the intersection of 0 along the horizontal axis, and 2 along the vertical axis. Your third session would have a 5.36% chance of being abandoned, given your previous session history.
As shown by the chart, the more sessions a student abandons, the more likely they are to abandon their next session (the trend shown by the red arrow). The more sessions the student completes, the less likely they are to abandon their next session (the trend shown by the green arrow).
This seems to confirm the solution, right? Abandons lead to even more abandons.
Not exactly.
We then looked at the total number of abandoned sessions to see what percent of all abandoned sessions were a result of prior sessions completed and prior sessions abandoned (Figure 2). This chart depicts that although power users (those who completed 21+ sessions) make up a considerable amount of the overall abandoned sessions, a majority of the impact comes from new users (those that have neither completed or abandoned a session). More broadly, 73% of the total abandoned sessions are a result of students that have previously abandoned 3 or fewer sessions. In other words, most abandoned sessions are driven by casual and first-time users.
With the above in mind, we sought to understand what caused first time users to abandon those sessions. We broke down the abandonment rate for first time-users (the cell at 0,0) by how far in advance the student scheduled the session.
We learned that the farther in advance a first-time user schedules a session, the more likely they are to abandon the session (Figure 3). This finding, combined with data from a student survey, made it clear that students were simply forgetting or running into scheduling conflicts.
Potential Solutions
With these initial findings in mind, we brainstormed potential solutions. These included:
- Sending a reminder to students 24 hours before their scheduled session
- Having tutors send a personalized message to students after a session is scheduled to establish a personal connection
- Socially validating the concept of tutoring by sharing statistics of how many of the student’s peers also received tutoring
- Monitoring sessions in order to discipline repeat offenders
And the winner was: sending a 24-hour reminder notification to students before their scheduled session! 🎊
The other solutions had more moving parts. Establishing a relationship between the student and tutor would require cooperation from both sides. Monitoring abuses of the system would not only take effort from someone on our team, but would convey to students that they were being micromanaged and disciplined. This may discourage students from reaching out for help in the first place — something we want to avoid above all else.
Within the experiment, we tested two variations of the reminder email. One version had a reminder with the session information, while the other also included a “reschedule” button that led students back to the platform to cancel the original session and schedule a new one.
We launched the experiment in November 2019 and ran it through March of 2020 to ensure that we captured a wide spectrum of student users; for example, during November we see a lot of “Finals Crammers”, while at the beginning of a semester, we see a lot of first-time and proactive studiers. We wanted to run this test long enough to cover that entire range, from finals to midterms.
Outcomes:
We found the experiment group that received either notification had a statistically significant, 17% lower abandonment rate compared to the control group.
What does 17% represent in terms of financial and student impact?
For Upswing, this reduction in abandoned sessions represents several thousand dollars in cost savings every year. This means that not only do we save money by not having to pay tutors for waiting in abandoned sessions, but that their time can be used to meet with students who do show up for their tutoring sessions.
Conclusion & Key Takeaways:
By simply reminding students of upcoming sessions 24 hours in advance, Upswing was able to reduce the abandonment rate by approximately 17% without any meaningful change in the cancellation rate.
In addressing the issue of student abandonments, we learned:
- Abandonments come mostly from first-time and heavy users, not so much the casual users
- The farther out a new user schedules a session, the more likely they are to no-show
- A 24-hour reminder can make a sizable difference in the rate at which students abandon sessions and takes far less time and effort than other solutions
About Upswing
Upswing is an online tutoring company that partners with colleges and universities around North America to help bring tutoring and advising online.
Emily Andre manages Upswing’s tutor community. She worked closely with Upswing’s Data Analyst, Arvind Bala, to write this article.
If you have questions about Upswing or this study, please reach out to Emily at emily.andre@upswing.io.
