I’m preparing for a talk with UCF data science students later this month on the statistical and machine learning tools we use for our Disney World wait time predictions.
After covering the basics – how we collect wait times, the hundreds of things we consider (everything from Extra Magic Hours to public school holidays, to the state of Brazil’s economy six months ago) – I wanted to say a little about how to handle the situation where you know in advance that Disney’s posted wait times will be wrong.
I’m comparing Buzz Lightyear’s actual wait times and posted wait times using the graph below. If the time you actually spent in line was exactly the posted wait time, every red dot shown below would fall on the black diagonal line.
As you can see, the red dots do not all fall on the diagonal. Some aren’t even close. The dot in the lower right that I highlighted is an example of someone waiting 5 minutes to ride Buzz Lightyear when the posted wait time said they’d wait 70 minutes.
The other dot I highlighted, in the upper left, represents someone actually waiting 65 minutes when the posted wait said they’d be there for 30 minutes.
The average difference between the actual wait time and the posted wait time at Buzz Lightyear is around 40% – roughly 8 minutes, plus or minus – out of 20 minutes (the average posted wait at Buzz). But as you can see, it varies. A lot.
Even a Crystal Ball Wouldn’t Help
Why is this important? Because it’s a kind of benchmark on how accurate a prediction can be.
Suppose you have a crystal ball that tells you exactly how many people will be in the Magic Kingdom tomorrow, and exactly when they’re planning to visit Buzz Lightyear. You still couldn’t predict the posted wait time at Buzz Lightyear any closer than +/- 8 minutes on average.
It’s clear from the graph above that the posted wait time at Buzz Lightyear seems to be based on something other than facts like the number of people in line, the Extra Magic Hours schedule, the economy, school calendars, and so on.
There are a couple of possible explanations here. Maybe we’re missing some critical data – Disney’s stock price at that instant in time, whether Mercury’s in retrograde – whatever.
Another possibility is that posting a predictable wait time just isn’t something Disney can achieve. You can certainly estimate, for example, how many people will show up to use FASTPASS+ at Buzz Lightyear in any hour-long time window. But the actual number of FASTPASS+ users who show up will still vary considerably. If Disney thinks it has already done as good a job as it can with posted wait times, then we’re likely to continue seeing what we’ve been seeing.
The good news – for the talk I’m giving to these budding data scientists, and for all of us – is that these wait times seem to behave in predictable ways in some recognizable situations.
For example, we’ve noticed that when Space Mountain closes unexpectedly, Buzz Lightyear’s actual wait time tends to be much closer to the posted wait time, because people in the area around Space Mountain will find Buzz and get in line. By identifying the sets of circumstances in which the relationship between Buzz’s posted wait and actual wait are firm, we’re able to chip away at that 8-minute margin.
Here’s how close our Buzz Lightyear predictions have been for the past few months:
|January 2017 MTD||11.2|
There’s still work to be done, I think, but overall this is good.