Are Wait Times More or Less Accurate For Individual Lightning Lane Attractions?
Disney World has a long history of inflating posted wait times. We’ve written articles and recorded videos talking about this behavior and why, to a degree, it’s necessary. Disney has to control crowds to a certain extent, and they are motivated to under-promise and over-deliver in order to make happier customers. It can be frustrating to know that people are being misled on purpose, but it wasn’t necessarily going to negatively impact anyone’s vacation. But with the advent of Genie (and, specifically, Genie+ and ILL), this sort of wait time inflation becomes a whole new kind of problematic. Now, on top of issues of crowd control and customer satisfaction, Disney is financially motivated to make customers think standby waits are longer than they actually are. Higher posted wait times may encourage you to think more positively about paying $15.98 per person to use Genie+ for a day, or to drop another $15.98 just to skip standby for Rise of the Resistance, or any of the other Individual Lightning Lane Attractions.
Disney has put out messaging that wait times shouldn’t be used for purchase decision-making. So there’s that. But given that the current posted wait time is displayed pixels away from the option to purchase your way around that wait, it’s unrealistic that most humans aren’t going to let that factor into their decision. It’d be nice to at least know that Disney is being responsible with their posted wait times – maybe getting more accurate, especially on attractions where it matters the most. Or at least not getting worse. So since we’ve had nearly a month of data-gathering, let’s see if that’s the case.
Explain the Math!
In order to judge the “fairness” of posted wait times, we need to have comparison groups. These are ways to slice and dice all of the standby wait time data to see when/where it might be different. To get meaningful information in this analysis, we’ll want to slice two different ways, to give us four comparison “groups”:
- Pre-Genie vs Post-Genie
We want an equivalent number of days pre-and post-Genie launch to see if wait time inflation behavior has changed. Certainly, we’d hope that overall it has gotten more accurate post-Genie, but at the very least, we’d want to see that it’s not significantly less accurate.
- Individual Lightning Lane Access (ILL) vs Non-ILL
Similarly, the biggest out of pocket cost for WDW visitors would come from purchasing ILL access for two rides each day. Just skipping Rise of the Resistance currently costs the same as a whole day of Genie+, and there’s reason to believe that cost could easily be increased. So, ideally, those ILL attractions are where Disney would be focusing its effort to best predict and accurately reflect the actual standby wait time. Logistic realities mean that there will always be some inflation in the posted times. But it’d be nice to see less inflation where pay-your-way-around is an option, compared to other attractions.
The Pre/Post Genie slice could prove slightly problematic, because there’s not a direct comparison there. Crowd levels were different in late September and early October than they were in late October and early November. Higher wait times generally means that Disney proportionally increases their posted wait times even further. So to account for that, we’ll introduce another “control” group of data, which is standby wait times from the exact same dates in 2019. 2020 was an anomaly, so we don’t want to use that as our comparison.
The metric we’ll be interested in here is the Actual Wait Time (the wait time the user timed from when they entered line until when they got in the ride vehicle) divided by the Posted Wait Time (which is captured when the user enters the line). If everything were perfectly predictable, the Posted Wait Time would be exactly equal to your Actual Wait Time, and that means Actual/Posted = 100%. In a totally wacky scenarios where every ride was a walk-on (essentially a 0 minutes wait, plus any walking and pre-shows), but Disney was still wildly inflating Posted Wait Times, Actual/Posted would be 0%. And Actual/Posted could go above 100% if you ended up waiting longer than what was posted. To summarize:
- Actual/Posted < 100% : The Posted Wait Time is longer than your Actual Wait Time
- Actual/Posted = 100% : The Posted Wait Time exactly predicts your Actual Wait Time
- Actual/Posted > 100% : The Posted Wait Time is shorter than your Actual Wait Time
Are Posted Wait Times More Accurate Post-Genie?
Let’s start with the good news! In the month leading up to the launch of Genie, actual wait times at non-ILL attractions were averaging just 57.3% of what was posted when the user entered line. That means, at those attractions, you could expect your wait to be almost half of what was posted when you entered line. In the month since Genie launched, actual wait times have averaged 62% of what was posted at the same attractions. By looking at the number of wait times submitted, and the standard deviation of the proportion, we can also get a confidence interval for each of those averages. Here, the confidence intervals don’t overlap. That means we can be significantly confident that posted wait times are more accurate than they were pre-Genie at non-ILL attractions. 62% still isn’t great – there’s a lot of room for improvement to get closer to 1. But it’s improved – inflation isn’t statistically as much of a problem.
Unfortunately, that’s pretty much where the good news ends. Over the same time period, we’ve also been collecting actual wait times for ILL attractions. Remember, these are the attractions where you’re given the option to buy your way around the standby line immediately below where you’re told what the posted wait time is. The more inflated that standby time is, the more people are going to be convinced that they should pay their way out of it.
Pre-Genie, posted wait times at ILL attractions were much more accurate than their non-ILL counterparts. In the month since Genie launched, that all changed. Statistically, posted wait times at ILL attractions are no longer any more accurate than posted wait times at non-ILL attractions. And they’ve gotten significantly less accurate compared to pre-Genie. That means, on average, there is a lot more inflation of posted wait times at ILL attractions compared to when you couldn’t pay your way around the standby line.
To give an example of the problem – let’s say that your actual wait in standby for Rise of the Resistance was 60 minutes. If you visited in early October and that was your actual wait, the posted wait time when you entered the line was probably something like 80 minutes. That’s a decent amount of inflation, but it could be necessary because of operational issues and unpredictability. But in late October or early November, that same actual wait of 60 minutes would have meant a posted wait time of something closer to 95 minutes. Same exact actual wait, with two rather different sets of expectations based on what was posted. Let’s look at some data from specific attractions to explore this further.
What Are Some Attraction-Specific Examples?
Frozen Ever After
At EPCOT, Frozen Ever After is one of the Individual Lightning Lane attractions. It’s not immune from the decreasing accuracy of posted wait times in the era of Genie. For this example, let’s say your wait at Frozen Ever After from the time you got in line until you boarded your boat was 30 minutes. In early October, that means the posted wait when you got into line was close to 40 minutes. But in late October or early November, that same 30 minute wait would’ve meant a posted wait time of closer to 50 minutes when you got into line. Would the difference between a 40 minute wait and a 50 minute wait mean you would decide to pay to skip standby? Maybe not. But it’s still misleading.
Runaway Railway is an attraction that has a pretty steady capacity, and should therefore have rather predictable wait times. But it too has seen a decrease in posted wait time accuracy with Genie. For this example, we’ll assume a 45 minute actual wait for Runaway Railway. In late September or early October, that actual wait would have meant approximately a 60 minute posted wait. But in late October or early November, it would’ve been more like 70 minutes. Now we’re getting into the realm of tipping decision making. Maybe I’d wait for an hour in standby, but anything over an hour I know my kids will get antsy and I consider paying my way around it. But if I pay to skip 70 minutes of line, it’d be unfortunate if I found out that I really would have only waited 45 minutes. I still save those 45 minutes, but not the 70 that I thought I would.
Seven Dwarfs Mine Train
Time for our last example, and it’s an eye-opening one. Seven Dwarfs Mine Train has long held the least accurate posted wait times of any of the attractions that are now Individual Lightning Lane Access. Even before the advent of Genie, if you actually waited for 60 minutes in line, that meant that on average the posted wait time when you entered line would have been 110 minutes. 110 minutes! For an hour long wait! So surely since there was so much room for improvement, things would have gotten better with Genie. Yeah? No. Instead, now that same 60 minute actual wait will mean that when you make the decision whether to get in line, your app and the sign will show a 130 minute wait. That’s huge, and I’m willing to guess it would absolutely influence some decision making regarding an ILL purchase.
But Maybe This Is Explained By Crowd Levels, Right?
Maybe it could be. We know that with high crowd levels, Disney increases posted wait times even more. It has to shift people around more, and make absolutely sure that you don’t wait longer than you expect. And late October/early November have certainly been more crowded than late September/early October. But we can give ourselves a control (comparison) group by pulling the same exact dates from 2019. Crowd levels saw similar trends throughout that time period and this year. And we can split the attractions into their same groups.
Interestingly, both non-ILL and ILL attractions saw increases in posted wait time accuracy when comparing late October/early November to late September/early October. It’s almost the exact same proportional increase that we see in the non-ILL attractions in 2021 – about 5% more accurate for the second half of the time period. But in 2019, that increase happened for the non-ILL attractions and the ILL attractions, instead of the ILL-only decrease in accuracy that we see in 2021.
So, we could still give Disney the benefit of the doubt and say that predicting ILL purchases and capacity/use has made just those attractions more difficult to predict in the era of Genie. And so they’re hedging their bets by giving the posted wait time some wiggle room. But it’s hard to explain away across-the-board decreases in accuracy, especially when those inflated times are displayed immediately adjacent to the option to pay your way around what appears to be a really long standby line.
What Does This Mean For You?
- Be a savvy consumer. You know that Disney has always and will always inflate their posted wait times. Take that into consideration before balking at a posted standby time and purchasing Individual Lightning Lane access.
- This doesn’t mean that you should never pay for Individual Lightning Lane Access. Even if I see Flight of Passage with a posted 100 minute wait time, it might be worth it for me to pay my way around what would still likely be a 60 minute wait. Just do some math in your head before blindly accepting the posted wait time.
- Help us to continue helping you by submitting your wait times! iOS users can now downloaded an updated version of Lines that will let you time both standby and Lightning Lane (Genie+ or ILL) waits. Then we can start understanding actual time saved and generate better recommendations.
Have you visited WDW in late October or early November? What was your experience like with Genie+ or Individual Lightning Lane Access? Let us know in the comments!
20 thoughts on “Are Wait Times More or Less Accurate For Individual Lightning Lane Attractions?”
Great analysis. Disney’s contempt for guests is astonishing, and seems to only be growing.
That analysis once again shows how beneficial the Lines App is! I will only use that when I am in the parks and I know people that scoff at it and say things like “I don’t want to have to plan my day, I want to be able to be spontaneous!” To me, that seems like they are only setting themselves up for quite the miserable experience if they are relying on those standby signs.
Keep up the great work!
I’m right there with you, Paul. I’m a planner through and through. I can plan out a window of time to be spontaneous in 🙂
If you are going to use % of accuracy as your metric for comparison, then it might be better using 100% as your gauge instead of 1. Of course percentages are really decimals but my less-math-savvy friends would be confused why 56% is less than 1.
This is an undeniable trend though. Look at that data! I do plan on purchasing some ILLs while I’m down in December (due to park-hopping to Remy) so if the Android app is out by then Ill helpfully contribute.
Excellent advice, Maggie! I will change the wording from 1 to 100%.
Love seeing data supporting what we long suspected would happen when Genie+ launched.
Wow. I mean we all assumed Disney might do this, but frankly this verges on fraud. The whole “don’t take wait times into account when making your purchase” is obviously just legalese. They clearly WANT you to take wait times into account when making your decision, hence the handy vicinity of the ILL purchase button. Just icky. Thank goodness we have touringplans to shed a little light on situation.
You’ve hit the point, Tiffany. Genie+ and ILL purchases could legitimately still be reasonable and useful. But you have to have the right data to make that decision. So it’s helpful to have as accurate of a prediction as possible about what wait(s) you’re purchasing your way around.
Can you compare wait-time accuracy for late October/early November 2019 to late October/early November 2021? You explained how the accuracy changed from late September/early October to late October/early November in each of those years, but I’m wondering how accuracy changed from late October/early November 2019 to the same time in 2021.
It feels like the wait-time inflation started increasing in magnitude months before Genie launched (perhaps so Disney’s motives would be less obvious?), but I’d love to see if the data actually backs up that hypothesis.
Ah, so you want to compare the actual percentages and not just the trends? Sure! I’ll go back and pull them again when I get a moment. I was focused on trends, and even my brain that generally locks numbers in didn’t commit them to memory.
I was in the parks 11/5 & 11/7. I bought Genie + both days and got a Lightening Lane for Rise and 7 Dwarves. While my wait for those rides was shorter than standby, it wasn’t walk on (15-20m each). My other Genie + LLs were shorter, but it felt like Disney wasn’t managing them well yet. You could see big groups assembling for the next LL time. The if a ride went down(which they were doing often), the big group would move to the next big ride en mass. Also my waits were longer than I remember fast passes being. Still buying the LLs was well worth it. I had limited time in the parks and wanted to maximize my time. Wait times projected by Disney were usually pretty accurate. I think the wait for Dumbo was long than projected by Disney and Touring Plans by about 15m. Otherwise, waits were close to estimates. One aside, there were a lot of rides stopping repeatedly while I was riding them during my two days. Small world, Jungle Cruise, Wedway are the ones that jump to mind.
Thanks for the data 🙂 I mean, it’s pretty disgusting and disappointing, but at this point I’m no longer surprised sadly. Unless this reverses itself over the coming months, I don’t plan to give Disney the benefit of the doubt. Maybe if it wasn’t for all their other nickle-and-diming lately, but they’ve lost the goodwill at this point.
If fewer people are entering via the lightning lane entrance, perhaps the standby is moving faster than expected. Any chance that lower than expected ILL$ sales might cause inaccuracy in Disney’s estimated standby times?
That _could_ be a reason, but based on the bragging on a recent call, and numbers of people getting into the Lightning Lanes, it’s probably not the real reason. There’s more-than-decent demand there.
If Disney wanted to be more accurate they could be. Anyone with a Magic Band or using My Disney Experience, Disney knows when that person entered the attraction and when they got on the ride vehicle. Knowing the rate of people entering the attraction and the rate of people boarding the ride vehicle they could easily estimate the wait for each person entering the queue.
We were at WDW 11/6-14 and stayed on property. Got Genie+ for MK and HS, ILL for RoTR and FoP, BG for Remy, and used EE to rope drop MMRR and SDMT. Used a combination of touring plans/re-optimizing and tip board wait times on the days we didn’t buy Genie+. For the longer wait rides, the posted wait time was accurate to +/- 5 mins. Did standby for FoP early evening because TP chart showed flat times of ~50 mins from 4pm onwards, as the day progressed the TP graph changed and by the time we were in line the wait time was predicted to be ~75 mins. Posted standby was 95 mins, we waited 92 mins. Not an ILL ride, but Navi River the TP prediction was ~45mins, posted 85 mins, we waited 85 mins. These were on CL1 predicted days. Have to say, it was unsettling to be touring without the trove of TP data powering our decisions. Nevertheless, we had an amazing time. Can’t wait to return with TP data in hand!
I realize the last thing you need is another metric. But for me it matters if it is more often that they are five minutes off or in addition with being inaccurate what is the magnitude of that inaccuracy. Perhaps just an indication of what fraction of the they are over 20% higher than actual? Of course the practical bottom line is use TP estimate of wait to decide whether to buy a pass. Never trust Disney for wait times, I haven’t for years because I have had TP and that has made a lot of difference. Even if you are being spontaneous you still want to make that momentary decision on accurate information
Oh Kathleen, you don’t know me very well 🙂 One of my favorite sayings is that good data invites more questions! So I love your ideas for more metrics. Will work on other ways to slice and dice this!
I’m a scientist myself- so I get it. One of the reasons I love this site so much is it is empowering us with data. Thanks for all your work.
My family of 4 visited in November and decided to do the G+ for each day of our trip. Overall it was a very good experience, although it did add ~$400 to our overall spending for the week we were there. We figured out the timing pretty quickly the first day, and also figured out quickly that regular Genie is pure crap and just a waste of screen real estate and code.
We did pay for ILL access on several rides, and used those windows to make our days fore efficient. It is worth $50 for our family of 4 to “lock in” our 7DMT opportunity before we leave the resort. It just sets the day up to be more exciting from the get-go.
Our ONLY operation complaint is how the times change from the first display screen to the second. It causes a lot of frustration and confusion and conflict with existing dining reservations – especially in a park like EPCOT where you may have a 25-minute walk across the park (little legs & all).