How Accurate Are The Park Recommendations?
Predicting crowd levels at Walt Disney World is like predicting the weather. Even a 90% chance of rain means that 10% of the people will conclude that you were wrong. The difference is that rain is an absolute event: you either got wet or you didn’t. Crowds at Disney World are subject to people’s perceptions. For example, one of our subscribers sent this in:
Bought your book and subscribed to TouringPlans.com. I printed out the crowd calendar for the time we were there which was June 8th through June 16th. In my humble opinion, your recommendations could not have been worse. We were there nine days, and for the last 6, after hitting big crowds at your recommended parks, we decided to go to the parks you said to avoid. We were walking on several attractions, and had minimal waits at most others. Either EVERYONE is subscribing to your site or you need to really go back to the drawing board on how you arrived at your recommendations. I’m guessing it’s the latter. Sorry, but this subscription was a big waste of money for me. I doubt this email makes it to your website!
Jim
Cypress, Texas
While we’d like to have 100% of Walt Disney World visitors be TouringPlans.com members, we, like Jim, know this is not yet the case. But, because the accuracy of our crowd estimates is really our critical mission, we wanted to give you a sense of how accurate we’ve been this summer.
Our Results
Since the launch of Crowd Calendar 2.0 in June, we’ve made more than 250 predictions for the best parks to visit, and the parks to avoid. (Parks that are neither the best nor the worst are “neutral.”)
- Our success rate at picking the park to avoid is over 99%.
- Our success rate at picking the best park is approximately 80%.
When we’re not correct picking the best park, it is almost always because a “best park” turned out to be “neutral” (11% of the time) or a park that was listed as “neutral” turned out to be “best park” (9% of the time). We think a bit of fine tuning on our rules for picking the best park will increase our success rate there.
- The number of times that a park listed as “best park” should have been ”avoid park” or vice-versa, is less than 1%.
We also predict a crowd level, from 1 to 10, for each day of the year. Since June, our crowd level predictions have been accurate to within 1 index point 90% of the time. (That is, if we predict a 5.5 for a specific day, 90% of the time the actual crowd level will be between 4.5 and 6.5.) Further, the crowd level bias is 0, meaning we’re equally likely to under-predict as we are to over-predict. (This is good, because it tells us our models don’t have an inherent tendency to overestimate or underestimate crowds.)
These results are based on wait time submissions by Lines users (please – keep submitting those times!) as well as wait times collected by our own researchers.
Remember These Key Tips
Our park recommendations are based not just on low waits times but what we like to call “optimal touring conditions.” If at mid-day you walk into a park that we list as “Avoid” you may experience manageable crowds and you may even walk on to a few rides. What Jim experienced — low wait times for some attractions – is completely possible. However, park conditions like Extra Magic Hours, Special Events and park hours may have made it difficult to tour effectively that day. On average, our recommendations will give you the best chance to tour the parks under optimal conditions, seeing fireworks, parades and completing a touring plan with a minimal total amount of time spent in line.
Optimal Touring Conditions are:
- Days with sufficient park hours to complete a touring plan
- Days without Extra Magic Hour Mornings
- Days without special events
To read a full break down of “optimal touring conditions,” read our Park Recommendations page.
The accuracy of our crowd statistics is a constant focus for us, and we’re always trying to improve it. We added the per park index to allow you to look at the absolute estimates for crowd level without considering touring conditions. If you are not worried about park hours, special events or Extra Magic Hours, then use the per park index to pick your day.
PS: In case you were wondering, we made 23 park predictions during Jim’s trip, and were correct on 19 of those (82%).
_______________________________________________________________________________________________
Estimates for the crowd calendar are derived from the sum of the (adjusted) peak wait time for all attractions at Walt Disney World. The crowd level index represents a percentile rank of that sum for all days between June 2006 though the end of the crowd calendar.
You May Also Like...
-
We're seeing a new series of keychains appearing at Walt Disney World. Made of leather and fabric, they look like those…
-
Last week, we showed you the newest in Disney collectible--the Orange Bird Park Pal. More of these items have been popping…
-
When I first saw the Disney Park Pals, I was so confused. What do they do? I thought to myself. I asked…
-
The parks are reopening soon (July 11 for the Magic Kingdom and Disney's Animal Kingdom and July 15 for Disney's Hollywood…
O.K. Our flight arrived 2 hours late cutting deep into kid sleep-hours. We had 2 unexpected almost hour-long bus waits and a hour+ wait for a taxi. Of the 75 items on my list for our 3 1/2 day visit to all 4 parks, we easily hit all but 3 or 4 (which we opted out of & Rockin Roller Coaster was down). In addition, we added at least 25 more counting other entertainment, shops, food, and characters. We also had choice seating at almost every attraction. I’m still in disbelief how well this went compared to my last non-planned visit.
-As for the numbers, the per-park estimates were dead-on.
-Though MK & HS were marked as “parks to avoid” due to early morning hours, visiting each on their early morning hour days was the right decision with a touring plan –and did give us purely 1 extra hour. Wait times would have been almost identical if we would have reversed days at the two parks –plus we benefited by having early morning hours.
-(Actually, per-park crowds would have been much less favorable if we would have reversed the two days we visited MK & HS. Wait times would have been longer).
-In general, wait times were in line with user reported times on similar historical days, however often they were much less. I think many users report what the permanent signs that are posted say, not noticing that there are no people waiting in line next to the sign.
-I have more confidence in the historical user-reported wait times (from days with similar per park crowd calendar numbers) than the calculated wait times here. I could also better match historical user-reported wait times to a time of day close to when I expected to be at the attraction. I know you can’t always have similar historical days to extract user-reported times from though –and automating this is difficult.
In summary, per-park numbers and user-reported wait times worked better for me than “parks to avoid” indicator and the calculated wait times here, respectively. Thanks again for the mountains & mountains of information and well-engineered touring process. Wow, what a trip.
Hi Chris, it is a great question. Like many statistics, the numbers don’t always follow what we would expect. In general, yes, we would expect that the trend would be the same. But recall what the crowd index really means:
The percentile rank of all wait times. So every day when we collect wait times we throw it into the mix and figure out how all days in the past (and future) rank against each other. It is possible for our estimates of per park levels to remain the same but when compared to all days, the overall rank is now higher. For example, if last week was less crowded than expected then that makes this week less desirable.
Each rank is taken individually so its possible for some to go up, others to go down and still others to stay the same.
Hope this helps.
I know there are many significant variables and weighting for each situation but I too have been curious about what Chris discussed. In trying to allocate my days to the appropriate parks I must choose between:
-Overall crowd numbers adjusted by extra hours & special events
-Individual park crowd numbers
-Trying to just stick with “best parks” and “parks to avoid”
Wow, with Crowd Calendar numbers of mostly 1-3, HS comes out of no-where with individual park numbers of 6.4 & 8.3 –and until recently was a “best park” before it hit 6.4!
Oct 21 22 23 24
MK 0.8 0.1 7.8 0.3
EP 3.6 0.4 0.4 0.4
HS 6.4 0.9 1.0 8.3
AK 0.5 4.5 1.0 2.6
Resort 2 1 4 2
I wish it were a little easier to determine the how to allocate your days. We HAVE TO go to all of the parks which forces us to choose when. 6.4 & 8.3 for HS scare us after seeing overall crowd numbers of 1 or 2. Just like thinking you will be riding Dumbo then realizing you are about to get into a Tower elevator. (O.K., well maybe just the Haunted Mansion).
Again, I do understand the potential variability in all of this. I guess it’s probably always best just to go with “best parks” & “parks to avoid”, and make your best guess on the others, right?
The one thing I would suggest is to make sure you account for the error rates in our per park index. If you are comparing two days that are within 2 of each other, chances are you will not notice a difference. For example, 6.4 and 8.3 aren’t as different as one might think. Consider that when you are forced to choose.
The Studios is the most volatile of the parks. We see crowds influx on one day for Fantasmic then disappear the next. By far, the most changes day to day are at the Studios.
The number one best way to beat the crowds on any day is to use a touring plan. If you do that, the crowds won’t be scary at all. We harp on this a lot but it is true. The real answer to:
“What day should I go”
is:
“Any day, but get there at 7:30 and use a touring plan”.
After comparing crowd numbers & expected wait times to actual wait times reported by users, I have a better feel of what to expect. My upcoming trip dates also have crowd numbers similar to those I looked at closely –so I feel more certain about possible wait times at each attraction.
I made these observations, which may or may not be correct:
-Most often, total crowd numbers will be a good guide to traffic, but anything can happen.
-Most often, for a single park, the individual park number can be compared on different days to reliably predict which day will have better wait times, but only if the number is very high one day and very low another.
-Hollywood Studios very often has an influx of traffic that is difficult to predict.
-Even on “2” WDW crowd days, there can be attractions by afternoon with 90+ minute waits and quickly depleted fastpasses.
-Individual park numbers may contradict what “best parks / parks to avoid” indicator recommends and can sometimes be more useful in choosing which parks to visit over several days. This requires comparing all individual park numbers on all days to look for huge differences –such as 0.8 at a park one day compared to 8 on another day.
-User reported times of actual waits are slightly higher than actual waits.
-Line wait times reported by users aren’t posted (or indirectly reflected) in “Lines”, only FP times are.
-Attractions that start out or briefly go offline often will have wait times double what was expected.
-And of course, –arriving very early and using a touring plan always significantly improves the efficiency of a visit.
I wanted to try and determine more exact wait times for my upcoming visit, so I looked at historical days with similar (total & individual park) crowd numbers and compared them to user reported wait times. I’m convinced now that much lower individual park crowd numbers should in general result in at least slightly better wait times. Quantifying these wait times at each attraction however seems to be more of an art than a science though.
Based on all this, I’m making a bold decision to visit MK on a “park to avoid” day with early morning hours and the Halloween parade —instead of on a day with none of these issues. (Oct 21 vs 23). I’m doing this for two reasons:
(a) I’d rather go when the MK crowd number is 0.8 than when its 7.8.
(b) With the same or better wait times, plus extra hours, plus already being there for the parade, it seems like a better option.
I hope I am correct in this assumption for my situation and my logic isn’t sending me to the wrong park due to my ignorance about how “best parks / parks to avoid” are derived.
O.K. my ignorance is no longer ignorant after discovering the explanation of differences between “Per Park Levels” and “Park Recommendations” at:
http://blog.touringplans.com/2010/08/24/per-park-crowd-levels-and-the-crowd-calendar/
I am sticking with the Per Park numbers to decide which parks on which days. I also have put historical user reported wait times from similar days & times in a table and will use those as a more fine-tuned estimate of wait times, for the specific times we expect to be at each attraction.
Thank you for such a great website, good advice, and all of the hard work. I can’t wait (pun intended) to see how it plays out. It has been really entertaining to have access to so much well-compiled data and good information….So much fun and I’m not even at Disney World yet.
Hi. I have always been a little unclear on the relationship between the per-park crowd levels and the overall crowd levels. It’s my understanding that they both reflect average wait times and therefore would be expected to “trend” together. You’ve said previously that you can’t average the per park levels to obtain the overall crowd level, but I always assumed that this was due to some parks being “weighted” more heavily than others in creating the overall score (and not because the per park levels were unrelated to the overall crowd level). The thing that makes me question this is that I’ve been tracking both overall crowd levels and per-park levels for my upcoming vacation and I noticed that the overall crowd levels for two days have increased while the per-park levels for those same days either decreased or remained the exact same. For example, last week (on 9/15) the overall crowd rating for Wednesday, 10/20 was a 4 and the per park ratings were 4 (MK), 6.8 (EP), 0.3 (HS), and 2.8 (AK). Yesterday (9/21) the overall crowd rating for the same day (Wednesday, 10/20) increased to a 5 but the per park ratings decreased a bit: 4 (MK), 6.7 (EP), 0.2 (HS), and 2.4 (AK). For another day during my upcoming trip (Friday, 10/22) the overall crowd level increased from a 1 to a 2 but the per park levels remained exactly the same. I’m wondering how this is possible? Shouldn’t the per park levels “trend” with the overall crowd level for a particular day? Thanks so much for any details you can provide here. Love your website!
Hi there, we went to WDW last July(09) and September(09) and we found TP to be very accurate. I would not go on a trip without the information from Touring Plans, as it really can help you make the most of your time on vacation at WDW. We may go again next month and we subscribed to TP to help us plan our trip, it is especially helpful if you are on a 4-5 day trip and you can’t afford not to make the most of your time. Thanks again and keep up the good work.
Just thought I would add that we were in the parks from 9/3/10 – 9/9/10 and the crowd levels did not seem very accurate. We were in the parks last year at the same time and had a different experience. On one particular day, the per park crowd number was .4 for Hollywood Studios and we waited in line for 60 minutes for Toy Story Mania and had similar experiences at all of the parks. We found the crowds to be quite different from what was projected and had to do alot of park hopping.
Sorry to hear that Susie. When you say you were in the parks last year at the same time and had a different experience what do you mean? Were the crowds smaller than this year?
As crazy as it seems, a 60 minute wait at Toy Story Mania is one of the lowest you will see. For 90% of the days since it opened, Toy Story Mania has had a peak wait time of 90 minutes.
Yes, there crowds were probably half of what we experienced this year. So, on a day that Hollywood Studios is marked a .4, the expected wait time for Toy Story is 60 minutes? Last year at this same time, we walked right on it.
Jim from Cypress here…..before I’m made out to be the new Disney villain, I did (shortly after the original one) send a follow up email to Len softening my stance quite a bit. We had just gotten home from a day when the heat index was 108 degrees. We were a little out of sorts to say the least. I’m not saying I take back what I wrote, because the strategy adopted worked for us. On a day Epcot was listed as the “best park”, the lines for EVERYTHING were unmanagable. We hopped to the Studios (a “park to avoid”) and had a great time with very short wait times. This strategy proved to be true while we were there.
Having said that, I still have a subscription to TouringPlans, and plan to keep it because I enjoy it. Also wish I could have used the “Lines” feature, but didn’t have a capable phone at the time. I think that would be a HUGE benefit. I also have bought copies of the “Unofficial Guide” for the last 10 years running.
I did apologize to Len for the harshness of the email, because that really wasn’t warranted. I guess it was just the heat talking.
I actually do believe that many people use this site, and that it does have an effect on where the crowds go, and that it CAN turn a “best” park into a nightmare.
Anyway, I do enjoy the site, and didn’t mean to create a defensive attitude with anyone on the staff.
Hey Jim,
Thanks a lot for the comment! No hard feelings on our side — we just want to get things right, and we’re really sorry we didn’t get them right on your trip.
Best,
-Henry
No problem! I really appreciate how quickly all of you respond to emails and comments. It shows that you care about customer feedback.
Best wishes as well,
Jim
Hey Jim,
Thanks for the clarification! We wanted to post your email because it shows that we’re open to feedback. Yours was succinct and clear, so it was a good model for us to use. No hard feelings, of course!
Thanks again!
Len
Not at all Jim, thanks for the follow-up. Your email was one of the things that prompted us to release the per park index. We weren’t doing a great job expressing the reasons that a park was listed as “Avoid” so we thought adding the per park index may help people decide for themselves.
I’m curious if you think that might help others with their planning?
Our main point in all this was to show that the recommendations are meant to show the best days to complete a touring plan while the per park index will show you an absolute comparison for wait times.
Fred,
Yes, I think the per park index would be quite helpful because it does break things down a little more. But even more than that, I wish I had had “Lines” on my trip. In my opinion, everyone experiences Disney in their own way, and many don’t need or want to get to all of the attractions in a given park. With “Lines”, they can see what’s going on at the attractions that they are most interested in, and then make an educated decision on which park would best suit them on a given day. I just see that feature as a more “real time” estimate, and it would therefore be more accurate than a prediction.
Man, all this talk is making me want to go back again!!!!!!
Jim
Just a quick question. I’m planning my trip and I do have park hopper. When scheduling my parks, I’m primarily looking at the “per park crowd levels.” Because I’m willing to change parks to adjust for special parties and such, are there any drawbacks that I’m not accounting for? Dec. 16 is an example. It has an avoid rating but a crowd rating of .9. I know there is a party that night that I have tickets for. What do you think of that way of planning?
Sorry left off the park in my example that the park is MK. Thanks for any reply. Love your site btw.
I believe it is a park to avoid because of the Christmas party…they pretty much always recommend avoiding MK on a party day because cutting the day off at 7pm makes it harder to finish the “ideal” touring plan.
On a 0.9 day, I don’t think you can go wrong. Especially if you follow one of the plans. My recommendation would be to tour first thing in the morning, take an afternoon break and then hit the party. Should be a great use of your time.
One of the things that users of the Crowd Calendar always need to consider is that the choice of parks is also going to depend on if you are a WDW resort guest, and if you have hopper passes. Hence, EMH parks are often designated as parks to avoid, but if you are a resort guest with hopping capability, they may be a best park, especially in the morning.
Unfortunately, there are probably too many variables like this for Len & Fred to provide recommendations for everyone’s individual scenario. That’s where experience comes in.
BTW, Len, welcome back from Scotland. I trust the golf was good. (How can it *not* be in Scotland?!)
Thanks for all you’ve done through the years. Lots of us have used your great information for a long time.
Even Disney sometimes totally misses crowd estimates. Several times this last year, Disney has added hours to the theme parks without any notice, and then added extra hours for the next couple of days.
Yes, there have been more adjustments to park hours in the the last 24 months than we have ever experienced. Typically, this points to economic influcences. When people’s spending habits change drastically as they have in the past few years, it becomes difficult to predict what will happen.
For critics or skeptics of the crowd calendar, I think there are two important things to keep in mind: 1) crowd levels are relative to each other (so a day 4 at WDW might not seem “un-crowded” but it is less crowded than a 5 or 6 day) and 2) crowds are subject to perspective and what folks are used to when they visit theme parks. A 6 day may seem like a 10 to someone who has only toured in the off-season and a 6 might feel like a 1 to someone who is accustomed to visiting in the summer or during holidays. If you’re not someone who has experienced various crowd levels at WDW then it might be more difficult to know what to expect from the numbers. Having toured in almost all WDW “seasons” the numbers seem pretty accurate to me, and helped me decide to go ahead and try out MLK weekend for this year’s trip…the 5’s and 6’s sound great compared to the 10’s during last year’s March spring break!
Thanks Betsy, this is by far our biggest challenge. We don’t want to simply make a calendar that expresses how the parks feel because, like you say, it depends on one’s perspective. Like everything else in the Unofficial Guide, we choose to take the data-driven approach.
Our calendar does a good job showing the relative wait times for each day. If we could devise a system to predict how crowded it would feel to everyone, well – wouldn’t that be something.
This information is great. I was wondering if you could elaborate more on how you you determine the accuracy of your wait time estimates. You noted that actual wait times are determined by submissions by Lines users and collected by your own team of users. I thought that the wait times submitted by Lines users were posted wait times, not the actual time they waited in lines? If this is true, how do you use these data to determine actual wait times, given what you have previously said about Disney’s posted wait times being less accurate than your estimates? I’d also be curious to know more about the number of data points your team of researchers are able to document for rides at various time points each day, given that it would be impossible to cover a large number of time points for various rides each day (given the almost infinite # of data points). Thanks!
Well Chris, that could be a blog post in itself but I’ll give it a try in 200 words or less.
Yes, Lines users submit the posted times (although we are working on a method to submit actual times as well). We know from research that Disney overestimates the wait time by about 20% on average but we can’t just simply reduce each submission by 20%. We take the posted times and apply a transformation model that tells us the “curve” for wait times each day at each attraction. This curve represents the actual wait times and allows us to calculate the peak time for that day. Sum all the peak wait times and rank each day in the database and voila!, we have something we can compare with what we predicted.
When we hit the parks ourselves we do 30 minute loops of the park recording the ups and down of the wait times at all attractions. Meanwhile, we’ll send someone to ride an attraction over and over all day, recording their actual wait time. We’ve had researches ride Space Mountain 11 times in a row, waiting as little as 5 minutes in the morning to 78 minutes in the afternoon.
Sounds like a glamourous job until it is time to test It’s A Small World.
Hey Keith,
Fred will be along to answer shortly. I think, however, the answer is the latter: wait times alone would have been different enough to change the park recommendation.
For example, one of the rules we use to say a park is a “best park” is if the wait time predictions are 10% lower than the average of the week around it. (So if today’s crowd level at the MK is 5.0, and the average for the week around today is 5.5 or higher at the MK, we’d recommend visiting the MK today.)
I think that a little bit of fine-tuning on that rule – say, making it 11% or 9% – would solve most of the differences we have between our recommendations and what actually happens. And the good news is that we have the capacity to model these what-if scenarios. I expect that over time, we’ll become very, very good at this.
Thanks again!
Len
That’s right Len,
Every day we look at the wait times and say to ourselves – if we were to go back and apply our rules for recommending parks what would we say for park X. If park X ends up the same (Best, Neutral or Avoid) then that was a success.
It is wait times that would make the difference. Park Hours, Extra Magic Hours and Special Events rarely change on the day of interest (although it does happen) so if we were not successful it is almost always because the wait times were sufficiently different from what we expected in relation to the days around it.
Thanks Len and Fred for the detailed reply. I figured that the hours/events wouldn’t change that often, so thanks for confirming.
Thanks so much for this detailed writeup Fred. I really appreciate the integrity you showed in quoting someone who didn’t have a positive experience, and also your obvious passion for getting the numbers as accurate as possible.
It’s taken me a while, but I think I’m finally understanding how the Recommended Parks and the Per-Park Crowd Levels all fit within the way I like to tour. (This is definitely due in large part to your explanatory blogs such as above.)
One specific question I was hoping you could answer. When you did the retrospective on summer predictions, what were you using to determine the “success rate” of your park recommendations? I mean, as you said, the Best Park could be due to things like long-enough park hours, EMH hours, or special events. So in the 20% failure of choosing the Best Park, are you saying that something unexpectedly changed regarding park hours/EMH/special events that changed the Best Park recommendation? Or do you mean that the wait times alone were different enough to change what would have been the Best Park recommendation?