Does a Longer Wait = Higher Satisfaction?

A few months ago, I did a generic correlation of average wait times to overall satisfaction for all attractions over the past three years. (You can read about the full results of that study here.) But I was very clear in that analysis that these were overall averages. So they represent a really wide view of what’s happening. But wouldn’t it be cool if we got to more specific detail? Like if we could figure out based on wait time data and survey data how long you waited, and what your exact satisfaction was given your exact wait time.

Oh hey! We have all that data and we can tie it together. I’ll be honest – the results here didn’t turn out the way I expected. My plan was to figure out, for each attraction, what the “break point” was for satisfaction. For example, at a 20 minute wait maybe EVERYONE loves Space Mountain. But once you get above that, satisfaction starts decreasing with your increased wait. I didn’t really find those breaking points, but what I found can still be informative.

Explain the Math!

TouringPlans users submit wait times through the Lines App (and if you don’t, you should – more numbers to crunch = more accurate predictions for your next visit). They can also complete a post-visit survey where they tell us how satisfying each attraction was on a scale of 1 to 5. Tying these two datasets together is where it gets tricky, but we can do it by taking the following steps:

1. The user has to match between the survey result and the submitted wait time
2. The attraction has to match between the survey result and the submitted wait time
3. The day the wait was timed has to fall between the start date and end date of the visit that the survey was about

And I actually took it one step further to make sure I didn’t muddy up the data. If the same user submitted multiple waits for the same ride during the same trip, I threw those out. We only have one attraction satisfaction rating, and I didn’t want to average out the wait times or do some other sort of aggregation. So I only pulled satisfaction scores for each attraction that met the above criteria and only had one wait time.

I only pulled wait times and surveys from the beginning of 2019 through July 2021. Even still, I had a sample size of over 7,750 different wait times that attraction satisfaction scores directly associated with them. Yay! That’s a decent sample. Bonus – each wait time could have more than one satisfaction score associated with it, because we stratify satisfaction by age group. So I can treat each wait-satisfaction pair as independent. Maybe grade schoolers have a different “break point” than “young adults”, or maybe they don’t. I can keep that information stored and see if it makes a difference. But now I have over 15,400 independent pairs in my sample.

Park-Wide Results Example: Magic Kingdom

Hold. The. Phone. Do you see what I see? Those green sections are getting bigger as the wait goes up! Does that mean I’m going to be happier if I wait longer? Slow your roll, let’s talk about it.

First, let’s figure out all of the crazy happening in this graph. The grayed out section to the far right shows that we don’t have enough wait times in those bins to make it a good sample, statistically. All of the other colors’ columns are standardized in height, and then the colors show what percentage of results were rated a 5, 4, 3, 2 or 1 in satisfaction. So more green = happier people. The black line with dots shows the overall average satisfaction score within each bin – the values for that line are over on the axis to the right.

So what I’m seeing here is that average satisfaction is, in fact, increasing with wait time. I can tell that by looking at the black bar, and by looking at each column and seeing more and more green as the wait times increase. Does that mean I can just plop a headline like “Who Needs To Plan? Wait Longer, Be Happier” on this graph and call it a day?

Nononono. The problem here is that my sample is at the wrong level. To the right you’ll see the average wait/satisfaction graph from my post 3 months ago. It’s a little itty-bitty, but we don’t need the details to see the trend. Across all WDW parks, if I was to eyeball a trendline for all of the colored dots, it would start near the bottom left and end near the top right. What does that mean? It means that attractions with higher satisfaction tend to have longer waits. So if all I’m doing is plotting wait time vs satisfaction for an entire park, yeah, it’s going to show that longer waits are more satisfying. Because those rides are more popular, and attract bigger crowds, and therefore you’ll wait longer for them. You expect it. You still enjoy the headliners. And the walk-ons might not be as satisfying, and that’s why they’re walk-ons.

Well, phooie. Okay, so the big fancy graph above tells me … that higher-satisfaction rides generally require longer waits. And I guess it shows that I can’t make some generic statement like “If any line at Magic Kingdom is longer than 20 minutes, you will be less satisfied”. But can I get more detailed information? My next step was to narrow to a specific attraction to see if the “break point” was visible at that level. I ran these graphs for a LOT of different attractions, and they pretty much always fell into two different “types”. Let’s explore each one separately.

Type 1: Always Awesome No Matter What

This attraction type is basically your headliners. They’re great rides that almost everyone enjoys, and it doesn’t really matter how long you wait, they’re still going to be fun. In fact, some of them even appear to increase in satisfaction as the wait gets longer. Let’s look at one example of this type of attraction from each park.

So what do you think is going on here? Are people really happier to wait longer for Toy Story Mania!? I think we can all agree that logically that doesn’t make sense. What is probably really happening behind the data is that we have some selection bias. You’re probably not going to get in an hour-long line for Toy Story Mania! unless you already know that you just LOVE that ride. If you’re unsure, or you’re unsatisfied because you keep getting outscored by your young child, you probably won’t even bother joining the queue if its lengthy. Knowing all of that, it’s not particularly surprising that there’s no breaking point in satisfaction here. These are big, popular headliners. They have high satisfaction no matter the wait.

Type 2: You Like It, Or You Don’t

The non-headliners, or less-popular attractions are where I would expect to see a decrease in satisfaction after a certain amount of time. Yeah, Na’vi River Journey is cool if I wait less than 10 minutes. After that though … I’d probably be less happy.

But it looks like that’s not what really happens. Even at really short waits, there are people that just aren’t impressed. And the proportion of people that aren’t impressed stays pretty much the same no matter the wait. So it’s possible that satisfaction with an attraction is only impacted by ride quality, and not by wait experience. But we also know that our selection bias problem is still here. Experienced folks that know whether they like the ride or not will balk before getting in line, so we won’t get a wait/satisfaction from them unless they’ve already pre-determined that they like the ride. So our proportion of highly satisfied folks at higher rates is likely not accurately reflecting reality.

What Does That Mean For You?

1. Here’s where I was going to helpfully tell you wat what point you should bail on lines for each ride. But it turns out we don’t have that information!
2. Instead, this is a good lesson in the difference between data and information, which is … context! Our data here is right, but we could easily infer incorrect information. Thankfully, we took a step back to understand what was really happening.

Do you have any personal breaking points for attractions? Do you remember the longest you ever waited for a ride, and what you think your satisfaction was?