Genie Testing: Genie and TouringPlans Head-to-Head
Yesterday, October 21st, was a big day for us at TouringPlans. It was our first planned head-to-head test with Genie – Disney’s new “plan your day” feature. Would Disney’s claims that it could pick attractions that delight you and help you avoid waits and have a magical day compete with what we’ve been doing for years? They certainly made big claims.
To review, here is what Disney wrote about Genie when it announced the October 19 launch date: “the complimentary Disney Genie service offers new features that let you create your best Disney day. These features include a personalized itinerary creator that effortlessly and smartly maps out an entire day inspired by the things you tell us you love – helping you make the most of your park time so you can have more fun.” And “Disney Genie continues to update your itinerary from morning to night as the day changes, so you can stay spontaneous and go with the flow.” Sounds ideal! But what happens in the real world?
The Set-Up
We wanted to test the ability of Genie to live up to two specific claims:
- The personalized itinerary helps you accomplish the things that you tell it that you love
- The itinerary continuously updates throughout the day
Both of these things are easily accomplished with a Touring Plan, by 1) selecting attractions that you want to experience, and 2) optimizing that plan throughout the day. Could Genie keep up?
We picked a set of 12 attractions from the COVID-19 Magic Kingdom for Adults One-Day Touring Plan:
- Big Thunder Mountain Railroad
- Buzz Lightyear’s Space Ranger Spin
- The Haunted Mansion
- it’s a small world
- Jungle Cruise
- The Many Adventures of Winnie the Pooh
- Peter Pan’s Flight
- Pirates of the Caribbean
- Seven Dwarfs Mine Train
- Space Mountain
- Splash Mountain
- Under the Sea
And we added a 30 minute lunch break. To start Genie with the same information, we picked the exact same attractions as the things that we “love,” or our must-dos for the day.
We set the following rules: Do whatever the plan tells you to do. No using Genie+ or ILL purchases. And we needed to start slightly after rope drop so that neither plan allowed you to take advantage of early entry or rope drop strategy. And we would wrap up our test slightly after 3 pm.
6 hours with Genie
Our test started at 9:25 with a kickoff at Casey’s Corner in Magic Kingdom. Our tester checked her Genie itinerary, which sent her immediately to Jungle Cruise, which had a 40 minute posted wait. She ended up only waiting 18 minutes. Not too bad.
But then things start to go off the rails, after Jungle Cruise, Genie tells her to go to the Swiss Family Treehouse – an attraction that not only wasn’t on her list of what she wanted to experience, but is an attraction that almost never has a wait. It’s a great attraction to slot during very peak crowds – not early in the morning. Nonetheless, she followed the plan and got in her climbing for the day.
Did Genie get back on track with her preferences after the Treehouse? No. She was next sent over to the Enchanted Tiki Room. While we can all agree that this show is a timeless treasure … it’s not really a great morning option. Again, almost never a wait.
Her third stop was a spin on the Magic Carpets – which means that 3 out of her first 4 attractions weren’t on her list of must-dos. The posted wait was 15 minutes, and the actual wait was 15 minutes. Not a horrible wait, but also not a great attraction.
By this point, our tester was significantly ahead of the times that Genie expected her to hit. But whenever she tried to refresh, all of the steps and times would stay the same. There was no way to mark a step as complete or to try to force a reshuffling. Definitely no continuous updates so far. What this meant was that she was sent to visit Tom Sawyer Island before Tom Sawyer Island opened. Since we were following the rules, she waited until the Island opened and then spent 30 minutes away from civilization. Again, not an ideal time to head to Tom Sawyer Island. It’s a relaxing place, but best saved for the busiest times on the busiest days. Especially when you have a list of 12 popular attractions to get to in a day. Genie is now 1-for-5.
Finally, at 11:40, Genie decided to allow our tester to experience another attraction that was on her list of must-dos. She was sent to Big Thunder Mountain Railroad, which had a posted wait of 45 minutes. So hey, 2-for-6. The actual wait ended up being a bearable 37 minutes.
After that wait, our tester took her lunch break. Genie suggested Golden Oak Outpost (another questionable suggestion given other nearby options with better food), but we let our tester ignore that recommendation. Something something Geneva Conventions.
After lunch, her plan had still never shuffled attractions or appreciably updated in any way. Her next step was supposed to be Haunted Mansion, but when she got there it was closed. Genie recognized this issue, and for the first time in the day, it updated the rest of her steps for the day. And it decided a good swap for Haunted Mansion (which was on her list of interests) was … Country Bear Jamboree (which wasn’t). That was another 12 minute wait and then a long show.
You might think Genie has been weird and unreliable so far, but JUST YOU WAIT. It gets even crazier. After exiting Country Bear Jamboree at 1:48 (which was a single step after an update of her schedule), Genie suggested that our tester grab a spot on Main Street for a Main Street Philharmonic performance … at 3:35. Almost 2 hours away! Attempts at forced refreshes were futile. If we hadn’t tried to hack the plan, that would’ve been the end of her test. She would’ve completed 2 of her 12 wish-list attractions, and a bunch of other randomly recommended experiences.
Luckily, we toyed around with how to “fix” her plan. She deleted the Philharmonic step. And then the cavalcade step after that. And then another cavalcade step after that. And finally, when there was a 4-hour gap in her plan, Genie was kind enough to update her plan with new steps again. It sent her to Space Mountain at 2:28, with a posted wait of 45 minutes. She waited 37 minutes and her test time was done.
Genie doesn’t have a way for you to tell it what time you plan on leaving the park, so even if we give it the benefit of the doubt and say that we were unfair by leaving mid-day, we could examine the rest of the suggested steps to see if it was just shifting all of her requested attractions to the end of the day. What were the remaining steps in her plan? Mad Tea Party, Under the Sea, Peter Pan’s Flight, Prince Charming Regal Carrousel, Dinner at The Lunching Pad, it’s a small world, Pirates of the Caribbean, Audio Tales (audio tour options), and Buzz Lightyear’s Space Ranger Spin. This would’ve left her with 4 attractions on her must-do list that Genie didn’t ever put in her plan, even with 12 hours in the park. Ouch.
So what’s the summary? Wait times were mainly low for this tester, but it was because her input regarding what she wanted to ride was almost completely ignored. She was shifted from low-popularity-attraction to low-popularity experience and didn’t get to try most headliners. She wasn’t even notified that Enchantment would be happening. Proximity seemed to be prioritized over almost anything else. There weren’t continuous updates throughout the day. In fact, Genie only updates when literally forced due to ride closure or exorbitant deletion of steps.
6 hours with a Touring Plan
Our TouringPlans tester also began her day at Casey’s Corner at 9:25. When she optimized her plan, it sent her to Buzz Lightyear’s Space Ranger Spin, where she arrived to a walk-on experience. Excellent start to a day.
After exiting the ride, she optimized again and headed to Haunted Mansion. A long walk, but a ride that would see increasing rides throughout the day. TouringPlans had her skip past all of her Fantasyland attractions because it was very crowded that morning. She arrived to Haunted Mansion at 9:39 with a 20 minute posted wait. She only waited 15 minutes, but the ride broke down 3 times while she was in her doom buggy – a sign of the trouble that would likely eventually lead to its closure that threw off our Genie tester.
Once she escaped the Mansion, TouringPlans sent her to the Jungle Cruise. By this point, Jungle Cruise had a 60 minute posted wait. Definitely more people in line than when the Genie tester got there earlier in the morning. Our TP tester had a 38 minute wait, and then went to see the back side of water.
After her cruise, she re-optimized and was sent to Peter Pan’s Flight, which had a 50 minute wait, but wasn’t predicted to see a drop in crowds for a while. She got in line and waited 46 minutes. It was now time for her lunch break, and she chose Tortuga Tavern.
After lunch, she was supposed to go to Splash Mountain, but when she re-optimized she saw that it was down, and she was directed to Big Thunder Mountain Railroad instead. There was a 45 minute posted wait (just like when the Genie tester entered line), and the actual wait was 41 minutes.
Her newly re-optimized plan then told her that it was a great time to head to Pirates of the Caribbean. The 40 minute posted wait ended up only actually being a 14 minute wait. That’s a breeze.
Once she got off of the boat, Splash Mountain had come back online, and her re-optimized plan sent her there next. There was a big 75 minute posted wait, which turned out to be an actual 53 minute wait. That gave our tester time to rush to one last attraction before our testing time was over.
The re-optimized plan told her to hustle over to it’s a small world, which had a posted wait time of 20 minutes. Only 15 minutes later, she was on her boat and enjoying one last classic attraction.
What were the results for her touring plan? In less than 6 hours, she was able to check off 8 of her ideal 12 attractions. Pretty impressive given that the 12 attractions are meant for a full park day. And if she had more time, she could’ve finished the next 4 steps (all of which she had picked in advance as attractions she was interested in) by around 6 pm.
Conclusions and Commentary
- The fundamental goal of any plan for your day in a Disney park should be: Helping you accomplish what you want to do. And that is where Genie failed. It failed spectacularly. And not because of time constraints or confusing input. From the outside, it appears to blatantly ignore the inputs other than picking one of the selected attractions for the first step. From there, proximity (not interest) ruled the recommendations.
- Genie did not come anywhere close to continuous updates. In almost 6 hours, it updated twice. Once when an attraction was closed, and once when our tester deleted enough steps to make an apparent 4 hour gap that Genie needed to close.
- Genie was willing to send our tester to a bunch of non-requested attractions that have low to zero waits in almost all conditions. But when met with a 2 hour break before a show, it failed to send her to other nearby attractions with low waits (like PeopleMover or Mad Tea Party or Carrousel or … many other options).
- Genie may eventually receive updates to meet its self-declared requirements of taking your interests into consideration, or to continuously update throughout the day. But until then, go ahead and opt for the app that already successfully does both of those things. Use a TouringPlan and avoid all of the mess.
Have any thoughts about the testing? Let us know what you think in the comments.
You May Also Like...
-
It's January, but go ahead and strike up "Pomp and Circumstance" already. We spotted these Class of 2020 Graduation Mickey Ears…
-
In this week's episode of TouringPlans Teaches, I'm going through how to plan a day at Hollywood Studios while stacking Genie+…
-
Yesterday (October 23rd) was a gorgeous Saturday in Hollywood Studios, and it was the last day of our marathon Genie testing…
-
Genie+ is Disneyland and Walt Disney World's FASTPASS-style ride reservation system. Starting today, Disney announced variable pricing for this popular add…
I’ve used your plans for 14 trips and will always use you! You and data are the real magic makers. I’d love to see if you’ve tried this again, maybe in another park. I’ve only used your plans since the advent of the FP+ system. Did you provide plans back in the dark ages of paper FastPasses? I’m eagerly awaiting the adaption of Touring Plans to the new Genie+ system. I can’t even begin to imagine how hard it will be to accommodate the Inability to plan LL access in advance, but I assume it will be a little like the old paper fast passes. Thanks for your hard work
And this is why TP has such a loyal following – you all rock.
Now – please don’t go and sell your product to Disney – no matter how much they offer. You know they’d just shut down the competition.
If Disney did what TP does, it wouldn’t work. The reason TPs work is because such a small percentage of people use them.
Just returned from a 5 night stay at Pop Century. The Genie was useless except for the two times we bit the bullet ($15 for Rise of the Resistance and $14 for Flight of Passage) – walked right on both rides. Touring Plans was it’s usual, dependable accurate self. At one point the Genie went back into the bottle and went to sleep and chaos ensued at all of the “Guest Experience” umbrellas around the park. Stuff that Genie back in the bottle and cork in it and toss it in the bottom of the deepest sea. Tour Plans I will stick with you!
So, here’s the thing though. You all aren’t able to conduct an unbiased test of the two systems because you have a justified vested interest in seeing Genie fail. If you have your professionals using your app, an accurate test would be to have someone from Disney who developed theirs and those parties to go head to head. Obviously, that’s not going to happen. A closer to unbiased test would probably be to get third parties with the same amount of knowledge about the two apps to test them. But anyone you try to recruit would probably end up being biased too since you’re the ones initiating the tests. This is just a fact of data collection in general. There are always implicit or explicit biases.
Anna has a great point. When I knew TP was going to conduct tests, I, too, was worried about biases. However, with what has been reported by the TP team about Genie not taking into account user preferences and in the case of the Hollywood Studios tests published today (October 24), Genie is clearly designed for the benefit of Disney and not the guest. The $100 upswell suggestion made by Genie to attend Droid Depot when it was not listed as a must-do in two different testers’ plans is the most amazing failure that I have read about Genie!
It’s still not unbiased testing and reporting. As kinks get worked out and people learn how to use the app, I think it’ll all work itself out. Everyone hated Fastpass+ when it first came out, and then everyone learned how to use it. What I like about Genie is that is does suggest things that aren’t necessarily even known to exist by people visiting, like the animal trails at DAK.
Anna, that’s the target audience for Genie: the uninformed. I find it bizarre that anyone does not know about the walking trails, but if they don’t and Genie suggests them to those who do no research, they will happily follow Genie’s guidance.
I am theorizing that Genie will turn out to be a win-win-win situation. The first win is to Disney for coming up with an app that will spread out the crowds and encourage upsells to the uninformed. The second win goes to those uninformed individuals who will go home on their vacations and tell people about all the short lines they encountered and the cheap skip the line tickets (as compared to what other parks sell). But the third win goes to us Touring Plans subscribers. If Genie is spreading out crowds, then the lines at the headliners are going to be shorter and we Liners are going to get more for our Disney dollar than we did during the paper fastpass and fastpass+ eras!
I would love to see what results are using Genie+. We will definitely be purchasing this (seems similar to the MaxPass system we used on a Disneyland trip a few years ago). I feel like I’m not able to effectively plan out our days with Touring Plans if we aren’t able to take that into account.
Hey Emerald! Originally we thought maybe not, but as the days progress, it’s pretty apparent that things are different post-Genie, so you can bet that we’ll be making sure to incorporate the differences.
I also agree Genie is useless recommendations. Used G+LL at MK and Epcot this week. Tried to select next LL after tapping in at an attraction, but it wouldn’t let me until I tapped in to the 2nd tap station at the end of the LL queue where the lines merge! Think about it: You’re inside the ride queue with little to do and you can’t select your next LL. Wasted time. I wish I had taken note of which rides had this 2nd tap in station, as not all did. Something else new that Tours might want to track and test.
Thank you for the replies to my post yesterday. I was curious if the addition of Genie and its possible effects on crowd distribution would make it so that wait times post-Genie predicted by Touring Plans will need to be recalibrated, using post-Genie behavior as its dataset?
Guests that are standing in line, aren’t spending money. So Genie spreads everyone out, and gets more people “experiencing diversions” that Disney hopes leads to more Guest-dollars being spent.
Sorry Disney, I’m sticking with my old friends at Touring Plans. Continued thanks and appreciation to everyone testing, collecting, and you Becky for reporting these past few days. Very insightful!
I suspect the actual intent of Genie is to redistribute traffic away from headliners towards Disney’s other attractions. This lowers wait times at those headliners and theoretically improves guest experience by allowing them to “enjoy” other attractions instead of spending hours in a line.
Ya, that’s the conclusion I’m coming to also. Which then leads to the conclusion that Genie is much more for Disney’s benefit than a guest.
This is exactly what I was thinking. Disney isn’t giving you a free “planning app” because they want to genuinely improve your experience and satisfaction. They want to more efficiently and predictably shuffle traffic around their park.
Wow, so disappointed in Disney, but it’s not unexpected. They’ve totally destroyed Walt’s vision of a fun, safe, (relatively) inexpensive place for families. Now the (USA, at least) parks have become yet more of a $$ grab. From what I’ve read here & other places, Genie is a @*#&% mess. As you said, it ignores what YOU want, & instead sends you where THEY want/need you to go/do & esp. BUY.
I get it, I do; they’re in business to make a profit. But at what cost? I thought the basis of any successful business model was “repeat customers”? Yet they treat their AP holders like junk… & now that attitude has spread corp-wide: treat everyone like pooh (& I don’t mean Winnie).
Thank you Becky, & TP staff, for telling us the real scoop. You’re also in business, but you seem to be making an honest try at reality-based comparison. Good luck!
P.S. When we visited WDW a few yrs ago with grandson, we used TP & thank gawd! Older medical-issues grandparents w/8 y.o. grandson… we wouldn’t have made it w/o TP! “Minimize walking” prob. literally saved our lives & sanity. Disney obvsly doesn’t want $$ from the ppl most likely to have it/willing to pay their exorbitant prices. Even if we were willing to pay for Genie+/ILL (ILL; how apropos!) which we aren’t — running around any park willy-nilly wouldn’t work for us. So, now WDW was a one-&-done for us. Add in the absolute lack of any magic or “pixie dust”… nah, we’ve got better things to do with our time & money. But again, congrats on your timely article!
This article shows what this teacher has been telling her students for years… homework pays off!! Touring Plans does exhaustive “homework” and plenty of testing and adapting. Genie obviously didn’t. If I were a first time visitor and all I saw was the Swiss Family Treehouse, Tiki Room and Tom Sawyer Island I would be spectacularly disappointed. Sounds like just walking from attraction to attraction and waiting for whatever was there would have been just as “productive” as following the Genie.
Out of curiosity, do you know what settings were used for “Plan Preferences” by the TouringPlans Tester? I would guess priority was given to lowest wait times over walking less, given the move from Space Ranger Spin to Haunted Mansion…
I actually really value these options on TouringPlans and couldn’t find an equivalent within Genie, which isn’t really surprising given how bad it seems to be at performing its basic functions, but still disappointing as well.
I also love that function! When we travel with our toddlers we toggle the walking speed down a notch or two and prioritize distance walked over wait time, really helps generate a more realistic & stroller friendly plan. Also love that touring plans can take into account rider swap. Does Genie have a rider swap function, so it knows you’ll take extra time at a step? Also, Becky, any word yet on how rider swap works with ILL or Genie+?
Ah! Same. It’s totally not worth dragging a stroller all the way across the park and then having to find a spot to park it before getting in line at each place just to save a couple of minutes. I’d rather park it once and tour that area effectively. We’re on the same page!
No tests with Rider Swap yet … it’s something I’m eager to know, since my husband and I had gotten really good at “gaming” FastPass+ so we could each get single FP and then rider swap to experience and essentially get and use twice as many FastPasses in a day. I’m hoping that same ability is there with Genie+, but it will likely be logistically more difficult because Genie, so far, hasn’t handled different party members having different plans well at all.
I just need them to open Main Street Barber Shop so that I have an excuse to visit and can test out all of the “with-children” questions myself!
I’m going to sprinkle a little pixie dust here for us Touring Plans subscribers.
If Genie is sending people to A-ticket attractions and “diversions” like Magic Carpets and Swiss Family Treehouse because of low waits (and to spread out the crowds), and if more people are using Genie because its free or they are just unaware of how great Touring Plans plans are, then that is going to be good for us TP subscribers! If Genie users are spending the early part of their day exploring the caves on Tom Sawyer island or staring at the Tiki Room birds, the lines for the E-ticket attractions will be shorter for the rest of us!
I am really looking forward to your Hollywood Studios tests. It looks like HS is where Genie+ and ILL may be a real plus, and perhaps a necessity to do as much as possible in one day. But I’m curious if a touring plan will accomplish the same thing.
And replying to myself because I forgot to check the notify for follow up comments box. 🙂
My thoughts exactly, but you said them better!
I rarely spend all day at a park (first time we went to WDW, I read in the unofficial guide that heading back to our hotel for a rest or a swim as the parks get crowded, and then come back in late afternoon evening was a good strategy – we have followed that for 25 years), so not being able to add an end time in Genie would leave me doing almost nothing I wanted to do in a limited amount of time. Seems like Genie is aimed at the spend-the-whole-day-in-the-park-and-get-grouchy-from-heat-and-crowds group. Maybe that’s why the 3-4 hour wait was in its recommended itinerary.
Sooo when is Genie getting put back in the lamp?? I love Disney so I honestly wanted Genie to succeed if anything just to save face for the company after a sea of bad publicity recently under the parks division. However, this level of crash and burn has to be a nasty PR sting for them. Also, Genie has a real obsession with Swiss Treehouse! Multiple blogs I’ve read say Swiss was within the first couple of attractions on all their plans. Also read somewhere that Genie wanted people to eat the Dapper Dans for lunch (gave recommendation for the Dans for lunch time). Still, we’ve got TP to take care of it (the right way) for us. Got all my TP plans set up for December!
I’m secretly glad about this because the masses will follow the ridiculous recommendations and allow us liners an even better advantage.
Wow. I had very low expectations for the genie itinerary service, but I am still blown away by how useless it appears to be.
Hey Becky, since I’m the first comment and you’ll probably see this, are we 100% sure that you need to tap into an attraction before you can make your next G+LL slot? Under the old fastpass/maxpass rules, you could book another FP/MP as soon as your current one became valid. If you book a G+LL for 9:30-10:30am at Big Thunder, are we positive that you can’t just book another slot at 9:30 (even if you haven’t tapped into BTMRR yet)?
Ha, should’ve refreshed first, I guess I’m not the first comment. My question stands, though. 😉
Your question stands 🙂 And yes, we’ve tested this out. You do need to tap in for your “timer” to reset and make a selection possible. Otherwise you have to miss the entire window (which Genie counts as “using” the reservation) or wait the 120 minutes.
But you can still cancel the G+LL slot up until the end time, right? If I have a 10-11am G+LL booked and I see I’m not going to make it, I can cancel that at 10:45am and immediately rebook that same ride for a later return time?
According to the first 2 days of testing, yes. You can cancel your 10-11 at 10:45 and immediately “get it back” and re-book. But we’ll keep testing in case Disney changes any functionality.
Does Genie capture information about the guests and their parties before making the suggestions, specifically regarding children (and their ages), mobility issues, the amount of time planned for the park for that day, etc.? How far out does their planning go, and did they provide suggestions for the requested rides that fell outside of your cut-off time of 3:00 pm?
It appears as if Genie’s strategy is to keep things leisurely; they’ll send guests to a land, then keep them there if it looks like crowds are light in that area (have to distribute those crowds) before sending them on their way. The Touring Plans itinerary looks great if I was traveling without kids, but the bouncing back & forth from land to land takes its toll and causes problems later in the day. Having not looked at the Genie interface, I’m curious what sort of data you can give it other than “these are the attractions I want to experience”; will they allow for additional info that may help fine tune the plan, or is it a “helpful guide” in name only?
It can capture two things: whether your party has height restrictions, and whether there are any mobility issues that need to be taken into consideration.
The “time end” is not something that’s easy to do right now (if it’s possible at all), so it’s hard to force it to plan for a timeframe tighter than “all day”. The only exception to that is if you buy Genie+ or ILL for a “second park”, it will learn that you’re going there eventually.
You can give it more than just a list of attractions. You can pick interest “areas” like princesses, or something like that. Which is very cool in theory for newbie guests that know generally what they like, but none of the specifics of WDW. But in reality we haven’t seen those interests significantly impacting the plan. More testing to come on that.
Customized TouringPlans allow you to set a “Walking Speed” (Very relaxed, Relaxed, Average, Fast) and “Waiting vs. Walking” (Minimize Waiting, Balance, Minimize Walking). Genie does neither.
Thanks Tiffany! Yeah, couldn’t have picked out better results if I had just made them up myself. And yes, you’re 2 steps ahead of us! Tomorrow at DHS we’ll be having one tester use TP with early entry and another using Genie/Genie+/ILL with a later arrival to see if savvy strategy balances out vs. “paying for all the things”.
This article is hilarious. In trying to compete with touring plans, Disney inadvertently proved there own product to be inferior. Sounds like Genie rose spectacularly to my worst expectations. Ineffective touring plan? Check. Hard sell for upgrades? Check. Shuffle people to less popular attractions/dining regardless of preferences? Check. Lack of meaningful updates? Check. Completely foreseeable bugs left unaddressed? Check. Hope you do another test taking into account rope drop strategy, as that is a big part of the touring plan magic.