Genie Testing: Hollywood Studios Double-Head-to-Head

Share This!

Yesterday (October 23rd) was a gorgeous Saturday in Hollywood Studios, and it was the last day of our marathon Genie testing plan – we’ve all earned a few days of rest … before we do even more testing! And this was our busiest day of testing yet – we had four different testers in Hollywood Studios, attempting to “do the whole park” before they left for the day. And we had two live streams (on YouTube and Instagram) to talk about what was happening in the afternoon. So what happened and what did we learn? Strap in – there’s a lot to cover!

Test 1: Knowledge vs Money

When we ran a Genie/TouringPlans head to head in the Magic Kingdom on Thursday, several people commented that we messed with the results by kicking off after rope drop, because all savvy TouringPlans users know that rope drop is the key to saving time. And while I don’t think all of the members of our community do rope drop, I agree that it’s a key time-saving strategy. So our first test of the day was to pit a savvy TouringPlans tester with a touring plan and early theme park entry against a non-touring plan, late arrival tester equipped with all of the money she needed to pay her way around lines. So what did their days look like?

Tester 1: Early Entry and a Touring Plan

Tester 1 was up bright and early for a stroll through Sunset Boulevard during Chrismalloween.

Tester 1 entered the park at 7:45 – well before the advertised early theme park entry time of 8:30. She was able to walk almost all of the way to the entrance of the plaza for Rock’n’Roller Coaster and line up with a cast member to be the first guests on that ride for the day. She used her Touring Plan to choose that step, and to re-optimize after every step for the entire rest of the day. Here was how her day progressed:

Our tester joined us for streaming and then called it a day, but she could have seen Beauty and the Beast at 2, walked on to MuppetVision and Star Tours (which were walk-ons at that point in the afternoon) and watched the Frozen Sing-Along Celebration at 3:30. She would’ve been done with her day by 4.

Our tester commented that this was a perfect and “golden” day. She admitted that she was tempted to follow her instincts rather than the plan several times. But if she had, for example, gotten in line for Rise of the Resistance instead of Rock’n’Roller Coaster first thing, that probably would’ve been the only attraction she did before the park officially opened, and then she would’ve lost her advantage over the rest of the crowds.

Early Theme Park Entry was absolutely key here. She finished three attractions before other guests could even get in the park. If she started 30 minutes later, she would’ve had a 20 minute wait for Toy Story Mania at 9:30 instead of 1 minute at 8:58. But it was the combination of early entry and informed strategy that allowed her to be successful and have such a fantastic day. If this test helped you recognize the value of a resort stay, talk to the TouringPlans Travel Agents – they’re top-notch and will get you set up.

Tester 2: Arrive Late and Buy All of the Things

Our Pay-Your-Way Tester paid her way to breeze all the way up to the room with “the circle robot and the lady” as one of our testers fondly refers to this duo.

Tester 2 was playing someone that didn’t want to get to the park before 11 am. Maybe that person has kids that don’t want to get up early, or needs a slower start in the morning. That being said, she still did wake up early to purchase Genie+ and make her first return time reservation. She ended up picking Slinky Dog Dash. Since she didn’t log in right at 7, the first available return time was 3 pm – 4 pm. Then, because she wasn’t a resort guest, she had to wait until 9 am to book the Individual Lightning Lane Access return times that she needed. We’ll look at those and the rest of her schedule below:

  • Grabbed Slinky Dog Dash return time (3:00-4:00) at 7:00
  • Grabbed Rise of the Resistance return time (5:20-6:20) at 9:00
  • Grabbed Runaway Railway return time (12:10-1:10) at 9:00
  • Grabbed Millennium Falcon return time (4:30-5:30) at 11:04
  • Entered the park a little after 11:00
  • Frozen Sing-Along Celebration at 11:30
  • Runaway Railway via Lightning Lane (LL) – 10 minute wait at 12:10
  • MuppetVision – 11 minute wait at 12:31
  • Star Tours – 1 minute wait at 1:02
  • Grabbed Tower of Terror return time (2:50-3:50) at 1:04
  • Walt Disney Presents – 25 minutes of relaxed touring
  • Beauty and the Beast at 2:00
  • Tower of Terror via LL – 10 minute wait at 2:50
  • Grabbed Rock’n’Roller Coaster return time (3:20-4:20) while in line for Tower of Terror
  • Slinky Dog Dash via LL – 5 minute wait at 3:05
  • Grabbed Toy Story Mania return time (4:35-5:35) while in line for Slinky Dog Dash
  • Rock’n’Roller Coaster via LL – 11 minute wait at 3:27
  • Grabbed Alien Swirling Saucer return time (3:50-4:50) while in line for Rock’n’Roller Coaster
  • Alien Swirling Saucers via LL – 3 minute wait at 4:19
  • Millennium Falcon: Smuggler’s Run via LL – 9 minute wait at 4:29
  • Toy Story Mania via LL – 6 minute wait at 4:50
  • Rise of the Resistance – 13 minute wait at 5:18

Wow. That’s a lot of steps because of all of the return time finding/reserving/using. But it was successful! The pay-your-way tester was able to complete virtually everything in the park between the hours of 11 am and 5:30 pm, typically the most crowded hours of the day. She banked return times for mid- and late-afternoon while doing low-wait or no-wait shows or attractions early in her day. That was HUGE. I’m willing to bet that it’s possible to grab return times all morning while doing another park, and then hop to Hollywood Studios at 2, burn all of the return times and do everything in the park after hopping.

That being said, our tester said it was mentally exhausting trying to keep track of all of the return times. Genie doesn’t push any notifications when your return windows open (or are about to open or about to close). That mental exhaustion, if you do keep up with it, is counter-balanced by the emotional rush of flying past everyone in long standby lines.

Conclusions of this test

In this case, knowledge and money just about balanced out. Both of our testers felt like they had really great days. One got to not rope drop, and one got to not spend additional money. But knowledge vs money may be a bit of a misnomer. Our pay-your-way tester wasn’t flying blind and just randomly grabbing whatever return times looked good. She knew which ones to get first before they booked up, and which to wait on. She knew when she’d be eligible to make another reservation. And she knew to visit the low- and no-wait attractions even before she booked return times for other longer-wait attractions, knowing she’d be able to get them later and it was best to spend peak hours on less popular attractions. So even pay-your-way requires some research or inside knowledge.

Test 2: Does Genie Care About My Preferences?

One of the key claims of the Genie plan for your day is that it takes your preferences into consideration and builds the best day based on what you tell it. Cool? Very cool. In theory. In Thursday’s test, all preferences were almost wholly ignored. Zero fun. So we wanted to test what two people with different inputs would experience as their output. They weren’t allowed to “work ahead” – they had to wait for the times that Genie suggested, because we wanted to mimic what inexperienced park visitors might do. And they weren’t allowed to “hack” Genie with any of the tricks (like mass step deletion to force optimization) we had learned in previous days.

Tester 3: Star Wars Fanatic

Our tester pretended that she only cared about Star Wars things. She picked Star Wars attractions as her must-dos, and noted her interest in Star Wars as a category. And her plan pretty much reflected her preferences! It was a minor miracle! I’ll list her initial plan here, with some breaks for notes:

  • Step 1: Walt Disney Presents at 9:10
    – Seriously? Literally anything else would’ve been a better “rope drop” suggestion. Don’t do Walt Disney Presents first. Ever.
  • Step 2: Toy Story Mania at 10:10 – 37 minute wait
    – Not a bad suggested attraction, but it’s over an hour after park opening. No need for that much downtime. And it resulted in a much longer wait than if she had gone earlier.
  • Step 3: Lunch at Ronto Roasters at 11:15 (more about this step later)
  • Step 4: Rise of the Resistance at 12:20 – 93 minute wait
    – This left an hour for lunch, which is potentially not unreasonable. But it was a long wait for Rise.
  • Step 5: Droid Depot at 3:10
    – Wait, WHAT?! Disney is upselling her to buy a $100+ droid? Okay, I mean, she selected all of the Star Wars things, but still … bold move. So we assumed she wouldn’t fall for that, and allowed her to attempt a Swaperoo!!!!!

    From now on, I will imagine Spiderman swinging in heroically when Iron Man yells “Swaperoo!”
  • Step 5 (swaperoo’d): Alien Swirling Saucers at 3:10 – 29 minute wait
    – This was the longest wait for Alien Swirling Saucers out of any of the four testers
  • Step 6: Millennium Falcon: Smugglers Run at 3:50 – 58 minutes
    – Ouch, another long wait. In fact, the longest wait (by 18 minutes) out of any of the four testers

At that point, our tester bailed for the day. She experienced 5 attractions between 9 am and 5 pm. And two of them were really long waits. If she had finished her plan, it would’ve looked something like this:

  • Step 7: Dinner at Docking Bay 7 at 5:30
  • MuppetVision at 6:35 (would’ve been a walk on)
  • Star Tours at 7:20 (would’ve been a walk on)
  • Tower of Terror at 8:10 (would’ve been a very low wait or walk on)

So at the end of the day, no Runaway Railway, no Slinky Dog Dash, no repeated Star Wars attractions that might have made her happy. And definitely not ideal waits. Things were spread out in a way that left odd gaps and breaks throughout the day. Our tester thought it felt awkward and not a pace she would choose for the future.

Tester 4: Blank Slate

When the area around like the Millennium Falcon looks like this, it’s almost definitely a good time to ride. But our “Blank Slate” tester got to sit around and enjoy the view instead.

The fourth and final tester (or in this case … Testa) followed a Genie plan after giving it no input or preferences. It provided very interesting contrasts to the more aggressive testers in the first matchup, as well as the Star Wars tester.

  • Step 1: Toy Story Mania at 9:30 – 20 minute wait
    This might not strike you as odd if you didn’t know that this tester was in the park at 8:30. Genie recognized the park location, but didn’t nothing to fill the first hour. It was painful to watch other guests being efficient with their time as he killed time waiting.
  • Step 2: Droid Depot at 10:40
    NO. STOP IT. Droid Depot recommendation for a Star Wars fan? I’ll give it a pass. Droid Depot recommendation for someone with no thoughts and preferences? No. Droid Depot as the second step of an entire day for that person? Absolutely not. Never. Stop it. Do not pass go. I’m appalled.
  • Step 2 (swaperoo’d): Walt Disney Presents at 10:10
    At least the swaperoo filled a little of the gap in time. But the swaperoo options were not great. Now both of our free Genie users were doing Walt Disney Presents and Toy Story Mania as their first two steps.
  • Step 3: Lunch at Ronto Roasters at 11:10
    Okay, 2-for-2 on recommendations for early lunch at Ronto Roasters. And it was very apparent that our testers weren’t the only ones that got this recommendation. Even though the park wasn’t packed, and 11:10 is a pretty early lunch, every single seat (indoors and outdoors) at Ronto Roasters was taken. Our tester ate lunch on a trash can. But he at least said that the veggie wrap was good. So that’s something.
  • Step 4: Millennium Falcon Smugglers Run at 12:10 – 40 minute wait
    This wasn’t a terrible wait, but it was the 3rd attraction of the day and already in the afternoon.
  • Step 5: Beauty and the Beast at 1:00
    This step was originally Star Tours at 2 pm at the beginning of the day. But when the tester exited Millennium Falcon, it was swapped out as a “better fit” according to Genie.
  • Step 6: Slinky Dog Dash at 3:00 – 54 minute wait
    This step was originally Rise of the Resistance at 3:00. When Rise of the Resistance went down (while Tester 1 was in line around 10:00), Genie took it off of the plan due to downtime. It never reappeared. 54 minutes isn’t a terrible wait for Slinky Dog Dash. But it was still not enjoyable for this tester after wasting a lot of time in the morning. Especially since our pay-your-way tester blew  past him in the Lightning Lane while he was in standby.
  • Step 7: Alien Swirling Saucers at 4:15 – 10 minute wait
  • Step 8: MuppetVision at 5:15 – 8 minute wait
  • Step 9: Dinner at ABC Commissary at 6:00
    Not a bad dining recommendation as far as quick service is concerned. Satisfaction has been increasing at this location.
  • Step 10: Vacation Fun (short) at 6:50 – 6 minute wait
    Our tester did not think this vacation day was fun.
  • Step 11: Tower of Terror at 7:35 – 8 minute wait
    Crowds were really emptying out of the park by this point. Wait times were really low for almost every attraction.
  • Step 12: Wonderful World of Animation at 8:45
    Our tester gave up before this point. The day was too disappointing to continue. But he won the endurance award for being in the park longer than any other tester!

This tester was probably the most disappointed and annoyed of any of our testers today. He evaluated the impact of this plan on him as “I can feel my will to live ebbing away”. The only thing longer than this tester’s day in the park is … his hair.

Conclusions

Despite the disappointment and annoyance levels of our Genie testers, we learned some very interesting things! When conditions changed, the Star Wars steps in his plan mysteriously disappeared, while they stayed on the plan for our Star Wars fanatic. It may be our first example of Genie potentially taking a preference into account. If Rise went down, and Disney knew that they would have to accommodate all ILL reservation holders for that time (and the rest of the day) as well as standby waiters, it makes total sense that they wouldn’t suggest anyone go to the ride for the rest of the day, perhaps unless they had already indicated that it was a must-do. But also, Disney pushed the hard upsell of Droid Depot to a non-Star Wars fan. Weird. And the pace of the day was soooo slooooow while still not hitting all (or many) headliners.

Overall Results

Qualitatively, the differences between the two groups of testers were blindingly obvious. Two testers felt great about their choices and how their day went (which is saying something, given that they’ve been going at this for 5 straight days now). Two testers felt let-down and like they wasted a lot of time.

But enough with the qualitative stuff. Let’s look at the numbers. For these summaries, we’ll assume that testers 1 and 3 finished out their day since we can easily pull the data to see what their waits would’ve been.

  • Tester 1 (Touring Plan and early entry) experienced 13 attractions in 7.5 hours. She waited in line for 200 minutes (24 of this was downtime for Rise). The average rating of the attractions that she experienced was 3.88.
  • Tester 2 (Pay-Your-Way) experienced the same 13 attractions in a little over 6.5 hours (but with prep time for up to 4 hours before she was in the park). She waited in line for 79 minutes. The average rating of the attractions that she experienced was 3.88. She paid a total of $40 (pre-tax) for her line-skipping powers. That was $15 for Genie+, $15 for Rise of the Resistance, and $10 for Runaway Railway.
  • Tester 3 (Genie – Star Wars Fanatic) experienced 8 attractions in 11.5 hours. She waited in line for 226 minutes. The average rating of the attractions that she experienced was 4.0.
  • Tester 4 (Genie – No Preferences) experienced 10 attractions in 12.5-13 hours (including the wasted first hour of the day). He waited in line for 143 minutes. The average rating of the attractions that he experienced was 3.78.

Tester 3 got to see all of the Star Wars things … but so did Testers 1 and 2, and they had more attractions in their day and less overall time in lines. It’s a rough comparison. And in Genie, there’s currently no way to set an end time (without pretending that you’re park hopping even if you’re not), so some bigger attractions get shoved to the end of the day, which means you’ll miss them if you don’t spend that full 13 hours in the park. And it’s apparent that just like Disney needs to bump up posted wait times to avoid angry guests, they’re doing the same thing with Genie plans. They don’t want you to feel like you’re falling behind, so they’re padding them with TONS of “bonus” time that equal a lot of waiting around.

Like we’ve mentioned before, the big miss here is lack of a feedback loop. You can’t mark steps complete, so Genie will never know if you’re running ahead (or behind for that matter). And you can’t tell it you really hated its suggestion unless you delete or force a swaperoo, and it doesn’t seem like even those actions are taken into consideration for the rest of the steps in your day.

We had low expectations for Genie today, and we were still disappointed. Genie+ and ILL were worth the purchase price, but Genie plans were a total disaster when compared to savvy strategy.

That wraps up our intense 5-day testing plan. We’ll be back tomorrow with an overall look at things we’ve learned and any tips/tricks we’d recommend based on current conditions. Remember, Disney will keep trying to tweak and update Genie, and we’ll keep testing and trying to stay one jump (or sixteen jumps) ahead. Have any questions or concerns you’d like to share? Leave a comment below!

 

You May Also Like...

Becky Gandillon

Becky Gandillon was trained in biomedical engineering, but is now a full-time data and analytics nerd. She loves problem solving and travelling. She and her husband, Jeff, live in St. Louis with their two daughters and they have Disney family movie night every Saturday. You can follow her on LinkedIn: https://www.linkedin.com/in/becky-gandillon/ or instagram @raisingminniemes

26 thoughts on “Genie Testing: Hollywood Studios Double-Head-to-Head

  • October 24, 2021 at 1:26 pm
    Permalink

    Great article but the question I am left with is how much did the Buy Everything Tester (Tester 2) end up paying extra for their Genie plus and lightning lane passes?

    Reply
    • October 24, 2021 at 1:39 pm
      Permalink

      Sorry about that, Mark! I edited the article to include that information. But the cost was:
      $15 for Genie+
      $15 for Rise of the Resistance
      $10 for Runaway Railway
      That’s a total of $40 (pre-tax)

      Reply
  • October 24, 2021 at 1:37 pm
    Permalink

    This was extremely helpful. Thank you.

    Reply
  • October 24, 2021 at 1:55 pm
    Permalink

    How much more time would it have been at the Falcon if she didn’t use the Single Rider Line?

    Just wondering if I am there with my family of 4, does this work? Or would the additional time at the Falcon change all the later times?

    Reply
    • October 24, 2021 at 2:05 pm
      Permalink

      Actual wait time at that time of the day was probably around 5 minutes – it was posted as 15. So it wouldn’t have pushed you too much “further behind” the test case.

      Reply
      • October 24, 2021 at 8:13 pm
        Permalink

        Great blog post. Thanks to the team for this detailed testing and commentary!!
        But this was a below-average attendance day (3/10). What happens if you are going to be there on an 8/10 or even 10/10 day. Can a TouringPlan with Genie+ still get you on all rides including ROTR and MMRR without ILL?

      • October 24, 2021 at 9:13 pm
        Permalink

        Good question, Rick. We could test out some theory, but we haven’t seen a crowd level that big in a long time. It’ll be a totally new ball game with those crowds/waits and Genie/Lightning Lane. So we’ll have to run some more tests.

  • October 24, 2021 at 1:55 pm
    Permalink

    Are there any plans for tests to mimic those who stay offsite and can’t do early entry but are at the parks when they are allowed in (the sort of rope drop crowd)?

    Reply
    • October 24, 2021 at 2:06 pm
      Permalink

      We can definitely put that on our wish list of more test cases (and yes, we have a wish list).

      Reply
  • October 24, 2021 at 4:24 pm
    Permalink

    I said this a couple of days ago, and I’ll say it again. The goal of Genie appears to be direct crowds away from their headliners toward their lesser attractions to better distribute crowds around their parks, with low wait times being the priority. (We can discuss the ethics of recommending some of those attractions separately.)

    I’m hoping that at some point, Touring Plans will be suggesting the best opportunities to use Lightning Lane. For infrequent guests who may wait years between visits, paying for LIghtning Lane to supplement a Touring Plan may make sense, when you consider the amount spent on airfare and on-side hotels.

    Reply
    • October 24, 2021 at 8:38 pm
      Permalink

      And I think that is actually a good thing for us TP subscribers! If Genie is steering the typical Disney guest who doesn’t plan like us Liners to A-ticket attractions that always have little or no wait, the lines for E-ticket attractions will be shorter and the need for Genie+ and individual,Lightning Lane purchases will be lessened.

      Reply
    • October 24, 2021 at 9:03 pm
      Permalink

      You can bet that at some point, we’ll have enough data to start suggesting when (or when not) to purchase things like Genie+ and Individual Lightning Lane Access, as well as which rides to grab return times for, and when. But it takes time to accumulate that sort of data. In the meantime, I’ll be writing up some anecdotal-based strategies tonight for a post tomorrow morning.

      Reply
  • October 24, 2021 at 4:48 pm
    Permalink

    What is a early entry rope drop for Rise typically look like? We are going with a large group 13 total and would prefer not to have to drop over $100 if the wait would only be 20-40 minutes if it’s the first thing I do.

    Reply
    • October 24, 2021 at 9:06 pm
      Permalink

      I don’t know if Early Theme Park Entry has been happening long enough for there to be a “typically”. It probably high depends on what the crowd level is for that day. Yesterday was crowd level 1, and it was posted at 45 minutes for the entire early entry period – you probably would’ve waited less than that. But two days before that, it was also a crowd level 1 day and it was posted at 65 minutes during early entry. If you can arrive even before Early Theme Park Entry “opens” (significantly early) and be toward the front of the pack, it might be worth it. If you arrive any time after the start of Early Theme Park Entry, I wouldn’t risk it.

      Reply
  • October 24, 2021 at 6:56 pm
    Permalink

    I counted 6 Lightening Lane rides for tester 2. I thought LL was limited to 2 rides?

    Reply
    • October 24, 2021 at 8:50 pm
      Permalink

      There are lightning lanes for many rides. Each park has two rides whose lightning lanes can be accessed by purchasing individual lightning lane access. For the other rides in the park, you can use the regular $15/per person per day Genie + to access the lightning lanes.

      Reply
    • October 24, 2021 at 9:11 pm
      Permalink

      Yeah, Mike, the terminology is confusing. The Lightning Lane refers to the physical queue. So she’s was in the Lightning Lane instead of the Standby queue.
      You are limited to purchasing two Individual Lightning Lane Access rides per day, but you can access as many as possible with Genie+ on top of those two purchases. ILL And Genie+ both allow you to use the (physical) Lightning Lane.

      Reply
  • October 24, 2021 at 8:55 pm
    Permalink

    Thank you for doing all this research, it is very helpful! I am a very confused about when you are allowed to get additional Genie + lightening lane passes (not the individual passes). Tester 2 was able to load up several that didn’t appear to be two hours waiting in-between each request later in the day. You can get your first Genie+ LL pass at 7am, when staying on property, and it is for use at 9am or after? Then I need to wait until 11am to get next LL pass, then 1pm, then 3pm? It seemed at around 3, she was able to get a few more LL passes without waiting 2 hours in-between. I would like to know when I am allowed to request additional LL passes, every two hours unless I use my current LL pass first, then I can get another one? And I have to wait until 11am to get my second LL pass even if I use my first pass before then? Sorry, this part is very confusing and I have seen some people say that if you get a LL pass for later in the day at a different park, you actually are able to get additional LL passes before then. This is mentally exhausting!

    Reply
  • October 25, 2021 at 9:49 am
    Permalink

    Since the cost is the same for now ($15), if someone were to purchase either Genie+ for the day or a Rise of the Resistance Lightning Lane (but not both), which would you recommend as the biggest value (and time-saver)?

    Reply
    • October 25, 2021 at 9:14 pm
      Permalink

      Ooh, Peter with the hard-hitting question. Time-savings-wise, you’re certainly going to save more time with Genie+ than with just Rise. But you’ll typically be avoiding several (or many) medium to medium-high waits instead of one potentially really high wait. But I’d rather time that one wait in such a way that I can potentially avoid crowds than balance all of the others throughout the day.

      Reply
  • October 25, 2021 at 12:15 pm
    Permalink

    It would be great to add a table of rides per hour in park, wait time divided by rides!

    Reply
  • October 25, 2021 at 3:23 pm
    Permalink

    I think the most surprising thing to me about these results was the brazen push by Genie toward expensive up charges. I mean we all knew there would be aggressive advertising for Genie+ and ILL, but sending random people to go buy a $100 droid, when they expressed no interest and hadn’t even ridden the main rides at the park, was pretty shocking, and a bit icky.

    Reply
    • October 25, 2021 at 9:21 pm
      Permalink

      Ha! I thought I might be the only one that called things “icky”. Glad there’s more than just me 🙂

      Reply
  • October 26, 2021 at 9:40 pm
    Permalink

    Great article, thank you. I’m surprised and confused that Tester 2 was able to grab a toy story mania return time before tapping in for Rock’n’Roller Coaster (and <2 hrs after previous). Did tapping in for Slinky count instead? Same thing happens later, grabbing Alien Swirling Saucers before Toy Story Mania.

    Reply
  • October 28, 2021 at 5:40 pm
    Permalink

    I’m curious if when you do these head to head tests with 1 person using EE do you include the time the EE person spends waiting for park to open in the results of time spent in line?

    Reply
    • October 28, 2021 at 9:05 pm
      Permalink

      No, we don’t include the time the person spends waiting for the park to open. We assume the early entry person waits for their time to enter, and that the regular rope drop person waits for their time to enter, so they’re likely equivalent.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *