How to Document Funnel Optimization A/B Tests Effectively?

As a Growth Manager for an online travel booking platform, you're documenting an A/B test to optimize the hotel search results page.

Evaluate the following documentation elements:


  • Element I: Hypothesis statement: "Displaying user ratings more prominently will increase click-through rates to hotel detail pages by 10%, based on customer survey data indicating that ratings are a primary decision factor."

  • Element II: Mid-test adjustment: Expanding the test to 100% of traffic after one week due to initial positive results, without documenting the decision process or potential risks.

  • Element III: Test parameters: 4-week duration, applied to 50% of desktop traffic, using Optimizely for implementation. Success defined as 10% CTR increase, with a plan to roll back if CTR decreases by more than 5%.

  • Element IV: Post-test analysis: CTR increased by 8%, falling short of the 10% goal. Unexpected finding: mobile users showed higher engagement than desktop. Recommendation to refine mobile layout for a follow-up test.

Which elements align with best practices for documenting A/B tests for funnel optimization?

Choose the closest answer: