▶  technical seo · S5E107 · · February 23, 2026

SEO Testing Secrets from SearchPilot’s Will Critchlow | Enterprise A/B Testing That Actually Works

Guest: Will Critchlow, CEO at SearchPilot
Hosts: Jon Clark & Joe DeVita

Play episode
0:00
Apple Podcast Spotify Youtube

ABOUT THIS EPISODE

What if your SEO “best practices” are actually costing you traffic? In this episode of the Page 2 Podcast, Jon Clark and Joe DeVita sit down with Will Critchlow, CEO and co-founder of SearchPilot and former founder of Distilled, to unpack the evolution of SEO experimentation, the transition from agency to software, and why instinct-driven SEO is no longer enough. Will shares the behind-the-scenes story of spinning SearchPilot out of Distilled just before the pandemic, how enterprise brands are running statistically rigorous SEO A/B tests across thousands of pages, and why even “obvious” fixes (like breadcrumb schema updates) can reduce traffic. We also dive deep into how SEO testing is changing in the age of LLMs. With ChatGPT and other AI tools emerging as traffic sources, traditional attribution models are breaking down—and experimentation is more critical than ever. If you care about measurable SEO impact, enterprise experimentation, or navigating the future of search, this episode is packed with insight.

In This Episode

  • Why “SEO best practices” can actually hurt your traffic
  • The real story behind spinning SearchPilot out of Distilled
  • Why most SEO tests fail—and what makes a strong hypothesis
  • How enterprise brands structure scalable SEO A/B testing programs
  • The surprising breadcrumb schema test that reduced traffic
  • Why title tag tests are high-risk, high-reward
  • How LLMs like ChatGPT are becoming measurable traffic sources
  • Why net impact matters more than isolated traffic gains
  • The shift from services to software—and leadership lessons along the way
  • Why focus, not innovation, is often the real competitive advantage

This episode is a masterclass in bringing scientific rigor to SEO and building a controllable performance engine with provable ROI.

Connect with Will Critchlow

LinkedIn
X
Threads
Bluesky

Chapters

  • 0:00

    Introduction to SEO and Testing

  • 2:59

    The Evolution of Distilled and SearchPilot

  • 6:07

    Transitioning from Service to Software

  • 9:01

    The Acquisition Journey and Its Challenges

  • 11:56

    Focus and Personal Growth in Business

  • 15:00

    SEO Testing Methodologies Explained

  • 18:12

    Understanding Test Parameters and Success Metrics

  • 20:53

    The Role of AI in SEO Testing

  • 23:47

    Leveraging User-Generated Content

  • 26:59

    The Importance of Crawl Data

  • 29:59

    Insights from Test Failures and Successes

  • 32:57

    Leadership Lessons and Personal Development

  • 36:09

    Final Thoughts and Resources

Resources mentioned