After keeping busy all summer and early fall with client meetings, events, and absorbing repeated acquisition announcements, it's great to be home again and conducting research!

Most notably, my latest round of research on the topic of online testing, titled "The State of Online Testing 2010," published this week.  This document is the output of a survey conducted with online testing users alongside the "The Forrester Wave™: Online Testing, Q3 2010" research process.

As you may be aware I'm a huge proponent of online testing.  I think it is an underutilized and highly effective site optimization tool, with a wide array of applications.  I frequently find myself uttering the rallying cry "Test, Test, and Test some more!" and I often refer to online testing as the wind tunnel of site optimization, a contained and structured environment for evaluating the content, promotions, and customer experience components of the online marketing mix.
















(Image source:

What did we learn about the state of online testing? The report showcases the data in it's full glory, but here are some highlights:

  • Online testing offers broad benefits but adoption varies. Despite broad applicability of online testing, users are primarily testing against revenue oriented goals they can directly measure, and adoption is significantly higher for B2C marketers in consumer oriented industries.
  • Online testing remains a challenging endeavor. Marketers need help with online testing and more often than not are working with third parties such as vendors and agencies to bolster their online testing programs.  Managing the testing process is by far the biggest obstacle users struggle with.
  • Ultimately, online testing is rewarding. Despite the challenges of online testing, users report tremendous success across a wide variety of metrics.

As a side note, I will be addressing the site optimization process, a crucial driver to online testing success, in an upcoming report.

Overall, I'm encouraged by this data. Yes, online testing programs require care and feeding, and must be considered a serious investment, but testing is a powerful and effective tool in the marketer's toolkit that is well worth the effort. It's also worth noting that while many current online testing programs are focused on direct revenue generation – and rightly so – online testing is applicable to more than driving orders, and we're only now scratching the surface of what online testing can accomplish.

PS – If you're in the mood for some more Customer Intelligence info, please check out my new Web analytics case study featuring National Instruments, it's a great look at how planning and collaboration pay dividends in the vendor selection and implementation process.  And the Forrester CI team has published "Customer Intelligence Trends to Watch in 2011," a look forward into the ongoing evolution of marketing knowledge – new data sources, increasing complexity, real-time intelligence, and its application throughout the enterprise.  Happy reading!