Brian Offenberger's marketing and sales keynote speaker blog

Posts Tagged ‘A/B Testing’

Improve Your Website’s Effectiveness with Google Website Optimizer

Posted on: July 24th, 2009 by Joe Giordano No Comments

Apogee Search Founder, Bill Leake, explains how to use Google Website Optimizer to improve your website effectiveness.

Bill Leake

Driving traffic to your website is only half the battle. Your site also needs to be constructed to convert those visitors into leads or sales. A valuable first step to making your website more effective is to utilize Google Website Optimizer.  Google Website Optimizer is a free tool that website owners can use to determine the most effective combination of content on their site through a variety of tests. The results from these tests can then be used to maximize conversion rates among existing traffic to your site.

GWO can conduct both A/B testing and multivariate testing. A/B testing will test two different pages against one another, and multivariate testing will test the various elements on the page to determine the best combination.

Strategies to keep in mind when using Google Website Optimizer:

  • Ask big questions. You will only receive answers for the things that you actively test. You should test drastic changes rather than slight changes to obtain statistically significant data.
  • Spend time developing well-written copy to test.
  • Wait for significant data. GWO will indicate when a sample is large enough to be useful.

Different situations in which one would want to use A/B testing versus multivariate testing:

  • Multivariate tests compare different elements of a page. Google uses a full factorial test so it is very easy to generate a large number of combinations even when testing just a few elements. These tests require a high amount of traffic to achieve statistical significance.
  • A/B testing is suited for sites with less traffic because it requires a much smaller sample size to achieve statistical significance. This is the best option for testing a complete site redesign.

Rules to follow when using Google Website Optimizer:

  • Wait for statistically significant data. As discussed earlier, initial results might reflect random chance, so wait until you have had enough traffic for the information to be significant.
  • Limit the elements being tested. If you create too many different combinations, it will take a great deal of time for GWO to produce meaningful results from your test.
  • Spend as much time developing content for a test as you would for a website. Even though this is a test, you are testing potential real options for your site. Do not get sloppy with the content just because it is a test.

Bill Leake is the CEO and Founder of Apogee Search, the largest search engine marketing firm in the Southwest, one of the 20 largest in North America and one of the fastest growing companies on the Inc. 500 list. In addition to leading Apogee Search, Bill also serves as the president of the Austin Interactive Marketing Association, and as the chairman of the SEMPO (Search Engine Marketing Professionals Organization) committee.

Multivariable vs. Multiple Variable Testing

Posted on: April 4th, 2007 by Joe Giordano No Comments

Here is an interesting article by this week’s show guest Seth Rosenblatt.

Seth Rosenblatt is the Vice President of Marketing and Business Development at Optimost, the leading provider of online multivariable testing solutions. Optimost is a technology and services company specializing in comprehensive real-time testing and conversion rate marketing. For more information about Optimost, please visit their website

Strategy guru Michael Porter of Harvard Business School once defined a global enterprise as a company “whose competitive position in one country is significantly affected by its position in other countries.” This was meant to distinguish a global company from a multinational company, which is merely a company that has operations in more than one country. Maybe that seems like a subtle distinction, but in reality this has real significance. The global firm targets the international market without distinguishing national or political boundaries – it integrates its business across political boundaries, coordinates its product chain and value chain, and thinks of the entire world as its marketplace. Although a multinational firm may gain some economies of scale from being in multiple countries, it roughly views its business a series of independent businesses in distinct marketplaces.

This distinction is a good metaphor for how marketers look at their web pages. Is your web page a series of unrelated content items, or do the items on the page have some connection to each other? For many this seems like a rhetorical question – of course your web page has interrelated elements. For example, it’s possible that certain headlines won’t make sense with certain main images, and if you have a price point quoted at the top of the page, any mention of that price elsewhere in the page better have that same price! Also, if you make a certain offer at the top of the page, does it reinforce the message to mention that offer again in the middle of the page, or does the redundancy hurt you?

But despite the universal agreement that everything on our web sites is interrelated, many marketers looking at testing their web pages don’t take into account these relationships and constraints. This is in part driven by the fact that the terminology we all use in this industry is very loose. For example, the word “multivariable” is used by different companies to mean different things. Unfortunately many companies use the term “multivariable” or “multivariate” when they really mean “multiple variable.” Just like global vs. multinational, there is an important distinction here. If you are testing four different content elements on your page simultaneously, you are doing “multiple variable” testing – essentially you are running four independent tests on the same page. Certainly this is more efficient than running a test on only a single element, but it both limits what values you can actually test and fails to take into account any of those relationships and constraints among elements on the page.

For example, if you run a “multiple variable” test, you can not test any value which potentially could conflict with another element on the page (e.g., you certainly don’t want the same image to show up in adjacent slots; you don’t want that headline about the motorcycle showing up under the picture of the car, etc.). “Multiple variable” testing severely restricts what you can do – analogous to the multinational corporation which is not taking advantage of the potential “global” relationships among its subsidiaries.

True “multivariable” testing, on the other hand, takes into account these relationships and constraints among the elements on these pages. It does require a more advanced methodology to get it right, but it gives the marketer the ability to really understand how elements on the page work together to both give the best conversion rate possible and to provide those learnings as to what is and what isn’t working. Many marketers make the mistake by assuming that some of the older methodologies, like Taguchi, will allow them to do this, where at best it will only allow “multiple variable” testing (more information on various “design of experiments” methodologies can be found on our web site at

So, next time you talk to someone about approaches to “multivariable” testing on your web site, make sure they’re not mixing it up with “multiple variable” testing – it would be as mistaken as a multinational company thinking it was a global powerhouse.

Read some Optimost success stories to learn how clients achieved success using our platform, If you would like someone to contact you from Optimost, please click here.

Measurable Results with Multivariable Testing

Posted on: April 2nd, 2007 by Joe Giordano No Comments

Seth RosenblattThis weeks guest on Online Marketing with RSS Ray is Seth Rosenblatt, Vice President of Business Development for Optimost.

Optimost – Optimost revolutionized online testing in 2001 with the first on-demand multivariable testing platform. Since then they have remained the leading provider of testing solutions by focusing on providing the most advanced multivariable testing technology available.

The main challenge facing any advertiser is presenting consumers with the right message to drive them to buy a particular product or perform a particular action. Many advertisers may test five or ten versions of their advertising, landing pages, or any other page in their sales funnel, to determine which creative performs best. These experiments may include variations in copy, layout, or images. Optimost’s advanced multivariable testing techniques allows advertisers to simultaneously test these factors and many more-up to millions of different permutations, in fact-for the same (or less) effort and expense.

Subscribe to our Blog

This blog covers everything about selling in the internet era. Join thousands of others and subscribe now.

Get blog updates from Brian Offenberger via RSS Feed Get blog updates from Brian Offenberger via email

Get updates via RSS feed or Email.

. .

Meet Brian

So who is this guy named
Brian Offenberger?

Meet him here >> Meet Brian Offenberger marketing strategist and keynote speaker
Keynote speaker and marketing strategist Brian Offenberger
Free Newsletter Get insider marketing and sales tips delivered directly to your inbox. Each month Brian sends each of his newsletter subscribers valuable resources to help keep their marketing plan one step ahead of the competition. Sign Up Today!