free PDF version of “Always Be Testing” - JohnQuarto.com

337 Pages·2008·11.53 MB·English

Advance praise for Always Be Testing: The Complete Guide to Google™ Website Optimizer
I can tell you two things. The first thing is: Testing works. If you’re not doing it well, you’re missing out.
The second? Bryan Eisenberg knows what he’s talking about. Time to get started, time to start testing.
—Seth Godin, Author of Meatball Sundae
Always Be Testing is a very readable and practical guide to landing page testing from an experienced
master practitioner. Bryan has emerged from the trenches with actionable advice and detailed testing tips
that will make your cash register ring more often. Pick up a copy before your competitors do!
—Tim Ash, Author of Amazon’s e-commerce bestseller Landing Page Optimization: The Definitive
Guide to Testing and Tuning for Conversions
If you want to become an expert in website testing, this is the place to start. The book’s depth and breadth
are particularly impressive, from why testing matters to what and how to test. Bryan and John have com-
bined years of experience in persuasive marketing with their deep knowledge of Google Website Optimizer
to deliver actionable insights that will make rock stars out of any marketing team.
—Tom Leung, Senior Business Product Manager, Google Website Optimizer
This is the best collection of practical testing advice that is available today. If you want to get ahead in
testing, then you and everyone involved in building and maintaining your website need to read it. Always
Be Testing offers basic and advanced testing principles, based on years of experience, that help you opti-
mize (with or without Google Website Optimizer) quickly and efficiently. The authors provide practical
exercises to help you evaluate your site objectively, tools to help you identify what to test, and resources
to expand your knowledge. This book should be required reading for anyone interested in helping online
visitors achieve their goals—it easily can save you the time and expense of unproductive results that are
due to common mistakes.
—Dylan Lewis, Senior Manager, Web Measurement, Intuit
By unlocking the secrets of Google Website Optimizer, Bryan Eisenberg and John Quarto-vonTivadar’s
clear prose combined with case studies allows any reader involved in an Internet business to go beyond the
typical implementation and redefine their business goals and execution.
—Mike Smith, Senior Vice President, Forbes.com
This book provides not only valuable insight into the importance of website testing and how to undertake
tests that provide meaningful data, but also strategies for dealing with website design and content ele-
ments that have a high-ranking advocate but that may be ineffective—or even counterproductive—when
it comes to creating sales.
—Kurt Peters, Editor in Chief, Internet Retailer
Always Be Testing is the essential handbook that gets you testing and keeps you testing. In this game, the
difference between Winners and Losers is that Winners take the time to figure out which of their ideas
stink. Losers don’t. This book is for everyone who wants to be a Winner!”
—Avinash Kaushik, Author of Web Analytics: An Hour A Day
9900663377ffffiirrss..iinndddd ii 66//2266//0088 1122::1122::1133 AAMM
Conversion is the Holy Grail for any online business. But testing for conversion—the competitive game
changer—has always seemed boring, tedious, and complicated. With Always Be Testing, the job of test-
ing your marketing is laid out in a practical fashion that anyone can grasp and do in a way that will
practically guarantee your conversion success and blow away your competition. I’m a big fan of Website
Optimizer, but this is the first thing I’ve read that really makes it seem practical and simple.
—John Jantsch, Author of Duct Tape Marketing
This is the 360-degree “firehose” of testing and optimization wisdom. Armed with the knowledge and
wisdom in this book, you could run an A/B split between you and your competition and come out the
winner. The result-oriented material in here is worth weeks of advanced marketing training.
—Sam Decker, Chief Marketing Officer, BazaarVoice
Testing has always been a bugbear, because it brings up so much data. And data is cumbersome. So we
avoid intense testing. This book points out not just how to go about testing, but more importantly, what
to test. And what’s supercool is that you can be up and running in under 15 minutes! That, as you can
imagine, is anything but cumbersome—and yes, it actually helps you get clear results instead of another
mountain of data.
—Sean D’Souza, www.psychotactics.com
A: Bryan Eisenberg delivers the What, Why, and How of closing significantly more online sales. B: This is
where Eisenberg’s “Call to Action” philosophy gets tactical and you get better results from your website.
C: Increase online sales without spending a dime or too much time. This book reveals what it takes and
how to use it. D: “Keep on Testin,’” Eddie Kendricks and Robert Crumb. E: All of the above.
—Jim Sterne, Targeting.com
I have been living and breathing A/B and multivariate testing for the past few years, as illustrated in this
book, and I have found that 25 years of experience in this field and best practices still fails in compari-
son to the feedback we get from our audience through testing. So many “best practices” mistakes can be
prevented by simply putting the subject to a test and letting your audience tell you how they best interact
with your product.
—Dino Bernardi, Vice President of Customer Experience, Experian Interactive
You’ve avoided serious testing on your website because it’s expensive and hard to understand, right? Well,
Google Website Optimizer has made testing free, and this book just took away your other excuse. Put this
book down now, or you just might double your revenue and give your customers the website they’ve always
wanted. Bryan and John tell you why and how to Always Be Testing. If you don’t do it, your competitors
will.
—Mike Moran, Coauthor of Search Engine Marketing, Inc., and Author of Do It Wrong Quickly: How
the Web Changes the Old Marketing Rules
Behind every great sales team is a marketer who understands the power of testing. Let’s face it, sales teams
would get no leads without a marketing team that expertly knew how to convert a visitor into a “close.”
Always Be Testing by Bryan Eisenberg teaches marketers just that … how effective testing leads to a
higher close rate at the very first moment of customer engagement.
—Jim Kukral, Internet Marketing Consultant, www.jimkukral.com
9900663377ffffiirrss..iinndddd iiii 66//2266//0088 1122::1122::1155 AAMM
Everyone talks about the importance of testing, yet few people really do it. Always Be Testing gives you
everything you need to become a testing expert. Tools, methods, theory, and practical examples, plus the
most important aspect: what to test and why. Keep this book on your desk and use it every day.
—John Marshall, Chief Technology Officer, Market Motive
Bryan and John have been optimizing websites for over a decade, helping their clients to earn millions
of dollars thanks to testing and Persuasion Architecture. They now share their mastery of testing in this
practical guide with clear explanations and exercises to ensure an efficient knowledge transfer. Whether
you’re new to online testing or a confirmed professional, this is the book you must read on the subject.
—René Dechamps Otamendi, OX2 Founder
I’m not sure the title does it justice—while a chunk is obviously about Website Optimizer, the book really
is so much more since it goes into developing a testing framework, explaining testing methodologies, and
suggested testing areas. I guess that’s why it’s called the complete guide!
—Maheesh Jain, Cofounder and Vice President of Business Development, CafePress.com
Bryan Eisenberg and John Quarto-vonTivadar take you through not just how to use Google’s free conver-
sion testing tool, but continually provide both philosophical reasons and practical tips about why busi-
nesses need to test. It’s like having a golf pro take you around a famous course, not just showing you how
to play, but also taking you deep into the mental process needed to win the game as well.
—Danny Sullivan, Editor-In-Chief, SearchEngineLand.com
This book is Eisenberg at his best. Train wrecks in the marketing world can be avoided by performing in-
depth analysis of potential interactions with your audience and being open to change. You can avoid the
epitome of marketing mistakes, the perfect storm of brand self-love.
—Kevin M. Ryan, Vice President and Global Content Director, Search Engine Strategies and Search
Engine Watch
B2B, B2C, or Agency … it doesn’t matter how (or what) you sell, you should always be testing. Bryan
Eisenberg (along with John Quarto-vonTivadar and Lisa T. Davis) have done it again! Always Be Test-
ing is a great read—even if you’re not using Google Website Optimizer (and, why aren’t you? … it’s free).
The insights, strategies, and tactics in Always Be Testing constantly give you the “why didn’t I think of
that” feeling. Always Be Testing should come with a cold compress for the many times you’ll be whacking
yourself on the forehead. After reading Always Be Testing, you’ll begin wondering why you did any mar-
keting campaigns any other way.
—Mitch Joel, President, Twist Image and Blogger/Podcaster, Six Pixels of Separation
For years companies have tried to develop the best website, even shamelessly copying from the so-called
“best of breed.” The reality is that although technology can be copied, although business processes can be
inspired from best practices, humans can’t be that easily copied. If one of the distinctive elements of any
company is its workforce, the decisive factor is understanding the client. And the only way to know if our
website works for our clients is to test, measure, and optimize our Persuasion Architecture®. A continuous
improvement process isn’t complete until we reach 100% success. . .in the meantime, read this book, learn
a whole lot about testing, see what others are doing, and get plenty of ideas to get going!
—Stephane Hamel, eBusiness Strategist and Web Analytics Consultant, Immeria.net
9900663377ffffiirrss..iinndddd iiiiii 66//2266//0088 1122::1122::1155 AAMM
Always Be Testing needs to be every web managers’ and marketers’ mantra. Testing doesn’t have to be
expensive and complicated to be effective. Always Be Testing is more than a how-to guide. Bryan Eisen-
berg and John Quarto-vonTivadar provide a valuable, easy-to-use framework to decide what to test, how
to test, and how to implement the findings. This will be one of the most dog-eared books in your library.
—Seth Romanow, Vice President and Chief Marketing Officer, Rainmaker Systems
During slow economic times, retailers of all shapes and sizes are trying to find affordable ways to bolster
conversion, leapfrog the competition, and enhance the customer experience. The way to achieve this is
with proper A/B and multivariate testing, and Bryan and John do a spectacular job of mapping out test-
ing best practices that are easy to follow and implement. Bryan has been a great speaker and resource to
the Shop.org community for many years, and this is just one more example of Bryan sharing his wealth of
knowledge and experience. Always Be Testing—buy it, live it, love it.
—Larry Joseloff, Vice President of Content, Shop.org
Any marketer who understands the Web will keep a copy of this book on the desk at all times.
—Pinny Gniwisch, Founder EVP Marketing, www.ice.com
For years I have been telling anyone who will listen that “If you’re not doing some type of testing, you’re
not really doing web analytics!” Well, apparently my friend Bryan Eisenberg has been listening! Always
Be Testing is by far the most definitive work on the subject of website testing and optimization ever written.
More importantly, Bryan, John, and Lisa manage to make this critical-but-occasionally-tedious subject
palatable, using clear language and tons of helpful examples. So now instead of jumping up and down
and yelling at people to “start testing,” I can simply pass out copies of this great book. Thanks Bryan!
—Eric Peterson, Chief Executive Officer and Principal Consultant, Web Analytics Demystified, Inc.
Forget best practices says Bryan Eisenberg, every company has a different optimal strategy. To get your
suits and geeks on the same page, you need data, not intuition. Eisenberg’s new book shows how to use
Google Website Optimizer to get that data by refining website alternatives to optimize conversion rates.
With his trademark style and humor, Eisenberg makes the complex math, like adequate sample size and
statistical power sound easy. The most valuable part of the book is the second half, where he shows 30 fac-
tors to test (color, images, headlines, calls to action. . .) with over 250 alternatives to experiment with to
maximize conversions.
—Andrew B. King, Author of Speed Up Your Site and Website Optimization
A lot of “buzz” gets created around the latest powerful and sophisticated technologies/tools. Bryan
Eisenberg and John Quarto-vonTivadar manage to demystify the cure-all perceptions of these tools by
focusing on the often glossed-over, but truly more difficult, time-consuming, basic questions of how to
form hypotheses for testing, how to come up with creative to test, and how to prioritize needle-moving
activities.
—Bernardo de Albergaria, Vice President and General Manager, eCommerce, Citrix Online
This is the real deal. We’ve already tested Bryan’s and John’s ideas to optimize conversion, and they are
winning ideas. This isn’t pie-in-the-sky stuff. It’s in-the-trenches, applicable, practical information that
helps you confirm what you do know, discover what you don’t know, and make more money online.
—Steve Snyder, Chief Operating Officer, CBS Interactive
9900663377ffffiirrss..iinndddd iivv 66//2266//0088 1122::1122::1155 AAMM
Always Be Testing
9900663377ffffiirrss..iinndddd vv 66//2266//0088 1122::1122::1155 AAMM
9900663377ffffiirrss..iinndddd vvii 66//2266//0088 1122::1122::1166 AAMM
Always Be Testing
THE COMPLETE GUIDE TO GOOGLE™ WEBSITE OPTIMIZER
BRYAN EISENBERG
JOHN QUARTOVONTIVADAR
LISA T. DAVIS
Wiley Publishing, Inc.
9900663377ffffiirrss..iinndddd vviiii 66//2266//0088 1122::1122::1166 AAMM
Acquisitions Editor: Willem Knibbe
Development Editor: Toni Ackley
Production Editor: Dassi Zeidel
Copy Editor: Kim Wimpsett
Production Manager: Tim Tate
Vice President and Executive Group Publisher: Richard Swadley
Vice President and Executive Publisher: Joseph B. Wikert
Vice President and Publisher: Neil Edde
Book Designer: Caryl Gorska
Compositor: Kate Kaminski, Happenstance Type-O-Rama
Proofreader: Kathy Pope, Word One
Indexer: Jack Lewis
Cover Designers: Rob Kirby, Ryan Sneed
Project Coordinator, Cover: Margaret Rowlands
Copyright © 2008 by Wiley Publishing, Inc., Indianapolis, Indiana
Published simultaneously in Canada
ISBN: 978-0-470-29063-7
No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic,
mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United
States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the
appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, [phone], fax (978)
646-8600. Requests to the Publisher for permission should be addressed to the Legal Department, Wiley Publishing, Inc., 10475
Crosspoint Blvd., Indianapolis, IN 46256, [phone], fax [phone], or online at http://www.wiley.com/go/permissions.
Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to
the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation
warranties of fitness for a particular purpose. No warranty may be created or extended by sales or promotional materials. The
advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the
publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the
services of a competent professional person should be sought. Neither the publisher nor the author shall be liable for damages
arising herefrom. The fact that an organization or Website is referred to in this work as a citation and/or a potential source of
further information does not mean that the author or the publisher endorses the information the organization or Website may
provide or recommendations it may make. Further, readers should be aware that Internet Websites listed in this work may have
changed or disappeared between when this work was written and when it is read.
For general information on our other products and services or to obtain technical support, please contact our Customer Care
Department within the U.S. at [phone], outside the U.S. at [phone] or fax [phone].
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in elec-
tronic books.
Library of Congress Cataloging-in-Publication Data
Eisenberg, Bryan.
Always be testing : the complete guide to Google website optimizer / Bryan Eisenberg, John Quarto-von Tividar.—1st ed.
p. cm.
ISBN 978-0-470-29063-7 (pbk.)
1. Internet marketing—Testing. 2. Electronic commerce. 3. Web site development. I. Quarto-von Tividar, John, 1964- II. Title.
HF5415.1265.E376 2008
658.8’72—dc22
2008022103
TRADEMARKS: Wiley, the Wiley logo, and the Sybex logo are trademarks or registered trademarks of John Wiley & Sons, Inc.
and/or its affiliates, in the United States and other countries, and may not be used without written permission. Google is a trade-
mark of Google, Inc. All other trademarks are the property of their respective owners. Wiley Publishing, Inc., is not associated
with any product or vendor mentioned in this book.
10 9 8 7 6 5 4 3 2 1
9900663377ffffiirrss..iinndddd vviiiiii 66//2266//0088 1122::1122::1166 AAMM
Dear Reader,
Thank you for choosing Always Be Testing: The Complete Guide to Google Website
Optimizer. This book is part of a family of premium-quality Sybex books, all written
by outstanding authors who combine practical experience with a gift for teaching.
Sybex was founded in 1976. More than 30 years later, we’re still committed to pro-
ducing consistently exceptional books. With each of our books, we’re working hard
to set a new standard for the industry. From the authors we work with to the paper
we print on, our goal is to bring you the best books available.
I hope you see all that reflected in these pages. I’d be very interested to hear your
comments and get your feedback on how we’re doing. Feel free to let me know what
you think about this or any other Sybex book by sending me an email at nedde@wiley.
com; if you think you’ve found a technical error in this book, please visit http://sybex.
custhelp.com. Customer feedback is critical to our efforts at Sybex.
Best regards,
Neil Edde
Vice President and Publisher
Sybex, an Imprint of Wiley
9900663377ffffiirrss..iinndddd iixx 66//2266//0088 1122::1122::1177 AAMM