Review of Testing March 2008

Good as his word, I got a copy of James Brausch’s Testing Newsletter within minutes of posting a trackback! (I do want to know how to do that.) 

This edition is 15 pages long, two-columns in a very readable Courier font. It uses James’s very direct, no-nonsense style to explain what he began with, and how he tested it.  The newsletter gives 1) an insight into how to go about testing, as in what to test, and 2) clear explanations of the data gathered.

The newsletter discusses in specific detail how “Diego” tweaks his sales page for his intern program– a free product that makes money for him. Once he collects the data about how much his business gains from each intern, he sets the value into MuVar and then checks the variables. Running the script process the variables is itself an intern task, making MuVar that much more valuable.

Once the visitor value is set, each part of the sales letter is tested, from the number of blank spaces at the top and the bottom of the page, to the headline and font color, each paragraph, and different pictures. In each case, he tested “nothingness” against the factor, and in some cases, “nothingness” won.   The letter explains each variable, and even comments that there is no way to tell why something works, or in the case of one sentence, why it stops working. 

Some factors are not as easy to control, such as testimonials from from interns.  But each factor that is tested shows which variation is a winner, or if two are nearly equal. It’s a fascinating look at how to do testing, but without software to manage the testing, it would be a nightmare to track so many variables.

As an intern for Braush, I was most surprised by the number of people who don’t work through the program. He says that over half of the people quit before finishing the first task. Amazing.  The work is not hard, and I’ve learned more from each task what it takes to get a business up and running.  I’m looking forward to the time when I have interns and my own subscription to Testing!