split test like a scientist

7 Steps To Run Split Tests Like A Scientist

andrew browne Split Testing Best Practices Leave a Comment

Sharing is caring!

AB testing is a scientific test that can help you optimize your Amazon private label listings.

 

Because this is a scientific test, you should use the scientific method to help you brainstorm test ideas and guide you.

 

After you brainstorm ideas on ways to optimize your listings, you can use Splitly, our AB split testing tool, to help you AB test your listings.

How To Optimize Your Private Label Listing

 

There are many reasons to split test your private label listings. However, the most obvious reasons are to optimize your private label listings to increase sessions, conversion rate, profit and sales.

 

AB testing is a perfect way to optimize your listings because it lets you compare two different variables in your listings, like two different lead pictures, while controlling for other variables, like days of the week.

 

Splitly is our split testing tool that can help you conduct effective split tests, which will in turn help optimize your listings. Despite Splitly being able to help you conduct split tests, you still need to come up with the variables to test and know the basics of how to conduct effective tests. 

 

One challenge that many sellers wrangle with is fighting their natural inclination and using data to drive their decisions. And this is exactly where split testing is so important. For example, if you were selling bedding, which of the two images below do you think would generate the most interest, and therefore clicks and sales?

pillow_packaging bed

Well, this is a trick question. There is no right answer, as the “better” image may change with the seasons, price, main keywords, or more.

 

That’s where the scientific method comes into play.

 

How To Brainstorm Split Test Ideas

 

You may be asking yourself “why do I need to use the scientific method to test my products? Can’t I just throw a bunch of changes into Splitly and see what happens?”

 

Well, that’s not really the case.  

 

The reason why you should use the scientific method is because, well,  AB testing is science – testing two variables while controlling for other variables, and then making decisions based on the data.

 

If you want accurate data that helps you optimize your listings, you have to think of this as  science.

 

Therefore, the scientific method will help come up with ideas on what to test and it will also guide you on how to test it.

 

Use 7 Steps of The Scientific Method

 

Below are the universal steps of the scientific method:

 

  1. Formulate a question
  2. Conduct research and observations and collect data
  3. Form a hypothesis
  4. Test this hypothesis
  5. Analyze the data from this test
  6. Interpret the data and draw conclusions
  7. Retest

 

You should use the above steps as a guide to help you create more powerful AB split tests for your Amazon private label listings. Below is a more indepth explanation of the above steps.

 

  1. Define a question:

 

Defining a question is a crucial part of any AB Test. It will shape your entire test, so make sure to evaluate this step thoroughly.

 

A good tip is that you should start with your pain points. What about your Amazon Private Label business bothers you? What do you want to improve?

 

Here are a few examples of pain points turned into a question that can be tested:

 

  • Why are my sales not increasing?
  • Why is my BSR not increasing?
  • Why is my competition staying firmly on the first page while I can not increase rank?
  • Why is my competition selling more than me?
  • How can I capture some of the market from my competition?
  • How can I increase my profits?
  • How can I increase daily sales?
  • How can I increase daily conversions?
  • How can I increase daily sessions?
  • Are my pictures good enough?
  • Should my title be longer?
  • Should I put more keywords in my title?
  • What bullet points should I be using?
  • What features of my product should I be highlighting?
  • How should I describe my product?
  • What is the optimal price for my product to increase sales?
  • What is the optimal price for my product to increase profit?

 

If you are a private label seller, you most likely have asked yourself one or more of these questions several times.

 

However, there are many other questions about your private label listings that you need to be asking. These are just a few examples.

 

Take the time to look at your listings and ask the tough questions. Also, look at your competition and see what they are doing.

 

Write down your pain points in the form of questions and then move on to the next step.

 

  1. Conduct research and observations and collect data:

 

After you formulate your questions, start conducting research and collect data on these questions.

 

For example, let’s say your question was “why is my competition selling more than me?”

 

You may want to start by doing some research on your niche and what those customers in that niche like. You may also do research on that general industry to see where it is projected to go. You could also ask friends and family to give you ideas.

 

You may also conduct observations on your competition. For example, you may want to look at your competitors’ pictures. Are their pictures better than yours? Are they using models in their pictures? Are they using infographics?

 

You can also look at their titles and compare them to your titles. What keywords are they using? Are they using long form content content or short form content?

 

Next, you could look at their bullet points. What features are they highlighting? What keywords are they using? Are they offering a guarantee?

 

Finally, you may look at their description. How are they describing their product? What features are they highlighting? Who are they addressing in their writing?

 

  1. Form a hypothesis:

 

Now that you have your question(s) and the baseline research you need, formulate a hypothesis based on your research.

 

For example, let’s say your question is “why is my competition still selling more than me?” and your research seems to show that all your competition has better photos than you.

 

Your competition uses professional photographs in all their listing pictures. They are also using infographics and models to show how their product is used. This is the main feature image for the LED Glowing dog collar 

dog_collars_-_LED

You, on the other hand, only use photos taken from your iPhone. You don’t have any models or infographics. Maybe your main image looks something like this: 

 

dog_collar_-_bad

Based on your research, this is the main difference between your listings and your competitions listings.

 

Therefore, your hypothesis might be one or more of the four below:

 

  • If I change my product pictures to professional pictures, my sales will increase
  • If I change my lead picture to that of a professional picture, my sales will increase
  • If I start using infographics in some pictures, my sales will increase
  • If I start using models to demonstrate how to use my product, my sales will increase

 

  1. Test this hypothesis:

 

Now that you have your hypothesis, you are ready to test. This is where Splitly comes in.

 

Let’s say you wanted to change your lead picture to test your hypothesis that “if i change my lead picture to that of a professional picture, my sales will increase”.

 

You would do this by getting a professional picture of your product and then letting Splitly test that picture as your lead picture against your old lead picture.

 

Splitly will make live each variable to your listing at least one day a week to control for the day of the week. This way we know that fluctuations in data are not due to the day of the week.

 

Once this test has reached a statistical significance of 90% or above, we consider the test complete and the data is ready to be analyzed.

 

  1. Analyze the data from this test:

 

Once your test hits a 90% statistical significance, we will automatically stop it. We now consider it  done and the data is ready for you to analyze.

 

Splitly records all changes in average daily sessions, average daily sales, conversion rates, and average daily profits, and puts this data into a neat, easy to read display.

 

You will be able to easily see how much your average daily sessions, average daily sales, average daily profits and conversion rate has increased or decreased and by what percent.

 

You will also see the likelihood that this change in your data was due to the variable you were testing. This is displayed as a percentage of likelihood that your variable outperformed or underperformed the original.

 

If it is a 90% chance or above that the variable outperformed the original, you can be confident that the change in data was due to the variable.

 

If it is a 90% chance or above that the variable underperformed the original, you can be confident that the change in data was due to the variable underperforming the original.

 

  1. Interpret the data and draw conclusions:

 

Splitly makes it very easy to interpret your test data.

 

As stated above, Splitly presents you with the differences in you average daily sales, sessions, profits and conversion rate in an easy to read display. Splitly also provides you with the percentage of increase or decrease, and the percentage that your variable likely caused that increase or decrease. 

 

However, the easiest way to visualize the impact of your split tests can be found in the dashboard. Here are the results of a recent user’s tests, which focused on improving conversion rate (the bottom left quadrant):

 

price test for increased conv rate

Now it’s your turn to draw a conclusion.

 

Let’s say after testing your original lead picture against one taken by a professional, your average daily sessions increased by 10%, your conversion rate went up by 5%, and your average daily sales went up by 5 a day. Also, Splitly notes that there is a 95% chance that the variable you were testing  outperformed the original.

 

This data is pretty conclusive that your new professional photo has caused these increases in your sessions, sales, and conversion rate, and should be a permanent change in your listing.

 

Now let’s say your average daily sessions dropped by 10%, your conversion rate went down by 5%, and your average daily sales went down by 5 a day, and there was a 95% chance that your variable underperformed the original.

 

Then it’s conclusive that this change in your lead picture caused a decrease in sales, sessions and conversions, and should not replace your original lead picture.

 

  1. Retest:

 

You may think your job is over after the you completed your test and drawn your conclusion.

 

However, you can never do enough optimizing of your listings. You should continue to retest this variable until you are confident it is fully optimized.

 

For example, now that you are done with split testing your lead picture, and your conclusion was that the professional photo increased sessions, conversions, and sales, your next step would be to retest this variable.

 

You may want to

 

  • test a new professional lead picture showing a different angle of your product
  • test pictures using models versus your original pictures
  • test pictures using infographics versus your original pictures
  • change the rest of your pictures to professional pictures and test these against your original pictures and several other professional pictures

 

Whatever you decide to retest, you should be focused on continually optimizing your listings, and never getting complacent.

Conclusion:

 

AB testing is a scientific test that can help you optimize your listings. Because this is a scientific test, you should use the scientific method to help you brainstorm test ideas and guide your tests.

 

The steps to the scientific method are

  1. Formulate a question
  2. Conduct research and observations and collect data
  3. Form a hypothesis
  4. Test this hypothesis
  5. Analyze the data from this test
  6. Interpret the data and draw conclusions
  7. Retest  

Use these steps to guide your AB testing on your Amazon private label listings and use Splitly to help you conduct these tests. Get started on your own split tests today!

Sharing is caring!

andrew browne

andrew browne

Code Wizard at Splitly
Software developer and Amazon seller from Ireland. Constantly searching for travel adventures, greasy burgers, and all things tech.
andrew browne

Leave a Reply

Your email address will not be published. Required fields are marked *