How a Negative Split Test Result Can Grow Your Business

andrew browne Split Testing Best Practices 1 Comment

Sharing is caring!

Splitly not only helps you pick out profitable changes in your listing that can help drive sales, but can also help prevent you from making changes to your listing that would decrease your sales.

 

By helping you identify changes to your listing that would decrease your sales, Splitly can help prevent you from making changes that you would consider mistakes. By helping prevent you from making these changes, Splitly can help save you thousands of dollars in lost sales and profit.

 

Therefore, you should always test any changes to your listing before you make them.

 

You Don’t Always Need A “Win”

 

Many Splitly users believe that in order to benefit from an AB testing tool like Splitly, they need to come up with the next killer variable that will drive their sales from a few a day to hundreds a day.

 

Of course, why else would you use an AB testing tool if you could not find variables that would help increase sessions, sales, conversions and daily profit?

 

However, AB testing tools like Splitly also help prevent you from making changes to your listing that would be huge mistakes.

 

Splitly Can Help Prevent You From Making Big Mistakes

 

A number of private label sellers will make a simple change to their listings because of a hunch they had. This change may have a positive effect on their sales metrics, or it could have a disastrously negative effect causing sales, sessions, conversions and profit to go down.

 

What Splitly allows you to do is to test this change before making it permanent. This allows you to get insight on whether this change will drive sales up or down before you make the change.

 

You will see, normally within 2 weeks, if this change is something you should keep. If Splitly’s data shows that this change actually caused a decrease in any of your stats, you now have concrete evidence that indicates you should not to make this change.

 

This is especially true when making changes to your listings that are so small you may not even realize they will have an impact.

 

For example, you may think it’s perfectly fine to change your price from 11.97 to 12.97. However, this could have a slight affect on your conversion rate, which you might not even notice until months later. By that time you may have made several other changes to your listings and won’t know which is causing your sales to decrease.

 

Instead, by testing this variable and seeing whether it does better or worse than the original, you can ensure that all your changes are profitable ones and avoid unprofitable mistakes.

 

By helping prevent you from making these unprofitable changes, Splitly can help save  thousands of dollars in lost sales and profit.

 

Splitly Identifies Which Change is Profitable or Detrimental

 

When you test a change in your listing with Splitly, it tests that change against your original listing. Splitly will then give you data on your average daily sales, sessions, profit and conversion rate of the original and of the variable.

 

Splitly will also tell you what percentage these metrics increased or decreased by, and the likelihood that these changes were caused by your variables.

 

By comparing your original listing against your changed listing (variable) and then presenting the changes in sales metrics, Splitly can help you pick variables that boost your sales, sessions, conversions and profit.

 

However, equally important is that Splitly will prevent you from making changes that hurt your listings by dropping your sales, sessions, conversions and profit.

 

This is because Splitly will show you clearly that the change in your listing underperformed the original by either decreasing sales, sessions, profit or conversion rates.

 

As you can see, this can be used as a preventive tool to help stop you from making changes that will hurt your listings.

 

Below are examples of tests run by real Splitly users that illustrates the power of testing before making changes in order to prevent making mistakes.

Changes to Your Sales Price Can Hurt Your Sales Metrics

 

You should always test price changes before making these changes to your listings because you may be overpricing or underpricing your product. This could lead to a reduction in your sales metrics.

 

You may think that a small change in your price couldn’t possibly hurt your sales metrics. I mean, what’s $2-$3 to most people anyway? That’s not even the costs of a Venti Iced Caramel Macchiato at Starbucks.

 

And so with that thinking, you may increase your sales price by $2-$3 in order to increase profit However, it would be a mistake to make this change without first testing it.

 

One of our users conducted this very test by changing his price from $14.97 to $17.97. This user tested this higher price against his original price, and ran his test for 17 days.

 

He found that his higher price dramatically reduced all of his sales metrics including average daily sessions, average daily sales, average daily profits, and his overall conversion rate.

 

This user went from 23 sales a day with his price at $14.97 to 12 sales a day with his price at $17.97.

 

This was with 100% chance that the increased price underperformed the original price. This means there is 0% probability that this change was caused by random fluctuations, and 100% probability it was caused by the change in price.

 

This was a decrease of 47%.

splitly amazon test results

 

In addition, his conversion rate went from 12% to 7.38%, with a 100% chance that his variable underperformed his original. This was a decrease of 46%.

 

Finally, this user’s profit went from $121.22 a day to $91.54 a day, with a 98.56% chance that the variable underperformed the original. This is a decrease of 23%.

 

results from splitly amazon product testing

 

If this user did not test this price change and made the decision to increase his product by $3 simply because he wanted to make more money per sale, he would have decreased his average sales by 47%, his conversion rate by 46%, and his profit by 23%.

 

Put another way, if he did not test this variable, he would have lost approximately $890 in one month! That’s almost double the most expensive Splitly subscription plan.

 

Even Small Price Changes Can Have a Negative Impact on Your Sales, So Make Sure You Test First:

 

Regarding the above example, you may be thinking that it makes sense that this user would experience such a large drop in his sales metrics. Although $3 may not seem like a lot, for a product that is $14.97, an $3 increase is a markup of over 20%.

 

However, if you were to assume that only big price changes affect your listings, you would be wrong. In fact, small price changes can have a big impact. That is why you must test even a small change to your price to make sure that it doesn’t hurt your sales metrics.

 

For example, one of our users changed his price from $29.97 to $29.99. This is only a markup of 2 cents! Despite how small the mark up was, the impact was huge.

 

His average daily sessions went from 234 to 209, a decrease of 10%. His conversion rate went from 8.11% to 7.05%, a decrease of 12%. Finally, his daily sales average went from 19 to 15, a decrease of 22%.
results from splitly ab test

splitly unedited

 

Therefore, if this user did not test this price difference of only 2 cents because he believed that the impact of such a trivial amount would be small, he would have decreased his sales by 22%.

 

However, because he tested this price change, he was able to see that changing his price was a mistake, and therefore could decide not to make this change.

 

Changes in Your Picture Order Can Have a Negative Impact on Your Sales Metrics:

 

You may not think that switching the order of your pictures would have an impact on your sales. However, the truth is that even switching your pictures orders could cause a decrease in your sales metrics.

 

Therefore, before switching your picture order, you should test different picture orders and make sure they do not hurt your metrics.

 

For example, one of our users switched his lead picture for what was originally his second picture. Therefore, his original second picture was now his lead picture and his original lead picture was now his second picture. He then tested this against the original order.

 

His original lead picture showed six products, as he was selling his product in a bundle of six. His original second picture, on the other hand, showed only one product. When he switched this order, his new lead picture showed one product, while his new second picture showed six.

 

Due to this change, his average daily sessions went down from 12 to 8, with a 99.81% chance of the variable underperforming the original. That’s a decrease of 33%.

 

His average daily sales went from 4 to 2, with a 98.42% chance of the variable underperforming his original. That’s a decrease of 54%.
splitly daily sessions chart

Finally his conversion rate went from 31.33% to 22.37%, with a 91.72% chance of the variable underperformed the original. That’s a decrease of 28%.

splityedited6
By testing this order, this user was able to find that his original order was helping him drive more sales. Therefore, Splitly prevented this user from making a change in his picture order that would have cost him more than double his average daily sales.

 

Changing Your Short Title to a Longer Title Can Reduce Your Sales Metrics:

 

You may have bought into the so called “gurus” telling you that you should keyword stuff your titles and use every possible character allotment. However, unless you test this, changing your title from a short version to a longer version may hurt your sales.

 

For example, one of our users took his original title, which was 117 characters, and tested it against a title that was 198 characters – close to the 200 limit imposed by Amazon.

 

This user’s average daily sessions went down over 39%, his page views dropped 49%, and his average daily sales dropped by 37%.

splitly 7

This again proves that you should always test a change in your listing before you make it permanent. If you test a change before you make it, you can be sure that the change you make is a proper one.

 

Conclusion:

 

Splitly can help you optimize your Amazon private label listings.

 

Splitly will not only help you test variables that will increase average daily sales, average daily sessions, average daily profits and conversion rates, but will also show you variables that will hurt your sales metrics.

 

By showing you variables that will hurt your sales metrics, Splitly helps prevent you from making permanent changes that can decrease your sales and profit.

 

When running a Splity AB test, if the change shows a decrease in your sales metrics, you will know that that change is not a good one and can revert back to the original listing.

Sharing is caring!

andrew browne

andrew browne

Code Wizard at Splitly
Software developer and Amazon seller from Ireland. Constantly searching for travel adventures, greasy burgers, and all things tech.
andrew browne

Comments 1

Leave a Reply

Your email address will not be published. Required fields are marked *