Take the guesswork out of book cover design with user testing (a Copyhackers case study)

You’ve probably heard this before:

Design is subjective.

Something that looks beautiful to one person might look like trash to the next.

Now, I’m the only designer at Copyhackers.

So coming up with a design that the majority of people find aesthetically pleasing can be difficult.

My first-draft designs are often based on what I think looks good, not what the general population thinks.

The problem with this approach is that I’m not the one buying our products. So what does it matter what I think?

So instead of assuming (you know the old saying…) what sort of design stands out to our consumers, we figured the best method is to, well, ask them.

This is where preference tests come in.

Preference testing for book cover design

Preference tests are the simplest way to determine what cover design your potential customers prefer.

At Copyhackers, we use UsabilityHub because it’s easy to use and offers a lot of value for the price.

However, there are a ton of testing tools out there, so don’t feel like our favorite option is the only option.

You also don’t necessarily need to use a tool like UsabilityHub. You can use the free option and just ask folks in your network.

However, I recommend using a tool.

That way, you’ll have unbiased opinions from strangers instead of people who may feel pressure to give positive feedback because they know you.

Re-releasing Copyhackers books from 2011: A cover design case study

In the summer of 2022, we decided to re-release a series of books Jo wrote back in 2011 from the Copyhackers vault.

(Because the books are mega helpful, why keep that knowledge to ourselves?)

And so, The Copyhackers Classics Series stepped out of the vault and into the light (for the first time in almost a decade).

The first book we re-released was Where Stellar Messages Come From, which is the basis of this case study.

When revisiting this book, the content was still incredibly valuable, but the cover design left something to be desired:

No shade to the original designer – visually, this cover looks fine.

But it doesn’t say much about the book’s content.

It has more of a “free workbook that you might skim through” vibe rather than a “this book is full of a ton of valuable information, and if you were to only buy one book about copywriting, it should be this book” vibe. Ja feel?

It also feels a little juvenile.

Now don’t get me wrong, design doesn’t always have to be super serious. But…

In this case, we’re trying to seriously help copywriters and would-be copywriters further their careers.

A juvenile design didn’t really communicate the value the reader would actually get out of the book.

So when approaching the redesign, we (we meaning I because, as I said, I am the only designer at Copyhackers) knew we were going to test our cover design against our top-performing competitor’s cover designs.

But we needed a new book cover design to test first

I started the cover design like I start every design project: 

With a blank page and a Google tab open for research. 

I researched our competitors’ cover designs to get an idea of what covers are the most clickable. 

At this point, the actual reviews of their books didn’t matter too much because it’s the first impression of the cover that makes people want to click on the Amazon listing to learn more about it. 

I also did some general research to find that consumers tend to gravitate toward book covers that are big, bold, and readable. I.e., Primary colors and large fonts tend to perform better than pastels and small fonts.

So I sketched out some ideas based on my research and my own opinions of what looked good, and then I refined those sketches into fully-fleshed-out designs:

Now, did I think this cover was perfect? No! Of course not!

I never think any design I do is perfect.

Design is like copywriting in that sense. There’s always room for optimization.

But what I did think about this design is that it was big, bold, and readable.

Also, this was one of three cover designs I did. And the CH Publishing team and I chose this one because of the points mentioned above.

Ok, so now we had the cover design we wanted to test,

We needed a couple of competitor’s cover designs to test against

Now you might be wondering, “Nicole, didn’t you just say that you created 2 other designs before? Why don’t you just test this cover against those?”

Sure, we could have done that.

But that wouldn’t tell us if our cover stood out against the top performers in our Amazon category.

If we did that, all the test would reveal is that customers like one version of our book cover, not that they prefer it over our competitors.

So the CH Publishing team searched through Amazon for two of the most popular cover designs in our category.

Similar to my initial research process, we focused on what was most popular:

• Was it a bestseller?
• Did it keep popping up in the search?
• Did it pop up even when we changed the search terms?

…We didn’t focus as much on books with the best reviews.

Eventually, we decided to test against these two covers:

I have my own opinions about these cover designs, but as I said before:

My opinion doesn’t matter much.

Preference tests are like a democratic election, it’s illegal to vote for yourself.

I’m obviously joking (it’s not illegal to vote for yourself).

But when it comes to design, you can’t let your personal opinions and biases get in the way.

You need to know – objectively – which book cover design performs better.

Book cover design preference test 1: Epic fail

Just like wanting high-converting copy, we also want high-converting design.

Ok, I’ll cut to the chase. Here’s how our first test went:

Uhhh, so our design lost. Big time.

But that’s okay because:

A) I’m used to it from back in the day when I was in Track & Field. And,

B) We now had a jumping-off point for the next iteration.

Now you’re probably like, “Nicole, how do you know what to iterate on? All you know is that people prefer other designs over yours.”

And that’s where you’re wrong.

The value of preference tests doesn’t come from the final score. The value is in the comments.

Basically, participants choose their preferred design and then leave a comment as to why they chose said design.

And we received some insightful comments from the participants.

So the first thing I did was look at the comments for Persuasive Copywriting to try and get some insight as to why that was the clear winner.

The comments helped us realize the direction we needed to take this design if we wanted it to stand out against other books on Amazon.

While our cover was certainly bold, you can see in the comments that it wasn’t doing what it needed to do:

Persuade people to click on it.

The main takeaway from the comments was:

• Graphics stand out and help better communicate what the book is about
• Subtitles help potential customers know what your book is about
• Color contrast helps your book stand out
• People really like yellow

But, there were participants that actually preferred our cover over the other two, so it was also important that we took their comments into consideration:

I didn’t want to change what was already working in our current design.

We also didn’t want to copy the other two designs. We just wanted to know how we can optimize ours.

So from these comments, we realized our power was in our simplicity.

The challenge was how do you make a simple design that still stands out.

The most obvious solution was to add a subtitle. So we did that.

The second most obvious solution was to add a graphic. So we did that too. And here’s the final design I came up with:

What I like about this design is that it has the same essence as the previous design we tested but communicates its ideas sooooooo much better.

I look at this and immediately know it’s a copywriting book.

Whereas before, it honestly could’ve been any genre.

However, note the differences – this design uses the same font and background, but it immediately feels much better than the previous design.

A lot of times, design comes down to a gut feeling. But remember what I said, my opinion doesn’t matter.

So let’s see what our participants thought:

The participants loved it too!

This was a huge win for us because we knew the yellow Persuasive Copywriting cover would be tough to beat (like I said, people like yellow).

But again, the comments were the most valuable.

Even though we already knew our cover won, we still wanted to make sure the right idea was being communicated:

These comments solidified our confidence in this book cover, so we decided to run with it.

And good thing we did because within 24 hours, we were a #1 new release on Amazon and hit lists in Business Communications, Marketing, and Professional & Technical Marketing.

Ultimately our preference testing process worked.

(Shout out to UsabilityHub for making it easy to get unbiased feedback on designs).

It was so easy, in fact, that we decided to do this process all over again with Jo’s new book:

Your First $1000: 12 actionable techniques to make great money in the next 7 days as a part-time freelance copywriter

Preference tests for a new book cover design

This testing process was very similar to our last, but since this was a brand new book, we didn’t have an original cover to improve upon, so we had to work from scratch.

Now, this isn’t necessarily more difficult to do, but when you have nothing to improve, everything you design is an improvement.

Basically, I sketched more ideas at the beginning of the process to narrow down the direction we wanted this design to go in.

After the more involved sketching process, the testing process was more or less the same.

The only difference is that to ensure the design communicated what we wanted it to communicate, we tested two totally different designs before we even started the revision process.

Here’s a breakdown of how it went:

• I drew about 8 sketches to choose the design direction
• Came up with 3 cover designs
• Chose our top two favorite designs
• Tested our favorite design against top competitors in our category (which came in last, with 28% of participants saying it was their preferred design)
• Tested our second favorite design (This one also came in last, but with 30% of participants saying it was their preferred design)

This was where we ended up having to trust our gut instinct.

Even though our second favorite design performed better in testing, we decided to scrap it and revise the first design instead.

Why? Because the first design felt like it matched the book’s energy far more than the second design.

And we felt really strongly about it (with design, sometimes the feeling you get is more important than the data you receive).

So if we could address some of the comments our test participants had, we could have a winner on our hands.

And that’s exactly what we did.

Then, when we tested the revised version of the first design (our favorite one), participants preferred our revised cover over the competition (and I gotta say, I prefer it too!)

And here it is…

Your First $1000 book cover reveal

Your First $1000 is now available in paperback and on Kindle

(BONUS free video series)