Three ways to end an opinion war

Jared Spool wrote an interesting article last week about ending the opinion war and how his company handles the task. The term opinion war is pretty self-explanatory; but just to be clear, it is when two parties having a vested interest in a project hold differing opinions about how something should be implemented.

Opinion wars can be difficult to navigate.  It can be nearly impossible to sway someone’s opinion on a matter. Many times because each party thinks it has comes to an opinion in a reasoned and rational way, which makes it difficult to see the reason and rationale behind the opposing opinion. This is what makes politics so entertaining/frustrating, right?

As a small business owner, you may argue with yourself more than anyone else about how something should be accomplished, but even still we have a few suggestions on how to come to a sound resolution.  

Spool offers two suggestions in his article.  We added the third.

1. Use an arbitrator. Each project has a not-so-senior project manager who has the full power to make the final decision. This person is tasked with evaluating the opinions, asking for advice from senior practitioners if needed, then banging the gavel. An interesting approach.
2. Let user research guide the way. This is the approach Is It Easy? specializes in. First and foremost, ongoing user research helps orient stakeholders’ opinion in the right way, staving off potential opinion wars. However, when a specific difference does arise, we’ll review what data we may already have about our users (personas, scenarios of use, interviews, surveys) for insights. Then, if needed, we’ll collect new research using prototypes.   
3. Use A/B testing to evaluate each opinion. Rather than using prototypes in a round of user testing, we’ll create two different versions of the same page that will go out live to your users. Each version will be exactly the same, the only difference being the element in question. Half of your users will see one version of the page (version A) and the other half will see the other version (version B). When you set a conversion goal, you can assess which design yields better results.

When should I use A/B testing?  We feel it works best in the following situations.

1. The elements in question are specific and discrete. A/B testing works for questions about particular images, text, colors, labels, and placement. We test one variable at a time to avoid confounding the results. If there are larger questions about whether or not your message is getting across to users or if your subscription or shopping cart flow is confusing, those are questions best left to user testing. You may learn from user testing that your shopping cart flow is confusing because of the placement, wording, or color of your checkout button, in which case we'll use A/B testing to discover the optimal design.
2. Each opinion can be tied to a conversion scenario. For each page version, we’ll want to set up a conversion page or some other way of tracking a “win.” A conversion is the desired behavior from the user, such as subscribing, placing an order, or submitting contact information. Therefore, we can track a conversion when a user lands on the “thank you” for subscribing, ordering, or contacting page. The design that leads to more conversions wins the opinion war.  
3. Results from qualitative user studies were inconclusive. If the element you’re arguing over is specific, sometimes you may not get a clear picture of what would serve the users better during user testing. User testing uses a small sample size, so questions such as which banner placement yields more clicks may not be conclusive. I love user testing, but it can be somewhat artificial because users adopt goals and motivations for the test they may not have in real situations. To get answers to very specific questions, we recommend A/B testing to see what converts a higher rate of customers over a large sample size.  

What are your thoughts? Have you had opinion wars over design decisions in your business? Tell us how you came to a resolution.