Why Does An Expairiment Ask Only One Question?

Why does pairLab ask just one question? Let's look at an example to find out.

If I'm a product manager and I'm developing a new tool for my cool web application, I have three goals I need to hit for success:

  • Figure out how I can improve my customers' lives with my new thing
  • Figure out where best to invest my resources as I build out MVP
  • Figure out which of my competitors are strongest in the market

If I analyze these goals, I see that I have to ask my customers three questions:

  • What are your problems?
  • What features would solve those problems?
  • What products are now doing the best job at solving them?

I could tweak these questions a little but I'd still be aiming for the same outcome: Find my my customers' pains and gains, build something that ends their pains and satisfies their gains, and aim for my competitors' weak spots. 

It ends up that these product questions are fiendishly complex because each is mutually exclusive of the others, and each has lots of potential answers I need to sort through (including answers I didn't think of). If I throw all that complexity at my customers at once, the likelihood that I'll get the clear answer I'm looking for is low.

And at the end of the day, what I really want is something that tells me what to do, like this:

Your users want X twice as much as they want Y, and they want Y 50% more more than they want Z. Users value Q so little, that if you take resources away from X, Y and Z to build Q, you're diminishing your chances for success.

That's what I want to know and that's what pairLab does. It does it by asking only one question. 

Instead of asking my customers lots of different qualitative and quantitative questions, which impose a nasty cognitive load on them (and makes me nuts as I use messy math to get the results), pairLab does the exact opposite. It asks participants to evaluate lots of answers to a single question.

Now I'm making life easier for my customers because their choices are dead simple. My life is easier, too, because evaluating data is no longer an excruciating process. 

There's a lot more power in one question than you think. That's where pairLab shines.

Posted by Christian
on March 2, 2022