How to Create a pairLab Survey

Create a New Survey

  • click the Create Survey button from the my expairiments page
  • you can also choose New … from the Expairiments drop down in the navigation bar

Field Definitions


Give your expairiment a name that you can remember. This is never shown to anyone and is used only by you.


This is what you are trying to answer with the expairiment

Write your question carefully and clearly

  • If you are familiar with the game, "Would you rather … ?" you know how to write a question. Like in the game, you use the question to create a dilemma between two choices for your survey participants. “What do you prefer more in a job?” “Which color best represents the beach?” “Which feature do you value more in a laptop?” A clear question gets survey participants to focus on the choice in front of them.
  • You also want your question to be simple enough that the survey participant doesn’t have to re-read it every time they vote on an idea pair.  Again, we want them to focus on the pair, not the question.
  • Finally, keep in mind that an expairiment answers only one question. You may need more than one expairiment to help you understand the problem you face.
    • For example, let’s say I’m building an intranet and I find that participants highly a value a tool to locate the offices of their colleagues.
    • I ask myself: What could this feature look like? Maybe I could build it with an offline map, maybe there’s a mobile wayfinding app, maybe I just need to show a generic map in the browser. I know from one expairiment I need this feature, but it takes an additional one to understand the details. So, multiple expairiments are good.

Survey Status

Is the survey open or closed to participants?

Note you need a minimum of three ideas to open a survey, even if it is opened through these radio buttons. You will see a flag on the survey page if you need more ideas.

Be sure to close your survey when you have enough votes; don’t keep a survey open so someone can later on find it and alter your results

Hello and Goodbye Messages

For the Hello message, be descriptive but not too long. Consider asking for help. Asking directly for help motivates many people to take the survey.

For the Goodbye message, a simple thank-you is appropriate, but you can say more.

Show Results to Participants

If you want your survey participants to see a simple display of the results when they complete the survey, choose the Yes radio

Goodbye Button Link

You can include a link to another site at the end of the survey by specifying a URI in this field

Demographic Segments

One is required and you can have up to three.

Each segment consists of a question field and a values field.

  • Keep the question simple
  • Add one segment per line for the values

Why are segments important? For several reasons:

  • They allow you to see the priority of subsets of your survey participants and to compare them; for example: men and women may have different priorities and you may need to know that
  • If you are looking to survey representative samples of certain groups that exist in a ratio, segments help you figure out if you are meeting your target. For example: In an expairiment I did at Princeton, I knew my ratio of Staff to Faculty was seven to one. A segment question helped me know when I needed to survey more of one of these groups because I hadn’t attracted enough of them.
  • Finally, segments can serve as a mini closed-ended survey. In that same study I asked survey participants if they believed the app I was building was needed, which I reported back to my bosses who were interested in that metric.

Participant Goals

This tells pairLab whether it should consider the participants as part of group with either common goals or individual goals. The reason is: people vote differently depending on how connected they feel to you or your organization. 

Research tells us that people who feel they belong to a group will adopt the group’s goals as their own (as opposed to people whose goals are mutually exclusive of each other’s). pairLab takes this into account. Your choice changes how the algorithm calculates results.

  • You can change this at any time
  • If you are not sure, keep the default
  • You may find that rank order isn’t affected, you are likely to see the pair scores among ideas tighten


Oh, you know what this means.

What's Next?

If you opened your survey, you will see new indicators appear:

  • The status indicator changes to open
  • If you don’t have enough ideas (which you won’t when you create a survey), you will see this yellow warning indicator
  • Finally, you see two buttons for accessing the survey
    • Click the Survey Link to get a direct link to the survey, which you can send to participants
    • If you wish to embed the survey in a web page, click Embed Code to get html code for an iframe.
Tagged as
Posted by Christian
on September 17, 2020