Writing a Questionnaire for a Conjoint Analysis Study

The Screener

The first section of a questionnaire generally contains the screening questions. These questions are used to determine if people are eligible for the study (e.g., are users of a particular product). They are used as well for collating quota information (i.e., ensuring an adequate cross-section of people in key sub-groups). For example, surveys generally are only open to people aged 18 or over, so there will be a quota on age. Other quotas might include gender, and geography, as well as whatever other screeners are applicable to the study of interest.

Explain how to do the first choice question

It is usually desirable to get people into the choice questions (also known as choice tasks) as soon as possible. This is because these questions are both taxing and boring. You want people to be answering them when their minds are as fresh as possible.

When the first question is presented, or, immediately prior to this, it is usually useful to:

• Provide relevant background information. For example, if your study is in a technology market and the choice model has attributes containing new technologies, it is usually a good idea to introduce these new technologies and ask questions that test comprehension immediately prior to the choice-based conjoint analysis questions.
• Encourage honesty and attention. For example,  “Your answers to this questionnaire will help determine what you can buy, so the more attention you devote to the survey, the more likely you are to be able to buy what you want”.

(Optional) Ask people to explain their first choice

In the questionnaire, it can be helpful to ask people to explain how they made their first choice. Ideally, this question will appear on the same screen immediately after they have made their first choice. This has two goals. One is to reinforce that the respondent's choices are important. The other is as a way of verifying that your data is good. For example, if you have a study which includes attributes for brand and price, and the person says something like "I chose the Chevrolet because it is a brand that I know will always have good prices", then you know you have a bit of a problem, as the assumption in a choice model is that people will rely on the attribute descriptions when making their choices.

Explain that the following choice questions vary

After the first question, it is usually a good idea to put in a screen saying something like "Now we are going to show you 11 questions, just like the one you completed. On each question the descriptions of the alternatives change, so please read them carefully." Without such a question, some respondents assume that there is an error and that they have been asked the same question twice!

Count down the number of choice questions

It is a good idea to make it clear how many questions have been answered and how many are left to do. This is because the questions are boring, and you do want to make people feel like it will not go forever. If asking a lot of questions, say more than 10, it may be a good idea to give the respondent a break in the middle (e.g., "Nice work. You are halfway done!").

Situational data and cueing

In some markets, the situation a person is in has a large impact on their choices. For example, what people choose to take to drink when going to a friend's house may be different from what they drink at home. In transportation studies, where people live is a big determinant of their options. It is usually useful to:

• Collect detailed information about the situation. This should include information regarding the specific attribute levels currently available to them (e.g., in a study of transport choice, collecting data on fuel costs, travel times, etc.).
• Occasion cue the choice questions. That is, rather than asking "which would you choose", try "Now thinking back to what you did last Saturday night if you had the following options available to you instead, which would you have chosen?" When occasion-cuing it is generally preferable to ask people about only one occasion. None of the standard choice models are applicable to dealing with situations where people are asked about multiple occasions.

Collect data for validation purposes

There are two types of information that can be useful when checking a choice model. First, it is useful to collect data that you expect to be correlated with the estimated utilities. For example, in a market with luxury goods, you may want to collect income or status consciousness and check that these are correlated with the estimated utilities. Second, it is usually a good idea to check information about current and historic behavior (e.g., brands previously purchased, last purchased, how often the consumer switches), so that you can verify and calibrate the model See 12 Techniques for Increasing the Accuracy of Forecasts from Choice Experiments for more information.

Profiling section of the questionnaire

There will usually be additional data requirements, such as demographics and attitudinal data.

Start and end-time information

It is usually helpful to collect data on the start and end-time. If it is possible, include the time taken to answer each of the choice questions. People identified as completing the questionnaire in an impossibly short amount of time can be excluded from further analyses.

Programming the questionnaire

Lastly, but most importantly, you need to make sure that the questionnaire is programmed correctly. You need to capture data about which questions were answered in which order by which people. Without this information, there is no way to analyze the data!

Get Started with Displayr

Now you've got your questionnaire ready, let us show you how Displayr, the most complete conjoint analysis software in the world, can help by booking a demo below.

I hope you found this information helpful