Skip to contents

Design Your Survey

design.Rmd

We strongly recommend using our Projoint Survey Design Tool. You can easily:

  • Set up attributes and levels
  • Randomize attributes and levels with weights
  • Add cross-attributes and cross-profile constraints
  • Automatically create a repeated task for intra-respondent reliability (IRR) estimation
  • Generate test responses for checking your design
  • 🚀 Export your survey directly for Qualtrics (in QDF format) ✅

Conjoint Survey Design Tool (FYI Only)

Why Designing Conjoint Surveys Was Challenging in the Past

1. Generate a JavaScript or PHP randomizer

Many researchers use Anton Strezhnev’s Conjoint Survey Design Tool (Link: conjointSDT) to produce a JavaScript or PHP randomizer.

JavaScript

The JavaScript randomizer can be inserted into the first screen of your Qualtrics survey using Edit Question JavaScript. Example screenshot:

The JavaScript runs internally within Qualtrics and generates embedded fields for each conjoint task.
For example:

  • "K-1-1-7" = value for the 7th attribute, first profile, first task
  • "K-5-2-5" = value for the 5th attribute, second profile, fifth task

PHP

Alternatively, the PHP randomizer must be hosted externally.

Example hosted on our server:
https://www.horiuchi.org/php/ACHR_Modified_2.php
(PHP file here)

This method was used in:
Agadjanian, Carey, Horiuchi, and Ryan (2023)

2. Modify your JavaScript or PHP randomizer

You may want to add constraints — for example, prevent ties between profiles.
To do this, you can manually modify your JavaScript or PHP.

In the future, projoint will offer easier ways to add constraints!
Until then, resources like OpenAI’s GPT-4 can help you edit scripts.

Example PHP snippet ensuring racial balance between profiles:

$treat_profile_one = "B-" . (string)$p . "-1-" . (string)$treat_number;
$treat_profile_two = "B-" . (string)$p . "-2-" . (string)$treat_number;
$cond1 = $returnarray[$treat_profile_one] == "White" && $returnarray[$treat_profile_two] == $type;
$cond2 = $returnarray[$treat_profile_two] == "White" && $returnarray[$treat_profile_one] == $type;

if ($cond1 or $cond2) {
  $complete = True;
}

If you have good examples of manual constraints, please email Yusaku Horiuchi!

3. Add conjoint tables with embedded fields in Qualtrics

After generating the randomizer, you must create HTML tables displaying embedded fields for each task.

Example of the first task:

Each conjoint study typically includes 5-10 tasks.

The embedded fields update across tasks:
e.g., "K-1..." for Task 1, "K-2..." for Task 2, and so on.

It’s easy to create a repeated task for intra-respondent reliability (IRR) estimation:

  • Copy the HTML for Task 1 later into the survey (e.g., after Task 5)
  • Flip Profile 1 and Profile 2 (swap the embedded field digits)

Example repeated task:

4. Sample Qualtrics survey template

We provide a full Qualtrics QSF file ready to use!

This file recreates a standard conjoint survey setup based on Mummolo and Nall (2017).

🌟 What’s Next?

Now that your survey is properly designed, you’re ready to collect responses in Qualtrics and import the data into R!

➡️ Continue to: Read Your Qualtrics Data
🏠 Home: Home