How to Compute D-Error for a Sawtooth Software CBC Experiment

In other articles I provide the mathematical definitions of D-error and worked examples of how to calculate D-error; but in the real world, most people will use existing tools to compute D-error. In this article I describe how to use Q and Displayr to compute the D-error for a design from a Sawtooth Software CBC experiment.

Loading the design

The Sawtooth design file should have the form shown below, with the first three columns indicating the version, task and concept (alternatives) and the subsequent columns containing levels for each attribute, represented by numbers starting from 1.

To load the design into Q, click on File > Data Sets > Add to Project > From File… and select the design file. Click OK in the Data Import dialog box. To load the design into Displayr, click on Home/Insert > Data Set and select the design file either locally from your computer or from an online source such as Dropbox. The design should appear as a new data set with each column in the design appearing as a variable.

Computing D-error

The D-error will be computed using R code though an R Output, which in Q is created by clicking on Create > R Output in the menu, and in Displayr is created by clicking on Insert > R Output in the ribbon menu. In the R CODE box on the right, enter the following code:

</pre>
library(flipChoice)

attribute.levels <- c(2,2,3)

version <- `Version`

task <- `Task`

alternative <- `Concept`

n.versions <- length(unique(version))

n.alternatives.per.question <- length(unique(alternative))

n.questions.per.version <- length(task) /

(n.versions * n.alternatives.per.question)

questions <- rep(rep(1:n.questions.per.version,

each = n.alternatives.per.question),

n.versions)

design <- cbind(version, task, questions, alternative,

`Attribute1`, `Attribute2`, `Attribute3`)

colnames(design)[1:4] <- c("Version", "Task",

"Question","Alternative")

DError(design, attribute.levels, effects = FALSE)

Replace the assignment to the variable attribute.levels in the second line with the appropriate vector. In this example, I've assigned c(2,2,3)because there are 2, 2 and 3 levels in the three attributes in the design.

Also replace the variable names (which are surrounded by backticks ``) with the variable names from your design. Remember to keep the backticks around each name to ensure that names containing spaces or special characters will still work.

Once you have modified the R Code, click on the Calculate button in the top right and the D-error should appear in the output. The default number of decimal places is 1; if this is insufficient, you can increase the number of decimal places shown via the toolbar in the top left of the window in Q or via the Number section of the Appearance tab.

Note that by setting effects = FALSE, I have chosen to use dummy coding instead of effects coding. Since I have not passed in priors for the parameters, D0-error will be computed in this case.

Specifying priors

Priors are specified as an extra parameter in the call to DError. D_P-error is computed when prior is a vector of parameters. For example:

</pre>
prior <- c(0.5, 1.0, -1.0, -2.0)

DError(design, attribute.levels, effects = FALSE, prior = prior)
<pre>

On the other hand, D_B-error is computed when prior is a matrix with two columns. The first and second columns of this matrix correspond to the means and standard deviations of the normal distributions of the prior parameters.

We hope you found this article helpful. Find out more about choice model experimental designs by heading on over to market research section of our blog.