Conjoint Analysis and Preference Elicitation

Summary of the Previous Session

Imagine you’re designing a stress reduction app with three components: mindfulness exercises, social and peer support, and physical exercise. Randomized Controlled Trials (RCTs) can tell you if the app reduces stress, Multiphase Optimization Strategy (MOST) can optimize which components work best together, Micro-Randomized Trials (MRTs) can determine the best time to send mindfulness prompts, and Sequential Multiple Assignment Randomized Trials (SMART) can adapt the app’s features based on user progress. But how do you know if users prefer a $5/month app with mindfulness exercises alone or a $15/month app that includes physician-guided exercise? That’s where conjoint analysis comes in. Unlike these experimental methods, conjoint analysis reveals what people value and how they choose—critical insights for designing a product users will adopt before you even build it.

In the last session, each one of you presented an excepriment that was using one of these methods to examine a rwsearch question of your interest.

Your familiarity with RCTs, MOST, MRT, and SMART equips you to test and refine interventions. Here’s a quick recap:

• RCTs: Measure the causal impact of one intervention versus another.

• MOST: Optimize intervention components for efficiency and effect.

• MRTs: Assess just-in-time effects of micro-interventions.

• SMART: Adapt strategies over time based on response.

Conjoint analysis, however, fills a unique gap by focusing on preference elicitation and trade-off measurement—key for business decisions in strategy, marketing, and product development.

Types of Questions Conjoint Analysis Addresses

Below are five types of questions conjoint analysis can tackle that RCTs, MOST, MRT, and SMART are not well suited for, with examples tied to our stress reduction app.

1. How do users weigh trade-offs between multiple attributes when making a choice?
   Conjoint Strength: Quantifies the relative importance and utility of attributes (e.g., cost vs. mindfulness vs. exercise) and reveals user priorities in a single decision.
   Why Others Fall Short: RCTs test if the app works but don’t dissect why one version is preferred; MOST optimizes components but doesn’t uncover preferences organically; MRT focuses on timing, not trade-offs; SMART adapts over time but doesn’t map static preferences.
   Example: “When choosing a stress app, how much more do users value physical exercise feature over social support feature, and what’s the tipping point for price?”

2. What combination of features maximizes user preference or market share without real-world testing?
   Conjoint Strength: Simulates market scenarios to predict preferences (e.g., via choice-based conjoint) before launch.
   Why Others Fall Short: RCTs require real-world tests; MOST evaluates existing components, not untested combinations; MRT and SMART focus on interventions, not static predictions.
   Example: “Which bundle (e.g., $10 with mindfulness and support vs. $20 with all three) will attract the most users?”

3. What are the psychological or emotional benefits driving consumer choices beyond observable features?
   Conjoint Strength: Benefit-based conjoint digs into motivations (e.g., “calmness” from mindfulness vs. “trust” from peer support) rather than just attributes.
   Why Others Fall Short: RCTs, MOST, MRT, and SMART measure outcomes or behaviors, not latent preferences or emotions.
   Example: “Do users prefer the app for the peace of mindfulness or the trust of peer support, and how much are they willing to pay for each?”

4. How sensitive are consumers to changes in specific attributes (e.g., price elasticity)?
   Conjoint Strength: Isolates attribute impacts (e.g., price changes) and calculates willingness-to-pay or sensitivity.
   Why Others Fall Short: RCTs test price effects but not in multi-attribute contexts; MOST and SMART optimize but don’t quantify sensitivity; MRT targets timing, not static impacts.
   Example: “How much will a $5 price hike affect preference compared to adding physical exercise?”

5. What is the optimal product positioning or segmentation strategy based on preference heterogeneity?
   Conjoint Strength: Segments users by preference patterns (e.g., cost-sensitive vs. peer support focused) for tailored strategies.
   Why Others Fall Short: RCTs test average effects, not individual variations; MOST and SMART refine interventions, not market segments; MRT focuses on micro-decisions, not positioning.
   Example: “Which users value social support over cost, and how should we position the app to target them?”

These questions don’t replace RCTs, MOST, MRT, or SMART—they complement them by providing the “why” and “what” to design better experiments or strategies. Conjoint analysis equips you to understand user preferences in a way that experimental methods alone cannot, making it a vital tool for your business research toolkit.

Foundations of Conjoint Analysis

Explanation of Principles

Definition: Conjoint analysis is a quantitative research method that assesses how individuals value different attributes of a product or service by observing their choices or rankings in simulated decision tasks. Unlike direct surveys that rely on stated preferences—where users explicitly report what they like—conjoint analysis infers preferences indirectly by analyzing trade-offs made during these tasks, a process rooted in revealed preference theory. This distinction is critical: insights derived from revealed preferences are generally more reliable than stated ones, as they reflect actual decision-making behavior rather than hypothetical or biased self-reports, providing a robust framework for understanding consumer priorities. For our stress reduction app, it helps us determine how much users value physical exercise, mindfulness, and social support when deciding to subscribe.

Core Principles:

1. Users Evaluate Bundles: Participants assess the whole package, not features in isolation.
   Example: A user might weigh mindfulness alongside an affordable price, not just mindfulness alone.

2. Trade-Offs Matter: Choices reveal what people are willing to sacrifice.
   Example: Would users pay $15/month for all three features (physical exercise, mindfulness, social support) or opt for $5/month with just mindfulness?

3. Utility Scores Quantify Preferences: Conjoint assigns numerical values (utilities) to each feature level, showing their relative importance.
   Example: If physical exercise scores 4, mindfulness 3, and social support 2, physical exercise drives choice most.

Implementing Conjoint Analysis in the Stress Reduction App

Steps in Conjoint Analysis:

1. Design Attributes and Levels: Define key features and their variations.
   

   • Physical Exercise: None, Basic (videos), Premium (personalized plans).

   • Mindfulness: None, Basic (guided meditations), Premium (live sessions).

   • Social Support: None, Basic (chat groups), Premium (peer coaching).

   • Price: $5/month, $10/month, $15/month.

2. Design the Choice Profiles

   In conjoint analysis, the experimental design determines the combinations of attribute levels (profiles) that respondents evaluate. For our stress reduction app, we have four attributes—Physical Exercise, Mindfulness, Social Support, and Price—each with three levels (e.g., None, Basic, Premium for features; $5, $10, $15 for Price). A full factorial design, which includes every possible combination, results in $3^4 = 81$ profiles. While this captures all main effects and interactions, presenting 81 options to respondents is impractical due to fatigue and time constraints.

   To address this, we use a fractional factorial design, selecting a smaller subset of profiles that still allows us to estimate key effects, typically the main effects (i.e., the independent influence of each attribute on choice). The minimum number of profiles needed depends on the degrees of freedom: with four attributes, each having three levels (2 degrees of freedom per attribute), we require at least $4 \times 2 = 8$ plus 1 (for the overall mean), or 9 profiles, to estimate main effects. In practice, a slightly larger set—such as 12 profiles—improves balance and robustness, ensuring reliable preference estimates.

   The goal is to create an efficient design that is orthogonal (attributes vary independently, minimizing correlation) and balanced (each level appears roughly equally often). For example, a set of 12 profiles might include combinations like Basic Physical, Premium Mindfulness, No Social, $10, alongside others, carefully chosen to represent the attribute space without redundancy. These profiles are then organized into choice sets—typically pairs or small groups of options, often with a “none” choice—for respondents to evaluate. This reduced design preserves the statistical power to infer preferences while making the experiment manageable, forming the foundation of our conjoint analysis.

3. Collect Data: Present options and ask respondents to choose or rank.
   Example Choice Set:

   • Option 1: $5/month, Basic Mindfulness, No Physical Exercise, No Social Support.

   • Option 2: $15/month, Premium Physical Exercise, Premium Mindfulness, Basic Social Support.

   Respondents pick their preferred option, revealing trade-offs.

4. Analyze Preferences: Use statistical models (e.g., regression or logit) to estimate utility scores.
   

Analyzing Simulated Data

The table below shows the 12 profiles as they are assigned to the 6 choice sets, with each profile defined by its attribute levels. The ‘Option‘ column indicates the option number within the choice set (1 or 2), and ‘OptionID‘ links to the profile’s original position in the design.

In this design:

• Each of the 6 choice sets contains 2 unique profiles, resulting in 12 profiles total (e.g., ChoiceSet 1 pairs profiles with levels 1-1-2-3 and 1-2-3-2).

• The profiles are a subset of the full factorial, selected to efficiently estimate main effects of the attributes on preferences.

• During simulation, a “none” option (all attributes at level 1) is added to each set as Option 3, making 3 options per set for respondents to choose from.

• The final dataset reflects choices from these sets, though not all profiles may be chosen due to utility differences and random variation.

This structure allows us to test preferences across a manageable number of options while covering a broad range of attribute combinations.

proc sql;
    create table profile_summary as
    select 
        Physical,
        Mindful,
        Social,
        Price,
        count(*) as COUNT, /* Count occurrences of each profile */
        (count(*) / 600.0) * 100 as PERCENT format=8.4 /* Percentage out of 600, rounded to 4 decimals */
    from conjoint_data
    group by Physical, Mindful, Social, Price
    order by COUNT desc; /* Sort by count in descending order for readability */
quit;

After designing the conjoint experiment with 12 unique profiles paired across 6 choice sets (as shown in the previous table), we simulate choices for 100 respondents, each evaluating all 6 sets. Each choice set offers 3 options: 2 profiles from the original design plus a “none” option (Physical=1, Mindful=1, Social=1, Price=1). Respondents select the option with the highest utility, adjusted by random noise, resulting in 600 total choices (6 sets $\times$ 100 respondents). The table below lists the unique profiles chosen, their frequency (COUNT), and percentage of total choices (PERCENT).

This output shows 11 unique profiles chosen out of the 600 simulated choices, including the “none” option (Obs 1: 1-1-1-1). Compared to the original 12 profiles designed (see previous table):

• Profile Coverage: The original design included 12 distinct profiles, but only 10 of these appear here (Obs 2–11), plus “none.” Two profiles from our original 12—1 1 2 3 (ChoiceSet 1) and 2 2 1 2 (ChoiceSet 3)—were not chosen. Their utilities were outmatched by competitors in their sets, leaving 10 of the 12 profiles plus "none" in the final 600 choices

We can examine how respondents selected among the three options in each of the 6 choice sets. Table 9.3 shows the frequency of chosen options within each choice set, based on 100 respondents per set, totaling 600 choices. These counts reflect respondent preferences driven by utility values and random noise, revealing which options were favored in each set.

proc sql;
    proc freq data=conjoint_data;
    tables ChoiceSet * ChosenOption / nopercent norow nocol out=choices_by_set;
    title "Frequency of Chosen Options Within Each Choice Set";
run;
quit;

In our conjoint analysis, we should expand the original simulated dataset from 600 to 1800 observations by adding 1200 additional rows to include all possible options within each choice set. Initially, the dataset contained only the 600 chosen options (one per respondent per choice set), reflecting the selections of 100 respondents across 6 sets. However, choice-based conjoint models, such as those fitted by PROC MDC or PROC PHREG, require data on all alternatives—chosen and unchosen—to estimate part-worth utilities by comparing preferences across options. By adding the 1200 unchosen options (2 per set $\times$ 600 sets), we created a full choice set structure with 3 options per set (2 profiles + “none”), enabling the model to assess the relative utility of each attribute level within the context of every decision, thus ensuring accurate and robust estimation of respondent preferences.

/* Recreate choice_sets_full (6 sets x 3 options) */
data choice_sets_full;
    input ChoiceSet Option Physical Mindful Social Price;
    datalines;
    1 1 1 1 2 3  /* ChoiceSet 1: Option 1 */
    1 2 1 2 3 2  /* ChoiceSet 1: Option 2 */
    1 3 1 1 1 1  /* ChoiceSet 1: Option 3 (none) */
    2 1 1 3 1 1  /* ChoiceSet 2: Option 1 */
    2 2 2 1 1 1  /* ChoiceSet 2: Option 2 */
    2 3 1 1 1 1  /* ChoiceSet 2: Option 3 (none) */
    3 1 2 2 1 2  /* ChoiceSet 3: Option 1 */
    3 2 2 2 3 1  /* ChoiceSet 3: Option 2 */
    3 3 1 1 1 1  /* ChoiceSet 3: Option 3 (none) */
    4 1 2 3 2 2  /* ChoiceSet 4: Option 1 */
    4 2 2 3 3 3  /* ChoiceSet 4: Option 2 */
    4 3 1 1 1 1  /* ChoiceSet 4: Option 3 (none) */
    5 1 3 1 3 2  /* ChoiceSet 5: Option 1 */
    5 2 3 2 1 3  /* ChoiceSet 5: Option 2 */
    5 3 1 1 1 1  /* ChoiceSet 5: Option 3 (none) */
    6 1 3 2 2 1  /* ChoiceSet 6: Option 1 */
    6 2 3 3 1 2  /* ChoiceSet 6: Option 2 */
    6 3 1 1 1 1  /* ChoiceSet 6: Option 3 (none) */
    ;
run;

/* Merge to create full choice set data (1800 rows) */
proc sql;
    create table full_choices as
    select 
        a.ChoiceSet,
        a.Respondent,
        b.Option,
        b.Physical,
        b.Mindful,
        b.Social,
        b.Price,
        case when a.ChosenOption = b.Option then 1 else 0 end as Chosen
    from conjoint_data a
    cross join choice_sets_full b
    where a.ChoiceSet = b.ChoiceSet;
quit;

/* Confirm totals */
proc freq data=full_choices;
    tables Chosen;
    title "Verification: Count of Chosen vs. Unchosen Options";
run;

In choice-based conjoint (CBC) analysis, we estimate part-worth utilities to understand how attribute levels influence respondent preferences across multiple options. Two SAS procedures, PROC MDC and PROC PHREG, can achieve this, though they originate from different econometric traditions: discrete choice modeling and survival analysis, respectively. Despite their distinct purposes, they are equivalent for CBC under specific conditions, as demonstrated below using econometric specifications.

Consider a choice set $C_{ir}$ for respondent $i$ in choice set $r$, containing $J = 3$ options (e.g., two profiles and “none”). Each option $j$ has attributes $X_{jir} = (X_{jir1}, X_{jir2}, \ldots, X_{jirk})$ (e.g., Physical, Mindful, Social, Price levels), and the respondent chooses one option, denoted by $y_{ir} = j$.

Conditional Logit Model (PROC MDC)

In PROC MDC, the conditional logit model assumes the utility of option $j$ for respondent $i$ in choice set $r$ is: $$V_{jir} = X_{jir} \beta = \beta_1 X_{jir1} + \beta_2 X_{jir2} + \cdots + \beta_k X_{jirk}$$ where $\beta$ is a vector of part-worth utilities. The probability of choosing option $j$ is: $$P(y_{ir} = j | C_{ir}) = \frac{e^{V_{jir}}}{\sum_{k \in C_{ir}} e^{V_{kir}}}$$ The log-likelihood is maximized over all choices: $$LL = \sum_{i=1}^N \sum_{r=1}^R \log \left( \frac{e^{X_{y_{ir}} \beta}}{\sum_{k \in C_{ir}} e^{X_{kir} \beta}} \right)$$ Here, $N = 100$ respondents, $R = 6$ choice sets, and $J = 3$ options per set. PROC MDC directly estimates $\beta$ using this formulation, requiring data with all alternatives (1800 rows: 6 $\times$ 100 $\times$ 3).

Stratified Cox Model (PROC PHREG)

In PROC PHREG, designed for survival analysis, the hazard rate for an event (e.g., death) at time $t$ is: $$h(t | X) = h_0(t) e^{X \beta}$$ where $h_0(t)$ is the baseline hazard, and $X \beta$ is the linear predictor. For CBC, we adapt this by:

• Treating the choice event as instantaneous (no time dimension).

• Defining “chosen” (Chosen=1) as the event, and unchosen options (Chosen=0) as censored.

• Stratifying by ChoiceSet and Respondent (600 strata), ensuring comparisons within each choice set.

The partial likelihood for a stratum (choice set $C_{ir}$) with one chosen option $j$ is: $$L_{ir} = \frac{e^{X_{jir} \beta}}{\sum_{k \in C_{ir}} e^{X_{kir} \beta}}$$ The total log-likelihood across all strata is: $$LL = \sum_{i=1}^N \sum_{r=1}^R \log \left( \frac{e^{X_{jir} \beta}}{\sum_{k \in C_{ir}} e^{X_{kir} \beta}} \right)$$ where $j$ is the chosen option in $C_{ir}$.

Equivalence

Comparing the log-likelihoods:

• PROC MDC: $LL = \sum_{i=1}^N \sum_{r=1}^R \log \left( \frac{e^{X_{y_{ir}} \beta}}{\sum_{k \in C_{ir}} e^{X_{kir} \beta}} \right)$

• PROC PHREG: $LL = \sum_{i=1}^N \sum_{r=1}^R \log \left( \frac{e^{X_{jir} \beta}}{\sum_{k \in C_{ir}} e^{X_{kir} \beta}} \right)$

Both are identical, as $y_{ir} = j$ represents the chosen option. In PROC PHREG, the baseline hazard $h_0(t)$ cancels out within each stratum (since only one event occurs), leaving the same utility form as conditional logit. Thus, the $\beta$ estimates (part-worth utilities) from PROC PHREG match those from PROC MDC’s conditional logit model, adjusted for parameterization (e.g., reference coding with ref=’1’).

/*Estimate Part-Worth Utilities with PROC MDC */
/*This does not seem to work*/
proc mdc data=full_choices;
    class Physical  Mindful Social Price ;
    model Chosen = Physical Mindful Social Price /
          type=clogit
          nchoice=3; /* 3 alternatives per choice set */
    id SetRespID_num; /* Numeric ID for each ChoiceSet-Respondent pair */
    title "Conditional Logit Model for Utility Estimation";
run;
/*Estimate Part-Worth Utilities with PROC PHREG */
proc phreg data=full_choices;
    class Physical (ref='1') Mindful (ref='1') Social (ref='1') Price (ref='1') / param=reference;
    model Chosen = Physical Mindful Social Price / ties=breslow;
    strata ChoiceSet Respondent;
    title "Conditional Logit Model for Utility Estimation";
run;

Interpreting Part-Worth Utilities from PROC PHREG

Table 9.4 below presents the maximum likelihood estimates, with reference levels set to 1 (None for features, $5 for Price), reflecting utilities relative to these baselines. Each parameter’s significance is assessed via chi-square tests, and hazard ratios indicate the relative likelihood of selection compared to the reference level.

The results reveal respondent preferences relative to the absence of features (None, level 1) and a $5 price point (Price level 1). For Physical Exercise, Basic (2) significantly increases utility by 0.419 (p=0.0005) over None, while Premium (3) slightly reduces it by -0.116 (p=0.4521), suggesting a preference for moderate exercise over none or premium options, though the latter is not significant.

Mindfulness shows a significant aversion, with Basic (2) decreasing utility by -0.473 (p=0.0004) and Premium (3) by -0.871 (p$<$0.0001) compared to None, indicating respondents disfavor mindfulness features.

Social Support follows a similar trend, with Basic (2) reducing utility by -0.330 (p=0.0015) and Premium (3) sharply decreasing it by -2.173 (p$<$0.0001), reflecting a strong rejection of social features, especially at premium levels.

For Price, $10 (2) slightly increases utility by 0.132 (p=0.3403), but not significantly, while $15 (3) significantly boosts it by 0.682 (p$<$0.0001), a counterintuitive result suggesting higher prices are preferred over $5. This anomaly likely stems from a simulation coding error (e.g., Price levels flipped: $5 as 3, $15 as 1), inconsistent with typical price sensitivity. At the end of the day, I am only a human and at this point, I really do not want to go back and fix my simulation code ;)

Required readings