The methodology begins with a commercially important buyer decision, establishes the conditions under which it will be observed, measures repeated recommendation behavior where appropriate, and reserves deeper investigation for gaps that are both meaningful and plausibly actionable.

Operating sequence

Measure → Diagnose → Prioritize → Test → Learn

1. Measure

Define the buyer decision, relevant geography, constraints, providers, consumer surfaces, prompt language, trial count, and validity rules before drawing conclusions. Preserve the exact prompt, timestamp, complete response, recommendation order, companies mentioned, visible citations, and environment provenance.

Where variation matters, use repeated independent observations in fresh conversations. Report exclusions separately and keep the denominator visible.

2. Diagnose

Decompose the observed answer into the decision, candidate set, represented beliefs, visible or stated evidence, and recommendation. Compare where the business wins, where competitors win instead, and how the pattern changes across buyer contexts and AI environments.

Diagnosis can identify plausible explanations and evidence gaps. It does not reveal undisclosed retrieval behavior or a model's private reasoning process.

3. Prioritize

Not every recommendation loss is an opportunity. A gap deserves deeper work only when the buyer decision matters economically, the business is genuinely a strong fit, the observed pattern is meaningful enough to act on, and a controllable variable may plausibly affect the outcome.

This step protects businesses from optimizing for noise, poor-fit customers, low-value decisions, or signals too unstable to support action.

4. Test

When intervention is justified, define the exact change, the evidence-based hypothesis, the success measure, the collection design, and the time boundary. Recommendation Engineering may involve improving accurate public evidence, clarifying entity information, strengthening decision-relevant proof, or another tightly scoped intervention.

Haenisch does not treat content production as the automatic answer and does not attempt to manipulate AI systems artificially.

5. Learn

Remeasure the same decision environment where comparability is possible. Record what remained constant, what changed, what else could explain the result, and how much uncertainty remains. Preserve the original observation and publish corrections or updates rather than silently rewriting history.

The advanced analytical sequence

  1. 01Observed Recommendation
  2. 02Decision Decomposition
  3. 03Evidence Diagnosis
  4. 04Intervention
  5. 05Remeasurement
  6. 06Causal Learning

This sequence is progressive, not automatic. Measurement does not guarantee diagnosis. Diagnosis does not guarantee a worthwhile intervention. A changed result does not automatically prove causation.

Observational evidence and causal claims

Most consumer AI recommendation studies are observational. They can show what happened under documented conditions and how outcomes were distributed across repeated trials. They may support hypotheses about relevant evidence or decision stages.

Causal language is appropriate only when the design meaningfully isolates the intervention, preserves a valid comparison, accounts for volatility and other changes, and produces a result strong enough to support that conclusion. Otherwise Haenisch reports association, difference, or a plausible explanation—not causal certainty.

Minimum reporting discipline

  • Exact research question, decision context, prompt, and frozen protocol date.
  • Provider, consumer surface, visible model label, sign-in state, temporary/private mode, locale, time zone, and collection timestamps where observable.
  • Planned, valid, and excluded observation counts with validity rules.
  • Operational definitions for mention, inclusion, recommendation, order, and citation.
  • Counts and percentages together, with prompt-level and provider-level distributions where relevant.
  • Raw-response retention and data-availability statements.
  • Limitations, conflicts, funding, external spend, and paid API use when relevant.
  • Version history, corrections, and dated remeasurements.

AI products, interfaces, and model behavior can change. A well-documented study is a bounded measurement of its collection conditions, not a timeless statement about every user or future answer.