Advanced CalibrationPro+

Multi-Arm Calibration

Test more than two survey variants simultaneously. The number of variants (including the control) is limited by your subscription plan.

Variant Limits by Plan

Bonferroni Correction

When comparing multiple challengers against a single control, the significance level is adjusted to control the family-wise error rate:

adjusted_alpha = alpha / number_of_challengers

For example, with alpha = 0.05 and 4 challengers, each pairwise comparison uses alpha = 0.0125. This tighter threshold increases the required sample size per variant.

Winner Detection

Winner detection uses pairwise mSPRT (mixture Sequential Probability Ratio Test), comparing each challenger independently against the control. The first challenger to beat the control at the adjusted significance level is declared the winner.

Multi-Objective Calibration

Configure how calibration rewards are computed by weighting three objective dimensions: completion, attentiveness, and fidelity. Attentiveness is one minus half a variant's straightlining rate and half its speeding rate; its weight is sent as quality.

Set a survey's weights with PUT /api/v1/surveys/:id/calibration/strategy (scope surveys:write), or the MCP tool update_calibration_strategy. Send strategy as one of the presets below, or custom with weights: { completion, quality, fidelity }, each between 0 and 1 and summing to within 0.05 of 1. The response is the weights stored.

A survey that sets no weights inherits them: each objective comes from the survey's own weights where it sets them, else the organisation's default, the platform's, and finally completion alone. The weights are scaled to sum to 1 when rewards are computed. The API sets weights but does not clear them; the dashboard's calibration page has "Use defaults" for that.

Strategy Presets

The presets the endpoint and the MCP tool accept, and the weights each stores.

Weighted Reward Formula

The calibration reward for each response is computed as:

reward = w_c * completion + w_q * attentiveness + w_f * fidelity

Where w_c, w_q, and w_f are the objective weights configured for the survey (see above).

Fidelity Score Components

The fidelity score is a composite of five sub-metrics that measure response trustworthiness:

Cross-Customer Calibration Priors

Look up whether a given category and question type combination historically produces meaningful calibration effects, based on anonymised data aggregated across all opted-in Surventrics customers.

Query Parameters

Example Request

curl -X GET "https://app.surventrics.ai/api/calibration/insignificance-prediction?category=satisfaction&questionType=likert" \
  -H "Authorization: Bearer YOUR_API_KEY"

Response

{
  "isInsignificant": true,
  "confidence": 0.97,
  "basedOnN": 312,
  "insignificanceRate": 0.97
}

Response Fields

How It Works

• Aggregated from anonymised calibration results across all Surventrics customers who have opted in to cross-customer data sharing.
• When there is no applicable prior for the category/type, the endpoint still returns 200 with isInsignificant: false and confidence: 0 (rather than an error), so callers can treat low confidence as “not enough data”.
• All contributing customers must have granted GDPR consent for anonymised data sharing.
• Insignificance threshold: >96% of past calibrations showed <2 percentage point effect.

Errors

Errors are returned with an HTTP status and a flat { "error": "<message>" } body (not the structured envelope used by the key-authenticated endpoints).