Advanced CalibrationPro+

Extend calibration with multi-arm variant testing, multi-objective reward weighting, and cross-customer insignificance priors.

Multi-Arm Calibration

Test more than two survey variants simultaneously. The number of variants (including the control) is limited by your subscription plan.

Variant Limits by Plan

Plan	Max Variants (incl. control)
Pro+	3
Business+	5
Enterprise	10

Bonferroni Correction

When comparing multiple challengers against a single control, the significance level is adjusted to control the family-wise error rate:

adjusted_alpha = alpha / number_of_challengers

For example, with alpha = 0.05 and 4 challengers, each pairwise comparison uses alpha = 0.0125. This tighter threshold increases the required sample size per variant.

Winner Detection

Winner detection uses pairwise mSPRT (mixture Sequential Probability Ratio Test), comparing each challenger independently against the control. The first challenger to beat the control at the adjusted significance level is declared the winner.

Multi-Objective Calibration

Configure how calibration rewards are computed by weighting three objective dimensions: completion, quality, and fidelity.

Not available via the public API. Objective weights are configured per survey in the dashboard (Settings → Calibration) and resolve as per-survey → per-org → platform default. There is currently no public endpoint to set them. The sections below document how the weights shape the calibration reward.

Strategy Presets

Use one of the built-in strategy presets, or supply custom weights.

Preset	Completion	Quality	Fidelity	Use Case
`completion`	1.0	0.0	0.0	Maximize survey completion rate
`quality`	0.5	0.35	0.15	Balance completion with response quality
`balanced`	0.34	0.33	0.33	Equal weight across all objectives
`fidelity`	0.15	0.15	0.7	Prioritize data integrity and trustworthiness

Weighted Reward Formula

The calibration reward for each response is computed as:

reward = w_c * completion + w_q * quality + w_f * fidelity

Where w_c, w_q, and w_f are the objective weights configured for the survey (see above).

Fidelity Score Components

The fidelity score is a composite of five sub-metrics that measure response trustworthiness:

Component	Description
Attention accuracy	Correct responses to attention-check questions
Time consistency	Response times within expected range for question type
Answer variance	Non-flat-lining — variation in responses across questions
Completeness	Percentage of optional questions answered
Engagement depth	Time spent on open-text and complex question types

Cross-Customer Calibration Priors

Look up whether a given category and question type combination historically produces meaningful calibration effects, based on anonymised data aggregated across all opted-in Surventrics customers.

GET/api/calibration/insignificance-prediction

Required scope:surveys:read

Plan:Pro+or higher

Rate limit:20/minute per org

Query Parameters

Name	Type	Description
`category`	string	Calibration category, required (e.g., satisfaction, demographics, product_feedback).
`questionType`	string	Question type, optional. Must be a valid question type (e.g., single_choice, likert, nps).

Example Request

curl -X GET "https://surventrics.ai/api/calibration/insignificance-prediction?category=satisfaction&questionType=likert" \
  -H "Authorization: Bearer YOUR_API_KEY"

Response

{
  "isInsignificant": true,
  "confidence": 0.97,
  "basedOnN": 312,
  "insignificanceRate": 0.97
}

Response Fields

Name	Type	Description
`isInsignificant`	boolean	true when the insignificance rate meets the threshold (>96% of past calibrations showed a <2pp effect).
`confidence`	number	Confidence in the prediction (0.0 - 1.0); equals the insignificance rate, and is 0 when there is no applicable prior.
`basedOnN`	number	Number of anonymised cross-customer observations the prediction is based on.
`insignificanceRate`	number	Fraction of past calibrations for this category/type that showed a less-than-threshold effect (0.0 - 1.0).

How It Works

Aggregated from anonymised calibration results across all Surventrics customers who have opted in to cross-customer data sharing.
When there is no applicable prior for the category/type, the endpoint still returns 200 with isInsignificant: false and confidence: 0 (rather than an error), so callers can treat low confidence as “not enough data”.
All contributing customers must have granted GDPR consent for anonymised data sharing.
Insignificance threshold: >96% of past calibrations showed <2 percentage point effect.

Errors

Errors are returned with an HTTP status and a flat { "error": "<message>" } body (not the structured envelope used by the key-authenticated endpoints).

Status	When
`400`	The required `category` query parameter is missing.
`500`	The prediction could not be computed.