Self consistency prompting

Self-consistency and ensemble prompting is a variance-reduction technique that samples multiple model decodings (commonly 5–40 chains) and aggregates them—typically by majority voting—to improve accuracy on reasoning and classification tasks. In practice this means generating N independent chains with stochastic decoding (e.g., temperature >0 or nucleus sampling) and returning the modal answer or a probability-weighted vote; the approach is defined in the chain-of-thought self-consistency literature and was shown to outperform single-chain decoding on multiple benchmarks. Wang et al. sampled 40 chains in the original chain-of-thought self-consistency experiments. Used in production, teams often select a final answer using modal vote or aggregate log-prob scoring to balance accuracy and calibration.

Mechanically, self-consistency prompting works by generating multiple independent Chain-of-Thought (CoT) or answer chains using stochastic samplers (temperature, top_p or nucleus sampling) and then aggregating outputs via majority voting or probability-weighted voting using token log-probabilities. This is analogous to bootstrap aggregating (bagging) in classical ensembles: sampling different decodings reduces variance while preserving systematic bias. Ensemble prompting and prompt ensembling achieve diversification by varying prompt templates, few-shot exemplars, or instruction phrasing rather than model weights. Practical tool support includes sampling controls in OpenAI and Hugging Face APIs and evaluation with calibration metrics such as expected calibration error (ECE). Beam search reduces diversity and is not a substitute.

A frequent misconception equates self-consistency prompting with a universal accuracy fix; in reality it is a variance-reduction strategy that trades roughly N× inference cost for robustness and interacts with prompt bias. For example, on arithmetic benchmarks chain-of-thought self-consistency can correct random decoding errors but will not fix a prompt that systematically produces an incorrect reasoning pattern, and prompt ensembling across diverse templates is required to address that failure mode. Majority voting can also amplify label imbalance in classification tasks unless votes are weighted by calibrated probabilities or adjusted using temperature and Platt-style calibration. Empirical gains typically show diminishing returns after tens of samples, so reporting sampling parameters (temperature, top_p, random seeds) is essential for reproducibility in production deployments. For instance, in imbalanced fraud-detection majority voting can reinforce dominant-class bias.

Operationally, best practice is to benchmark a baseline deterministic decode (temperature=0) against a self-consistency prompting run with calibrated sampling (for example temperature 0.7–1.0, top_p 0.9, 20–40 chains), evaluate with accuracy and calibration metrics such as ECE and F1, and measure cost per effective improvement (latency and tokens). For low-cost deployments, light ensembles can use prompt ensembling over 3–5 templates with probabilistic voting or log-prob weighting to capture orthogonal signal without full N× model cost. A pragmatic ramp is N=5 → N=20 while tracking marginal accuracy per sample. This page contains a structured, step-by-step framework.