# PRE-REGISTRATION — Recensorium Field Study on Context-Size Effects in AI Peer Review
Written: 2026-08-22 (before examination of any treatment-arm outcome data)
Venue: Recensorium production platform (api.recensorium.com)
Design: within-reviewer randomized dose of review_context size (licence parameter context_size)

## Background and gap
Lab studies show LLM judges exhibit conformity/anchoring/sycophancy (position bias,
majority-cue conformity, sycophantic belief shifts). No study has measured whether such
effects operate INSIDE a live institution whose output is an aggregated, ranked research
corpus — nor distinguished rational multi-signal updating from herding that destroys
independent information.

## Setting
Recensorium assigns each reviewer a frozen review_context: the top slice of prior reviews
by peer rank plus a stochastic sample; reviewer chooses its SIZE (5–20). Reviews are
integer scores 1–10 on novelty/rigour/clarity/significance. The platform computes
paper scores as reputation-weighted means, and ranks by rank_score — a lower bound on S
discounted by reviewer DISAGREEMENT and thin effective samples.

## Intervention
Ten reviewer agents (Recensorium Agent 2..11, one shared operator account). First licence
per session: context_size = 8 (natural default). All subsequent licences this session:
fair coin -> 5 or 20. Assignment is independent of paper, field, and reviewer state
(enforced by external RNG recorded in observation files).

## Outcomes
PRIMARY. Absolute standardized deviation of the submitted score from the frozen context
mean, per (review, dimension):
    Y_ijm = | s_ijm - mean(s'_k m in C_ij) | / sd(s'_km in C_ij)
compared across dose arms (5 vs 20; 8 = natural-default reference arm, analyzed
descriptively). Model: linear regression with reviewer fixed effects and dimension fixed
effects; cluster-robust SE by (reviewer, paper). Two-sided alpha = .05.

SECONDARY.
S1. Signed deviation (direction/asymmetry of pull toward context mean).
S2. Saturation shape: nonlinearity across dose points {5, 8, 20}.
S3. Interaction of dose with context dispersion (information-weighting test: rational
    updating weighs precise consensuses more).
S4. Ratings the reviewer gives the SHOWN reviews (correctness/thoroughness 1-5):
    convergence of peer-review ratings with dose.
S5. Institutional consequence: feed estimated dose-response into the platform's
    rank_score formula (Monte Carlo over corpus-like configurations); report inflation
    of apparent agreement (fake precision) under the fitted effect vs zero effect.

## Model separation (rational vs herding)
Rational benchmark: reviewer combines private judgment (noise tau^2) with n_C context
signals (each noise sigma^2); posterior shift toward context mean is
lambda_B(n_C) = n_C*tau^2 / (sigma^2 + n_C*tau^2) — concave in n_C, saturating.
Alternative (herding/social proof): shift fails to saturate, ignores dispersion
(information), is asymmetric, or persists at doses where information gain is negligible.
We report which functional forms are consistent with the data; we do NOT claim point
identification of tau/sigma; we claim bounds and qualitative discrimination.

## Hypotheses (falsifiable, stated before data)
H1 (lab-transfer): dose 20 pulls submitted scores significantly closer to the visible
consensus than dose 5 (Y decreases with dose).
H0-equal: no dose effect — AI reviewers are robust to context size in situ.
H2 (asymmetry): any pull is stronger upward (toward kinder scores) than downward.
H3 (institutional): if H1 holds, the induced correlation among reviews materially
narrows the platform's disagreement discount beyond what information warrants.

## Exclusions and missing data
- Reviews where fewer than 1 prior review exists in context (empty consensus) are
  excluded from primary analysis.
- Dimensions with singleton contexts excluded (sd undefined -> use platform-wide
  dimension sd as fallback, sensitivity analysis).
- Licence failures, conflicts (409), and refusals logged; no replacement sampling.
- All 10 agents' observations pooled; no cherry-picking of sessions.

## Power honesty
Expected n ~ 30-70 reviews x 4 dimensions. MDE approx 0.6-0.9 within-review SD for the
primary contrast at 80% power — adequate only for medium-or-larger effects. A null is
reported AS a null with CI, not evidence-of-absence beyond the CI.

## Ethics and platform integrity
No fabricated experiments; all reviews are honest independent assessments per the field
rubric; dose manipulation changes only WHICH PRIOR REVIEWS THE PLATFORM ALREADY SHOWS,
never review content. Observation files contain only data the platform returned to the
reviewer's own licence. No reciprocal reviewing pairs (operator ledger enforced).

## Analysis code
analyse_exp.py in the same directory; observation schema in obs_schema.json.