Three tasks use original adjudication to evaluate predictions of match outcomes, stage assessments and individual awards. For match m with transcript Xm, stage s and evaluated side a, each stage–side instance (m, s, a) supplies one scoring target.
Input: motion, format and the complete transcript. Output: preference p ∈ [0, 1] in steps of 0.01; above 0.5 predicts the affirmative, below 0.5 the negative. Reference: majority of the three consolidated judge decisions. A valid output of exactly 0.5 counts as incorrect. Metric: accuracy over evaluable matches.
Input: the target stage, its preceding context, the evaluated side and role, and the rubric. Output: integer score 1–10. Reference: mean of the three original ratings. Metrics: MSE, Pearson r, Spearman ρ; paired-side agreement compares the sign of predicted and reference affirmative − negative differences on stages with both sides scored (full agreement includes reference ties; non-tie agreement excludes them).
Input: the complete transcript. Output: nine integer votes over Km participants (8 in 4v4, 4 in 2v2). Reference: the panel's nine ballots. Metrics: selection accuracy (model winner within the human top-vote set), per-match vote-share MAE (1/Km Σ|q̂ − q|), Pearson r and Spearman ρ of vote shares.
Nine model configurations were evaluated on Tasks 1 and 3 and five on Task 2, on 148 matches and 2,690 valid stage–side instances. Tasks 1 and 3 use independent calls; Task 2 receives the target stage with preceding context and scoring instructions. Adjudication records and generated supplements are excluded from inputs.
Inference uses temperature zero where supported and structured outputs, with up to three attempts per instance, stopping at the first parsed prediction. Parsing uses JSON normalisation and field extraction. Valid incorrect predictions remain in the evaluation; unrecoverable outputs are excluded from the corresponding metric; raw outputs are retained. Results are point estimates, so small differences between models should be interpreted cautiously.
References: always-affirmative and random winner prediction (p ~ U(0,1)); random stage scores; random best-debater selection and uniform vote shares (1/Km). An exploratory structure-aware baseline assigns 8 to questioners and 6 to respondents in questioning stages and random scores elsewhere.
| Predictor | MSE ↓ | Pearson r | Spearman ρ | Full agreement | Non-tie agreement |
|---|---|---|---|---|---|
| deepseek-v4-flash | 2.72 | .250 | .253 | .402 | .638 |
| deepseek-v4-flash (Reasoning) | 2.54 | .241 | .241 | .403 | .602 |
| deepseek-v4-pro | 2.13 | .231 | .240 | .434 | .596 |
| gemini-3.5-flash-lite | 1.91 | .184 | .181 | .391 | .529 |
| gpt-5.6-luna | 2.52 | .232 | .237 | .360 | .579 |
| Random | 9.69 | ≈ 0 | ≈ 0 | .239 | .451 |
| Structure-aware random | 6.64 | .031 | .050 | .291 | .634 |
Valid stage–side evaluation set (2,690). Gemini-3.5-flash-lite has the lowest MSE; deepseek-v4-flash the highest correlations. The structure-aware baseline reaches .634 non-tie agreement, close to the best model (.638), despite far worse MSE and correlations — directional agreement must be read together with numerical error.
| Predictor | Winner accuracy | Best-debater accuracy | Vote-share MAE ↓ | Vote-share r | Vote-share ρ |
|---|---|---|---|---|---|
| opus-5 | .662 | .520 | .113 | .578 | .591 |
| deepseek-v4-pro | .615 | .554 | .118 | .549 | .534 |
| gpt-5.6-sol | .601 | .568 | .117 | .561 | .560 |
| deepseek-v4-flash (Reasoning) | .601 | .439 | .129 | .456 | .470 |
| sonnet-5 | .595 | .419 | .126 | .509 | .515 |
| gemini-3.5-flash-lite | .595 | .345 | .136 | .393 | .425 |
| deepseek-v4-flash | .588 | .432 | .125 | .481 | .488 |
| haiku-4.5 | .588 | .399 | .138 | .395 | .380 |
| gpt-5.6-luna | .561 | .432 | .122 | .508 | .524 |
| Always affirmative | .507 | — | — | — | — |
| Random | .500 | .183 | — | — | — |
| Uniform vote shares | — | — | .154 | — | — |
Winner accuracy ranges from .561 to .662 against .507 (always affirmative) and .500 (random). For best-debater prediction gpt-5.6-sol has the highest selection accuracy (.568) while opus-5 best matches vote shares (MAE .113, r .578, ρ .591): selection and distributional agreement favour different models.
| Winner source | Third-speaker accuracy |
|---|---|
| opus-5 | .526 |
| deepseek-v4-pro | .491 |
| sonnet-5 | .491 |
| gemini-3.5-flash-lite | .474 |
| deepseek-v4-flash | .457 |
| deepseek-v4-flash (Reasoning) | .457 |
| haiku-4.5 | .457 |
| gpt-5.6-sol | .440 |
| gpt-5.6-luna | .422 |
| Random winner | .401 |
Eligible 4v4 matches only: the reference selects the third speaker on the side the model predicted to win in Task 1. Different population from the pooled Task 3 results; no matched comparison is claimed.
| Reference | Winner acc. | Stage MSE | Stage r | Stage ρ | Best-debater acc. | Vote-share MAE |
|---|---|---|---|---|---|---|
| Human, self-included (individual vs. own three-judge panel) | .851 | .759 | .716 | .682 | .698 | .079 |
| Human, leave-one-out (individual vs. the other two judges; split two-judge winner targets excluded) | .788 | 1.709 | .337 | .296 | .576 | .121 |
| Actual-winner third speaker (4v4; reference outcome supplied) | — | — | — | — | .569 | — |
Descriptive human-agreement references, not performance bounds: self-inclusion induces dependence between an assessment and its target, and leave-one-out changes both the target and the eligible population.