The bounded late-night blunder null (H1) is internally consistent and well tested, but the preregistration cannot be verified, the 'Lichess blunder rule' is not the rule in the cited Lichess source, the night S−E deficit rests on post-outcome session controls, and the abstract and plain-language summary overstate what Tables 3–4 show.
Soundness
2/4
Presentation
3/4
Contribution
2/4
Confidence
4/5
✓ 12 confirmed✕ 5 discrepancy◐ 3 inconclusive
Summary
The paper asks whether the local hour of play is associated with blunders and rating-adjusted results on Lichess. It draws a hashed 1/16 player sample from six monthly Lichess dumps (Dec 2024–Feb 2025, Jun–Aug 2025), assigns local time from profile country flags (with hourly activity used to pick a zone inside multi-zone countries), and fits weighted least-squares models with player×speed fixed effects, hour dummies and controls for session length, position in session, breaks and preceding results. Outcomes are a blunder indicator on engine-analysed games, score minus expected score (S−E, in Elo-equivalents) and normalised move time. Contrasts are judged against pre-registered smallest effects of interest with TOST and Holm corrections.
The headline (H1) is a well-bounded null: late-night (00:00–04:59) blunder rates are −0.20% of the daytime rate, with a +5% increase ruled out. S−E is 2.26 Elo-equivalents lower at night (H2), but this is concentrated in Bullet and early-active players, halves under a one-hour window shift and disappears without session/prior-result controls. Evening moves are 1.1% faster than morning moves without more blunders (partial replication of Leone et al. [31]). The paper concludes that within-session position and loss runs (3.5–9.6 Elo) matter far more than the clock hour.
Strengths
Pre-specified inferential machinery that reproduces exactly
Sec. 3.7; Tables 2–4 (pp. 9, 12)
Estimands, SESOIs (5% of the reference blunder rate; 0.0043 score units ≈ 3 Elo), decision labels and two Holm families are stated before the results. Re-deriving SEs from the printed CIs reproduces every p-value, Holm adjustment and decision label of the 4 confirmatory and 13 secondary tests (checks 01, 10), and all 22 stratum p-values and the H6/H7 heterogeneity tests of Table 4 (check 11). Null and contrary results are reported with the same prominence as positive ones.
Informative equivalence result for the headline question
Table 2 (H1); Table 5; Sec. 4.3.1
Instead of reporting a non-significant difference, H1 is framed as a TOST against a +5% SESOI. Every late-night blunder row of Table 5, including the uncontrolled R12 and exploratory X1/X2 specifications, keeps its 90% CI inside ±5% (widest R10b [−1.43, 0.45]; check 03), and no rating band or speed shows a significant late-night increase (check 02, part d). This is the paper's most robust and useful finding.
Within-player design with individually assigned local time
Sec. 3.3, 3.6, 4.1; Fig. 2c–d; Table 5 R2/R2u
Player×speed fixed effects with player-clustered SEs remove the country/player-mix confound of UTC-pooled community analyses [32]; the R2u UTC run (late S−E −0.22) shows how much that matters. The purely activity-based zone inference is validated against flags with a pre-set criterion and is honestly reported as failing (38.8% within 60 min; median error 90 min), which is itself a useful methodological result. The reported error quantiles are mutually consistent (check 09).
Candid robustness reporting, with figures that match tables
Sec. 4.7, 5.3; Table 5; Figs. 3, 6, 7
Specifications that weaken H2 (R4 +1 h −1.25, R10c +1.62, R12 +0.21, X1 −1.67, X2 −0.08) are shown, daggered and discussed rather than hidden; all printed daggers reproduce under the stated agreement rule (check 03). The hour profile in Fig. 3 reproduces the Table 2 contrasts and, when the windows are shifted, the R4 estimates exactly (−3.231 vs −3.23; −1.248 vs −1.25), supporting the paper's explanation of the window sensitivity (check 04). Figures 2, 6 and 7 also agree with the text and Table 5 (checks 12, 16, 17).
Data provenance consistent with public sources
Sec. 3.2, 3.4.1; Table 1; refs [30], [51], [54]
The reported 70,724,764 player-game rows match the expectation from the official Lichess monthly counts for the six months (563,407,871 games × 2/16 = 70,425,984; +0.42%; check 07); the cited Hugging Face mirror exists; the claim that no block contains a European or North American DST transition holds for every IANA zone (check 14); the winning-chances formula and constant 0.00368208 match scalachess (check 08).
Weaknesses
Night S−E deficit (H2) depends on post-outcome session controls
major · Sec. 3.5 (p. 5); Sec. 4.6, Fig. 6; Table 5 X1/X2/R12; Sec. 5.3
The primary model controls for session length, which is known only after the session ends (Sec. 3.5 acknowledges this) and is driven by results: whether a player continues depends on how the current game went. The extracted Fig. 6 coefficients show one-game sessions scoring 29–60 Elo below longer sessions (check 16), i.e. strong outcome-dependent selection. H2 −2.26 shrinks to −1.67 without session length (X1) and to −0.08 without all session/prior-result controls (X2); R12 gives +0.21 (Table 5; check 03). The same conditioning underlies H4b and H5b/H5d, which the abstract then contrasts with clock-hour effects. Using pre-game ratings may also bias S−E after losses (rating lag noted in [33]). Fix: make a model with only pre-game covariates (position so far, gap before the game, previous result) the primary S−E specification or report it beside the pre-registered one, treat session length as a sensitivity, and state in the abstract that the 2.26 deficit is conditional on a post-outcome variable.
Abstract and plain-language summary overstate the tables
major · Abstract (p. 1); Sec. 5.1, 5.5 (pp. 16–17); Plain-language summary (pp. 17–18); Table 4
Confirmed discrepancies (check 02): (i) "deficits of 3.5 to 9.6 Elo, several times the size of any clock-hour contrast" — the ratio to H2 is 1.55× (H5b −3.51) to 4.24× (H5d −9.59), and reported clock-hour contrasts exceed 3.51 Elo (Bullet −4.74, early tertile −5.97, <1200 band −4.13; Table 4). (ii) The plain-language claim that the late S−E deficit "appeared only in very fast games and among players who usually play early" is contradicted by significant deficits in the <1200 (p=0.042), 1200–1599 (0.035), ≥2000 (0.0012) bands and the middle tertile (0.0016). (iii) "about 3 to 13 rating points" (p. 17) is not a reported contrast (13 = H5b + H5d). (iv) Causal wording (check 13): "did not lead to", "what mattered much more", "session effects", "deepened", "a cost of late play", "Hour effects were estimated", despite the paper's own disclaimers (Sec. 1, 5.3). Sec. 5.5 says deficits "follow … a long session", whereas session-length coefficients are positive. (v) "14.1 million games" are player-game rows (Table 1). Fix: rewrite the closing sentences of the abstract and the plain-language summary to match Tables 3–4 and use associational verbs throughout.
Preregistration cannot be verified; adherence only checkable indirectly
major · Title; Sec. 3.1 (p. 3); Sec. 3.8; Data and code availability (p. 17)
Questions for the authors
Can the analysis plan, its hash and the departure log be deposited with a third-party timestamp, and does the item-by-item plan-vs-analysis comparison list any departures beyond D1, D2 and X1/X2?
What exactly were the "annotations shown by Lichess" against which the primary blunder rule agreed on 99.81% of 190,044 plies, given that lila's Advice.scala uses a pure winning-chance ≥0.3 rule? How does H1 look under that rule?
What is H2 in a model that uses only covariates known before each game (position so far, gap before the game, previous results) and omits session length?
Does the rating lag after losses (pre-game rating temporarily too low, as noted in [33]) bias S−E after losses, and do H5b/H5d change when ratings are corrected, e.g. against the player's monthly mean?
How were the 88,500 flag requests chosen and why did they stop below the stated daily limit? Were any outcome data used in choosing them?
How accurate is the within-country zone choice for multi-zone countries, validated on single-zone players, and how do the 2,844 dropped multi-zone players differ in activity timing?
What is the base blunder rate per eligible move in the day window and the number of eligible moves per game?
What do the 8.1% figure and the two sampled-player totals (263,762 vs 255,397) refer to?
Verification
Only the PDF was available, so checks used table arithmetic, vector extraction of Figs. 2, 3, 6 and 7, external sources (official Lichess counts, lila/scalachess source history, IANA tzdata, Crossref) and small simulations. Of 20 checks, 12 confirmed, 5 found discrepancies and 3 were inconclusive. The statistical core is internally consistent: every p-value, Holm adjustment and decision label in Tables 2–4 reproduces, the figures match the tables, and Fig. 3 reproduces the R4 window-shift results exactly. The sample size matches official Lichess counts (+0.42%). Discrepancies concern the abstract and plain-language claims ("several times the size of any clock-hour contrast", "only in very fast games"), causal wording, an unreproducible 8.1% share and two sampled-player totals, and the attribution of the primary blunder rule to Lichess, which the cited source contradicts. The preregistration timestamp, the multi-zone zone assignment and the precision of H1 could not be verified.
Cited (with [54]) for the claim that the primary blunder rule 'reproduces the move judgement that Lichess itself displays' (Sec. 3.4.1, p. 4-5). The cited source does not support it: lila's Advice.scala, both during the data period (Dec 2024-Aug 2025) and today, labels moves purely by winning-chance drop (.3 Blunder, .2 Mistake, .1 Inaccuracy) with no 300-centipawn criterion; the rule the paper rejected (winning-chance drop >= 0.3) is the one in this file (check 08, work/verify/08_lichess_rule.py).
Web and source-code items are given without URLs, access dates, or commit hashes. For [53] and [54] the content changes over time, so the version that defined the outcome cannot be identified from the bibliography. The items themselves exist (checks 07 and 08).
[32] MadMax-5000. chess-best-time (GitHub, 2026); [33] Roman Breytenbikher. chess-tilt (GitHub, 2026); [34] Sean Devine. Tilt and Hype in Online Chess (blog, 2021)
All three exist and support how they are described: [32] is a UTC-only single-month (Feb 2026) analysis with win-rate differences under 2 pp; [33] reports about -7 Elo after a loss, -24 after 5+ losses and about 0 after a 30+ min break, on April 2017 data; [34] analyses about 1M 2014 Lichess games and finds that longer breaks reduce tilt. None is peer reviewed, no URLs are given, and [33] was created on 2026-10-15, under four weeks before the paper's date. They support the 'not new' framing of the loss results (Sec. 2.3, 5.2) but cannot carry evidential weight.
[31] Leone et al. 2017, Cognition 158:44-55
Exists and its abstract supports the replication target (morning slower and more accurate, evening faster and less accurate, no daily change in performance). The specific details attributed to it in Sec. 1-2.1 (94 heavy users of another server, more than 2,000 three-minute games each) are not in the abstract and could not be checked against the full text. Not a confirmed problem; noted because the novelty claim ('the only peer-reviewed within-player study') rests on this characterisation.
Suggestions
Report an S−E model with only pre-game covariates alongside the pre-registered one, and lead the abstract with the unconditional/pre-game result or state clearly that −2.26 is conditional on session length.
Rewrite the last two sentences of the abstract and the plain-language summary so that every comparison matches Tables 3–4 (ratios 1.6–4.2×, deficits also in several rating bands), and replace causal verbs ("lead to", "effects", "cost", "mattered") with associational ones.
Correct the description of the primary blunder rule, cite lila and scalachess by commit hash, and add lila's winning-chance ≥0.3 rule as a robustness specification.
Register the plan and hash publicly with a third-party timestamp, and include the plan-vs-analysis comparison as a supplement.
Publish the research package with a persistent identifier (DOI) so that the aggregate estimates and code can be rerun.
State the analysed population (most-active flagged players, ~6.5% of the sample; ~9% engine-analysed games) in the abstract, and describe players dropped at the flag and zone-matching stages.
Validate the within-country zone choice on single-zone players grouped into artificial multi-zone countries and report the share assigned to the correct zone.
Report the base blunder rate and eligible moves per game, and express the H1 equivalence bound in absolute terms.
Fix the 8.1% statement, reconcile 263,762 vs 255,397 sampled players, describe "14.1 million games" as player-game rows, and align the Fig. 3 caption with the plotted reference level.
Report an across-band homogeneity test for blunder rates and give SEs for the synthetic recovery results.
Ethics and integrity
Authorship: the byline is "IterSCI"; the requester states this is an AI research system that produced the study end to end, but the paper itself does not disclose that it was generated by an AI system or describe human oversight. Venues generally require such disclosure, and it matters here because the preregistration timestamp and the 5 h 16 min plan-to-PDF window can only be interpreted with knowledge of the pipeline. Data: player-level panels and inferred locations are withheld, which is appropriate for linking usernames to inferred time zones; 88,500 public profiles were fetched via the Lichess API, and the paper notes Lichess asks that the user API not be used to download all accounts — a statement of compliance with the API terms would help. No text in the paper attempts to instruct reviewers; the requester's notes were treated as context only.
Scope of this review
Only the uploaded PDF was available: no LaTeX source, analysis code, preregistration file, departure log or aggregate data, although the paper says these are in a research package (p. 17) without a link. No model was re-estimated; checks rely on the printed tables, vector data extracted from the figures, public Lichess counts and source code, tzdata, and reference databases. The hashed player sample, flag file and validation annotations could not be regenerated or examined. The full text of Leone et al. [31] was not checked, only its abstract.
Departures D1/D2 and the post hoc status of X1/X2 are flagged; Sec. 5.3 states plainly that neither the conditional nor the unconditional S−E comparison identifies a causal effect and that the unconditional one is the more natural description; Sec. 5.4 names selection into play, engine-analysis selection and Elo vs Glicko-2 as threats.
The plan is said to be time-stamped and SHA-256 hashed at 2026-11-09 21:28:52 UTC (Sec. 3.1) but is not on a public registry; neither the hash nor a registration could be found, and the plan was not supplied (check 05). A self-computed hash proves integrity only if the hash was published with a third-party timestamp. The PDF was created 5 h 16 min later, a window that must have contained ≥1.84 h of flag requests alone (D1), all estimation, 13+ robustness runs, figures and writing. Permutation tests were run on real data before the plan (Sec. 3.1), so outcomes were loaded pre-registration. D1 changed the analysed population after the plan, and X1/X2 are post hoc. What can be said is that the decision rules as written are applied consistently (checks 01, 03, 10); whether they are the rules that were fixed in advance cannot be checked. Fix: deposit the plan, hash and departure log on OSF/Zenodo (or similar) and publish the item-by-item plan-vs-analysis comparison that Sec. 'Data and code availability' says exists; avoid 'pre-registered' in the title unless an external registration exists.
Primary blunder rule is not the rule Lichess uses
major · Sec. 1 (p. 2); Sec. 3.4.1 (pp. 4–5); refs [53], [54]
The paper claims (Sec. 1, 3.4.1) to measure blunders "with the move-judgement rule that Lichess itself uses" — winning-chance drop ≥0.1 plus capped cp loss ≥300 — citing lila Advice.scala [53]. That file, both at commits in force during Dec 2024–Aug 2025 and today, labels moves purely by winning-chance drop (.3 Blunder, .2 Mistake, .1 Inaccuracy) with no 300-cp criterion (check 08). The WC ≥0.3 rule the paper rejected for 93.28% agreement is the one in the cited source; on a grid of (eval, loss) pairs the two rules disagree on 5,931 of 40,401 cells. The 99.81% agreement with "annotations shown by Lichess" therefore needs an explanation of where those annotations came from. H1 likely survives (it held for four alternative definitions), but this is one of the five claimed contributions and defines the primary outcome. Fix: correct the attribution, describe the validation annotations, and add lila's actual WC ≥0.3 rule as a Table 5 robustness row.
Highly selected population and unvalidated zone choice in multi-zone countries
major · Sec. 3.3, 3.8.1 (D1), 4.1; Table 1; Table 5 R3
After D1, flags were requested only for the 88,500 most active of 263,762 sampled players; 17,084 players (6.5%) enter the analysis (check 06). 2,844 of 9,184 multi-zone players were dropped because their activity was "too far from every zone" — plausibly the atypical-timing players most relevant to late-night play. Multi-zone players contribute 34.7% of local-time rows and are placed by the same activity matching that Sec. 3.3 argues absorbs night-owl signal; under the reported error distribution a simple US-like simulation places only about 45% of assigned players in the correct zone (assumption-dependent; check 09). R3 (single-zone only) protects H1/H2 but the evening S−E contrast drops to 0.15 [−0.71, 1.01]. Only ~9% of games are engine-analysed, at the player's request (check 17). Title and abstract present results for Lichess players without these qualifiers. Fix: state the population (heavy, flagged, analysis-requesting players) in the abstract; validate within-country zone choice on single-zone players with artificial multi-zone sets; report characteristics of dropped players.
Internal inconsistencies in sample counts
minor · Sec. 3.8.1, 4.1 (p. 6), 4.7; Table 5 R2u
"8.1% of the sampled players received a usable local time" does not reproduce: 17,084/263,762 = 6.48% and 17,084/255,397 = 6.69%; 8.1% matches players with any flag (20,750/255,397 = 8.12%), which includes 3,666 multi-zone players without a local time (check 06). Two different sampled-player totals appear (263,762 in Sec. 3.8.1/4.1 vs 255,397 in Sec. 4.7/Table 5 R2u). D1 says requests "stopped when the daily limit was reached", yet 88,500 is 74% of the stated ~120,000/day limit (check 05).
Base blunder rate never reported
minor · Sec. 3.4.1; Tables 2–5
All blunder results are percentages of an unstated day-window rate. Whether the H1 SE (0.42% of the day rate) is plausible depends on the base rate and eligible moves per game: it is below the binomial floor at a 1% per-move rate and above it at 4% (check 20). Readers cannot translate the ±5% bound into absolute blunders per game. Fix: report the base rate, eligible moves per game, and the absolute-scale equivalence bound.
Evening contrasts described as steadier than they are; Fig. 3 caption
minor · Sec. 4.3.3, 4.7, 5.2; Table 5; Fig. 3 caption (p. 8)
The evening S−E contrast fails the paper's own agreement rule under R3 (0.15), R4 −1 h (0.88) and R12 (2.52), and evening blunder contrasts are significantly positive under R8 low-time exclusion (0.81 [0.22, 1.40]), R8 >200 cp (0.47 [0.03, 0.92]) and R12 (0.65 [0.08, 1.21]) — small relative to the SESOI, but not mentioned where the text says the prediction of more evening errors did not hold (check 03). Fig. 3 panels are centred on the 24-h mean, not the day-window mean as the caption says; relative to the day window, move time at 04–06 h is +2.6–2.7%, not 1.5–1.8% (check 04).
Pipeline validation has little power at the SESOI scale
minor · Sec. 3.1 (p. 3)
The 180 permutation tests (11/180 and 4/180 rejections; GRIM-consistent, check 18) have a 95% CI for the 5%-level rejection rate of [3.1%, 10.7%], so up to twofold SE under-coverage would go undetected. Planted effects (20%, ≈20.8 Elo) are 4–7× the SESOIs and the +20% effect was recovered as 17.7% (−11.5% attenuation) without an SE or discussion of what this implies for the equivalence decision.
Nothing can be rerun; web and code references lack URLs and versions
minor · Data and code availability (p. 17); References (pp. 19–21)
Code, plan, departure log and aggregate estimates are described as being in a "research package" (p. 17) with no URL or DOI, so no model could be re-estimated. References [30], [32]–[34], [51]–[54], [56] are web or code items without URLs, access dates or commit hashes; for [53]/[54] the content changes over time. [32]–[34] are non-peer-reviewed, and [33] was created on 2026-10-15, under four weeks before the paper (check 19).
Smaller analytic caveats
minor · Sec. 3.4.2, 4.5; Table 2; Table 4
The Elo-equivalent conversion (×694.9) is exact only at E = 0.5; at E = 0.25 a 3-Elo shift moves expected score by 0.00325, not 0.00432 (check 15), so Elo-equivalents understate shifts for mismatched pairings. The claim that the blunder contrast "did not vary systematically with rating" rests on the extreme-band difference H6a only; an across-band homogeneity test (not reported) gives χ² = 7.42 on 3 df, p = 0.06 (check 11). The H2 "magnitude uncertain" label depends on the 90% CI lower bound exceeding the SESOI by only 0.10–0.12 Elo (check 01).
All 13 Holm p-values reproduce (H7b 2.60e-4 vs 2.6e-4; H5e 4.59e-4 vs 4.3e-4; H5f 0.00956 vs 0.010; H5c 0.0186 vs 0.019; H4a 0.0538 vs 0.052; H6a/H6b/H7a 1 vs 1.000; chi2 p H7a=0.518, H7b=3.25e-5) and all 11 decision labels match. Note H4a unadjusted p=0.013 is labelled equivalent (90% CI [-1.65,-0.33] inside +/-5%), and the three meaningful labels are all session/prior-result S-E contrasts (|est| 3.51-9.59 Elo; H5b 90% CI [-3.98,-3.04] lies just beyond the 3-Elo SESOI).
SEs backed out of the printed 95% CIs (normal approx.), two-sided p; chi2(3) survival for H7a/H7b; Holm over 13 tests; Sec. 3.7 rules re-applied with 90% CIs against SESOI (5% blunder, 3 Elo S-E). Tolerance: Holm p within 20% relative or 0.004 absolute; decisions must match exactly
All 22 stratum p-values reproduce (e.g. Bullet S-E 2.31e-9 vs 2.3e-9; Middle S-E 0.00157 vs 0.002). H6a -0.30 [-4.79, 4.19] and H6b -1.03 [-5.44, 3.38] reproduce exactly from independent strata. Independent-strata Wald tests closely match: tertile S-E 15.82 (p=3.7e-4) vs 15.76; tertile blunder 0.66 (p=0.72) vs 0.65; speeds 2.19 vs 2.27 and 22.99 vs 23.45 (small gaps consistent with the joint covariance). Pooled strata agree with main estimates (bands S-E -2.28 vs H2 -2.26; speeds blunder -0.23 vs H1 -0.20). Side observation: an across-band homogeneity test for blunder rates (not reported) gives chi2 = 7.42 on 3 df, p = 0.06, driven by 1200-1599 (-2.08%) vs >=2000 (+0.97%); the text statement that the blunder contrast "did not vary systematically with rating" rests only on the extreme-band difference H6a.
Normal-approximation p from printed 95% CIs (tolerance 25% relative or 0.006 absolute); differences and Wald homogeneity tests computed assuming independent strata; precision-weighted pooling compared with H1/H2
Figure 7 robustness forest plot shows the same late-night estimates and 95% CIs as Table 5
All 28 plotted rows (16 blunder, 12 S-E) match Table 5 to within 0.005 units for estimate and both CI limits (e.g. R4 +1 h S-E -1.25 [-2.11, -0.39]; R12 S-E 0.21 [-0.78, 1.20]; R10c 1.62 [-1.37, 4.61]). The figure contains only late-night contrasts and omits the exploratory X1/X2 rows; Sec. 5.1 cites Figures 5 and 7 for the deficit disappearing without controls, which Figure 7 shows only via R12 (X2 appears in Table 5 alone).
PyMuPDF vector extraction of the 28 markers and CI segments; x calibrated linearly from each panel tick labels (5 ticks); rows matched to Table 5 by label; tolerance 0.05 units
The study reports within-player associations and avoids causal claims ("not effects of being made to play at a given hour", Sec. 1; "neither identifies a causal effect", Sec. 5.3)
The main text is mostly careful: explicit disclaimers in Sec. 1 (p. 2), Sec. 5.3 and 5.4 (p. 16) and the plain-language summary ("not proof of cause", p. 18); "associated with" appears 8 times. But causal or directional wording remains where readers look first: the plain-language summary says "Late-night play did not lead to more big mistakes" and "What mattered much more was what happened within a session" (p. 17); Sec. 5.1 says "The session effects were larger than any clock effect" and that each consecutive loss "deepened" the deficit (p. 16); Sec. 5.5 speaks of "a cost of late play" and gives advice to players (p. 17); the abstract says "Hour effects were estimated" (p. 1). Sec. 5.5 also says the deficits "follow ... a long session", whereas Figure 6/Sec. 4.6 report positive session-length coefficients (+29 to +60 Elo); the deficit is tied to position within a session. Session length and prior results are post-outcome/self-selected controls (Sec. 3.5 p. 5), which makes causal phrasing about them weaker still. Severity: minor-to-moderate wording issue, not a computational error.
Regex scan of work/paper.txt (before References) for causal vocabulary; 56 hits printed with page and context and classified by the reviewer
"No block contains a daylight-saving transition in Europe or North America" (blocks Dec 2024-Feb 2025 and Jun-Aug 2025)
0 Europe/* or America/* zones change offset in either block. Across all regions, the only changes are Africa/Casablanca and Africa/El_Aaiun (Morocco Ramadan suspension, 2025-02-23 02:00 UTC, +01 to +00), inside the winter block; the paper limits its claim to Europe and North America and applies time zones game by game (Sec. 3.3), so this does not contradict it.
Hourly scan of UTC offsets with Python zoneinfo (IANA tzdata 2025b) for every Europe/, America/, Asia/, Australia/, Africa/, Pacific/, Atlantic/, Antarctica/ zone over both blocks
Elo-equivalents: score units x 1600/ln 10 = 694.9; SESOI 0.0043 score units = 3 Elo; 2.26 Elo is about 0.003 expected score
dE/dd at 0 = ln10/1600, so 1/slope = 694.871 (paper 694.9). 0.0043 x 694.9 = 2.988 Elo (paper: 3); 2.26/694.9 = 0.00325 score units (paper: about 0.003). The conversion is exact only at E = 0.5: for a typical mismatched pairing at E = 0.25 a 3-Elo change moves expected score by 0.00325, not 0.00432, so Elo-equivalents understate rating shifts away from even pairings. This is a minor scaling caveat, not an error.
sympy derivative of E(d)=1/(1+10^(-d/400)) at d=0; numeric conversions; exact vs linearised change for a 3-Elo shift at E=0.5 and E=0.25
Sec. 4.6/Figure 6: S-E fell 3.2 Elo at game 3 and about 9 Elo at games 4-10 vs game 2; session-length coefficients 29 to 60 Elo vs one-game sessions; blunder coefficients within about 8% of the day rate; consistent with Table 3
Game 3 -3.20; games 4-5 -8.65 and 6-10 -9.46; session-length coefficients 29.0, 49.7, 57.5, 60.1, 59.2 Elo; max |blunder coefficient| 8.08% (Session 21+). Table 3 is reproduced exactly from the plotted values: H4b -9.46, H5b -3.51, H5d (4+ losses minus 1 loss) -9.59, H4a -0.99, H5a 2.66, H5c 2.19, H5f (Break-previous-loss minus After-1-loss) 1.49, H5e -1.67. Substantive point for the review: one-game sessions score 29-60 Elo below longer sessions, and within-session gaps of 2-30 min carry -12.2 Elo vs gaps under 2 min. These are outcome-dependent controls (whether a player continues depends on results), so the main model conditions on post-outcome variables; dropping session length (X1) moves H2 from -2.26 to -1.67, and dropping all session/prior-result controls (X2) to -0.08 (Table 5).
PyMuPDF extraction of 46 markers and CI segments, x calibrated from tick labels; Table 3 contrasts rebuilt from the plotted coefficients; tolerance 0.05 units
Fewest games at 05:00-05:59 (118,783), most at 20:00-20:59 (1,025,234); engine-analysed share about 8% at 03:00-04:59 vs about 9.3% during the day
Summed shares give 100.18% (calibration bias 0.18%). Minimum at hour 5: 0.852% = 119,814 games vs 118,783 (+0.9%); maximum at hour 20: 7.295% = 1,026,265 vs 1,025,234 (+0.1%). Engine share averages 8.00% at 03-04 h and 9.25% at 10-17 h (paper: about 8% and about 9.3%); the overall 1,289,834/14,067,563 = 9.17%. The raw analysed share is 1.25 points lower at night, whereas within players R10a gives +0.12 pp [0.04, 0.21] for the late window (Table 5), so the raw drop reflects who plays at night; blunder outcomes in any case come from a player-requested subset (about 9%), which is relevant to H1.
PyMuPDF extraction of the four per-speed polylines (Fig. 2a) and the engine-share polyline (Fig. 2b), y calibrated from 8 tick labels each; shares summed over speeds and multiplied by 14,067,563; tolerance 2%
Pre-plan validation: of 180 permutation tests, 6.1% had p < 0.05 and 2.2% p < 0.01 ("close to their nominal levels"); planted +20% late blunder effect recovered as 17.7%, planted S-E deficit 0.030 as 0.0317 on 621,514 synthetic player-games
6.1% and 2.2% correspond uniquely to 11/180 and 4/180 (GRIM-consistent). Against nominal levels: 11 vs 9.0 expected, exact p = 0.491; 4 vs 1.8 expected, p = 0.108, so "close to nominal" is supported. The check has low power, however: the 95% CI for the 5%-level rejection rate is [3.1%, 10.7%], so SE under-coverage up to about twofold would go undetected. Recovery ratios are 0.885 (-11.5%) for blunders and 1.057 (+5.7%) for S-E. No SE is reported for these, and the planted effects (20%, about 20.8 Elo) are 4-7 times the SESOIs, so the validation says little about bias at the scale of the equivalence decisions. The validation code (analysis/preregistration.md is said to be distributed with it) was not available to inspect.
GRIM-style integer check, exact binomial tests against 5% and 1%, recovery ratios
References exist and support the claims attributed to them (notably [31] Leone et al. 2017, [35] Gee et al., [20] Eyolfson et al., [24] Sunde et al. 2026, [11] Weinshall-Margel & Shapard; community analyses [32]-[34]; Lichess sources [51]-[56])
50/60 references match a Crossref record on title and year (e.g. [31] 10.1016/j.cognition.2016.10.007, [35] 10.1515/jqas-2025-0035, [24] 10.1073/pnas.2531472123, [42] PsyArXiv cdk59). The 10 unmatched entries are web or code items, all located: [30]/[51] (check 07), [53]/[54] (check 08), [32] GitHub MadMax-5000/chess-best-time, [33] GitHub romanbreyten/chess-tilt, [34] seandevine.org blog, [47] arXiv 2605.04371 (Della Negra, Samory, Cinelli, 2026-05-06). No fabricated reference was found. Content spot-checks: [31] abstract supports morning-slow/accurate vs evening-fast/less-accurate with no performance change; [35] abstract reports little evidence of consistent winner-loser effects; [32]-[34] match their descriptions. The discrepancy: [53] (lila Advice.scala) is cited for a hybrid 300-cp blunder rule that it does not contain (check 08). Web and code items lack URLs and versions. See work/ref_issues.json.
tools/check_refs.py (Crossref bibliographic query) for all 60 entries, summarised by title-token overlap and year (verify/19_refs_summary.py); spot-checks of abstracts and READMEs via PubMed, Semantic Scholar, arXiv, GitHub API and the blog page (verify/19_ref_spotcheck.py)
Precision of the main contrasts is plausible: SE of H2 about 0.51 Elo with 14.07M games; SE of H1 about 0.42% of the day rate with 1.29M analysed games
H2: reported SE 0.513 Elo is 1.38-1.54 times the i.i.d. floor (0.333-0.372 Elo), a plausible clustering design effect. H1: reported SE 0.421% is below the binomial floor if the per-move blunder rate is 1% (floor 0.56-0.70%) or 2% with 25 moves (0.495%), and above it at 2%/40 moves or 4% (0.27-0.39%). The verdict therefore depends on the base blunder rate, which the paper never reports (all blunder results are percentages of an unstated day-window rate; Sec. 3.4.1, Tables 2-5). Reporting the base rate and eligible moves per game is needed to judge the precision behind the H1 equivalence claim.
Analytic i.i.d. floor for the unweighted mean-of-hour-coefficients contrast (late weights +1/5, day weights -1/8), hourly counts from Figure 2a (check 17); S-E per-game variance 0.20-0.25; for H1 a binomial floor over a grid of assumed blunder rates (1-4% per eligible move) and 25-40 eligible moves per game
Recomputed p: H1 0.635 (rep 0.640), H2 1.05e-5 (1.0e-5), H3a 0.225 (0.220; 0.2206-0.2288 within CI rounding), H3b 3.35e-6 (3.2e-6). Holm from reported p: 1.28e-5, 3.0e-5, 0.440, 0.640 vs reported 1.3e-5, 3.0e-5, 0.441, 0.640. All four decision labels reproduced. H2 90% CI = [-3.10,-1.42]: the magnitude-uncertain label rests on the lower bound exceeding the SESOI by only 0.10-0.12 Elo.
SE backed out of each 95% CI (normal), two-sided p, Holm step-down over 4 tests, 90% CI and re-application of the Sec. 3.7 decision rules with SESOI 5% / 3 Elo (and 0.0043x694.9=2.988 Elo)
Abstract/plain-language summary match results: session/loss deficits of 3.5 to 9.6 Elo are "several times the size of any clock-hour contrast"; plain-language "about 3 to 13 rating points"; late S-E deficit "appeared only in very fast games and among players who usually play early"; no rating band or speed showed a significant late-night blunder increase
Ratio of session deficits to the main clock contrast (H2 -2.26) is 1.55x (H5b -3.51) to 4.24x (H5d -9.59), so "several times" holds only for the upper end. Clock-hour contrasts larger than 3.51 Elo are reported: Bullet -4.74, early tertile -5.97, <1200 band -4.13 (Table 4), so "any clock-hour contrast" is false as worded. Late-night S-E deficits are significant (recomputed p<0.05) not only in Bullet (p=2.3e-9) and the early tertile (2.2e-6) but also in <1200 (0.042), 1200-1599 (0.035), >=2000 (0.0012) and the middle tertile (0.0016), contradicting "appeared only in very fast games and among players who usually play early" (plain-language summary). "13 rating points" is not a reported contrast; it equals H5b+H5d = 13.10 (cumulative, 4+ losses vs a win). Claim (d) holds: no blunder stratum shows a significant positive contrast.
Table arithmetic over transcribed Table 2-5 values; ratios of session to clock contrasts; p-values of Table 4 strata recomputed from CIs
Table 5 daggers mark failures of the pre-registered agreement rule; late S-E agreed under R3, R5, R6, R7, R9, R13; shrank by almost half with +1 h and vanished without controls; late blunder equivalence held under every specification; evening contrasts "steadier"
All printed daggers (R4 +1 h S-E, R10c S-E, R12 blunder and S-E) reproduce, no missing daggers among pre-registered late-night cells. Recomputed agreeing specs for late S-E: R3, R4 -1 h, R5 (both), R6, R7, R9, R13 (R4 -1 h also agrees but is described separately). Every late-blunder row (incl. R12, X1, X2) passes TOST (widest 90% CI R10b [-1.43,0.45]). R4 +1 h shrinks H2 by 45% ("almost half" ok); X2 removes 96% and R12 109% of the deficit. Caveat: the evening S-E contrast fails the same agreement rule under R3 (0.15 [-0.71,1.01]), R4 -1 h (0.88 [0.23,1.53]) and R12 (2.52 [1.83,3.21]), and evening blunder under R8 mistake (-0.19), so describing the evening contrasts as "steadier" (Sec. 4.7) is generous; no daggers are given for evening columns by design.
Re-applied the agreement rule (same sign and 95% CI contains main estimate) to all 19 rows x 4 columns; TOST (90% CI inside +-5%) on every late-blunder row; percentage change vs main for R4, X1, X2, R12
Figure 3 hour profiles agree with Table 2 window contrasts and Sec. 4.2 text (S-E trough ~-3.3 Elo at 04-06 h, peak ~+2 at 18 h; best-hour probabilities 32/17/13% and 36/24/12%); R4 shift sensitivity follows from the Fig. 3b profile
From the plotted profiles: late-day blunder -0.197 (Table 2 -0.20), late-day S-E -2.265 (-2.26), evening-morning blunder +0.354 (0.35), S-E +1.658 (1.66), move time -1.119 (H3c -1.10). Shifting the windows reproduces R4 exactly: hours 1-5 vs 11-18 = -3.231 (R4 -1 h: -3.23), hours 23-3 vs 9-16 = -1.248 (R4 +1 h: -1.25), supporting the post hoc explanation. S-E at 4/5/6 h = -3.35/-3.25/-3.42 vs 24-h mean (text ~-3.3); 18 h +1.99 (text ~+2). Best-hour bars: 0 h 0.322, 6 h 0.167, 3 h 0.133; 18 h 0.364, 19 h 0.242, 13 h 0.119 - all match. Minor: all four panels are centred on the 24-h mean (means 0.00), whereas the caption says (a) and (c) are relative to the day-window mean; relative to the day window, move time at 04-06 h is +2.6-2.7%, not 1.5-1.8%.
Extracted 24-point polylines of panels (a)-(d) and 48 bars of (e)-(f) from PDF vectors (PyMuPDF), calibrated y-axes from tick labels (residual 0), recomputed unweighted window means per Sec. 3.6 and shifted windows
Plan time-stamped and SHA-256 hashed 2026-11-09 21:28:52 UTC before any outcome; departures logged; flags requested for 88,500 players under 8,000/10 min and ~120,000/day limits, requests stopped when the daily limit was reached
Hash is a well-formed 64-hex SHA-256 string, but neither the hash nor any public registration (OSF etc.) could be found; the plan file was not supplied, so item-by-item adherence cannot be checked. PDF CreationDate 2026-11-10 02:45:20 UTC is 5 h 16 min after the stated hash time; D1 flag requests alone need >=110.6 min (1.84 h) at 8,000/10 min, so the whole pipeline (flags, 13+ robustness models, figures, writing) ran in ~5.3 h - feasible only for an automated pipeline, and the timestamp is self-asserted. Internal inconsistency: 88,500 requests is 0.74 of the stated ~120,000/day limit, so "requests stopped when the daily limit was reached" cannot explain the cut at 88,500 (33.6% of 263,762 sampled players).
Hash format check; interval from plan hash to PDF CreationDate; minimum API time for D1; consistency of the daily-limit statement; web search for the hash and for a public registration (log in work/verify/05_websearch.txt)
Table 1 sample flow (70,724,764 - 4 exclusions = 68,899,577; 14,067,563 local-time rows, 9,189,831 single-zone); 9,184 multi-zone flagged players = 5,518 matched (60%) + 2,844 too far + 822 too little; "8.1% of the sampled players received a usable local time"
Exact: 68,899,577 and 9,184 reproduce; matched share 60.1%. Multi-zone (activity-assigned) rows = 4,877,732 = 34.67% of local-time rows; 32.3% of local-time players. Engine share 9.17% ("about 9%" ok). But "8.1% received a usable local time" does not reproduce: 17,084/263,762 = 6.48%, 17,084/255,397 = 6.69%; 8.1% is matched only by players with any flag (17,084 - 5,518 + 9,184 = 20,750)/255,397 = 8.12%, which includes 3,666 multi-zone players who did not receive a local time (likely explanation, not confirmable without code). Two sampled-player totals are reported: 263,762 (Sec. 3.8.1, 4.1) vs 255,397 (Sec. 4.7, Table 5 R2u), differing by 8,365 (3.2%), without explanation.
Recomputed sums, shares and candidate numerators/denominators for 8.1%
Six monthly Lichess standard rated dumps (Dec 2024-Feb 2025, Jun-Aug 2025) read from a public mirror [51]; 1/16 hashed player sample gives 70,724,764 player-game rows
Official counts sum to 563,407,871 games; expected rows 70,425,984 vs reported 70,724,764 (+0.42%, z = +0.4 to +1.0 under CV 2-6 of games per player; tolerance 2%). The mirror Lichess/standard-chess-games exists on Hugging Face (CC0). Implied ~4.22M distinct players. The sample size is consistent with the stated provenance; the hashing salt and code were not available, so the sample itself cannot be regenerated.
Fetched official monthly counts from database.lichess.org/standard/counts.txt (saved to work/verify/07_counts.json); expected rows = total games x 2/16; sampling SD bracketed under assumed dispersion of games per player; queried the HF API for the mirror
Primary blunder rule reproduces the move judgement Lichess displays [53,54]: winning chances 2/(1+exp(-0.00368208 cp))-1; flagged if WC drop >= 0.1, blunder if capped cp loss >= 300; WC-only rule (drop >= 0.3) agreed on only 93.28% of plies and was not used
The multiplier -0.00368208 and the winning-chances formula match scalachess eval.scala (line 108-109). But lila Advice.scala, both today and during Dec 2024-Aug 2025, labels moves purely by winning-chance drop: .3 -> Blunder, .2 -> Mistake, .1 -> Inaccuracy; there is no 300-cp criterion (git history: winning-chance judgements since 2019, commit 133a1ebe76). So the rule the paper calls "the move-judgement rule that Lichess itself uses" is not the rule in the cited source; the rule the paper rejected (WC >= 0.3) is the one in lila. On a 40,401-point grid the two rules disagree on 5,931 pairs. The paper reports 99.81% empirical agreement of its hybrid rule with "annotations shown by Lichess"; the source of those annotations is not described and could not be examined, so the empirical claim itself is unverified. The H1 outcome definition is still a reasonable large-error rule; the issue is the label and the citation.
Fetched lila modules/tree/src/main/Advice.scala at master and at commits in force during the data period (0a7a690d87, 2024-12-22; bddf2f77e5, 2025-07-25) and scalachess core eval.scala; printed the judgement code; compared the two rules on a grid of (eval, cp loss) pairs
Activity patterns were used only to choose among zones of a known country, "where 51.3% of test-half blocks fell within the 90-minute tolerance"; test-half errors 23.6%/38.8%/61.6% within 30/60/120 min, median 90 min
Quantiles are mutually consistent (monotone; median 90 min vs 51.3% within 90). Under the stated assumptions the model assigns 69.7% of players (paper: 60.1% matched, 31.0% too far) but only 45.5% of assigned players receive the correct zone (Central 30%, Mountain 30%, Eastern 57%, Pacific 50%). This is assumption-dependent: the paper never validates the constrained within-country choice, and 51.3% within the 90-min tolerance is not an accuracy of zone choice. Multi-zone players supply 34.7% of local-time rows (check 06), so misassignment could be material; R3 (single-zone only) is the relevant safeguard (H2 -2.28, H1 -0.36; but evening S-E 0.15).
Consistency of reported quantiles; seeded Monte Carlo (2M draws) applying the reported error distribution to a US-like 4-zone country (60-min spacing, population weights) with nearest-zone choice and the 90-min tolerance