Summary
The paper links every FARS death in the contiguous US for 2000–2024 to county daily maximum temperature from nClimGrid-Daily, giving a panel of 28.4 million county-days. It fits PPML models with county×year×month fixed effects and Tmax bins at lags 0–3, with standard errors clustered by state and by date. Separate models cover pedestrians, cyclists and vehicle occupants. The confirmatory contrast is the cumulative RR for Tmax ≥90 °F versus 60–70 °F. That RR is 1.010 for pedestrians, 1.143 for cyclists (Holm p = 0.093) and 1.032 for occupants. None of the six confirmatory tests is rejected, and there is no evidence of age heterogeneity.
The pre-specified falsification test, which uses next-day Tmax, is strongly positive in all three groups. The authors attribute this to nClimGrid-Daily labelling each grid with the date of the morning observation. They then add a post hoc 'aligned' window, which gives larger associations (cyclists 1.409, pedestrians 1.120). Further material covers attributable-death estimates, an activity-volume bound, 19 sensitivity analyses and subgroup breakdowns. The study describes itself as pre-registered against a self-time-stamped plan, but that plan is not public.
Strengths
Headline estimates reproduce from public data
Sec. 2.2–2.5; Tables 1, 2, 5, 6, 8 (S3), 11
We rebuilt the panel independently from the FARS CSVs and the nClimGrid-Daily county files and wrote our own PPML estimator. This reproduced the published numbers to within 0.003 on the RR scale, CIs included. Cyclists gave 1.142 (1.025, 1.272) against the paper's 1.143 (1.026, 1.273), and pedestrians gave 1.011 against 1.010. The S3 and aligned models and every lag RR also matched. The panel size (28,372,440), the ≥90 °F county-day count (2,884,882) and the pedestrian and cyclist death totals matched exactly. Sections 2.2–2.5 are therefore described precisely enough to replicate (checks 02, 03, 05).
Transparent reporting of null and failed tests
Abstract; Sec. 2.8, 3.2, 3.6; Table 2; Table 8
All six confirmatory tests are reported with raw and Holm-adjusted p-values, and the recomputed values agree (check 01). The failed falsification test appears in the abstract, Sec. 3.6 and Table 8 and is not buried. The aligned window is labelled post hoc (Sec. 2.8, Table 11 note). Such candour about an ambiguous confirmatory family is uncommon.
Appropriate core design for weather–crash panels
Sec. 2.5–2.6, Eq. (1); Table 2
The model uses county×year×month fixed effects in PPML with distributed lags and precipitation controls, following Leard & Roth. An occupant comparator from the same data system and a joint covariance for the rate-ratio ratios (RRRs) are sensible choices. The RRR point values and their CIs are internally coherent (check 10).
Careful handling of per-trip versus population risk
Sec. 3.8, 4.3; Table 10
The activity bound in Table 10 and the discussion in Sec. 4.3 show clearly how much exposure change would explain each RR. All 18 entries recompute exactly (check 14), and the note states that the volume change is treated as known.
Internally consistent arithmetic throughout
Tables 2–11; Fig. 6
We parsed 326 RR (CI) triplets and found all of them log-symmetric within rounding (check 12). Cumulative RRs equal the products of the lag RRs (check 06). All Table 8 consistency flags follow the stated rule (check 07). The BH q-values reproduce (check 08), and the attributable-death identities hold (check 13).
Weaknesses
Abstract and conclusions state a cyclist effect the confirmatory analysis does not support
major · Abstract (last sentence); Sec. 4.1; Sec. 4.6; Table 2; Table 8 (S3)
The abstract closes with 'Hot days were associated with a modest rise in cyclist deaths', and Sec. 4.6 says the hottest days 'carried a modest excess of cyclist deaths'. The pre-registered test H1b was not rejected (Holm p = 0.093, which we confirm in check 01). Adding the next-day term (S3) gives a cyclist cumulative RR of 0.976 (0.871–1.092); our rebuild gives 0.975 (0.869, 1.093). Only the post hoc aligned model is clearly positive. Sec. 4.1 says the estimate is 'stable across most pre-specified checks', but only 16 of 24 contrasts exclude 1 (check 07). The plain-language summary ('suggestive rather than firm') is better calibrated than the abstract. Fix: in the abstract and Sec. 4.6, state that no confirmatory test was rejected and that the cyclist association is not robust to the falsification model. Present the aligned results as hypothesis-generating. (Check 16, discrepancy.)
The confirmatory exposure window is mistimed by one day
major · Sec. 2.4, 2.8, 4.4; Table 6; Table 8 (S3); Table 11
We compared nClimGrid-Daily with calendar-day ASOS station Tmax at 8 stations. Label d+1 tracks calendar day d best at all 8: correlations of daily changes are 0.76–0.93 at d+1 and −0.02 to 0.50 at d (check 04). The pre-registered 'lag 0–3' window (labels d…d−3) therefore effectively measures lags 1–4 and never covers the afternoon of the crash day. The S3 'falsification' term is in fact the same-day exposure. All six confirmatory tests, the attributable deaths in Table 9 and the abstract headlines, including 'no detectable change in pedestrian deaths', all rest on this mistimed contrast. The pedestrian cumulative RR of 1.010 is itself a lag-0 excess (1.126) offset by later deficits (check 06). Durre et al. [34] document the dating, so it could have been caught at the planning stage. Fix: say plainly that the confirmatory estimates answer a different question from the one posed. Drop or qualify 'no detectable change in pedestrian deaths' (the aligned RR is 1.120, 1.062–1.181). Plan a confirmatory test of the aligned window on independent data or a held-out period.
'Pre-registered' cannot be verified and is overstated
major · Title; Abstract; Sec. 1 (last para); Sec. 2.1; Sec. 2.9; Sec. 5
Neither the plan, its digest nor the deviation log was available, and no registry or repository link is given. Sec. 2.1 says the plan was not lodged publicly. The plan timestamp is 03:52:57 UTC on 10 Nov 2026 and the PDF was created at 09:16:55 UTC the same day, 5 h 24 min later (check 15). Every download, the build of the 28M-row panel, every model fit and the write-up would have to fit into that window. A self-recorded timestamp is not evidence that the plan predates the data. Sec. 2.9 reports 'seventeen departures' but describes only 10. Two of those, the purpose-written estimator with a modified two-way VCV and the stacked comparisons from separate fits, bear directly on confirmatory tests, which conflicts with 'None of these departures changed a confirmatory test'. Fix: remove 'Pre-registered' from the title, or deposit the plan and digest with a third party and publish the full deviation log. Describe all 17 deviations.
Northeast pedestrian CI is about half the width the described method gives
major · Table 3 (Pedestrians: region); Fig. 1; Sec. 3.5
Table 3 reports Northeast pedestrians at 0.862 (0.833–0.891). The log-SE of 0.0172 is smaller than the national SE of 0.0222, which comes from 7× as many deaths, and is 0.29× the √N expectation. Our independent regional refit reproduces the point estimate (0.858) but gives SE 0.0348 and CI (0.802, 0.919), 2.02× wider. For the other seven regional estimates the ratio is only 1.02–1.21. The Northeast cyclist CI also widens to (0.984, 1.488) and then includes 1. With rebuilt SEs, the pedestrian region heterogeneity χ² falls from about 36.6 to 17.5. It remains significant, but the 'most significant secondary result' is overstated. The likely cause is the small-sample correction with only 9 clusters. Fix: recompute the regional SEs with region-specific cluster counts, use a wild-cluster bootstrap with few clusters, and report the regional results with that caveat. (Check 09, discrepancy.)
Causal and mechanistic language exceeds the design
major · Sec. 2.6; Sec. 3.7; Table 9; Table 11; Sec. 4.4 (para 2); Sec. 6
'Deaths attributable to days with Tmax ≥90 °F' (Sec. 2.6, 3.7, Table 9, Table 11) puts counts on non-significant and mistimed RRs; for example, 146 pedestrian deaths come from an RR of 1.010 (0.967–1.055). Sec. 4.4 states 'heat acts on occupants mainly through daytime driving … and on pedestrians through night-time exposure'. That claim rests on exploratory, post hoc subgroup contrasts, and the pre-registered pedestrian day/night q is 0.271. The plain-language summary ends with a policy recommendation. Fix: use 'excess deaths associated with' and report attributable counts only with their intervals and the falsification caveat. Recast Sec. 4.4 as hypotheses.
Claimed code, plan and person-level data not supplied
minor · Sec. 2.5; Sec. 5
Sec. 5 states that all code, the plan and the person-level file are 'distributed with this article' and that running the code 'reproduces all reported numbers'. None of them came with the submission. The purpose-written PPML estimator is said to be validated against an established package to about 1e-9, but no evidence is shown. Our reproduction succeeded only because we wrote our own estimator and had to assume the precipitation cut-points and holiday definitions. Fix: release the materials and state the precipitation bins and holiday periods in Sec. 2.5.
Several references are misdescribed
minor · Sec. 1 (paras 2–3); Sec. 4.2; References [16], [18], [21], [24], [26], [44]
[26] is said not to state its data or design in its abstract, but the abstract does state them (FARS 2016–2022, state-day panels, GHCN-Daily). It also reports no cyclist result, which matters for the novelty claim. [21] did not find bicyclists to have the 'largest' increase (another category had 3.77), and it highlights steeper night-time risk, not stronger daytime effects. [16] covered all fatal crashes, not only drivers. [44] does not report that cyclist deaths rise with temperature. [18] has a single author but is cited 'et al.'. The [24] pedestrian claim could not be verified (check 17, discrepancy).
Absence of evidence on age presented as evidence of absence
minor · Abstract; Table 3; Table 4; Sec. 4.2
The abstract says 'RRs did not differ' by age and concludes 'no age group at distinctly higher relative risk'. The cyclist age-band CIs are 0.833–1.808 for children and 0.713–1.649 for older people, which cannot exclude large differences. Sec. 4.2 acknowledges this, but the abstract does not. Fix: say the study was underpowered to detect age differences of plausible size.
Provenance statements slightly inconsistent
minor · Sec. 1 (para 1); Sec. 2.2; Sec. 2.9; Table 1
Sec. 2.2 claims that 99.993% of deaths were placed. The paper's own totals imply 173 dropped deaths (99.979%), about 105 of them with unknown county, a group Sec. 2.9 does not mention (check 02). The 7,552 contiguous pedestrian deaths in 2022 are correct for the revised final file. However, the figure exceeds the national total of 7,522 in the cited source [3], so the 'within 0.02%' agreement in Sec. 2.2 cannot be against that source (check 11). Fix: name the comparison totals and give a full accounting of exclusions.
Asymmetric sensitivity-consistency rule
minor · Sec. 2.7; Sec. 3.6; Table 8; Fig. 6
The rule 'same direction and CI covers the primary RR' is lenient for the wide cyclist CIs and harsh for the near-null pedestrian estimate, where any point estimate below 1 counts as inconsistent (e.g. S12 0.992, 0.947–1.039). The summary counts of 20/24 and 13/25 are therefore not comparable across groups (check 07). Fix: also report how many contrasts exclude 1, and the range of estimates.
Questions for the authors
Can the plan v1.0 file and its SHA-256 digest be produced, along with evidence from an independent third party (registry, trusted timestamp, repository commit) that it existed before the FARS download? Can the full log of 17 deviations be provided?
How was the Northeast pedestrian SE in Table 3 computed? Was the small-sample factor based on the region's 9 clusters or the national 49? Does the CI survive a wild-cluster bootstrap?
Was the nClimGrid-Daily date convention checked before the plan was fixed? If not, would the authors present the aligned-window estimates as a new hypothesis to test on held-out data, rather than as the substantive estimate?
With labels d+1…d−2, do the attributable-death estimates and the 'no detectable change in pedestrian deaths' conclusion still hold?
What precipitation cut-points and holiday periods were used? How were Connecticut's 2022 planning regions and other county-boundary changes handled?
Which published national totals give the 'within 0.02%' agreement in Sec. 2.2, given that the cited source [3] reports 7,522 pedestrian deaths for 2022?
Verification
We ran 17 checks: 13 confirmed, 3 discrepancies and 1 not checkable. Using only public NHTSA and NOAA data, we rebuilt the 28,372,440 county-day panel and the death counts, and refitted the PPML models with an independently written estimator. Panel size, exposure counts and pedestrian and cyclist totals match exactly. The primary, S3, aligned and lag-specific RRs reproduce within 0.003 (checks 02, 03, 05). Holm and BH adjustments, the lag products, the sensitivity flags, the RRR ratios, the CI symmetry, the activity bound and the attributable-death arithmetic all check out. An empirical comparison with station data confirms the one-day dating offset that the authors invoke, and this also shows the confirmatory window is mistimed (check 04). The discrepancies are threefold: the Northeast pedestrian CI is about half the width the method gives (check 09), the abstract and conclusions overstate the cyclist finding (check 16), and several references are misdescribed (check 17). Adherence to the pre-registration could not be checked because the plan was not supplied (check 15).
Confirmatory family (Table 2): p-values for H1a/H1b/H3a/H3b from Wald CIs, chi2 p for H2a/H2b, Holm-adjusted p (H1b 0.093, H3b 0.376); no test rejectedConfirmedAbstract; Section 3.2; Table 2
Recomputed p: H1a 0.654 (claimed 0.654), H1b 0.0151 (rounding range 0.0144–0.0159; claimed 0.016), H2a 0.644 (0.643), H2b 0.923 (0.923), H3a 0.463 (range 0.448–0.478; claimed 0.453), H3b 0.0747 (0.075). Holm: H1b 0.091–0.096 (claimed 0.093), H3b 0.373–0.375 (claimed 0.376), others 1.000. No test rejected at 0.05 after Holm. Tolerance ±0.005 on p and ±0.01 on Holm p.
script: 01_holm_confirmatory.pyRecovered log-scale SE from each printed 95% CI, recomputed two-sided normal p (propagating ±0.0005 rounding of printed values), chi2(2) survival for age tests, Holm step-down over the six p-values
FARS provenance: 135,546 pedestrian, 19,965 cyclist and 679,655 non-motorcycle occupant deaths in the contiguous US 2000–2024 with the Table 1 age/sex/day–night splits; 4,723 contiguous pedestrian deaths in 2000 and 7,552 in 2022; 99.993% of deaths placed on a county-day; 85 deaths with unknown dayConfirmedSection 1; Section 2.2; Section 2.9; Section 3.1; Table 1
Recounts: ped 135,571 (paper 135,546, +0.018%), cyc 19,968 (19,965), occ 679,800 (679,655, +0.021%). The excess equals the unplaceable deaths almost exactly (ped 25, cyc 3, occ 143), so the paper counts placed deaths. Age, sex and day/night splits agree within 0.05%; hour-unknown differs by about 5% because of HOUR = 24 coding (29 ped). Contiguous ped 2000: 4,724 (claimed 4,723); 2022: 7,552 (exact). Unknown day of month: 86 deaths (claimed 85). Minor inconsistency: the paper totals imply 173 dropped ped/cyc/occ deaths (99.979% placed, including about 105 with unknown county, which Section 2.9 does not mention), not the stated 99.993%. The definitions in Section 2.3 are reproducible from public data.
script: 02_fars_counts.pyDownloaded all 25 NHTSA FARS national CSV zips (2000–2024), streamed the person files, and counted INJ_SEV = 4 by PER_TYP (5; 6–7; 1/2/9 in non-motorcycle body types) for contiguous states, with age, sex and crash-hour splits; tabulated non-placeable deaths (unknown county or day)
Independent reproduction of headline estimates: cumulative RR 1.143 (cyclists) and 1.010 (pedestrians); S3 next-day RR 1.379 (cyclists); cyclist cumulative RR 0.976 once the lead is addedConfirmedEq. (1); Table 2; Table 5; Table 6; Table 8 (S3); Table 11
Rebuilt estimates against the paper. Cyclists: primary 1.142 (1.025, 1.272) vs 1.143 (1.026, 1.273); S3 lead 1.377 (1.266, 1.499) vs 1.379 (1.267, 1.500); S3 cumulative 0.975 (0.869, 1.093) vs 0.976 (0.871, 1.092); aligned 1.406 (1.280, 1.544) vs 1.409 (1.285, 1.545); lag 0–3 RRs 1.222/0.963/1.060/0.915 vs Table 6 1.223/0.963/1.060/0.915. Pedestrians: primary 1.011 (0.967, 1.056) vs 1.010 (0.967, 1.055); lags 1.126/0.960/0.994/0.941 vs 1.126/0.960/0.994/0.940; S3 lead 1.142 (1.095, 1.191) exact; S3 cumulative 0.947 (0.906, 0.990) vs 0.947 (0.906, 0.989); aligned 1.120 (1.062, 1.180) vs 1.120 (1.062, 1.181). Every difference is ≤0.003 on the RR scale, CIs included. Fit sizes: cyclists 519,908 rows in 17,050 cells; pedestrians 2,392,081 rows in 78,525 cells. The headline numbers are reproducible from public data with the described method. The rebuild also confirms that the S3 lead term absorbs the cyclist association.
script: 03_reproduce_ppml.pyWrote a PPML estimator from scratch: county×year×month FE profiled out, Newton on β, two-way (state, date) clustered sandwich with G/(G−1) and eigenvalue truncation. Applied it to the rebuilt panel (check 05: nClimGrid-Daily county files + FARS 2000–2024) with 8 Tmax bins × lags 0–3, 5 precipitation bins × lags 0–3, day of week and approximate holiday indicators. Precipitation cut-points (0.25/2.5/10/25 mm) and holiday period definitions had to be assumed. Fitted the primary, S3 (adds next-day Tmax bins) and aligned (labels d+1…d−2) models for cyclists and pedestrians, and the regional models. Script work/verify/03_reproduce_ppml.py (+ panel_lib.py)
nClimGrid-Daily assigns each daily grid to the date of the morning observation, so Tmax labelled d+1 mostly describes the afternoon of day d (rationale for the post hoc aligned window and explanation of the failed falsification test)ConfirmedSection 2.8; Section 4.4; Table 11 note; ref [34]
The documentation says the grids interpolate ‘morning and midnight observations’ but does not state the date convention. Empirically, label d+1 matches calendar day d best at 8/8 stations: correlations of daily changes at labels d−1/d/d+1 are, for example, Chicago −0.22/0.02/0.93, Miami −0.22/0.00/0.90, Atlanta −0.18/0.08/0.92, DFW −0.22/0.13/0.89, Phoenix −0.02/0.50/0.76. Mean absolute level differences are smallest at d+1 (e.g. Chicago 0.80 °C vs 3.16 °C at d). This supports the paper’s explanation, and it also means the pre-registered window (labels d…d−3) essentially never contains the crash-day afternoon. The confirmatory ‘lag 0–3’ contrast is effectively a lag 1–4 contrast, and the S3 ‘falsification’ term is the same-day exposure. This was knowable before data collection, so the confirmatory estimates headlined in the abstract measure a mis-timed exposure.
script: 04_nclimgrid_dating.py(A) Read the NCEI nClimGrid-Daily user guide and the Durre et al. 2022 abstract (Crossref); the full text could not be retrieved (publisher and NOAA repository returned 403). (B) Empirical test: for 8 large-county first-order ASOS stations in GHCN-Daily (calendar-day Tmax), correlated day-to-day station Tmax changes on day d with nClimGrid county Tmax changes at labels d−1, d, d+1, 2003–2012 (n = 3,652 days each)
Panel: 3,107 counties × 9,132 days = 28,372,440 county-days; 2,884,882 (10.2%) county-days with Tmax ≥90 °F and 4,208,875 (14.8%) in 60–70 °F; days ≥90 °F carry 13.2% / 17.1% / 13.2% of pedestrian / cyclist / occupant deathsConfirmedSection 2.2; Section 3.1; Table 1
3,107 × 9,132 = 28,373,124. The 684 missing county-days are exactly Broomfield County before 15 Nov 2001, and the rebuilt panel has 28,372,440 county-days. Rebuilt counts match exactly: Tmax ≥90 °F on 2,884,882 county-days (10.2%) and 60–70 °F on 4,208,875 (14.8%), a 0 difference. Linked deaths: pedestrians 135,546 and cyclists 19,965 (both exact), occupants 679,657 (paper 679,655, +2). Share of deaths on ≥90 °F days: 13.2% / 17.1% / 13.2%, all exact. The exposure coding uses unrounded conversion of the two-decimal °C values; rounding °F to 0.1 would change the ≥90 count to 2,913,067.
script: 05_panel_exposure.pyRebuilt the county-day panel from the 604 nClimGrid-Daily county ‘scaled’ monthly files (tmax, prcp; 1999-12 to 2025-01). Mapped NCEI state codes to FIPS (DC is filed as NCEI 18511), removed Broomfield CO before 15 Nov 2001, converted °C to °F, and linked the FARS deaths from check 02 after remapping legacy codes (12025→12086, 46113→46102, 51560→51005, 51515→51019, 51678→51163). Script work/verify/05_panel_exposure.py with work/verify/panel_lib.py
Cumulative RR equals the product of lag-specific RRs (ped 1.010, cyc 1.143, occ 1.032; S2 lags 0–7 0.933/1.090/1.048; S2 lags 0–3 1.016/1.133/1.028)ConfirmedTable 5; Table 6; Table 8 (S2); Section 3.3
Products: ped 1.0100 (claimed 1.010), cyc 1.1423 (1.143), occ 1.0329 (1.032); S2 0–7: 0.9328/1.0913/1.0484 (0.933/1.090/1.048); S2 0–3: 1.0166/1.1328/1.0288 (1.016/1.133/1.028). All within rounding tolerance (≤0.005). This confirms that the null pedestrian cumulative RR is a lag-0 excess (1.126) offset by deficits at lags 1–3 (0.960, 0.994, 0.940).
script: 06_lag_products.pyMultiplied the printed lag RRs and compared with the printed cumulative RRs, with tolerance from 3-dp rounding of each factor
Sensitivity analyses: cyclists consistent in 20/24, pedestrians 13/25, occupants 24/24 under the rule same direction and CI covers primary RR; pedestrian range 0.916–1.076ConfirmedSection 2.7; Section 3.6; Table 8; Figure 6
Recomputed 13/25 (ped), 20/24 (cyc), 24/24 (occ); every printed C/N flag agrees with the rule, and the pedestrian range is 0.916–1.076 as stated. Under a stricter reading (consistent and CI excludes 1), only 16/24 cyclist contrasts remain. For pedestrians, because the primary RR is 1.010, any point estimate below 1 is classed N even when it is statistically indistinguishable (e.g. S12 0.992, 0.947–1.039). The rule is therefore lenient for wide cyclist CIs and harsh for the near-null pedestrian estimate, which affects the reading of ‘stable across most pre-specified checks’ (Section 4.1). For occupants, S9 counts as consistent because the occupant night-time reference (0.984) is itself below 1.
script: 07_sensitivity_flags.pyTranscribed all 73 cells of Table 8, re-applied the stated rule (S9 against the night-time subgroup estimate) and counted; also counted cells that additionally exclude 1
Secondary heterogeneity family: chi2 → p and Benjamini–Hochberg q-values (0.339, <0.001, 0.271, 0.003, 0.990, 0.339, 0.456, 0.385)ConfirmedSection 2.6; Section 3.5; Table 3; Figure 1
All 8 p-values and q-values reproduce within ±0.002 (e.g. ped blocks chi2 17.08 df 3 → p 0.0007, q 0.0027 vs printed 0.003; ped region q < 0.001). A first pass flagged cyclist sex because the printed chi2 is rounded to 0.00; p = 0.990 corresponds to chi2 ≈ 0.0002, which is consistent. The independent-subgroup approximations (Q = 2.02, 36.57, 3.04, 15.94, 0.00, 4.36, 0.61, 4.04) are close to the printed stacked-model statistics, so the heterogeneity statistics are internally coherent. The ped region result still depends on the Northeast SE (see check 09).
script: 08_heterogeneity_bh.pyRecomputed p from printed chi2/df (propagating 0.01 rounding of chi2), applied BH over the 8 tests, and recomputed an approximate Wald chi2 (Cochran Q) from subgroup log-RRs and CI-derived SEs assuming independent subgroups
Northeast pedestrian RR 0.862 (0.833–0.891) from 19,513 deaths and 9 state clusters (and Northeast cyclists 1.223 (1.031–1.450) from 2,369 deaths)DiscrepancyTable 3 (Pedestrians: region); Figure 1; Section 3.5; Section 4.4; Table 3 region test (chi2 = 33.85, BH q < 0.001)
(1) The printed Northeast pedestrian log-SE of 0.0172 is the smallest of all 15 pedestrian subgroup SEs, smaller even than the national 0.0222 from 7× as many deaths. That is 0.29× the √N expectation (standardised residual −3.22). (2) The independent refit reproduces the point estimate, 0.858 vs 0.862, but gives a log-SE of 0.0348 and a CI of (0.802, 0.919), 2.02× the paper’s SE. For the other seven regional estimates the ratio of rebuilt to paper SE is 1.02–1.21. So only the Northeast pedestrian interval is anomalously narrow, roughly half the width obtained with the described method. Using rebuilt SEs, the pedestrian region heterogeneity statistic (independent-subgroup approximation) falls from 36.57 to 17.45 (p = 0.0006), so the q < 0.001 conclusion is weakened but survives. The paper’s own caution about nine clusters (Section 4.4) is warranted, but the printed CI understates the uncertainty. The cyclist Northeast CI is also narrower than the rebuild (SE ratio 1.21; rebuilt 1.210 (0.984, 1.488) includes 1). (Regional refits were run with work/verify/03_reproduce_ppml.py, regions_ped / regions_cyc.)
script: 09_subgroup_se_scaling.py(1) Regressed log SE on log deaths across all printed subgroup estimates (work/verify/09_subgroup_se_scaling.py). (2) Refitted the primary model separately for each Census region with clusters restricted to the region’s states, using the independent estimator from check 03 (work/verify/03_reproduce_ppml.py, regions_ped / regions_cyc), and compared SEs
RRR estimates equal RR_group/RR_occ with a joint covariance: ped/occ 0.979 (0.925–1.036), cyc/occ 1.107 (0.990–1.238), motorcyclists 1.128 (1.059–1.202), aligned 1.044/1.314/1.082ConfirmedSection 2.6; Table 2; Table 10; Table 11
All six point ratios match within rounding (e.g. 1.010/1.032 = 0.9787 vs 0.979; 1.143/1.032 = 1.1076 vs 1.107). Implied correlations: ped/occ −0.36 (primary) and −0.28 (aligned), pooled/occ −0.29, cyc/occ −0.06/+0.05, motorcyclists/occ +0.33. The negative ped–occ correlation is unusual for outcomes sharing county-day weather and holiday shocks, but it widens the RRR intervals (SE 0.0289 vs 0.0253 under independence), so it is conservative and does not change H3a. Flagged as an observation: the joint-score construction cannot be audited without code.
script: 10_rrr_covariance.pyRecomputed each ratio from the printed RRs; backed out the implied covariance and correlation of the two log-RRs from the three CI widths (Var(log RRR) = Va + Vb − 2Cov)
78% of 2022 pedestrian deaths occurred in the dark [3]; national pedestrian totals agree with published FARS-based totals within 0.02%ConfirmedSection 1 (p.1); Section 2.2; ref [3]
TSF 813 590 reports 7,522 pedestrians killed in 2022 and 78% in the dark; the 78% matches. The TSF is based on the 2022 Annual Report File. The current 2022 final file gives 7,593 (50 states + DC), +0.94% vs the TSF, and 7,552 contiguous, exactly as in the paper. The paper’s numbers therefore come from the revised final file, which is correct. However, the contiguous 2022 figure (7,552) exceeds the national total in the source it cites ([3]: 7,522), and the ‘within 0.02%’ agreement in Section 2.2 cannot hold against that source; the paper should name which published totals it compared against (minor).
script: 11_nhtsa_tsf.pyDownloaded NHTSA Traffic Safety Facts DOT HS 813 590, extracted the total and dark share, and compared them with the current FARS 2022 final-file count
All reported RR (95% CI) triplets are Wald intervals symmetric on the log scaleConfirmedSection 2.5; Tables 2–11; text
326 triplets parsed; 0 outside rounding tolerance and 0 estimates outside their interval. No transcription mismatches between estimates and intervals were detected.
script: 12_ci_symmetry.pyParsed every x (l, u) / x (l–u) triplet from the extracted paper text and tested |log x − (log l + log u)/2| against the 3-dp rounding tolerance
Attributable deaths (Table 9): cyclists 417 (16.7/yr), pedestrians 146 (5.9/yr), pooled 589 (23.5/yr), age rows; hot-lag deaths 25,895 + 4,889 = 30,784ConfirmedSection 3.7; Table 9
All 12 rows admit an unrounded total consistent with both printed values. Pedestrians 5.9/yr needs an unrounded total in [146.25, 146.5) and pooled 23.5/yr needs [588.5, 588.75); both are feasible but narrow, so this is not a discrepancy. Hot-lag deaths: 25,895 + 4,889 = 30,784 (exact), and the pooled age rows equal ped + cyc age rows exactly (1,668; 23,928; 4,799). Attributable shares (ped 0.56%, cyc 8.5%) lie inside the AF ranges implied by the lag coefficients (ped −11.5% to 11.2%; cyc −13.5% to 22.9%).
script: 13_attributable_arith.pyChecked per-year = total/25 under printed rounding (range of unrounded totals compatible with both printed values), additive identities across groups and age rows, and that attributable/hot-lag-deaths lies within the AF range implied by the Table 6 lag coefficients
Activity bound: volume change reproducing RR = RR − 1 (+1.0%, +14.3%, +3.0%) and per-trip RR = RR/(1 + Δ) for Δ = −20..+20%, with CIsConfirmedSection 3.8; Section 4.3; Table 10
18/18 entries match within 3-dp rounding (e.g. cyclists Δ = −10%: 1.143/0.9 = 1.2700 (1.1400, 1.4144) vs printed 1.270 (1.140, 1.414)). The per-trip CIs treat the assumed volume change as known without error, which the table note makes clear.
script: 14_activity_bound.pyRecomputed all 18 table entries (3 volume changes, 15 per-trip RRs with CIs) from the primary RRs and CIs
Pre-registration: plan v1.0 time-stamped 10 Nov 2026 03:52:57 UTC before any FARS record was downloaded; 6 confirmatory + 8 secondary tests + 19 sensitivity analyses; 17 deviations recorded; none changed a confirmatory testNot checkableAbstract; Section 1 (last para); Section 2.1; Section 2.9; Section 4.5; Section 5; title page
The supplied inputs contain only paper.pdf and request.json, with no plan, digest or deviation log. The paper prints no SHA-256 value and no registry or repository link, and Section 2.1 says the plan ‘was not lodged on a public registry’. So adherence to the plan cannot be verified. The internal counts do add up: all 19 sensitivity analyses are reported (S1–S17 in Table 8, S18–S19 in Table 4), there are 6 confirmatory tests and 8 BH-adjusted secondary tests. However, Section 2.9 describes only 10 of the ‘seventeen’ departures. Two of the described changes do touch confirmatory tests: the purpose-written estimator and modified two-way covariance affect all six tests, and the ‘stacked comparisons from separate fits’ affect H3a/H3b. That sits uneasily with ‘None of these departures changed a confirmatory test’. Timing: the PDF CreationDate is 2026-11-10 09:16:55 UTC, 5 h 24 min after the plan timestamp. Data download, panel build, every fit and typesetting all fall in that window, and the plan carries only a self-recorded timestamp. Calling the study ‘Pre-registered’ in the title is stronger than this evidence supports.
script: 15_prereg_consistency.pyScript lists the supplied inputs and searches the paper for a digest, registry URL or plan; computes the interval between the plan timestamp, the paper date and the PDF CreationDate; counts the reported analysis families (S1–S19 in Table 8 and Table 4; 6 confirmatory rows; 8 BH tests from check 08); and enumerates the deviations described in Section 2.9
Title and abstract match the results and causal language is appropriate; abstract concludes ‘Hot days were associated with a modest rise in cyclist deaths, no detectable change in pedestrian deaths and no age group at distinctly higher relative risk’DiscrepancyTitle; Abstract (last sentence); Section 4.1; Section 4.4 (para 2); Section 4.6; Section 6 (last sentence); Table 2; Table 8 (S3); Table 9
Every abstract number matches the body and tables (0 missing). All six Table 2 rows are as stated, with 0 of 6 tests rejected after Holm, and the abstract does report Holm p = 0.093 for cyclists. The concluding sentences are another matter. The abstract (‘Hot days were associated with a modest rise in cyclist deaths’) and Section 4.6 (‘the hottest days carried a modest excess of cyclist deaths’) state the cyclist association as a finding. Yet the pre-registered confirmatory test H1b was not rejected (Holm p = 0.093), the S3 model with the next-day term gives a cyclist RR of 0.976 (0.871, 1.092), and only the post hoc aligned model is clearly positive. The plain-language summary (‘suggestive rather than firm’) is better calibrated than the abstract, yet it ends with a policy line, ‘hot days call for attention to the safety of people cycling’. Section 4.1 says the cyclist estimate ‘was stable across most pre-specified checks’; 20 of 24 contrasts are consistent, but only 16 of 24 exclude 1 and S3 is null. Causal or mechanistic wording appears in ‘Deaths attributable to days with Tmax ≥90 °F’ (Section 2.6, Section 3.7, Table 9, with point counts given for non-significant RRs) and ‘heat acts on occupants mainly through daytime driving … and on pedestrians through night-time exposure’ (Section 4.4). The latter is drawn from exploratory, post hoc subgroup contrasts. ‘No age group at distinctly higher relative risk’ reflects non-rejection with wide intervals (cyclist children 0.833–1.808, older 0.713–1.649); the paper acknowledges this in Section 4.2 but not in the abstract. The title says ‘Pre-registered’, but no plan is available (see check 15).
script: 16_abstract_vs_results.pyScript extracts every number in the abstract and locates it outside the abstract; matches the abstract values to Table 2 rows; lists each abstract or conclusion statement beside the corresponding confirmatory test outcome and sensitivity result; greps for causal and mechanistic terms
References exist and support what they are cited for ([44], [24], [16], [2], [21], [26] and the 2025–2026 items)DiscrepancyReferences; Section 1 (paras 2–3); Section 4.2
All 44 references exist ([2] DOT HS 813 794 is the April 2026 FARS Analytical User’s Manual 1975–2024; [18] exists, PMID 41111441, though the checker mismatched it). Several citations misdescribe their sources. [26]: the paper says its data source, spatial unit and fixed effects ‘are not stated in the publicly available abstract’, yet the abstract states FARS 2016–2022, state-day panels, GHCN-Daily and its adjustment terms, and reports motorcyclists 2.141 and pedestrians 1.213 (no cyclist or age result). [21]: 2.31 [1.59, 3.38] is correct, but bicyclists were not the ‘largest’ increase (motor vehicles on other roadways had 3.77). The source also stresses a steeper night-time slope, whereas the paper cites it for effects ‘stronger by day’. [16] covered all fatal crashes in May–September 2001–2011, so it was not ‘restricted to drivers’. [44] evaluates passing laws with weather as a control; nothing in its abstract supports ‘cyclist deaths rise with temperature’. The [24] pedestrian and 65+ statements could not be verified (full text blocked). Details are in work/ref_issues.json.
script: 17_references.pyRan tools/check_refs.py (all 44 resolved via Crossref, two with low match scores). Spot-read sources via PubMed (16, 18), Crossref abstracts (24, 26), Semantic Scholar (44), the Springer full text (21) and the NHTSA PDF (2). Script work/verify/17_references.py greps each source for the cited facts
References
44 references checked
- [26] O. Tatsuki. Hotter-than-normal days and traffic-crash mortality ... SSRN Working Paper 7246106, 2026
Mischaracterised. Section 1 says the working paper's 'data source, spatial unit, fixed effects and age bands are not stated in the publicly available abstract'. The public abstract (Crossref, doi 10.2139/ssrn.7246106) does state its data source (FARS 2016–2022, GHCN-Daily), its spatial unit (state-day panels) and its adjustment terms (spatial, seasonal, long-term-trend, day-of-week). The paper also says [26] reports larger associations for 'pedestrians, cyclists and motorcyclists', but the abstract gives motorcyclists 2.141 and pedestrians 1.213 and does not mention cyclists or age. This bears on the novelty framing ('closest overlap').
- [21] Hsu, Quistberg, Pérez-Ferrer, Rodríguez. Discover Cities 3:61, 2026
The 2.31 RR for bicyclist fatal/severe crashes at the 99th percentile is correct. However, Section 1 says heat was associated with 'the largest relative increase for severe and fatal bicycle crashes'. In the source, crashes involving motor vehicles on other roadways had the highest RR (3.77 [1.04, 13.61]), with bicyclists second. The paper also cites [21] for heat effects being 'stronger by day'. The source reports a higher daytime RR (1.46 vs 1.15) but a steeper night-time slope at extreme heat, 'indicating a greater increase in risk under extreme nighttime heat'.
- [16] Wu, Zaitchik, Gohlke. Accident Analysis & Prevention 119:195–201, 2018
Section 1 describes the study as 'restricted to drivers'. Per the PubMed abstract (PMID 30048841), the outcome was all fatal traffic crashes in May–September 2001–2011, with subgroups by driver age; it was not restricted to drivers. Its warm-season-only window is not mentioned. The 'middle-aged drivers' citation in Section 4.2 is supported (56–65-year-old drivers, 8.2%).
- [44] Cody Nehiba. Give me 3': Do minimum distance passing laws reduce bicyclist fatalities? Economics of Transportation 14:9–20, 2018
Cited in Section 4.2 as 'state-month analyses in which cyclist deaths rise with temperature'. The study is a difference-in-differences evaluation of passing laws, and weather is a control. The abstract (Semantic Scholar) does not mention a temperature finding. The support is at most an indirect control coefficient and should not be presented as evidence on heat.
- [24] R. Pan et al. Environmental Research Communications 6:085011, 2024
Could not verify. The abstract reports only the all-transport-accident association (1.47% per °C) and says nothing about pedestrians or people aged 65+. The full text was blocked by a publisher captcha, so the claims 'pedestrian association was null' and 'aged 65 and over ... close to the all-age estimate' remain unchecked. The citation lacks volume and article number.
- [18] David C. Schwebel et al. J Inj Violence Res 17(1):97–101, 2025
The reference exists (PMID 41111441), but it is single-authored, so 'et al.' is wrong. tools/check_refs.py matched it to the wrong Crossref record (score 37.9). The paper summarises it as showing correlation 'across states but not with year-to-year temperature change'. That omits its national annual correlation (r = 0.61), but the change-from-previous-year analysis was indeed null.
- [3] NCSA. Pedestrians: 2022 data. Traffic Safety Facts DOT HS 813 590, 2024
The reference exists; its 78% in the dark matches. It reports 7,522 pedestrian deaths in 2022 (Annual Report File). The paper's '0.02%' agreement with published totals cannot hold against this source: the rebuilt contiguous count is 7,552, and the final-file national count is 7,593 (+0.94%). See check 11.
- [6] Danesh, Monazzam, Hadei. Human Factors, 2025
Minor. Crossref gives a 2026 issue date (22 Jul 2026) and five authors (with Zakerian and Sakhaei); the citation lists three authors and 2025. The paper also uses [6] to support 'heat acts on occupants mainly through daytime driving'. The meta-analysis found only reaction-time impairment in 7 experimental studies (about 207 participants).
Suggestions
Rewrite the last sentence of the abstract and Sec. 4.6 so that they report the confirmatory outcome: no test rejected, and the cyclist association not robust to the next-day term.
Drop 'Pre-registered' from the title unless a third party can verify the plan timestamp. Publish the plan, the digest and all 17 deviations.
Treat the aligned window as the substantive exposure only after a prospective or held-out confirmation, for example by registering the aligned analysis and testing it on 2025–2026 FARS data.
Recompute the regional (and other few-cluster) CIs with region-specific cluster corrections or a wild-cluster bootstrap. Revise Table 3, Fig. 1 and the heterogeneity tests accordingly.
Replace 'attributable' with 'excess deaths associated with', and drop or heavily qualify the mechanistic sentence in Sec. 4.4 and the policy line in Sec. 6.
Release the estimator code with a numerical comparison against an established package (e.g. fixest/ppmlhdfe) on a subset, and state the precipitation bins and holiday definitions.
Correct the descriptions of [16], [18], [21], [26] and [44], and reassess the novelty framing against [26].
Give a full flow of exclusions: unknown county, unknown day, AK/HI/PR, Broomfield before 2001-11-15. Name the published totals used for the 0.02% comparison.
Ethics and integrity
No concerns about the data: FARS and nClimGrid are public and de-identified. Integrity points: the sole listed author is an AI system (IterSCI), and no human takes accountability or gives a conflict-of-interest statement. The pre-registration rests on a self-recorded timestamp only 5 h 24 min before the PDF was created and cannot be independently verified. Sec. 5 says code and data are distributed, but they were not supplied. The requester's notes were treated as context only; they contained no instructions directed at the reviewer.
Scope of this review
We received only the PDF. The plan, digest, deviation log, code and person-level file described in Sec. 5 were absent, so adherence to the pre-registration and the 7 undescribed deviations could not be examined. The reproduction (checks 03, 05, 09) used public FARS and nClimGrid-Daily data with our own estimator, and the precipitation cut-points and holiday definitions were assumed. The secondary, occupant, motorcyclist and most sensitivity models (S1–S19 other than S3) were checked for internal consistency only and were not refitted. The full text of [24] and of Durre et al. [34] could not be retrieved.