The descriptive core is carefully built and its numbers check out: a raw 0.180 log-point gap, transparent pre-trend diagnostics, and tables that reproduce. The headline interpretation does not hold as stated: 'about half reflects trajectories' and 'lasting gains' rest on a post hoc fake-date estimator that the paper's own trend benchmarks contradict, and the preregistration cannot be verified.
Soundness
2/4
Presentation
3/4
Contribution
3/4
Confidence
3/5
✓ 9 confirmed✕ 5 discrepancy◐ 3 inconclusive○ 1 not checkable
Summary
The paper links the first names of main characters in 864 major English-language films and series released in 1986–2020 (selected from Wikidata/Wikipedia by present-day sitelinks) to US Social Security Administration name counts. It runs a stacked event study in which each of 1,332 character names is compared with five never-on-screen control names matched on the pre-release level and the k=−3..−1 trend. The primary estimate is the mean log-rate gap over event years k=0..3: 0.180 log points (19.7%, 95% CI 0.148–0.211), or about 354,000 excess births.
The paper reports its own pre-trend problems prominently. Held-out leads (k) sit about 0.10 above controls. A fake release date eight years early yields 0.121. A post hoc within-name difference between true- and fake-date estimates (0.096, 0.052–0.143) is presented as the more conservative release-window estimate, and from it the abstract and conclusions infer that about half of the raw gain reflects pre-existing trajectories. The preregistered rule for flagging individual names as 'made popular' identifies none; a post hoc count suggests about 81. The profile keeps rising to (0.534), so no decay half-life is identified. Heterogeneity by sex, format, genre and state is small, apart from action/adventure.
Strengths
Unfavourable pre-trend diagnostics are reported up front
Abstract; Sec. 3.4; Sec. 4.1; Fig. 1a,b; Table 3 (R8, R13)
Leads at k=−6..−4 are held out of matching and used as a diagnostic. The paper adds a fake-date benchmark (release date −8 years) and a long-trend matching variant (R13). It then puts the bad news in the abstract: leads ≈0.10 and a fake-date mean of 0.121. Many event studies in this area show only post-release curves. Here the reader is told directly that part of the raw gap predates release.
Departures and post hoc analyses are labelled in the tables
Sec. 3.8; Tables 2 and 4; Abstract
Sec. 3.8 lists the R13 addition and the three post hoc analyses P1–P3 with their motivation. P-rows carry an asterisk in Tables 2 and 4, and the abstract labels the P2 release-window estimate and the 81-name count as post hoc. This makes the confirmatory/exploratory boundary auditable, even though some summary sentences later blur it (weakness 3).
Reported numbers are internally consistent and reproduce from what is printed
Fig. 1; Tables 1–5; Sec. 4.4; Sec. 4.7
Digitising Fig. 1 reproduced all 12 quoted coefficients within 0.002 (check 05). Refitting the decay model to the digitised curve gives vs the reported −0.115 (check 07). The state meta-analysis (FE mean, Q, p, , , ) recomputes from Table 5 (check 12), and so do all ten genre BH q-values (check 13). 31 of 32 count identities hold (check 10), and p-values agree with their bootstrap CIs (check 14). Nothing suggests fabricated or transcribed-wrong numbers.
Weaknesses
'About half reflects trajectories' rests on a post hoc estimator with an unstated assumption and is metric-dependent
major · Abstract (last sentence); Sec. 1; Sec. 3.8; Sec. 4.1–4.2; Sec. 5.1; Sec. 5.5; Table 2 (P2 rows); Table 3 (R10, R13); Table 4 (P2)
The closing sentence of the abstract, the contribution paragraph of Sec. 1 and Sec. 5.5 all state that about half of the raw gain reflects pre-existing trajectories. The basis is P2, a post hoc true-minus-fake-date difference (0.096 vs 0.180). Three problems follow.
(i) The fraction is not robust to the metric. It is 0.47–0.56 on the unweighted log scale, but in the paper's own birth counts the fake-date excess is 14,170 against 240,444 at true dates, about 6% (Table 2). In names above chance it is about 12% (check 02).
(ii) The paper's other pre-trend benchmarks point the other way. The leads show treated names falling relative to controls toward release (0.171, 0.111, 0.076). Linear de-trending (R10) raises the estimate to 0.301, and our extrapolation of the quoted leads reproduces 0.31–0.35 (check 03). R13 raises it to 0.197. The paper adopts the fake-date benchmark without arguing why it beats these.
(iii) P2 is conservative only if selection bias at the fake date equals selection bias at the true date. The paper never states this assumption. In an illustrative simulation with zero true effect, P2 is biased in either direction depending on how writers pick names (check 04). Digitising Fig. 1b also shows the fake-date curve accelerating from 0.059/yr to 0.101/yr exactly at the true release years (check 05), so the difference mixes a long trend with a release-time kink.
Fix: state the identifying assumption; report the share on all metrics; reconcile with R10/R13 or report a bracket (e.g. 0.10–0.30) rather than 'about half'; estimate a trend break at the true date directly; and label the sentence as post hoc wherever it appears.
'Lasting gains' / 'persists for at least a decade' is not identified
major · Abstract; Sec. 4.4; Sec. 5.5; Sec. 6; Fig. 1a,b
The abstract ('lasting gains'), Sec. 5.5 ('the gain persists for at least a decade') and the plain-language summary ('The rise did not fade … still gaining') read the raw profile as a durable release effect. But the fake-date profile in Fig. 1b also rises steadily over ten years, to 0.750 at fake (about 0.06/yr before the true release, check 05). The trajectory-adjusted P2 estimate covers only . The part of the rise that is release-related is therefore not separated from the pre-existing trend. Under the paper's own logic, the continued rise is at least partly trajectory.
Questions for the authors
Under what assumption is the true-minus-fake-date difference (P2) a conservative estimate? Why should it be preferred over the linear de-trended (R10, 0.301) and long-trend matched (R13, 0.197) estimates, which imply that the raw estimate understates the release-window change?
Does 'about half reflects trajectories' survive when measured in births (fake-date 14,170 vs true-date 240,444 excess births, about 6%) or in names above chance? If not, which metric does the abstract refer to?
Can the frozen plan, its hash target, and the deviation log be made public with an independent timestamp? Which pre-release analyses (including the leads test that motivated R13) were run before the 04:32 UTC freeze, and when were the SSA and Wikidata files downloaded relative to it?
What is the true-minus-fake-date difference at k=4..10? Without it, what supports 'the gain persists for at least a decade'?
Is the lead mean of 0.103 (0.079–0.127) computed on all 1,332 names while the three quoted leads use the 1,172 balanced names? What is the mean lead in the balanced sample?
How do the estimates change with not-yet-treated character names as controls, instead of names that never appeared on screen?
Why was the BH correction in Table 1 applied over nine hypotheses rather than the ten stated, and was this specified in the plan?
What happened to the 319 Set B events not in the 1,211-name Set B analysis? Why do Set B names reach 5 births less often than the placebo expectation?
Verification
18 checks were run, each with its own script and log: 9 confirmed, 5 discrepancies, 3 inconclusive, 1 not checkable. Only the PDF was available, so the checks recompute from printed numbers, the digitised Fig. 1, the Table 5 state estimates, a public copy of the SSA data to 2017, and the cited sources. The reported numbers are internally sound: Fig. 1 coefficients, the decay fit, the state meta-analysis, the genre BH q-values, the p/CI agreement and 31 of 32 count identities all reproduce. The discrepancies concern interpretation and bookkeeping, not fabricated values. The 'about half' trajectory share holds only on the log scale (≈6% in births), and trend-based benchmarks contradict its direction (checks 02–03). Table 1 BH uses m=9 rather than the stated ten. The lead mean does not equal the mean of the quoted leads. Small count and conversion inconsistencies remain. An illustrative simulation shows that the fake-date difference is not guaranteed to be conservative under writer selection (check 04). Preregistration adherence could not be verified because the plan was not provided (check 17).
9Confirmed
5Discrepancy
3Inconclusive
1Not checkable
✕DiscrepancyAbstract; Sec. 3.5; Sec. 4.1-4.3; Sec. 4.6; Table 1; Table 3; Sec. 6
[10] La Ferrara, Chong, Duryea (2012), Soap Operas and Fertility: Evidence from Brazil, AEJ: Applied 4(4)
Cited in Sec. 2.1 as Globo areas being '18 to 24% more likely' to have a novela character's name among the 1994 top-20 names. In the source (working-paper Table 9), 0.18-0.24 are linear-probability coefficients, i.e. 18-24 percentage points on a base probability of 0.085 (covered areas: 0.329). The relative increase is far larger than 18-24%. The unit is misstated, though the direction of the claim is supported (work/verify/18_refs_spot.log).
[17] Little Sleepies (2025), Characters vs. Celebrities: Who's the Inspiration for Baby Names?
Commercial blog/industry report with no URL or access date; Crossref returns an unrelated work. It is load-bearing in Sec. 5.2: the 'uncontrolled mean of 98%' and the film-vs-series and sex-difference comparisons rest on it alone. Not verifiable as cited.
[18] BeenVerified (2024), BeenVerified Study Finds Impact of Disney Movies on Baby Names
Press release/grey literature with no URL; Crossref match is an unrelated 2010 article. Not verifiable as cited (used only as context in Sec. 2.1).
[6] Verma and Bhardwaj (2020), Game of Thrones: Impact on U.S. Baby Names and Network Analysis, B.Tech. project report, Jaypee University
Unpublished student report with no URL or repository; Crossref best match is an unrelated FFT-architecture paper. Existence could not be confirmed. Cited in Sec. 1 for the rise of names such as Arya.
[11] Napierski-Prancl (2016), Brandy, You're a Fine Name, Studies in Popular Culture 38(2):41-54
Exists (confirmed via web search; Crossref best match is an unrelated 2009 article because the journal has no DOIs). Publisher listing gives pp. 41-53, not 41-54. The paper uses it to support 'peaks within about three years and then falls' (Sec. 1, 5.2); the source tracks girls' names five years before and after a Billboard chart year, which roughly supports 'rise and fall within five years' (Sec. 2.1) but not specifically a three-year peak.
[19] Ludwig, Hightower, Engels, McGee (2019), Analyzing Influences on U.S. Baby Name Trends, SMU Data Science Review 2(3)
No Crossref record (best match is a different title with no year); the journal is not DOI-registered. Existence not confirmed in this review. Context-only citation.
[42] Social Security Administration (2026), Background Information: Popular Baby Names
Web page cited without URL or access date. It is the primary data source for the whole study, so the exact file, download date and version (the national and state files are revised over time) should be given for provenance.
Suggestions
Rewrite the abstract's last sentence and Sec. 5.5 to report the release-window change as a bracket across the paper's benchmarks (fake-date P2 ≈ 0.10; R13 0.197; R10 0.301). Label the fake-date-based fraction as post hoc and metric-dependent.
Estimate a trend break at the true release date directly (e.g. a kink in the slope of τ around k=0, using k≤−1 for the pre-slope). Fig. 1b already suggests a 1.7× acceleration there.
Extend the true-vs-fake comparison to before claiming persistence, and replace 'lasting' in the title-level summary until it is done.
Ethics and integrity
No text in the paper or the requester notes attempts to instruct the reviewer. According to the requester's notes, the study was produced end to end by an AI research system (IterSCI). The paper itself lists only 'IterSCI' as author and contains no statement on AI authorship, human oversight, or a contact responsible for the data and preregistration. The 'preregistered' label in the title rests on a self-certified, unpublished freeze made on the paper's own date. Together these make independent accountability for the preregistration claim impossible at present. An explicit authorship and accountability statement and a public registry deposit are recommended. The data (aggregate public SSA counts with small cells suppressed, and Wikidata) raise no consent or privacy concerns.
Scope of this review
Only the PDF was available. The analysis code, the frozen plan and hash target, the deviation log, the processed results file and the data documentation that the paper says are 'released with this article' were not provided, so no result could be rerun from source and the preregistration could not be compared with the paper. ssa.gov blocks scripted downloads (HTTP 403). The national row count was checked only against the CRAN babynames copy (to 2017), and the state file and the 2018–2025 national rows were not checked. The title list was not reconstructed from Wikidata. Figure values were digitised from embedded rasters (tolerance 0.02). Excess births (check 11) and the fake-date simulation (check 04) are order-of-magnitude and illustrative reconstructions, not replications. References [9] and [10] were checked against working-paper versions. The novelty claim ('no US study has provided' a controlled estimate) was not tested by a systematic literature search.
The requester asked specifically about preregistration adherence, data provenance, causal language, and whether the title and abstract match the results. These are addressed in weaknesses 1–3, 6 and 7 and in checks 02, 09, 15–17.
λ=−0.1147
I2
τ2
∑n=9,146
Systematic sampling and careful matching
Sec. 3.2–3.5
Titles are drawn by a fixed rule (top 40 films and top 20 series per year), not hand-picked famous cases, so the much-cited Madison/Arya anecdotes are not selected on the outcome. Controls are matched by Mahalanobis distance on level and trend, with phonetic/spelling neighbours excluded and standardized balance of 0.013 (level) and 0.002 (trend). Inference uses a title-clustered bootstrap with BH correction within families.
Null and negative results are reported straight
Table 1; Sec. 4.3–4.6
The preregistered timing hypothesis (H4: peak in k=1..3; share of replicates 0.002), the decay hypothesis (H5, λ<0, p = 1), the sex, format and adaptation contrasts (H7–H9) and the failed 'made popular' rule are all reported as unsupported, not reframed. The contrast with the qualitative 'tsunami curve' literature [4, 11] is stated plainly.
Broad robustness battery
Table 3; Fig. 3; Sec. 4.3; Table 2
Thirteen specifications (R1–R13) are all positive, ranging from 0.153 (R7) to 0.301 (R10). Six stay within 0.005 of the primary estimate. Excess births are given with bootstrap intervals and their concentration across titles (median 33, mean 410 per title; 354,276/864 = 410.0 checks).
Accurate positioning against prior evidence
Sec. 2.1–2.3; Sec. 5.2
The figures attributed to Disdier et al. [9] check out against the source: about 10% at mean name age, 62% for new names, no effect beyond about 31 years, and 180/927/4,845 titles (check 18). The summary of the staggered-DiD literature [27–33] is correct in broad strokes. The industry figure of 98% [17] is rightly presented as an uncontrolled upper benchmark.
βk
k=10
k=0..3
k=4..10
The decay test (H5) is computed correctly (λ=−0.115 reproduced, check 07). However, 'not identified' under the rule is read in Sec. 4.4 as 'beyond ten years', and in Sec. 5.5 as evidence against short-lived fads. With a trend-dominated average profile and staggered individual peaks, the test cannot show either.
Fix: extend the true-minus-fake comparison to k=4..10 (feasible for event years ≤ 2015). Until that is done, replace 'lasting'/'persists' with a statement that the raw gap keeps widening, of unknown attribution.
Preregistration cannot be verified, and adherence is incompletely documented
major · Title; Sec. 3.1; Sec. 3.8; Table 1 note; Abstract; Sec. 1; Sec. 5.5; Sec. 6; Code and data availability
The title rests on 'preregistered'. The plan was frozen internally on 10 Nov 2026 at 04:32:17 UTC, the same date the paper carries. It was not deposited with a third-party registry (OSF, AsPredicted), and the plan, deviation log and code were not available for review (check 17). A reader therefore cannot confirm the freeze time, the plan's content, or that 'confirmatory analyses are reported exactly as planned'.
Where adherence can be checked, there are gaps. Table 1 states BH 'within the ten hypotheses', but every q-value reproduces only with m=9, i.e. H6 was dropped (check 09). The result is harmless but it is an unlisted departure. Four of 15 headline claims are post hoc, and the abstract's closing sentence, Sec. 1 and Sec. 5.5 state the post hoc 'about half' without the label. The genre result in the abstract is exploratory (X4) and unlabelled (check 13). The plain-language summary says 'the plan for the analysis was fixed and locked before any results were looked at'. That sentence does not tell the lay reader that the central interpretation came afterwards.
Fix: publish the frozen plan and the hash target with an independent timestamp. Add a plan-vs-paper table covering every hypothesis, list the m=9 change as a departure, and label every post hoc or exploratory claim in the abstract, introduction, conclusions and summary.
The preregistered 'made popular' rule was foreseeably near-unattainable, leaving RQ2 answered only post hoc
major · Sec. 3.6; Sec. 4.3; Table 2 and note; Sec. 5.4
With about 500 placebo stacks per baseline decile, the smallest attainable p is about 1/501. BH at q<0.05 over m=1,332 names then needs at least 54 names that beat every placebo in their decile before any single rejection is possible (check 08). The observed minimum q of 0.18 implies about 15 such names. The rule is thus not impossible 'at this sample size', as Sec. 4.3 and 5.4 say. It is underpowered by design. A pre-freeze power check on the synthetic stacks the plan already used (Sec. 3.5) would have shown this, and 20,000–50,000 placebo stacks would bring the requirement down to 6–14 floor names. One of the three research questions ('how many names do releases make popular') therefore rests entirely on the post hoc P1 count of about 81.
Fix: state that this was a design flaw. Rerun the preregistered rule with a larger placebo reference, or use a parametric or extreme-value tail approximation, and report that alongside P1.
Control and placebo names come from a different population than character names
major · Sec. 3.4; Sec. 3.6; Sec. 4.1
The control pool excludes every name used by a main character in any major title from 1975 to 2025, and also excludes phonetic and spelling neighbours. The placebo pool is built the same way. Controls are therefore names that writers never chose. Character names, by the paper's own argument (Sec. 1), are chosen because they are fashionable or rising. Matching on a three-year level and trend does not remove differences in longer-run dynamics: the held-out leads differ by about 0.10, and R13 changes the estimate. The placebo distribution then measures noise among never-chosen names, not the selection process, which weakens both the 'made popular' calibration and the P1 count.
Fix: add a not-yet-treated design (character names whose release falls 8 or more years later serve as controls), in the spirit of the staggered-DiD estimators the paper cites [29, 30]. Also report balance on the k=−6..−4 leads and on longer-run curvature.
Causal vocabulary in places outruns the associational design
minor · Abstract; Sec. 4.3; Table 2; Sec. 5.5; Sec. 6
Sec. 5.3 is careful: the estimate is 'an association … not an experimental effect'. Other places use causal framing. Examples are 'excess births' (Abstract, Table 2), 'attributes about 0.10 … to the release window' (Abstract), 'How many names were made popular' (Table 2 title, Sec. 4.3), 'how long the effect lasts' and 'the boost has no measurable half-life' (Sec. 6), and 'names that a release made popular' (Sec. 5.5). The abstract's closing verb 'coincide' is appropriate. Given weaknesses 1, 2 and 5, the estimand should be described consistently as a gap relative to matched controls.
Fix: use 'births above the matched-control counterfactual' for excess births, and 'associated with' or 'coincides with' throughout, including the plain-language summary.
Data provenance is underspecified
minor · Sec. 3.2; Sec. 4.7; Table 4 (X5); Code and data availability; Ref. [42]
The title list cannot be reconstructed from Sec. 3.2. 2,148 films / 40 per year = 53.7 years and 944 series / 20 per year = 47.2 years, which are non-integer and mutually inconsistent, and the year range is not stated (check 15). No Wikidata snapshot or query date is given; the paper says only 'November 2026'. Selection uses present-day sitelinks, a survivorship criterion; Sec. 5.4 acknowledges this. SSA file checksums and release dates are deferred to documentation that was not provided, and [42] has no URL. The national row count (1,316,981) is plausible against the CRAN copy of the SSA data to 2017 (check 16), but the state file count could not be checked. 'Western states' (X5) is undefined; it matches the 13 Census West states exactly (check 12).
Fix: give the title-year window, the exact SPARQL query and timestamp, the SSA file dates and checksums in the paper, and the region definition.
Several internal inconsistencies in counts and summaries
minor · Sec. 3.3; Sec. 4.1; Sec. 4.2; Sec. 4.3; Table 1; Table 2; Sec. 6
(a) Sec. 4.1 gives leads of 0.171, 0.111 and 0.076 and a 'mean over these three years' of 0.103. Their mean is 0.119 (check 06). The figures reconcile only if 0.103 is the full-sample mean and the leads are for the 1,172 balanced names, and this affects the diagnostic that motivated departure 1.
(b) 1,714 − 71 − 350 = 1,293 ≠ 1,332. The gap is explained only by an undocumented overlap of 39 names, which the R5/R6 sample sizes imply (check 10).
(c) Set B is defined in Sec. 3.3 as names listed in at most two pre-years (1,530 events). Sec. 4.3 analyses 1,211 Set B names that had no listed pre-year, with 319 unaccounted for. Set B names also reach 5 births less often than placebo names (122 vs 176.3), which is unexplained.
(d) The plain-language summary's '12% boost' converts 0.121 log points as 12% rather than 12.9%, unlike the 0.180 → 19.7% conversion elsewhere (check 01).
(e) H1 carries two CIs ([0.147, 0.214] vs [0.148, 0.211]).
State estimates average well below the national estimate without comment
minor · Sec. 4.7; Table 5; Fig. 5
The fixed-effect mean of the Table 5 state estimates is 0.121 (n-weighted 0.123; reproduced, check 12), about two-thirds of the national 0.180. Possible reasons include restriction to names common in a state, different matching, or composition. The paper does not discuss the gap, although it bears on whether the national estimate is driven by a subset of names.
Some cited evidence is misstated or unverifiable
minor · Sec. 1; Sec. 2.1; Sec. 5.2; Refs. [6], [10], [11], [17], [18]
[10] La Ferrara et al. is cited as Globo areas being '18 to 24% more likely' to have a novela name in the 1994 top 20. In the source these are linear-probability coefficients, i.e. 18–24 percentage points on a base of about 0.085, which is a much larger relative effect (check 18). [11] supports a rise-and-fall within five years, not specifically a peak 'within about three years'. The comparison in Sec. 5.2 against an uncontrolled 98% rests on [17], a commercial blog with no URL. [6] and [18] are grey literature that could not be located.
The 'short-lived fads' conclusion over-generalises from an average profile
minor · Sec. 5.5; Sec. 4.4; Table 2
Sec. 5.5 says claims that screen media 'routinely create short-lived naming fads are not supported'. An average over 1,332 names with staggered and heterogeneous individual peaks can rise monotonically even if many individual names spike and fall. The paper's own Table 2 shows the gains concentrated in a few titles. Answering this needs name-level profiles, for example the share of P1 names whose own series peaks within 3 years.
Sample flow and table arithmetic: 3,244 = 1,714 + 1,530; 1,332 after excluding 71 common and 350 multi-title names; subgroup splits; R5/R6/R9/R12 n; Table 2 excess and share columns; 410 births per title; Set B description
Minor. 31 of 32 identities hold exactly: all subgroup and genre splits sum to 1,332 or 1,172; Table 2 excesses and shares (80.7, 36.2, 7.5, -5.0; 0.110, 0.361, 0.057, 0.329, 0.066) and 354,276/864 = 410 all match. One identity fails: 1,714 - 71 - 350 = 1,293, not 1,332. This reconciles if 39 names are both common and multi-title, and R5 n = 1,364 (= 1,332 + 32) and R6 n = 1,643 (= 1,332 + 311) imply exactly that overlap. So the numbers are coherent but the overlap is not documented. Set B is inconsistent: Sec. 3.3 defines it as listed in at most two pre-years (1,530 events), while Sec. 4.3 analyses 1,211 Set B names that had no listed pre-release year, leaving 319 unexplained. Set B names reached 5 births less often than placebo names (122 vs 176.3 expected), which is unexplained.
Recomputed 32 sums, differences and shares; solved for the overlap implied by the exclusions and the R5/R6 sample sizes
Excess births over k=0..3 total 354,276 (247,215-476,998) across 1,332 names; R4 count-weighted boost 0.167
The reconstruction gives 219,000-282,000 excess births (claimed/implied = 1.26-1.62; 1.48 at 1.75 M births per sex-year). That is the same order of magnitude as the claimed 354,276 and overlaps its CI lower bound of 247,215. The result depends on the assumed birth totals and on the shape of the rate distribution above the 75th percentile, which is not reported. The figure is plausible but cannot be verified without the name-level data. Check 16 (CRAN copy of the SSA national file) gives 1.80 M (F) and 1.94 M (M) listed births per year in 1986-2017. That puts the relevant reconstruction at the N = 1.75-1.9 M rows: 239,000-282,000 implied, so the claim is 1.26-1.48 times the reconstruction.
Order-of-magnitude reconstruction: truncated lognormal fitted to the reported median and IQR of pre-release rates (Sec. 3.3), times an assumed 1.6-1.9 M listed births per sex per year, over 4 years, times exp(0.167)-1
State heterogeneity: fixed-effect mean 0.122, Q = 81.0 (df 49, p = 0.003), I^2 = 0.39, DL tau^2 = 0.0008, 9,146 name-state pairs; Q = 53.0 gives p = 0.321; West vs other 1,978 / 7,168
All 8 quantities are reproduced within tolerance: FE mean 0.1212 vs 0.122; Q 81.49 vs 81.0 (from rounded SEs); p 0.0024 vs 0.003; I^2 0.399 vs 0.39; tau^2 0.00085 vs 0.0008; df 49 (50 states with SE); sum n 9,146 exactly; p for Q = 53.0 is 0.321-0.324 given rounding. West = the 13 Census West states gives exactly 1,978 / 7,168, a definition the paper does not state. The state-level means (FE 0.121, n-weighted 0.123) sit well below the national primary estimate of 0.180, and the paper does not discuss this gap.
Inverse-variance meta-analysis recomputed from the Table 5 theta and SE (VT has no SE): FE mean, Cochran Q, chi-square p, I^2, DerSimonian-Laird tau^2; sum of n; West pair counts under alternative region definitions
Genre-family BH q-values (Table 4): only action/adventure survives (p 0.002, q 0.020); sci-fi/fantasy p 0.048, q 0.216
All 10 q-values are reproduced within 0.001 (tolerance 0.003): action 0.002*10/1 = 0.020; family, comedy, sci-fi and crime 0.216; horror and romance 0.506 vs 0.505; musical 0.5175 vs 0.517. Only action/adventure has q < 0.05. These contrasts are exploratory (X4). The abstract reports the action/adventure result (only action and adventure titles stood out) without flagging it as exploratory.
Step-up Benjamini-Hochberg over the 10 reported genre p-values
Reported p-values are consistent with their 95% bootstrap intervals (Tables 1 and 4)
No row flagged. Normal-approximation p agrees closely in most rows (H7 0.057 vs 0.0570; crime 0.076 vs 0.072; horror 0.354 vs 0.352), and every two-sided CI that excludes 0 has p < 0.05. The largest relative gap is X5 West-other, reported p 0.012 vs normal 0.0026. This is within the stated slack and plausible for a bootstrap-share p, but the CI and p for this contrast appear to come from different procedures. Intervals are near-symmetric (asymmetry 0.88-1.28).
Normal approximation with SE = (U-L)/3.92 and one- or two-sided p as stated in the table notes, for 22 rows; flagged factor-2 deviations and CI/p sign incoherence
3,092 major titles (2,148 films, 944 series) from the top 40 films and top 20 series per release year by sitelinks
The film and series counts imply different, non-integer numbers of years: 2,148/40 = 53.7 and 944/20 = 47.2. No window reproduces both; 1975-2025 (the control-pool window, 51 years) would give 2,040 films and 1,020 series. The counts may be explained by unfilled quotas, ties, or different windows, but the paper states neither the year range of the title list nor the Wikidata/Wikipedia snapshot date. The title list therefore cannot be reconstructed from the description. This is a data-provenance gap, not a demonstrated error.
Computed the number of years implied by the stated per-year quotas and compared with candidate windows
SSA national file has 1,316,981 name-sex-year rows for 1980-2025; state file 3,726,757 rows
Partial, plausibility-level check. Exact count for 1980-2017 is 1,060,844 rows. The claimed total implies 32,017 rows/yr for 2018-2025, vs 33,009/yr in 2013-2017 and 32,469 in 2017, which continues the observed slow decline (tolerance 10%). The national row count is therefore plausible. Rows for 2018-2025 and the state file (3,726,757 rows) could not be checked because no accessible copy was available. The paper gives no file checksums or SSA release dates; it says they are in the data documentation, which was not provided.
ssa.gov blocks scripted downloads (HTTP 403), so counted rows in the CRAN babynames 1.0.1 package (SSA national data to 2017) and extrapolated 2018-2025
Analysis plan frozen 10 Nov 2026 04:32:17 UTC before any post-release outcome (SHA-256 given); confirmatory analyses reported exactly as planned; two departures
The hash is a well-formed 64-hex SHA-256, but the frozen plan, deviation log and code were not provided (inputs contain only the PDF). The paper names no public registry (OSF, AsPredicted). The freeze time is on the same date as the paper. The freeze time, plan content, and the claim that confirmatory analyses are reported exactly as planned cannot be verified. Audit result: 4 of 15 headline claims are post hoc (P2 release-window 0.10; about half reflects trajectories; ~81 names; tens of names). The abstract labels P2 and the 81-name count as post hoc, but its closing sentence (about half ... trajectories), Sec. 1 and Sec. 5.5 state the post hoc result without that label. The genre result in the abstract is exploratory (X4) and unlabelled. Check 09 shows one unlisted analytic choice (BH over 9, not 10, hypotheses).
Audit script: checked hash format and which materials were provided; classified each headline claim in Abstract and Conclusions as preregistered confirmatory, preregistered exploratory or post hoc using the paper own labels
Load-bearing references exist and support the figures attributed to them: [9] ~10% for average-age names, ~62% for new names, none beyond ~31 years, 180/927/4,845 titles, 1967-2002; [10] Globo areas 18-24% more likely to have a novela name in the 1994 top 20; [7] 14% television; [4] tsunami curve.
43 of 43 references have a Crossref record, but the best match is a different work for [6] (FFT adders), [11] (family vacations), [17] and [18] (grey literature). [1] matched only a 2002 book review. Every spot-checked figure is in the source. [9] has 'falls to 10% for a name with the mean age', 'jumps by 62%', 'negative for ages over 31', '180 major movies, 927 broadcast TV shows, and 4845 popular songs' and '1967-2002'. [7] has 'by 14% of the parents'. [10] has the top-20 names of 1994 births and Globo coefficients 0.1835-0.244. Caveat for [10]: those coefficients are linear-probability effects in percentage points, on a base of 0.085 vs 0.329. The paper's '18 to 24% more likely' misstates the unit; the relative increase is 287% from the means. [11] exists (Studies in Popular Culture 38(2), 2016), but the web listing gives pp. 41-53 vs the cited 41-54. The [9] and [10] checks used working-paper versions, not the published versions.
Parsed work/refs_checked.json from tools/check_refs.py (43 refs). Regex search of open-access full texts downloaded on 2026-11-10: the Disdier-Head-Mayer working papers, the Bocconi 2008 working paper of La Ferrara et al., and Vandebosch 1998 from ans-names.pitt.edu. Evans 2007 checked on the ANS site search.
Minor. 14 of 15 checks match (e.g. exp(0.180)-1 = 19.72% vs 19.7%; 0.096 (0.052-0.143) rounds to 0.10 (0.05-0.14); 80.7 -> 81; n-weighted girls/boys mean 0.1798 vs 0.180; 0.216-0.121 = 0.095 vs 0.096). One mismatch: plain-language summary (Sec. 6) says fake dates give an apparent 12% boost, but 0.121 log points = 12.9% under the paper own conversion (tolerance 0.5 pp); 12% is the log value read as a percentage. Also two CIs for H1 ([0.147,0.214] Table 1 vs [0.148,0.211] elsewhere), disclosed in Sec. 4.2. The abstract word lasting is attached to the raw profile; the differenced estimate covers only k=0..3.
Recomputed exp(x)-1 conversions, rounding of quoted values and cross-section consistency
About half of the raw estimate reflects pre-existing trajectories (fake 0.121 vs true 0.216 on 1,001 names; within-name difference 0.096)
On the unweighted log scale the share is 0.47-0.56 (0.121/0.216 = 0.560; 1-0.096/0.180 = 0.467), so about half holds there. In the paper own birth counts it does not: fake-date excess births 14,170 vs true-date 240,444 = 0.059 (about 6%); excess names above chance per name (7.5/1001)/(80.7/1332) = 0.124. The trend-based adjustments go the other way: R10 implies -0.67 and R13 -0.09 (raw understates). The headline fraction therefore depends on the metric and the benchmark chosen, and the metric-dependence is not disclosed. The estimate behind it (P2) is post hoc (Sec. 3.8).
Computed the trajectory share implied by each metric the paper reports: log-scale means, percentage scale, excess births (Table 2 P2), excess names above chance (Table 2 P1), and the trend-based adjustments R10/R13 (Table 3)
Pre-release trajectories inflate the raw estimate, while leads show treated names falling relative to controls and trend adjustments R10 (0.301) and R13 (0.197) raise the estimate
Leads slope -0.0475/yr (leads only) or -0.036/yr (leads+baseline); the implied trend-adjusted boost is 0.306-0.346, compared with paper R10 = 0.301 and raw 0.180. The direction (raw understates) and approximate size of R10 are reproduced from the quoted leads. This is opposite in sign to the fake-date benchmark (0.096 < 0.180). The paper has two pre-trend benchmarks that disagree in sign. The abstract and conclusions adopt only the fake-date one and do not argue why.
Fitted linear trends to the quoted leads (0.171, 0.111, 0.076 at k=-6..-4), with and without the zero baseline, extrapolated to k=0..3 and computed the trend-adjusted boost
The true-minus-fake-date difference (P2) is the more conservative estimate and is robust to the trends visible in the data.
Illustrative only, because the DGP is ours. With random selection, A_raw = -0.001 and P2 = -0.003, so the estimator is unbiased. Under selection, P2 does not go to zero and its sign depends on how writers choose names. momentum_latent: A_raw 0.254, A_fake 0.078, P2 +0.176. momentum_observed: P2 -0.055. trough: P2 +0.114. rising_long: P2 -0.144. Only the trough rule reproduces the paper's lead pattern, positive and declining toward k=-4: simulated 0.324/0.242/0.158 vs the paper's digitised 0.172/0.112/0.077 (check 05). Under that rule P2 is biased upward with zero true effect. P2 therefore does not guarantee a conservative estimate. Its identifying assumption, that selection bias at the true date equals bias at the fake date, is neither stated nor tested in the paper. A_raw had the wrong sign under the trough rule, so the simulation does not reproduce the paper jointly. That is why it is not a discrepancy.
Monte Carlo (numpy, seed 20261110), zero true release effect, local-linear-trend log-rate series with AR(1) slope (rho 0.85), 30 release years x 25 names x 4 reps, 5-NN Mahalanobis matching on B and k=-3..-1 trend, fake date = R-8 with matching redone, under five writer-selection rules.
Figure 1 coefficients quoted in text: beta_0 0.090, beta_1 0.170, beta_5 0.295, beta_10 0.534; leads 0.171/0.111/0.076; placebo 0.012 at k=10; R13 lead -6 0.005 and k=10 0.578; fake-date 0.030 at fake k=0 and 0.750 at fake k=10; peak over k=0..5 at k=5
All 12 quoted values are reproduced within 0.002 (tolerance 0.02), e.g. beta_10 claimed 0.534 vs digitised 0.534, fake k=10 0.750 vs 0.748, R13 k=10 0.578 vs 0.579. The peak over k=0..5 is at k=5 (beta_4 0.285, beta_5 0.295). Additional finding: the fake-date curve rises 0.059/yr on average over fake k=1..7 (true -7..-1) and 0.101/yr over fake k=8..10 (true release years 0..2), a 1.7x acceleration at the true release. The text describes this curve only as also kept rising (Sec. 4.4, 5.3). This kink is itself evidence of a release-window change. It also suggests the P2 difference mixes a long pre-existing rise with a release-time acceleration.
Digitised the embedded raster of Fig. 1 by marker colour, with axes calibrated from tick marks detected in the image; also computed year-on-year increments of the fake-date curve
Mean held-out lead over k=-6..-4 is 0.103 (95% CI 0.079-0.127), with leads 0.171, 0.111, 0.076
The mean of the three quoted leads is 0.119 (digitised 0.120), against the claimed 0.103 (diff +0.016, tolerance 0.002). The numbers reconcile only if 0.103 is the mean over all 1,332 names while the quoted leads refer to the 1,172 balanced names. In that case the 160 omitted names (event years 2016-2020) would need a mean lead of -0.017. The text presents both figures in one sentence without saying the samples differ. Minor, but it affects the pre-trend diagnostic that motivated departure 1.
Arithmetic mean of the quoted leads and of the digitised Fig. 1a leads; solved for the lead mean that the 160 non-balanced names would need if 0.103 is the full-sample mean
Decay rate lambda = -0.115 from NLS fit for k = k_p..10 with k_p = 5; beta_10 exceeds the peak; half-life not identified
With beta_peak free: lambda = -0.1147 (SE 0.007), vs claimed -0.115 (tolerance 0.01). With beta_peak fixed at beta_5: -0.125. Log-linear: -0.119. beta_10/beta_5 = 1.81 >= 0.75, so the half-life is not identified under the preregistered rule. The computation is right. The wording in Sec. 4.4 (which places it beyond ten years) reads more into a not-identified result than the rule supports, given that the fake-date profile rises as well.
scipy curve_fit of beta_peak*exp(-lambda(k-5)) on the digitised beta_5..beta_10 from Fig. 1a (check 05), with beta_peak free, with beta_peak fixed, and log-linear OLS
The preregistered made-popular rule could not be met at this sample size: p floor about 0.002 with ~500 placebo stacks per decile; after BH over 1,332 names no q falls below 0.18
The quoted numbers are internally consistent: p floor 1/501 = 0.0020, and min q = 0.18 implies about 15 names beating every placebo in their decile. But the rule is not impossible at this sample size, contrary to the wording could not be met. It needs at least 54 names at the floor (or, e.g., all 146 P1 names each beaten by at most 1 of 500 placebos). The failure is a foreseeable power/discreteness problem of the planned placebo size. With about 20,000 or 50,000 stacks the requirement falls to 14 or 6 floor names. A pre-freeze power check (the plan already ran 579 synthetic stacks, Sec. 3.5) could have caught this. Research question 2 is therefore answered only post hoc.
Exact Benjamini-Hochberg arithmetic for m = 1,332 with p floor 1/501; number of floor-valued p needed; number of floor names implied by min q = 0.18; effect of larger placebo references
Table 1 BH q-values computed within the ten hypotheses: 0.003, 0.003, 0.003, 0.004, 1.000, -, 0.085, 0.182, 0.208, 0.005
Minor. All nine reported q-values reproduce exactly with m = 9 (e.g. H9 0.1859/8 = 0.208, H8 0.1429/7 = 0.183, H7 0.0579/6 = 0.0855, H10 0.0039/5 = 0.0054). With m = 10 as stated, H7 = 0.095, H8 = 0.203, H9 = 0.231 (3 mismatches beyond 0.001). The correction was applied over the nine testable hypotheses, with the untested H6 dropped. This is a reasonable choice, but it differs from the stated family of ten and is not listed as a departure from the plan. No accept/reject decision changes.
Recomputed step-up Benjamini-Hochberg q from the reported p-values (<0.001 taken as the 1/1000 bootstrap floor) with m = 9 and m = 10
[1] Lieberson (2000), A Matter of Taste, Yale University Press
Crossref matched only a 2002 book review (Language, doi:10.1353/lan.2002.0063). The book itself is well known and exists; no problem with the content attributed to it. Listed for completeness.
[23] Varnum and Kitayama, Psychological Science 22(2), 2011; [25] Candia et al., Nature Human Behaviour 3(1), 2019; [39] Chen and Roth, QJE 139(2), 2024
Minor: Crossref gives 2010, 2018 and 2023 (online-first dates) vs the cited print years. The volume/issue cited matches the print year, so this is not an error, only a date-convention difference.
k=4..10
Deposit the frozen plan, hash target, code and deviation log in a public registry with a DOI. Add a plan-vs-paper table that also records the m=9 BH family.
Add a not-yet-treated control design and report balance on held-out leads and longer pre-trends.
Rerun the preregistered 'made popular' rule with 20,000–50,000 placebo stacks per decile, or with a tail approximation, so that RQ2 has a confirmatory answer.
State the title-year window, the Wikidata query and snapshot time, the SSA file dates and checksums, and the West region definition in the paper itself.
Fix the internal inconsistencies: lead mean, the 39-name overlap, the Set B definition and n, 12% vs 12.9%, and the two H1 CIs.
Correct the description of [10] to percentage points, and replace or support the grey-literature comparison [17] in Sec. 5.2.
Add an authorship and accountability statement naming the system, its operator, and a contact for the preregistration and data.