Review by CheckPaper · Reviewed 16 Oct 2026

Testing black hole metrics with binary black hole inspirals

Name: Review: Testing black hole metrics with binary black hole inspirals
Rating: 3
Author: CheckPaper

Zhe Zhao, Swarnim Shashank, Debtroy Das, Cosimo Bambi

Read the PDF ↗arXiv 2510.04703 ↗

https://checkpaper.app/r/q7j9e3ntge

3/10

Major revision

16 claims · individually verified

A useful catalogue of metric-to-ppE mappings whose inputs reproduce exactly. However, a phase-sign mismatch, a wrong PN order for Kerr-Sen and untruncated percentiles mean that every number in Table 1 and Fig. 1 must be recomputed; only the qualitative 'consistent with Kerr' conclusion survives.

Soundness: 1/4
Presentation: 2/4
Contribution: 2/4
Confidence: 4/5

✓ 8 confirmed✕ 8 discrepancy

Summary

The paper derives the leading-order inspiral phase correction for twelve non-Kerr black-hole spacetimes: Kerr-Newman, Bardeen, asymptotically safe gravity (ASG), a loop-quantum-gravity (LQG) black hole, non-commutative geometry (NCG), Kerr-Sen, Ghosh-Kumar, rotating Kazakov-Solodukhin (RKS), Ghosh, Tinchev, rotating Einstein-Born-Infeld (EBI) and five Balart-Vagenas mass functions. It treats the lighter body as a test particle on a circular equatorial geodesic of the deformed metric (Sec. 2, Eqs. 4-10) and keeps the GR quadrupole flux (Eq. 13). The resulting stationary-phase correction is matched to the ppE/TIGER parameterisation (Eq. 17), so that each deformation parameter is a fixed multiple of one LVC phase-deviation posterior, (Eq. 29), with (1PN) or (2PN).

Download full report

Summary

Strengths

Transparent, uniform pipeline whose algebra checks out

Sec. 2.2-2.3, Eqs. (17)-(18), (29); Sec. 3.1-3.12 β/K pairs

Every metric goes through the same chain: $\Psi_{\rm NGR}$ to $\beta$ , $b$ and $K$ , with all intermediate coefficients printed. Readers can therefore redo each step. I re-derived all twelve $\beta \to K$ conversions, the $u$ exponents $b=(i-5)/3$ and the $\eta^{-i/5}$ powers with exact arithmetic, and all agree (check 12). This transparency is also what made the errors below detectable.

PN order and magnitude correct for 11 of 12 metrics

Eqs. (19)-(23), (33), (41), (50), (59), (67), (81), (89), (97), (104), (113), (120)

I expanded each metric at large areal radius and ran it through a pipeline calibrated on the GR $\varphi_2$ . For 15 metric/mass-function cases (all except Kerr-Sen), the PN order and the magnitude of the phase coefficient match exactly, e.g. Bardeen 225/64, LQG 5/8 and NCG $5/(6\sqrt\pi)$ (check 06). The PN coefficients - in Eqs. (19)-(23) equal the LALSimulation aligned-spin TaylorF2 coefficients identically (check 15).

Input posteriors are reproducible from public LVC data

Sec. 2.3 p. 5; Sec. 4.1 Eqs. (129)-(130)

The quoted GW170608 numbers reproduce to every printed digit from the public GWTC-1 TGR samples (SEOBNRv4_ROM): $\delta\hat\varphi_2=-0.09^{+0.24}_{-0.25}$ , and (check 04). The choice of GW170608 as the single most constraining GWTC-1 event is also correct (check 11). Its 90% width is 3.10, against 3.70-8.88 for the other events.

Useful side-by-side compilation of literature bounds

Table 1 (p. 19); closing paragraphs of Sec. 3.1-3.9

Table 1 puts GW, EHT and X-ray bounds for the same twelve spacetimes in one place. Of the literature values I traced to their sources, nearly all are transcribed correctly: Vagnozzi+ EHT bounds, Tripathi+ $r_2<0.011$ , Bambi $|g/M|<0.41$ , and the GW170608 eccentricity bounds 0.036, 0.12 and 0.166 (check 16). The exception is the ASG X-ray entry (see weaknesses).

Eccentricity systematic addressed with reproducible numbers

Sec. 4.1, Eqs. (125)-(130), pp. 21-22

Sec. 4.1 is a welcome addition: it gives explicit eccentric PN coefficients and quantifies the shift of the inferred posteriors. Evaluated per sample at $f_0=10$ Hz, the quoted $\varphi_2^{\rm ecc}=-0.0229$ , and the shifted posteriors , all reproduce (checks 09, 10).

Weaknesses

Sign convention mismatch flips every K and hence every bound

major · Sec. 2.2 p. 4 (SPA phase), Eq. (17), Eqs. (33)-(35) and all β/K pairs in Sec. 3; Table 1; Fig. 1

On p. 4 the SPA phase is written $\Psi=2\phi(t_0)-2\pi f t_0$ , the opposite sign to the TaylorF2/LAL convention in which the ppE mapping of Eq. (17) and the LVC $\delta\hat\varphi_i$ are defined (Newtonian term $+3/(128\eta v^5)$ ). I calibrated a symbolic pipeline by reproducing $\varphi_2=3715/756+55\eta/9$ exactly. In the LAL convention it gives $\Psi_{\rm NGR}=+\frac{5}{48}q^2u^{-1}\eta^{-2/5}$ for Kerr-Newman, so $K=+9/40$ , whereas Eq. (35) has $K=-9/40$ . Eq. (33) matches the derived term only in the paper's own SPA convention (check 01). The same factor $-1$ appears in all 15 cases (check 06).

Why it matters. 75% of the $\varphi_2\delta\hat\varphi_2$ samples and 70% of the $\varphi_4\delta\hat\varphi_4$ samples are negative (check 04). Flipping therefore moves most of each posterior out of the physical region. With the corrected sign and truncation to the physical range, the 90% upper limits are KN/EBI/BV (paper 0.48), LQG 0.046 (0.08), NCG 0.061 (0.11), Ghosh 0.139 (0.25), Bardeen 0.300 (0.52) and Ghosh-Kumar 0.918 (1.56). RKS, whose is positive, loosens from 0.42 to 0.883 (check 03). Every number in the GW column of Table 1 and every Fig. 1 panel changes. The qualitative "consistent with Kerr" conclusion survives, because zero stays inside every interval.

Fix. Use one phase convention throughout (preferably LAL), re-derive the signs of Eqs. (33)-(122) and recompute Table 1 and Fig. 1. A one-line GR calibration of the sign, as in check 01, would help readers.

Kerr-Sen deformation is a 1PN, not a 2PN, effect

major · Sec. 3.6, Eqs. (70)-(76); Table 1 Kerr-Sen row; Fig. 1 Kerr-Sen panel; Sec. 5

Written in the areal radius $R^2=r(r+r_\alpha m)$ , the $a =$ limit of Eqs. (71)-(72) is . This is the Reissner-Nordström form, so the correction enters at 1PN with on . The paper instead places it at 2PN with and on . An independent 50-digit computation of the exact circular-orbit energy confirms this: as , identical to Kerr-Newman at and . A 2PN effect would make this ratio vanish (check 02).

Reported intervals are untruncated percentiles, contradicting the stated prior truncation

major · Sec. 3.1-3.12 result paragraphs; Sec. 4 p. 19; Table 1 GW column; LaTeX source comments

Sec. 4 (p. 19) says negative values are excluded and only $0<\delta<\delta_{\rm extremal}$ is kept. Yet all 12 reported medians and upper ends equal the median and 95th percentile of the untruncated distribution, e.g. KN 0.134/0.489, quoted as $0.13^{+0.35}$ and . None of the 12 match the range-restricted posterior (check 03). Commented-out lines in the LaTeX source show the original two-sided intervals: KN , RKS and GK . Only the lower error bar was then overwritten with median. For RKS the negative median was replaced by "0.0", so is not a statistic of any distribution.

Figure 1 does not show what its caption and Sec. 4 describe

major · Figure 1 (p. 20), caption and panel labels; Sec. 4 p. 19

I parsed the 12 vector PDFs (checks 07, 08). Line widths do follow the caption: median 1.5 pt, interval 1.0 pt. However:

(i) Each violin body is an untruncated Gaussian KDE (max width deviation $\le1.9\times10^{-3}$ of the peak; a truncated KDE misses by 0.76-1.00), with the unphysical part hidden by white rectangles. Only 28-33% of the visible area lies below the thick "median" line.
(ii) In the RKS panel the median line sits at $b^2=-0.268$ , outside the axes, while the label says 0.0.

Data source and waveform model are misidentified

major · Sec. 2.3 p. 5; Sec. 4 p. 19 footnote; ref. [70]; Table 1 and Fig. 1 captions

The footnote on p. 19 and ref. [70] cite the GWTC-2 TGR release (zenodo 5172704), and the text and the captions of Table 1 and Fig. 1 name "SEOBNRv4P". The parametrized-test archive of that release (par.zip, 630 files) covers only 26 O3a events and has no GW170608 file (check 05). The quoted inputs instead reproduce exactly from the GWTC-1 TGR release LIGO-P1900087, whose README labels the samples SEOBNRv4_ROM (aligned spin, non-precessing). The IMRPhenomPv2 samples do not match (check 04). SEOBNRv4P is a different, precessing model. For reproducibility, cite the correct release and waveform.

Table 1 comparisons are not like-for-like; ASG X-ray entry is a different parameter

major · Table 1 (p. 19) and caption; Sec. 3.3 (source line 440), 3.4, 3.5, 3.8; Sec. 5

Table 1 mixes parameters and confidence levels. GW bounds are on $q^2$ , $g^2$ , $\sqrt\vartheta$ and , while EHT/X-ray bounds are on , , and . The GW column is at 90% (really a 95th percentile), EHT at 1σ/2σ, and X-ray at 90% or 3σ (check 13). Converted to common units, GW vs EHT , and GW is than EHT (1σ); the paper does not remark on the latter. The RKS EHT entry constrains "", which is never defined. The NCG EHT bound applies only to the Lorentzian NCG profile, and the bound is the LIRBH-2 value at (check 16).

Twelve constraints carry two pieces of information; single event; untested dissipative assumption

major · Eq. (29); Eq. (13) and p. 3; Sec. 1 last paragraph; Sec. 2.3; Sec. 5

By Eq. (29), every bound is a fixed rescaling of either $\varphi_2\delta\hat\varphi_2$ or $φ_{}$ from one event. The paper never says so explicitly, and it matters for how much the catalogue adds beyond the LVC paper and refs. [Cardenas-Avendano+2020, Shashank & Bambi 2022, Das+2024]. A rough 5-event combination of the GWTC-1 SEOB posteriors would narrow the intervals by about 0.57 (check 11, approximate). It is not attempted, although for universal couplings such as or it is natural.

Eccentricity estimate mixes absolute and fractional coefficients; e0 = 0.1 is not conservative

minor · Sec. 4.1, Eqs. (125)-(130), pp. 21-22

Eq. (125) adds $\varphi_i^{\rm ecc}$ , an absolute shift of the $v^i$ coefficient, to $δ {\hat{φ}}_{}$ , which is fractional. Under the consistent fractional prescription the shifts are only 3.8% and 0.3% of the medians, not 25% and 15% (check 09). The paper's numbers are thus an overestimate, which is harmless for the conclusion but inconsistent. On the other hand, is called conservative while the cited Wu+2020 bound is . At that value the shifts become 67% and 41% of the medians, still only about 26% and 13% of the 90% half-widths. The reference frequency Hz needed to reproduce Eq. (128) is not stated next to it (check 10). The text gives "~20%" after Eq. (129) and "~25%" after Eq. (130), and the dominant 0PN eccentric term is not discussed.

Bardeen 'regularity' rationale wrong; spin-dependent extremal bounds ignored

minor · Eqs. (32), (37), (73); Sec. 3.1, 3.2, 3.6

Eq. (37) says regularity requires $q\le\sqrt{16/27}\approx0.77$ . The threshold is right, but it is the condition for a horizon to exist. The Bardeen spacetime is regular for all : the Kretschmann scalar at is 6144, 180.6 and 8.43 for , 0.9 and 1.5 (check 14). Truncation bounds are taken at , although KN requires and Kerr-Sen . The effect is small for GW170608's low spins, but it should be stated.

Notation and bibliographic slips

minor · Eqs. (11), (22), (24), (29), (126)-(127); Sec. 3.8; ref. [61]

$\delta$ denotes both the deformation parameter (Eq. 29) and the mass difference in Eq. (22), which Eq. (24) calls $\xi$ .
Eq. (11), $\omega^2=L^2/r^4\simeq f(r,a,\delta)$ , is only the Newtonian relation.

Verification

I ran 16 scripted checks, all on disk under work/verify/. Eight confirmed the paper's numbers:

the LVC GW170608 inputs (to every printed digit);
the PN coefficients;
all β→K algebra;
the magnitudes and PN orders for 11 of 12 metrics;
the event choice;
the Table 1 unit conversions;
the eccentric phase shifts.

Eight found discrepancies. Four affect the main results:

a global sign mismatch between the paper's SPA convention and the ppE mapping, which flips every K (01);
the Kerr-Sen correction is 1PN, not 2PN (02);
every reported interval is an untruncated 5/50/95th percentile, contrary to the stated truncation (03), and Fig. 1 draws untruncated KDEs under white masks (07, 08);
the cited data release and waveform name do not match the samples actually used (05).

The other discrepancies are an X-ray bound in Table 1 attributed to the wrong parameter (16) and a wrong 'regularity' rationale for the Bardeen range (14), which changes no number.

The qualitative 'consistent with Kerr' conclusion survives. The quantitative content of Table 1 and Fig. 1 does not: corrected 90% upper limits are typically 40-45% smaller for 1PN metrics and about 2× larger for RKS.

8Confirmed

8Discrepancy

Kerr-Newman leading phase correction Psi_NGR = -5 q^2/(48 u eta^{2/5}) (Eq. 33) and mapping q^2 = -(9/40) phi_2 dphi_2 (Eq. 35) with beta = (3/128) phi_i dphi_i eta^{-i/5} (Eq. 17)DiscrepancySec. 2.2 p. 4 (SPA convention Psi = 2 phi(t0) - 2 pi f t0), Eqs. (13), (17), (33)-(35)
Calibration reproduces phi_2 exactly. Derived C = +40/9 (Psi_NGR = +5 q^2/(48 u eta^{2/5}) in the LAL convention), so q^2 = +(9/40) phi_2 dphi_2; paper Eq. (35) has K = -9/40. Magnitude agrees exactly (ratio 1), sign is opposite. The paper's Eq. (33) equals the derived term written in its own stated SPA convention (Psi = 2 phi - 2 pi f t, Newtonian term -3/(128 u^{5/3})), but Eq. (17) is the LAL-convention ppE mapping (Newtonian +3/128), so conventions are mixed and K has the wrong sign. With the GR flux in x instead of r(omega) |C| would be 40/27, so the magnitude also depends on the flux prescription; the paper's value corresponds to r(omega) in Eq. (13).
sympy: circular equatorial geodesic of f = 1 - 2m/r + q^2 m^2/r^2, E(x) and flux (32/5) eta^2 r(omega)^4 omega^6 (paper Eq. 13), SPA phase built in the TaylorF2/LAL convention; pipeline calibrated by reproducing phi_2 = 3715/756 + 55 eta/9 from the GR 1PN energy and flux exactly. Compared the convention-independent ratio Psi_NGR/Psi_Newtonian = C q^2 v^2 (i.e. phi_2 dphi_2 = C q^2) with the paper.
script: 01_sign_convention.py
Kerr-Sen deformation r_alpha enters at 2PN (b = -1/3) with Psi_NGR = -405 r_alpha/(128 u^{1/3} eta^{4/5}) and r_alpha = -(1/135) phi_4 dphi_4DiscrepancySec. 3.6, Eqs. (70)-(76); Table 1 row Kerr-Sen; Fig. 1 Kerr-Sen panel
(a) f(R) = 1 - 2m/R + r_alpha m^2/R^2 + 0/R^3 + ...: the deformation is Reissner-Nordstrom-like and enters at 1PN (i = 2) with |C| = 40/9, i.e. |K| = 9/40 on phi_2 dphi_2, not at 2PN with K = -1/135 on phi_4 dphi_4. With the literal static form of Eq. (70) (g_thth = r^2) the coefficient doubles (|C| = 80/9), still 1PN. (b) dE/dr_alpha / x^2 = -0.17446, -0.16742, -0.16674 for x = 1e-2, 1e-3, 1e-4 (a = 0), tending to -1/6, identical to Kerr-Newman dE/dq^2 to all printed digits (ratio 1.0) also at a = 0.3; a 2PN effect would make this ratio vanish as x -> 0. Hence the Kerr-Sen row (r_alpha < 0.58) should be derived from the 1PN posterior and coincides with the Kerr-Newman/EBI mapping; the X-ray comparison (0.011 vs 0.58) in Sec. 5 changes accordingly.
(a) sympy: g_tt of Eqs. (71)-(72) at a=0 re-expressed in the areal radius, then the calibrated pipeline of check 01; (b) independent mpmath (50 digits) computation of the exact prograde equatorial circular-orbit energy E(x) for Kerr-Sen and Kerr-Newman at a = 0 and a = 0.3, derivative w.r.t. the deformation at fixed x = (m Omega)^{2/3} for x = 1e-2, 1e-3, 1e-4.
script: 02_kerr_sen.py
GW constraints of Sec. 3 / Table 1 at 90% with negative values excluded (0 < delta < delta_extremal): KN q^2 = 0.13^{+0.35}_{-0.13} (<0.48), Bardeen 0.12^{+0.40} (<0.52), ASG 0.18^{+0.60} (<0.78), LQG 0.02^{+0.06} (<0.08), NCG 0.03^{+0.08} (<0.11), Kerr-Sen 0.13^{+0.45} (<0.58), GK 0.36^{+1.20} (<1.56), RKS 0.0^{+0.42} (<0.42), Ghosh 0.07^{+0.18} (<0.25), Tinchev 0.18^{+0.60} (<0.78), EBI and BV 0.13^{+0.35} (<0.48)DiscrepancySec. 3.1-3.12 result paragraphs; Sec. 4 p. 19; Table 1 GW column
All 12 reported medians and upper ends equal the median and 95th percentile of the UNTRUNCATED distribution (e.g. KN 0.134 [5%: -0.209, 95%: 0.489]; RKS median -0.268, 95th 0.418, reported as 0.0^{+0.42}); 0/12 match the physical-range posterior described in Sec. 4. The commented-out LaTeX values (e.g. KN 0.13^{+0.35}{-0.34}, RKS -0.27^{+0.69}{-0.71}, GK 0.36^{+1.20}_{-1.12}) equal the untruncated 5-50-95% values exactly; only the lower error was replaced by -median. Restricting to the physical range with the paper's K gives e.g. KN median 0.197, 90% UL 0.441, 95th 0.521; Kerr-Sen 0.231/0.527/0.640; GK 0.623/1.422/1.727; RKS keeps only 24.9% of samples (median 0.204, 90% UL 0.555). With the sign corrected (check 01) only ~25-30% of each 1PN/2PN posterior is physical and the 90% upper limits become e.g. KN/EBI/BV 0.28, LQG 0.046, NCG 0.061, Ghosh 0.139, Bardeen 0.300, ASG 0.455, Tinchev 0.459, GK 0.918, while RKS loosens to 0.883 and Kerr-Sen (1PN) 0.277. Restriction by sample rejection approximates a prior truncation because the LVC prior is uniform in dphi_i.
delta = K phi_i dphi_i per GW170608 SEOBNRv4_ROM sample with the paper's K; percentiles of (a) the untruncated distribution and (b) the samples restricted to the paper's physical range (0 < delta < delta_max), plus (c) the same with the sign of K reversed per check 01 (and the 1PN mapping for Kerr-Sen per check 02). Tolerance 0.01.
script: 03_table1_intervals.py
GW170608 inputs: dphi_2' = -0.09^{+0.24}{-0.25}, dphi_4' = -0.41^{+1.29}{-1.38}, phi_2 dphi_2 = -0.60^{+1.52}{-1.58}, phi_4 dphi_4 = -17.93^{+55.92}{-60.11}ConfirmedSec. 4.1, Eqs. (129)-(130), p. 22
SEOB: dphi_2 = -0.0942 +0.2411 -0.2469; phi_2 dphi_2 = -0.5951 +1.5230 -1.5777; dphi_4 = -0.4134 +1.2912 -1.3812; phi_4 dphi_4 = -17.9302 +55.9224 -60.1106 - all match the quoted values to the printed digits. The LVC Table V pickle gives SEOB dphi_2 = -0.09 (-0.25, +0.24), dphi_4 = -0.4 (-1.4, +1.3). The IMRPhenomPv2 samples do not match (phi_2 dphi_2 = -0.43 +1.35 -1.24), so the paper used the SEOBNRv4_ROM GWTC-1 samples. 75.2% of phi_2 dphi_2 and 70.1% of phi_4 dphi_4 samples are negative.
Recomputed medians and symmetric 90% intervals from the public LVC GWTC-1 TGR samples (LIGO-P1900087 PGen.tgz, GW170608_GWTC-1-TGR.hdf5, SEOB = SEOBNRv4_ROM datasets, 20071 and 17428 samples), phi_2 and phi_4 per sample from Eqs. (21), (23); cross-checked with the LVC Table V pickle and the IMRPhenomPv2 datasets. Tolerance 0.01 (0.02 for phi_4 dphi_4).
script: 04_lvc_inputs.py
Constraints use the publicly released 'Tests of GR with BBHs from GWTC-2 - Full Posterior Sample Data Release' (zenodo 5172704) for GW170608 with the SEOBNRv4P waveform modelDiscrepancySec. 4 p. 19 (footnote with zenodo link), ref. [LVC-web]; Sec. 2.3 p. 5; Sec. 3 result paragraphs; Table 1 and Fig. 1 captions
Record 5172704 (doi 10.7935/903s-gx73) holds imr, liv, rin, sim, par, ech zips. par.zip has 630 members covering 26 O3a events (S190408an ... S190924h) and no GW170608/S170608 file; the only GW170608 files in the release are ech_S170608_IMR(E).h5 (echo search) and sim_S170608_pv2_* (spin-induced quadrupole test). The paper's inputs reproduce exactly (check 04) from the GWTC-1 TGR release (LIGO-P1900087), whose README labels the samples SEOBNRv4_ROM (aligned spin); the string 'SEOBNRv4P' does not occur. The data source and waveform name in the paper should be corrected (SEOBNRv4P is a different, precessing model).
Queried the Zenodo API for record 5172704, listed the par.zip (parametrized tests) central directory live by HTTP range requests and searched member names for 170608; searched the step-1 listings of all six zips; read the waveform label in the GWTC-1 TGR release README (LIGO-P1900087).
script: 05_provenance.py
Magnitudes and PN orders of the phase-correction coefficients: Bardeen 225/64 (2PN), ASG 75/32 (2PN), LQG 5/8 (1PN), NCG 5/(6 sqrt(pi)) (1PN), Ghosh-Kumar 75/64 (2PN), RKS 5/96 (1PN), Ghosh 5/24 (1PN), Tinchev 75/32 (2PN), EBI 5/48 (1PN), all five Balart-Vagenas mass functions 5/48 (1PN), KN 5/48 (1PN)ConfirmedSec. 3.1-3.12, Eqs. (33), (41), (50), (59), (67), (81), (89), (97), (104), (113), (120)
For all 15 metric/mass-function cases other than Kerr-Sen the PN order and the magnitude agree exactly: C_derived/C_paper = -1 in every case (e.g. Bardeen f = 1 - 2/R + 3q^2/R^3 gives |C| = 150 = (128/3)(225/64); LQG f = 1 - 2/R + 6 A_lambda/R^2 gives |C| = 80/3; NCG 8 sqrt(theta/pi)/R^2 gives |C| = 320/(9 sqrt(pi)); RKS -b^2/2R^2 gives |C| = 20/9). The uniform factor -1 is the sign issue of check 01 (not counted again here). All five Balart-Vagenas mass functions reduce to f = 1 - 2/R + q^2/R^2, identical to Kerr-Newman and EBI. Kerr-Sen is treated in check 02. Spin cross-terms were not examined (a = 0 only).
sympy: for each metric at a = 0, large-R expansion of g_tt in the areal radius (ADM-normalised, linear in the deformation; LQG with M_b = M_w; NCG with exact atan series; EBI with the 2F1 series; each of the five Balart-Vagenas mass functions separately), then the calibrated geodesic + SPA pipeline of check 01 with the r(omega) flux. Compared C = (128/3) beta to the paper as exact ratios.
script: 06_phase_coeffs.py
Figure 1 violin plots show the posteriors, with 'the thick horizontal line mark[ing] the median and the thin horizontal line mark[ing] the 90% credible interval', consistent with the quoted valuesDiscrepancyFigure 1 (p. 20), caption and panel labels; Sec. 3.1-3.12 result paragraphs
The line widths follow the caption in all 12 panels (median 1.5 pt, interval 1.0 pt). The line values match the quoted numbers to within 0.009, e.g. KN 0.1339/0.4889 against 0.13/0.48 and Ghosh-Kumar 0.3586/1.5608 against 0.36/1.56. Three problems were found. (1) The lines are the 5/50/95th percentiles of the untruncated samples: the full body has 0.498-0.501 of its area below the thick line. The visible body, which is cut at the physical range by white rectangles, has only 0.28-0.33 of its area below the 'median' line. A reader therefore sees a 'median' that is not the median of the distribution drawn. (2) RKS panel: the thick median line sits at b^2 = -0.2678, outside the visible axes [0, 1.5], while the label and text say 0.0. (3) Ghosh-Kumar panel label reads 'g = 0.36^{+1.20}_{-0.36}', but the constrained quantity is g^2 (Eq. 83, Table 1). The lower thin lines lie at negative values, off-axes (e.g. KN -0.2088); for 5 panels they are off-page and could not be recovered.
Decompressed the 12 matplotlib vector PDFs of Figure 1 and parsed the path operators and line widths. Recovered the data-to-page mapping from tick marks and decoded tick labels (fit residual <= 2e-9), then measured the y-value and stroke width of each horizontal line and the share of violin area below each line, for the full polygon and for the visible (white-masked) part. Decoded the panel-label text and compared it with the text and Table 1.
script: 07_fig1_lines.py
Figure 1 panels are the range-restricted posteriors of the deformation parameters derived from the GW170608 phi_i dphi_i posteriors via Eq. (29)DiscrepancyFigure 1 (p. 20) and caption; Sec. 4 p. 19 ('we exclude any negative values and only consider 0 < delta < delta_extremal')
The 12 panels contain only two distinct shapes, one for i = 2 and one for i = 4. Outlines agree to <= 1e-8 of the peak after rescaling by K, as Eq. (29) implies, so EBI, BV and KN are the same figure. The untruncated KDE reproduces every body: its support matches exactly and the maximum width difference is <= 1.9e-3 of the peak width, only at the tail tips. The truncated KDE misses by 0.76-1.00. The thick and upper thin lines equal the untruncated median and 95th percentile to within 2e-9. The physical range is imposed only by white masks over the drawn body, and KN and ASG are masked rather than cut at their upper bound of 1. A truncated posterior would have a different median/95th percentile: KN 0.197/0.522 instead of 0.134/0.489, Bardeen 0.199/0.487, Kerr-Sen 0.231/0.640, Ghosh-Kumar 0.623/1.727, and RKS 0.204/0.666 with only 24.9% of samples kept. The figure therefore does not show the range-restricted posterior that Sec. 4 says is used. This is the same root issue as check 03, seen in the figure.
Using the polygons extracted by check 07, rescaled every violin outline by 1/|K|, normalised it to unit area and compared panels pairwise. Regenerated Scott-bandwidth Gaussian KDEs of K phi_i dphi_i from the GWTC-1 TGR SEOB samples (check 04), both untruncated and truncated to the physical range, and compared their widths with the plotted widths at the plotted y-nodes. Compared the line positions with untruncated and truncated percentiles.
script: 08_fig1_shapes.py
Eccentricity (e0 = 0.1, evaluated at 20 Hz) shifts phi_2 dphi_2 from -0.60^{+1.52}{-1.58} to -0.45^{+1.52}{-1.57} and phi_4 dphi_4 from -17.93^{+55.92}{-60.11} to -15.30^{+55.93}{-60.00}; relative contributions '~20%' (later '~25%') and '~15%'; eccentricity is subdominant.ConfirmedSec. 4.1, Eqs. (125), (128)-(130), p. 22; Abstract; Conclusions
Prescription (a) reproduces Eq. (130) within tolerance: -0.451 +1.524 -1.575 (claimed -0.45 +1.52 -1.57) and -15.301 +55.932 -60.005 (claimed -15.30 +55.93 -60.00). The relative shifts are 0.0229/0.09 = 25.4% and 0.0617/0.41 = 15.0%, so the '~20%' after Eq. (129) contradicts the '~25%' after Eq. (130); this is minor. Methodological caveats are not counted as a discrepancy. (1) Eq. (125) adds phi_i^ecc, an absolute shift of the v^i coefficient, to dphi_i, which is a fractional deviation. Under the fractional prescription (b) the shifts are only 3.8% and 0.3% of the medians, so the paper's estimate is conservative by a factor of phi_i (about 6.4 and 44). (2) Under the paper's own prescription, the 'observationally allowed range' quoted in the abstract includes e0 = 0.166 (Wu+2020). That value gives shifts of 66.6% (phi_2) and 40.8% (phi_4) of the median, which are still only 25.6% and 12.6% of the 90% half-widths. So 'subdominant relative to statistical uncertainty' holds, but '~25%' is not an upper bound over the allowed range. (3) The interpretation of e0 as defined at f0 = 10 Hz is taken from check 10.
Per-sample recomputation on the GW170608 SEOBNRv4_ROM posteriors (GWTC-1 TGR release, check 04), with phi_i from Eqs. (21), (23) and phi_i^ecc from Eqs. (126)-(127) at chi = 2. Two prescriptions: (a) the paper's Eq. (125), dphi_i = dphi_i' - phi_i^ecc; (b) a fractional shift dphi_i' - phi_i^ecc/phi_i, which is consistent with the LAL ppE definition (coefficient phi_i(1 + dphi_i)). Repeated for e0 = 0.036, 0.12 and 0.166. Tolerance 0.01 (phi_2 product) and 0.02 (phi_4 product).
script: 09_ecc_shift.py
phi_2^ecc = -0.0229^{+0.0007}{-0.0001} and phi_4^ecc = -0.0617^{+0.0049}{-0.0007} at f = 20 Hz, e0 = 0.1 (Eqs. 126-128)ConfirmedSec. 4.1, Eqs. (126)-(128), p. 21-22
At f0 = 10 Hz the script obtains phi_2^ecc = -0.0229 +0.0007 -0.0001 and phi_4^ecc = -0.0617 +0.0049 -0.0007, an exact match to 4 decimals. Other f0 values do not match (f0 = 20 Hz gives -0.1077 and -0.2996). The reference frequency f0 = 10 Hz is therefore implied but never stated next to Eqs. (126)-(128); only the eccentricity bounds are said to be 'at the reference frequency of 10 Hz'. None of the 26 coefficients (e.g. 2045665/348096, 5795368945/350880768) appears literally in the Moore et al. 2016 source; only the 0PN numbers 2355 and 1462 do. The paper's coefficients are presumably re-expanded or combined forms rather than copies, so the transcription itself remains unverified. This check confirms only the evaluation.
Evaluated Eqs. (126)-(127), as transcribed in the paper, per GW170608 SEOB posterior sample (eta from detector-frame masses) at chi = f/f0 = 20/10. Reported the median and 5th/95th percentiles, and scanned f0 over 5-20 Hz. Downloaded the arXiv source of Moore et al. 2016 (1605.00304) and searched it, ignoring whitespace and braces, for each of the 26 rational coefficients printed in Eqs. (126)-(127).
script: 10_ecc_coeffs.py
GW170608 'has a long detected inspiral phase and provides the most stringent constraints on possible deviations from the Kerr metric'ConfirmedSec. 2.3, p. 5
Five events carry the SEOB dphi_2/dphi_4 datasets. GW170608 gives the narrowest interval among them: the phi_2 dphi_2 width is 3.101 against GW150914 3.701, GW151226 4.125, GW170814 4.955 and GW170104 8.876; the phi_4 dphi_4 width is 116.0 against 127.3-305.6. The single-event claim is therefore confirmed. However, the approximate 5-event combination would narrow the intervals by a factor of about 0.57 (phi_2 dphi_2 width 1.77) and 0.56 (phi_4 dphi_4 width 65.1). This is relevant if the deformation parameter is taken to be universal (e.g. a fixed coupling); the paper does not discuss stacking. The combination is approximate (KDE product with no prior removal, and GW150914 alone excludes zero at 90% for phi_2 dphi_2), so it is offered as context and not as a result.
For every GWTC-1 event in the LVC GWTC-1 TGR release (PGen.tgz) that has dphi_2/dphi_4 SEOB posteriors, computed phi_i dphi_i per sample and the width of its 5th-95th percentile interval, then ranked the events. Also formed an approximate multi-event combination: the product of Gaussian KDEs on a common grid, with no prior division, assuming one universal deformation parameter.
script: 11_event_choice.py
K constants (delta = K phi_i dphi_i): KN -9/40, Bardeen -1/150, ASG -1/100, LQG -3/80, NCG -9 sqrt(pi)/320, Kerr-Sen -1/135, Ghosh-Kumar -1/50, RKS +9/20, Ghosh -9/80, Tinchev -1/100, EBI -9/40, BV -9/40, obtained by equating beta with Eq. (17); b = (i-5)/3ConfirmedEqs. (17)-(18), (29) and the beta/K pairs (34)-(35), (42)-(43), (51)-(52), (60)-(61), (68)-(69), (75)-(76), (82)-(83), (90)-(91), (98)-(99), (105)-(106), (114)-(115), (121)-(122)
All 12 rows pass all four tests, e.g. (3/128)/(-5/48) = -9/40, (3/128)/(-225/64) = -1/150 and (3/128)/(-5/(6 sqrt(pi))) = -9 sqrt(pi)/320. Eq. (17) with u = eta^{3/5} pi m f gives eta exponents -2/5 and -4/5, as used. The algebra from beta to K is therefore correct; the errors in K come from the sign of beta (check 01) and the Kerr-Sen PN order (check 02), not from this step.
Exact sympy arithmetic on coefficients transcribed from draft3.tex; each is re-grepped on its source line. Compared Psi_NGR with beta, the u power with b = (i-5)/3 and the stated b, the eta power with -i/5, and K with (3/128)/beta.
script: 12_K_constants.py
Table 1 cross-column conversions and comparisons: Kerr-Sen EHT q/m < 0.6, 0.75 written as sqrt(r_alpha) < 0.6, 0.75; Ghosh EHT g/m < 0.8, g < m written as h < 0.32, 0.5 via h = g^2/2m^2; Bardeen sqrt(16/27) ~ 0.77; X-ray bounds stronger than GW where available; 'constraints from both methods are consistent with our GW results'ConfirmedTable 1 (p. 19) and caption; Sec. 3.6, 3.8, 3.9 last paragraphs (source lines 570, 651, 697); Conclusions p. 22
The conversion arithmetic is correct: sqrt(r_alpha) < 0.6 is equivalent to r_alpha < 0.36; g/m < 0.8 and g < m give h < 0.320 and h < 0.500; sqrt(16/27) = 0.7698. The Conclusions' statement that X-ray bounds are stronger holds: ASG 0.05 vs 0.78, Kerr-Sen 0.011 vs 0.58, Bardeen q < 0.41 -> q^2 < 0.168 vs 0.52. The following presentation problems, all minor, are relevant to the requester's focus on the table. (1) The rows mix parameters (q vs q^2, g vs g^2, theta vs sqrt(theta), sqrt(r_alpha) vs r_alpha) and confidence levels: GW 90%, EHT 1 sigma/2 sigma, X-ray 90% or 3 sigma. Readers must convert to compare. In common units, GW q^2 < 0.48 is q < 0.693 against EHT q < 0.8; GW g^2 < 1.56 is g < 1.249 against EHT g < 1.4; GW sqrt(theta) < 0.11 is theta < 0.0121, tighter than EHT theta < 0.02 (1 sigma), which the paper does not remark on. (2) In the RKS row the EHT column constrains 'l', which is never defined; it appears only at source line 651 and in Table 1, while the GW column constrains b^2. (3) The GW column inherits the sign error and the untruncated percentiles (checks 01-03). With the corrected values the GW bounds become KN q^2 < 0.276, Kerr-Sen r_alpha < 0.277, Ghosh h < 0.139, NCG sqrt(theta) < 0.061 and LQG A_lambda < 0.046, which changes every GW-vs-EHT comparison in the table. (4) EHT bounds refer to Sgr A* and GW bounds to a stellar-mass binary; 'consistent' implicitly assumes a universal dimensionless parameter, which is not discussed.
Arithmetic on the Table 1 and Sec. 3 numbers using the paper's stated relations r_alpha = q^2/m^2 and h = g^2/2m^2. Put GW and EHT bounds in the same parameter for KN, GK and NCG. Recomputed the sign-corrected GW 90% upper limits (checks 01-03) for comparison. Grepped the source for a definition of 'l'.
script: 13_table1_conversions.py
Physical ranges: KN a^2 + q^2 m^2 < m^2 (Eq. 32); Bardeen 'regularity requires' q <= sqrt(16/27) ~ 0.77 (Eq. 37); ASG 0 < omega~ < 1 'where BHs have two event horizons' (Eq. 46); Kerr-Sen 0 < r_alpha < 2 (Eq. 73)DiscrepancyEqs. (32), (37), (46), (73) (source lines 312, 356, 413, 555); truncation used for Table 1 and Fig. 1
Every numeric threshold is correct: KN q^2 = 1.000000 at a = 0 and 0.910000 at a = 0.3; Bardeen q = 0.769800 = sqrt(16/27); ASG omega~ = 1.000000, since at gamma = 0 the metric reduces exactly to f = 1 - 2r/(r^2 + omega~), RN-like; Kerr-Sen r_alpha = 2.0 at a = 0 and 1.4 = 2(1-a) at a = 0.3. The justification of Eq. (37) is wrong, however. The Bardeen spacetime is regular for every q: the Kretschmann scalar tends to 6144, 32000000/177147 and 2048/243 for q = 0.5, 0.9 and 1.5. q <= sqrt(16/27) is the condition for a horizon to exist, not for regularity. Above the threshold the object is a horizonless regular spacetime, not a singular one, so the stated rationale for truncating the posterior is incorrect, although the bound itself is the right one for a black hole. The spin dependence of the bounds (KN q^2 < 1 - a^2, Kerr-Sen r_alpha < 2(1 - a)) is ignored in the truncation. The effect is small for GW170608's low spins, and the a = 0 bounds are used. This discrepancy is minor and does not change any reported number.
For m = 1, counted horizon roots on a log grid from 1e-4 to 1e2 with 2e5 points while bisecting the deformation parameter for the value at which horizons disappear. Cases: KN and Kerr-Sen at a = 0 and a = 0.3; Bardeen; ASG with gamma = 0. Evaluated the Bardeen Kretschmann scalar as r -> 0 (sympy limit and r = 1e-1, 1e-3, 1e-5) for q = 0.5, 0.9 and 1.5.
script: 14_horizons.py
PN phase coefficients phi_2, phi_3, phi_4 in Eqs. (19)-(23), TaylorF2/PhenomD form including aligned-spin termsConfirmedEqs. (19)-(24), p. 5 (source lines 232-237)
phi_2 = 3715/756 + 55 eta/9 is reproduced exactly. phi_3 and phi_4 equal the LAL aligned-spin TaylorF2 coefficients identically (difference 0). The LAL non-spinning 4PN term 5(3058.673/7.056 + 5429/7 eta + 617 eta^2)/72 equals 15293365/508032 + 27145 eta/504 + 3085 eta^2/72, and the spin-orbit term x_i(25 + 38 x_i/3) equals the paper's 113/3 delta chi_a + (113/3 - 76 eta/3) chi_s. Spot values at eta = 0.2427, chi_s = 0.03, chi_a = 0.01: phi_2 = 6.3972, phi_3 = -49.256, phi_4 = 45.648. One minor notation issue: Eq. (22) uses delta for the mass difference, which Eq. (24) defines only as xi, while delta also denotes the deformation parameter in Eq. (29).
sympy: (1) derived phi_2 from E_2 and F_2 via psi_2 = (20/9)(2E_2 - F_2). (2) Rebuilt phi_3 and phi_4 from the individual-spin TaylorF2 coefficients of LALSimulation (LALSimInspiralPNCoefficients.c, downloaded from git.ligo.org; the script confirms each constant string is present in the file) for aligned spins with kappa_i = 1, rewrote them in chi_s, chi_a and xi, and compared them symbolically with the paper. (3) Spot-evaluated at eta = 0.2427.
script: 15_pn_coeffs.py
Literature bounds quoted in the last paragraph of each Sec. 3.x and in Table 1 (EHT and X-ray columns), plus the GW170608 eccentricity bounds in Sec. 4.1, are reproduced correctly from the cited works.DiscrepancySec. 3.1-3.9 closing paragraphs (source lines 335, 440, 487, 528, 570, 604, 651, 697); Table 1 (p. 19, source lines 899-913); Sec. 4.1; refs [23] Vagnozzi:2022moj, [41] Zhou:2020eth, [45] Afrin:2022ztr, [38] Bambi:2014nta, Tripathi 2103.07593, Romero-Shaw 1909.05466, O'Shea 2107.07981, Wu 2002.05528, [61] Tinchev
Most quoted numbers match their sources exactly (string match):

Vagnozzi gives Q < 0.8M and 0.95M (RN), omega~ < 0.9M^2 at 1 sigma (ASG, metric A = 1 - 2Mr/(r^2+omega~)), and theta < 0.02M^2 and 0.04M^2 (this applies to Lorentzian NCG only; Gaussian NCG is unconstrained). It also gives q_m < 0.6M and 0.75M (Sen), |g| < 0.8M and M (Ghosh/GCSV), |ell| < 0.2M and M (Kazakov-Solodukhin), and k < 1.4M and 1.6M (Ghosh-Kumar, A = 1 - 2M/sqrt(r^2+k^2)).

Afrin: L_q in [0, 0.0821) at 1 sigma and [0, 0.1834) at 2 sigma at theta_0 = 0 deg, for the second LIRBH model (the abstract reads 'L_q < 0.0423 and 0.0821 for the two LIRBHs').

Tripathi: r_2 < 0.011 (90% CL) with Delta = r(r+r_2) - 2Mr + a^2, matching r_alpha.

Bambi: |g/M| < 0.41 (and a stricter < 0.28, not quoted).

Romero-Shaw: GW170608 0.036. O'Shea: GW170608 < 0.12. Wu Table 4: GW170608 e^90_max(10 Hz) = 0.166.

DISCREPANCY (ASG X-ray, source line 440 and Table 1): the paper states 'omega~ < 0.05 at the 90% of confidence level [Zhou:2020eth]' for the Bonanno-Reuter omega~ (gamma = 0). Zhou et al. constrain a different parameter in a different metric: the Held et al. modified Kerr metric with G_N(k) = G_0/(1+gamma G_0 k^2), k^2 = G_0 M r^3/rho^6, giving 'gamma~ < 0.047 at 90% confidence'. Vagnozzi, the paper's other source, quotes the same result as 'omega~ < 0.05 at 1 sigma'. So both the parameter identification and the confidence level of the 0.05 X-ray entry are inconsistent with the cited work. The Conclusions claim (source line 1064) that X-ray bounds are consistent with and stronger than the GW bounds therefore rests on a parameter mismatch in the ASG row.

Smaller issues:

The paper's RKS 'l' (line 651) is Vagnozzi's ell and is undefined in the paper.

The A_lambda EHT bound (line 487) is the LIRBH-2 bound at theta_0 = 0 deg. Neither the model, the parameter map to A_lambda, nor the inclination is stated; at theta_0 = 90 deg the bounds are 0.1170 and 0.2566.

The NCG EHT bound holds only for the Lorentzian NCG metric.

The Tinchev DOI 10.6122/CJP.20150810 returns 404 on Crossref. arXiv 1512.09164 gives the journal ref as 'Chin. J. Phys. 53, 110113 (2015)'.
Downloaded the arXiv source of each cited paper (Wu: PDF via pdftotext) and printed the exact sentence or table row holding each quoted number, together with the metric definitions, so parameters can be compared. Resolved the Tinchev DOI on Crossref and doi.org, and looked up the arXiv journal-ref.
script: 16_ref_spotcheck.py

References

90 references checked

[41] Zhou:2020eth (Zhou et al., arXiv:2005.12958), cited at Sec. 3.3 (source line 440) and Table 1 ASG row

Cited for 'omega~ < 0.05 at the 90% confidence level' on the Bonanno-Reuter ASG parameter omega~ (gamma = 0 metric). Zhou et al. instead constrain gamma~ < 0.047 (90%) in the Held et al. modified Kerr metric, with G_N(k) = G_0/(1+gamma G_0 k^2) and k^2 = G_0 M r^3/rho^6. This is a different parameter and metric. Vagnozzi:2022moj quotes the same result as 'omega~ < 0.05 at 1 sigma'. The parameter identification and the confidence level do not match the cited work (check 16).

[45] Afrin:2022ztr (arXiv:2209.12584), cited at Sec. 3.4 (source line 487) and Table 1 LQG row

The quoted A_lambda < 0.0821 (1 sigma) and 0.1834 (2 sigma) are the L_q bounds for the second loop-inspired regular BH (LIRBH-2) at inclination theta_0 = 0 deg. At theta_0 = 90 deg they are 0.1170 and 0.2566, and LIRBH-1 gives 0.0423. The paper states neither the model, the inclination, nor the mapping from L_q to its A_lambda.

[23] Vagnozzi:2022moj (arXiv:2205.07787), cited throughout Sec. 3 and Table 1

The quoted numbers are reproduced correctly, but some are taken out of context:

The RKS bound 'l < 0.2, 1' (source line 651) refers to Vagnozzi's Kazakov-Solodukhin parameter ell, which the paper never defines.
The NCG bound theta < 0.02, 0.04 holds only for the Lorentzian NCG metric; the Gaussian NCG is unconstrained.
The Ghosh-Kumar bound is on k (A = 1 - 2M/sqrt(r^2+k^2)), but Table 1 places it next to the GW bound on g^2 without converting.

[61] Tinchev (Chin. J. Phys., DOI 10.6122/CJP.20150810)

Crossref returns 404 for this DOI, and tools/check_refs.py could not find it. arXiv 1512.09164 gives the journal reference as 'Chin. J. Phys. 53, 110113 (2015)', so the page number in the bibliography should be checked.

[70] LVC-web data release (zenodo 5172704, 'Tests of GR with BBHs from GWTC-2')

Cited as the source of the GW170608 parametrized-test posteriors, but this release has no GW170608 parametrized-test samples. The values used match the GWTC-1 TGR release (SEOBNRv4_ROM, PGen), not GWTC-2 or 'SEOBNRv4P' (checks 04 and 05).

[90] Moore:2016qxz (Moore et al., arXiv:1605.00304), cited for the eccentric phase coefficients (Eqs. 126-127 in the paper's appendix)

None of the 26 transcribed numerical coefficients appears literally in the cited source; only 2355 and 1462 are found. The transcription is therefore unverified. The quoted phi^ecc values are reproduced only with f0 = 10 Hz, while the text says 20 Hz (checks 09 and 10).

[82] Wu:2020zwr (arXiv:2002.05528), cited in Sec. 4.1

The cited GW170608 bound e^90_max = 0.166 at 10 Hz is correct (Table 4). However, the paper's 'conservative' choice e0 = 0.1 is below this bound, so it is not conservative with respect to the reference the paper itself cites. At e0 = 0.166 the shifts are 66.6% and 40.8% of the medians (check 09).

[38] Bambi:2014nta (arXiv:1401.4640), cited at Sec. 3.2 and Table 1 Bardeen row

|g/M| < 0.41 is quoted correctly (3 sigma, from a* > 0.95). The same paper also gives the stricter |g/M| < 0.28 (from a* > 0.983), which the paper does not mention (minor).

Testing black hole metrics with binary black hole inspirals

Summary

Download full report

Testing black hole metrics with binary black hole inspirals

Summary

Download full report

Strengths

Transparent, uniform pipeline whose algebra checks out

PN order and magnitude correct for 11 of 12 metrics

Input posteriors are reproducible from public LVC data

Useful side-by-side compilation of literature bounds

Eccentricity systematic addressed with reproducible numbers

Weaknesses

Sign convention mismatch flips every K and hence every bound

Kerr-Sen deformation is a 1PN, not a 2PN, effect

Reported intervals are untruncated percentiles, contradicting the stated prior truncation

Figure 1 does not show what its caption and Sec. 4 describe

Data source and waveform model are misidentified

Table 1 comparisons are not like-for-like; ASG X-ray entry is a different parameter

Twelve constraints carry two pieces of information; single event; untested dissipative assumption

Eccentricity estimate mixes absolute and fractional coefficients; e0 = 0.1 is not conservative

Bardeen 'regularity' rationale wrong; spin-dependent extremal bounds ignored

Notation and bibliographic slips

Questions for the authors

Verification

References

Suggestions

Scope of this review