# HOLDOUT_PROTOCOL.md

## Purpose

This file locks the journal-facing validation protocol for the RBFT Galaxy Phase Tuner reproducibility package. It is intended to separate exploratory discovery results from confirmatory claims before journal submission.

## Frozen law and no-cheat boundary

The law layer remains fixed throughout the protocol:

```text
v_f^4 = G M_b a_Theta
```

The value of `aTheta_m_s2` is read once from `config.json`:

```text
aTheta_m_s2 = 1.631702075962e-10
```

No galaxy, lensing system, cluster, phase class, or holdout fold is allowed to receive its own `aTheta`. The Galaxy Phase Tuner measures detector-space quantities only; it does not alter the law-level acceleration scale.

## Data lineage locked for this protocol

Use the raw SPARC files and cleaned outputs already included in the reproducibility package:

```text
data/raw/Table1.mrt.txt
data/raw/Table2.mrt.txt
outputs/01_clean_sparc/sparc_table1_clean.csv
outputs/01_clean_sparc/sparc_table2_clean.csv
outputs/01_clean_sparc/sparc_merged_clean.csv
```

The current cleaned package contains 175 galaxy-level rows and 3,391 radial rows. The phase detector currently analyzes 165 galaxies with sufficient usable radial coverage.

## Detector quantities

The detector begins from the observed-to-baryonic acceleration ratio:

```text
g_obs(r) = V_obs(r)^2 / R
g_bar(r) = V_bar(r)^2 / R
Phi(r) = g_obs(r) / g_bar(r) = V_obs(r)^2 / V_bar(r)^2
y(r) = ln Phi(r)
sigma_y(r)^2 = (2 e_Vobs(r) / Vobs(r))^2 + sigma_floor^2
```

The locked reference baryonic trace is:

```text
V_bar^2 = |V_gas|^2 + 0.5 V_disk^2 + 0.5 V_bul^2
sigma_floor = 0.06
gas_mode = unsigned
```

## Primary confirmatory target

The primary target is radial phase drift:

```text
B_y = y_outer - y_inner
```

This is preferred over signed `theta` for confirmatory testing because `theta` is reference-gauge dependent.

## Primary pre-declared predictor

The primary baryonic structural predictor is:

```text
f_gas = 1.33 M_HI / (0.5 L36 + 1.33 M_HI)
Psi_bar = log(R_HI/R_disk) + log(R_HI/R_eff) + log(SB_eff) - log(f_gas)
```

`Psi_stress = Psi_bar - log(R_last/R_HI)` may be reported as a secondary observed-coverage diagnostic, not as the main baryonic-only claim.

## Locked feature list for confirmatory models

Use only the following Table1-derived baryonic/geometric features for the primary model:

```text
ln_f_gas
logSBeff
ln_RHI_over_Rdisk
ln_RHI_over_Reff
```

`ln_Rlast_over_RHI` is allowed only in a secondary observed-coverage model because it depends on radial observation coverage.

## Deterministic holdout split

For journal confirmation, assign galaxies by deterministic SHA-256 hash of the galaxy name:

```python
split_key = int(sha256(Galaxy).hexdigest()[:8], 16) % 10
train      = split_key in [0,1,2,3,4,5]
validation = split_key in [6,7]
test       = split_key in [8,9]
```

The current 165-galaxy detector sample gives:

```text
train: 95 galaxies
validation: 42 galaxies
test: 28 galaxies
```

The test set must not be used for selecting predictors, reference galaxies, polynomial degree rules, spike thresholds, or wording of success criteria.

## Existing cross-validation checkpoint

The current package includes an exploratory fixed 5-fold cross-validation checkpoint using `KFold(n_splits=5, shuffle=True, random_state=42)`. For the primary target `B_y_outer_minus_inner_lnPhi`, the current package reports:

```text
N = 162
CV R2 = 0.244809
MAE = 0.379910
Spearman(pred,true) = 0.521831
```

These cross-validation values support the method but should be labeled as cross-validated discovery/support unless the deterministic holdout split above is run without modification.

## Multiple-testing control

The primary confirmatory correlation is `Psi_bar` versus `B_y_outer_minus_inner_lnPhi`. In the current package it has:

```text
Spearman r = 0.502598
raw p = 9.428170e-12
BH-FDR q = 2.828451e-10
Bonferroni p = 2.828451e-10
```

All other correlations should be treated as exploratory unless they survive the `multiple_testing_control.csv` correction and then replicate in the locked holdout protocol.

## Reference-gauge robustness

The reference-gauge robustness file repeats the phase-angle calculation using:

```text
NGC7331
NGC5907
NGC7814
NGC5055
```

For journal language, signed `theta` should be reported with circular statistics and reference-gauge caveats. Confirmatory language should emphasize `B_y` and absolute/circular phase statistics rather than relying on one signed reference gauge.

## Pass/fail criteria for a journal claim

A cautious journal-facing claim is supported only if the locked test set satisfies all of the following:

1. `Psi_bar` predicts `B_y` with the same sign as the discovery result.
2. The test-set Spearman correlation for `Psi_bar` vs `B_y` is positive and statistically non-random under the pre-declared test.
3. Cross-validated or holdout predictive performance for `B_y` beats a mean-only baseline.
4. Reference-gauge robustness does not erase the population-level phase grouping.
5. Lensing claims are labeled as pilot/proxy unless full convergence-map or arc-inversion tests are supplied.

## Language rule

Use this wording:

```text
The tested signatures are not uniquely explained by dark matter.
```

Do not use this wording in journal submission:

```text
Dark matter is disproven.
```

## Current status

This file pre-registers the next journal-facing holdout protocol. The existing package already reproduces the cleaned pipeline and exploratory/cross-validated results. A final journal submission should run the deterministic holdout split and add its output table before claiming confirmatory prediction.
