[ms609-agent]

Adds an opt-in, tighter upper bound to NNIDist(). This is the follow-up to #222 that was deferred there.

What it does

NNIDist(tree1, tree2, regionTips = n) appends a seventh element, region_upper. Each region of unmatched edges with up to n leaves (n ≤ 16) contributes the exact NNI distance between its two subtrees, found by bidirectional BFS over the region's tree space. Larger regions contribute the region's best_upper term (its diameter or bound), as do regions with polytomies and searches that visit more than 2²¹ trees.

Reconciling regions one at a time is a valid transformation, so region_upper is an upper bound and is never above best_upper. It is not documented as exact. Li, Tromp & Zhang (1996), Lemma 3, show that a shortest path can break a split the two trees share.

With regionTips = 0L (the default), the output is unchanged.

Implementation (src/nni_exact.cpp)

• Regions are found with union-find over tree 1's unmatched internal edges; a binary root is suppressed.
• One leaf stands for the subtree beyond each of a region's k + 3 boundary edges. Projecting both trees' splits onto those leaves gives C0 and C1 directly, so the region never has to be located in tree 2.
• The BFS stores each split as a 16-bit mask polarised away from leaf 0. A tree is its sorted splits packed into 4 × 64 bits. An NNI on cluster S replaces it with X ∪ Q or Y ∪ Q, where X and Y are S's children and Q is its sibling.
• Internal .NNIExact(tree1, tree2, budget) returns the exact distance for trees of up to 16 leaves. It serves as the test oracle and is used by the benchmark.

Validation

• lower ≤ exact ≤ region_upper ≤ best_upper and symmetry hold on all 5,565 six-leaf pairs, all 446,985 seven-leaf pairs, and 311,850 eight-leaf pairs (30 random trees × all 10,395).
• region_upper == exact held on every pair checked. That covers n ≤ 8, plus 270 near-identical pairs at n = 12–16. This is empirical: Li et al. note that Day's decomposability conjectures hold up to 7 leaves, and Lemma 3's counterexample needs large n. The 6/7-leaf testthat sweep asserts this equality as a regression check, against an independent R BFS.
• The tests also cover rerooting (including on the shared split), one-to-many lists, the size cap, polytomies in either tree, and the budget fallback.

Benchmarks (benchmark/nni-region-upper.R, release install, 1 core)

Cost of one region with no shared splits (median / max seconds; share solved within budget):

leaves	≤ 9	10	11	12	13–16
median s	< 0.01	0.01	0.14	2.7	~4 (budget exhausted)
max s	0.02	0.17	0.70	3.9	~4
solved	100 %	100 %	100 %	60 %	0 %

Tightness against the exact distance at n = 7, 8 (means):

pairs	lower	best_upper	region_upper	exact
n = 7 random	3.65	6.25	4.51	4.51
n = 7, 4 random NNIs	2.41	3.54	2.53	2.53
n = 8 random	4.68	9.04	6.31	6.31
n = 8, 4 random NNIs	2.60	3.67	2.70	2.70

Larger trees (regionTips = 16; means of 50 pairs, ms per pair):

n	perturbation	lower	best_upper	region_upper	ms/pair
16	10 NNIs	6.16	9.60	6.48	0.6
16	20 NNIs	8.92	17.84	10.72	111
16	random	12.68	59.08	59.08	3951
50	20 NNIs	15.14	21.18	15.58	0.6
100	20 NNIs	17.16	20.30	17.24	0.4
100	random	96.88	739.92	739.92	0.4

The bound pays off for similar trees, where regions stay small; there it nearly closes the gap to lower. Unrelated trees form one large region, which gains nothing. At n = 12–16 they also pay the full search budget, about 2–4 s per pair, before falling back.

Recommendation

• Ship it as opt-in with default regionTips = 0L. The docs say that up to 11 leaves costs under a second per region, and that larger regions pay off only when the subtrees are similar.
• NNI_EXACT_BUDGET stays at 2²¹, so most 12-leaf regions solve; a failed search costs about 4 s.

Notes

• The branch was stacked on NNIDist(): make lower bounds valid; adopt Fack theorem 3 #222 and now includes main after NNIDist(): make lower bounds valid; adopt Fack theorem 3 #222 merged. The merge kept main's best_upper-second ordering.

🤖 Generated with Claude Code