Restrict Wyner/exact/functional generative models to generator states

CHANGELOG.md

@@ -7,6 +7,17 @@ brute-force reference or a closed-form value and has a regression test.
 
 ### Breaking changes
 
+- `minimal_generative_model`, `wyner_generative_model`,
+  `functional_generative_model` and `generative_complexity()` optimize only
+  over generator states: besides `I[S+ : S- | G] = 0` they impose
+  `I[G_0 X_0 : S-_1 | G_1] = 0` and `I[S+_0 : X_0 G_1 | G_0] = 0`, the
+  conditions under which the HMM built from the channel generates the process.
+  The reported value is the returned model's own (no functional fallback), the
+  exact value upper-bounds the generative complexity, and models carry
+  `markov_residual`. The functional model, previously returned even when it
+  did not reproduce the process, now always does. The unconstrained values are
+  `BidirectionalEpsilonMachine.wyner_common_information()` and
+  `exact_common_information()`.
 - `correction="bonferroni"` is the default for every CSSR learner (process,
   subtree, transducer, stack), and process CSSR counts its length-`(L+1)`
   resolution tests; pass `correction=None` for the old behavior.
@@ -47,6 +58,11 @@ brute-force reference or a closed-form value and has a regression test.
 
 ### Fixes
 
+- The exact/Wyner generative-model channel was read off dit's optimum with
+  the reverse-state axis permuted whenever the joint support listed a
+  higher-indexed reverse state first (dit numbers symbols by first
+  appearance). The channel then failed to screen `S+` from `S-`.
+
 - **Shifts:** `SoficShift.topological_entropy` counts words on a
   right-resolving presentation (parallel same-label edges no longer inflate it);
   new `ShiftOfFiniteType.topological_entropy`; `trim_transient` actually prunes

docs/generators/bidirectional_epsilon_machine.rst

@@ -137,4 +137,4 @@ API
 ===
 
 .. autoclass:: BidirectionalEpsilonMachine
-   :members: from_forward, from_pair, joint_distribution, step_distribution, forward_epsilon_machine, reverse_epsilon_machine, entropy_rate, statistical_complexity, excess_entropy, bidirectional_crypticity, minimal_generative_model, wyner_generative_model, functional_generative_model, gacs_korner_generative_model, generative_complexity, predicted_information, bound_information, ephemeral_information, structural_ephemeral_information, parallel_edge_information, bound_structural_information, bound_parallel_edge_information, information_anatomy, caekl_causal_information
+   :members: from_forward, from_pair, joint_distribution, step_distribution, forward_epsilon_machine, reverse_epsilon_machine, entropy_rate, statistical_complexity, excess_entropy, bidirectional_crypticity, minimal_generative_model, wyner_generative_model, functional_generative_model, gacs_korner_generative_model, generative_complexity, wyner_common_information, exact_common_information, predicted_information, bound_information, ephemeral_information, structural_ephemeral_information, parallel_edge_information, bound_structural_information, bound_parallel_edge_information, information_anatomy, caekl_causal_information

docs/generators/generative_models.rst

@@ -11,41 +11,45 @@ be *generated* with less memory by a non-unifilar machine. ``sofic`` builds
 these minimal generators from a
 :class:`~sofic.generators.bidirectional_epsilon_machine.BidirectionalEpsilonMachine`,
 each realizing a different **common information** between the forward causal
-state :math:`S^+` and the reverse causal state :math:`S^-`.
+state :math:`S^+` and the reverse causal state :math:`S^-`, restricted to
+auxiliaries that are the states of a generator (see `Lifted generators`_).
 
 The generative complexity :math:`C_g = \H{G}` of a presentation with generative
-states :math:`G` obeys :math:`C_g \le C_\mu`, and the four constructions here
-sit in a fixed order:
+states :math:`G` obeys :math:`C_g \le C_\mu`. The three generator
+constructions here sit in the order
 
 .. math::
 
-   K[S^+ : S^-] \;\le\; \op{I}{S^+ : S^-} \;\le\;
-   \text{(exact)} \;\le\; \text{(Wyner)}, \quad \text{(functional)}
+   \op{I}{S^+ : S^-} \;\le\; \text{(Wyner)} \;\le\; \text{(exact)} \;\le\;
+   \text{(functional)} \;\le\; \min(C_\mu^+, C_\mu^-),
+
+where the Wyner value is :math:`\op{I}{(S^+, S^-) : G}` and the others are
+:math:`\H{G}` (the first two up to optimizer accuracy).
 
 * :func:`minimal_generative_model` — the **exact common information**
-  :cite:`Kumar2014`: the minimum-entropy auxiliary :math:`G` rendering
-  :math:`S^+` and :math:`S^-` conditionally independent. This gives the
-  smallest-state generator and is found by a stochastic optimizer.
+  :cite:`Kumar2014`: the minimum-entropy generator state :math:`G` rendering
+  :math:`S^+` and :math:`S^-` conditionally independent, found by a
+  stochastic optimizer. Its entropy bounds :math:`C_g` from above.
 * :func:`wyner_generative_model` — the **Wyner common information**
-  :cite:`Wyner1975`, minimizing :math:`\op{I}{(S^+, S^-) : G}` subject to
-  conditional independence.
+  :cite:`Wyner1975`, minimizing :math:`\op{I}{(S^+, S^-) : G}` over the same
+  generator states.
 * :func:`functional_generative_model` — the **functional common information**:
   the smallest *deterministic* function of :math:`(S^+, S^-)` that separates
-  the two causal states.
+  the two causal states and is a generator state, found by a partition search.
 * :func:`gacs_korner_generative_model` — the **Gács–Körner common information**
   :cite:`GacsKorner1973`: the combinatorial meet :math:`S^+ \wedge S^-`, the
   largest variable that is simultaneously a deterministic function of both.
   This captures only the conserved "core" (phase / ergodic structure) and is
   often trivial for mixing processes.
 
 Each returns a (generally non-unifilar) :class:`~sofic.generators.mealy.MealyHMM`
-subclass whose ``generative_complexity()`` is the corresponding common
-information. The exact, Wyner, and functional models reproduce the source
-process. The Gács–Körner model does **not**: the meet need not render
-:math:`S^+` and :math:`S^-` conditionally independent, so it is the
-common-information variable presented as an HMM rather than a generator —
-``is_equal_process(source)`` is ``False`` in general (the golden mean collapses
-to an i.i.d. coin), and its ``entropy_rate()`` is its own, not the source's.
+subclass whose ``generative_complexity()`` is its state entropy. The exact,
+Wyner, and functional models reproduce the source process. The Gács–Körner model
+does **not**: the meet need not render :math:`S^+` and :math:`S^-` conditionally
+independent, so it is the common-information variable presented as an HMM
+rather than a generator — ``is_equal_process(source)`` is ``False`` in general
+(the golden mean collapses to an i.i.d. coin), and its ``entropy_rate()`` is its
+own, not the source's.
 
 .. code-block:: python
 
@@ -59,12 +63,52 @@ to an i.i.d. coin), and its ``entropy_rate()`` is its own, not the source's.
    exact = bidir.minimal_generative_model()     # stochastic optimizer
    exact.generative_complexity()                # <= C_mu
 
-The optimizers mirror the corresponding ``dit`` common-information routines and
-require ``dit`` (a core dependency). The convenience methods
+The optimizers start from the corresponding ``dit`` common-information routines
+and require ``dit`` (a core dependency). The convenience methods
 :meth:`~sofic.generators.epsilon_machine.EpsilonMachine.minimal_generative_model`
 and friends on an :class:`~sofic.generators.epsilon_machine.EpsilonMachine`
 build the bidirectional presentation first.
 
+Lifted generators
+=================
+
+The common informations of :math:`S^+` and :math:`S^-` constrain only
+:math:`\op{I}{S^+ : S^- \mid G} = 0`. The HMM built from a channel applies
+:math:`p(g \mid s^+, s^-)` at both ends of every bidirectional edge (the
+*lift*), and that HMM generates the source process, with :math:`G` jointly
+distributed with the past and future as the channel prescribes, only if the
+lifted state sequence is Markov:
+
+.. math::
+
+   \op{I}{G_0 X_0 : S^-_1 \mid G_1} = 0, \qquad \op{I}{S^+_0 : X_0 G_1 \mid G_0} = 0.
+
+The exact, Wyner, and functional constructors impose both conditions, so every
+point they search is a generator state (Ellison et al. :cite:`Ellison2009` for
+the bidirectional machine). The exact value, which
+:meth:`~sofic.generators.epsilon_machine.EpsilonMachine.generative_complexity`
+returns, bounds the generative complexity of Löhr & Ay :cite:`LohrAy2009` from
+above. ``markov_residual`` reports the largest constraint information (bits) at
+the returned channel.
+
+For first-order Markov processes the constrained and unconstrained values
+coincide. They differ already for the second-order Markov process in which
+every 1 is followed by at least two 0s: its unconstrained exact (and functional)
+optimum, of entropy 1.406, is a deterministic variable whose value
+:math:`(C, R_{01})` ("the past ends in 10 and the future starts with 01") can
+only be preceded by a value that does not know the past ended in 10, so no HMM
+has it as its state; the constrained value is 1.5. The stochastic optimizer is
+local and seeded (``starts``, ``rng``); on harder processes more starts can
+find lower values.
+
+The unconstrained values remain properties of the process — they equal the
+common informations of the past and future — and are available as measures:
+
+.. code-block:: python
+
+   bidir.wyner_common_information()   # C[S+ : S-] = C[past : future]
+   bidir.exact_common_information()   # G[S+ : S-] = G[past : future]
+
 API
 ===
 
@@ -76,6 +120,12 @@ Constructors
 .. autofunction:: functional_generative_model
 .. autofunction:: gacs_korner_generative_model
 
+Measures
+--------
+
+.. autofunction:: wyner_common_information
+.. autofunction:: exact_common_information
+
 Models
 ------
 

docs/references.bib

@@ -680,6 +680,17 @@ @inproceedings{Kumar2014
   archivePrefix = {arXiv},
 }
 
+@article{LohrAy2009,
+  author  = {L{\"o}hr, Wolfgang and Ay, Nihat},
+  title   = {On the Generative Nature of Prediction},
+  journal = {Advances in Complex Systems},
+  volume  = {12},
+  number  = {2},
+  pages   = {169--194},
+  year    = {2009},
+  doi     = {10.1142/S0219525909002143},
+}
+
 @article{Schreiber2000,
   author  = {Schreiber, Thomas},
   title   = {Measuring Information Transfer},

sofic/generators/bidirectional_epsilon_machine.py

@@ -605,9 +605,21 @@ def gacs_korner_generative_model(self, **kwargs: Any) -> Any:
         return gacs_korner_generative_model(self, **kwargs)
 
     def generative_complexity(self, **kwargs: Any) -> float:
-        """C_g = H[G] for the minimal generative model."""
+        """H[G] of the minimal generative model, an upper bound on C_g."""
         return self.minimal_generative_model(**kwargs).generative_complexity()
 
+    def wyner_common_information(self, **kwargs: Any) -> float:
+        """Wyner common information C[S+ : S-] = C[past : future] (no generator requirement)."""
+        from sofic.generators.minimal_generative_model import wyner_common_information
+
+        return wyner_common_information(self, **kwargs)
+
+    def exact_common_information(self, **kwargs: Any) -> float:
+        """Exact common information G[S+ : S-] = G[past : future] (no generator requirement)."""
+        from sofic.generators.minimal_generative_model import exact_common_information
+
+        return exact_common_information(self, **kwargs)
+
     def information_anatomy(self) -> dict[str, Any]:
         """Return ρ_μ, b_μ, r_μ, h_μ, E, χ, χ±, and the structural/gauge refinement.