P3: standards-first SCL-driven SV profiles and IEC/IEEE 61850-9-3 PTP synchronization #24

Goal

Turn the proven ESP32-P4 SV publisher from a hard-coded bench image into a standards-aware, interoperable process-bus source whose configuration is compiled from SCL/profile data on the host and executed deterministically on ESP32-P4.

The architecture keeps wire-observed facts, configured expectations, and profile claims separate; PTP traffic visibility and SV smpSynch truthfulness remain separate; unknowns remain unknown instead of being silently promoted to compliance claims.

Standards-facing architecture

SCD/CID/ICD/IID
      |
      v
Host SCL/profile compiler
      |- stream identity: MAC / APPID / VLAN / PCP
      |- svID / confRev / nofASDU / SmvOpts
      |- dataset order and element types
      |- sample-rate basis / counter policy
      |- scaling / quality policy / synchronization policy
      v
Versioned binary SvPublisherProfile
      |
      v
ESP32-P4 deterministic runtime
      |- prebuilt fixed templates
      |- waveform/fault engine
      |- exact smpCnt policy
      |- canonical SV publisher
      |- PTP disciplined clock + truthful smpSynch
      |- timing/evidence telemetry

SCL/XML parsing must stay out of the realtime hot path and preferably out of the ESP32 entirely. The host compiles an immutable/versioned binary profile that the ESP validates before activating.

Reuse / align with existing ARStack capabilities

ARStack already has an SCL model/parser and Sampled Values codecs. Extend those instead of creating a second configuration model. The runtime profile should be derivable from SclSampledValuesStream and dataset entries.

SV profile engine requirements

• Introduce a transport-neutral SvPublisherProfile / compiled-profile model in the core library.
• Compile a profile from an SCL SampledValueControl + Communication/SMV address + bound DataSet.
• Preserve explicit field-presence options from SmvOpts rather than guessing.
• Support APPID, destination MAC, VLAN ID, PCP, svID, confRev, dataset reference presence, smpRate, smpMod, smpSynch, nofASDU, payload layout and sample-counter policy.
• Reject ambiguous/unresolved SCL bindings instead of silently inventing a dataset.
• Keep profile family labels evidence-aware: Generic IEC 61850-9-2, IEC 61869-9, and Legacy 9-2LE compatibility must not be claimed from heuristics alone.
• Add deterministic profile validation and golden-frame tests.
• Add versioned host->ESP binary profile serialization with checksum/version/length validation.
• Keep diagnostic mirror identity completely separate from canonical process-bus identity.

PTP / synchronization architecture

Use a layered, evidence-oriented PTP design and make the ESP32-P4 path hardware-timestamped and synchronization-capable rather than a software-only lab publisher.

Layers:

1. PTP wire codec — IEEE 1588v2 / IEC 61588 L2 0x88F7 message parsing/serialization.
2. Passive monitor — source/domain/message/sequence visibility without making a synchronization claim.
3. Power-profile validator — explicit IEC/IEEE 61850-9-3 constraints and diagnostics.
4. Hardware clock — ESP32-P4 EMAC IEEE1588v2 time enabled through the pinned toolchain.
5. Servo/discipline state — offset/path-delay estimation and lock/holdover/unlocked state.
6. SV synchronization policy — smpSynch derives from measured lock state; no unconditional global-sync claim in standards mode.
7. Timestamp evidence — scheduled deadline vs EMAC hardware TX timestamp statistics.

ESP32-P4 hardware timing acceptance

Use EMAC IEEE1588v2 hardware timestamp capability as the device-side source of truth rather than host USB-Ethernet arrival timestamps.

• Enable EMAC PTP hardware time on the tested toolchain version.
• Capture actual hardware TX timestamps for SV frames where supported by the driver API.
• Correlate smpCnt, scheduled target time, submit time and hardware TX time.
• Export min/max/mean/P95/P99 TX timing error and missed-deadline counts.
• Add 60 s / 4000 fps evidence run, then 60 s / 4800 fps.
• Do not use host USB-Ethernet arrival jitter as the device conformance timing metric; keep it as capture-path evidence only.

Current bench evidence that motivates this split

The current GPTimer publisher sustains ~4000 frames/s with device-side missed=0, while the host USB-Ethernet capture path reports roughly the same average throughput but batches/perturbs arrival timestamps and produces >300 us excursions. Sequence integrity remains continuous and reconstructed 50 Hz waveforms are correct. This strongly suggests the next timing gate must move to EMAC hardware timestamps / trusted reference capture rather than trying to tune firmware against host capture jitter.

UX target

Open SCL
  -> detect SV streams
  -> select stream
  -> validate address + dataset + SmvOpts
  -> show exact/compatible/conflict findings
  -> compile profile
  -> deploy to ESP32-P4
  -> run waveform/fault scenario
  -> show PTP lock + smpSynch state + hardware TX timing evidence

Public-repository naming policy

Keep project-owned public documentation vendor-neutral. Do not add commercial competitor/product names, comparison tables, logos, screenshots, or marketing references. Standards and technical platform/dependency identifiers required to build or reproduce the project are allowed.

Safety / claim boundary

A successfully decoded SV frame is not a conformance certificate. Keep packet interoperability, SCL/configuration interoperability, semantic compatibility, timing evidence, synchronization evidence and relay interoperability as separate gates.

Depends on P2 timing evidence issue #21 and the hardware bring-up in PR #19.

P3-A profile foundation scope preserved from #33 · tracking consolidated 2026-10-23

This section replaces overlapping issue tracking, not P3-A acceptance. #33's own Oct 7 audit said P3-A0 is substantially complete on then-main (#140/#141/#142/#143/#145), while P3-A1 device binary integrity and P3-A2 hardware/profile breadth must remain explicitly tracked. We keep exact remaining requirements in the canonical P3 parent and focused OPEN children:

P3-A0 (host parser and profile correctness; still regression-protected):

• Preserve SmvOpts optional-field semantics, exact configured MAC/APPID/VLAN/PCP, svID/confRev, nofASDU, sample basis/mode, explicit FCDA order/type, synchronization policy, and profile provenance. Never make configured intent identical to observed PCAP evidence automatically.
• Expand structured FCDA without daName using DataTypeTemplates and FC semantics; fail closed for unsupported or ambiguous layout, no arbitrary vendor-derived fallback.
• Preserve deterministic rational 4000/4800 (and explicitly supported other) cadence, distinguish sampling rate/ASDU/frame rate; do not equate SmpPerSec to validated counter wrap without independent policy.
• Retain vendor-neutral anonymous 4800 4I4V fixture and read-only SCL/CID -> normalized profile CLI, with no proprietary identifiers in public repository.
• Preserve profile-family explicit/evidence-backed Generic IEC 61850-9-2, IEC 61869-9 and legacy compatibility claims; do not guess family from APPID/MAC naming.

P3-A1 device integration (OPEN #36 owns acceptance):

• Versioned host→ESP32-P4 binary profile V1 and checksummed/chunked transaction was merged via P3-A1: native ARStack Studio binary profile deployment and installer candidate #151 at historical main 9e70a49; source behavior remains in current recovery branch CHECKPOINT MERGE: Auto-Live and 5-file capture on PCAP-proven SV/PTP baseline #288. This is software-code presence only.
• Prove source-matched ESP hardware profile deploy and STOPPED-only atomic activation, schema/length/checksum validation, prebuilt templates and no XML/heap in high-rate TX. Physical 4000/4800, EMAC timestamp/CPU/queue budget, long soak and canonical wire proof remain open in P3-A1: compiled SV device profile and deterministic ESP32-P4 activation #36/P2: deterministic ESP32-P4 Sampled Values publisher and timing evidence #21.
• Test more than one validated SCL-driven profile: 4000/50, 4800/60 or second-aligned, explicit vs structured FCDA, SmvOpts variants, alternate stream identity, multi-ASDU/non-4I4V only when layout/capability explicit.

P3-A2 configured vs observed resolution (OPEN #35 owns):

• Measured SV wire fingerprint, deterministic configured-vs-observed MATCH/DIFF/MISSING_CONTEXT/UNSUPPORTED, provenance of user-confirmed overlay, PCAP/PCAPNG capture safety, independent profile-family compatibility; no silent runtime override or brand branch.

Adjacent open gates: #40 vendor SCL dialect quarantine; #139 standards/virtual WirePlan; #21 physical EMAC TX/jitter/4800; #154 PTP↔SV and external IED acceptance. The P3 parent #24 stays OPEN; no claim that P3 was finished by closing the redundant #33 tracking issue.
Original tracking: #33 .

P3-H completed foundation + remaining requirements (transferred from #139, 2026-10-23)

This is an issue-tracking consolidation, not an acceptance statement. The old #139 P3-H recovery/source ledger is archived; the historical source itself, comments and PR links remain accessible. Main P3 standards owner stays OPEN here in #24, specialized children #21/#35/#36/#40 and PTP #154/#162.

Already merged SOFTWARE, preserve these invariants:

• P0 canonical 802.1Q VID 0..4094, reject VID 4095; PCP 0..7; permit SV priority-tagged VID0 but preserve stricter PTP policy where applicable. Source merged #140, including upstream compiler, runtime, PTP builder and edge tests; valid 4I4V frame bytes unchanged.
• P0.1 encoder-produced fixed-bounds per-ASDU BER WirePlan patch offsets/widths; remove runtime byte-pattern search/second parser, avoid hidden secondary TX failure authority. Source merged #141.
• P1 distinct sample basis/rate, ASDUs per Ethernet frame, rational publication cadence and explicit counter policy, no universal 1ASDU assumption. Source merged #142.
• Explicit Generic IEC61850-9-2 vs legacy 9-2LE vs IEC61869-9 family provenance/capability. Source merged #143.
• Host IEC61869-9 profile variant, ordered datasets and rational engineering scaling. Source merged #145. Host-only standards semantics ≠ physical ESP deployability/conformance.

Still OPEN P2+ / future architecture gates (do not mark completed):

• Hardware EMAC SV TX egress timestamps correlated to GPTimer deadline, actual smpCnt/sample epoch/profile generation, not host NIC arrival jitter. Bound and measure ISR/queue overhead; 4000 and 4800 fps with ≥60s saved PCAP; owner P2: deterministic ESP32-P4 Sampled Values publisher and timing evidence #21.
• Validate beyond 1-ASDU 4I4V: deterministic multi-stream scheduler, exact profile admission and packet/capability gating, no realtime XML/heap/BER rescans.
• Process-bus admission planner with versioned device capability vs standards validity vs SCL/observed runtime evidence; fail closed on ambiguous/unsupported.
• PRP/HSR network adapter layer as an explicit later roadmap feature, not silently implied by VLAN/PTP readiness.
• Independent multi-vendor/real relay interop, 4000/4800 physical matrix, and proper standards-family claims. Current local bench source clock is not a certified UTC source; PTP lock/smpSynch field evidence owner M2: continuously PTP-disciplined SV timeline, truthful smpSynch and IED sync-alarm acceptance #154/M3: unified PTP–SMV context supervisor, continuous PPS↔smpCnt, dual clock authority and bounded recovery #162.
• Exact-head C++/embedded/security/Windows tests and no valid golden-wire regressions for new code; no physical timing/conformance claims from builds alone.

Preserve old parallel-work handoff protocol: start from current accepted main + latest checkpoint rather than reimplementing old milestones; read NORTH_STAR.md/docs/SMV_INJECTOR_STATUS.md; own disjoint files; avoid duplicate standards authority/second PTP servo; record exact base/head SHA and rollback/test evidence in each future PR. Original historical anchor: #139.