LeapFlow

Everything is a signal

LeapFlow is a signal-driven, self-evolving agent framework. Virtual signals (IM apps, OS events, web) and physical signals (robot arms, depth cameras, scientific instruments, IoT) flow through one pipeline into observation → learning → decision-control loops. Knowledge compounds across episodes: skills, device trust, and environment understanding accumulate — the agent gets smarter the more you use it.

News

v0.6.0 — Benchmark qualification suite (36 scenarios, 12 external integrations), governed hardware degradation coordinator, failure fingerprints, CUA window-target contract, adaptive context retention.
v0.5.0 — LeapRobot: governed robot control, local/remote VLA inference, real-time servo, teleoperation + episode export; hc.v1 descriptors, robot-arm & ROS 2 transports.
v0.4.0 — Engine decomposition (6 delegates, 5 sub-packages), BM25 progressive tool disclosure, PCD prefix-commitment, cache-hit telemetry, skill curator, scheduler productization.

Previous releases

v0.3.0 — Event-sourced self-evolution pipeline, leap evolve CLI, LeapBoard evolution & causal views.
v0.2.0 — Teacher/student world-model self-evolution, Hardware Context Protocol (hc.v0), 8 generic hardware tools, LeapBoard hardware lens, device preview pipeline.
v0.1.1 — DeepSeek Harness plugin compatibility, restricted Node.js bridge, component-aware verdicts.
v0.1.0 — First-class plugin subsystem: adaptive evolution, marketplace (Ed25519 signing), sandbox isolation, Gateway adapter registry.
v0.0.9 — Real-time signal pipeline, TUI thinking display, cross-session history, long-task convergence hardening.
v0.0.8 — Cross-platform Windows support (DaemonTransport), journey test layer with cassette-backed CI.
v0.0.7 — 1M-class context windows, self-calibrating token estimator, concurrent-TUI session isolation.

Quick Start

1. Install

pip install leapflow

From source

git clone https://github.com/modelscope/leapflow.git && cd leapflow
uv sync --all-extras --no-extra leapspace

2. Configure LLM — launch the TUI and set your provider (key goes to the local secret vault, never plaintext):

leap

/config llm set --base-url https://dashscope.aliyuncs.com/compatible-mode/v1 \
  --model qwen3.8-flash --api-key sk-...

Any OpenAI-compatible endpoint works — DashScope, OpenAI, DeepSeek, Anthropic, etc.

Shell / CI

leap config llm set \
  --base-url https://api.openai.com/v1 \
  --model gpt-6.1-sol \
  --ask-api-key

3. Go

leap                          # interactive TUI
leap "summarize this repo"    # one-shot
leap --mock-host              # no native execution backend

Have hardware? See Physical AI.

Features

Coding Tools

Locate → read → edit → verify code, out of the box:

Tool	What it does
`repo_map`	Project orientation — languages, structure, entry points
`code_search`	Regex search (ripgrep)
`edit_file`	Anchored search-replace or unified diff — rejects ambiguous edits
`code_intel`	Document symbols (Python AST)
`git_query` / `git_write`	Structured git operations
`test_run` / `lint_check`	Auto-detect and run test / lint suites
`web_fetch`	Fetch pages and JSON APIs — read-only, cached, retried
`terminal_session`	Persistent shell sessions (opt-in)

Teach & Learn

leap teach "open Safari and bookmark this page"

Record your actions → distill into parameterized skills → earn autonomy through success: DRAFT → CANDIDATE → VERIFIED → PRODUCTION.

LeapBoard

/board                  # session analysis (default)
/board finance          # finance lens
/board templates add ./custom.yaml

Real-time monitoring dashboard with Server-Driven UI. Bring your own YAML template — one file, zero code.

Gateway

❯ connect to Telegram

Turn any IM channel (Feishu, DingTalk, Telegram, Slack, …) into a signal boundary. Inbound events are classified, filtered, and routed; each chat gets an isolated session. Setup is conversational — no config files to edit.

Physical AI

LeapFlow bridges LLM intelligence and real-world hardware through a three-layer protocol stack and a layered control hierarchy:

  LHP Fabric          cross-node data plane (JSON Lines)
  ────────────────    inventory · reads · writes · halts · permits · fleet
  HCP                 per-device contract
  ────────────────    6-method transport: open / close / read / write / probe / halt
  Southbound          transport plugins
  ────────────────    ROS2 · Feetech · RealSense · VLA · MCP · media · simulated …

  System-2  (LLM)     0.2–1 Hz     task planning → SubtaskGoal
                          ↓
  System-1.5 (VLA)    5–30 Hz      observation + goal → action chunks
                          ↓
  System-1  (servo)   100–1000 Hz  PID / impedance — never crosses the fabric
                          ↓
                      actuators     escalation flows back up ↑

The host sends goals; the node decomposes them locally. A 1 kHz servo loop stays safe regardless of network latency.

Protocol once, devices for free — new device = one YAML on the node; new protocol = one transport plugin; new node = one fleet entry.

Hardware tools

Group	Tools	Risk
Inspect	`hw_list` · `hw_describe` · `hw_read` · `hw_status`	read-only
Act	`hw_configure` · `hw_actuate` · `hw_dispense` · `hw_batch_actuate`	approval-gated
Safety	`hw_estop`	no approval — node-local
Control	`hw_control_start` · `hw_control_stop` · `hw_control_status`	leased deadman
Fleet	`hw_fleet_status` · `hw_fleet_discover` · `hw_fleet_halt_all`	multi-node
Policy	`hw_policy_infer` · `hw_policy_episode` · `hw_policy_list`	VLA inference

Governance

Admission before visibility — unverified or halt-less devices are demoted to read-only.
Signed execution permits — Ed25519, single-use, nonce-protected, TTL-limited; node verifies and still re-runs local safety.
Halt never waits — node-local, bypasses approval and network.
Progressive trust — autonomy earned per device/action on verified success, lost on failure.

Bundled physical-AI stack

Robot arms (leapflow.robot) — LeRobot-derived: serial bus (Feetech), teleop, camera capture, episode export, policy registry (vla_local / vla_remote / position_control).
VLA as a device — policy server (gRPC or HTTP) appears as an HCP virtual device; inference is a CONFIGURE effect — physical motion happens only when the returned action is written to an actuator.
Depth cameras — RealSense integration; frames as governed binary channels.
ROS2 — sidecar transport over Unix socket; topics/services/actions map into HCP.

Deployment modes

Mode	Description
Host only	Edge node has no LeapFlow runtime.
Host + node	Host agent + edge `leap hw serve` (recommended).
Node only	Edge node operates independently.

Host + node setup:

# Edge node
pip install leapflow
openssl rand -hex 24 > ~/.leapflow/.lhp-token && chmod 600 ~/.leapflow/.lhp-token
LEAPFLOW_LHP_TOKEN="$(cat ~/.leapflow/.lhp-token)" \
  leap hw serve --node-id edge01 --host 0.0.0.0 --port 8767

# Host
export EDGE_LHP_TOKEN='paste-the-token-here'
export LEAPFLOW_FLEET_NODES='[{"node_id":"edge01","endpoint":"tcp://<ip>:8767","token_env":"EDGE_LHP_TOKEN"}]'
leap daemon restart && leap hw list

Device onboarding

leap hw init my_sensor          # generate hc.v1 template
leap hw validate my_sensor      # parse + check
leap hw probe my_sensor         # connectivity test
leap hw verify my_sensor --as operator   # digest-bound verification
leap hw permit-key              # Ed25519 keypair for permits

Architecture

Everything is a Signal

  Virtual                              Physical
  IM · OS · clipboard · web            arms · cameras · instruments · IoT
          │                                    │
          └──────────────┬─────────────────────┘
                         ▼
              ┌─────────────────────┐
              │  Unified Pipeline   │
              │  normalize → gate → │
              │  dedup → memory →   │
              │  causal fusion      │
              └────────┬────────────┘
                       ▼
              ┌─────────────────────┐
              │  OODA Engine        │
              │  Observe → Orient → │
              │  Decide → Act       │
              └─────────────────────┘

Three nested loops on the pipeline:

Loop	Scope	Example
Decision	per turn	Depth adapts to task difficulty; bounded frames compose
Learning	per outcome	World-model verdicts, skill distillation, trust accrual
Evolution	per environment delta	Typed capability requirements → resolution before acquisition

Everything is a Plugin

Every capability arrives behind a typing.Protocol (@runtime_checkable). One lifecycle applies to all: reversible registration, fiber state machine, hot-reload, trust ledger.

Protocol	Role
`ToolPlugin`	Expose tools to the LLM
`GatewayAdapterPlugin`	IM platform adapter
`LLMProviderPlugin`	LLM inference backend
`SignalSource` / `ActiveSignalSource`	Signal sources
`HardwareTransport`	Southbound device protocol (entry point)
`InferenceStrategy`	VLA / policy inference (entry point)
`CVProcessor`	Computer-vision processor

Reference

TUI Commands

Command	Description
`/status`	Runtime and backend status
`/tool`	Inspect available tools
`/config`	View or update settings (tab-completion)
`/model`	Show or switch the active model
`/usage`	Context and token usage
`/board`	Open the monitoring dashboard
`/gateway`	Platform connection status
`/orient`	Layered orientation and re-entries

Safety

Action	Behavior
Safe shell (`ls`, `git status`)	Auto-execute
Dangerous shell (`sudo`, `rm -r`)	Approval prompt — [a]lways for session
Destructive (`rm -rf /`, `mkfs`)	Always blocked
File writes (safe types)	Auto-approve
Hardware actuation	Approval-gated; `hw_estop` always allowed

Configuration

leap config list                  # all settings
leap config set memory.working_max_tokens 12000

Inside the TUI: /config list, /config set … — hot-reloads the active session.

Common keys

Key	Default	Description
`llm.model`	—	Primary model
`llm.base_url`	—	OpenAI-compatible endpoint
`llm.context_length`	`1000000`	Context budget
`memory.working_max_tokens`	`12000`	Working memory budget
`tools.terminal_session_enabled`	`false`	Persistent terminal sessions
`dashboard.enabled`	`true`	LeapBoard
`evolution.enabled`	`false`	Self-evolution

Advanced

Self-Evolution

Disabled by default. When enabled, LeapFlow observes environment deltas and evolves its plugin composition under governance:

leap config set evolution.enabled true

Pipeline: observation → adaptation verdict → capability gap → resolution-first check → proposal → LLM code generation → multi-stage validation → dual approval → install at DRAFT trust → progressive accrual.

Even when disabled, the world model still produces verdicts and distils knowledge. See docs/world_model_plugin_harness_evolution.md.

Plugin Self-Management

The agent manages its own plugins at runtime: plugin_list, plugin_generate, plugin_install, plugin_reload, plugin_disable, plugin_remove, plugin_rollback.

All mutations require HIGH-risk approval. Trust: DRAFT → CANDIDATE → VERIFIED → PRODUCTION. Untrusted plugins run sandboxed (subprocess JSON-RPC). Marketplace plugins are Ed25519-signature verified.

Adaptive-Depth Execution

The agent loop adapts depth to task difficulty. Hard tasks earn wider budgets; recursive subagents run isolated OODA frames.

Config: agent.subagent_full_loop, agent.reentry_enabled, agent.calibration_enabled. Inspect with /orient. Design doc: docs/design/adaptive_depth_ooda.md.

Workflow Copilot (Preview)

Multi-tier next-action predictor: L0 hash match (<1ms) → L1 n-gram (<5ms) → L2 embedding (<50ms) → L3 LLM+RAG (200–2000ms). Display overlay pending; predictions are computed and cached internally.

Native Execution Backend (macOS)

For desktop perception — screen capture, accessibility tree, input injection:

brew install trycua/tap/cua-driver && leap host doctor

Without this, use --mock-host.

Multi-Profile

LEAPFLOW_PROFILE=work leap

Each profile owns its own daemon, config, credentials, and databases under ~/.leapflow/profiles/<name>/.

Custom Gateway Platform

Drop a YAML manifest in ~/.leapflow/profiles/<profile>/gateway/manifests/, implement the adapter Protocol (connect, send, disconnect), and say "connect to my_platform" in the TUI.

CLI Reference

Command	Description
`leap`	Interactive TUI
`leap "question"`	One-shot chat
`leap teach [goal]`	Record and distill a skill
`leap run [prompt]`	Execute a matched skill
`leap skills list\|show\|export\|import`	Skill library
`leap evolve`	Run the learning boundary
`leap board`	LeapBoard dashboard
`leap hw list\|serve\|verify\|estop\|…`	Hardware / LHP node
`leap daemon start\|stop\|restart\|status`	Manage leapd
`leap config list\|set\|show`	Configuration
`leap host doctor`	Execution backend check

Flags: --mock-host, --thinking

Development

make setup && make test && make lint

More

make test-unit              # mock layer, parallel
make test-e2e               # real journeys
make test-impact BASE=origin/main
make test-live              # real provider (needs API key)

src/leapflow/
├── cli/              ├── engine/           ├── hardware/
├── robot/            ├── tools/            ├── plugins/
├── llm/              ├── memory/           ├── perception/
├── signal_fusion/    ├── causal/           ├── world_model/
├── evolution/        ├── learning/         ├── skills/
├── copilot/          ├── monitor/          ├── dashboard/
├── gateway/          ├── platform/         ├── security/
├── daemon/           ├── scheduler/        ├── domain/
├── storage/          ├── recording/        └── analysis/

Troubleshooting

Symptom	Fix
`cua-driver not found`	`brew install trycua/tap/cua-driver`
`MCP connection failed`	`leap host doctor`
`LLM API key is empty`	`/config llm set --api-key …`
Accessibility denied	System Settings → Privacy & Security → grant
Device `writable: 0`	`leap hw verify` — unverified devices are read-only
`permit_required`	`leap hw permit-key` and configure issuer keys

License

Apache 2.0 — see LICENSE.

ModelScope · Star · Issues · Discussions

LeapFlow — Learning and Evolving from Actual Practice.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
assets		assets
docs		docs
scripts		scripts
src		src
tests		tests
tools		tools
.gitignore		.gitignore
AGENTS.md		AGENTS.md
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
pyproject.toml		pyproject.toml
uv.lock		uv.lock

Navigation Menu

Repository navigation

About

Repository files navigation

LeapFlow

News

Quick Start

Features

Coding Tools

Teach & Learn

LeapBoard

Gateway

Physical AI

Architecture

Everything is a Signal

Everything is a Plugin

Reference

TUI Commands

Safety

Configuration

Advanced

Development

Troubleshooting

License

About

Releases

Packages

Used by

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

LeapFlow

News

Quick Start

Features

Coding Tools

Teach & Learn

LeapBoard

Gateway

Physical AI

Architecture

Everything is a Signal

Everything is a Plugin

Reference

TUI Commands

Safety

Configuration

Advanced

Development

Troubleshooting

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages