diff --git a/README.md b/README.md index 892cf11..fbdc74d 100644 --- a/README.md +++ b/README.md @@ -1,18 +1,16 @@ # langchain-submilli -Code mode for [LangChain](https://docs.langchain.com/oss/python/langchain/overview) and -[deepagents](https://docs.langchain.com/oss/python/deepagents/overview), with a policy the -model can't talk its way around. +[Submilli](https://github.com/submilli/submilli-runtime) is a dedicated secure runtime for +code-mode agents, with built-in isolation, governance, and permissions. -`SubmilliMiddleware` gives your agent a tool that runs TypeScript programs on -[submilli-server](https://github.com/submilli/submilli-runtime). Every program runs under a -Blueprint, a policy you write in YAML, on behalf of the user your application names. Before a -program touches the outside world, such as an HTTP request, a file, a secret, a model call, or -one of your own tools, the server checks the call against the Blueprint. +`langchain-submilli` is the Python integration for +[LangChain](https://docs.langchain.com/oss/python/langchain/overview) and +[Deep Agents](https://docs.langchain.com/oss/python/deepagents/overview). It lets your agent +execute TypeScript code in Submilli's runtime. [Docs](https://submilli.ai/docs/) · -[Connect deepagents](https://submilli.ai/docs/tutorials/connect-deepagents/) · -[Package README](packages/langchain-submilli#readme) · +[Connect Deep Agents](https://submilli.ai/docs/tutorials/connect-deepagents/) · +[Package reference](packages/langchain-submilli#readme) · [Submilli runtime](https://github.com/submilli/submilli-runtime) · [Discord](https://discord.gg/VphpukeGGj) · [Website](https://submilli.ai) @@ -21,448 +19,160 @@ one of your own tools, the server checks the call against the Blueprint. [![PyPI: submilli](https://img.shields.io/pypi/v/submilli?label=submilli)](https://pypi.org/project/submilli/) [![CI](https://github.com/submilli/submilli-python/actions/workflows/ci.yml/badge.svg)](https://github.com/submilli/submilli-python/actions/workflows/ci.yml) [![License: Apache 2.0](https://img.shields.io/badge/license-Apache%202.0-blue)](LICENSE) -[![Discord](https://img.shields.io/discord/1553732232928165958?label=discord)](https://discord.gg/VphpukeGGj) -```python -from deepagents import create_deep_agent -from langchain_submilli import SubmilliMiddleware +## Let your coding agents do the work -agent = create_deep_agent( - model="anthropic:claude-sonnet-5", - middleware=[SubmilliMiddleware(blueprint="support")], -) -await agent.ainvoke({"messages": [...]}, context={"user_id": "u_ada"}) -``` - -## Why - -Think about someone who wants their customer support agent to issue a refund of up to $500 -for platinum clients, and up to $100 for every other tier. They could add it to the prompt as a -safeguard, but because of the nature of models, it will likely work only *some* of the time. -When Submilli runs the agent's code, the owner of the workflow defines these guardrails in -advance, and the runtime enforces them, outside the model's control. - -Code mode and programmatic tool calling started a movement toward agents that write code -instead of calling tools one by one. Submilli is a runtime built for those agents. The agent -submits TypeScript, and Submilli compiles it and runs it in WebAssembly for isolation. There is -no `node:http` or `node:fs`: programs reach the outside only through Packages, and each -Package defines its operations in terms you can write rules about, such as "allow a refund up -to $500, only for customer 123". - -![A support agent creates a replacement with $18 shipping. A $42 replacement is blocked until the developer raises the Blueprint limit from $25 to $50.](https://raw.githubusercontent.com/submilli/submilli-runtime/main/.github/assets/submilli-demo.gif) - -## Packages - -| Package | Install | What it is | -| --- | --- | --- | -| [`langchain-submilli`](packages/langchain-submilli) | `pip install langchain-submilli` | `SubmilliMiddleware` for deepagents and any LangChain v1 agent | -| [`submilli`](packages/submilli) | `pip install submilli` | The SDK for submilli-server's HTTP API, sync and async, for any Python harness. `langchain-submilli` is built on it | - -Both need Python 3.11 or later and submilli-server 0.4.1 or later. Each package has its own -README, changelog, and version, and is released on its own. - -## Quick start - -**1. Install Submilli and start the server.** The installer brings the `submilli` CLI and -`submilli-server`: - -```sh -curl -fsSL https://submilli.ai/install.sh | sh # macOS and Linux; Windows: irm https://submilli.ai/install.ps1 | iex -export SUBMILLI_SERVER_TOKEN=$(openssl rand -hex 32) -submilli-server & # listens on 127.0.0.1:8128 -``` - -**2. Register a Blueprint.** This one allows only computation, which is enough to try the -middleware. It declares `userId`, which the middleware fills from `user_id` in the run's -context: - -```yaml -# blueprint.yaml -kind: blueprint -name: hello -variables: - userId: - required: true -default: deny -``` - -```sh -submilli server blueprint apply blueprint.yaml -``` - -**3. Run an agent.** In the shell where `SUBMILLI_SERVER_TOKEN` is set: - -```sh -pip install "langchain-submilli[deepagents]" "langchain-anthropic>=1.0" -``` - -```python -import asyncio - -from deepagents import create_deep_agent -from langchain_submilli import SubmilliMiddleware - -agent = create_deep_agent( - model="anthropic:claude-sonnet-5", - middleware=[SubmilliMiddleware(blueprint="hello")], -) - - -async def main() -> None: - result = await agent.ainvoke( - {"messages": [{"role": "user", "content": "What is the 40th Fibonacci number? Compute it."}]}, - context={"user_id": "u_ada"}, - ) - print(result["messages"][-1].text) - - -asyncio.run(main()) -``` - -The middleware is async only: use `ainvoke` or `astream`. It reads the server from -`SUBMILLI_SERVER` (default `http://127.0.0.1:8128`) and the token from `SUBMILLI_SERVER_TOKEN`. - -## What the agent gets - -Six tools, with descriptions the server writes for the Blueprint: - -| Tool | Does | -| --- | --- | -| `submilli__typescript__execute` | Runs a program. Its description teaches the model the language, with the Blueprint's Packages and rules filled in | -| `submilli__typescript__last_run` | The thread's last run, with its full console | -| `submilli__typescript__packages__search` | Searches the Packages a program may import | -| `submilli__typescript__packages__docs` | A Package's documentation and declarations | -| `submilli__typescript__builtins__list` | The built-ins in scope without an import | -| `submilli__typescript__builtins__docs` | Declarations of built-ins | - -The model writes code and nothing else. The Blueprint, the user, and any secrets are set by -your application, where the model can't reach them. - -## Examples +Copy this prompt into your coding agent to connect your LangChain or Deep Agents project to +Submilli: -### A rule on an operation's arguments +
+Show prompt -A Blueprint's `permissions` block says what programs may do. Package authors publish the -capabilities their Package supports, and you grant all or some of them. Variables, bound by -your application for each session, let a rule depend on who the agent is working for. Here the -agent may list charges and apply credits of up to $500, only for the customer the application -bound. A different customer, or a larger amount, fails: - -```yaml -variables: - customerId: - required: true - -permissions: - main: - - capability: acme.com/charges.list - filter: customerId == ${vars.customerId} - action: allow - - capability: acme.com/credits.apply - filter: customerId == ${vars.customerId} and amount <= 50000 # cents - action: allow -``` - -The middleware fills `customerId` from the run's context, so one shared agent serves every -customer: - -```python -middleware = SubmilliMiddleware( - blueprint="support", - variables=lambda context: {"customerId": context["customer_id"]}, -) +```text +Connect this project’s LangChain or Deep Agents agent to Submilli. Follow +https://submilli.ai/docs/tutorials/connect-deepagents/. +Keep the existing model. Show me how existing application tools will be +available to the agent, and which calls will go through Submilli. +Show me the Blueprint and code changes. Test with mocked model responses. ``` -When the policy refuses a call, the tool result has `error.kind` `permission_denied`, naming -the capability, and tells the model to report the denial rather than route around it. - -### A deep agent with a notebook per user - -The [research example](packages/langchain-submilli/examples/research) searches the web, reads -pages, and keeps notes that outlive the conversation, one directory per user. The Blueprint -mounts each user's notes at `/notes` and grants the web search Package: - -```yaml -variables: - userId: - required: true - -packages: -- '@submilli/jina' - -vfs: - mode: per_session - cwd: /notes - mounts: - /notes: - mode: named - volume: notes - subPath: ${vars.userId} # each user sees only their own notes - -default: deny - -permissions: - main: - - capability: jina.ai/search - action: allow - - capability: jina.ai/read - action: allow - - capability: fs.read - action: allow - - capability: fs.write - action: allow - # ... and the Package's own HTTP and secret grants; see the full file -``` - -The agent is built once and shared. Each run brings its user in the context, and each thread -keeps one server session, saved by the checkpointer: - -```python -from typing import TypedDict - -from deepagents import create_deep_agent -from langgraph.checkpoint.memory import InMemorySaver -from langchain_submilli import SubmilliMiddleware - - -class Context(TypedDict): - user_id: str - - -submilli = SubmilliMiddleware( - blueprint="research", - # deepagents' own file tools keep files in the conversation's state. Hide them, - # so that notes go through Submilli. - hide_filesystem_tools=True, -) - -agent = create_deep_agent( - model="anthropic:claude-sonnet-5", - system_prompt=INSTRUCTIONS, - middleware=[submilli], - checkpointer=InMemorySaver(), - context_schema=Context, -) - -config = {"configurable": {"thread_id": "t1"}} -result = await agent.ainvoke( - {"messages": [{"role": "user", "content": "What's new in Rust? Save a note with your sources."}]}, - config, - context=Context(user_id="u_ada"), -) -await submilli.aclose_thread(agent, config) # when the conversation is over -``` - -Its `check.py` runs the agent with a scripted model, so it needs no API key, and asserts that -two users who write the same note each see only their own. - -### Your own tools in programs, with approvals - -Programs can call your application's LangChain tools (programmatic tool calling). Pass them as -`ptc`, and name the ones a person must approve in `interrupt_on`: +
-```python -from typing import TypedDict - -from langchain.agents import create_agent -from langchain.tools import ToolRuntime, tool -from langgraph.checkpoint.memory import InMemorySaver -from langchain_submilli import SubmilliMiddleware - - -class Orders(TypedDict): - count: int - total: float - - -@tool -def query_orders(customer: str, runtime: ToolRuntime) -> Orders: - """Orders of one customer.""" - return db.orders(customer, user=runtime.context["user_id"]) +## Why +[Code mode](https://submilli.ai/docs/why/#the-industry-already-agrees) lets agents write code +instead of calling tools one at a time. The agent submits code, and +Submilli compiles and runs it in WebAssembly for isolation. Programs have no direct access to the +network or host filesystem. Programs use [Packages](https://submilli.ai/docs/packages/) to access +external systems, and [Blueprints](https://submilli.ai/docs/blueprints/) define which operations +are allowed. See [the execution challenges](https://submilli.ai/docs/why/#this-code-is-a-stranger) +for why isolation alone is not enough. -@tool -def send_email(to: str, body: str) -> str: - """Send an email.""" - ... +## Example +Suppose your support agent can send replacements with shipping costs up to $25. A prompt can ask +the model to respect that limit, but the model can still generate code that exceeds it. Set the +limit in a Blueprint, and Submilli enforces it. In the video, a replacement with $18 shipping is +allowed and a replacement with $42 shipping is blocked. That replacement succeeds after the +developer raises the limit to $50. -agent = create_agent( # any LangChain v1 agent, not only deepagents - model="anthropic:claude-sonnet-5", - middleware=[ - SubmilliMiddleware( - blueprint="shop", - ptc=[query_orders, send_email], - interrupt_on={"send_email": True}, - ) - ], - checkpointer=InMemorySaver(), -) -``` +https://github.com/user-attachments/assets/e601c541-400b-4ac1-ad96-23dc0b0e6f90 -Programs import them as the `@tools` Package, typed from your tools' schemas and return -annotations: +## Install -```ts -import tools from "@tools"; +Python 3.11 or later is required. -function main(): string { - const orders = tools.query_orders({ customer: "c_42" }); - if (orders.count > 10) { - tools.send_email({ to: "sales@example.com", body: `c_42 has ${orders.count} orders` }); - } - return `${orders.count} orders`; -} +```sh +pip install "langchain-submilli[deepagents]" ``` -The Blueprint decides which tools programs may call, through the `tools.call` capability. It -is denied by default: +For a different LangChain v1 agent, install `langchain-submilli` without the `deepagents` +extra. To use submilli-server directly from Python, install the +[`submilli` SDK](packages/submilli/README.md) instead. -```yaml -permissions: - main: - - capability: tools.call - filter: tool == "query_orders" or tool == "send_email" - action: allow -``` +Follow [Connect Deep Agents](https://submilli.ai/docs/tutorials/connect-deepagents/) for the +complete setup, including the server, Blueprint, model provider, agent, and session cleanup. -Each `send_email` call pauses the graph with LangChain's human-in-the-loop request, while the -run waits on the server. `approve`, `edit`, `respond`, and `reject` work as they do for -`HumanInTheLoopMiddleware`. See [Your tools in programs](packages/langchain-submilli#your-tools-in-programs). +## Use your existing LangChain tools -### Per-user secrets +Your agent can call your existing LangChain tools from the code it runs through Submilli. Pass +those tools to `SubmilliMiddleware` with `ptc` to make them available to the program. -`secrets` maps the run's context to the Blueprint's `harness` secrets. They are sent to the -server when a session opens and are never stored in the agent's state, so a program can act -with the user's own token without the model ever seeing it: +The [integration guide](https://submilli.ai/docs/tutorials/connect-deepagents/#your-tools-in-programs) +explains how these calls work. See the [LangChain configuration](packages/langchain-submilli#application-tools) +for the `ptc` option. -```python -middleware = SubmilliMiddleware( - blueprint="github", - variables=lambda context: {"userId": context.account_id}, - secrets=lambda context: {"GITHUB_TOKEN": context.github_token}, -) -``` +## Next steps -### Without LangChain +To control what the agent may do, [create a Blueprint](https://submilli.ai/docs/blueprints/start-a-blueprint/) +and [register it on your server](https://submilli.ai/docs/server/register-a-blueprint/). The +[permissions reference](https://submilli.ai/docs/reference/permissions/) covers rules for the +operations your agent needs. -The `submilli` SDK talks to the same server from any Python code: +To pass application context or credentials to Submilli, see the [middleware settings](packages/langchain-submilli#middleware-reference). +The [session reference](packages/langchain-submilli#sessions-retries-and-results) explains reuse +across turns and cleanup. -```python -from submilli import AsyncSubmilli +## What the Submilli runtime provides -async with AsyncSubmilli() as client: - session = await client.sessions.create("hello", variables={"userId": "u_ada"}) - result = await client.sessions.run( - session.id, - "export function main(): string { return `${1 + 1}`; }", - idempotency_key="call-1", - ) - print(result.result) # "2" -``` +- [Blueprints](https://submilli.ai/docs/blueprints/) define rules for operation arguments, + bind values per session, and deny everything else by default. +- [Packages](https://submilli.ai/docs/packages/) wrap APIs in operations the Blueprint can + govern. Credentials can be attached by the host without being returned to the program, and + [curated integrations](https://submilli.ai/docs/reference/curated-packages/) cover common + services. +- [Resource limits](https://submilli.ai/docs/server/set-limits/) bound memory, execution time, + stack depth, model tokens, and more. +- The [audit trail](https://submilli.ai/docs/reference/audit-trail/) records refusals, runs, + sessions, and administrative changes. +- [MCP servers](https://submilli.ai/docs/reference/mcp-servers/) can be exposed as Packages, so + Blueprint rules apply to their tools. +- [Server controls](https://submilli.ai/docs/server/run-the-server/) include HTTPS, API tokens + with administrative and user roles, and an encrypted secret store. -See the [`submilli` README](packages/submilli#readme) for sessions, files, tool calls, and -errors. - -## How it behaves - -- **One session per thread.** The first `execute` in a thread opens a server session bound to - the Blueprint and the run's variables. With a checkpointer, a resumed thread, or one continued - in another process, keeps the same session and its files. -- **One user per session.** A thread whose variables change raises - `SubmilliSessionMismatchError` before the model is called. One user's programs never run in - another user's session. -- **Safe retries.** Every execute call sends an `Idempotency-Key` derived from the tool call id, - so a repeated call replays the recorded result instead of running twice. -- **An audit trail.** Each tool message's `artifact` carries the run's `execution_id`, its place - in the server's [audit trail](https://submilli.ai/docs/reference/audit-trail). - -The [package README](packages/langchain-submilli#readme) covers configuration, lost sessions, -and how `SubmilliMiddleware` compares with `CodeInterpreterMiddleware` from -`langchain-quickjs`. - -## What Submilli brings - -- [Blueprints](https://submilli.ai/docs/blueprints) with rules on an operation's arguments, - bound per session, and everything denied by default. -- [Packages](https://submilli.ai/docs/packages) that wrap your APIs and hold the credentials, so - generated code never sees a secret. Curated Packages for GitHub, Slack, Gmail, Google Drive - and Calendar, Linear, Notion, Sentry, and web search are - [included](https://github.com/submilli/submilli-runtime/blob/main/packages/README.md). -- [Limits](https://submilli.ai/docs/server/set-limits) on memory, time, stack depth, model - tokens, and more. A failing run ends alone, and the rest of the server keeps serving. -- An [audit trail](https://submilli.ai/docs/reference/audit-trail) of every refusal, run, - session, and admin change. -- MCP servers as Packages, with rules on their tools. -- HTTPS, API tokens with admin and user roles, and an encrypted secret store. - -To run the server in production, use -[Docker Compose](https://submilli.ai/docs/server/deploy-with-compose), the -[Helm chart](https://submilli.ai/docs/server/deploy-on-kubernetes), or -[systemd](https://submilli.ai/docs/server/deploy-on-linux). The -[Submilli docs](https://submilli.ai/docs/) cover -[writing Blueprints](https://submilli.ai/docs/blueprints/start-a-blueprint/) and the -[HTTP API](https://submilli.ai/docs/reference/http-api). +For production, deploy with +[Docker Compose](https://submilli.ai/docs/server/deploy-with-compose/), +[Kubernetes](https://submilli.ai/docs/server/deploy-on-kubernetes/), or +[Linux and systemd](https://submilli.ai/docs/server/deploy-on-linux/). ## Status -Submilli is young and moving quickly. Breaking changes are called out in each package's -changelog. If you're thinking of using it, we'd love to talk to you: contact us at -[hello@submilli.ai](mailto:hello@submilli.ai) or on [Discord](https://discord.gg/VphpukeGGj). +Submilli is young and moving quickly. Breaking changes are recorded in the +[`langchain-submilli` changelog](packages/langchain-submilli/CHANGELOG.md) and +[`submilli` changelog](packages/submilli/CHANGELOG.md). To talk with us, email +[hello@submilli.ai](mailto:hello@submilli.ai) or join +[Discord](https://discord.gg/VphpukeGGj). ## Development -The repository is a [uv workspace](https://docs.astral.sh/uv/concepts/projects/workspaces/). +This repository is a [uv workspace](https://docs.astral.sh/uv/concepts/projects/workspaces/). In the workspace, `langchain-submilli` uses `packages/submilli` directly. When published, it depends on `submilli` from PyPI. ```sh uv sync --all-packages --all-groups -``` - -Formatting and linting run over the whole repository: - -```sh uv run ruff check . uv run ruff format . ``` -Type checks and tests run inside each package, with that package's settings: +Run type checks and tests inside each package: ```sh cd packages/submilli # or packages/langchain-submilli -uv run mypy # strict +uv run mypy uv run pyright -uv run pyright --verifytypes submilli --ignoreexternal # public API fully typed; langchain_submilli there -uv run pytest # unit tests, against a fake of the HTTP API +uv run pyright --verifytypes submilli --ignoreexternal # packages/submilli +uv run pyright --verifytypes langchain_submilli --ignoreexternal # packages/langchain-submilli +uv run pytest ``` -The integration tests start their own submilli-server from `$SUBMILLI_BIN`, a directory that -holds `submilli-server`, or from `PATH`. Put a release build in `.submilli-bin/`, which git -ignores, and point at it: +Integration tests start submilli-server from `$SUBMILLI_BIN` or `PATH`. A release build can +live in the ignored `.submilli-bin/` directory: ```sh SUBMILLI_BIN=$PWD/../../.submilli-bin uv run pytest tests/integration # submilli SUBMILLI_BIN=$PWD/../../.submilli-bin uv run pytest tests/integration_tests # langchain-submilli ``` -Some `langchain-submilli` tests also need a server with the `research` Blueprint; -`packages/langchain-submilli/scripts/test-server.sh` starts one. +Some `langchain-submilli` integration tests also need the `research` Blueprint. +`packages/langchain-submilli/scripts/test-server.sh` starts a suitable server. + +From the repository root, after changing `packages/submilli/src/submilli/_async_client.py`, +regenerate the sync client: -After changing `packages/submilli/src/submilli/_async_client.py`, regenerate the sync client -with `uv run python packages/submilli/scripts/generate_sync.py`. CI checks it is up to date. +```sh +uv run python packages/submilli/scripts/generate_sync.py +``` ## CI and releases -CI runs ruff, then for each package mypy, pyright, `--verifytypes`, and the unit tests on -every supported Python. It also runs the integration tests against the submilli-server -release named in `.github/workflows/ci.yml`, and checks that each wheel builds and installs on -its own. +CI runs formatting, linting, type checks, unit tests, integration tests, and standalone wheel +installation checks. -A package is released by pushing a tag with its name and version, matching its -`pyproject.toml`: `submilli-v0.1.0` or `langchain-submilli-v0.1.0`. The release workflow builds -that package and publishes it to PyPI through trusted publishing. Release `submilli` before a -`langchain-submilli` that needs its new version. +Release a package by pushing a tag that matches its `pyproject.toml` version: +`submilli-v0.1.0` or `langchain-submilli-v0.1.0`. The release workflow publishes that package +to PyPI through trusted publishing. Release `submilli` before a `langchain-submilli` version +that depends on it. ## License diff --git a/packages/langchain-submilli/README.md b/packages/langchain-submilli/README.md index e3663de..9591b82 100644 --- a/packages/langchain-submilli/README.md +++ b/packages/langchain-submilli/README.md @@ -1,352 +1,137 @@ # langchain-submilli -`langchain-submilli` runs a LangChain agent's code-mode tool calls as TypeScript programs on -[submilli-server](https://github.com/submilli/submilli-runtime). Each program runs under the -policy of a Blueprint, on behalf of the user named in the run's context. +[Submilli](https://github.com/submilli/submilli-runtime) is a dedicated secure runtime for +code-mode agents, with built-in isolation, governance, and permissions. -v1 ships `SubmilliMiddleware`, which works with -[deepagents](https://docs.langchain.com/oss/python/deepagents/overview) and any LangChain v1 -agent that takes middleware. +`langchain-submilli` is the Python integration for +[LangChain](https://docs.langchain.com/oss/python/langchain/overview) and +[Deep Agents](https://docs.langchain.com/oss/python/deepagents/overview). It lets your agent +execute TypeScript code in Submilli's runtime. ## Install +Python 3.11 or later and submilli-server 0.4.1 or later are required. + ```sh pip install langchain-submilli ``` -deepagents is not a dependency, because the middleware works with any LangChain v1 agent. -Install it with `pip install "langchain-submilli[deepagents]"`. The HTTP API itself is the -[`submilli`](https://github.com/submilli/submilli-python) SDK, which this package depends on; -its client, results, and errors are imported from `submilli`. - -```python -from deepagents import create_deep_agent -from langchain_submilli import SubmilliMiddleware - -agent = create_deep_agent( - model="anthropic:claude-sonnet-5", - middleware=[SubmilliMiddleware(blueprint="research")], -) -await agent.ainvoke({"messages": [...]}, context={"user_id": "u_ada"}) -``` - -The middleware needs a running submilli-server with the Blueprint registered. See -[Run a server](#run-a-server). It is async only: use `ainvoke` or `astream`. - -## What the agent gets - -Six tools, with the server's names and with descriptions the server writes for the Blueprint: - -| Tool | Does | -| --- | --- | -| `submilli__typescript__execute` | Runs a program. Its description teaches the model the language, with the Blueprint's Packages and rules filled in | -| `submilli__typescript__last_run` | The thread's last run, with its full console | -| `submilli__typescript__packages__search` | Searches the Packages a program may import | -| `submilli__typescript__packages__docs` | A Package's documentation and declarations | -| `submilli__typescript__builtins__list` | The built-ins in scope without an import | -| `submilli__typescript__builtins__docs` | Declarations of built-ins | - -The descriptions come from `GET /v1/blueprints/{name}/prompt`. Each middleware instance fetches -them once, before the first model call. The model writes code and nothing else. The Blueprint, -the user, and any secrets are set by your application, where the model cannot reach them. - -## What happens on each tool call - -**One session per thread.** The first time the model calls `execute` in a thread, the -middleware opens a session on the server. It binds the Blueprint and the variables from this -run's context. The session's id and variables go into the agent's state as `submilli_session`. -With a checkpointer, they are saved before any program runs, so a resumed thread, or one -continued in another process, keeps using the same session. Discovery tools need no session, -unless you [expose your own tools](#your-tools-in-programs). - -Ending a turn does not end the session, because the session belongs to the thread. End it with -`await middleware.aclose_thread(agent, config)`. Otherwise the Blueprint's `idle_timeout` -(default 24 hours) ends it. - -**One user per session.** Before each run, the middleware compares the context's variables with -the variables the thread's session was opened with. If they differ, it raises -`SubmilliSessionMismatchError` before the model is called. One user's programs never run in -another user's session. - -**Idempotent retries.** Every execute call sends an `Idempotency-Key` derived from the -LangChain tool call id. The id is used as is when it is printable ASCII of at most 120 bytes; -otherwise its sha256 hex digest is used. When the same tool call reaches the server twice, the -program runs once and the second call gets the recorded result. This happens when a thread is -resumed from a checkpoint taken before its tools finished, or when a retry middleware repeats -a call. Both tool messages carry the same `execution_id`. When the server refuses a key, the -model gets a tool result that says nothing ran and what to do. The refusals are -`idempotency_conflict`, `idempotency_incomplete`, `idempotency_in_progress`, and -`idempotency_unavailable`. On `idempotency_incomplete`, the model is told the earlier attempt's -side effects are unknown, and to check before it retries. - -**Policy and denials.** The server checks every capability a program uses against the -Blueprint, such as a file write, an HTTP request, a secret, or a model call. The tool result -is JSON with `result`, `console`, `error`, and `denied`, the same shape submilli-runtime's -Vercel AI SDK example returns. When the policy refuses a call, `error.kind` is -`permission_denied` and names the `caller`, `capability`, and `source`. `denied` is `true`, -and a note tells the model to report the denial to the user rather than route around it. -`execution_id` and `session_id` are in the tool message's `artifact`, not in the model's -content, so your application can link each run to the server's -[audit trail](https://submilli.ai/docs/reference/audit-trail). - -**When the server loses something.** When a session has ended, for example by idle timeout, -the server answers `404 unknown session`. The middleware opens a new session and runs the -program there. The tool result tells the model that the earlier session's files are gone. -Sessions survive a server restart, and so do their harness secrets, which the server keeps -encrypted with its secret-store key. If the server answers `409 session_requires_secrets` -anyway, the middleware sends the context's secrets with `rebind` and retries once. - -`interrupt_on` approvals apply to `submilli__typescript__execute` as they do to any other tool: -a person approves the whole program before it runs. To approve single calls a program makes to -your own tools, see [Your tools in programs](#your-tools-in-programs). - -## Your tools in programs - -With submilli-server 0.4.1 or later, programs can call your application's own tools, such as a -LangChain `@tool` that queries your database or sends an email. This is programmatic tool -calling (PTC), and the option has the same name as in `langchain-quickjs`. Pass the tools as -`ptc`: - -```python -from typing import TypedDict - -from langchain.tools import ToolRuntime, tool - - -class Orders(TypedDict): - count: int - total: float - - -@tool -def query_orders(customer: str, runtime: ToolRuntime) -> Orders: - """Orders of one customer.""" - return db.orders(customer, user=runtime.context["user_id"]) - +Deep Agents is optional. Install the extra when you use it: -@tool -def send_email(to: str, body: str) -> str: - """Send an email.""" - ... - - -middleware = SubmilliMiddleware( - blueprint="shop", - ptc=[query_orders, send_email], - interrupt_on={"send_email": True}, -) +```sh +pip install "langchain-submilli[deepagents]" ``` -Programs import them as the `@tools` Package, typed from your tools' schemas: - -```ts -import tools from "@tools"; +Follow [Connect Deep Agents](https://submilli.ai/docs/tutorials/connect-deepagents/) for a +complete setup with a server, Blueprint, model provider, agent, and session cleanup. -function main(): string { - const orders = tools.query_orders({ customer: "c_42" }); - if (orders.count > 10) { - tools.send_email({ to: "sales@example.com", body: `c_42 has ${orders.count} orders` }); - } - return `${orders.count} orders`; -} -``` +## Middleware reference -- **Running a call.** Each call stops the run on the server. The middleware runs the tool - through a LangGraph `ToolNode`, with the run's state, context, and store, as the agent would, - and resumes the run with the result. Time spent waiting doesn't count against the run's time - limit. -- **Return values.** The argument types come from the tool's argument schema. The return type - comes from its return annotation. Return a `TypedDict` or a Pydantic model for a typed - object. A `dict[str, X]` reaches the program as `unknown`. A tool with no JSON return - annotation returns its text. -- **Errors.** A tool that raises throws `Error` in the program, which the program can catch. -- **Policy.** The Blueprint decides which tools programs may call, through the `tools.call` - capability and its `tool` filter. It is denied by default, and the server logs a warning - when a session declares tools the Blueprint can't use: - - ```yaml - permissions: - main: - - capability: tools.call - filter: tool == "query_orders" or tool == "send_email" - action: allow - ``` - -- **Approvals.** `interrupt_on` takes the `HumanInTheLoopMiddleware` settings for exposed - tools: `True`, or an `InterruptOnConfig` with `allowed_decisions`, `description`, and - `when`. A call that needs approval interrupts the graph with a `HITLRequest` of one action, - while the run waits on the server. Resume it with one decision: - - ```python - [pending] = (await agent.aget_state(config)).interrupts - await agent.ainvoke(Command(resume={pending.id: {"decisions": [{"type": "approve"}]}}), config, context=...) - ``` - - `approve` runs the call. `edit` runs it with edited arguments, for the same tool. `respond` - answers for the tool without running it. `reject` throws `PermissionDeniedError` in the - program, with the rejection message, and the model is told the denial is final. A program - that makes several such calls pauses once for each. The agent needs a checkpointer. A run - waits up to the Blueprint's `tool_call_timeout` (10 minutes by default) for each answer, so - raise it for approvals that take longer. - - This approval is separate from deepagents' and LangChain's own `interrupt_on`, which only - sees tool calls the model makes. If you approve a tool there, approve it here too before you - expose it. - -- **What the application sees.** The execute tool message's `artifact` lists the program's - calls under `tool_calls`, each with its arguments, its outcome, and the decision, when there - was one. -- **State updates.** A tool that returns a `Command`, such as one that writes deepagents' - `files`, has its state update applied with the execute result. When several calls write - the same key, dict values are merged and anything else keeps the last value. -- **Discovery.** With exposed tools, `packages__search` and `packages__docs` answer for the - thread's session, which includes `@tools`. The first such call opens the session. -- **Sessions.** A session keeps the tools it was opened with. When the exposed tools change, - the thread's next call opens a new session. - -The exposed tools are not given to the model directly. Pass them to the agent as well if the -model should call them itself. - -## Configuration +`SubmilliMiddleware` is async only. Use `ainvoke` or `astream`. ```python SubmilliMiddleware( blueprint="research", - server=None, # default: $SUBMILLI_SERVER, then http://127.0.0.1:8128 - token=None, # default: $SUBMILLI_SERVER_TOKEN - variables=None, # default: {"userId": context.user_id} - secrets=None, # default: none + server=None, + token=None, + variables=None, + secrets=None, hide_filesystem_tools=False, - client=None, # a submilli.AsyncSubmilli, instead of server and token - ptc=None, # your tools, for programs to call as @tools (programmatic tool calling) - interrupt_on=None, # which of them need approval, as for HumanInTheLoopMiddleware + client=None, + ptc=None, + interrupt_on=None, ) ``` -`variables` and `secrets` take the run's `runtime.context` and return `dict[str, str]`. A fixed -mapping also works. Build the agent once and share it; each run brings its own user: - -```python -middleware = SubmilliMiddleware( - blueprint="research", - variables=lambda context: {"userId": context.account_id, "region": context.region}, - secrets=lambda context: {"USER_TOKEN": context.github_token}, -) -``` - -By default, `user_id` in the context becomes the Blueprint variable `userId`. The context can be a -dict, a dataclass, or a Pydantic model. A context without `user_id` binds no variables. A -Blueprint that requires `userId` then refuses the session, and the run raises -`submilli.BadRequestError`. - -Secrets go to the Blueprint's `harness` secrets. They are sent when a session opens and when -the server asks for them again. They are never stored in the agent's state. A session keeps -the secrets it was opened with: when they change, for example a rotated user token, end the -thread's session with `aclose_thread` so the next call opens one with the new values. - -`hide_filesystem_tools=True` removes deepagents' own file and shell tools from the agent: -`ls`, `read_file`, `write_file`, `edit_file`, `delete`, `glob`, `grep`, and `execute`. The -model then reaches files only through programs. Those tools keep files in the conversation's -state, so without the flag a model asked to "save a note" may save it there, and the note -never reaches the server. The middleware filters the tools from each model request, the way -deepagents' own harness profiles do, and refuses a call that names one. The flag does not -reach deepagents' built-in `general-purpose` subagent, which does not receive your middleware. -To deny the file tools there too, pass deepagents' -`permissions=[FilesystemPermission(operations=["read", "write"], paths=["/**"], mode="deny")]` -as well. - -## Run a server - -Install Submilli. The installer puts both `submilli`, the CLI, and `submilli-server` on your -`PATH`: - -```sh -curl -fsSL https://submilli.ai/install.sh | sh # macOS and Linux -irm https://submilli.ai/install.ps1 | iex # Windows (PowerShell) -``` - -Start the server with a token. It listens on `127.0.0.1:8128`, which is where the middleware -looks by default: - -```sh -export SUBMILLI_SERVER_TOKEN=$(openssl rand -hex 32) -submilli-server & -``` - -Register a Blueprint. This one allows only computation, which is enough to try the middleware. -It declares `userId`, which the middleware fills from `user_id` in the run's context: - -```yaml -# blueprint.yaml -kind: blueprint -name: hello -variables: - userId: - required: true -default: deny -``` - -```sh -submilli server blueprint apply blueprint.yaml -``` +| Setting | Contract | +| --- | --- | +| `blueprint` | Name of a Blueprint already [registered on the server](https://submilli.ai/docs/server/register-a-blueprint/); used for every program | +| `server` | submilli-server URL; defaults to `$SUBMILLI_SERVER`, then `http://127.0.0.1:8128` | +| `token` | Server token; defaults to `$SUBMILLI_SERVER_TOKEN` | +| `variables` | Fixed `dict[str, str]`, or a function from the run context to one; defaults to mapping context `user_id` to Blueprint variable `userId` | +| `secrets` | Fixed `dict[str, str]`, or a function from the run context to one, for the Blueprint's `harness` secrets | +| `hide_filesystem_tools` | Hides and refuses Deep Agents' file and shell tools on this agent. Subagents need separate configuration. | +| `client` | Existing `submilli.AsyncSubmilli`; replaces `server` and `token` | +| `ptc` | Application tools that programs may call through the `@tools` Package | +| `interrupt_on` | LangChain human-in-the-loop settings for calls to tools in `ptc` | + +The middleware gives your agent the six tools below. Before the first model call, it fetches the +instructions for using them from the Submilli server. -Run the agent where `SUBMILLI_SERVER_TOKEN` is set, with `SubmilliMiddleware(blueprint="hello")`. -Set `SUBMILLI_SERVER` too if the server is somewhere other than `http://127.0.0.1:8128`. - -To give programs HTTP, files, secrets, or model calls, grant them in the Blueprint. The -`examples/research/` directory in the [repository](https://github.com/submilli/submilli-python/tree/main/packages/langchain-submilli/examples/research) -holds a Blueprint that searches the web and keeps notes, with its `prompt.txt` and `note.ts`. -These pages cover the rest: - -- [Install](https://submilli.ai/docs/install/) and [Run the server](https://submilli.ai/docs/server/run-the-server/) -- [Register a Blueprint](https://submilli.ai/docs/server/register-a-blueprint/), with its secrets and Packages -- [Start a Blueprint](https://submilli.ai/docs/blueprints/start-a-blueprint/) and the [Blueprint file reference](https://submilli.ai/docs/reference/blueprint-file/) -- Deploying: [Linux](https://submilli.ai/docs/server/deploy-on-linux/), [Compose](https://submilli.ai/docs/server/deploy-with-compose/), [Kubernetes](https://submilli.ai/docs/server/deploy-on-kubernetes/) -- The [HTTP API reference](https://submilli.ai/docs/reference/http-api), which lists every endpoint this package calls - -For development, `scripts/test-server.sh` starts a throwaway server with the `research` -Blueprint in `.submilli-test/` at the repository root. - -## Compared with `CodeInterpreterMiddleware` - -[`CodeInterpreterMiddleware`](https://docs.langchain.com/oss/python/deepagents/interpreters) -from `langchain-quickjs` fills the same slot. It adds a code tool to a deep agent, each thread -keeps its own state, and programs call the agent's tools through programmatic tool calling -(PTC). Both middlewares take those tools as `ptc=[...]`, and programs call them as -`tools.(...)`. - -| | `CodeInterpreterMiddleware` | `SubmilliMiddleware` | -| --- | --- | --- | -| Language | JavaScript | A strict TypeScript subset, compiled to WebAssembly | -| Where code runs | QuickJS, in the agent's process | submilli-server, a separate process | -| Reaching the outside | Only the `ptc` tools | Packages and capabilities the Blueprint grants: files, HTTP, secrets, model calls; and the `ptc` tools | -| `ptc` tools | Called as `await tools.camelCaseName(...)`, declared as returning `Promise` | Called as `tools.name(...)`, typed from each tool's argument schema and return annotation | -| Permission boundary | The `ptc` allowlist | The Blueprint's policy, checked on each capability call, `ptc` tools included | -| Approvals | `interrupt_on` for the whole program; `ptc` calls are not approved | `interrupt_on` for the whole program, and for each call to a `ptc` tool | -| Subagents | `await task({...})` from a program | Not yet; planned for the next version | -| State across calls | JavaScript globals persist within a thread | Each program starts fresh; files and session state last as long as the session, when the Blueprint grants them | -| Per-user binding | — | Variables and secrets from the run's context, fixed per session | -| Retries | — | `Idempotency-Key` per tool call; a repeated call replays | -| Record of runs | — | `execution_id` per run, in the server's audit trail | -| Needs | `pip install` | A running submilli-server | - -The `langchain-quickjs` documentation says its interpreter is "not a host-memory isolation -boundary". +| Tool | Does | +| --- | --- | +| `submilli__typescript__execute` | Runs a program | +| `submilli__typescript__last_run` | Returns the thread's last run and full console | +| `submilli__typescript__packages__search` | Searches Packages available to the program | +| `submilli__typescript__packages__docs` | Returns a Package's documentation and declarations | +| `submilli__typescript__builtins__list` | Lists built-ins available without an import | +| `submilli__typescript__builtins__docs` | Returns declarations for selected built-ins | + +## Sessions, retries, and results + +- The first program run opens a Submilli session for the conversation. LangGraph can save the + session ID with the conversation state using a [checkpointer](https://docs.langchain.com/oss/python/langgraph/persistence), + so later turns reuse it. To reuse it after your application restarts, use persistent storage + for the checkpointer. Close the session with `await middleware.aclose_thread(agent, config)`. +- Blueprint variables are fixed when the session opens. A later run on the same thread with + different values raises `SubmilliSessionMismatchError` before the model is called. +- Harness secrets are sent when a session opens and are never stored in agent state. End the + session before using rotated secrets. +- Each execute call sends an `Idempotency-Key` derived from the LangChain tool call id. A + repeated call replays the recorded result. The middleware reports conflicts and incomplete, + in-progress, or unavailable replays without silently rerunning the program. +- When a session has ended, the middleware opens a replacement. If a restarted server asks for + the session's secrets, it rebinds them once and retries. +- The execute tool message contains `result`, `console`, `error`, and `denied`. Its artifact + contains `execution_id`, `session_id`, and any application-tool calls. + +Submilli enforces the Blueprint only for code that runs through Submilli. Other LangChain +tools keep their own permission boundaries. + +## Application tools + +Let programs reuse your existing LangChain tools. A model-written program can combine calls, +loop over results, branch on conditions, and process the results in one run. + +The `ptc` parameter (programmatic tool calling) tells `SubmilliMiddleware` which LangChain tools +programs can use. Submilli makes those tools available to the +program through the [`@tools` Package](https://submilli.ai/docs/tutorials/connect-deepagents/#your-tools-in-programs), +so the program can import and call them. In your Blueprint, grant [`tools.call` with a filter for +each allowed tool name](https://submilli.ai/docs/reference/http-api#application-tools). + +The middleware runs each call through a LangGraph `ToolNode` with the run's state, context, and +store. Input types come from the tool schema, and return types come from annotations. Use a +`TypedDict` or Pydantic model for typed objects. A raised exception becomes an `Error` in the +TypeScript program. + +Tools in `ptc` are not given to the model directly. If you also pass them to the agent, direct +model calls bypass Submilli's Blueprint checks. + +A session keeps the application-tool catalog it opened with. Changing the catalog causes the +thread's next call to open a new session. + +## Submilli documentation + +- [Install the CLI and server](https://submilli.ai/docs/install/) +- [Create and register a Blueprint](https://submilli.ai/docs/blueprints/start-a-blueprint/) +- [Configure permissions](https://submilli.ai/docs/reference/permissions/) +- [Run the server](https://submilli.ai/docs/server/run-the-server/) +- [Inspect the audit trail](https://submilli.ai/docs/reference/audit-trail/) +- [Use the HTTP API](https://submilli.ai/docs/reference/http-api) ## Development This package lives in the [submilli-python](https://github.com/submilli/submilli-python) -workspace, next to the `submilli` SDK it depends on. See the repository's README for setup, -checks, and releases. In `packages/langchain-submilli`: +workspace. See the repository README for workspace setup, checks, and releases. ```sh uv run pytest # unit tests; the HTTP API is mocked -uv run pytest tests/integration_tests # needs a server; see below +uv run pytest tests/integration_tests # requires a server ``` -The integration tests use a scripted model, so they need no model API key. Most of them run -against a server at `$SUBMILLI_SERVER` with the `research` Blueprint, and are skipped when none -is reachable. The others start their own server from `$SUBMILLI_BIN` or `PATH`. +The integration tests use a scripted model and need no model provider key. Most use a server +at `$SUBMILLI_SERVER` with the `research` Blueprint; the others start a server from +`$SUBMILLI_BIN` or `PATH`. ```sh export SUBMILLI_SERVER_TOKEN=$(openssl rand -hex 32) @@ -354,9 +139,8 @@ scripts/test-server.sh & SUBMILLI_SERVER=http://127.0.0.1:8128 uv run pytest tests/integration_tests ``` -Design notes, and the problems in submilli-runtime found while building this package, are in -[NOTES.md](https://github.com/submilli/submilli-python/blob/main/packages/langchain-submilli/NOTES.md). +Implementation notes are in [NOTES.md](https://github.com/submilli/submilli-python/blob/main/packages/langchain-submilli/NOTES.md). ## License -Apache-2.0, as submilli-runtime. +Apache-2.0.