Pi 1.0 and Pi Durable: Why AI Agents Need Extra Wrapping

This is the typical path of an agent project in a company.

In a demo, the agent is given three tools and a short task, and the pilot succeeds. In production, the agent is connected to 1C, PIM, CRM, and a warehouse system, bringing the number of tools to forty. Sessions last an hour or longer.

The agent responds more slowly, the token bill grows several-fold, and the task stops halfway because one of the APIs timed out.

The manager sees the spending but not the result.

After a quarter, the project is shut down and AI is labeled a failure. In stories like these, the model is usually not the problem.

The context window is the agent's working memory, and every token in it is paid for on every call.

Descriptions of forty tools, intermediate API responses, and step history fill this memory before the agent even starts the task.

### Codemode In the traditional approach, the agent calls tools one by one, and each response enters the context in full. In Codemode, the model writes a short script: the script calls the required tools, filters the data, and returns only the result to the model. In a publication on code execution with MCP, Anthropic gave an example where this approach reduced usage from 150,000 to 2,000 tokens, a 98.7% reduction.

Savings in a specific project depend on the task, but the order of magnitude is illustrative. ### Lazy tool loading

The agent keeps a short tool catalog in context and loads the full description only for the tool needed at the current step.

Forty connected systems no longer occupy memory during steps where they are not involved. ### Cache warming

Providers cache the unchanged beginning of a request: the system prompt, instructions, and tool descriptions. With Anthropic, reading from the cache costs about 10% of the base price for input tokens.

During cache warming, this prefix is sent in advance, so the user's first real request is cached and the response arrives faster. ###

Durable execution is a long-established integration technique: after each step, the process state is saved, and after a failure, the process resumes from the last checkpoint.

If an external API fails on the agent's thirtieth step, durable execution allows that single call to be retried.

Without it, the task must be restarted and the tokens paid for a second time.

This describes the principle; the implementation in Pi Durable should be verified in the project documentation.

Around the same time, Latent Space published an interview with Alex Zhang from MIT about recursive language models (RLMs).

In this approach, the long context is stored as a variable in the code execution environment.

The model writes code that examines this context, splits it into parts, and recursively calls itself on the fragments.

This is the same idea as in Codemode: the model manages its memory through code and does not load unnecessary content into it. Latent Space ranks

Zhang alongside the podcast's previous guests. Shunyu

After its 2024 release, Yao created Operator at OpenAI and now serves as Tencent's chief AI scientist.

Jack Morris, a 2025 guest, became a co-founder of Engram, and the company was valued at $600 million. OpenAI, Tencent, and venture investors pay for people who know how to manage model context.