An AI Agent Is Worth Its Completed Task

Why token prices say little about an AI agent's value, with AWS and Condé Nast cases and business metrics that matter.

  • What AWS showed
  • Why token price is misleading
  • First, measure what the task costs today
  • Where agents usually break down

Trigger

  1. In one week, AWS published five analyses of agents on Bedrock.

  2. They include a three-agent pipeline that mixes a music track over several days, searches 140,000 videos at Condé Nast, and runs RAG on insurance claims.

  3. The most useful business insight came in the announcement of the GPT-6.1 Sol model: calculate an agent's economics across the entire task.

  4. Token price is one line item in this estimate, while the budget usually goes elsewhere.

What AWS showed

  1. The case studies cover different topics. Together, they show how agents operate in production. - Condé Nast.

  2. Editors at Vogue, GQ, Vanity Fair, and Wired spent an average of 250 minutes on each search through an archive of 140,000 videos.

  3. The old search could see only titles and descriptions and could not look inside the video.

  4. The company built multimodal search that indexes video content. - Insurance claims.

  5. Answers about payouts are stored in different places: adjuster notes, repair estimates, police reports, payment registers, and scans. AWS brings them together in a knowledge base with RAG—generating answers based on the documents found. - Music pipeline.

  6. One agent generates audio on an instance GPU, while two others open the .wav file it recorded and refine it.

  7. The work takes several days, while serverless sessions are limited to a few hours, so AWS moves agents to dedicated instances. - GPT-6.1 Sol. The announcement says it directly: the agent collects data, calls tools, tries approaches, fixes errors, and checks the result.

  8. Every incorrect step adds model calls, latency, and manual work.

Why token price is misleading

  1. The agent works in a loop: decide, act, verify, then decide again.

  2. Task cost consists of the number of iterations, the cost of each iteration, waiting time, and the employee minutes spent investigating failures. An example with illustrative figures.

  3. The cheaper model completes the task in 12 steps and hands it over to a human in one-third of cases.

  4. The model that costs twice as much per token completes the task in five steps and hands one task in ten to a human.

  5. Even the API bill for the second model is lower: five steps at twice the price cost less than twelve at the single-step price.

  6. If you add the employee’s salary, the difference increases several times over.

  7. The figures here are hypothetical, but the mechanics work exactly this way, and you can test them on your own data in a week.

  8. That is why models are compared by the cost of successfully completing one task, not by the price list.

First, measure what the task costs today

  1. The most useful figure in the Condé Nast case is the before number: 250 minutes spent searching.

  2. The company knew how much the task cost without AI and was therefore able to calculate the impact.

  3. Most pilots I have seen started without this kind of measurement, and six months later no one could say whether they had paid off.

  4. Time to usefulness (TTU) is measured in the units that reflect the problem: editor minutes, adjuster hours, or days until a product appears in the storefront.

  5. If an agent has not moved any of these figures after a quarter, it is a demonstration, and the CFO will sooner or later call it exactly that.

Assess where AI can deliver impact in your process

Where agents usually break down

  1. In the insurance case, AWS presents two queries.

  2. The customer asks whether their claim has been approved.

  3. The claims adjuster needs all open auto claims over $10,000 from last month.

  4. The first query is classic RAG: find the documents and summarize them.

  5. The second is selecting fields with filters and totals, which vector search handles poorly: it returns similar texts and cannot guarantee a complete list. In a working setup, engineers separate these queries: semantic questions go to RAG, while reporting questions use a structured query to the accounting system.

  6. This is exactly where demos most often fail to reach production.

Infrastructure for long-running tasks

AWS’s music pipeline demonstrates the second problem.

A task lasting several days survives restarts, session changes, and errors by individual agents.

The model’s context lasts only until the end of the session, so state is stored externally: in files, queues, or a database.

Agents pass artifacts to one another

In enterprise integration, a bus has long played this role. Orders, stock levels, and item cards pass through Apache Kafka or Datareon, so a failure of one participant does not interrupt the process.

Agents operate under the same conditions, and the engineering solutions already exist.

How we assemble this at KT.Team

  1. In AI-native integration projects, we start with three things. A unified gateway to models.

  2. The LLM & Security Gateway routes calls to Claude, OpenAI, GigaChat, YandexGPT, and Qwen.

  3. The gateway tags each call with a task ID, so cost is calculated per completed task.

  4. It also masks personal data, and the model can be changed without rewriting the agents. Tools through MCP.

  5. The agent works with 1C, PIM on Akeneo or Pimcore, and Elasticsearch through MCP servers with explicit permissions. For each server, we define what the agent may read and change, and log every action it takes.

  6. Hybrid search. We build RAG on Elasticsearch: semantic vector search works together with exact field filters. The query “all open claims above $10,000”

  7. It returns the complete list, while summaries of similar documents remain for semantic questions. In catalog projects, we apply the same approach to generating item-card text, facets, and taxonomy in PIM. We measure two things: how many minutes a content manager spends on an item card before and after implementation, and what share of cards pass moderation without edits.

Verdict

An agent is evaluated using three figures: the cost of one completed task, the share of tasks solved without human intervention, and employee minutes before and after implementation. If a team cannot provide these figures, it does not yet know whether its agent works. Models will continue to get cheaper, and companies that calculate the full cost of a task will see these savings in their reporting sooner than others.

Discuss the article: An AI agent is worth as much as…

Enter your email or phone number so we can get back to you.

Send via: