rokan.

Rokan and Langfuse

Langfuse records every step your agent takes, scores it with evals and manages your prompts. Rokan starts where a Langfuse trace shows a failure at one customer: it finds why it failed there, writes the fix, and replays the fix on every customer's past runs before it ships. Rokan reads Langfuse exports, so you keep Langfuse and add Rokan next to it.

Updated . Langfuse described from langfuse.com on that date.

What Langfuse does

Langfuse calls itself "Open Source Agent Evals & Observability". Its "hierarchical traces capture every LLM call, tool invocation, and retrieval step", and you can filter them by user, session, cost, latency or your own metadata. It runs evals with "LLM-as-a-judge, heuristic functions, or human review", keeps prompts outside your code with deployments and rollbacks, and has a playground, experiments, datasets and human annotation. It is MIT-licensed and can be self-hosted.

If you want to see what your agent did and measure how well it did it, Langfuse covers that.

What Rokan adds

  • The why, for one customer. Rokan reads that customer's failing runs with their own rules, hours and config, and says in plain words why the agent failed there and not elsewhere.
  • The fix, written. A prompt or config edit, or a code change as a branch.
  • A replay on every customer. Before the fix ships, Rokan replays it on every customer's past runs. If it would break a run that works today at another customer, it is blocked or scoped to the one customer.
  • Your approval. A person on your team approves every fix.

When Langfuse alone is enough

  • You have one customer, or several who use the agent the same way.
  • Your team reads traces, finds the cause and writes the fix quickly.
  • You already keep datasets from each customer and run a Langfuse experiment before every prompt change. That covers much of what Rokan's replay does.

When you'd add Rokan

  • The agent fails at one customer and works at the others, and the traces don't say why.
  • Each customer has its own rules, hours or setup, and a fix for one keeps breaking another.
  • Nobody on the team has time to read a week of traces per customer and write the fix.

How to send a Langfuse export to Rokan

In Langfuse, open the tracing table and filter it to the window you care about. Filters carry over to the export. Click the export button, choose CSV or JSON, then download the file from Project Settings, Exports (Langfuse docs). The Langfuse SDKs and API work too.

Send 20 to 50 runs, from more than one customer if you can, plus your prompt and config. If the export doesn't show which customer a run belongs to (a user ID, a session or a metadata field), tell us the field in one line. The steps are on the trial page.

What Rokan does not do

Rokan's replay is a model playing the user's side against your prompt and rules. It does not call your tools or your voice stack. Rokan is also early: it has no paying customers yet.