MixedEXP-06iones labUpdated on 20 September 2026
Jarvis & OpenClaw: autonomous agents on our support tickets
The questionCan an AI agent handle the first level of our maintenance tickets on its own without degrading service quality?
Verdict
Triage and qualification are reliable and fast. Autonomous resolution remains too unpredictable: we keep the agent as a copilot, never as a decision-maker.
DecisionAdopted for triage with human validation. Autonomous resolution ruled out for now.
- Status
- Mixed
- Duration
- 6 weeks
- Team
- 2 developers, 1 run lead, 1 project manager
- Stack
- OpenClaw
- Jarvis
- MCP
- Redmine
- Claude
- Postgres
91%correctly qualified ticketscategory + priority, across 412 tickets
34%resolved without interventionsimple cases only
6.8 smedian latency per ticketfrom reading to proposal
7risky actions blockedby the guardrails, over 6 weeks
01
Context
Our run teams receive several hundred tickets every week: bugs, minor change requests, usage questions, incidents. First-line handling consists of reading, qualifying, prioritising and routing these tickets before a developer picks them up. It is repetitive work, but it requires understanding the project context and the client’s history.
We already had an internal assistant, Jarvis, able to answer questions about our projects from our documentation. The lab’s question was how to move from an assistant that answers to an agent that acts: read the ticket, consult the repository and the database, propose a qualification and, for simple cases, apply the fix.
02
What we tested
We set up two configurations in parallel on a copy of our Redmine instance, fed with real, anonymised tickets from three maintenance projects.
- Jarvis in agent mode: a single model, read access to tickets, code and documentation through MCP servers, no write actions.
- OpenClaw in orchestration mode: a planner agent that delegates to specialised sub-agents (code reading, bug reproduction, drafting the reply), with write access to a dedicated branch.
- A shared set of guardrails: allow-list of actions, token budget per ticket, blocking of any command touching the production database, mandatory human validation before any client comment.
Each ticket handled by an agent was then reviewed by a developer, who scored the proposed qualification (category, priority, component) and, where relevant, the fix. We processed 412 tickets over six weeks, 138 of them with a fix attempted.
03
Results
- 91%
- correct qualification (category and priority)
- 34%
- fixes accepted without changes
- 22%
- fixes accepted after edits
- 0.04 EUR
- average cost per triaged ticket
On triage, the result exceeds our expectations. Both configurations qualify more than nine tickets out of ten correctly, and the proposed priority is the one the lead would have chosen in 87% of cases. The internal first-response time drops from several hours to a few seconds, which makes it possible to route a critical incident before a human has even read it.
On resolution, it is a different story. OpenClaw cleanly resolves a third of the simple tickets (text to correct, validation rule, configuration). But of the 138 attempts, 18 fixes were technically functional and yet wrong: the agent fixed the symptom without understanding the business rule, or changed an intentional behaviour. These cases are the most dangerous, because they pass the tests.
- Median latency of 6.8 s for a triage, but up to 4 minutes for a fix attempt including bug reproduction.
- The guardrails blocked 7 actions in six weeks, including two deletions of test data that the agent considered “obsolete”.
- OpenClaw uses on average 11 times more tokens than Jarvis for a marginal qualification gain (+2 points).
- The agents are excellent at writing the technical summary of a ticket the client described poorly: a unanimous time saver among developers.
“The agent saved me time on every ticket, except the ones where it made me lose trust. And those you can’t spot at first glance.”
04
What we conclude
Autonomy is not a switch, it is a dial. On reading, understanding and qualification, the agent is already better than a rushed human, and we have no reason to go without it. On decision and modification, reliability is not sufficient for a client who pays us precisely to avoid surprises.
OpenClaw’s multi-agent orchestration is attractive on paper, but its cost and complexity are not justified for triage. It becomes interesting only when the task genuinely requires several independent steps (reproduce, fix, test). We keep it for those cases, in isolated-branch mode with mandatory review.
05
What’s next
- Jarvis for triage has been deployed on our internal run projects since September, with human validation before any comment visible to the client.
- The fixes proposed by OpenClaw arrive as pull requests, never merged automatically.
- Next iteration: measure whether an agent trained on a project’s full history reduces the “functional but wrong” fixes.