Observability & SRE Agent
Incidents, the root cause analysis, approving a fix and setting up Slack, Teams and email.
Showing instructions for Xebit VoiceCloud.Showing instructions for Xebit VoiceKit, on your own infrastructure. Switch with the selector at the top. Updated 22 October 2026.
Observability watches every layer of a live voice agent: call quality, the model and speech providers, audio and transcription, orchestration, campaigns, human handoffs and the infrastructure underneath. When something breaks, its SRE agent opens an incident, writes the root cause analysis (RCA) and proposes a fix. Nothing changes in production until an engineer on your team approves it.
What it watches
| Area | Signals |
|---|---|
| Call quality and latency | Response time on every turn, silence and dead air, dropped and abandoned calls, barge in and talk over |
| Model and speech providers | LLM errors, timeouts and slowdowns; speech to text and text to speech per provider; which agents each provider affects |
| Audio and transcription | Recordings stored, transcription jobs completing, recordings and transcripts ready for replay |
| Orchestration, scheduler and campaigns | Turn handling and business rules, scheduled calls going out on time, campaign progress, retries and pacing |
| Human in the loop | Handoffs reaching a person, live transfers completing, calls waiting on a human |
| Infrastructure and agent behaviour | Nodes, GPUs and queues; failed tool calls; prompt regressions after a change; off script answers |
Underneath sit per call and per node metrics, transcripts, event logs and replay. For reviewing individual calls and outcomes, see Monitoring.
Incidents
Signals are checked against thresholds you set, per agent and per campaign, while calls are running. When a threshold is crossed, an incident opens with:
- a severity;
- the agents and campaigns affected;
- when it started;
- a timeline that fills in as the incident develops.
View an incident
- Sign in to the dashboard at
voice.xebit.aiand open Observability. - Sign in to the Xebit VoiceKit admin console on your host and open Observability.
- Open the incident from the list. You see the timeline, severity, the RCA with its evidence, the affected calls and the proposed fix.
- Replay any affected call with its transcript and event log side by side.
The root cause analysis
The SRE agent correlates signals across layers in the same window: which provider, which module, which node and which campaign moved first. It then writes the RCA in plain language, with the evidence behind it: metrics, event logs and the affected calls, ready to replay. A typical RCA tells you what the caller experienced, where the cause was, how many calls it touched and why it happened.
Approving a fix
Each incident ends with one recommendation: a specific change or runbook step, what it will touch and why.
- Read the RCA and the proposed fix in the incident.
- Check the evidence: replay one or two affected calls and look at the metrics it cites.
- Approve to apply the fix, or reject it.
- If you reject it, the incident stays open with everything the agent found, so your team can take it from there.
The SRE agent never changes production on its own. Voice agents talk to your customers, often in regulated work, and an automatic change can reach every call in a campaign before anyone looks. The agent finds and explains the problem; a person makes the change.
Set up Slack, Teams and email
The console always holds the full incident. Notifications make sure someone sees it without watching a dashboard.
Slack or Microsoft Teams
- Open the Observability notification settings in the dashboard or admin console.
- Connect your Slack workspace or Microsoft Teams, and choose the channel your on call team already watches.
- Save, and check that the next incident or update arrives in the channel.
- In the same settings, add the people who should receive incident notices and the finished RCA, including those who never open Slack.
Thresholds
Set thresholds per agent and per campaign, for example the response time or dropped call rate that should open an incident. Start with values you are comfortable with, then tighten them once you know what normal looks like for each agent.
By deployment
| Xebit VoiceCloud | Xebit VoiceKit on prem | Xebit VoiceKit air gapped | |
|---|---|---|---|
| Observability runs in | Xebit's infrastructure | Your network | Your network |
| Incidents and RCAs in the console | Yes | Yes | Yes |
| Slack, Teams and email | Yes | Yes, where your network allows | Only channels reachable inside your network |
| Who approves a fix | Your engineers | Your engineers | Your engineers |
On Xebit VoiceKit, Observability also watches your nodes, GPUs and queues, so capacity problems in your own environment show up as incidents.
Next steps
- Monitoring Overview: reviewing calls, outcomes and agents
- Latency Problems
- More on the capability: Observability