Quick answer
When evaluating voice vendors for production calls, the most actionable metric is the 95th‑percentile (p95) turn latency. This metric captures the worst‑case latency you can expect for 95 % of the user‑initiated turn requests in a real‑world workload. By measuring p95 you can compare vendors on the same footing, ensuring that the occasional outlier does not skew the overall user experience.
Turn latency is the time from when a caller speaks to when the system returns a spoken response. It is a sum of Speech‑to‑Text (STT), AI processing (LLM), and Text‑to‑Speech (TTS) latency, plus any network jitter or queueing delays. In practice, vendors report a total latency figure and break it down per component in a latency scorecard.
A low p95 latency guarantees that most users hear a response within the expected time window, reducing caller frustration and improving call completion rates. It also aligns with compliance thresholds for certain industries, where exceeding a maximum turn time can trigger audit flags. For voice AI vendors, a consistently low p95 is a differentiator that customers can validate in a bake‑off.
How Voxovo AI works on Voxovo
To conduct a bake‑off, first set a realistic workload that mirrors your typical call volume and mix of intents. Use a test harness that can simulate concurrent callers, record timestamps at each pipeline boundary, and aggregate the data into a CSV or database. Ensure the test runs long enough to capture the tail distribution, typically 30 min to an hour per vendor.
Voxovo AI provides a built‑in latency scorecard that timestamps every stage of the voice pipeline. The dashboard exposes per‑turn metrics for STT, LLM, TTS, and the overall turn, all keyed by the caller session ID. By exporting these metrics you can compute the p95 directly from the vendor’s own instrumentation.
A typical turn in Voxovo follows this sequence: caller speech → STT (Voxovo 1.0 Speed or Smart) → AI prompt → LLM (Gemini or custom) → TTS (Cartesia default, ElevenLabs optional) → audio playback. Each stage emits a millisecond timestamp, allowing you to isolate bottlenecks, such as a slow LLM or a congested TTS endpoint.
Computing the p95 is straightforward: sort all turn‑completion times, find the value at the 95th percentile index, and record it. In a CSV you can use a simple script or spreadsheet formula: `P95 = SORT(times)[floor(0.95 * N)]`. Repeat the calculation for each component if you want to see which stage dominates.
Implementation checklist
When interpreting the p95, compare it against your SLA. If your target is 500 ms and a vendor reports 650 ms, you have a 150 ms margin that may be unacceptable in a high‑stakes call center. Conversely, a vendor with a 400 ms p95 but a 600 ms tail in rare cases may still be preferable if the tail is negligible for your use case.