CallsQA Trial Batch Dashboard

CallsQA v0 trial batch: STT and evaluator comparison · run 20260709T140804256811Z
7cases
28ok steps
0errors
$0.2859known cost
371.053sstep time

HU 1 via local MLX Whisper

hu-1-mlx

$0.0284
546transcript chars
70.4sstep time
$0.0284known cost
mlx_transcript: $0.0000, 8.6s
Gemma 4 26B
3 / 3 / 2
cost $0.0178 · 51.3s · outcome unknown

The transcript is truncated and ends abruptly. While the agent's initial greeting was professional, the conversation ends immediately after the agent verifies the order number, leaving the customer's primary concern (the product replacement) unaddressed in the text. The agent's tone was also somewhat blunt when correcting the customer's order number.

GPT-5.5
0 / 0 / 0
cost $0.0106 · 10.4s · outcome unknown

A transcript jelentősen hiányos vagy megszakadt. Csak a problémafelvetés és a rendelési szám bekérése látható, de nincs elegendő információ az ügyintézés teljes hangnemének, érthetőségének, hatékonyságának vagy kimenetelének értékeléséhez.

HU 1 via OpenAI gpt-4o-transcribe

hu-1-openai-gpt4o-transcribe

$0.0286
629transcript chars
46.9sstep time
$0.0286known cost
openai_transcript: $0.0065, 5.5s
Gemma 4 26B
4 / 4 / 3
cost $0.0109 · 31.4s · outcome unresolved

The transcript ends abruptly mid-sentence. The customer realized they provided an incorrect order number and apologized, meaning the initial complaint regarding the product size discrepancy was never addressed.

GPT-5.5
4 / 5 / 4
cost $0.0112 · 10.1s · outcome partial

The agent was polite and quickly asked for the order number, then clarified that the provided number did not match their order-number format. The customer's underlying issue was not resolved because it appears they may have contacted the wrong company or referenced the wrong order, and the transcript is cut off at the end.

HU 1 via ElevenLabs Scribe v2

hu-1-elevenlabs-scribe

$0.0283
727transcript chars
50.3sstep time
$0.0283known cost
elevenlabs_transcript: $0.0040, 9.6s
Gemma 4 26B
5 / 5 / 3
cost $0.0097 · 28.0s · outcome unresolved

The agent was polite and clearly communicated the specific format required for the order number. However, the call was inefficient in terms of resolution because the customer provided the wrong information and ended the call before the actual complaint (the product size discrepancy) could be addressed.

GPT-5.5
4 / 5 / 4
cost $0.0147 · 12.7s · outcome unresolved

Az ügyintéző udvarias és segítőkész volt, gyorsan kérte a rendelési számot, majd jelezte, hogy a megadott szám nem megfelelő formátumú. Az ügyfél ezután arra következtetett, hogy talán rossz számot hívott, és megszakította az ügyintézést. A probléma nem oldódott meg, és nem történt további azonosítási próbálkozás.

45998 OpenAI Whisper channel merge

45998-openai-channel

$0.0410
12681transcript chars
41.1sstep time
$0.0410known cost
source_transcript: n/a, 0.0s
Gemma 4 26B
5 / 5 / 5
cost $0.0118 · 33.9s · outcome partial

The operator handled the follow-up call very professionally. She clearly explained the new logistics consolidation model and proactively addressed the customer's specific pain point regarding package identification (labels being covered by other carriers). The call concluded with a clear, actionable next step: the operator will provide a comprehensive price list for all applicable countries and investigate the possibility of adding identifiers to the packages.

GPT-5.5
4 / 4 / 3
cost $0.0292 · 7.2s · outcome partial

Operator was polite and helpful, explained the consolidation/delivery concept and confirmed next steps. The customer's questions were mostly addressed, but final resolution depends on a follow-up email with pricing for supported countries and confirmation of package labeling options.

45998 MLX Whisper channel merge, strict filtered

45998-mlx-channel

$0.0496
14629transcript chars
41.9sstep time
$0.0496known cost
source_transcript: n/a, 0.0s
Gemma 4 26B
4 / 5 / 4
cost $0.0120 · 34.6s · outcome partial

The operator was professional and polite. They successfully addressed the customer's specific pain point regarding the difficulty of identifying packages due to overlapping labels and proposed a solution (adding order numbers/names). While the pricing inquiry was not resolved during the call, a clear follow-up action was established (the operator will send a price list for the relevant countries).

GPT-5.5
4 / 5 / 3
cost $0.0376 · 7.3s · outcome partial

Operator was polite, apologetic, patient, and engaged with the customer's operational concerns. She explained the proposed consolidation/delivery model and confirmed the customer's needs, but the explanation was lengthy and sometimes indirect. The call ended with a clear next step: operator will verify available countries, pricing, and whether packages can be labeled with customer name/surname.

45998 existing ElevenLabs Scribe v2 channel merge

45998-scribe-existing-channel

$0.0396
10563transcript chars
41.0sstep time
$0.0396known cost
source_transcript: n/a, 0.0s
Gemma 4 26B
5 / 5 / 4
cost $0.0099 · 28.5s · outcome partial

The operator was highly professional, empathetic, and used personal experience to build rapport with the customer regarding the labeling issue. While the pricing request was not resolved during the call, the operator established a clear follow-up plan to provide a comprehensive price list and investigate the labeling solution.

GPT-5.5
4 / 5 / 3
cost $0.0297 · 12.5s · outcome partial

The operator was polite, apologetic, patient, and empathetic. She explained the Zlín consolidation/direct delivery concept and confirmed next steps, but some answers were lengthy and not fully definitive during the call. The customer’s main requests were acknowledged, with follow-up promised for pricing by country and confirmation of package labeling with the client name.

45998 fresh ElevenLabs Scribe v2 full audio

45998-scribe-fresh-full

$0.0704
9058transcript chars
79.5sstep time
$0.0704known cost
elevenlabs_transcript: $0.0362, 42.4s
Gemma 4 26B
5 / 5 / 5
cost $0.0082 · 23.7s · outcome resolved

The agent demonstrated excellent empathy and professionalism. She effectively addressed the customer's specific pain point regarding package identification (labels being covered by other carriers) and provided a clear solution path. She also proactively expanded the scope of the service to include other countries the customer mentioned. The call ended with clear next steps: sending a comprehensive price list and investigating labeling options.

GPT-5.5
4 / 5 / 3
cost $0.0260 · 13.3s · outcome partial

Agent was polite, apologetic, empathetic, and engaged with the customer's concerns. She explained the proposed consolidation and delivery process and confirmed the next steps, but some answers were lengthy and not fully concrete. The customer's main request remains pending: pricing for all applicable countries and confirmation of whether the customer name can be added to parcels.