Model Is Having Trouble Understanding

[alfiesal]

Hi,

I’ve been testing this tool in conjunction with the Twilio integration and noticed that the model has significant difficulty understanding what is being said to it.

However, when I use the OpenAI Playground, I don’t encounter this issue. There, the model accurately comprehends about 99% of what is spoken.

[Evanc123]

+1, it seems to drop ~20-40% of my sentences compared to the playground. I note that another twilio/openAI demo repo also reports a similar problem: twilio-samples/speech-assistant-openai-realtime-api-node#31

they speculate that:

Given that the performance difference is so pronounced, I suspect the issue is related to the audio quality or transmission when using Twilio’s phone integration. It seems like the model struggles to correctly interpret spoken words despite them being understood perfectly in other environments.

[alfiesal]

I also tested Eleven Labs in combination with Twilio, and there are no problems there.