Discussion on "🐍 Answer the phone! with Python"

Kevin Naidoo · 2024-07-07T14:14:16.595Z

Python is a powerful language that can do many things. Most people use it for machine learning or web development, but did you know that it can also interact with hardware and other services like SIP? 💡 Check out my newest article on how to build a...

Amazing article ! Thanks a lot for the insights.

I am currently contemplating building an in-house client for SIP virtual phone numbers.

We are building an AI conversational agent on top open AI realtime audio and after diving into how the VOIP works I realized it was not difficult to cut out Twilio. Since it would represent most of our costs in the future it makes sense to remove it.

My issue is that I am not sure if the client alone would be a good enough replacement. In your article you mention audio quality, background noise, silence being handled properly by Twilio.

I can't find relevant information online on the behind the scene added value of using Twilio as an SIP client.

The worst would be to build the client ourselves (which is not difficult) and underestimate what we really goes on in the backend at Twilio.

Do you have any insights or resources on the matter ?

Your help would be greatly appreciated!

Happy new year

Hi John, Happy 2025, hope you have an awesome year ahead :-)

Thanks for reading, and glad this was useful to you.

Twilio is not a SIP client, they provide a managed VOIP solution, so basically, you can buy telephone numbers from them and programmatically connect code to a real-time call.

With the media streams, they fork the audio from the real-time call and stream it to your WebSocket server.

The WebSocket server then receives the audio and you can transcribe or pass that on to a real-time model and then send the audio back to the client.

You can achieve the same by setting up Asterisk, but that is a bit more complicated and a pain to manage.

One problem with Twilio is that the audio is streamed at 8Khz and is base64 encoded so there is a conversion step that might slow down your call. Here's some documentation on the Twilio approach: twilio.com/docs/voice/tutorials/consume-real-time…

A better approach would be WebRTC(I built a simple client here: github.com/kevincoder-co-za/zazu-voiceai) or just build a SIP WebSocket server and use a provider like 3cx.

I did play with a few libraries, before going with Twilio (it was an MVP so shipping fast was essential). Maybe they'll be of use:

github.com/sipsorcery-org/sipsorcery mizu-voip.com/Software/SIPSDK/JavaSIPSDK.aspx pjsip.org github.com/emiago/sipgo linphone.org/en/voip-unified-communications-softw…

Mizu worked nicely, I just didn't have much time to fully implement the SDK and it's a commercial product so there are licensing costs.

Hope this helps.

Hello, I just have one question. Were you able to play audio files like mp3 and wav over the phone? I have real problem with that plus it is Europe it supports puma (alaw).

Hi Zaid, thanks for the question. Yes, you can but you need to format the audio into a WAV and ensure that the sample rate is 8000 Hz(mono channel).

Then you can send that WAV file back to the user:

audio = AudioSegment.from_mp3("your_file.mp3")

audio = audio.set_frame_rate(8000).set_channels(1)

audio.export("output.wav", format="wav")

f = wave.open('output.wav', 'rb')

frames = f.getnframes()

data = f.readframes(frames)

f.close()

call.write_audio(data)

I suggest looking into my web socket tutorial over at: kevincoder.co.za/how-i-used-voice-ai-to-bring-ima…

You don't need to use WebRTC if you need the phone system, you can use Twilio media streams or forward the call from a PBX server like Asterisk.

Hope this helps.

Kevin Naidoo Thank you for the response. Actually I have tried that before and it makes the audio very noise (painful for the ear) and unbearable. I have a bit better audio by processing the raw audio into g711 (alaw) format but the RTP couldn't send the packets or the stream in un choppy manner. I have perfectly working pipeline for voice conversation on laptop but not over the phone.

I see. Yeah PyVOIP is more for educational purposes, its not efficient for production. You probably should use pjsua/pjsip or twilio media streams for your use case.

Discussion

🐍 Answer the phone! with Python

Responses(2)

Recent in Forum

Search Hashnode

🐍 Answer the phone! with Python

Responses(2)

Recent in Forum