Is there open source alternative for VAPI or retellai?

Yes. Have you cheched livekit? Vapi is also built on media framework by daily.co. They have an open source version of voice ai called pipecat. You can check it over github.