The recent three-hour podcast featuring computer scientist Lex Fridman and Prime Minister Narendra Modi gained widespread attention. Streamed in Hindi, English and even Russian, it has been called the “best dubbing” to date by many. The hyperrealistic AI translations, nearly indistinguishable from the original, were made possible by ElevenLabs, a three-year-old AI startup.
The technology was also deployed to translate Lex’s interview with Ukrainian President Volodymyr Zelenskyy in Kyiv in English, Ukrainian, and Russian.
While this is not the first time PM Modi has used AI for translation, it was a crucial opportunity for ElevenLabs to showcase its technology by helping world leaders reach a global audience in multiple languages.
Siddharth Srinivasan, who is heading ElevenLabs in India, told AIM that India is ElevenLabs’ biggest market yet, and the company is actively expanding its team in the country to build the future of voice AI.
He believes the opportunity is immense for a country like India, given its diversity in languages and accessibility needs.
Srinivasan revealed that ElevenLabs is nearing a team of 10 employees in the country, currently focusing on business-related roles, and has plans for further expansion.
This aligns with the trend of Western labs and startups establishing their presence in India. OpenAI and Perplexity are also reportedly preparing to expand in the country soon.
Last month, the Poland-based startup closed another funding round, raising $108 million at a valuation of $3.3 billion. ElevenLabs went from a weekend project to a startup in April 2022, when founders Mati Staniszewski and Piotr Dabkowski set out to solve poor dubbing with lifelike voice synthesis.
Relevance of Voice UI
While there is significant competition in the voice AI space, including traditional tech companies, ElevenLabs’ differentiation in the Indian market comes from catering to different accents and languages accurately.
ElevenLabs offers several distinct models for its AI audio technology, specifically tailored for different use cases, such as text-to-speech (TTS) and speech-to-text (STT).
“We have the fastest model in the world in Indic languages, which is…speech-to-text language,” Srinivasan added.
As of now, the company supports 11 Indian languages, with plans to expand further. However, Srinivasan noted, “If you get about eight to 11 languages on both sides, [you cover] 70% of India.”
Limited digital data for many Indian languages remains an obvious challenge. ElevenLabs is addressing this by building datasets through strategic partnerships and community involvement.
Overall, the company’s multilingual AI model supports 29 languages, providing highly realistic, emotional voices in each. Moreover, their conversational AI model supports 32 languages, enabling natural, real-time conversations. They’ve also developed an advanced speech-to-text model in 99 languages, including 11 Indian languages.
Growing Creator Economy in India
ElevenLabs has effectively productised its technology to cater to content creators and developers. Their API is user-friendly, allowing large-scale integration, particularly appealing to developers and businesses looking for scalable solutions.
Podcasters generally are a huge target audience for the company.
In addition to Fridman, Indian tech podcaster Varun Mayya employs ElevenLabs’ technologies for his own brand and other companies. The company has also partnered with neuroscientist and podcaster Andrew Huberman to dub content for his Huberman Lab podcast into Hindi and Spanish. Their partnerships with Spotify help in producing AI-narrated audiobooks.
Srinivasan mentioned how Star Sports uses its technology to dub Steve Smith’s voice in Hindi and Tamil, and localising cricket content for a broader audience.
In terms of new features and partnerships, last year ElevenLabs launched GenFM (a feature similar to Google’s NotebookLM) for multispeaker podcasts and partnered with Pocket FM to help writers turn stories into audio.
In addition to the content creation market, industries of focus in India include customer support and education.
Recently, the company also hosted a hackathon in Bengaluru as part of its global tour, with India recording the highest number of registrations.
The showcased projects included an AI agent for emotional support, an AI video framework with designated roles for content creation, and a voice AI solution for rural India to enhance accessibility in areas where chatbots are limited.
Future of AI is Not Unipolar
Srinivasan, who comes with a decade of experience at YouTube, understands the creator ecosystem in the country. He believes the future of AI will not be limited to just one interface – although voice will be a massive pillar of it.
He pointed to a consistent trend in media and technology towards multimodal interactions, and even going forward people will engage with AI across voice, text, video, and visuals.
“Even before the rise of generative AI, people were engaging with technologies like Alexa, Google Assistant, and Siri,” he said.
But, he is confident that voice will remain the most natural interface, as it continues to be the most basic and widely used form of communication, deeply embedded in both consumer and business interactions.
Srinivasan also acknowledged the risks of deepfakes and emphasised that ElevenLabs uses moderation, permission for consent, and traceability to prevent misuse.
