Building a Local Voice Assistant with LLMs and Neural Networks on Your CPU Laptop
Please enjoy the read: Free link!
With the rise of multimodal Large Language Models (LLMs), we can now interact with them in more ways than just typing text, like using audio inputs. OpenAI has recently released a voice feature for ChatGPT, allowing one to talk directly with the chat platform. This opens up a myriad of novel opportunities and applications built around it.
As machine learning and Data Science practitioners, it's an exciting time to be involved. Using OpenAI's realtime speech to speech APIs, you can create a voice assistant powered by these multi-modal LLMs. However, if you are interested in the open-source libraries, you can build a voice assistant as well, completely in a local environment and without subscriptions to proprietary APIs!
Why local voice assistant?
- Data privacy
- No API calls limit
- Fine-tuning models
First, I am sure most people who use mainstream generative AI chatbots are aware of the data that was transmitted through their servers. A lot of people may be concerned about the data privacy issue and leak of information.
Second, using proprietary APIs can be subject to the API calls limitation. For example, the OpenAI's realtime API is rate-limited to approximately 100 simultaneous sessions for Tier 5 developers, with lower limits for Tiers 1–4.
Third, the LLMs hosts behind these proprietary API gates are powerful but are not fine-tuned or tailored to your specific domain. On the other hand, a locally hosted LLMs-based voice assistant allows you do inference without transferring data over to the cloud server. And you can choose lightweight LLMs to fine-tune and deploy on a CPU machine (i.e. a laptop or mobile device). How nice is that!
