AI For Business | The Right Question: LLMs VS Compact (Small) Language Models
On July 18, we launched our smart assistant Exie, which allows business owners, marketers, and SMM specialists to hire a virtual assistant to manage communication with users in their Instagram business account.
The solution to our business challenge presented us with a choice: to either use a large LLM system, artificially limiting its capabilities, or to opt for a smaller system. And we decided to go with ChatGPT.
To prevent the neural network from hallucinating and generating a bunch of nonexistent facts, we have to significantly restrict it in the context in which it should operate.
If you’re in eCom, you definitely don’t want your online-store AI-assistant to tell your customers how to create SEO texts or solve programming tasks. It’s really important to properly limit the context that the neural network operates in.
RAG architecture became a solution for managing the conversation context.
Here’s an amazing illustration of how all LLMs work in conjunction with RAG architecture in the current realities (including our product Exie).
Specifically, in our case, conversation context is based on the information provided to us by the user. Essentially, this information serves as Exie’s knowledge base, which allows it to respond to users only within the scope of what they have trained it on.
A fair question pops up:
Why do we need to use an LLM for such a small context? Why not use compact language models instead?
It is a good call to rely on a compact language model rather than using an LLM, and that will be enough for each specific case.
However, since we provide our app to users worldwide speaking different languages, using an LLM (in our case, OpenAI ChatGPT) allows us to work effectively in any supported language.
It was important for us to ensure that businesses could convey information about their company or product to the end customer in any language, regardless of the language in which the assistant was trained.
That is why it was easier for us to limit a large LLM model while maintaining multilingual support than to add other languages to smaller models with limited capabilities.
You can message Exie in Instagram Direct at https://ig.me/m/exie.app to see how it works.
Create your own assistant for Instagram Business https://exie.app/