.png)
2 hours ago
1
A Practical Guide to Cost-Effective Enterprise Integration and Real-World Use Case
As LLM popularity surged, many organizations began to integrate AI models into their solutions, ranging from simple chat bots to complex AI agents. Our firm was no different, and my department, in particular, focused on analytics.
To better understand this new AI reality, I first explored free models via the Ollama package in my free time, which proved to be a very helpful tool to start with. I then reviewed the simple example, “How to Set Up and Run DeepSeek R1 Locally With Ollama,” provided by . It served as an excellent kickstarter.
The next step was to design a simple RAG-powered AI chatbot for one of our products as a Proof of Concept (POC). The goal was to find the most optimal solution that would leverage the same infrastructure and resources we already had in place, thereby saving us money and time in developing these AI-powered capabilities.
CH1: POC Chatbot Journey
What did we need for basic AI Chatbot? We needed a a) database capable to store not only chat data, i.e. chat history, user interactions etc. but also a knowledge base which would be easily searchable and feed into chat context. In the context of LLM chatbots, that the data must be stored in the form of embeddings so that when searched, it can give the best related data and not only. Also we needed b) a reliable AI SDK for both frontend and backend development of chat bot. c) access to AI models and API.
So what did we have in place?
a) A Postgres database in the AWS cloud, NodeJs application and an AI Gateway with all necessary models mainly from Anthropic and OpenAI. Postgres can store embeddings with the pgvector extension and it is available in AWS RDS Postgres too (just run CREATE EXTENSION vector in db console). For small and medium application (which is 90% of use cases in big corporations) Postgres pgvector will solve most of the use cases.
b) The JavaScripts AI SDK has all the necessary tools to develop AI-powered applications with React, Vue and Svelte.
c) Our firm’s central platform provides reliable access to most AI models. Alternatively, one can just go to theofficial websites of those model providers and buy access to their APIs (i.e the OPENAI API Platform ). You can stick to free models, by running, for example, Ollama as a service and use its API, however, our experience indicated that free models did not give us satisfactory results.
Long story short, we had all we needed to kickstart our AI chatbot POC, and on a very high level, the POC design looked like this.
Press enter or click to view image in full size
Our POC was very successful: our chatbot was able to retrieve the necessary data from the database to provide context for the chat and deliver logical answers. However, the primary factor behind this success was the fact that we were able to integrate AI chatbots and agents into our existing infrastructure and resources within just 2–4 weeks.
CH2: Integration of AI Agents into our existing products
As a next step, we had to define what would be our template for AI Agents and Chats integrated inout our existing and future app based on successful POC? We already had the answer, but we needed more specific list:
- Cloud Provider: AWS
- Database Engine: Postgres
- ORM : Drizzle
- Application Stack: Node.js and React for app development, utilizing the AI SDK as a core component.
- AI Models: Anthropic and OPENAI
When we started our first AI integration into an existing app, we spent around two weeks for an end-to-end solution, following this choice of tools and simple architectural design. While we expect our architecture and design will become more complicated (not to mention many details we’ve simplified here for the sake of clarity) as we provide more complex solutions, we have a robust foundation until that time.
If you want to see a glimpse of Postgres + NodeJs + AI RAG architecture, check my demo repo.
CH3: The Importance of AI Tools in Corporate Language and How RAG Solves Our Problems
One of the core challenges we addressed during our first Proof of Concept (POC) was the terminological gap between the unstructured language used by our users in the chat interface and our firm’s formal, internal data structure.
To bridge this, one of our key tools operates as a natural language interpreter. It translates a client’s natural language question into our firm’s proprietary terminology, allowing our AI agent to correctly map the request to a definitive concept and select the appropriate next action.
For a simple analogy, consider a pharmacy business (though our product deals with analytical data and formal terminology): If a client asks, “Give me something against a headache,” the interpreter first translates this into the actionable, internal query: “The user has a headache and is requesting an Over-The-Counter (OTC) medicine.” This specific translation allows the agent to efficiently search its catalog of medicines (our database with embeddings) and return the best, most relevant response. This process of translation is vital for handling the formal, structured data inherent in our analytical products.
Read also:
