From 'RAG's to Riches: How to Leverage AI to Get More Out of Your Company's Data
Engaging with Data like Never Before with AI
Artificial Intelligence (AI) is evolving rapidly, transforming the way we interact with technology and handle data. AI models like GPT-4 by OpenAI have the potential to enable radical change in business processes, marking a new era in digital communication. Today we’re here to explore the real-world use cases for AI, and how it can make a meaningful impact in your business.
Imagine being able to “talk to” your company’s data as if it were a team member, asking questions about sales trends or operational efficiencies. Instead of writing complex SQL queries, or sifting through spreadsheets, you could simply ask, “What were our best-selling products last June?” or “How has operational downtime trended over the past six months?” This is the sort of revolutionary change that AI allows. While this sounds very compelling, there are some obstacles to overcome to make this vision a reality.
Train AI That Works With Your Data
Large Language Models (LLMs) like GPT-4 offer immense potential in the field of data analytics, but their application comes with inherent challenges. One of these is the fact that GPT-4 is trained on a large dataset that does not include your company's data. ChatGPT can tell you the capital of France or write python code, but it cannot answer questions about private data that it wasn’t trained on. This presents a problem because we can't feasibly retrain or fine tune the model every time new data is added to a dataset.
There's also a limit to how large a prompt can be when interacting with the model due to token restrictions. For GPT-4, the API has a cap of 32,000 tokens per prompt. Put simply, you can only “feed” the model with roughly 25,000 words per question you ask. This might seem substantial but consider the case of a 1,000,000-row SQL database of sales data or a 100-page PDF document. These would far exceed this input limit, making it impossible to feed the entire data set into the AI model’s prompt.
Azure Cognitive Search + ChatGPT
So, how can we leverage the power of a chatbot to interact with our data in a meaningful way? The solution lies in a design pattern called Retrieval-Augmented Generation (RAG). This new approach uses a combination of Azure Cognitive Search, and GPT-3.5 to overcome the aforementioned limitations and “talk to” your company's data in natural language. Let’s explore how this solution works.
Azure Cognitive Search is a service that sifts through your company’s data both on prem and in the cloud, to return contextually relevant information. This data can include documents like PDFs, databases, or even images. If an employee asks the chatbot, “What were our best-selling products last month?” the question is directed to Cognitive Search to find the relevant information in the vast sea of data. Then, the relevant data is fed back into GPT-3.5 with the original question. The net effect is a ChatGPT-like experience with your organization's private data.
Let’s Chat!
At FreshBI, we’ve lowered the barriers, risk and cost of developing world-class Power BI dashboards so that you can unlock the value in your data.
Contact us through the scheduling app to start a conversation about how our data visualization consultants can design your best Power BI dashboards today.
Our Latest Blogs
About FreshBI
Operating throughout USA and Canada, FreshBI is a boutique Microsoft Partner using Power BI to unlock the value trapped in your data so that your business can succeed.