What’s The Best Way To Connect Your Business Data To AI?

In Brief

Generative AI is transforming business intelligence by enabling secure, data-driven decision-making at scale, using tools like RAG, agentic AI, and integrated BI platforms to deliver actionable insights directly to users while protecting sensitive information.

Generative AI is rewriting the playbook for data-driven business strategy. Laborious processes are becoming automated and conversational, greasing the wheels for a new era of “decision intelligence,” characterized by the simple and precise surfacing of powerful insights exactly when and where they’re needed. It’s a world where AI instantly surfaces the trends that executive leaders need to make decisions quickly and with confidence

Over the last two years, we’ve seen massive leaps forward in AI’s business intelligence capabilities, but there’s a caveat. Before organizations can embrace generative business intelligence, they need to connect AI models to their highly-sensitive business data in a way that won’t leave it exposed

Vectorization, RAG, MCP and Agent Skills are among the formats and protocols that help to bridge the gap, but in this emerging space, no single solution has emerged as the industry standard. Of course, uploading confidential financial reports and personally identifiable information to a public-facing AI platform like ChatGPT is about as secure as posting it directly to Instagram

The moment someone feeds a spreadsheet to these services, there’s no telling if or when it might be leaked publicly, explains Cheryl Jones, an AI specialist at NetCom Learning. “One of the foremost ChatGPT security risks is the potential for inadvertent data leakage,” she writes in a blog post. “Employees might input confidential company information, customer data, or proprietary algorithms into ChatGPT, which could then be used in the model’s training data or exposed in future outputs to other users.”

From RAG to Rich BI Insights

Rather than asking ChatGPT directly, many organizations are investing in creating customized chatbots powered by proprietary LLMs connected to corporate databases. One way to do this is to use a technique known as “retrieval augmented generation” or RAG, which dynamically beefs up the knowledgeof LLMs by retrieving and incorporating external data into AI responses, improving their accuracy and relevance. It’s a way to “fine tune” an AI model without actually changing its algorithms or training.

RAG systems gather data from external sources and break it down into small, manageable chunks, drawing from numerical embeddings stored in a vector database, making them searchable for LLMs. This allows the LLM to surface data chunks that are relevant to the user’s query, before adding them to the original prompt so it can generate a response that’s informed by the connected data

“The foundation of any successful RAG system implementation is a modular architecture that connects raw data to a language model through intelligent retrieval,” explains Helen Zhuravel, director of product solutions at Binariks. “This structure allows teams to keep responses accurate, current, and grounded in internal knowledge, without retraining the model on every update.”

But RAG is not immune to the security issues associated with feeding data directly to AI chatbots, and it’s not a complete solution. RAG alone doesn’t enable LLMs to deliver conventional business intelligence, as the models are still designed to spit out their insights in a conversational way. RAG has none of the traditional building blocks of BI platforms. In order to generate thorough, interactive reports and dashboards, organizations will also need to integrate comprehensive business logic, a data visualization engine and data management tools with the LLM

Ready Made GenBI in a Box

Fortunately, organizations also have the option of purchasing ready-made generative BI systems such as Amazon Q in QuickSight, Sisense and Pyramid Analytics, which look and feel more like traditional BI platforms. The difference is they’re natively integrated with LLMs to enhance accessibility

With its plug-and-play architecture, Pyramid Analytics can connect third-party LLMs directly to data sources such as Databricks, Snowflake and SAP. This eliminates the need to build additional data pipelines or format the data in any special way. To protect sensitive information, Pyramid avoids sending any raw data to the LLM at all

In a blog post, Pyramid CTO Avi Perez explains that user queries are separated from the underlying data, ensuring that nothing leaves the customer’s controlled environment. “The platform only passes the plain-language request and the context needed for the language model to generate the recipe needed to answer your question,” he notes

For instance, if someone asks a question about sales and costs across different regions, Pyramid will only pass the query and limited information to the LLM, such as the metadata, schemas and semantic models required for context. “The actual data itself isn’t sent,” Perez says. “The LLM will use its interpretive capabilities to pass us back an appropriate recipe response which the Pyramid engine will then use to script, query, analyze and build content.”

Other Generative BI platforms handle the AI-database connection differently. Amazon Q in QuickSight addresses security questions by keeping everything siloed within AWS environments. In addition, Amazon promises to avoid using customer prompts and queries to train the underlying models that power Amazon Q, so as to prevent data leakage that way

Generative BI platforms make business intelligence accessible and easy to navigate. Because they offer conversational interfaces, non-technical users can engage with them using natural language prompts to dig up the answers they need. They can also use AI to automatically build dashboards and visualizations that can assist users who need to explore their data further

Users can even generate entire reports and contextual summaries, transforming static data into explainable stories, making it easier to understand trends and anomalies.

Actionable Insights with Agentic BI

In order to try and make business intelligence more actionable, some organizations have opted to apply RAG pipelines with foundational “agentic AI” technologies such as Agent Skills and the Model Context Protocol (MCP). The goal is to transform BI from a passive reporting tool into an autonomous system that understands key insights and can even execute tasks based on what they discover

Agent Skills refers to a library of modular capabilities developed by Anthropic that enable AI agents to perform specific actions, such as creating PDF files, calling a specific API or performing complex statistical calculations. These skills can be activated by agents whenever needed, allowing them to perform work on behalf of humans

Meanwhile, MCP is an open, universal standard that connects LLMs and external data sources and software tools. It enables AI agents to access live systems and tools in a secure and structured way, without needing to build custom connectors

These technologies have synergies that fit the scope of business intelligence, combining to create a new kind of agentic BI workflow. If a user asks a question such as “Why are sales down in the South?”, the agent will use MCP to pull in the specific context required to answer that question, such as the user’s role and access permissions, previous reports they’ve accessed and live data from the company’s CRM platform

Then, the agent will use RAG to retrieve relevant data, such as regional marketing plans, meeting transcripts and so on, to identify reasons for the sales dip. After finding the answer, the agent will employ Agent Skills to take actions, such as generating a summary report, notifying the responsible sales team and updating the budget forecast in the ERP

Cisco CMO Aruna Ravichandran is extremely bullish about Agentic BI and its potential to make “connected intelligence” pervasive throughout the workplace. “In this new era, collaboration happens without friction,” he predicts. “Digital workers anticipate needs, coordinate tasks in the background and resolve issues before they surface.”

Despite the optimism, RAG, MCP and Agent Skills remain in the experimental phase, and many are skeptical about their long-term adoption. There’s no standard framework in place for building agentic BI workflows, and so, for now at least, they will likely remain exclusive to larger organizations with the resources and talent to dedicate to such projects

Everyone Gets AI Enhanced Decision Making

LLM data access is, in a sense, a last-mile obstacle on the way of true decision intelligence, where powerful insights can be surfaced by anyone the moment they’re needed. Once it’s cracked, decision-making will no longer be confined to analyst teams or the executive suite, but instead become embedded in the fabric of daily business operations

More and more employees are getting involved in strategic problem solving, which has profound implications. Organizations that successfully integrate their own data with AI-driven analytics are essentially transforming corporate information from a siloed asset into the language of decisive action that every employee speaks.