Enterprise data can make a generative AI application useful for organization-specific questions. It can also be subject to access restrictions, privacy requirements, and legal obligations. Before using it in a chatbot, identify the business use case, the data needed, and who is authorized to use the application. section 1.7 section 1.9
For a chatbot that answers questions using internal information:
- Select relevant, high-quality data and manage its privacy and legal requirements before ingestion and indexing. Data quality affects answer quality. section 1.9
- Use retrieval-augmented generation (RAG) to bring relevant data into an LLM-based answer. Custom-training an LLM on enterprise data is another approach, but is more involved and unavailable to most users. section 8.0
- Plan retrieval, chunking, embeddings, and a vector database around the data’s structure. Production use requires attention to security, scale, performance, and cost. section 8.9
Internal enterprise chatbot
An enterprise chatbot ingests and indexes relevant internal data so authorized users can ask proprietary questions. Its deployment also calls for governance, responsible-AI guidance, and continued monitoring and testing; connecting a model to data is not the whole implementation. section 1.9
RAG can make answers more current and specific to an organization than simply increasing an LLM’s context window. That benefit does not remove the need to manage the underlying data or engineer its connection to source systems carefully. section 8.8 section 8.9
