 |
International Journal of Computer & Organization Trends (IJCOT) |  |
| © 2025 by IJCOT Journal | ||
| Volume - 15 Issue - 2 | ||
| Year of Publication : 2025 | ||
| Authors : Mehul K Bhuva | ||
| DOI : 10.14445/22492593/IJCOT-V15I2P302 |
How to Cite?
"Real World Implementation of a RAG-based Chat App Using Microsoft AI Foundry: A Practical Approach to Building Enterprise-Level Conversational AI Solutions" International Journal of Computer and Organization Trends vol. 15, no. 2, pp. 15-22, 2025. Crossref, https://doi.org/10.14445/22492593/IJCOT-V15I2P302
Abstract
This paper presents a detailed examination of implementing a Retrieval Augmented Generation (RAG) based chat application using Microsoft AI Foundry. The research addresses the critical gap between theoretical RAG architectures and practical enterprise implementations that effectively navigate privacy concerns, organizational knowledge integration, and scalability challenges. Unlike previous implementations focusing primarily on academic evaluations or isolated technical components, our approach provides a comprehensive framework for developing production-grade conversational systems that seamlessly blend proprietary knowledge with large language model capabilities. The study explores the architectural components, technical challenges, and optimization techniques involved in building an enterprise-ready RAG solution, introducing novel methods for adaptive document chunking, hybrid retrieval mechanisms, and context-sensitive prompt engineering. Performance metrics reveal significant improvements in response accuracy (87% compared to 63% in baseline models), contextual relevance, and user satisfaction compared to both traditional chatbot implementations and conventional RAG approaches. The paper further contributes valuable insights into real-world implementation considerations, including enterprise system integration, knowledge management practices, and scalability planning, which have been largely overlooked in the existing literature. These findings offer crucial guidance for organizations seeking to bridge the gap between theoretical RAG capabilities and practical business applications.
Keywords
Retrieval augmented generation, Rag, Microsoft AI foundry, Conversational AI, Knowledge retrieval, Enterprise chatbots, Vector databases, Semantic search, Natural language processing, Large language models.
References
[1] Eleni Adamopoulou, and Lefteris Moussiades, “Chatbots: History, Technology, and Applications,” Machine Learning with Applications, vol. 2, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[2] Hind Benbya et al., “Complexity and Information Systems Research in the Emerging Digital World,” MIS Quarterly, vol. 44, no. 1, pp. 1-17, 2020.
[Google Scholar] [Publisher Link]
[3] Rishi Bommasani et al., “On the Opportunities and Risks of Foundation Models,” arXiv preprint, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Tom Brown et al., “Language Models are Few-shot Learners,” Advances in Neural Information Processing Systems, vol. 33, 2020.
[Google Scholar] [Publisher Link]
[5] Thomas H. Davenport, and Rajeev Ronanki, “Artificial Intelligence for the Real World,” Harvard Business Review, vol. 96, no. 1, pp. 108-116, 2018.
[Google Scholar] [Publisher Link]
[6] Jan Deriu et al., “Survey on Evaluation Methods for Dialogue Systems,” Artificial Intelligence Review, vol. 54, pp. 755-810, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[7] Kelvin Guu et al., “REALM: Retrieval-augmented Language Model Pre-training,” International Conference on Machine Learning, pp. 3929-3938, 2020.
[Google Scholar] [Publisher Link]
[8] Urvashi Khandelwal et al., “Generalization through Memorization: Nearest Neighbor Language Models,” arXiv Preprint, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Patrick Lewis et al., “Retrieval-augmented Generation for Knowledge-intensive NLP Tasks,” Advances in Neural Information Processing Systems, 2020.
[Google Scholar] [Publisher Link]
[10] Michael McTear, Zoraida Callejas, and David Griol, The Conversational Interface, Springer International Publishing, 2016.
[Google Scholar] [Publisher Link]
[11] Gregoire Mialon et al., “Augmented Language Models: A Survey,” arXiv preprint, 2023. (Preprint)
[CrossRef] [Google Scholar] [Publisher Link]
[12] Yingqi Qu et al., “RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-domain Question Answering,” arXiv preprint, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[13] Li Zhou et al., “The Design and Implementation of XiaoIce, An Empathetic Social Chatbot,” Computational Linguistics, vol. 46, no. 1, pp. 53-93, 2020.
[CrossRef] [Google Scholar] [Publisher Link]