We’ve all seen those futuristic computers in science fiction movies—machines capable of understanding human speech, executing complex tasks, and instantly responding to user requests. Virtual assistants that can control an entire technological ecosystem without ever needing a keyboard or mouse. What once seemed out of reach is now closer to reality than ever, thanks to advancements in artificial intelligence (AI).
This article offers a feedback perspective on one of the key components of this rapidly evolving technology: Retrieval-Augmented Generation (RAG). At Wemanity, our developers carried out a Proof of Concept (PoC) to explore the possibilities offered by RAG in managing support interventions for a development team. Here, we will discuss the challenges, solutions, and results of this experimentation.
1. Integrating Enterprise Data: Towards Intelligent Utilization of Generative AI
The integration of AI into work environments is no longer a novelty. A prime example is Microsoft Copilot, embedded in development environments, which has revolutionized repetitive code creation, particularly for tasks such as generating unit tests or quickly bootstrapping projects. By learning from public code repositories and documentation, AI tools can offer automated completions that significantly speed up these tasks. However, when it comes to developing more complex code, developers have observed that the necessary code review to avoid errors can greatly reduce the tool’s initial efficiency.
At the same time, tools like ChatGPT have taken a central role in daily workflows. ChatGPT stands out by generating tailored text, making it useful for technical writing (reports, release notes), as well as for translation, information retrieval, and learning new frameworks. Our developers at Wemanity use it alongside Copilot to assist in analyzing complex code, creating automation scripts, and suggesting code optimizations.
Our advice: Enrich AI with specific data.
Although these tools are already powerful, the question of their customization to meet a company’s specific needs remains crucial. Standard AI models do not incorporate internal data, which is often essential for decision-making or managing complex projects. This is where Retrieval-Augmented Generation (RAG) comes into play. This AI framework enhances results by integrating a company’s internal data, enabling the generation of relevant and context-aware responses.
2. Development of the PoC for Generative AI: Technical Challenges and Innovative Solutions
To narrow the scope of the RAG experimentation, the PoC was focused on developing a micro-program to assist developers in reporting daily support tasks. The goal was to test the feasibility of a system capable of recording voice reports and structuring them in a way that makes them easily retrievable through a secondary voice command. A basic example of such retrieval would be a query like, “What incidents occurred this week?”, allowing the system to fetch relevant intervention reports from the specified period.
To carry out this project, the developers designed an application based on OpenAI GPT, with the sole functionality of microphone activation. The user then summarizes, in natural language, the details of the completed support activity, including all relevant information.
The program then converts the voice input into text using OpenAI’s API.
Once generated, the text is restructured into two categories: semantic information (relevant keywords) and technical information (such as dates or user identifiers). The semantic information is stored in a vector format, enabling searches based on the meaning of words, while the technical information is recorded in a traditional database. When a user reactivates the microphone to retrieve information, the same method is used to structure the request’s content. The semantic and technical data are then leveraged to search through existing activity reports and isolate the most relevant ones. These selected reports are then analyzed by the AI to generate a relevant response to the initial query, enriched with all the necessary information.
Excerpt from the prompt used in this context for extracting information from the user-generated transcript:
“You are an AI designed to extract and structure information from a transcript. Generate a concise, accurate JSON object with information from the transcript. Output only the JSON object, with no additional text.”
This prompt is designed to organise the results of the voice-to-text conversion process into a structured format (JSON), enabling efficient integration into the storage system.
3. Results and Challenges Encountered in the Use of Generative AI
The results of this PoC were promising. By capturing voice exchanges over a given period, we were able to execute queries such as “Summarize the database-related events that occurred between Monday and Wednesday, and list the different participants,” and obtain the expected results. However, several challenges emerged, particularly in ensuring predictive consistency in responses.
The sequencing of actions within the prompt proved to be crucial, while leveraging advanced options of the OpenAI API, such as the system and user roles, helped refine the instructions sent to the AI. Additionally, technical adjustments were necessary, including implementing code to properly parse and structure the generated responses, ensuring the smooth operation of the entire program.
Interestingly, the majority of the time was spent designing and fine-tuning the instructions for the AI engine, highlighting that the required expertise differs from those typically needed in modern software development.
4. The Future of Generative AI in Business: New Horizons
Les possibilités offertes par le RAG
This experimentation has highlighted the potential of RAG systems while emphasizing that advanced use cases for AI in business are still to be defined and explored. One possible approach could be the development of multiple RAG systems with limited scopes, communicating within a framework similar to a microservices architecture. The possibilities for configuration are numerous, and they must be carefully selected to align with evolving business needs and vision, especially during the adoption phase of a new technology.
At the same time, data security remains a major concern. Granting AI access to sensitive information requires strict management of access rights, permissions, and data protection, adding another layer of complexity.
Additionally, the cost of AI queries is a significant challenge that needs to be addressed, although it was intentionally excluded from the scope of this PoC.
Major challenges to overcome.
The future of artificial intelligence appears to be moving toward increasingly advanced personalization, where systems like RAG naturally integrate a company’s internal data. On a larger scale, it is easy to envision a near future where AI personal assistants, developed by industry giants, seamlessly interact with personal data on the devices they come with, combining information from calendars, emails, and other data sources.
These voice assistants will not only provide contextually relevant responses but also suggest activities based on user habits or execute specific actions within compatible applications. This evolution refines the way we interact with software, bridging the gap between science fiction and reality.
The insights gained from this PoC demonstrate that despite the impressive advancements in AI, technical and organizational challenges remain. However, it is clear that the integration of AI, and more specifically RAG, into daily business processes will continue to grow. Whether it is for optimizing knowledge management or automating complex tasks, AI is emerging as an indispensable tool. As these technologies evolve, they open up exciting new possibilities for the business world.
Generative AI, like ChatGPT, produces automated content based on existing data. Retrieval-Augmented Generation (RAG) enhances these capabilities by integrating specific internal company data, enabling more precise and context-aware responses for complex tasks.
The PoC demonstrated the feasibility of a structured voice transcription system, enabling the management and retrieval of technical reports through voice queries. It successfully responded to requests such as “What incidents occurred this week?” with relevant and actionable results.
The challenges include internal data security, query costs, and prompt tuning to ensure response consistency. Additionally, adoption requires extensive customization to meet the specific needs of businesses.
