Azure AI Building RAG Application Solution: Using Azure Open AI to generate embedding on PDF documents and executing vector quires for RAG Applicaiton
In the previous article, we have seen how to create Azure AI Search Service, Azure Open AI, and Azure Blob Storage using Azure Portal. Furthermore, we have uploaded PDF documents in the Azure Blob storage. We have seen how to create Azure AI Search Data Source, Index, and Indexer using Code. In this article, we will see the implementation of the RAG solution. As shown in Figure 1, the Step 1 is already completed in.
Figure 1: RAG Application
The project creation is already explained in previous article. In this article we need to modify the AzureAISearchServiceManager class. Since, we will be using the PDF documents for embedding, we need to make sure that the PDF contents from PDF documents are split and they are embedded.
Step 1: Modify the AzureAISearchServiceManager class by adding method as shown in List 1.
public static List<string> SplitTextContentsChunks(string text, int maxCharsPerChunk = 7000) { var chunks = new List<string>(); var sentenceBlock = Regex.Split(text, @"(?<=[\.!\?])\s+"); // Split by punctuation followed by space var currentChunk = new StringBuilder(); foreach (var sentence in sentenceBlock) { if (currentChunk.Length + sentence.Length > maxCharsPerChunk) { chunks.Add(currentChunk.ToString()); currentChunk.Clear(); } currentChunk.Append(sentence).Append(" "); } if (currentChunk.Length > 0) { chunks.Add(currentChunk.ToString()); } return chunks; }
Listing 1: The SplitTextContentsChunk
As shown in Listing 1, the SplitTextContentsChunks() method is used to chunk the text based on the length.
Step 2: In the project, we have the Models folder. In this folder, add a new class file named RequestResponses.cs. In this class file, we will add code for the Prompt class and ResponsePrompt class as shown in Listing 2.
namespace Core_RAG_Service.Models { public class Prompt { public string? Query { get; set; } } public class ResponsePrompt { public string? Query { get; set; } public string? Answer { get; set; } public List<string>? Documents { get; set; } } }
Listing 2: Prompt and ResponsePrompt classes
Step 2: In the project add the PdfPig package. In the AzureAISearchServiceManager class, add the GenerateEmbeddingsForBlobDocuments() method as shown in Listing 3. In this method following operations:
- Create an instance of the BlobServiceClient class using the Azure Storage Account Blob connection string that is stored in appsettings.json file.
- Creating an instance of BlobContainerClient class using the GetBlobContainerClient() method of the BlobServiceClient class. This method is passed with the Blob Container name that is stored in appsettings.json file.
- The instance of the AzureOpenAIClient class is created using its Endpoint and Key. This instance is used invoke the GetEmbeddingClient() method of the AzureOpenAIClient. This method is passed with the text embedding deployment name that we have created in Azure AI Foundry as shown in previous article.
- Furthermore, the SearchClient instance is created using Azure AI Search Service Endpoint, Index name, and AzureKeyCredential object.
- The method now further iterates with every blob from the Blob Container. In this loop the PDF document from the Blob Container is loaded using Open() method of the PdfDocument class of the PdfPig package. We have the GetPages() method of the PdfDocument class. This method allows to extract contents from the PDF file. These contents are further passed to the SplitTextContentsChunks() method.
- Within this loop, we have another loop. This loop iterates with the text from the chunk and generated embedding by passing the chunk to the GenerateEmbeddingAsync() method of the EmbeddingClient class of which instance is created using GetEmbeddingClient() method of the AzureOpenAIClient class.
- In next step, this embedding is passed to an instance of the IndexedAISearchDocumentModel class to its text_vector property along with the content, metadata_storage_name, metadata_storage_path, etc. properties. This instance now represents the vectorized document.
- This vectorized document using IndexedAISearchDocumentModel class is uploaded to the Azure AI Search Service Index using Create() method of the IndexDocumentsBatch class.
- The Index is populated with the vectorized documents.
private async Task GenerateEmbeddingsForBlobDocuments() { try { string textContent = string.Empty; var blobServiceClient = new BlobServiceClient(azureBlobStorageConnectionString); var containerClient = blobServiceClient.GetBlobContainerClient(azureBolbContainerName); ResponsePrompt promptResponse = new ResponsePrompt(); _openAIClient = new AzureOpenAIClient(new Uri(azureOpenAIEndpoint), new AzureKeyCredential(azureOpenAIKey)); var embeddingClient = _openAIClient.GetEmbeddingClient(azureOpenAIEmbeddingDeploymentName); var searchClient = new SearchClient( new Uri(searchEndpoint), indexName, _credential ); await foreach (BlobItem blobItem in containerClient.GetBlobsAsync()) { var blobClient = containerClient.GetBlobClient(blobItem.Name); // Get the stream from Blob Storage var blobStream = await blobClient.OpenReadAsync(); // Extract text using PdfPig using (var pdf = UglyToad.PdfPig.PdfDocument.Open(blobStream)) { foreach (var page in pdf.GetPages()) { extractedText += page.Text + "\n"; } } // Split the extracted text into manageable chunks var textChunks = SplitTextContentsChunks(extractedText, 7000); foreach (var chunk in textChunks) { var embeddingResponse = await embeddingClient.GenerateEmbeddingsAsync( new List<string> { chunk } ); var embedding = embeddingResponse.Value[0].ToFloats().ToArray(); var doc = new IndexedAISearchDocumentModel { chunk_id = Convert.ToBase64String(Encoding.UTF8.GetBytes($"{blobItem.Name}_{chunk.GetHashCode()}")), content = chunk, metadata_storage_name = blobItem.Name, metadata_storage_path = blobClient.Uri.ToString(), text_vector = embedding }; var batch = IndexDocumentsBatch.Create( IndexDocumentsAction.MergeOrUpload(doc) ); await searchClient.IndexDocumentsAsync(batch); } } } catch (Exception ex) { } }
private async void CreateIndex() { ................................... // Let's Generate embeddings for existing blob documents await GenerateEmbeddingsForBlobDocuments(); }
public async Task<ResponsePrompt> SearchInDocument(string userQuery) { ResponsePrompt promptResponse = new ResponsePrompt(); _openAIClient = new AzureOpenAIClient(new Uri(azureOpenAIEndpoint), new AzureKeyCredential(azureOpenAIKey)); var embeddingClient = _openAIClient.GetEmbeddingClient(azureOpenAIEmbeddingDeploymentName); // var length = embedding.Length; var searchClient = new SearchClient( new Uri(searchEndpoint), indexName, _credential ); var response = await embeddingClient.GenerateEmbeddingsAsync( new List<string> { userQuery } ); ReadOnlyMemory<float> embedding = response.Value[0].ToFloats(); var vectorQuery = new VectorizedQuery(embedding.ToArray()) { KNearestNeighborsCount = 10, // 3 Fields = { "text_vector" }, }; var options = new SearchOptions { VectorSearch = new VectorSearchOptions { Queries = { vectorQuery } }, Select = { "chunk_id", "content", "metadata_storage_name", "metadata_storage_path", "text_vector" }, Size = 3 //100 is old value }; var results = await searchClient.SearchAsync<IndexedAISearchDocumentModel>("*", options); var filteredDocuments = new List<IndexedAISearchDocumentModel>(); await foreach (var result in results.Value.GetResultsAsync()) { filteredDocuments.Add(result.Document); } List<string> documentContents = new List<string>(); // And then add the Name of the document foreach (var doc in filteredDocuments) { documentContents.Add($"[{doc.metadata_storage_name}]"); } // Adding the PDF Contents here documentContents.Add(extractedText); var indexedDocuments = string.Join("\n", documentContents); string prompt = $@" You are AI Two-Wheeler Policy Knowledge Generator, which helps user for getting detailed information of two wheeler policy details like damage, policy period, Liability to third parties based on the context provided. You don't have access to external knowledge, so answer based on the current context. context : {indexedDocuments} user: {userQuery}, "; var resp = await kernel.InvokePromptAsync(prompt); promptResponse.Answer = resp.ToString(); promptResponse.Query = userQuery; var matchingPDFs = results.Value.GetResults() .Where(r => r.Score > 0.5) // Optional: filter out weak matches .GroupBy(r => r.Document.metadata_storage_name) .Select(group => new { FileName = group.Key, MatchCount = group.Count(), TopSnippet = group.First().Document.content.Substring(0, 200), StoragePath = group.First().Document.metadata_storage_path }) .ToList(); string references = string.Empty; foreach (var pdf in matchingPDFs) { references += $"{pdf.FileName} \n"; } promptResponse.Documents = new List<string> { references}; return promptResponse; }
- The instance of the AzureOpenAIClient class is created using its Endpoint and Key. This instance is used invoke the GetEmbeddingClient() method of the AzureOpenAIClient. This method is passed with the text embedding deployment name that we have creat cled in Azure AI Foundry as shown in previous article.
- Furthermore, the SearchClient instance is created using Azure AI Search Service Endpoint, Index name, and AzureKeyCredential object.
- The method further generates embedding on the user query (the prompt) passed to it.
- The method further creates a VectorQuery based on the embedding generated of the user query. This query targets to the text_vector filed of the Index schema.
- Further, the SearchOptions is created that configure the VectorSearch using the VectorQuery and then ten configure the result to be retrieved using the Select property of the SearchOptions.
- These options are passed as input parameter to the SearchAsync() method of the SearchClient class.
- The SearchAsync() method returns the matched documents based on the vector query. These documents are further passed to the defined system prompt so that the RAG is implemented.
- Furthermore, the code of the method retrieves the PDF document name form which the result is generated.
using System.Text; using System.Text.RegularExpressions; using Azure; using Azure.AI.OpenAI; using Azure.Search.Documents; using Azure.Search.Documents.Indexes; using Azure.Search.Documents.Indexes.Models; using Azure.Search.Documents.Models; using Azure.Storage.Blobs; using Azure.Storage.Blobs.Models; using Core_RAG_Service.Models; using Microsoft.Extensions.AI; using Microsoft.SemanticKernel; namespace Core_RAG_Service.Services { public class AzureAISearchServiceManager { Kernel kernel; AzureKeyCredential _credential; AzureOpenAIClient _openAIClient; SearchIndexClient _indexClient; IConfiguration configuration; string searchKey, searchEndpoint; IConfiguration _configuration; string indexName=string.Empty; string indexerName = string.Empty; string azureAIDataSourceName = string.Empty; string azureBolbContainerName = string.Empty; string azureBlobStorageConnectionString = string.Empty; string azureOpenAIKey = string.Empty; string azureOpenAIEndpoint = string.Empty; string azureOpenAIChatDeploymentName = string.Empty; string azureOpenAIEmbeddingDeploymentName = string.Empty; string extractedText = string.Empty; public AzureAISearchServiceManager(IConfiguration configuration) { _configuration = configuration ?? throw new ArgumentNullException(nameof(configuration)); searchKey = _configuration["AzureSettings:AzureAISearchKey"] ?? throw new ArgumentNullException("AzureSettings:AzureAISearchKey"); searchEndpoint = _configuration["AzureSettings:AzureAISearchEndPoint"] ?? throw new ArgumentNullException("AzureSettings:AzureAISearchEndPoint"); _credential = new AzureKeyCredential(searchKey); _indexClient = new SearchIndexClient(new Uri(searchEndpoint), _credential); azureOpenAIEndpoint = _configuration["AzureSettings:AzureOpenAIServiceEndpoint"] ?? throw new ArgumentNullException("AzureSettings:AzureOpenAIServiceEndpoint"); azureOpenAIKey = _configuration["AzureSettings:AzureOpenAIServiceApiKey"] ?? throw new ArgumentNullException("AzureSettings:AzureOpenAIServiceApiKey"); azureOpenAIChatDeploymentName = _configuration["AzureSettings:AzureOpenAIServiceChatDeploymentName"] ?? throw new ArgumentNullException("AzureSettings:AzureOpenAIServiceChatDeploymentName"); azureOpenAIEmbeddingDeploymentName = _configuration["AzureSettings:AzureOpenAIServiceEmbeddingDeploymentName"] ?? throw new ArgumentNullException("AzureSettings:AzureOpenAIServiceEmbeddingDeploymentName"); azureAIDataSourceName = _configuration["AzureSettings:AzureAISearchDataSourceName"] ?? throw new ArgumentNullException("AzureSettings:AzureAISearchDataSourceName"); azureBlobStorageConnectionString = _configuration["AzureSettings:AzureBLOBConnectionString"] ?? throw new ArgumentNullException("AzureSettings:AzureBLOBConnectionString"); azureBolbContainerName = _configuration["AzureSettings:AzureBLOBStorageContainerName"] ?? throw new ArgumentNullException("AzureSettings:AzureBLOBStorageContainerName"); indexName = _configuration["AzureSettings:AzureAISearchIndexName"] ?? throw new ArgumentNullException("AzureSettings:AzureAISearchIndexName"); indexerName = _configuration["AzureSettings:AzureAISearchIndexerName"] ?? throw new ArgumentNullException("AzureSettings:AzureAISearchIndexerName"); var builder = Kernel.CreateBuilder(); builder.AddAzureOpenAIChatCompletion( azureOpenAIChatDeploymentName, azureOpenAIEndpoint, azureOpenAIKey ); kernel = builder.Build(); CreateIndex() ; } private async void CreateIndex() { var fields = new List<SearchField> { new SimpleField("chunk_id", SearchFieldDataType.String) { IsKey = true, IsFilterable = true }, new SearchableField("content") { IsSortable = false, IsFilterable = false, IsFacetable = false }, new SimpleField("metadata_storage_name", SearchFieldDataType.String) { IsFilterable = true, IsSortable = true }, new SimpleField("metadata_storage_path", SearchFieldDataType.String) { IsFilterable = true, IsSortable = false }, new VectorSearchField("text_vector", 3072, vectorSearchProfileName: "vector-config") }; var definition = new SearchIndex(indexName) { Fields = fields, VectorSearch = new VectorSearch { Algorithms = { new HnswAlgorithmConfiguration("hnsw-config") }, Profiles = { new VectorSearchProfile("vector-config", "hnsw-config") } } }; _indexClient.CreateOrUpdateIndex(definition); var dataSource = new SearchIndexerDataSourceConnection( name: azureAIDataSourceName, type: SearchIndexerDataSourceType.AzureBlob, connectionString: azureBlobStorageConnectionString, container: new SearchIndexerDataContainer(azureBolbContainerName) ); var indexerClient = new SearchIndexerClient(new Uri(searchEndpoint), _credential); indexerClient.CreateOrUpdateDataSourceConnection(dataSource); var indexer = new SearchIndexer( name: indexerName, dataSourceName: dataSource.Name, targetIndexName: indexName ); indexer.Parameters = new IndexingParameters { BatchSize = 1, MaxFailedItems = -1, MaxFailedItemsPerBatch = -1, }; indexerClient.CreateOrUpdateIndexer(indexer); // RunIndexer(); // Let's Generate embeddings for existing blob documents await GenerateEmbeddingsForBlobDocuments(); } private void RunIndexer() { var indexerClient = new SearchIndexerClient(new Uri(searchEndpoint), _credential); indexerClient.RunIndexer(indexerName); } public static List<string> SplitTextContentsChunks(string text, int maxCharsPerChunk = 7000) { var chunks = new List<string>(); var sentenceBlock = Regex.Split(text, @"(?<=[\.!\?])\s+"); // Split by punctuation followed by space var currentChunk = new StringBuilder(); foreach (var sentence in sentenceBlock) { if (currentChunk.Length + sentence.Length > maxCharsPerChunk) { chunks.Add(currentChunk.ToString()); currentChunk.Clear(); } currentChunk.Append(sentence).Append(" "); } if (currentChunk.Length > 0) { chunks.Add(currentChunk.ToString()); } return chunks; } private async Task GenerateEmbeddingsForBlobDocuments() { try { string textContent = string.Empty; var blobServiceClient = new BlobServiceClient(azureBlobStorageConnectionString); var containerClient = blobServiceClient.GetBlobContainerClient(azureBolbContainerName); ResponsePrompt promptResponse = new ResponsePrompt(); _openAIClient = new AzureOpenAIClient(new Uri(azureOpenAIEndpoint), new AzureKeyCredential(azureOpenAIKey)); var embeddingClient = _openAIClient.GetEmbeddingClient(azureOpenAIEmbeddingDeploymentName); var searchClient = new SearchClient( new Uri(searchEndpoint), indexName, _credential ); await foreach (BlobItem blobItem in containerClient.GetBlobsAsync()) { var blobClient = containerClient.GetBlobClient(blobItem.Name); // Get the stream from Blob Storage var blobStream = await blobClient.OpenReadAsync(); // Extract text using PdfPig using (var pdf = UglyToad.PdfPig.PdfDocument.Open(blobStream)) { foreach (var page in pdf.GetPages()) { extractedText += page.Text + "\n"; } } // Split the extracted text into manageable chunks var textChunks = SplitTextContentsChunks(extractedText, 7000); foreach (var chunk in textChunks) { var embeddingResponse = await embeddingClient.GenerateEmbeddingsAsync( new List<string> { chunk } ); var embedding = embeddingResponse.Value[0].ToFloats().ToArray(); var doc = new IndexedAISearchDocumentModel { chunk_id = Convert.ToBase64String(Encoding.UTF8.GetBytes($"{blobItem.Name}_{chunk.GetHashCode()}")), content = chunk, metadata_storage_name = blobItem.Name, metadata_storage_path = blobClient.Uri.ToString(), text_vector = embedding }; var batch = IndexDocumentsBatch.Create( IndexDocumentsAction.MergeOrUpload(doc) ); await searchClient.IndexDocumentsAsync(batch); } } } catch (Exception ex) { } } public async Task<ResponsePrompt> SearchInDocument(string userQuery) { ResponsePrompt promptResponse = new ResponsePrompt(); _openAIClient = new AzureOpenAIClient(new Uri(azureOpenAIEndpoint), new AzureKeyCredential(azureOpenAIKey)); var embeddingClient = _openAIClient.GetEmbeddingClient(azureOpenAIEmbeddingDeploymentName); // var length = embedding.Length; var searchClient = new SearchClient( new Uri(searchEndpoint), indexName, _credential ); var response = await embeddingClient.GenerateEmbeddingsAsync( new List<string> { userQuery } ); ReadOnlyMemory<float> embedding = response.Value[0].ToFloats(); var vectorQuery = new VectorizedQuery(embedding.ToArray()) { KNearestNeighborsCount = 10, // 3 Fields = { "text_vector" }, }; var options = new SearchOptions { VectorSearch = new VectorSearchOptions { Queries = { vectorQuery } }, Select = { "chunk_id", "content", "metadata_storage_name", "metadata_storage_path", "text_vector" }, Size = 3 //100 is old value }; var results = await searchClient.SearchAsync<IndexedAISearchDocumentModel>("*", options); var filteredDocuments = new List<IndexedAISearchDocumentModel>(); await foreach (var result in results.Value.GetResultsAsync()) { filteredDocuments.Add(result.Document); } List<string> documentContents = new List<string>(); // And then add the Name of the document foreach (var doc in filteredDocuments) { documentContents.Add($"[{doc.metadata_storage_name}]"); } // Adding the PDF Contents here documentContents.Add(extractedText); var indexedDocuments = string.Join("\n", documentContents); string prompt = $@" You are AI Two-Wheeler Policy Knowledge Generator, which helps user for getting detailed information of two wheeler policy details like damage, policy period, Liability to third parties based on the context provided. You don't have access to external knowledge, so answer based on the current context. context : {indexedDocuments} user: {userQuery}, "; var resp = await kernel.InvokePromptAsync(prompt); promptResponse.Answer = resp.ToString(); promptResponse.Query = userQuery; var matchingPDFs = results.Value.GetResults() .Where(r => r.Score > 0.5) // Optional: filter out weak matches .GroupBy(r => r.Document.metadata_storage_name) .Select(group => new { FileName = group.Key, MatchCount = group.Count(), TopSnippet = group.First().Document.content.Substring(0, 200), StoragePath = group.First().Document.metadata_storage_path }) .ToList(); string references = string.Empty; foreach (var pdf in matchingPDFs) { references += $"{pdf.FileName} \n"; } promptResponse.Documents = new List<string> { references}; return promptResponse; } } }
using Core_RAG_Service.Models; using Core_RAG_Service.Services; var builder = WebApplication.CreateBuilder(args); // Add services to the container. builder.Services.AddSingleton<AzureAISearchServiceManager>(); // Learn more about configuring OpenAPI at https://aka.ms/aspnet/openapi builder.Services.AddOpenApi(); builder.Services.AddCors(options => { options.AddPolicy("cors", builder => builder.AllowAnyOrigin() .AllowAnyMethod() .AllowAnyHeader()); }); var app = builder.Build(); // Configure the HTTP request pipeline. if (app.Environment.IsDevelopment()) { app.MapOpenApi(); } app.UseHttpsRedirection(); app.UseCors("cors"); app.MapPost("/api/search", async (AzureAISearchServiceManager searchService, Prompt userPrompt) => { var result = await searchService.SearchInDocument(userPrompt.Query); if (result == null ) { return Results.NotFound("No results found."); } return Results.Ok(result); }); app.Run();
using System.Net.Http.Json; using Blazor_ClientApp.Models; namespace Blazor_ClientApp.Services { public class PolicyClientService { private readonly HttpClient _httpClient; public PolicyClientService(HttpClient httpClient) { _httpClient = httpClient; } public async Task<ResponsePrompt> GetPolicyAsync(Prompt prompt) { var response = await _httpClient.PostAsJsonAsync("https://localhost:7081/api/search", prompt); if (response.IsSuccessStatusCode) { return await response.Content.ReadFromJsonAsync<ResponsePrompt>(); } else { throw new HttpRequestException($"Error fetching policy: {response.ReasonPhrase}"); } } } }
@page "/policy" @using Blazor_ClientApp.Models @using Blazor_ClientApp.Services @using Markdig @inject PolicyClientService PolicyService <h3>RAG Component</h3> <table class="table table-bordered table-dark table-striped"> <tbody> <tr> <td> Please provide your question here: </td> <td> <textarea class="form-control" placeholder="Enter your question" @bind="promptText"></textarea> </td> <td> <button class="btn btn-primary" @onclick="SubmitQuestion">Submit</button> </td> </tr> </tbody> </table> <hr/> <div class="alert alert-info"> <p> <strong>Note:</strong> </p> { @responsePrompt?.Answer } <br/> <strong>Source:</strong> @if (responsePrompt != null) { if (responsePrompt?.Documents != null) { @foreach (var item in responsePrompt?.Documents) { <strong>@item</strong> <br /> } } } <strong>Status:</strong> { @statusMessage } </div> <hr/> @code { private ResponsePrompt responsePrompt = new ResponsePrompt(); private Prompt prompt = new Prompt(); private string promptText = string.Empty; private string statusMessage = string.Empty; private async Task SubmitQuestion() { try { if (string.IsNullOrWhiteSpace(promptText)) { statusMessage = "Please enter a question."; return;Step 7: } prompt.Query = promptText; responsePrompt = await PolicyService.GetPolicyAsync(prompt); statusMessage = "Response received successfully."; } catch (Exception ex) { statusMessage = $"Error: {ex.Message}"; } } }
<div class="nav-item px-3"> <NavLink class="nav-link" href="policy"> <span class="bi bi-list-nested-nav-menu" aria-hidden="true"></span> Policy </NavLink> </div>