.NET 8 and Azure OpenAI

• .NET 8: .NET is a free, cross-platform, development framework open-sourced by Microsoft. It was released in November, 2023. It’s used to build web, mobile, desktop, games, IoT and other applications.
  
  
• Azure OpenAI: It offers REST API access to OpenAI's robust language models encompassing Ada, Babbage, Curie, GPT-3, GPT-3.5, GPT-4, DALL-E, Codex, and Embeddings model series.
  
  
• Awarded experts: Cazton team comprises of experts that have been awarded the Microsoft Regional Director and Most Valuable Professional awards. Our experts happen to be Azure insiders, ASP.NET advisors, Azure Cosmos DB insiders and Docker insiders.
  
  
• Top clients: At Cazton, we help Fortune 500, large, mid-size and startup companies with web and app development, deployment, consulting, recruiting services and hands-on training services. Our clients include Microsoft, Google, Broadcom, Thomson Reuters, Bank of America, Macquarie, Dell and more.

Microsoft .NET Conf presentation by Chander Dhall, CEO of Cazton.

Build a Chatbot with Azure OpenAI and .NET 8

In this blog post, I will showcase how to build a simple yet powerful chatbot using Azure OpenAI and .NET 8. The chatbot will be able to answer questions based on a provided PDF file without any explicit training. I recently gave a presentation on this topic at the Microsoft .NET Conf 2023. I will be using the key concepts from that presentation and the code I demonstrated to structure this blog post.

Introduction

Chatbots have become ubiquitous in recent years. We see them on websites, in messaging apps, and interacting with us through voice assistants. Under the hood, most chatbots today use some form of artificial intelligence (AI) to understand natural language queries and respond appropriately. In my session, I built a chatbot using Azure OpenAI Service and .NET 8. Azure OpenAI gives access to OpenAI models like GPT-3 and GPT-4 right from the Azure cloud. These models can understand complex language tasks without any training data. We will use this capability to build a chatbot that can read and comprehend a PDF file. Users will be able to ask questions based on the content of the file, and the chatbot will respond with relevant answers.

Prerequisites

To follow along, you will need:

• Access to the Azure portal to create an Azure OpenAI resource.
• .NET 8 SDK installed
• A code editor like Visual Studio Code
• Basic familiarity with C# and ASP.NET Core

I won't cover all the nitty-gritty details of setting up Azure OpenAI in this post. But I'll call out the key steps and parameters we need to integrate the service in our app.

Creating an Azure OpenAI Resource

Azure OpenAI is a managed offering from Microsoft that provides easy access to OpenAI models. Here are the high-level steps to create an Azure OpenAI resource:

• Go to the Azure portal and create an Azure OpenAI resource. You may need admin approval if you don't have access.
• Once the resource is provisioned, go to "Keys and Endpoints" to get the endpoint and keys.
• Under "Deployment Management", create a deployment to select one of the available models like GPT-3 or GPT-4.
• Make note of the deployment name - we will use this later.

The key things we need from above are:

• Endpoint URL
• API key
• Deployment name of the model

These will allow us to authenticate and send requests to the Azure OpenAI service.

Comparing Azure OpenAI and OpenAI

It's easy to get confused between Azure OpenAI and OpenAI. Here's a quick comparison:

• OpenAI is the company that has built models like GPT-3, Codex, DALL-E etc.
• Azure OpenAI is Microsoft's cloud offering that gives access to OpenAI models with added security, compliance and enterprise capabilities.

So, Azure OpenAI runs OpenAI models in the Azure cloud while taking care of scalability, reliability and other platform concerns. This allows developers to focus on just using the models in their applications.

Project Setup

With the theory out of the way, let's jump into the code! I used Visual Studio Code on macOS for this demo app, but you can follow along on Windows too.

Creating the Project:

Let's start by scaffolding an MVC application in .NET 8:

$ dotnet new mvc -o NetConf 
$ cd NetConf  
$ code . 

This creates a basic "NetConf" MVC app and opens it in VS Code. Some key files and what we'll use them for:

• .env: For storing Azure OpenAI credentials
• packages.config: Required NuGet packages
• script.sh: Script to restore the packages
• Program.cs: Initialize Azure OpenAI client
• ChatController.cs: Main bot logic
• Views/Chat: Chat UI

I like to maintain dependencies in packages.config and use a script to restore them all at once. This helps avoid version conflicts.

Setup Azure OpenAI Credentials:

The best practice is to store confidential credentials like API keys outside your app code. .NET 8 has great support for managing secrets through app settings and environment variables.

Let's create a .env file with our Azure OpenAI credentials:

AZURE_OPENAI_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxx
AZURE_OPENAI_ENDPOINT=https://xxxxxxx.openai.azure.com
AZURE_OPENAI_MODEL=gpt4-v2

This file won't get checked-into source control, so our keys are safe!

In Program.cs, we load this file and read the values:

Env.Load(); 
 
string endpoint = GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT");   
string key = GetEnvironmentVariable("AZURE_OPENAI_KEY"); 
string model = GetEnvironmentVariable("AZURE_OPENAI_MODEL");

Using Environment Variables is a standard way to manage secrets across various platforms.

Creating the Chatbot

The main logic of our chatbot will reside in the ChatController. Let's scaffold a basic controller:

public class ChatController : Controller 
{ 
    public IActionResult Index() 
    { 
        return View();  
    } 
}

We'll add a POST method called GetResponse that accepts the user's message and returns the bot's response:

[HttpPost] 
public async Task<ActionResult> GetResponse(string userMessage)   
{
}

Inside this method, we will:

• Initialize an OpenAI Client with our credentials.
• Build the payload for the chat completion API.
• Call the chat completion and return the bot's response.

Here is how we initialize the client:

OpenAIClient client = new OpenAIClient(new Uri(endpoint), new AzureKeyCredential(key));

We pass the endpoint and API key to authenticate. Next, we need to construct the chat prompt with the user message:

var chatOptions = new ChatCompletionsOptions() 
{ 
    Messages = new List<ChatMessage>() 
    { 
        new ChatMessage(ChatRole.User, "What is Azure OpenAI?"),
    }   
}

The ChatCompletionsOptions contains the chat history and parameters to configure the request. We are starting with just the user's message.

Finally, we can call the chat completion API:

Response<ChatCompletion> response = await client.CreateChatCompletionAsync(model, chatOptions); 
string reply = response.Value.Choices[0].Message.Content;

We pass the model name and chat options. The response contains possible replies from which we use the first one. And that's it! We return the bot's reply back to the browser to display.

Here is the full method: 

public async Task<ActionResult> GetResponse(string userMessage) 
{ 
    OpenAIClient client = new OpenAIClient(new Uri(endpoint), new AzureKeyCredential(key)); 

    var chatOptions = new ChatCompletionsOptions() 
    { 
         Messages = new List<ChatMessage>() 
         { 
             new ChatMessage(ChatRole.User, userMessage), 
         } 
    };   
 
    Response<ChatCompletion> response = await client.CreateChatCompletionAsync(model, chatOptions); 
    string reply = response.Value.Choices[0].Message.Content; 
 
    return Json(new { Response = reply }); 
}

Building the Chat UI

For the UI, we need:

• A <div> to display the chat messages.
• A textbox for the user to type their message.
• A Send button to submit the message.

Here is the code for Views/Chat/Index.cshtml:

<div id="chatBox"></div> 
<input type="text" id="userMessage" /> 
<button onclick="sendMessage()">Send</button> 
 
<script> 
  function sendMessage() {  
    // Send message to server 
    // Display responses 
  } 
</script>

We still need the JavaScript to actually call our API and display the responses.

Here is what the sendMessage function looks like:

function sendMessage(){ 
  var userMessage = $("#userMessage").val();   
 
  $.post("/Chat/GetResponse", {userMessage: userMessage}, function(data) {
    $("#chatBox").append("<p>User: " + userMessage + "</p>"); 
    $("#chatBox").append("<p>Bot: " + data.Response + "</p>"); 
  }); 
}

We take the user's message and make an AJAX POST call to our API. The data comes back containing the Response which we display. And that's it! Our simple chatbot is ready!

Testing the Chatbot

Let's take our bot for a spin! Run the app and go to the /chat page.

Try asking it a question like:

What is the capital of France?

You should see a friendly response with the answer:

The capital of France is Paris.

Let me try another one:

What is Azure OpenAI?

And this time it responds with:

Unfortunately I do not have enough context to definitively explain what Azure OpenAI is.

This makes sense because GPT models have a limited knowledge capacity tuned to certain time periods. Without additional context, it does not know about Microsoft Azure OpenAI service specifically.

This leads us to the next step of enhancing our chatbot...

Ingesting Custom Data

Our chatbot works reasonably well for casual chat. But for enterprise use-cases, we need to customize it with our own business data. Let's teach our bot about Azure OpenAI by providing it a PDF file on the topic. We want the bot to answer questions based on the content of this file, instead of its own limited knowledge.

I have added an Azure.pdf file that gives an overview of the Azure OpenAI service. The techniques we implement will work for PDFs, DOCX files or even raw text snippets.

At a high-level, we will:

• Extract text content from the PDF.
• Pass this text to the GPT model along with the user's question.
• GPT will analyze both and return an answer based on the PDF context.

Here is the code to extract text from a PDF using iTextSharp library:

private string GetTextFromPdf(string filePath) 
{ 
    PdfDocument pdf = new PdfDocument(new PdfReader(filePath)); 
    StringBuilder text = new StringBuilder();    
 
    for(int i=1; i<=pdf.GetNumberOfPages(); i++)
    { 
        PdfPage page = pdf.GetPage(i);  
 
        ITextExtractionStrategy strategy = new SimpleTextExtractionStrategy(); 
        string pageText = PdfTextExtractor.GetTextFromPage(page, strategy); 
 
        text.Append(pageText); 
    }    
 
    pdf.Close(); 
     
    return text.ToString();                 
}

Here:

• We load the PDF using iTextSharp.
• Loop through pages extracting text from each one.
• Append the text from all pages into a single string.
• Return the combined string.

To integrate this with our chatbot, we add a new method called GetResponseFromPdf.

public async Task<ActionResult> GetResponseFromPdf(string userMessage)   
{ 
    string pdfText = GetTextFromPdf("Data/Azure.pdf"); 
     
    // Create completion as before  
    // Pass PDF text as additional context 
}

We extract text from the PDF and pass it in the chat prompt:

var chatOptions = new ChatCompletionOptions() 
{ 
    Messages = new List<ChatMessage>() 
    { 
        new ChatMessage(ChatRole.System, "The following context is from an Azure OpenAI PDF."), 
        new ChatMessage(ChatRole.System, pdfText), 
        new ChatMessage(ChatRole.User, userMessage), 
    } 
}

This prepends the PDF text to the user's question to provide relevant context. The rest of the method stays the same. We call the completion API and return the response. With these changes, our bot now has the context to answer queries about Azure OpenAI specifically based on the ingested PDF content.

Let's re-run the app and try some questions:

What are the capabilities of Azure OpenAI?

It neatly summarizes the capabilities like we saw in the PDF:

Azure OpenAI Service provides access to powerful AI models like GPT-3 and Codex for language, code generation and more. It combines advanced capabilities of OpenAI with enterprise grade security, compliance and regional deployment on Microsoft Azure.

Next question:

How does Azure OpenAI compare to OpenAI API?

And again it responds based on the PDF context:

Azure OpenAI allows using OpenAI models like API access with added benefits of security, scalability and compliance features from Microsoft Azure cloud platform.

With just 4 lines of highlighted text from a PDF, we are able to customize and significantly enhance our chatbot's capabilities tailored to our business domain.

The complete code for this blog is available on GitHub.

Limitations and Enhancements

While we have a solid starting point for our chatbot, there is still room for improvement using more advanced techniques:

• Token Limit: OpenAI models have a token (word) limit per request. For GPT-3 it's 2048 tokens which falls short for large documents. This means the bot can only process a small chunk of text at a time. We can use retrieval and summarization techniques to compress content before sending to the model. There are also streaming and chaining methods to handle long conversations within the limits.
  
  
• Relevance: With more text, there is higher chance of irrelevant or contradictory responses. We can train a model like T5 or Codex for our domain to give more focused answers. Techniques like RAG (Retrieval Augmented Generation) also help the model ground its responses in the provided context for consistency.
  
  
• Integration: For real applications, the bot would need integration into communication channels like webchat, SMS, email etc. We can build these integrations using Azure Communication Services and other platform tools. Tracking usage metrics and model optimizations are also important long-term considerations.

Hopefully this post gave you a good overview of creating and customizing chatbots using the latest AI capabilities offered by Azure OpenAI service!