Azure OpenAI on Your Data: A Comprehensive Guide

Azure OpenAI on Your Data is a powerful tool that allows you to leverage the capabilities of OpenAI's advanced AI models directly within your own data and applications.

With Azure OpenAI, you can easily integrate OpenAI's AI models into your own data and workflows, unlocking new levels of automation and intelligence.

This integration enables you to tap into the vast capabilities of OpenAI's models, such as text generation, image recognition, and natural language processing.

By doing so, you can create more personalized and intelligent experiences for your users, and drive business growth and innovation.

Data ingestion is a crucial step in getting your data into Azure OpenAI. Azure OpenAI On Your Data uses a specific process to ingest data from Azure blob storage, local files, or URLs into Azure AI Search.

To ingest data, you'll need to create two indexers, two indexes, two data sources, and a custom skill in the Azure AI Search resource. A chunks container is also created in the blob storage.

If you're using a schedule to trigger the ingestion, the process starts from step 7. Azure OpenAI's preprocessing-jobs API implements the Azure AI Search customer skill web API protocol to process the documents in a queue.

For managed identities used in service calls, only system-assigned managed identities are supported. User-assigned managed identities aren't supported.

If you're using an API key, store it securely elsewhere, such as in Azure Key Vault. Don't include the API key directly in your code, and never post it publicly.

Ingestion Architecture is a crucial part of the data ingestion process, and it's essential to understand how it works. Azure OpenAI uses a specific process to ingest data from various sources.

The process starts with creating two indexers, two indexes, two data sources, and a custom skill in the Azure AI Search resource. This is a critical step, as it sets the foundation for the ingestion process.

The chunks container is created in the blob storage, which is where the data will be stored temporarily. This is where the data will be processed before being indexed.

If the schedule triggers the ingestion, the ingestion process starts from step 7. This is where the magic happens, and the data starts being processed.

Azure OpenAI's preprocessing-jobs API implements the Azure AI Search customer skill web API protocol, and processes the documents in a queue. This is a complex process, but it's necessary for accurate data ingestion.

Only system-assigned managed identities are supported for managed identities used in service calls. User-assigned managed identities aren't supported, so be sure to use the correct type.

Here is a summary of the ingestion architecture process:

Data Ingestion is a crucial step in making the most out of your data. You can use Azure OpenAI on your data to automate customer assistance, making it easier to provide help to your customers.

Automating customer assistance can be a game-changer for businesses. By using Azure OpenAI, you can provide 24/7 support to your customers, increasing customer satisfaction and reducing the workload on your support team.

Real-time document search is another use case for Azure OpenAI. This feature allows you to quickly find specific information within your documents, saving you time and increasing productivity.

The possibilities are endless with Azure OpenAI. You can also use it for industry/competitive insights, helping you stay ahead of the competition and make informed business decisions.

Here are some other use cases for Azure OpenAI on your data:

To use Azure OpenAI On Your Data fully, you need to set one or more Azure RBAC roles. Azure Role-based access controls (Azure RBAC) for adding data sources is a crucial step.

You can restrict the documents that can be used in responses for different users with Azure AI Search security filters. This is done by enabling document-level access, which trims the search results based on user Microsoft Entra group membership.

To enable document-level access, you need to register your application and create users and groups, index your documents with their permitted groups, and make sure each sensitive document in the index has a security field value set to the permitted groups of the document.

Here are some recommended models for MongoDB Atlas: gpt-4 (0613)gpt-4 (turbo-2024-04-09)gpt-4o (2024-05-13)gpt-35-turbo (1106)

To configure data access and control, you need to consider document-level access control. This feature is only supported when you select Azure AI Search as your data source.

You can restrict the documents that can be used in responses for different users with Azure AI Search security filters. This allows you to trim search results based on user Microsoft Entra group membership.

To enable document-level access, you can only do so on existing Azure AI Search indexes. For more information on how to secure your data, see Use Azure OpenAI On Your Data securely.

Azure OpenAI On Your Data provides a way to restrict access to sensitive information, giving you more control over who can see what.

If you're using a basic or standard pricing tier, or if it's your first time setting up all your resources securely, you should skip creating a shared private link.

To create a shared private link, you need to connect to your Azure OpenAI resource from your search resource. Select Resource type as Microsoft.CognitiveServices/accounts and Group ID as openai_account.

This process can be a bit tricky, but it's a crucial step to ensure secure data access and control. With a shared private link, step 8 of the data ingestion architecture diagram is changed from bypass trusted service to shared private link.

Document-Level Access Control is a feature that restricts the documents used in responses for different users with Azure AI Search security filters. This feature is supported for Azure AI search only.

To enable document-level access, you need to index your documents with their permitted groups. This means adding a security field to each document with the schema: {"name": "group_ids", "type": "Collection(Edm.String)", "filterable": true}. You can use the default field name "group_ids" or a custom field name like "my_group_ids" and map it in index field mapping.

Each sensitive document in the index must have this security field value set to the permitted groups of the document. You can retrieve and cache users' groups to pass the filter parameter in each API request.

Here's an example of how to pass the filter parameter in an API request:

Note that document-level access is only enabled when the permitted groups field is mapped in Azure OpenAI Studio. If it's not mapped, document-level access is disabled.

The default chunk size is 1,024 tokens, but you might find a different size more effective for your chatbot's performance.

Adjusting the chunk size can be a trial-and-error process, but start by considering the nature of your dataset. A smaller chunk size is generally better for datasets with direct facts and less context.

A small chunk size like 256 produces more granular chunks, which can potentially cost less and reduce noise and distraction.

However, smaller chunks also pose a potential problem: important information might not be among the top retrieved chunks, especially if the number of retrieved documents is set to a low value like 3.

Consider changing the chunk size if you're still not getting the desired results. Here are some scenarios to consider:

Keep in mind that altering the chunk size requires your documents to be re-ingested, so it's useful to first adjust runtime parameters like strictness and the number of retrieved documents.

Federated SharePoint searches allow for a more distributed and secure approach to data access.

Using Azure OpenAI Service on your data with Elastic as a vector database is a powerful combination for this purpose.

Azure OpenAI On Your Data provides intelligent search enabled for your data, with semantic search enabled by default if you have both semantic search and keyword search.

You can customize the field mapping when adding your data source to define the fields that will get mapped when answering questions. To customize field mapping, select Use custom field mapping on the Data Source page when adding your data source.

There are several search types available, including keyword search, semantic search, vector search, and hybrid search. Keyword search performs fast and flexible query parsing and matching over searchable fields, using terms or phrases in any supported language, with or without operators.

Semantic search improves the precision and relevance of search results by using a reranker (with AI models) to understand the semantic meaning of query terms and documents returned by the initial search ranker.

Vector search enables you to find documents that are similar to a given query input based on the vector embeddings of the content.

The following table summarizes the search options available:

Automatic index refreshing is supported for Azure Blob Storage only, allowing you to keep your Azure AI Search index up-to-date with your latest data.