Azure AI Foundry - Part4 - Deploy and use a generative AI model

Azure AI Foundry supports deploying large language models (LLMs). In this article, we will see how to deploy a model and use it.

Azure AI Foundry Portal

• Select your project - My assets - Models + endpoints - Deploy model
• Click Deploy base model
• Select the model you want to deploy (here I am selecting gpt-4.1) and click Confirm

• You can see the deployment details like capacity (token per minute), resource location etc. 
• Click on Create resource and deploy

• Now it will start creating the resource and this step may take a minute or so.
• Once it is done, it will take you to the following page where you can see the mode details on the model you just deployed.

• Click on Open in playground to test the model.
• Once the chat playground is open, you will see your deployment, and under that you will see a section where you can give the model instructions and context. An example is given in the following screenshot. Once the model instructions and context are provided make sure to click Apply changes button. 
• Now you can click on Generate prompt, provide the query and click on Send.
• You can also set values for limiting the maximum output token for the model response, temperature, frequency penalty etc. under the Parameters section.

• A sample response is provided in the following screenshot.

• To see the sample code, you can click on View code.

• You can also see code samples and authentication using API key as shown below.

• Metrics (total requests, token count, etc.) related to your LLM model deployment can be found on the following page.

 

Python

Sample code to interact with the model can be found in my GitHub repo.

Hope it was useful. Cheers!

Azure AI Foundry Blog Series

Azure AI Foundry is a comprehensive suite of tools and services designed to accelerate the development and deployment of AI solutions on the Azure platform. Throughout this blog series, we will cover various aspects of Azure AI Foundry.

Part1 - Create project
Part2 - Language translation using AI Services
Part3 - Abstractive text summarization
Part4 - Deploy and use a generative AI model

Azure AI Foundry - Part3 - Abstractive text summarization

In this article, I will show you how to use Azure Cognitive Services for text summarization. 

Azure AI Foundry portal

• AI Services - Language + Translator

• Summarize Information - Summarize text

• Select a connected AI service resource or create a new one.

• Playgrounds - Summarize Information - Summarize text

Python

Sample code to summarize a PDF can be found in my GitHub repo. Following is an example of a resume summary: 

Hope this was useful. Cheers!

Azure AI Foundry - Part2 - Language translation using AI Services

In this article, I will show you how to use an Azure AI Service available within the Azure AI Foundry project. We'll use the language translator as an example.

Azure AI Foundry portal

• Select AI Services. 
• Click on Language + Translator.

• Select Translation.

• Select Text Translation.

• Click on Try with your own.

• Here I am translating language from English to Malayalam. Take a look at the Connected Azure AI Services, you can see it is already connected to one. Incase if it is not connected to an Azure AI Services resource, you can click on Create a new AI Services resource, select a region, provide an AI Services name if you like to and click Create and connect. 

• You can also view the sample code by clicking on View code.

• Here is the sample code in Python and when you scroll down you can find the Resource key and Region details.

Python

import os, requests, uuid, json

resource_key = 'resource_key_here'
region = 'region_here'
endpoint = 'https://api.cognitive.microsofttranslator.com/'

# If you encounter any issues with the base_url or path, make sure
# that you are using the latest endpoint: https://docs.microsoft.com/azure/cognitive-services/translator/reference/v3-0-translate
path = '/translate?api-version=3.0'
params = '&to=ml'
constructed_url = endpoint + path + params

headers = {
    'Ocp-Apim-Subscription-Key': resource_key,
    'Ocp-Apim-Subscription-Region': region,
    'Content-type': 'application/json',
    'X-ClientTraceId': str(uuid.uuid4())
}

# You can pass more than one object in body.
body = [{
    'text' : 'where are you right now?'
}]
request = requests.post(constructed_url, headers=headers, json=body)
response = request.json()

print(json.dumps(response, sort_keys=True, indent=4, separators=(',', ': ')))

Sample output:

What you see is a unicode string and once it is converted you will see the corresponding Malayalam text. 

Curl

curl -X POST "https://api.cognitive.microsofttranslator.com/translate?api-version=3.0&to=ml" \
> -H "Ocp-Apim-Subscription-Key: your_key_here" \
> -H "Ocp-Apim-Subscription-Region: your_region_here" \
> -H "Content-Type: application/json" \
> -d "[{'Text':'where are you now?'}]" -v

Sample output:

Hope this was useful. Cheers!

References

• Translator Translate Method - Azure AI services | Microsoft Learn
• What is Azure Text Translation? - Azure AI services | Microsoft Learn
• Q: How do I use Azure Cognitive Services like Translator? | Microsoft Community Hub

Azure AI Foundry - Part1 - Create project

Azure AI Foundry is a unified platform that helps you design, customize, build, test, deploy, and manage generative AI applications. In this article, I will show you how to create a project and a hub, which are the first steps to building your AI solution. Let's get started!

Create project using the portal

• Sign in to the Azure AI Foundry portal https://ai.azure.com.

• Click Create project.
• The project will have an auto generated name or you can provide one. 
• You can also notice that it creates a new Hub, and Storage account, Key Vault and AI Services under a new resource group.
• Click Create.
• The project is getting created now. This may take a minute or two.

• Once it's done, it will take you to this overview page.

• On the Azure AI Foundry portal, you can use the Management center to configure/ get more details about your project, connected resources, models, endpoints etc.

• Under the Hub or Project properties, if you select the Resource Group, it will open a new browser tab and navigate to the Azure portal where you can see all the Azure resources that have been created to support your hub and project.

Create project using Azure CLI

Note: Remove any existing installation of the ml and azure-cli-ml extensions and install new.

• az extension remove -n azure-cli-ml
• az extension remove -n ml
• az extension add -n ml
• az extension update -n ml
  
  
• az login
• az account set --subscription "subscription_id"
• az group create --name "resource_group_name" --location "location_name"
  
  
• az ml workspace create --kind hub --resource-group "resource_group_name" --name "hub_name"

• $hub_id = "you will get this id from the output of previous step"

• az ml workspace create --kind project --hub-id $hub_id --resource-group "resource_group_name" --name "project_name"

In the next part we will explore the AI Services that are available within your Azure AI Foundry project. Hope this was useful. Cheers!

References

• Create an Azure AI Foundry project in Azure AI Foundry portal - Azure AI Foundry | Microsoft Learn
• Install and set up the CLI (v2) - Azure Machine Learning | Microsoft Learn
• How to create a hub using the Azure Machine Learning SDK/CLI - Azure AI Foundry | Microsoft Learn

VMware PowerCLI 101 - part10 - Verify vSphere cluster state

To verify the most common selected status attributes of the cluster:

Get-Cluster | Get-VMHost | Select Name,@{N='HAState';E={$_.ExtensionData.Runtime.DasHostState.State}},ConnectionState,PowerState,@{N='OverallStatus';E={$_.ExtensionData.OverallStatus}},@{N='ConfigStatus';E={$_.ExtensionData.ConfigStatus}},@{N='InMaintenanceMode';E={$_.ExtensionData.Runtime.InMaintenanceMode}},@{N='RebootRequired';E={$_.ExtensionData.Summary.RebootRequired}},@{N='BootTime';E={$_.ExtensionData.Runtime.BootTime}} | ft

> Get-Cluster | Get-VMHost | Select Name,@{N='HAState';E={$_.ExtensionData.Runtime.DasHostState.State}},ConnectionState,PowerState,@{N='OverallStatus';E={$_.ExtensionData.OverallStatus}},@{N='ConfigStatus';E={$_.ExtensionData.ConfigStatus}},@{N='InMaintenanceMode';E={$_.ExtensionData.Runtime.InMaintenanceMode}},@{N='RebootRequired';E={$_.ExtensionData.Summary.RebootRequired}},@{N='BootTime';E={$_.ExtensionData.Runtime.BootTime}} | ft

Name      HAState           ConnectionState PowerState OverallStatus ConfigStatus InMaintenanceMode RebootRequired BootTime
----      -------           --------------- ---------- ------------- ------------ ----------------- -------------- --------
10.90.1.4 connectedToMaster       Connected  PoweredOn        yellow       yellow             False          False 8/27/2024 7:31:10 AM
10.90.1.5 master                  Connected  PoweredOn        yellow       yellow             False          False 9/6/2024 9:08:09 PM

Hope it was useful. Cheers!

Revisiting Storage Performance Benchmarking

Few years ago, I had the opportunity to explore the intricacies of storage performance benchmarking using tools like FIO, DISKSPD, and Iometer. Those studies provided valuable insights into the performance characteristics of various storage solutions, shaping my understanding and approach to storage performance analysis. As I prepare for an upcoming project in this domain, I find it essential to revisit my previous work, reflect on the lessons learned, and share my experiences. This blog post aims to provide a comprehensive overview of my benchmarking journey and the evolving landscape of storage performance studies.

Recent advancements 

The field of storage technology has seen significant advancements in recent years. The rise of NVMe and storage-class memory technologies has also redefined high-end storage performance, offering unprecedented speed and efficiency. These advancements highlight the dynamic nature of storage performance benchmarking and underscore the importance of staying updated with the latest tools and methodologies.

Challenges

Benchmarking storage performance is not without its challenges. One of the primary difficulties is ensuring a consistent and controlled testing environment, as variations in hardware, software, and network conditions can significantly impact results. Another challenge is the selection of appropriate benchmarks that accurately reflect real-world workloads, which requires a deep understanding of the specific use cases and performance metrics. Additionally, interpreting the results can be complex, as it involves analyzing multiple metrics such as IOPS, throughput, and latency, and understanding their interplay. These challenges necessitate meticulous planning and a thorough understanding of both the benchmarking tools and the storage systems being tested.

Prior works

Following are some of the articles on storage benchmarking that I’ve published in the past:

• Benchmarking vSphere environment using HCIBench
• vSAN performance benchmarking considerations
• Stress test your storage system using Iometer
• Storage performance benchmarking of Kubernetes using FIO StatefulSet
• Benchmarking Kubernetes using K-Bench

Custom storage benchmarking framework

While there are numerous storage benchmarking tools available, such as VMFleet and HCIBench, I wanted to highlight a custom framework I developed a few years ago. Here are some reasons why we created this custom tool:

• Great learning experience: It provided valuable insights into how things work.
• Customization: Being a custom framework, it allows you to add or remove features as needed.
• Flexibility: You can modify multiple parameters to suit your requirements.
• Custom test profiles: You can create tailored storage test profiles.
• No IP assignment needed: There’s no need for IP assignment or DHCP for the stress test VMs.
• Centralized log collection: It offers centralized log collection for detailed analysis.

You can access the scripts and readme on my GitHub repository:

https://github.com/vineethac/vsan_cluster_storage_benchmarking_with_diskspd

Here is an overview.

• Profile Manifest: All storage test profiles are listed in profile_manifest.psd1. You can define as many profiles as you want.

• VM Template: A Windows VM template should be present in the vCenter server.

• Benchmarking Manifest: Details of vCenter, cluster name, VM template, number of stress test VMs per host, etc., are provided in benchmarking_manifest.psd1.

• Deploy Test VMs: deploy_test_vms.ps1 will deploy all the test VMs with pre-configured parameters.

• Start Stress Test: start_stress_test.ps1 will initiate the storage stress test process for all the profiles mentioned in profile_manifest.psd1 one by one.

• Log Collection: All log files will be automatically copied to a central location on the host from where these scripts are running.

• Cleanup: Use delete_test_vms.ps1 to clean up the stress test VMs from the cluster.

Note: These scripts were created about five years ago, and I haven’t had the opportunity to refactor them according to current best practices and new PowerShell scripting standards. I plan to enhance them in the coming months!

This overview should provide you with a clear understanding of the overall process and workflow involved in the storage benchmarking process. I hope it was useful. Cheers!