Azure Arc Overview

Indirectly connected then the view from the Azure portal is read only, this allows you to see an inventory of your SQL managed instances but you can’t do anything with them from the portal and local tools must be used to make changes. Directly connected of course offers the most benefits and allows users to use Azure Resource manager (ARM) APIs, Azure CLI and the Azure portal and is a similar experience to using the service in Azure. Currently within this category the only service which is GA is Azure SQL Managed Instance. These Kubernetes clusters appear as Azure Resources, becoming a platform to deploy Azure Services to. Azure Arc enabled Kubernetes works with any Cloud Native Computing Foundation (CNCF) certified Kubernetes clusters and this includes clusters running in other cloud native platforms such as AWS and GCP.

Onboard the Azure VM as an Azure Arc-enabled server

The host node /models folder will be mapped to /models folder within the deployed container pod. This method was not used for this deployment since minkube Kubernetes version was incompatible with resource types required for Azure GitOps agents. Further both deployments uses the same tools, install methods and were monitored centrally from Azure cloud. Instead of local VM node, steps were performed on the Azure cloud VM node and the Azure cloud Kubernetes instance. The Azure Kubernetes instance was deployed under the same Azure Resource Group as the earlier minikube local Kubernetes instance. The steps for launching the test client pod and executing the demo inference script is provided in the Appendix.

Azure Arc-enabled vSphere allows the automatic discovery and onboarding of VMware machines to Azure Arc as well as enables self service. This provides an inventory of all of your cloud resources and again allows you to apply Tags and Azure policies to these resources. The Multicloud connector enabled by Azure Arc went in to public preview in June 2024, this allows organizations to connect non-Azure Public Cloud resources to Azure.

The inference request is a sequence of 10 images sent one at a time to classify the objects in the images using resnet model uploaded earlier to the OVMS. The script uses grpc API to make inference request to OVMS container pod. These processors include Intel® Deep Learning Boost Vector Neural Network Instructions (VNNI), based on Intel Advanced Vector Extensions 512 (AVX-512) for optimized and improved inference performance.

1 OVMS DEPLOYMENT ON MINIKUBE

There will be no surprise that the service which has made it to GA is of course machine learning. That said all of these PaaS services are currently in public preview and available to try out. All of these services are currently still in public preview and have been since the launch of Azure Arc. This provides options around how much data is sent to Azure, how users interact with the Arc Data Controller and the option you choose may impact the availability of data services. We’ll come on to Azure Services next but the first thing to know is that they require an Azure Arc enabled Kubernetes platform as a deployment target. You can also configure clusters and deploy applications using GitOps based config management.

The steps for installing OVMS in minikube environment is provided in the Appendix (below). The model data was provided in OpenVINO optimized format (.bin and .xml files). A Resnet50 trained model’s data was hosted in the OVMS pod.

Azure Arc: Using an Azure Windows VM as an Arc-enabled server for learning and training

While it’s strongly not advised to install Azure Arc-enabled servers on an Azure VM for production scenarios, it is possible to configure Azure Arc-enabled servers on an Azure VM for training and testing purposes only. To do this, copy the content of the OnboardingScript.ps1 into a new script file in PowerShell ISE on the VM, and then run the script. The final step is to onboard the Azure VM as an Azure Arc-enabled server. After that, click the Run Script button to execute the script. Then, in a new script file, copy the content of the script you copied above or downloaded from GitHub. After copying the script or downloading it from GitHub, open it in Visual Studio Code or your preferred editor.

The inference request included specifying the Resnet50 model name to be used for inference. The image classification script was used that submits a list of images one at a time to OVMS for inference. The OVMS deployment can verified with a demo Inference application. The Kubernetes OVMS deployed state details are shown in figure below. As a proof-of-concept, a Kubernetes deployment was setup at one of the Intel onsite lab locations. This benefit comes along with utilizing a “single pane” for management and application deployment of disperse Kubernetes clusters from Azure Cloud as show in Solution Overview Figure 1 below.

• The model data was provided in OpenVINO optimized format (.bin and .xml files).
• Organisations can make use of role based access control to manage and delegate access to teams across the business.
• The inference request included specifying the Resnet50 model name to be used for inference.
• To prepare the Azure VM for onboarding to Azure Arc, we need to connect and log in to it in order to perform the necessary steps for configuration.
• The final step is to onboard the Azure VM as an Azure Arc-enabled server.

2 DEMO INFERENCE SCRIPT EXECUTION ON MINIKUBE

I spoke previously about the fact that the majority of organisations today are adopting a hybrid approach to their cloud strategy and discussed some of the reasons for this. The IP address is the Kubernetes cluster service IP assigned to the OVMS service. Instead of docker hub for source container image, a private azure registry can be created and the docker image can be pulled into the private registry. The model path in helm install is set to match the host folder /models/resnet created in step 1 above.

To use the script, start by saving a copy as “Create-Azure-Windows-VM-with-VNet-and-RG-for-Azure-Arc-learning.ps1” or downloading it directly from GitHub. You can either use the Azure Portal or, for a faster setup, run the Azure PowerShell script below. In this blog post, I’ll walk you through the easy setup process using Azure PowerShell, a PowerShell script, and the free Azure Bastion Developer SKU. This can be particularly useful for testing or learning purposes when you don’t have access to any on-premises machines. In any normal scenario, you can’t connect an Azure VM as an Azure Arc-enabled server because it’s already represented in Azure Resource Manager, with all native Azure features and capabilities easily accessible if needed. These professionals have the options to create their own models or use models built from open-source platforms like PyTorch.

Model data is required by OVMS in a specific directory and file format to support Inference requests for the models (Reference 7). A dual socket server configured with 2 x Intel(R) Xeon(R) Gold 6348 CPUs (3rd generation Xeon scalable processors, codename Ice Lake) was setup with Windows Server 2002 Hyper-V. Azure Arc-Enabled Kubernetes enables centralized management of heterogenous and geographically separate Kubernetes clusters from Azure public cloud. Kubernetes based offerings from different vendors are supported and they need not be based on Azure Kubernetes Service (AKS) (Reference 2).

Azure prerequisites

The same steps used for the on premise minikube instance was used for OVMS deployment. A Kubernetes cluster was deployed in Azure cloud using Azure Kubernetes service. The local minikube based Kubernetes deployment was Azure Arc enabled via a connection procedure to the Azure cloud (Reference 6). OVMS is based on the same architecture as TensorFlow Serving and Inference as a service is provided via gRPC or REST API, making it easy to consume by applications that require AI model inference.

Azure Arc-enabled Application Services

• I prefer using Azure Bastion for this, and fortunately, for Dev/Test purposes, you can now easily set up an Azure Bastion Developer, which is a free, lightweight version of the Azure Bastion service.
• That said all of these PaaS services are currently in public preview and available to try out.
• In this blog post, I’ll show you how to accomplish this using Azure PowerShell, a standard PowerShell script, and the Azure Bastion Developer host to configure everything in an easy and free way.
• This method was not used for this deployment since minkube Kubernetes version was incompatible with resource types required for Azure GitOps agents.
• If you followed the previous steps to deploy a new VM, no extensions, such as the Azure Monitor agent, should be installed.

It will also cover centrally monitoring the 2 deployments from Azure public cloud. It also includes OpenVINO Model Server (OVMS) for serving high performance machine learning models as a service (Reference 4). Currently, his main focus is on the Microsoft Hybrid Cloud Platform, and especially on Microsoft Azure and the Azure hybrid services. As a Microsoft Certified Trainer (MCT), his strength is assisting companies in the transformation of their businesses to the Cloud by implementing the latest features, services, and solutions. Then, go to the Azure Portal and navigate to the Azure VM you deployed or want to onboard to Azure Arc. To make these steps easier, you can either copy the PowerShell script below or download it from GitHub.

To prepare the Azure VM for onboarding to Azure Arc, we need to connect and log in to it in order to perform the necessary steps for configuration. If you want to use a different VM size than the default “Standard_B2ms“, run the script with the following parameters. You can then run the script, with the required parameters.

Applications or services packaged as containers can be deployed and managed with the same Kubernetes based eco-system tools in the public cloud, on premise or Edge locations. In this article we demonstrated Intel OpenVINO based inference deployment on 3rd Generation Intel Xeon processors, both on-premise and in Azure cloud with Kubernetes. The consistency and mobility of deploying inference applications using OpenVINO in a hybrid Kubernetes based cloud environment was demonstrated.

At present there are only a couple of Azure services that are generally available (GA) with many still in preview. For instance, once a virtual machine (VM) is projected in to Azure it looks and feels very much like an Azure VM, you can then view it along side all of your other Azure VMs. It is important to understand that Azure Arc is not single service, it is not something you deploy and boom you have all of the capabilities of Azure Arc and are charged for a single service.

This application development and deployment model enables scale and agility in today’s hybrid and multi-cloud environments. This script will create a virtual network (VNet) in a specified resource group, deploy an Azure VM running Windows Server 2022 within the same resource group, and connect the VM to a subnet in the VNet. Keep in might you need an Arc-enabled Kubernetes platform to deploy these services too. There are services which are free for Azure VMs such as Azure Policy Guest Configuration and Azure Automation services like Update Management which are charged for per server for Azure Arc enabled Servers. If you want to take advantage of services such as Azure Monitor etc on your Arc connected servers then these will be charged just as if you were using these on your Azure VMs are they are an Azure Service. Azure Machine learning can be used by Machine Learning (ML) professionals, data scientists and engineers in their day to day workflows to train and deploy models as well as manage machine learning operations.

To get started, we’ll first provision a new Azure VM in a dedicated resource group within the sandboxed subscription. However, there are a few steps you can take as a workaround to onboard an Azure VM running Windows Server as an Arc-enabled server. ⚠️ Keep in mind that this blog post and the steps to configure an Azure VM as an Arc-enabled server are meant exclusively for testing and research. You can go ahead and Azure Arc enable servers or use Azure Arc enabled VMware vSphere and get centralized visibility of your assets, tag them, place them in resource groups etc at no extra cost. The Data Science team will then be able to discover a list of available compute targets and instance types in the Machine learning workspace and these can be used for training or inference workloads.