Introductio

n
• Azure AI services provides a cloud-based platform for building artificial
intelligence capabilities into your applications.

• Like any software service, you should monitor AI services to track costs,
identify utilization trends, and detect potential issues.

• After completing this module, you'll be able to:

• Monitor Azure AI services costs.

• Create alerts and view metrics for Azure AI services.
• Manage Azure AI services diagnostic logging.
Monitor
cost

• One of the main benefits of using cloud services is that you can gain cost
efficiencies by only paying for services as you use them.

• Some Azure AI services resources offer a free tier with restrictions on use,
which is useful for development and testing; and one or more billed tiers that
incur charges based on transactions. The specific billing rate depends on the
resource type.
Monitor
cost
Plan costs for AI
services
• Before deploying a solution that depends on AI services, you can estimate costs by
using the Azure Pricing Calculator.

• To use the pricing calculator to estimate AI services costs, create a new estimate
and select Azure AI services in the AI + Machine Learning category.

• Then select the specific AI service API you plan to use (for example, Azure AI Text
Analytics), the region where you plan to provision it, and the pricing tier of the
instance you plan to use; and fill in the expected usage metrics and support option.

• To create an estimate that includes multiple AI services APIs, add additional Azure
AI services products to the estimate.

• After you've created an estimate, you can save it. You can also export it in
Microsoft Excel format.
Monitor
cost
View costs for AI
services
• In common with other Azure resources, you can view details of accumulated
costs for AI services resources in the Azure portal.

• To view costs for AI services, sign into the Azure portal and select your
subscription.

• You can then view overall costs for the subscription by selecting the Cost
analysis tab.

• To view only costs for AI services, add a filter that restricts the data to reflect
resources with a service name of Cognitive Services.
Create
alerts
• Microsoft Azure provides alerting support for resources through the creation of alert
rules.

• You use alert rules to configure notifications and alerts for your resources based on
events or metric thresholds.

• These alerts will ensure that the correct team knows when a problem arises.
Create
alerts
Create
alerts
• Alert rules
• To create an alert rule for an Azure AI services resource, select the resource in
the Azure portal and on the Alerts tab, add a new alert rule. To define the alert
rule, you must specify:

• The scope of the alert rule - in other words, the resource you want to
monitor.

• A condition on which the alert is triggered. The specific trigger for the alert
is based on a signal type, which can be Activity Log (an entry in the activity
log created by an action performed on the resource, such as regenerating its
subscription keys) or Metric (a metric threshold such as the number of errors
exceeding 10 in an hour).

• Optional actions, such as sending an email to an administrator notifying

them of the alert, or running an Azure Logic App to address the issue
automatically.

• Alert rule details, such as a name for the alert rule and the resource group in
View
metrics
• Azure Monitor collects metrics for Azure resources at regular intervals so that
you can track indicators of resource utilization, health, and performance.

• The specific metrics gathered depend on the Azure resource. In the case of
Azure AI services, Azure Monitor collects metrics relating to endpoint requests,
data submitted and returned, errors, and other useful measurements.

View metrics in the Azure portal

• You can view metrics for an individual resource in the Azure portal by selecting
the resource and viewing its Metrics page.

• On this page, you can add resource-specific metrics to charts. By default an

empty chart is created for you, and you can add more charts as required.

• For example, the following image shows the Metrics page for an AI services
resource, showing the count of total calls to the service over a period of time.
View
metrics
View metrics in the Azure portal
View
metrics
View metrics in the Azure portal
• You can add multiple metrics to a chart and choose appropriate aggregations
and chart types.

• When you're satisfied with chart, you can share it by exporting it to Excel or
copying a link to it, and you can clone it to create a duplicate chart in the
Metrics page - potentially as a starting point for a new chart that shows the
same metrics in a different way.
View
metrics
Add metrics to a dashboard
• In the Azure portal, you can create dashboards that consist of multiple
visualizations from different resources in your Azure environment to help you
gain an overall view of the health and performance of your Azure resources.

• To create a dashboard, select Dashboard in the Azure portal menu (your

default view may already be set to a dashboard rather than the portal home
page).

• From here, you can add up to 100 named dashboards to encapsulate views for
specific aspects of your Azure services that you want to track.

• You can add a range of tiles and other visualizations to a dashboard, and
when viewing metrics for a specific resource in a chart, as described
previously, you can add the chart to a new or existing dashboard.

• In the following image, two charts showing metrics for an AI services resource
have been added to a dashboard.
View
metrics
Add metrics to a dashboard
Manage diagnostic
logging
• Diagnostic logging enables you to capture rich operational data for an Azure AI
services resource, which can be used to analyze service usage and
troubleshoot problems.

Create resources for diagnostic log

storage
• To capture diagnostic logs for an AI services resource, you need a destination for
the log data.

• You can use Azure Event Hubs as a destination in order to then forward the data
on to a custom telemetry solution, and you can connect directly to some third-
party solutions; but in most cases you'll use one (or both) of the following kinds
of resource within your Azure subscription:
Manage diagnostic
logging
Create resources for diagnostic log
storage
• Azure Log Analytics - a service that enables you to query and visualize log
data within the Azure portal.

• Azure Storage - a cloud-based data store that you can use to store log
archives (which can be exported for analysis in other tools as needed).

• You should create these resources before configuring diagnostic logging for your
AI services resource.

• If you intend to archive log data to Azure Storage, create the Azure Storage
account in the same region as your AI services resource
Manage diagnostic
logging
Configure diagnostic settings
• With your log destinations in place, you can configure diagnostic settings for
your AI services resource.

• You define diagnostic settings on the Diagnostic settings page of the blade for
your AI services resource in the Azure portal. When you add diagnostic settings,
you must specify:

• A name for your diagnostic settings.

• The categories of log event data that you want to capture.

• Details of the destinations in which you want to store the log data.

• In the following example, the diagnostic settings store all available log data and
metrics in Azure Log Analytics and Azure Storage.
Manage diagnostic
logging
Configure diagnostic settings
Manage diagnostic
logging
View log data in Azure Log Analytics

• It can take an hour or more before diagnostic data starts flowing to the
destinations, but when the data has been captured, you can view it in your
Azure log Analytics resource by running queries, as shown in this example.
Manage diagnostic
logging
View log data in Azure Log Analytics
Exercise - Monitor Azure AI
services
Knowledge check
1. How should you collect telemetry for your Azure AI Services resource for later
analysis?

a) Create an alert.
b) Configure diagnostic settings.
c) Create a dashboard.

2. You are defining an alert that notifies you when a key regeneration event is
recorded in the activity log for your Azure AI Services resource. What should you
do?

d) Specify a Scope of Activity Log.

e) Specify an Action that uses an Azure Logic App to read the activity log.
f) Specify a Condition with a Signal Type of Activity Log.
Knowledge check
3. You are viewing a metric for your Azure AI Services resource in a chart. You want
to combine the chart with visualizations of other resources and data. What should
you do?

a) Add the chart to a dashboard.

b) Share the chart.
c) Clone the chart.