Contents

Azure Red Hat OpenShift

Overview
About Azure Red Hat OpenShift
Tutorials
Create and manage an Azure Red Hat OpenShift cluster
1 - Create an Azure Red Hat OpenShift cluster
2 - Scale an Azure Red Hat OpenShift cluster
3 - Delete an Azure Red Hat OpenShift cluster
How-to guides
Set up your Azure Red Hat OpenShift dev environment
Create an Azure AD tenant
Create an Azure AD app object and user
Deploy a standalone prometheus in an Azure Red Hat OpenShift cluster
Reference
Azure Red Hat OpenShift CLI
Azure Red Hat OpenShift REST APIs
Resources
Common questions
Troubleshooting
Supported resources
Regional availability
Red Hat OpenShift documentation
Azure Roadmap
 Azure Red Hat OpenShift
7/29/2019 • 2 minutes to read • Edit Online

The Microsoft Azure Red Hat OpenShift service enables you to deploy fully managed OpenShift clusters.
Azure Red Hat OpenShift extends Kubernetes. Running containers in production with Kubernetes requires
additional tools and resources, such as an image registry, storage management, networking solutions, and logging
and monitoring tools, all of which must be versioned and tested together. Building container-based applications
requires even more integration work with middleware, frameworks, databases, and CI/CD tools. Azure Red Hat
OpenShift combines all this into a single platform, bringing ease of operations to IT teams while giving application
teams what they need to execute.
Azure Red Hat OpenShift is jointly engineered, operated, and supported by Red Hat and Microsoft to provide an
integrated support experience. There are no virtual machines to operate, and no patching is required. Master,
infrastructure and application nodes are patched, updated, and monitored on your behalf by Red Hat and
Microsoft. Your Azure Red Hat OpenShift clusters are deployed into your Azure subscription and are included on
your Azure bill.
You can choose your own registry, networking, storage, and CI/CD solutions, or use the built-in solutions for
automated source code management, container and application builds, deployments, scaling, health management,
and more. Azure Red Hat OpenShift provides an integrated sign-on experience through Azure Active Directory.
To get started, complete the Create an Azure Red Hat OpenShift cluster tutorial.

Access, security, and monitoring

For improved security and management, Azure Red Hat OpenShift lets you integrate with Azure Active Directory
(Azure AD ) and use Kubernetes role-based access control (RBAC ). You can also monitor the health of your cluster
and resources.

Cluster and node

Azure Red Hat OpenShift nodes run on Azure virtual machines. You can connect storage to nodes and pods,
upgrade cluster components, and use GPUs.

Virtual networks and ingress

You can connect an Azure Red Hat OpenShift cluster to an existing virtual network via peering. In this
configuration, pods can connect to other services in a peered virtual network.
See Connect a cluster's virtual network to an existing virtual network for details.

Kubernetes certification
Azure Red Hat OpenShift service has been CNCF certified as Kubernetes conformant.

Next steps
Learn the prerequisites for Azure Red Hat OpenShift:
Set up your dev environment
 Tutorial: Create an Azure Red Hat OpenShift cluster
7/9/2019 • 6 minutes to read • Edit Online

This tutorial is part one of a series. You'll learn how to create a Microsoft Azure Red Hat OpenShift cluster using
the Azure CLI, scale it, then delete it to clean up resources.
In part one of the series, you'll learn how to:
Create an Azure Red Hat OpenShift cluster
In this tutorial series you learn how to:
Create an Azure Red Hat OpenShift cluster
Scale an Azure Red Hat OpenShift cluster
Delete an Azure Red Hat OpenShift cluster

Prerequisites
IMPORTANT
This tutorial requires version 2.0.65 of the Azure CLI.
Before you can use Azure Red Hat OpenShift, you'll need to purchase a minimum of 4 Azure Red Hat OpenShift reserved
application nodes as described in Set up your Azure Red Hat OpenShift development environment.

Before you begin this tutorial:

Make sure that you've set up your development environment, which includes:
Installing the latest CLI (version 2.0.65 or above)
Creating a tenant if you don't already have one
Creating an Azure Application object if you don't already have one
Creating a security group
Creating an Active Directory user to sign in to the cluster.

Step 1: Sign in to Azure

If you're running the Azure CLI locally, open a Bash command shell and run az login to sign in to Azure.

az login

If you have access to multiple subscriptions, run az account set -s {subscription ID} replacing
{subscription ID} with the subscription you want to use.

Step 2: Create an Azure Red Hat OpenShift cluster

In a Bash command window, set the following variables:
 IMPORTANT
Choose a name for you cluster that is unique and all lowercase or cluster creation will fail.

CLUSTER_NAME=<cluster name in lowercase>

Choose a location to create your cluster. For a list of azure regions that supports OpenShift on Azure, see
Supported Regions. For example: LOCATION=eastus .

LOCATION=<location>

Set APPID to the value you saved in step 5 of Create an Azure AD app registration.

APPID=<app ID value>

Set 'GROUPID' to the value you saved in step 10 of Create an Azure AD security group.

GROUPID=<group ID value>

Set SECRET to the value you saved in step 8 of Create a client secret.

SECRET=<secret value>

Set TENANT to the tenant ID value you saved in step 7 of Create a new tenant

TENANT=<tenant ID>

Create the resource group for the cluster. Run the following command from the same Bash shell that you used to
define the variables above:

az group create --name $CLUSTER_NAME --location $LOCATION

Optional: Connect the cluster's virtual network to an existing virtual network

If you don't need to connect the virtual network (VNET) of the cluster you create to an existing VNET via peering,
skip this step.
If peering to a network outside the default subscription then in that subscription, you will also need to register the
provider Microsoft.ContainerService. To do this, run the below command in that subscription. Else, if the VNET
you are peering is located in the same subscription, you can skip the registering step.
az provider register -n Microsoft.ContainerService --wait

First, get the identifier of the existing VNET. The identifier will be of the form:
/subscriptions/{subscription id}/resourceGroups/{resource group of
VNET}/providers/Microsoft.Network/virtualNetworks/{VNET name}
.
If you don't know the network name or the resource group the existing VNET belongs to, go to the Virtual
networks blade and click on your virtual network. The Virtual network page appears and will list the name of the
network and the resource group it belongs to.
Define a VNET_ID variable using the following CLI command in a BASH shell:

VNET_ID=$(az network vnet show -n {VNET name} -g {VNET resource group} --query id -o tsv)

For example: VNET_ID=$(az network vnet show -n MyVirtualNetwork -g MyResourceGroup --query id -o tsv

Create the cluster

You're now ready to create a cluster. The following will create the cluster in the specified Azure AD tenant, specify
the Azure AD app object and secret to use as a security principal, and the security group that contains the
members that have admin access to the cluster.
If you are not peering your cluster to a virtual network, use the following command:

az openshift create --resource-group $CLUSTER_NAME --name $CLUSTER_NAME -l $LOCATION --aad-client-app-id

$APPID --aad-client-app-secret $SECRET --aad-tenant-id $TENANT --customer-admin-group-id $GROUPID

If you are peering your cluster to a virtual network, use the following command which adds the --vnet-peer
flag:

az openshift create --resource-group $CLUSTER_NAME --name $CLUSTER_NAME -l $LOCATION --aad-client-app-id

$APPID --aad-client-app-secret $SECRET --aad-tenant-id $TENANT --customer-admin-group-id $GROUPID --vnet-peer
$VNET_ID

NOTE
If you get an error that the host name is not available, it may be because your cluster name is not unique. Try deleting your
original app registration and redoing the steps with a different cluster name in [Create a new app registration] (howto-aad-
app-configuration.md#create-a-new-app-registration), omitting the step of creating a new user and security group.

After a few minutes, az openshift create will complete.

Get the sign in URL for your cluster
Get the URL to sign in to your cluster by running the following command:

az openshift show -n $CLUSTER_NAME -g $CLUSTER_NAME

Look for the publicHostName in the output, for example:

"publicHostname": "openshift.xxxxxxxxxxxxxxxxxxxx.eastus.azmosa.io"

The sign in URL for your cluster will be followed by the publicHostName value. For example:
https://
https://openshift.xxxxxxxxxxxxxxxxxxxx.eastus.azmosa.io . You will use this URI in the next step as part of the app
registration redirect URI.

Step 3: Update your app registration redirect URI

Now that you have the sign in URL for the cluster, set the app registration redirect UI:
1. Open the App registrations blade.
2. Click on your app registration object.
3. Click on Add a redirect URI.
4. Ensure that TYPE is Web and set the REDIRECT URI using the following pattern:
https://<public host name>/oauth2callback/Azure%20AD . For example:
 https://openshift.xxxxxxxxxxxxxxxxxxxx.eastus.azmosa.io/oauth2callback/Azure%20AD
5. Click Save

Step 4: Sign in to the OpenShift console

You're now ready to sign in to the OpenShift console for your new cluster. The OpenShift Web Console enables
you to visualize, browse, and manage the contents of your OpenShift projects.
You'll need a fresh browser instance that hasn't cached the identity you normally use to sign in to the Azure
portal.
1. Open an incognito window (Chrome) or InPrivate window (Microsoft Edge).
2. Navigate to the sign-on URL that you obtained above, for example:
https://openshift.xxxxxxxxxxxxxxxxxxxx.eastus.azmosa.io

Sign in using the user name you created in step 3 of Create a new Azure Active Directory user.
A Permissions requested dialog will appear. Click Consent on behalf of your organization and then click
Accept.
You are now logged into the cluster console.

Learn more about using the OpenShift console to create and built images in the Red Hat OpenShift
documentation.

Step 5: Install the OpenShift CLI

The OpenShift CLI (or OC Tools) provide commands for managing your applications and lower-level utilities for
interacting with the various components of your OpenShift cluster.
In the OpenShift console, click the question mark in the upper right corner by your sign-in name and select
Command Line Tools. Follow the Latest Release link to download and install the supported oc CLI for Linux,
MacOS, or Windows.
 NOTE
If you do not see the question mark icon in the upper right corner, select Service Catalog or Application Console from the
upper left-hand drop-down.
Alternately, you can download the oc CLI directly.

The Command Line Tools page provides a command of the form

oc login https://<your cluster name>.<azure region>.cloudapp.azure.com --token=<token value> . Click the Copy to
clipboard button to copy this command. In a terminal window, set your path to include your local installation of
the oc tools. Then sign in to the cluster using the oc CLI command you copied.
If you couldn't get the token value using the steps above, get the token value from:
https://<your cluster name>.<azure region>.cloudapp.azure.com/oauth/token/request .

Next steps
In this part of the tutorial, you learned how to:
Create an Azure Red Hat OpenShift cluster
Advance to the next tutorial:
Scale an Azure Red Hat OpenShift cluster
 Tutorial: Scale an Azure Red Hat OpenShift cluster
7/9/2019 • 2 minutes to read • Edit Online

This tutorial is part two of a series. You'll learn how to create a Microsoft Azure Red Hat OpenShift cluster using
the Azure CLI, scale it, then delete it to clean up resources.
In part two of the series, you learn how to:
Scale a Red Hat OpenShift cluster
In this tutorial series you learn how to:
Create an Azure Red Hat OpenShift cluster
Scale an Azure Red Hat OpenShift cluster
Delete an Azure Red Hat OpenShift cluster

Prerequisites
Before you begin this tutorial:
Create a cluster by following the Create an Azure Red Hat OpenShift cluster tutorial.

Step 1: Sign in to Azure

If you're running the Azure CLI locally, run az login to sign in to Azure.

az login

If you have access to multiple subscriptions, run az account set -s {subscription ID} replacing
{subscription ID} with the subscription you want to use.

Step 2: Scale the cluster with additional nodes

From a Bash terminal, set the variable CLUSTER_NAME to the name of your cluster:

CLUSTER_NAME=yourclustername

Now let's scale the cluster to five nodes using the Azure CLI:

az openshift scale --resource-group $CLUSTER_NAME --name $CLUSTER_NAME --compute-count 5

After a few minutes, az openshift scale will complete successfully and return a JSON document containing the
scaled cluster details.

Next steps
In this part of the tutorial, you learned how to:
Scale an Azure Red Hat OpenShift cluster
Advance to the next tutorial:
Delete an Azure Red Hat OpenShift cluster
 Tutorial: Delete an Azure Red Hat OpenShift cluster
7/9/2019 • 2 minutes to read • Edit Online

This is the end of the tutorial. When you're finished testing the sample cluster, here's how to delete it and its
associated resources so you don't get charged for what you're not using.
In part three of the series, you learn how to:
Delete an Azure Red Hat OpenShift cluster
In this tutorial series you learn how to:
Create an Azure Red Hat OpenShift cluster
Scale an Azure Red Hat OpenShift cluster
Delete an Azure Red Hat OpenShift cluster

Prerequisites
Before you begin this tutorial:
Create a cluster by following the Create an Azure Red Hat OpenShift cluster tutorial.

Step 1: Sign in to Azure

If you're running the Azure CLI locally, run az login to sign in to Azure.

az login

If you have access to multiple subscriptions, run az account set -s {subscription ID} replacing
{subscription ID} with the subscription you want to use.

Step 2: Delete the cluster

Open a Bash terminal and set the variable CLUSTER_NAME to the name of your cluster:

CLUSTER_NAME=yourclustername

Now delete your cluster:

az openshift delete --resource-group $CLUSTER_NAME --name $CLUSTER_NAME

You'll be prompted whether you want to delete the cluster. After you confirm with y , it will take several minutes
to delete the cluster. When the command finishes, the entire Resource Group and all resources inside it, including
the cluster, will be deleted.

Deleting a cluster using the Azure portal

Alternately, you can delete the associated resource group of your cluster through the online Azure portal. The
name of the resource group is the same as your cluster name.
Currently, the Microsoft.ContainerService/openShiftManagedClusters resource that is created when you create the
cluster is hidden in the Azure portal. In the Resource group view, check Show hidden types to view the resource
group.

Deleting the resource group will delete all of the related resources that get created when you build an Azure Red
Hat OpenShift cluster.

Next steps
In this part of the tutorial, you learned how to:
Delete an Azure Red Hat OpenShift cluster
Learn more about using OpenShift with the official Red Hat OpenShift documentation
 Set up your Azure Red Hat OpenShift dev
environment
7/9/2019 • 3 minutes to read • Edit Online

To build and run Microsoft Azure Red Hat OpenShift applications, you'll need to:
Purchase Azure virtual machine reserved instances.
Install version 2.0.65 (or higher) of the Azure CLI (or use the Azure Cloud Shell).
Register for the AROGA feature and associated resource providers.
Create an Azure Active Directory (Azure AD ) tenant.
Create an Azure AD application object.
Create an Azure AD user.
The following instructions will walk you through all of these prerequisites.

Purchase Azure Red Hat OpenShift application nodes reserved

instances
Before you can use Azure Red Hat OpenShift, you'll need to purchase a minimum of 4 Azure Red Hat OpenShift
reserved application nodes, after which you'll be able to provision clusters.
If you are an Azure customer, purchase Azure Red Hat OpenShift reserved instances through the Azure portal.
After purchasing, your subscription will be activated within 24 hours.
If you are not an Azure customer, contact sales and fill out the sales form at the bottom of the page to start the
process.
Refer to the Azure Red Hat OpenShift pricing page for more information.

Install the Azure CLI

Azure Red Hat OpenShift requires version 2.0.65 or higher of the Azure CLI. If you've already installed the Azure
CLI, you can check which version you have by running:

az --version

The first line of output will have the CLI version, for example azure-cli (2.0.65) .
Here are instructions for installing the Azure CLI if you require a new installation or an upgrade.
Alternately, you can use the Azure Cloud Shell. When using the Azure Cloud Shell, be sure to select the Bash
environment if you plan to follow along with the Create and manage an Azure Red Hat OpenShift cluster tutorial
series.

Register providers and features

The feature, Microsoft.Solutions , Microsoft.Compute , Microsoft.Storage ,
Microsoft.ContainerService AROGA
Microsoft.KeyVault and Microsoft.Network providers must be registered to your subscription manually before
deploying your first Azure Red Hat OpenShift cluster.
To register these providers and features manually, use the following instructions from a Bash shell if you've
installed the CLI, or from the Azure Cloud Shell (Bash) session in your Azure portal:
1. If you have multiple Azure subscriptions, specify the relevant subscription ID:

az account set --subscription <SUBSCRIPTION ID>

2. Register the Microsoft.ContainerService AROGA feature:

az feature register --namespace Microsoft.ContainerService -n AROGA

3. Register the Microsoft.Storage provider:

az provider register -n Microsoft.Storage --wait

4. Register the Microsoft.Compute provider:

az provider register -n Microsoft.Compute --wait

5. Register the Microsoft.Solutions provider:

az provider register -n Microsoft.Solutions --wait

6. Register the Microsoft.Network provider:

az provider register -n Microsoft.Network --wait

7. Register the Microsoft.KeyVault provider:

az provider register -n Microsoft.KeyVault --wait

8. Refresh the registration of the Microsoft.ContainerService resource provider:

az provider register -n Microsoft.ContainerService --wait

Create an Azure Active Directory (Azure AD) tenant

The Azure Red Hat OpenShift service requires an associated Azure Active Directory (Azure AD ) tenant that
represents your organization and its relationship to Microsoft. Your Azure AD tenant enables you to register, build,
and manage apps, as well as use other Azure services.
If you don't have an Azure AD to use as the tenant for your Azure Red Hat OpenShift cluster, or you wish to create
a tenant for testing, follow the instructions in Create an Azure AD tenant for your Azure Red Hat OpenShift cluster
before continuing with this guide.

Create an Azure AD user, security group and application object

Azure Red Hat OpenShift requires permissions to perform tasks on your cluster, such as configuring storage.
These permissions are represented through a service principal. You'll also want to create a new Active Directory
user for testing apps running on your Azure Red Hat OpenShift cluster.
Follow the instructions in Create an Azure AD app object and user to create a service principal, generate a client
secret and authentication callback URL for your app, and create a new Azure AD security group and user to access
the cluster.

Next steps
You're now ready to use Azure Red Hat OpenShift!
Try the tutorial:
Create an Azure Red Hat OpenShift cluster
 Create an Azure AD tenant for Azure Red Hat
OpenShift
5/29/2019 • 2 minutes to read • Edit Online

Microsoft Azure Red Hat OpenShift requires an Azure Active Directory (Azure AD ) tenant in which to create your
cluster. A tenant is a dedicated instance of Azure AD that an organization or app developer receives when they
create a relationship with Microsoft by signing up for Azure, Microsoft Intune, or Microsoft 365. Each Azure AD
tenant is distinct and separate from other Azure AD tenants and has its own work and school identities and app
registrations.
If you don't already have an Azure AD tenant, follow these instructions to create one.

Create a new Azure AD tenant

To create a tenant:
1. Sign in to the Azure portal using the account you wish to associate with your Azure Red Hat OpenShift
cluster.
2. Open the Azure Active Directory blade to create a new tenant (also known as a new Azure Active Directory).
3. Provide an Organization name.
4. Provide an Initial domain name. This will have onmicrosoft.com appended to it. You can reuse the value
for Organization name here.
5. Choose a country or region where the tenant will be created.
6. Click Create.
7. After your Azure AD tenant is created, select the Click here to manage your new directory link. Your
new tenant name should be displayed in the upper-right of the Azure portal:

8. Make note of the tenant ID so you can later specify where to create your Azure Red Hat OpenShift cluster.
In the portal, you should now see the Azure Active Directory overview blade for your new tenant. Select
Properties and copy the value for your Directory ID. We will refer to this value as TENANT in the Create an
Azure Red Hat OpenShift cluster tutorial.

Resources
Check out Azure Active Directory documentation for more info on Azure AD tenants.

Next steps
Learn how to create a service principal, generate a client secret and authentication callback URL, and create a new
Active Directory user for testing apps on your Azure Red Hat OpenShift cluster.
Create an Azure AD app object and user
 Azure Active Directory integration for Azure Red Hat
OpenShift
7/9/2019 • 4 minutes to read • Edit Online

If you haven't already created an Azure Active Directory (Azure AD ) tenant, follow the directions in Create an
Azure AD tenant for Azure Red Hat OpenShift before continuing with these instructions.
Microsoft Azure Red Hat OpenShift needs permissions to perform tasks on behalf of your cluster. If your
organization doesn't already have an Azure AD user, Azure AD security group, or an Azure AD app registration to
use as the service principal, follow these instructions to create them.

Create a new Azure Active Directory user

In the Azure portal, ensure that your tenant appears under your user name in the top right of the portal:

If the wrong tenant is displayed, click your user name in the top right, then click Switch Directory, and select the
correct tenant from the All Directories list.
Create a new Azure Active Directory global administrator user to sign in to your Azure Red Hat OpenShift cluster.
1. Go to the Users-All users blade.
2. Click +New user to open the User pane.
3. Enter a Name for this user.
4. Create a User name based on the name of the tenant you created, with .onmicrosoft.com appended at the end.
For example, yourUserName@yourTenantName.onmicrosoft.com . Write down this user name. You'll need it to sign in
to your cluster.
5. Click Directory role to open the directory role pane, and select Global administrator and then click Ok at the
bottom of the pane.
6. In the User pane, click Show Password and record the temporary password. After you sign in the first time,
you'll be prompted to reset it.
7. At the bottom of the pane, click Create to create the user.

Create an Azure AD security group

To grant cluster admin access, the memberships in an Azure AD security group are synced into the OpenShift
group "osa-customer-admins". If not specified, no cluster admin access will be granted.
1. Open the Azure Active Directory groups blade.
2. Click +New Group.
3. Provide a group name and description.
4. Set Group type to Security.
5. Set Membership type to Assigned.
Add the Azure AD user that you created in the earlier step to this security group.
 6. Click Members to open the Select members pane.
7. In the members list, select the Azure AD user that you created above.
8. At the bottom of the portal, click on Select and then Create to create the security group.
Write down the Group ID value.
9. When the group is created, you will see it in the list of all groups. Click on the new group.
10. On the page that appears, copy down the Object ID. We will refer to this value as GROUPID in the Create an
Azure Red Hat OpenShift cluster tutorial.

Create an Azure AD app registration

You can automatically create an Azure Active Directory (Azure AD ) app registration client as part of creating the
cluster by omitting the --aad-client-app-id flag to the az openshift create command. This tutorial shows you
how to create the Azure AD app registration for completeness.
If your organization doesn't already have an Azure Active Directory (Azure AD ) app registration to use as a service
principal, follow these instructions to create one.
1. Open the App registrations blade and click +New registration.
2. In the Register an application pane, enter a name for your application registration.
3. Ensure that under Supported account types that Accounts in this organizational directory only is
selected. This is the most secure choice.
4. We will add a redirect URI later once we know the URI of the cluster. Click the Register button to create the
Azure AD application registration.
5. On the page that appears, copy down the Application (client) ID. We will refer to this value as APPID in the
Create an Azure Red Hat OpenShift cluster tutorial.

Create a client secret

Generate a client secret for authenticating your app to Azure Active Directory.
1. In the Manage section of the app registrations page, click Certificates & secrets.
2. On the Certificates & secrets pane, click +New client secret. The Add a client secret pane appears.
3. Provide a Description.
4. Set Expires to the duration you prefer, for example In 2 Years.
5. Click Add and the key value will appear in the Client secrets section of the page.
6. Copy down the key value. We will refer to this value as SECRET in the Create an Azure Red Hat OpenShift
cluster tutorial.
For more information about Azure Application Objects, see Application and service principal objects in Azure
Active Directory.
For details on creating a new Azure AD application, see Register an app with the Azure Active Directory v1.0
endpoint.

Add API permissions

1. In the Manage section click API permissions.
2. Click Add permission and select Azure Active Directory Graph then Delegated permissions
3. Expand User on the list below and make sure User.Read is enabled.
4. Scroll up and select Application permissions.
5. Expand Directory on the list below and enable Directory.ReadAll
6. Click Add permissions to accept the changes.
7. The API permissions panel should now show both User.Read and Directory.ReadAll. Please note the warning in
Admin consent required column next to Directory.ReadAll.
8. If you are the Azure Subscription Administrator, click Grant admin consent for Subscription Name below. If
you are not the Azure Subscription Administrator, request the consent from your administrator.

IMPORTANT
Synchronization of the cluster administrators group will work only after consent has been granted. You will see a green circle
with a checkmark and a message "Granted for Subscription Name" in the Admin consent required column.

For details on managing administrators and other roles, see Add or change Azure subscription administrators.
Resources
Applications and service principal objects in Azure Active Directory
Quickstart: Register an app with the Azure Active Directory v1.0 endpoint

Next steps
If you've met all the Azure Red Hat OpenShift prerequisites, you're ready to create your first cluster!
Try the tutorial:
Create an Azure Red Hat OpenShift cluster
 Deploy a standalone Prometheus instance in an
Azure Red Hat OpenShift cluster
7/11/2019 • 4 minutes to read • Edit Online

This article describes how to configure a standalone Prometheus instance that uses service discovery in an Azure
Red Hat OpenShift cluster.

NOTE
Customer admin access to Azure Red Hat OpenShift cluster isn't required.

Target setup:
One project (prometheus-project), which contains Prometheus and Alertmanager.
Two projects (app-project1 and app-project2), which contain the applications to monitor.
You'll prepare some Prometheus config files locally. Create a new folder to store them. Config files are stored in the
cluster as secrets, in case secret tokens are added later to the cluster.

Sign in to the cluster by using the OC tool

1. Open a web browser, and then go to the web console of your cluster (https://openshift.random -
id.region.azmosa.io).
2. Sign in with your Azure credentials.
3. Select your username in the upper-right corner, and then select Copy Login Command.
4. Paste your username into the terminal that you'll use.

NOTE
To see if you're signed in to the correct cluster, run the oc whoami -c command.

Prepare the projects

To create the projects, run the following commands:

oc new-project prometheus-project
oc new-project app-project1
oc new-project app-project2

NOTE
You can either use the -n or --namespace parameter, or select an active project by running the oc project command.

Prepare the Prometheus configuration file

Create a prometheus.yml file by entering the following content:
 global:
scrape_interval: 30s
evaluation_interval: 5s

scrape_configs:
- job_name: prom-sd
scrape_interval: 30s
scrape_timeout: 10s
metrics_path: /metrics
scheme: http
kubernetes_sd_configs:
- api_server: null
role: endpoints
namespaces:
names:
- prometheus-project
- app-project1
- app-project2

Create a secret called Prom by entering the following configuration:

oc create secret generic prom --from-file=prometheus.yml -n prometheus-project

The prometheus.yml file is a basic Prometheus configuration file. It sets the intervals and configures auto discovery
in three projects (prometheus-project, app-project1, app-project2). In the previous configuration file, the auto-
discovered endpoints are scraped over HTTP without authentication.
For more information about scraping endpoints, see Prometheus scape config.

Prepare the Alertmanager config file

Create an alertmanager.yml file by entering the following content:

global:
resolve_timeout: 5m
route:
group_wait: 30s
group_interval: 5m
repeat_interval: 12h
receiver: default
routes:
- match:
alertname: DeadMansSwitch
repeat_interval: 5m
receiver: deadmansswitch
receivers:
- name: default
- name: deadmansswitch

Create a secret called Prom-Alerts by entering the following configuration:

oc create secret generic prom-alerts --from-file=alertmanager.yml -n prometheus-project

Alertmanager.yml is the Alert Manager configuration file.

 NOTE
To verify the two previous steps, run the oc get secret -n prometheus-project command.

Start Prometheus and Alertmanager

Go to openshift/origin repository and download the prometheus-standalone.yaml template. Apply the template to
prometheus-project by entering the following configuration:

oc process -f https://raw.githubusercontent.com/openshift/origin/release-3.11/examples/prometheus/prometheus-
standalone.yaml | oc apply -f - -n prometheus-project

The prometheus-standalone.yaml file is an OpenShift template. It will create a Prometheus instance with oauth-
proxy in front of it and an Alertmanager instance, also secured with oauth-proxy. In this template, oauth-proxy is
configured to allow any user who can "get" the prometheus-project namespace (see the -openshift-sar flag).

NOTE
To verify if the prom StatefulSet has equal DESIRED and CURRENT number replicas, run the
oc get statefulset -n prometheus-project command. To check all resources in the project, run the
oc get all -n prometheus-project command.

Add permissions to allow service discovery

Create a prometheus-sdrole.yml file by entering the following content:
 apiVersion: template.openshift.io/v1
kind: Template
metadata:
name: prometheus-sdrole
annotations:
"openshift.io/display-name": Prometheus service discovery role
description: |
Role and rolebinding added permissions required for service discovery in a given project.
iconClass: fa fa-cogs
tags: "monitoring,prometheus,alertmanager,time-series"
parameters:
- description: The project name, where a standalone Prometheus is deployed
name: PROMETHEUS_PROJECT
value: prometheus-project
objects:
- apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: prometheus-sd
rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- pods
verbs:
- list
- get
- watch
- apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: prometheus-sd
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: prometheus-sd
subjects:
- kind: ServiceAccount
name: prom
namespace: ${PROMETHEUS_PROJECT}

To apply template to all the projects that you want allow service discovery, run the following commands:

oc process -f prometheus-sdrole.yml | oc apply -f - -n app-project1

oc process -f prometheus-sdrole.yml | oc apply -f - -n app-project2

To have Prometheus to gather metrics from itself, apply the permissions in prometheus-project.

NOTE
To verify that Role and RoleBinding were created correctly, run the oc get role and oc get rolebinding commands.

Optional: Deploy example application

Everything is working, but there are no metrics sources. Go to the Prometheus URL (https://prom-prometheus-
project.apps.random -id.region.azmosa.io/). You can find it by using following command:
 oc get route prom -n prometheus-project

IMPORTANT
Remember to add the https:// prefix to beginning of the host name.

The Status > Service Discovery page will show 0/0 active targets.
To deploy an example application, which exposes basic Python metrics under the /metrics endpoint, run the
following commands:

oc new-app python:3.6~https://github.com/Makdaam/prometheus-example --name=example1 -n app-project1

oc new-app python:3.6~https://github.com/Makdaam/prometheus-example --name=example2 -n app-project2

The new applications should appear as valid targets on the Service Discovery page within 30 seconds after
deployment.
For more details, select Status > Targets.

NOTE
For every successfully scraped target, Prometheus adds a data point in the up metric. Select Prometheus in the upper-left
corner, enter up as the expression, and then select Execute.

Next steps
You can add custom Prometheus instrumentation to your applications. The Prometheus Client library, which
simplifies Prometheus metrics preparation, is ready for different programming languages.
For more information, see the following GitHub libraries:
Java
Python
Go
Ruby
 Azure Red Hat OpenShift FAQ
6/13/2019 • 2 minutes to read • Edit Online

This article addresses frequently asked questions (FAQs) about Microsoft Azure Red Hat OpenShift.

How do I get started?

Before you can use Azure Red Hat OpenShift, you'll need to purchase a minimum of 4 Azure Red Hat OpenShift
reserved application nodes.
If you are an Azure customer,purchase Azure Red Hat OpenShift reserved instances through the Azure portal.
After purchasing, your subscription will be activated within 24 hours, after which you'll be able to provision
clusters.
If you are not an Azure customer, contact sales and fill out the sales form at the bottom of the page to start the
process.
Refer to the Azure Red Hat OpenShift pricing page for more information.

Which Azure regions are supported?

See Supported resources for a list of global regions where Azure Red Hat OpenShift is supported.

Can I deploy a cluster into an existing virtual network?

No. But you can connect an Azure Red Hat OpenShift cluster to an existing VNET via peering. See Connect a
cluster's virtual network to an existing virtual network for details.

What cluster operations are available?

You can only scale up or down the number of compute nodes. No other modifications are permitted to the
Microsoft.ContainerService/openShiftManagedClusters resource after creation. The maximum number of compute
nodes is limited to 20.

What virtual machine sizes can I use?

See Azure Red Hat OpenShift virtual machine sizes for a list of virtual machine sizes you can use with an Azure
Red Hat OpenShift cluster.

Is data on my cluster encrypted?

By default, there is encryption at rest. The Azure Storage platform automatically encrypts your data before
persisting it, and decrypts the data before retrieval. See Azure Storage Service Encryption for data at rest for
details.

Can I use Prometheus/Grafana to monitor my applications?

Yes, you can deploy Prometheus in your namespace and monitor applications in your namespace.

Can I use Prometheus/Grafana to monitor metrics related to cluster

health and capacity?
No, not at current time.

Is the Docker registry available externally so I can use tools such as

Jenkins?
The Docker registry is available from https://docker-registry.apps.<clustername>.<region>.azmosa.io/ However, a
strong storage durability guarantee is not provided. You can also use Azure Container Registry.

Is cross-namespace networking supported?

Customer and individual project admins can customize cross-namespace networking (including denying it) on a
per project basis using NetworkPolicy objects.

Can an admin manage users and quotas?

Yes. An Azure Red Hat OpenShift administrator can manage users and quotas in addition to accessing all user
created projects.

Can I restrict a cluster to only certain Azure AD users?

Yes. You can restrict which Azure AD users can sign in to a cluster by configuring the Azure AD Application. For
details, see How to: Restrict your app to a set of users

Can a cluster have compute nodes across multiple Azure regions?

No. All nodes in an Azure Red Hat OpenShift cluster must originate from the same Azure region.

Are master and infrastructure nodes abstracted away as they are with
Azure Kubernetes Service (AKS)?
No. All resources, including the cluster master, run in your customer subscription. These types of resources are put
in a read-only resource group.

Is Open Service Broker for Azure (OSBA) supported?

Yes. You can use OSBA with Azure Red Hat OpenShift. See Open Service Broker for Azure for more information.

I am trying to peer into a virtual network in a different subscription but

getting Failed to get vnet CIDR error.
In the subscription that has the virtual network, make sure to register Microsoft.ContainerService provider with
az provider register -n Microsoft.ContainerService --wait
 Troubleshooting for Azure Red Hat OpenShift
5/29/2019 • 2 minutes to read • Edit Online

This article details some common issues encountered while creating or managing Microsoft Azure Red Hat
OpenShift clusters.

Retrying the creation of a failed cluster

If creating an Azure Red Hat OpenShift cluster using the az CLI command fails, retrying the create will continue
to fail. Use az openshift delete to delete the failed cluster, then create an entirely new cluster.

Hidden Azure Red Hat OpenShift cluster resource group

Currently, the Microsoft.ContainerService/openShiftManagedClusters resource that's automatically created by the
Azure CLI ( az openshift create command) is hidden in the Azure portal. In the Resource group view, check
Show hidden types to view the resource group.

Creating a cluster results in error that no registered resource provider

found
If creating a cluster results in an error that
No registered resource provider found for location '<location>' and API version '2019-04-30' for type
'openShiftManagedClusters'. The supported api-versions are '2018-09-30-preview
, then you were part of the preview and now need to purchase Azure virtual machine reserved instances to use the
generally available product. A reservation reduces your spend by pre-paying for fully managed Azure services.
Refer to What are Azure Reservations to learn more about reservations and how they save you money.

Next steps
Try the Red Hat OpenShift Help Center for more on OpenShift troubleshooting.
Find answers to frequently asked questions about Azure Red Hat OpenShift.
 Azure Red Hat OpenShift resources
5/29/2019 • 2 minutes to read • Edit Online

This topic lists the Azure regions and virtual machine sizes supported by the Microsoft Azure Red Hat OpenShift
service.

Azure regions
See Products available by region for a current list of regions where you can deploy Azure Red Hat OpenShift
clusters.

Virtual machine sizes

Here are the supported virtual machine sizes you can specify for the compute nodes in your Azure Red Hat
OpenShift cluster.

IMPORTANT
Each VM has a different number of drives that can be attached. This may not be as immediately clear as memory or CPU size.
Not all VM sizes are available in all regions. Even if the API supports the size you specify, you might get an error if the size is
not available in the region you specify. See Current list of supported VM sizes per region for more information.

Compute node sizes

The following compute node sizes are supported by the Azure Red Hat OpenShift REST API:

SIZE VCPU RAM

Standard D4s v3 4 16 GB

Standard D8s v3 8 32 GB

Standard D16s v3 16 64 GB

Standard D32s v3 32 128 GB

- - -

Standard E4s v3 4 32 GB

Standard E8s v3 8 64 GB

Standard E16s v3 16 128 GB

Standard E32s v3 32 256 GB

- - -

Standard F8s v2 8 16 GB
 SIZE VCPU RAM

Standard F16s v2 16 32 GB

Standard F32s v2 32 64 GB

Master node sizes

The following master / infrastructure node sizes are supported by the Azure Red Hat OpenShift REST API:

SIZE VCPU RAM

Standard D4s v3 4 16 GB

Standard D8s v3 8 32 GB

Standard D16s v3 16 64 GB

Standard D32s v3 32 128 GB

Next steps
Try the Create a Azure Red Hat OpenShift cluster tutorial.