Microsoft Azure

Pulkit Gulati

Automate ARM templates deployment with Azure DevOps pipelines

In continuation to my previous post, I’m deploying the Hub and Spoke model using the ARM templates from Azure Repo using Azure pipeline. For this example, I’m only showing 2 ARM templates deployment for Log Analytics and Hub Vnet. The rest can be done in the same way.

Create a Service Connection in your Azure Project as shown below:

Below is the yaml file that I’m using to create multi-stage pipeline, where each stage is running a job:

name: $(BuildDefinitionName)_$(date:yyyyMMdd)$(rev:.r)
 
trigger: none
 
pr: none
 
stages :        
  - stage: arm_loganalytics_deploy
    jobs:
      - job: arm_loganalytics_deploy
        steps:
              - checkout: self
 
              - task: AzureResourceManagerTemplateDeployment@3
                inputs:
                  deploymentScope: 'Resource Group'
                  azureResourceManagerConnection: 'AzureVSE'
                  subscriptionId: '1bcd68af-e392-4b66-9558-697bd7e8dc91'
                  action: 'Create Or Update Resource Group'
                  resourceGroupName: 'azhubspoke-rg'
                  location: 'Japan East'
                  templateLocation: 'Linked artifact'
                  csmFile: '$(System.DefaultWorkingDirectory)/loganalytics-workbook/loganalytics.json'
                  deploymentMode: 'Incremental'
  
  - stage: arm_hubvnet_deploy
    jobs:
      - job: arm_hubvnet_deploy
        steps:
              - checkout: self

              - task: AzureResourceManagerTemplateDeployment@3
                inputs:
                  deploymentScope: 'Resource Group'
                  azureResourceManagerConnection: 'AzureVSE'
                  subscriptionId: '1bcd68af-e392-4b66-9558-697bd7e8dc91'
                  action: 'Create Or Update Resource Group'
                  resourceGroupName: 'azhubspoke-rg'
                  location: 'Japan East'
                  templateLocation: 'Linked artifact'
                  csmFile: '$(System.DefaultWorkingDirectory)/hub-vnet/hub-vnet.json'
                  deploymentMode: 'Incremental'

You can also validate the parameters of ARM template using the following parameter below csmFile in yaml file e.g.:

csmParametersFile: '$(System.DefaultWorkingDirectory)/hub-vnet/hub-vnet.parameters.json'

Run the pipeline and verify the results.

Pulkit Gulati

Create Azure Hub and Spoke model using ARM templates

The Hub and Spoke model is a popular Architecture for Teams who are migrating their Workloads to Cloud environment incrementally and still keep some workloads on-prem. Following are the main components that make-up Hub and Spoke model:

  1. Hub Virtual Network that holds your common components like VPN or Express Route Gateway, Azure Firewall, Azure Bastion Host etc. These components can be common to different environments like Dev, Staging, Prod etc. for better cost management.
  2. Spoke Virtual Network which have isolated workloads. These can hold VMs or other PaaS services like App Service that connect to On-Prem network via the Hub network gateway transit. There can be any number of Spokes.
Example Hub and Spoke Architecture

The benefits of hub and spoke configuration include cost savings, overcoming subscription limits and workload isolation.

Another example from the MS docs I found useful is as shown below:

I’m going to create the architecture shown above using ARM templates. You can find the templates for different components here. I’ve broken down the combined template into multiple templates that can be run in the following order by deploying them as Custom templates in Azure Portal and will be created in an existing Resource Group:

  1. Log Analytics workbook
  2. Hub (includes vpn gateway, firewall, bastion)
  3. Spoke1
  4. Spoke2
  5. Vnet Peerings
  6. Azure Sentinel
  7. Azure KeyVault

MS docs URL shared above contains more details about these components. Breaking down the templates into separate components gives you more control in creating a automated flow using say Azure pipelines. You can also later add or remove components easily as per your requirement.

Other options can be using Azure Blueprints for creating a minimal architecture and building from there on.

A similar architecture can created using Terraform as per MS docs here. But while running Terraform seems to be erroneous in Cloud shell and sometimes becomes unresponsive as per my experience.

Pulkit Gulati

Change ApplicationInsights Azure resource configuration in existing Web App

Application Insights is a Service on Microsoft Azure that lets you understand what users are actually doing on your App.
It also lets you diagnose any issues with it’s Powerful analytics tools and works with platforms including .Net, Java and Node.js.

The App Insights Instrumentation key is what is required to link your App with the resource on Azure.
If you already have an existing App Insights resource created through Visual Studio and you need to change it, then you can create another resource manually from the Azure Portal.

Once the App Insights resource is created, copy the Instrumentation key and replace it in your ApplicationInsights.config file. This lets you switch the ApplicationInsights resource for your Application.

Look for the InstrumentationKey tag in your ApplicationInsights.config file and replace. You might also need to change the InstrumentationKey in the HomePage JavaScript under Views folder added by App Insights SDK.

Start debugging your App and verify with your Live Metrics Stream in the App Insights resource that it is working.

Pulkit Gulati

Accessing the Ubuntu vm created on Azure via vnc server on Mac

Currently I’ve setup the Ubuntu Server 18.04 LTS from the Azure marketplace and I’m trying to access it via VNC Server setup on the Linux machine. Also, you’ll need a vnc client like RealVNC or you can also use the screen-sharing client available on your Mac.

Login via SSH:

First you need to login to your Linux VM as a non-root user which you’ve created while setting up the VM. To spin up a new Linux VM, you can check out this post. You can use the Cloud shell to connect to your VM using the non-root username and password to your machine via SSH. Use the Connect menu of your VM and copy the SSH command to run in the Cloud shell.

ssh your_user_name@IP_Address

You just need to replace the your_user_name and IP_Address parts in the above command. Enter the password you’re prompted for to complete the Login as SSH.

Install the required packages:

We now need to install the required packages like Xfce desktop environment and VNC Server which are not bundled in the Ubuntu OS by default. Xfce is a free and open-source desktop environment for Unix and Unix like Operating Systems.

Update list of packages:
$ sudo apt update
Install Xfce Desktop environment and wait for the installation to complete:
$ sudo apt install xfce4 xfce4-goodies
Install the VNC Server:

$ sudo apt install tightvncserver

Complete the initial configuration and provide the setup password:
$ vncserver

Providing a view-only password is optional. You’ll get the below Output as the initial configuration completes:

Creating default startup script /home/your_user_name/.vnc/xstartup Starting applications specified in /home/your_user_name/.vnc/xstartup Log file is /home/your_user_name/.vnc/your_hostname:1.log

Configure VNC Server:

The VNC Server is by default configured on the port 5901 and display port :1. VNC can launch multiple instances on other ports like :2, :3 and so on.

Let’s first kill the current instance for further configuration that we require:

$ vncserver -kill :1

Output:

Killing Xtightvnc process ID <ID>

Backup the xstartup file before modifying:

$ mv ~/.vnc/xstartup ~/.vnc/xstartup.bak

Create a new xstartup file and open in editor:

$ nano ~/.vnc/xstartup

Add the following lines to your file in the nano editor and save it:

~/.vnc/xstartup
#!/bin/bash
xrdb $HOME/.Xresources
startxfce4 &

This is making certain settings to the graphical desktop like colours, themes and fonts. The last line is starting the Xfce desktop. Now, let’s convert the file to an executable and restart:

$ sudo chmod +x ~/.vnc/xstartup
$ vncserver

Now, let’s connect to the VNC Server from your Mac by creating a SSH tunnel and use Screen-sharing client to connect.

Run this command on your Mac terminal:

$ ssh -L 5901:127.0.0.1:5901 -C -N -l your_user_name your_server_ip

Do replace the your_user_name with your sudo non-root username and your_server_ip with the IP Address of your Linux VM. Provide the password when prompted for your username.

Now, open your screen sharing App available in the Finder Go Menu on your Mac that says “Connect to Server…”.

Click on Connect and provide your password when prompted again and you’ll see the Xfce Desktop running via Screen-sharing.

Pulkit Gulati

Spin up Linux VM on Microsoft Azure

Microsoft Azure provides a multiple ways to create Virtual Machines for Windows or Linux along with their multiple market variations. They are:

  1. Azure portal UI at https://portal.azure.com
  2. Azure CLI
  3. Azure PowerShell commands

I’ll be using the Azure portal UI to create a marketplace image for Ubuntu Linux. Some of the UI features may change in future, but the crux will remain pretty much similar more or less.

Open Marketplace for VM Images on Azure portal

Marketplace image

Fill up the VM Image details

Create Image1

  1. Create or select Resource Group.
  2. Give a suitable name.
  3. Select region based on your geographic availability.
  4. For personal use redundancy is not required. You can change Availability options based on Availability Zone or Availability set.
  5. Select the Marketplace image for the available Ubuntu version.
  6. Select a machine size based on vcpus, memory and IOPS requirement. Of course, check the cost factor.


Setup Authentication using Password or SSH public key

You can simply use Username and Password for authentication otherwise use SSH public/private key pair.

For generating SSH public key, use Putty gen for Windows or ssh-keygen on Linux and OSX. You can download a suitable Putty client for windows here.

  1. Generate RSA 2048-bit key and follow the instructions by the tool.
  2. Save the Private key file as .ppk
  3. Save the Public key file as .pub
  4. Export the Private key file as .openssh format using the Conversions menu if this key will be used by an external SSH client such as on Linux.

PuttyKeygen

For the Admin account, put a suitable Username and SSH public key generated above starting with “ssh-rsa” as shown below. Make sure the key is copied as is without any modifications.

generateSshAdmin


Add Disks information

It’s better to use Premium SSD for optimal performance. If you have additional disks already created you can attach it with the VM at this step or you can also do this later.

Create Image2

Networking

This step creates a Virtual Network, a subnet and a Public IP for the VM. All these are added to the same Resource Group while creating these new resources. You can also select if you have these existing resources.

Create Image3


Allow Inbound Ports

You can select the required ports e.g. SSH for connecting using SSH public/private key-pair or RDP to connect using Username and Password.

inbound ports

Management

Keep these to default if you prefer. You can enable/disable any option based on requirement. I turned off Boot diagnostics as it required to create a Storage account so I switched it off, as I don’t require it for a test VM.

Create Image4

Guest Config

You can provide additional post-deployment configuration options using extensions like chef and puppet or Cloud init for Linux.

Create Image5

Tags

You can add various tags to categorize resources for consolidated billing and automation management.

Create Image6

Review your provided details in the next step and click on Create. Wait for the deployment to succeed.

Accessing the VM

For accessing the VM, check if you have inbound port rules set up to access using Public IP address with RDP or SSH. Use Putty configuration client to SSH into the VM using port 22 on Windows machine. From a Unix like system including MacOS, use the following command:

ssh <username>@<computer name or IP address>

For details on how to connect to your Ubuntu Linux VM from your Mac machine, check out this post.