Tag: devops

Posted on by — Leave a comment

Azure DevOps Terraform Provider

If you work everywhere as a code you will probably need to check Azure DevOps terraform provider. It is created and maintained from Microsoft and you can use it in order to have your DevOps tool as a code.

https://registry.terraform.io/providers/microsoft/azuredevops/latest/docs

In order to getting started you will need to create a PAT token and give it the access based on the actions that you need to do.

When the token is ready you will need to set two environmental variables on the machine that you work. The first one is AZDO_PERSONAL_ACCESS_TOKEN which should be your token. The second one will be your org URL AZDO_ORG_SERVICE_URL

export AZDO_PERSONAL_ACCESS_TOKEN= TOKEN
export AZDO_ORG_SERVICE_URL= https://dev.azure.com/geralexgr

Finally you are ready to deploy your IAC Azure DevOps configurations.

Lets see the below example.

# Make sure to set the following environment variables:
#   AZDO_PERSONAL_ACCESS_TOKEN
#   AZDO_ORG_SERVICE_URL
terraform {
  required_providers {
    azuredevops = {
      source = "microsoft/azuredevops"
      version = ">=0.1.0"
    }
  }
}

resource "azuredevops_project" "project" {
  name = "My Awesome Project"
  description  = "All of my awesomee things"
}

resource "azuredevops_git_repository" "repository" {
  project_id = azuredevops_project.project.id
  name       = "My Awesome Repo"
  initialization {
    init_type = "Clean"
  }
}

resource "azuredevops_build_definition" "build_definition" {
  project_id = azuredevops_project.project.id
  name       = "My Awesome Build Pipeline"
  path       = "\\"

  repository {
    repo_type   = "TfsGit"
    repo_id     = azuredevops_git_repository.repository.id
    branch_name = azuredevops_git_repository.repository.default_branch
    yml_path    = "azure-pipelines.yml"
  }
}

When above code runs it will create a new project with the name My Awesome Project. Inside the project a new git repo will be initialized and a new pipeline will be created inside this repository.

You can find the usage example below.

https://github.com/microsoft/terraform-provider-azuredevops

Youtube video:

Posted on by — Leave a comment

Tasks, jobs, stages templates combined for Azure DevOps

In this article we will examine the power of templates for Azure DevOps pipelines. Templates let you define reusable content, logic, and parameters and function in two ways. You can insert reusable content with a template or you can use a template to control what is allowed in a pipeline.

When you build complex automation scenarios for your organizations, you will need to use stages, jobs and tasks as they would contain multiple environments and configuration settings.

You can find some of the reasons why you would need to follow this approach on my previous article

Azure DevOps best practices – jobs and stages

In this article we will examine a azure devops pipeline which contains stages, jobs and tasks. Those will be created inside templates and they will be called from the main pipeline. A high level view of the architecture can be found in the below picture.

My code structure is shown below. There is a folder for the appropriate templates and a main pipeline which is located in another folder and will refer the templates folder.

stage.yml
The stage.yml file will contain the template code for the stages. It has as parameter the name which will be given in the stage.

parameters:
  name: ''

stages:

- stage: ${{ parameters.name }}
  jobs:    
  - template: job.yml
    parameters:
      name: ${{ parameters.name }}_build_job

job.yml
The job.yml file will contain the template code for the jobs. It has as parameter the name which will be given in the job and also a variable sign which will indicate if a task will be executed.

parameters:
  name: ''
  sign: false

jobs:

- job: ${{ parameters.name }}
  displayName: running ${{ parameters.name }} 
  steps:

  - template: step.yml
    parameters:
      name: task1

  - ${{ if eq(parameters.sign, 'true') }}:
    - script: echo sign is requested
      displayName: sign task

step.yml
The step.yml file will contain the template code for the steps. It has as parameter the name which will be given in the task. 

parameters:
  name: ''

steps:

- script: echo ${{ parameters.name }}
  displayName: running  ${{ parameters.name }}

main.yml
The main.yml file is the main reference of the pipeline and the one that will be called. If you need to add more stages on it, you would only have to add another -template section under the stages.

trigger:
- none

variables:
- template: templates/vars.yml  
pool:
  vmImage: $(myagent)

stages:
- template: templates/stage.yml  
  parameters:
    name: "App_Env1"

By executing the pipeline we can locate that we have one stage that is not visible (as it is the only one) and under this stage a job has been created for the task1 which we added on our template.

Find more about azure devops templates on my Udemy course:

Mastering Azure Devops CI/CD Pipelines with YAML | Udemy

Posted on by — Leave a comment

Scan azure devops repositories for credentials and passwords

DevSecOps practices are important for organizations especially when it comes to code repositories. Your code should avoid hard coded passwords and secrets for many reasons as a leak may occur. In this guide I will examine how you can massively scan Azure DevOps repositories for security leaks as passwords and secrets with gitleaks utility.

https://github.com/gitleaks/gitleaks

Simon has provided a very useful script that you can use in order to download all your repositories from Azure DevOps.

Cloning all repositories from Azure DevOps using Azure CLI – Simon Wahlin

When you execute the script, all the repositories will be downloaded inside your project folder.

Then you will need to install gitleaks and execute for each repository.

$folder_for_cleanup = "C:\Users\geralexgr\Documents\AzureRepos"
Get-ChildItem $folder_for_cleanup | Sort -Property FullName | ForEach-Object {
                gitleaks detect -s $_.FullName -v >> gitleaks-results.txt
                echo "######################################################################################################" >> gitleaks-results.txt
            }

The scan will go through each repository and scan for leaks. The output will be stored in gitleaks-result text file.

Posted on by — Leave a comment

Automatic rollback procedure for Azure DevOps

Azure devops pipelines provide a variety of tools for automated procedures. One mechanism that administrators can build using the YAML structure is an automated rollback mechanism during a deployment.

This means that after a deployment you can revert the previous state using your YAML tasks without having to redeploy. Another case would be a broken deployment which can be identified by monitoring tools and then a validation could approve or not the final release. This is exactly depicted in the below image. After releasing a version we have a validation step that requires manual approval from an administrator. If the validation is approved the release will proceed else the rollback will be triggered.

This mechanism is described below with YAML. Release stage includes release, validation and rollback jobs. Release job performs the actual release. Validation will depend on release job and will continue only if is approved. The rollback job will run only if validation failed which means that an administrator canceled the approval.

trigger: none
pr: none

stages:

- stage: releaseStage
  jobs:

  - deployment: release
    displayName: Release
    environment:
      name: dev
      resourceType: VirtualMachine
    strategy:
      runOnce:
        deploy:
          steps:
            - task: PowerShell@2
              displayName: hostname
              inputs:
                targetType: 'inline'
                script: |
                    deployment script here...
  
  - job: validation
    dependsOn: release
    pool: server
    steps:
    - task: ManualValidation@0
      inputs:
        notifyUsers: 'admin@domain.com'
        instructions: 'continue?'
        onTimeout: reject

  - deployment: rollback
    displayName: rollback
    dependsOn: validation
    condition: failed()
    environment:
      name: dev
      resourceType: VirtualMachine
    strategy:
      runOnce:
        deploy:
          steps:
            - task: PowerShell@2
              displayName: rolling back
              inputs:
                targetType: 'inline'
                script: |
                    rollback script here..
                    Write-Host "rollback"

When the release can be verified from the administrator the rollback will be skipped. This is the case when the validation is approved from the user.

Validation task will ask the user for a review.

On the other hand if validation is rejected the rollback stage will run.