Tag: devops

Posted on by — Leave a comment

Scan azure devops repositories for credentials and passwords

DevSecOps practices are important for organizations especially when it comes to code repositories. Your code should avoid hard coded passwords and secrets for many reasons as a leak may occur. In this guide I will examine how you can massively scan Azure DevOps repositories for security leaks as passwords and secrets with gitleaks utility.

https://github.com/gitleaks/gitleaks

Simon has provided a very useful script that you can use in order to download all your repositories from Azure DevOps.

Cloning all repositories from Azure DevOps using Azure CLI – Simon Wahlin

When you execute the script, all the repositories will be downloaded inside your project folder.

Then you will need to install gitleaks and execute for each repository.

$folder_for_cleanup = "C:\Users\geralexgr\Documents\AzureRepos"
Get-ChildItem $folder_for_cleanup | Sort -Property FullName | ForEach-Object {
                gitleaks detect -s $_.FullName -v >> gitleaks-results.txt
                echo "######################################################################################################" >> gitleaks-results.txt
            }

The scan will go through each repository and scan for leaks. The output will be stored in gitleaks-result text file.

Posted on by — Leave a comment

Automatic rollback procedure for Azure DevOps

Azure devops pipelines provide a variety of tools for automated procedures. One mechanism that administrators can build using the YAML structure is an automated rollback mechanism during a deployment.

This means that after a deployment you can revert the previous state using your YAML tasks without having to redeploy. Another case would be a broken deployment which can be identified by monitoring tools and then a validation could approve or not the final release. This is exactly depicted in the below image. After releasing a version we have a validation step that requires manual approval from an administrator. If the validation is approved the release will proceed else the rollback will be triggered.

This mechanism is described below with YAML. Release stage includes release, validation and rollback jobs. Release job performs the actual release. Validation will depend on release job and will continue only if is approved. The rollback job will run only if validation failed which means that an administrator canceled the approval.

trigger: none
pr: none

stages:

- stage: releaseStage
  jobs:

  - deployment: release
    displayName: Release
    environment:
      name: dev
      resourceType: VirtualMachine
    strategy:
      runOnce:
        deploy:
          steps:
            - task: PowerShell@2
              displayName: hostname
              inputs:
                targetType: 'inline'
                script: |
                    deployment script here...
  
  - job: validation
    dependsOn: release
    pool: server
    steps:
    - task: ManualValidation@0
      inputs:
        notifyUsers: 'admin@domain.com'
        instructions: 'continue?'
        onTimeout: reject

  - deployment: rollback
    displayName: rollback
    dependsOn: validation
    condition: failed()
    environment:
      name: dev
      resourceType: VirtualMachine
    strategy:
      runOnce:
        deploy:
          steps:
            - task: PowerShell@2
              displayName: rolling back
              inputs:
                targetType: 'inline'
                script: |
                    rollback script here..
                    Write-Host "rollback"

When the release can be verified from the administrator the rollback will be skipped. This is the case when the validation is approved from the user.

Validation task will ask the user for a review.

On the other hand if validation is rejected the rollback stage will run.

Posted on by — Leave a comment

Azure batch run task with container image through az cli and json rest api

Azure Batch can be a great tool for instant batch processing as it creates and manages a pool of compute nodes (virtual machines), installs the applications you want to run, and schedules jobs to run on the nodes. The important thing using this service is that there is no additional charge for using Batch. You only pay for the underlying resources consumed, such as the virtual machines, storage, and networking.

Azure Batch documentation – Azure Batch | Microsoft Learn

In this post I will demonstrate how one can create a new job and task from az cli for batch service. The trick in this implementation will be the json that is provided as input for the task definition as not all available options are provided from az cli.

The available az cli options are shown below.

https://learn.microsoft.com/en-us/cli/azure/batch/task?view=azure-cli-latest#az-batch-task-create

One important missing configuration will be the container image that can be provided in the task trough Azure portal but not with az cli.

In order to create a task using az cli and bypass this issue, you can use the json-file parameter. This option will trigger the creation using the rest api and provide the parameters for the container image.

When there is a batch service pool available, you will need to create a job.

az batch account login -g RESOURCE_GROUP -n NAME
az batch job create --id JOB_NAME --pool-id POOL_NAME

Then you can create a new task using a json file.

az batch task create --job-id JOB_NAME --json-file

Task – Add – REST API (Azure Batch Service) | Microsoft Learn

The JSON file can be created as shown below.

{   
  "id": "azcli-task",
  "displayName": "azcli-task",
  "commandLine": "azcli-task",
  "containerSettings": {
    "containerRunOptions": "--rm --workdir /app",
    "imageName": "registry.azurecr.io/batchcontainer"
  }
}

When you execute the command you will get an output from the rest API for the created task.

output omitted

Finally you can find the new created task on Azure portal.

Posted on by — Leave a comment

Install linux azure devops agent on docker container

As we previously examined how we can create a containerized azure devops agent running on a windows machine, we will now go through the same procedure but with linux OS.

You can read the windows container azure devops agent article using the below link:

Install windows azure devops agent on docker container

The first thing that you will need is a virtual machine that runs docker. When this requirement is fulfilled you can jump on the image building. In order to build your image you will need your Dockerfile and the instructions for the agent.

You can read the rest of the article on Medium using the link below:

View at Medium.com

A detailed deployment video can be found on my Udemy course:

https://www.udemy.com/course/mastering-azure-devops-cicd-pipelines-with-yaml/