Azure DevOps stages and jobs on build pipelines could save you a lot of minutes when you deal with more complex setups and heavy projects. In this article I will analyze how you can use jobs and stages to maximize performance and parallelism.
Lets take the starter pipeline example on Azure DevOps (code below). Although this is perfect for a single task scenario, it will execute tasks on a row without parallelism. The output of this pipeline will be two printed messages on your debug window.
trigger:
- main
pool:
vmImage: ubuntu-latest
steps:
- script: echo Hello, world!
displayName: 'Run a one-line script'
- script: |
echo Add other tasks to build, test, and deploy your project.
echo See https://aka.ms/yaml
displayName: 'Run a multi-line script'
The two tasks that are included (script task) are located under steps. Those steps are part of a job that is not present as it is the only one.
The hierarchy of a pipeline can include the below: Stages -> Jobs -> Steps -> Tasks
On the starter pipeline two tasks are included under steps that belong on a single job. Jobs big advantage is that they can run in parallel. Take for example a big pipeline that includes a lot of jobs 30 and more. It would be time killing to wait all these jobs execute one by one. Keep in mind that one job can also fail and you will have to start the process again.
If you have a single job and all tasks are included on it, you have to use continueOnError if you do not want to stop the pipeline on a task failure.
The below example shows two jobs that can be executed on parallel based on your Azure DevOps configuration (this may include more costs on your subscription). As you can see the failure of the first job will not affect the second job that will be executed. If you are also eligible for parallel jobs, these can run simultaneously if you do not have constraints between them.
pool:
vmImage: ubuntu-latest
trigger: none
pr: none
jobs:
- job:
displayName: Install npm and yarn tools
steps:
- task: Npm@1
displayName: Install npm
inputs:
command: 'install'
workingDir: '$(Pipeline.Workspace)/s'
- task: PowerShell@2
displayName: Install yarn
inputs:
targetType: 'inline'
script: |
npm install -g yarn
- job:
displayName: build code and publish artifact
steps:
- task: PowerShell@2
displayName: build code
inputs:
targetType: 'inline'
script: |
Write-Host "this task will be executed"
- task: PowerShell@2
displayName: this task will fail
inputs:
targetType: 'inline'
script: |
fail
Lets now examine the power of stages. The stage includes multiple jobs as you can see from the example below. A typical production environment will include stages for QA -> DEV -> Production deployments.
This approach big advantage is that you can rerun failed jobs separately and also rerun the whole stage in separation from each other. As a typical build pipeline may take a lot of minutes to complete by using stages you do not have to rerun the whole pipeline on a task failure.
trigger:
- none
pool:
vmImage: ubuntu-latest
stages:
- stage: BuildApp
displayName: Build Apps
jobs:
- job: BuildFrontendApp
displayName: Build Frontend App
steps:
- script: echo building frontend app
displayName: build frontend app
- script: echo running unit tests for frontend app
displayName: unit tests frontend
- job: BuildBackendApp
displayName: Build Backend App
steps:
- script: echo building backend app
displayName: build backend app
- script: echo running unit tests for backend app
displayName: unit tests backend
- stage: DeployDev
displayName: Deploy to DEV environment
jobs:
- job: DeployFrontendDev
displayName: Deploy frontend to DEV
steps:
- checkout: none
- script: echo deploying frontend app to DEV
displayName: deploy frontend app to DEV
- job: DeployBackendDev
displayName: Deploy backend to DEV
steps:
- checkout: none
- script: echo deploying backend app to DEV
displayName: deploy backend app to DEV
- stage: DeployProd
displayName: Deploy to PROD environment
jobs:
- job: Failjob
displayName: Running this job will fail
steps:
- checkout: none
- script: kati
displayName: deploy frontend app to PROD
- job: DeployBackendProd
displayName: Deploy backend to PROD
steps:
- checkout: none
- script: echo deploying backend app to PROD
displayName: deploy backend app to PROD
Video tutorial on YouTube:
