Posted on Leave a comment

Export Github Organization Dependencies as text

Github has introduced a powerful feature to get complete view on your project dependencies. This feature is available per project but also for a whole organization which could contain hundred of projects.

You can get all your dependencies for your organization using the below link (ORG should be replaced with your GitHub organization name)

https://github.com/orgs/ORG/insights/dependencies

I was searching a method to get massively these dependencies exported as a file. I could obtain this information using the below code with the inspect element section of Chrome browser.

elements = document.getElementsByClassName("js-navigation-open");
for (var i=0; i <elements.length; i++ ){
    var singleElement = elements[i];
    console.log(singleElement.outerText);

The result is a list with the names of the dependencies.

Unfortunately this method will only get the dependencies that are present on each page that you are currently browsing.

Posted on 5 Comments

Trigger Azure Devops build pipelines using REST API

In some cases you may need to schedule pipelines execution on nights but the schedules yaml feature cannot accomplish your needs. This could could happen if you have to provide parameters as inputs on the pipelines for a specific project export/import. In this case you could trigger your pipelines with REST APIs using ansible or another scripting tool.

In this article I will explain how you could trigger pipelines execution using the REST API of Azure Devops.

First things first you should create a personal access Token (PAT) in order to get the access required to run the pipelines on your organization. You can accomplish that by pressing your profile icon and selecting the sub-menu shown below.

You can specify the expiration of the token along with the access permissions. For the simplicity of deployment I gave it full access. You will need to copy the token somewhere as it will not be accessible after the creation.

In order to trigger a new build we would have to use the POST HTTP verb. Along with that we will need the repository name, the project name and the build pipeline ID. The below URL uses build API of version 6.1-preview.6 with the parameters

Organization: GeralexGR
Project: test-project

https://dev.azure.com/GeralexGR/test-project/_apis/build/builds?api-version=6.1-preview.6

Using CURL we can get information of runs for a specific pipeline. For example in order to get the latest runs for pipeline with ID:11. In the below example you should replace PAT with your provisioned personal access token.

curl  --user '':'PAT' --header "Content-Type: application/json" --header "Accept:application/json" https://dev.azure.com/GeralexGR/test-project/_apis/pipelines/11/runs?api-version=6.1-preview.1

In order to implement a run of a build pipeline using REST APIs I will use Postman.

The selected verb is POST and the URL is the one mentioned above. By giving the URL on the input field, POSTMAN will automatically enumerate Query parameters.

In Authorization tab you should select Basic Auth. The username can be empty and the password will be your PAT.

On Headers tab, add Content-Type as application/json

In the body, you should specify the build pipeline ID in the JSON format that is shown in the picture. If you press Send, you will successful trigger your build pipeline with ID 11.

The next step is to edit the pipeline to include also input parameters.

In order to do that, I will use the latest version of the REST API that uses the runs URL instead of builds.

As shown in the picture the request URL will be pipelines/buildID/runs

https://dev.azure.com/GeralexGR/test-project/_apis/pipelines/11/runs?&api-version=6.1-preview.1

The logic of the pipeline will only print the parameters that has been provided as input on the REST API.

trigger:
- none

pr: none 

pool:
  vmImage: ubuntu-latest

parameters:
  - name: name
    displayName: Tell me your name.
    type: string
    
steps:
- script: echo ${{parameters.name}} "succesfully triggered this build from a REST API call"
  displayName: 'print message from automatically created pipeline'

You can see in the result that my name is printed.

Azure Devops Rest API documentation:

https://docs.microsoft.com/en-us/rest/api/azure/devops/pipelines/runs/run-pipeline?view=azure-devops-rest-6.0

Youtube video:

Posted on 5 Comments

Pass parameters from build to release pipelines on Azure devops

When you need to pass parameters between your build and release pipelines it could be a real struggle if you do not want to use variable groups. Variable groups can accomplish the requested (to pass values between build and release pipelines) but this scenario is not useful for a parametric input which is a common case when deploying a project. You can accomplish that either by using a plugin from another publisher or by following the publish artifacts procedure that I will describe.

In order to pass variables between your build and release pipelines you can create/export a file containing your variable on your build agent. This file should be exported as build artifact and then downloaded on the release pipeline.

The below build pipeline implements the functionality I described. The exported file is named projectname.txt and will be located on artifacts folder on your build agent inside folder drop. For example C:\agent\1\a\drop

trigger:
- dev

pool:
  vmImage: windows-latest

parameters:
  - name: powerenvironment
    displayName: Where to deploy?
    type: string

steps:

- task: PowerShell@2
  inputs:
    targetType: 'inline'
    script: |
      $variable = '${{parameters.powerenvironment}}'
      $variable | Out-File $(Build.ArtifactStagingDirectory)\projectname.txt
      Get-Content $(Build.ArtifactStagingDirectory)\projectname.txt

- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)'
    ArtifactName: 'drop'
    publishLocation: 'Container'

When you run the pipeline you will be asked for a parameter. I gave this entry my name which will be passed on release.

My release pipeline will download the build artifacts and get the value of the file. The release pipeline includes two steps.

The first one downloads the folder drop from build artifacts. The projectname.txt is located there.

Then the powershell will print the contents of the projectname.txt

You can check the result and verify you get the parameter input value from the powershell script.

Bonus content:

You can also write your input parameter as a variable on the build agent and reference this value on a later step. This should be again a powershell step on your release pipeline.

$myVariable = "ProjectName";
$myValue = Get-Content $(System.ArtifactsDirectory)/drop/projectname.txt;

Write-Host "##vso[task.setvariable variable=ProjectName]($myValue)";

Lastly create a dump archive step to reference the input parameter from the build pipeline.

In order to test, use in the archive path the ProjectName variable.

Run the pipeline and verify that input parameter is correct (I used gerasimos).

Youtube video:

Posted on Leave a comment

There is not enough space on the disk – Azure Devops build containers

If the retention policies are not defined properly on Azure DevOps you may encounter out of space issues on your windows build containers. Take for example the below build, which stopped as it could not copy the necessary files on the build/temp folder.

In order to fix the build, you can connect to the windows server which runs the containers and get a prompt for the one that failed.

docker exec -it agent-name-01 cmd.exe

You can then check for the available free space:

Navigate to folder work and delete some build folders to free up space.