Recently working on a pipeline, I wanted to restore multiple transaction log backups on a MS SQL server. The files were located under R:\files\incremental.
You can use below powershell in order to do the restore automatically.
The above powershell will try to restore the X transaction log in the database. If the one is already restored, it will skip, else the log will be restored.
In this article I will explain how you can build your nodeJS application on Azure Devops. For the purposes of this demo I have created a hello world javascript application and stored it under a folder called node inside my repository with the name project.
We will use the folder structure later on for the build tasks.
The node folder contains two files, index.js and package.json.
The package.json file has been created automatically after initialization of a new project with
npm init
In order to build this project using npm you should define under the scripts section the commands that will be executed using a specific target.
For example in order to compile application I will execute
npm run build
but the build instructions have to be configured under the package.json file.
{
"name": "nodetest",
"version": "1.0.0",
"description": "Test project for AzureDevops using nodeJS",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"dev": "node index.js",
"build": "node index.js"
},
"author": "",
"license": "ISC",
"dependencies": {
"express": "^4.18.1"
}
}
As shown above, the build and dev targets have a simple node command that will start the application.
The index.js file is a console output javascript command.
console.log("Hello from node js app");
In order to build this node app using devops I will use the build in Npm@1 task. First I will execute the install command in order to install dependencies. Then I will run npm run with the target build. This will execute the node index.js command and will execute the node application.
trigger:
- none
pool:
vmImage: ubuntu-latest
steps:
- task: Npm@1
displayName: npm install
inputs:
command: 'install'
workingDir: '$(Build.SourcesDirectory)/node/'
- task: Npm@1
displayName: npm run build
inputs:
command: 'custom'
workingDir: '$(Build.SourcesDirectory)/node/'
customCommand: 'run build'
After pipeline run the Hello world message will print on the output.
Find this guide on Youtube:
For some data processing scenarios I had to create an automation that would download some files from a storage account, perform actions on them (python, custom tools) and lastly upload the processed files again in the storage account.
A high level diagram is visible below:
In order to automate this scenario I used a custom devops agent on azure devops and assigned a managed identity on this agent (virtual machine) on the storage account in order to interact with it without using credentials.
Then I only used powershell and az cli to download and upload the files on the storage account.
The three pipeline tasks that are required to perform the upload, processing, download actions can be found below.
The json object is used to download a specific file based on the requirements for example the first entry on chronological order. This is why the sort-object -descending is used.
trigger:
- none
pool: mypool
steps:
- task: PowerShell@2
displayName: download file from blob storage
inputs:
targetType: 'inline'
script: |
az login --identity
$json = az storage blob list --container-name blobcontainer --account-name storageaccountname --prefix "folder1/subcategory"
--auth-mode login | ConvertFrom-Json | Sort-Object -Descending { $_.properties.lastModified }
$filename = "custom_name"
az storage blob download --file "C:\devopsdir\$filename"
--name $json[0].name --container-name blobcontainer
--account-name storageaccountname --auth-mode login
Write-Host "##vso[task.setvariable variable=downloadfilename]$filename"
pwsh: true
- task: PowerShell@2
displayName: run python commands
inputs:
targetType: 'inline'
script: |
python $(Build.SourcesDirectory)/python/something.py
- task: PowerShell@2
displayName: upload file to storage account
inputs:
targetType: 'inline'
script: |
$name = "$(downloadfilename)" + "_" + (get-date -format "yyyyMMdd")
az storage blob upload --file "C:\devopsdir\$name" --name "folder1/subcategory/$name" --container-name blobcontainer --account-name storageaccountname --auth-mode login
The result will be then uploaded on the storage account.
Most times you will need to store logs for your azure resources in order to troubleshoot when things do not work as expected. Diagnostic settings for an app service can be enabled from the pane under Monitoring.
Then you should configure the diagnostic settings that will point which logs should be forwarded.
You can choose from the available categories shown below.
Lets now discover how we can enable diagnostic settings for an app service using terraform.
Create a file for example diagnostic_settings.tf and apply. The below configuration will enable all diagnostic settings categories.
resource "azurerm_monitor_diagnostic_setting" "diag_settings" {
name = "diag-settings"
target_resource_id = azurerm_windows_web_app.app_service1.id
log_analytics_workspace_id = local.log_analytics_workspace_id
log {
category = "AppServiceHTTPLogs"
enabled = true
retention_policy {
enabled = false
}
}
log {
category = "AppServiceConsoleLogs"
enabled = true
retention_policy {
enabled = false
}
}
log {
category = "AppServiceAppLogs"
enabled = true
retention_policy {
enabled = false
}
}
log {
category = "AppServiceAuditLogs"
enabled = true
retention_policy {
enabled = false
}
}
log {
category = "AppServiceIPSecAuditLogs"
enabled = true
retention_policy {
enabled = false
}
}
log {
category = "AppServicePlatformLogs"
enabled = true
retention_policy {
enabled = false
}
}
metric {
category = "AllMetrics"
retention_policy {
enabled = false
days = 30
}
}
}
You can also perform the same using a loop and a local variable in order to minimize code and make it more readable.
Assign a new variable inside your locals.tf file.
log_analytics_log_categories = ["AppServiceHTTPLogs", "AppServiceConsoleLogs","AppServiceAppLogs","AppServiceAuditLogs","AppServiceIPSecAuditLogs","AppServicePlatformLogs"]
Then perform terraform apply.
resource "azurerm_monitor_diagnostic_setting" "diag_settings" {
name = "diag-rule"
target_resource_id = azurerm_windows_web_app.app_service1.id
log_analytics_workspace_id = local.log_analytics_workspace_id
dynamic "log" {
iterator = entry
for_each = local.log_analytics_log_categories
content {
category = entry.value
enabled = true
retention_policy {
enabled = false
}
}
}
metric {
category = "AllMetrics"
retention_policy {
enabled = false
days = 30
}
}
}
After applying terraform all the settings will be enabled.
