Custom build tasks in TFS 2015

2015-07-21

Since I upgraded my team’s private TFS instance to TFS 2015 RC1, followed by RC2, the whole team has been working with TFS 2015 quite a lot. Of course one of the major features is the new build engine and we’ve given that quite a ride. From cross platform builds on Mac and Linux to custom build tasks, we’ve accomplished quite a lot. Seeing as during yesterday’s Visual Studio 2015 launch, Brian Harry stated that it was ‘quite easy’ to build your own tasks, I figured I’d give a short write-down of our experiences with custom tasks.

Preface

From the moment I upgraded our R&D server to RC1, we’ve been working with the new build system. Up until RC2 it was only possible to add custom build tasks, but we weren’t able to remove them. On top of that, the whole process isn’t documented quite yet. Seeing as we quite often add NuGet packages to a feed and didn’t want to add a, not very descriptive, PowerShell task to all of our build definitions, we decided to use this example for a custom task and see how it would fare.

Prerequisite one: What is a task?

To make a custom build task, we first need to know what it looks like. Luckily Microsoft has open-sourced most of the current build tasks in https://github.com/Microsoft/vso-agent-tasks which gave us a fair idea of what a build task is:

a JSON file describing the plugin
a PowerShell or Node.JS file containing the functionality (this post will focus on PowerShell)
an (optional) icon file
optional resources translating the options to another language

Now the only thing we needed to find out was: how to upload these tasks and in what format?

Good to know:

To make sure your icon displays correctly, it must be 32×32 pixels
The task ID is a GUID which you need to create yourself
The task category should be an existing category
Visibility tells you what kind of task it is, possible values are: Build, Release and Preview. Currently only Build-type tasks are shown

Prerequisite two: How to upload a task?

We quickly figured out that the tasks were simply .zip files containing the aforementioned items, so creating a zip was an easy but then we needed to get it there. By going through the github repository’s, we figured out there was a REST-API which controls all the tasks and we figured that by doing a PUT-call to said endpoint we could create a new task, but also overwrite tasks.

The following powershell-script enables you to upload tasks:

param(
   [Parameter(Mandatory=$true)][string]$TaskPath,
   [Parameter(Mandatory=$true)][string]$TfsUrl,
   [PSCredential]$Credential = (Get-Credential),
   [switch]$Overwrite = $false
)

# Load task definition from the JSON file
$taskDefinition = (Get-Content $taskPath\task.json) -join "`n" | ConvertFrom-Json
$taskFolder = Get-Item $TaskPath

# Zip the task content
Write-Output "Zipping task content"
$taskZip = ("{0}\..\{1}.zip" -f $taskFolder, $taskDefinition.id)
if (Test-Path $taskZip) { Remove-Item $taskZip }

Add-Type -AssemblyName "System.IO.Compression.FileSystem"
[IO.Compression.ZipFile]::CreateFromDirectory($taskFolder, $taskZip)

# Prepare to upload the task
Write-Output "Uploading task content"
$headers = @{ "Accept" = "application/json; api-version=2.0-preview"; "X-TFS-FedAuthRedirect" = "Suppress" }
$taskZipItem = Get-Item $taskZip
$headers.Add("Content-Range", "bytes 0-$($taskZipItem.Length - 1)/$($taskZipItem.Length)")
$url = ("{0}/_apis/distributedtask/tasks/{1}" -f $TfsUrl, $taskDefinition.id)
if ($Overwrite) {
   $url += "?overwrite=true"
}

# Actually upload it
Invoke-RestMethod -Uri $url -Credential $Credential -Headers $headers -ContentType application/octet-stream -Method Put -InFile $taskZipItem

Good to know:

Currently only ‘Agent Pool Administrators’ are able to add/update or remove tasks.
Tasks are server-wide, this means that you will upload to the server, not to a specific collection or project.

Creating the actual task

So like I said, we’ll be creating a new task that’s going to publish our NuGet packages to a feed. So first we need to decide what information we need to push our packages:

The target we want to pack (.csproj or .nuspec file relative to the source-directory)
The package source we want to push to

For this example I’m assuming you’re only building for a single build configuration and single target platform, which we’ll use in the PowerShell-script.

First we’ll make the task definition. As I said, this is simply a JSON file describing the task and its inputs.

{
   "id": "61ed0e1d-efb7-406e-a42b-80f5d22e6d54",
   "name": "NuGetPackAndPush",
   "friendlyName": "Nuget Pack and Push",
   "description": "Packs your output as NuGet package and pushes it to the specified source.",
   "category": "Package",
   "author": "Info Support",
   "version": {
      "Major": 0,
      "Minor": 1,
      "Patch": 0
   },
   "minimumAgentVersion": "1.83.0",
   "inputs": [
      {
         "name": "packtarget",
         "type": "string",
         "label": "Pack target",
         "defaultValue": "",
         "required": true,
         "helpMarkDown": "Relative path to .csproj or .nuspec file to pack."
      },
      {
         "name": "packagesource",
         "type": "string",
         "label": "Package Source",
         "defaultValue": "",
         "required": true,
         "helpMarkDown": "The source we want to push the package to"
      }
   ],
   "instanceNameFormat": "Nuget Pack and Push $(packtarget)",
   "execution": {
      "PowerShell": {
         "target": "$(currentDirectory)\\PackAndPush.ps1",
         "argumentFormat": "",
         "workingDirectory": "$(currentDirectory)"
      }
   }
}

This version of the task will be a very rudimentary one, which doesn’t do much (any) validation, so you might want to add that yourself.

[cmdletbinding()]
param
(
   [Parameter(Mandatory=$true)][string] $packtarget,
   [Parameter(Mandatory=$false)][string] $packagesource
)

####################################################################################################
# 1 Auto Configuration
####################################################################################################

# Stop the script on error
$ErrorActionPreference = "Stop"

# Relative location of nuget.exe to build agent home directory
$nugetExecutableRelativePath = "Agent\Worker\Tools\nuget.exe"

# These variables are provided by TFS
$buildAgentHomeDirectory = $env:AGENT_HOMEDIRECTORY
$buildSourcesDirectory = $Env:BUILD_SOURCESDIRECTORY
$buildStagingDirectory = $Env:BUILD_STAGINGDIRECTORY
$buildPlatform = $Env:BUILDPLATFORM
$buildConfiguration = $Env:BUILDCONFIGURATION

$packagesOutputDirectory = $buildStagingDirectory

# Determine full path of pack target file
$packTargetFullPath = Join-Path -Path $buildSourcesDirectory -ChildPath $packTarget

# Determine full path to nuget.exe
$nugetExecutableFullPath = Join-Path -Path $buildAgentHomeDirectory -ChildPath $nugetExecutableRelativePath

####################################################################################################
# 2 Create package
####################################################################################################

Write-Host "2. Creating NuGet package"

$packCommand = ("pack `"{0}`" -OutputDirectory `"{1}`" -NonInteractive -Symbols" -f $packTargetFullPath, $packagesOutputDirectory)

if($packTargetFullPath.ToLower().EndsWith(".csproj"))
{
   $packCommand += " -IncludeReferencedProjects"

   # Remove spaces from build platform, so 'Any CPU' becomes 'AnyCPU'
   $packCommand += (" -Properties `"Configuration={0};Platform={1}`"" -f    $buildConfiguration, ($buildPlatform -replace '\s',''))
}

Write-Host ("`tPack command: {0}" -f $packCommand)
Write-Host ("`tCreating package...")

$packOutput = Invoke-Expression "&'$nugetExecutableFullPath' $packCommand" | Out-String

Write-Host ("`tPackage successfully created:")

$generatedPackageFullPath = [regex]::match($packOutput,"Successfully created package '(.+(?<!\.symbols)\.nupkg)'").Groups[1].Value
Write-Host `t`t$generatedPackageFullPath

Write-Host ("`tNote: The created package will be available in the drop location.")

Write-Host "`tOutput from NuGet.exe:"
Write-Host ("`t`t$packOutput" -Replace "`r`n", "`r`n`t`t")

####################################################################################################
# 3 Publish package
####################################################################################################

Write-Host "3. Publish package"
$pushCommand = "push `"{0}`" -Source `"{1}`" -NonInteractive"

Write-Host ("`tPush package '{0}' to '{1}'." -f (Split-Path $generatedPackageFullPath -Leaf), $packagesource)
$regularPackagePushCommand = ($pushCommand -f $generatedPackageFullPath, $packagesource)
Write-Host ("`tPush command: {0}" -f $regularPackagePushCommand)
Write-Host "`tPushing..."

$pushOutput = Invoke-Expression "&'$nugetExecutableFullPath' $regularPackagePushCommand" | Out-String

Write-Host "`tSuccess. Package pushed to source."
Write-Host "`tOutput from NuGet.exe:"
Write-Host ("`t`t$pushOutput" -Replace "`r`n", "`r`n`t`t")

To finish up, don’t forget to add a .png logo to your task ;-)
You should now be able to add a custom task to your build pipeline from the “Package” category:

Words of warning

Tasks can be versioned, use this to your advantage. All build definitions use the latest available version of a specific task, you can’t change this behavior from the web interface, so always assume the latest version is being used.

If you don’t change the version number of your task when updating it, the build agents that have previously used your task will not download the newer version because the version number is still the same. This means that if you change the behavior of your task, you should always update the version number!

When deleting a task, this task is not automatically removed from current build definitions, on top of that you won’t get a notification when editing the build definition but you will get an exception on executing a build based on that definition.

Tasks are always available for the entire TFS instance, this means that you shouldn’t include credentials or anything that you don’t want others to see. Use ‘secret variables’ for this purpose: