So we moved to the new awesome scripted build from VSTS and GitHub as our main code repository. But how do we handle the build and deployments? In this post I will show all the tips and tricks we did to get it working. Please note that this is a continues improving plan, but it will get you started.
Code setup
First our code setup. In the root of our project we have a couple of folders to not mix up code with other tools. So we have the following folders defined:
- src (for sources)
- build (for build scripts, see versioning below)
- tools (for external tools)
Build setup
For the build server we've created the following steps. See the specific configurations below each name.
Build Tab
Delete files
Contents: **\<Namespace>*.nupkg (this is because we don't do a clean checkout each time)
Powershell
Script Filename: build/ApplyVersionToAssemblies.ps1
NuGet Installer
Path to Solution: **\*.sln
Visual Studio Build
Solution: **\*.sln
MSBuild Arguments: /p:RunOctoPack=true
Visual Studio Test
Test Assembly: **\bin\$(BuildConfiguration)\*test*.dll; -:**\xunit.runner.visualstudio.testadapter.dll; -:**\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll;-:**\<NameSpace>.TestUtils.dll
NuGet Publisher
Path/Pattern to nupkg **\bin\**\MeteoGroup.RouteGuard*.nupkg
NuGet Server Endpoint: Octopus Deploy (see Octopus Deploy, below)
Triggers Tab
The triggers trigger only the master, feature and hotfix branches. All other branches are not automatically build. The branch prefixes for feature and hotfix are based on the
GitFlow naming, so if we want to use GitFlow, the naming is at least the same.
General Tab
Build Number Format: <YourProjectName>_2.$(Year:yy)$(DayOfYear)$(rev:.r)
Version Updater
The version updating script is a nifty little thing that uses regex to set the version number of the build in your assemblies. And with the assemblies properly versioned, your OctoPack will also use the proper version and so will your deployment. This brings the awesomeness that everything is connected to each other!
It will create versions like 2.16109.1.0 (for master branch builds) and 2.16109-feature-<FEATURENAME> for feature builds. The versions should be SemVer 1 compatible due to limitations in the NuGet 2.0 protocol. Safe the powershell script below in the build directory as ApplyVersionToAssemblies.ps1
##-----------------------------------------------------------------------
## <copyright file="ApplyVersionToAssemblies.ps1">(c) Microsoft Corporation.
## This source is subject to the Microsoft Permissive License.
## See http://www.microsoft.com/resources/sharedsource/licensingbasics/sharedsourcelicenses.mspx.
## All other rights reserved.</copyright>
##-----------------------------------------------------------------------
# Look for a 0.0.0.0 pattern in the build number.
# If found use it to version the assemblies.
#
# For example, if the 'Build number format' build process parameter
# $(BuildDefinitionName)_$(Year:yyyy).$(Month).$(DayOfMonth)$(Rev:.r)
# then your build numbers come out like this:
# "Build HelloWorld_2013.07.19.1"
# This script would then apply version 2013.07.19.1 to your assemblies.
# Enable -Verbose option
[CmdletBinding()]
# Regular expression pattern to find the version in the build number
# and then apply it to the assemblies
$BuildVersionRegex = "\d+\.\d+\.\d+"
$FileVersionRegex = "\d+\.\d+\.\d+\.\d+"
$VersionTagRegex = "refs\/heads\/(\w*)\/([\w-]*)"
# If this script is not running on a build server, remind user to
# set environment variables so that this script can be debugged
if(-not ($Env:BUILD_SOURCESDIRECTORY -and $Env:BUILD_BUILDNUMBER))
{
Write-Error "You must set the following environment variables"
Write-Error "to test this script interactively."
Write-Host '$Env:BUILD_SOURCESDIRECTORY - For example, enter something like:'
Write-Host '$Env:BUILD_SOURCESDIRECTORY = "C:\code\FabrikamTFVC\HelloWorld"'
Write-Host '$Env:BUILD_BUILDNUMBER - For example, enter something like:'
Write-Host '$Env:BUILD_BUILDNUMBER = "Build HelloWorld_0000.00.00.0"'
exit 1
}
# Make sure path to source code directory is available
if (-not $Env:BUILD_SOURCESDIRECTORY)
{
Write-Error ("BUILD_SOURCESDIRECTORY environment variable is missing.")
exit 1
}
elseif (-not (Test-Path $Env:BUILD_SOURCESDIRECTORY))
{
Write-Error "BUILD_SOURCESDIRECTORY does not exist: $Env:BUILD_SOURCESDIRECTORY"
exit 1
}
Write-Verbose "BUILD_SOURCESDIRECTORY: $Env:BUILD_SOURCESDIRECTORY"
# Make sure there is a build number
if (-not $Env:BUILD_BUILDNUMBER)
{
Write-Error ("BUILD_BUILDNUMBER environment variable is missing.")
exit 1
}
Write-Verbose "BUILD_BUILDNUMBER: $Env:BUILD_BUILDNUMBER"
Write-Verbose "BUILD_SOURCEBRANCH: $Env:BUILD_SOURCEBRANCH"
# Get and validate the version data
$VersionData = [regex]::matches($Env:BUILD_BUILDNUMBER,$BuildVersionRegex)
switch($VersionData.Count)
{
0
{
Write-Error "Could not find version number data in BUILD_BUILDNUMBER."
exit 1
}
1 {}
default
{
Write-Warning "Found more than instance of version data in BUILD_BUILDNUMBER."
Write-Warning "Will assume empty version tag."
}
}
$VersionTagData = [regex]::matches($Env:BUILD_SOURCEBRANCH,$VersionTagRegex)
switch($VersionTagData.Captures.Groups.Count)
{
0 {}
3
{
$VersionTag = $VersionTagData.Captures.Groups[1].value + '-' + $VersionTagData.Captures.Groups[2].value
}
default
{
Write-Error "Invalid version tag data in BUILD_SOURCEBRANCH."
}
}
$NewVersion = $VersionData[0].value
Write-Verbose "Version: $NewVersion"
if($VersionTag){
Write-Verbose "VersionTag: $VersionTag"
}
# Apply the version to the assembly property files
$files = gci $Env:BUILD_SOURCESDIRECTORY -recurse -include "*Properties*","My Project" |
?{ $_.PSIsContainer } |
foreach { gci -Path $_.FullName -Recurse -include AssemblyInfo.* }
if($files)
{
Write-Verbose "Will apply $NewVersion to $($files.count) files."
foreach ($file in $files) {
$filecontent = Get-Content($file)
attrib $file -r
$FileVersion = $NewVersion + ".0"
$filecontent -replace $FileVersionRegex, $FileVersion | Out-File $file
if($VersionTag) {
Add-Content $file "`n[assembly: AssemblyInformationalVersion(`"$NewVersion-$VersionTag`")]"
Write-Verbose "$file.FullName - version tag applied"
}
else {
Write-Verbose "$file.FullName - version applied"
}
}
}
else
{
Write-Warning "Found no files."
}
Octopus Deploy
You've probably already heard of
Octopus Deploy. Your build server builds, and Octopus Deploys.
In the previous step you've seen that we've created a service endpoint in VSTS. You can add it by clicking on the Settings button and add a service.
With the versioning in place, everything will work fine in Octopus, and because we have names like 'Feature' or 'Hotfix' in the package, you can even setup channels to quickly deploy hotfix patches to production and allow feature packages only to be deployed on your development environment. But this is something we still need to setup (maybe in a future blog post).
Because the applications in the end don't know the release version / environment name, we've created a Variable set which is called 'Default Environment' and added the following 2 keys in it:
All applications that need to do something with it, can now use the Version and Environment name (in our case we use it to log the version to LogStash).
When you want to update more that one project, the
octo.exe is there to help you, with this super simple tool, you can create releases and deploy multiple projects at once. For example:
@echo off
set SERVER=http://<YOUR OCTOPUS SERVER>/
set APIKEY=<API KEY>
set PACKAGEVERSION=<PACKAGE VERSION FROM VSTS, when not Feature/Hotfix, add .0 to it>
set TO=Development
set RELEASENOTES="<RELEASE NOTES>"
octo create-release --server %SERVER% --releasenotes=%RELEASENOTES% --apiKey %APIKEY% --packageversion %PACKAGEVERSION% --project "<Project Name> Api"
octo create-release --server %SERVER% --releasenotes=%RELEASENOTES% --apiKey %APIKEY% --packageversion %PACKAGEVERSION% --project "<Project Name> Application"
octo create-release --server %SERVER% --releasenotes=%RELEASENOTES% --apiKey %APIKEY% --packageversion %PACKAGEVERSION% --project "<Project Name> Data Ingestor"
octo create-release --server %SERVER% --releasenotes=%RELEASENOTES% --apiKey %APIKEY% --packageversion %PACKAGEVERSION% --project "<Project Name> Product Worker"
octo deploy-release --server %SERVER% --apiKey %APIKEY% --releaseNumber %PACKAGEVERSION% --deployto %TO% --waitfordeployment --project "<Project Name> Api"
octo deploy-release --server %SERVER% --apiKey %APIKEY% --releaseNumber %PACKAGEVERSION% --deployto %TO% --waitfordeployment --project "<Project Name> Application"
octo deploy-release --server %SERVER% --apiKey %APIKEY% --releaseNumber %PACKAGEVERSION% --deployto %TO% --waitfordeployment --project "<Project Name> Data Ingestor"
octo deploy-release --server %SERVER% --apiKey %APIKEY% --releaseNumber %PACKAGEVERSION% --deployto %TO% --waitfordeployment --project "<Project Name> Product Worker"
You'll be amazed with the nice colored logging that comes out of this beauty. Now a 5 O'clock deployment is nothing more than a click away (but don't do it :)) !
We're still in the process of improving the flow of the deployment, but I like the progress we made so far. If you have any tips or questions, let me know in the comments below!
Happy deploying!
Luuk
