Enable different configuration for multi agent pipeline with Azure DevOps

Some project has a testing that we can not run parallel. In my case, one test create and remove a blog and others refer that. Obviously it can’t run at the same time. However, you might want to reduce the testing time with multi agent strategy. You can change the connection string per agent.

Azure DevOps multi agent Integration Testing

I’ll explain how to do it and how to avoid a pit hole.

Don’t set the Environment Variables

12 Factor app recommend to store config to Environment Variables. A lot of app might follow this strategy. Azure DevOps doesn’t allow you to dynamically configure and set Environment Variables. A lot of blogs mention how to do it. However, it seems outdated. Currently when I tested it, we can’t set Environment Variables during the pipeline. For the static Environment Variables per environment, we can do it. However, if you want to have dynamically configured Environment Variables, it won’t work. Please see how to set Environment Variables as Pipeline Variables.

Variables - Azure Pipelines

When I test it (05/03/2019), We can set Environment Variables for User scope. However, If you want to get Process level Environment Variables, you can’t get it. On your PC, once you set the User level Environment Variables, you can refer it as Process level. However, it is not for Azure DevOps. I assume that before execute a task, they might remove all Process revel Environment Variables and set these from Pipeline Variables. I’m not sure however, according to my observation, it looks like that.

In short, Forget to set the environment variables during executing pipeline.

Override Configuration file

Instead, the strategy that currently work is override the config file. For example, On the .Net app, it has app.config, we can have configuration like this. Just override the sensitive information during the pipeline.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <appSettings>
    <add key="StorageConnectionString" value="UseDevelopmentStorage=true;DevelopmentStorageProxyUri=http://127.0.0.1:10002/" />
    <add key="ServiceBusConnectionString" value="" />
    <add key="TaskHubName" value="test" />
  </appSettings>

Configure the variable per agent

If you have five agents, you can configure something like this.

Variables per agents

Create a PowerShell task with inline.

PowerShell task

Code

This is the whole picture. I’ll explain some parts.

Code

While running the multi agent pipeline, it looks like this.

multi agent pipeline

As you can see, we have three agents. It is called “Job”. You can get the Value of the JobName with AGENT_JOBNAME It is predefined variables. Pipeline Variables can available as Environment Variables. However, it have some rule to convert to them. Make is upper case and ‘.’ will be ‘_’. Now you can get the Job name. e.g. Job1 As you can see, I’d like to get the number, it is different for each Job. $jobid=$jobname.Substrinb($jobname.Length-1, 1) will get the id e.g. 1 .

Predefined build variables - Azure Pipelines

This code is overridden by the pipeline variables. That is why I use case to make them override with value. $(ServiceBusConnectionString1-5) will be replaced by Azure Pipeline values.

$serviceBusConnectionString = $null
switch ($jobid) {
  "1" {  $serviceBusConnectionString = "$(ServiceBusConnectionString1)"
            break}
  "2" {  $serviceBusConnectionString = "$(ServiceBusConnectionString2)"
            break}
    "3" {  $serviceBusConnectionString = "$(ServiceBusConnectionString3)"
            break}
    "4" {  $serviceBusConnectionString = "$(ServiceBusConnectionString4)"
            break}
    "5" {  $serviceBusConnectionString = "$(ServiceBusConnectionString5)"
            break}
    default      { Write-Host "$($jobid)"; break; }}

Finally, it override the ServiceBusConnectionString with new value.

$filePath = ".\test\Some.ServiceBus.Tests\app.config
"$appConfig = [xml](cat $filePath)
      :
$appConfig.configuration.appSettings.add | foreach {
    Write-Host $_.key  #.ServiceBusConnectionString= $servcieBusConnectionString
     if ($_.key -eq "ServiceBusConnectionString")
    {        $_.value = $serviceBusConnectionString    }} $appConfig.Save($filePath)

Conclusion

We can enable the pipeline works. It was 37 min for the testing time. Now the integration testing is 10 min.

Integration Testing in 10 min

What I want to share with you guy is, forget about the dynamic environment variables strategy. I stick to the strategy, and I lost almost one day. However, now we make it!

Additional Learning (07/03/2019)

After some improvement of this pipeline, we encounter the issue, for failing tests. It works locally, however, we can’t figure out at that time. In this case we simply cross out the script. For the test part we do something like this.

[TestCategory("DisabledInCI")] 
[TestMethod] 
public async Task TestCreateIfNew()

Then we can configure this until the problem has been solved.