Got web.config transformation problem in your Azure DevOps?
Preface (Skip this section if you are in rush)
Well! You have been sprinting since two weeks and tomorrow is your day. You need to present your piece of enhancement you have coded since now to your team and business users. You merrily test it in your local environment and check-in the code to your feature branch and let the devops does its holy CI/CD and deploy your changes to the development branch. You get your car keys, suit-up for home, you don’t care about that half drunk coffee and you happily call it a day.
On driving back home, your subconscious mind is still running all the scenarios and it ticks it all right. This boosts your confidence for yet another day of success tomorrow for your demo day. You come home, fresh-up and then reach for a remote control to catch your favorite Netflix show that night. But, instead your fellow developer will call you and surprises you that the development box reading all the data from production server. Yikes! Isn’t that a worst way to end your day? :-) Yeah. That’s my story!
I faced this problem for the first time when I and the team had to migrate all our projects from Jenkins DevOps to Azure DevOps. This was something we under-estimated. After all, its a Microsoft DevOps and they are leaders in the DevOps! But, hey. Go figure. You cant be this lazy. Who in the world would know this tiny bit of problem will back fire. Until you know it, its bit of a pain.
What was my problem, anyway?
Okay. Its not so uncommon. Most of us face this if you are new to DevOps and coming off of a traditional deployment practice.
We got build and delivery pipelines for our web-applications. One of our app has web.config and its variations to support Dev, RC, Test and Production environments. We got native transformations inside the project to support automatic transformation of web.config at the time of build and deployment. It was once worked out with Slow-cheetah plugin to transform the web.config file and Jenkins to deploy the solution. We had no problems back then.
We took the whole git repository and migrated all to Azure DevOps (we use on-prem devops btw). And configured both CD and CI for the application. It all worked out fine. But, later we uncovered this issue. The final web.config which gets deployed to any target environment was always based off of web.release.config. It never picked up the desired web.<environment>.config for Dev, RC, etc.
We use to get this error in the deploy (CD) pipeline every time it was run.
2020–08–03T23:31:48.9981190Z ##[warning]Unable to apply transformation for the given package. Verify the following.
2020–08–03T23:31:48.9982309Z ##[warning]1. Whether the Transformation is already applied for the MSBuild generated package during build. If yes, remove the <DependentUpon> tag for each config in the csproj file and rebuild.
2020–08–03T23:31:48.9982777Z ##[warning]2. Ensure that the config file and transformation files are present in the same folder inside the package.
And, we follow build-once-deploy-everywhere philosophy. So, we expect the build artifact/output to be more generic as possible. Something like this.
Without going much into each detail, I will just list down the changes we did for it to work. These are the information gathered from reading Microsoft articles, referring Stack Overflow, some trial and errors. And I welcome any suggestions to improvise.
Steps taken to fix the transformation problem
In your project
- Open your csproj (or equivalent) file in a notepad.
- Find and remove ‘<DependentUpon>’ tag related to web.config.
- Change the tag of name ‘<None>’ to ‘<Content>’. And remove <DependentUpon> tag from each of the web.<env>.config elements. It should look something like below after doing the changes.
4. Save the file. Now, open the project in visual studio. Make sure all the web configs are included and displayed in the project in isolation and not under the web.config root.
5. Select each web.config file and change the file property “Copy to Output Directory” to “Copy if newer”. Save the project.
In Azure DevOps-Build Pipeline
6. In the ‘Build solution’ under the Agent, make sure the value in the ‘MSBuild Arguments’ has following arguments.
/p:DeployOnBuild=true
/p:WebPublishMethod=Package
/p:PackageAsSingleFile=true
/p:SkipInvalidConfigurations=true
/p:PackageLocation=”$(build.artifactstagingdirectory)\\”
/p:TransformWebConfigEnabled=false
/p:AutoParameterizationWebConfigConnectionStrings=false
/p:MarkWebConfigAssistFilesAsExclude=false
/p:ProfileTransformWebConfigEnabled=false
/p:IsTransformWebConfigDisabled=true
Arguments in bold are mostly important to disable transformation during the build time.
7. Also, you can try checking ‘Clean’ checkbox as shown in the picture below.
In Azure DevOps-Delivery Pipeline
8. Make sure the pipeline stage names are properly named to match the webconfig names. Ex: The stage name for web.Development.config will be simply ‘Development’ and so on.
9. Check the ‘XML Transformation’ flag under the ‘IIS Web App Deploy’ task. You might have already did this as its more obvious. But listing here for the sake of completeness.
10. Running a build pipeline. Download the build generated artifact zip package. Verify that the package folder should have all the web configs.
11. Also, compare the package web.config with your project web.config. Both should be same and there should not be any transformation.
I think these are it! Should get you going…
Troubleshooting items
- You might encounter object reference exception while delivery pipeline is run during the deployment. There could be various reasons. Some of the common are.
(a) Beside web.release.config, other web configs might also have the remove debug tag below. This could have gotten there by accident or simply by copy-paste without us being aware of the nuances.
<compilation xdt:Transform=”RemoveAttributes(debug)” />
This would happen when the debug attribute is already removed but the second attempt is made with the different webconfig as the source. The way ctt.exe transforms is it will first apply web.release.config, then apply the desired config file as per the stage name. So, these errors are common.
(b) You are trying to transform remove/add/replace tags against the tags in target file which aren’t simply there. So, pay attention to these by taking one nice look again at the web configs.
That is all for now. I will add more as I come across others.
2. The resulting web.config has release configuration (web.release.config) rather than the desired target environment. This would happen when your target web.<env>.config has no transformation instructions. For example: You got three web.configs.
(1) web.config — base config
(2) web.dev.config — for your development environment. This is the exact copy/paste of web.config.
(3) web.release.config — for your production.
When you run the deploy for ‘Dev’ release pipeline. The resulting web.config contains transformations from web.release.config instead from web.dev.config. You may wonder what business release config got here?
The cause may be that your web.dev.config doesnt have any transformation. By nature the deploy pipeline uses ctt.exe. The way it works in Azure Devops’ release pipeline is that ctt.exe always starts by transforming web.config with web.release.config.
Later, it will parse for the target deployment web.config (in our case, it is web.dev.config) and then apply the transformation second time on the transformed web.config again.
In this case, if the web.dev.config doesn’t have any transform instructions to add, remove, replace the keys, attributes, connection strings etc. It would simply not care to transform the intermediate web.config which was initially transformed based off of web.release.config. So, please make sure the transformation instructions in your web.dev.config is proper and it does so as if you are transforming web.release.config instead of web.config.
Bonus items
Sample web.configs I tried experimenting with. Watch out for highlighted items which will be transformed.
Web.Config (Base version)
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<connectionStrings>
<add name="DefaultConnection"
connectionString="Data Source=(LocalDb)\\MSDB;DbFilename=aspcore-local.mdf;" />
</connectionStrings>
<system.web>
<compilation debug="true" strict="false" explicit="true" targetFramework="4.7.2"/>
<httpRuntime targetFramework="4.7.2"/>
<pages>
<namespaces>
<add namespace="System.Web.Optimization"/>
</namespaces>
<controls>
<add assembly="Microsoft.AspNet.Web.Optimization.WebForms" namespace="Microsoft.AspNet.Web.Optimization.WebForms" tagPrefix="webopt"/>
</controls>
</pages>
</system.web>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<dependentAssembly>
<assemblyIdentity name="Antlr3.Runtime" publicKeyToken="eb42632606e9261f"/>
<bindingRedirect oldVersion="0.0.0.0-3.5.0.2" newVersion="3.5.0.2"/>
</dependentAssembly>
<dependentAssembly>
<assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed"/>
<bindingRedirect oldVersion="0.0.0.0-12.0.0.0" newVersion="12.0.0.0"/>
</dependentAssembly>
<dependentAssembly>
<assemblyIdentity name="WebGrease" publicKeyToken="31bf3856ad364e35"/>
<bindingRedirect oldVersion="0.0.0.0-1.6.5135.21930" newVersion="1.6.5135.21930"/>
</dependentAssembly>
</assemblyBinding>
</runtime>
<system.codedom>
<compilers>
<compiler language="c#;cs;csharp" extension=".cs"
type="Microsoft.CodeDom.Providers.DotNetCompilerPlatform.CSharpCodeProvider, Microsoft.CodeDom.Providers.DotNetCompilerPlatform, Version=2.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
warningLevel="4" compilerOptions="/langversion:default /nowarn:1659;1699;1701"/>
<compiler language="vb;vbs;visualbasic;vbscript" extension=".vb"
type="Microsoft.CodeDom.Providers.DotNetCompilerPlatform.VBCodeProvider, Microsoft.CodeDom.Providers.DotNetCompilerPlatform, Version=2.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
warningLevel="4" compilerOptions="/langversion:default /nowarn:41008 /define:_MYTYPE=\"Web\" /optionInfer+"/>
</compilers>
</system.codedom>
<appSettings>
<add key="testKey1" value="false" />
<add key="testKey2" value="1" />
</appSettings>
</configuration>Web.Development.Config
<?xml version=”1.0" encoding=”utf-8"?><configuration xmlns:xdt=”http://schemas.microsoft.com/XML-Document-Transform">
<connectionStrings>
<add name=”MyDB”
connectionString=”Data Source=ReleaseSQLServer_dev;Initial Catalog=MyReleaseDB;Integrated Security=True”
xdt:Transform=”Insert” />
</connectionStrings>
<appSettings>
<add xdt:Transform=”Replace” xdt:Locator=”Match(key)” key=”testKey1" value=”true_dev” />
<add xdt:Transform=”Replace” xdt:Locator=”Match(key)” key=”testKey2" value=”2_dev” />
</appSettings>
<system.web>
<compilation debug="true" xdt:Transform="SetAttributes(debug)" />
</system.web>
</configuration>Web.Release.Config
<?xml version="1.0" encoding="utf-8"?><!-- For more information on using web.config transformation visit https://go.microsoft.com/fwlink/?LinkId=125889 --><configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
<connectionStrings>
<add name="MyDB"
connectionString="Data Source=ReleaseSQLServer;Initial Catalog=MyReleaseDB;Integrated Security=True"
xdt:Transform="Insert" />
</connectionStrings>
<appSettings>
<add xdt:Transform="Replace" xdt:Locator="Match(key)" key="testKey1" value="true" />
<add xdt:Transform="Replace" xdt:Locator="Match(key)" key="testKey2" value="2" />
</appSettings>
<system.web>
<compilation xdt:Transform="RemoveAttributes(debug)" />
</system.web>
</configuration>
Result web.config run with Stage ‘Development’
<?xml version="1.0" encoding="utf-8"?>
<!--
For more information on how to configure your ASP.NET application, please visit
https://go.microsoft.com/fwlink/?LinkId=169433
-->
<configuration>
<connectionStrings>
<add name="DefaultConnection" connectionString="Data Source=(LocalDb)\\MSDB;DbFilename=aspcore-local.mdf;" />
<add name="MyDB" connectionString="Data Source=ReleaseSQLServer;Initial Catalog=MyReleaseDB;Integrated Security=True" />
<add name="MyDB" connectionString="Data Source=ReleaseSQLServer_dev;Initial Catalog=MyReleaseDB;Integrated Security=True" />
</connectionStrings>
<system.web>
<compilation strict="false" debug="true" explicit="true" targetFramework="4.7.2" />
<httpRuntime targetFramework="4.7.2" />
<pages>
<namespaces>
<add namespace="System.Web.Optimization" />
</namespaces>
<controls>
<add assembly="Microsoft.AspNet.Web.Optimization.WebForms" namespace="Microsoft.AspNet.Web.Optimization.WebForms" tagPrefix="webopt" />
</controls>
</pages>
</system.web>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<dependentAssembly>
<assemblyIdentity name="Antlr3.Runtime" publicKeyToken="eb42632606e9261f" />
<bindingRedirect oldVersion="0.0.0.0-3.5.0.2" newVersion="3.5.0.2" />
</dependentAssembly>
<dependentAssembly>
<assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" />
<bindingRedirect oldVersion="0.0.0.0-12.0.0.0" newVersion="12.0.0.0" />
</dependentAssembly>
<dependentAssembly>
<assemblyIdentity name="WebGrease" publicKeyToken="31bf3856ad364e35" />
<bindingRedirect oldVersion="0.0.0.0-1.6.5135.21930" newVersion="1.6.5135.21930" />
</dependentAssembly>
</assemblyBinding>
</runtime>
<system.codedom>
<compilers>
<compiler language="c#;cs;csharp" extension=".cs" type="Microsoft.CodeDom.Providers.DotNetCompilerPlatform.CSharpCodeProvider, Microsoft.CodeDom.Providers.DotNetCompilerPlatform, Version=2.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" warningLevel="4" compilerOptions="/langversion:default /nowarn:1659;1699;1701" />
<compiler language="vb;vbs;visualbasic;vbscript" extension=".vb" type="Microsoft.CodeDom.Providers.DotNetCompilerPlatform.VBCodeProvider, Microsoft.CodeDom.Providers.DotNetCompilerPlatform, Version=2.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" warningLevel="4" compilerOptions="/langversion:default /nowarn:41008 /define:_MYTYPE=\"Web\" /optionInfer+" />
</compilers>
</system.codedom>
<appSettings>
<add key="testKey1" value="true_dev" />
<add key="testKey2" value="2_dev" />
</appSettings>
</configuration>Result web.config run with Stage ‘Release’
<?xml version="1.0" encoding="utf-8"?>
<!--
For more information on how to configure your ASP.NET application, please visit
https://go.microsoft.com/fwlink/?LinkId=169433
-->
<configuration>
<connectionStrings>
<add name="DefaultConnection" connectionString="Data Source=(LocalDb)\\MSDB;DbFilename=aspcore-local.mdf;" />
<add name="MyDB" connectionString="Data Source=ReleaseSQLServer;Initial Catalog=MyReleaseDB;Integrated Security=True" />
</connectionStrings>
<system.web>
<compilation strict="false" explicit="true" targetFramework="4.7.2" />
<httpRuntime targetFramework="4.7.2" />
<pages>
<namespaces>
<add namespace="System.Web.Optimization" />
</namespaces>
<controls>
<add assembly="Microsoft.AspNet.Web.Optimization.WebForms" namespace="Microsoft.AspNet.Web.Optimization.WebForms" tagPrefix="webopt" />
</controls>
</pages>
</system.web>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<dependentAssembly>
<assemblyIdentity name="Antlr3.Runtime" publicKeyToken="eb42632606e9261f" />
<bindingRedirect oldVersion="0.0.0.0-3.5.0.2" newVersion="3.5.0.2" />
</dependentAssembly>
<dependentAssembly>
<assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" />
<bindingRedirect oldVersion="0.0.0.0-12.0.0.0" newVersion="12.0.0.0" />
</dependentAssembly>
<dependentAssembly>
<assemblyIdentity name="WebGrease" publicKeyToken="31bf3856ad364e35" />
<bindingRedirect oldVersion="0.0.0.0-1.6.5135.21930" newVersion="1.6.5135.21930" />
</dependentAssembly>
</assemblyBinding>
</runtime>
<system.codedom>
<compilers>
<compiler language="c#;cs;csharp" extension=".cs" type="Microsoft.CodeDom.Providers.DotNetCompilerPlatform.CSharpCodeProvider, Microsoft.CodeDom.Providers.DotNetCompilerPlatform, Version=2.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" warningLevel="4" compilerOptions="/langversion:default /nowarn:1659;1699;1701" />
<compiler language="vb;vbs;visualbasic;vbscript" extension=".vb" type="Microsoft.CodeDom.Providers.DotNetCompilerPlatform.VBCodeProvider, Microsoft.CodeDom.Providers.DotNetCompilerPlatform, Version=2.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" warningLevel="4" compilerOptions="/langversion:default /nowarn:41008 /define:_MYTYPE=\"Web\" /optionInfer+" />
</compilers>
</system.codedom>
<appSettings>
<add key="testKey1" value="true" />
<add key="testKey2" value="2" />
</appSettings>
</configuration>References
- https://stackoverflow.com/questions/57955114/how-to-stop-msbuild-from-transforming-web-config-in-azure-devops-pipelines
- https://stackoverflow.com/questions/54254446/how-to-include-my-config-transformation-files-in-the-web-deploy-zip
- https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/transforms-variable-substitution?view=azure-devops&tabs=Classic
