Dallas CSharp SIG - MSBuild Fundamentals

On 8/5/2010, Leblanc presented “MSBuild Fundamentals” at the Dallas C# Sig. If you are interested in seeing the video, Shawn Weisfeld has captured and archived it on his blog and blip.tv. This post reiterates what was discussed in the video, and gives you a better format for quick reference scanning.

The video is a little rough at the start, but hits some good points once it gets to the demo and hands on stages. The video covers in detail on how we built some of our own custom tasks to help distributed teams.

What is MSBuild

MSBuild is a tool in the continuous integration domain. MSBuild is the scripting language of choice for solving, building, and deployment problems in .NET. Any projects created by Visual Studio become an MSBuild file. Team Foundation Server can also use MSBuild directly, or indirectly, with Windows Workflow.

The goal of continuous integration is to reduce deployment risk. This is done by verifying that whatever is checked into version control can be checked out; successfully build, pass unit tests, and pass the integration test suite on schedule. This ensures that no one works on an island, and that deployment builds are consistent and repeatable without human intervention.

Here are two common .NET continuous integration tools that I use to manage my projects and how I have integrated MSBuild. ci team foundation server, tfsci teamcity

Syntax & Semantics

Prior to Visual Studio 2010, syntax needed to be memorized. Nowadays, though, Visual Studio 2010 provides intellisense for “.proj” files, thereby lowering the bar to entry. The syntax is what the parser cares about. The parser being the MSbuild engine used: Microsoft.NET\Framework\v4.0.30319\MSBuild.exe

Notice that that path is pointing to the .net framework 4 version of the MSBuild.exe file. Similarly, the ToolsVersion attribute on the Project element must match the framework version.

A basic template is as follows:

          <Project ToolsVersion="4.0" DefaultTargets="Setup" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<!-- Global Variables -->
<!-- Import MSBUILD Tasks -->
<!-- Targets -->
<Target Name="UpdateAfterSvn" DependsOnTargets="IdentifyBuildVariables">
	<!-- ####### add more tasks here ####### -->
	<RemoveDir Directories="$(TemporaryFolder)" />
</Target>
</Project>

        

The recommended approach to execute the MSbuild project is to have a batch file launch the process. The following bat file will allow you to right click on the bat file and let you run as administrator while keeping the local project directory.

          @setlocal enableextensions
@cd /d "%~dp0"
%windir%\Microsoft.NET\Framework\v4.0.30319\msbuild.exe MSBuild\run-after-update.proj /t:UpdateAfterSvn /fl /flp:v=diag
pause

        

The semantics in the following section will provide the structure, together with how it will be interpreted by the MSbuild engine.

Scripting Language Constructs

Special Characters

Variables - label that points to a single value

          <PropertyGroup>
	<ExternsMSBuildFolder>$(MSBuildStartupDirectory)\Externs\msbuild</ExternsMSBuildFolder>
	<DatabaseFolder>$(MSBuildStartupDirectory)\Database</DatabaseFolder>
	<MyMSBuildFolder>$(MSBuildStartupDirectory)\MSBuild</MyMSBuildFolder>
	<TemporaryFolder>$(MSBuildStartupDirectory)\tmp</TemporaryFolder>
	<OutputFolder>$(MSBuildStartupDirectory)\tmp</OutputFolder>
</PropertyGroup>

<PropertyGroup>
	<ZipTask>$(ExternsMSBuildFolder)\ExecTasks\7-Zip\7z.exe</ZipTask>
	<PscpTask>$(ExternsMSBuildFolder)\ExecTasks\pscp.exe</PscpTask>
	<PsftpTask>$(ExternsMSBuildFolder)\ExecTasks\psftp.exe</PsftpTask>
	<AltovaXmlTask>$(ExternsMSBuildFolder)\ExecTasks\AltovaXml.exe</AltovaXmlTask>
</PropertyGroup>

        

The variable value can later be referenced using ‘$’ surrounded with parenthesis:

          <Exec Command=""$(AltovaXmlTask)"  /in "%(WebSites.WebConfigFile)" /xslt2 "$(ExternsMSBuildFolder)\ExecTasks\AltovaXmlIdentity.xslt" > "%(WebSites.WebConfigFile)-tmp.config"" />

<RemoveDir Directories="$(TemporaryFolder)" />

        

Collections - label that points to multiple values

          <ItemGroup>
  <Directories Include="$(OutputFolder); $(TmpFolder)" />
</ItemGroup>

        

The collection values can later be referenced using ‘@’ surrounded with parenthesis:

          <RemoveDir Directories="@(Directories)" />
<Message Text="@(Directories, '; ')">

        

All directories are printed on the same line seperated with ’; ’

Working with collections also opens up the concept of metadata and removing items from greedy matches:

          <Target Name="Push" DependsOnTargets="Obfuscate;Package">
	<ItemGroup>
		<DeployWwwFiles Include="$(OutputFolder)\Www\Www.WebApplication\**\*.*" />
		<DeployServiceFiles Include="$(OutputFolder)\Services\**\*.*" />
	</ItemGroup>

	<Copy
		  SourceFiles="@(DeployWwwFiles)"
		  DestinationFiles="@(DeployWwwFiles->'$(DeployWww)\%(RecursiveDir)%(Filename)%(Extension)')" />
	<Copy
		  SourceFiles="@(DeployServiceFiles)"
		  DestinationFiles="@(DeployServiceFiles->'$(DeployServices)\Services\%(RecursiveDir)%(Filename)%(Extension)')" />
</Target>

        

Here we use the copy task to individually move files, from inside the Websites collection, to an OutputFolder.

To create the destination files we use ’@ ’, which addresses the same collection as the source, but we then use ‘->’ to make a projection. The projection is done inside the MSBuild engine, before assigning the projected values to the DestinationFiles property on the Copy task. Inside the projection we show an example of using single value variables, PropertyGroup value ‘$(OutputFolder)’, custom metadata retrieved with ‘%(DirName)’, and built in metadata created by MSBuild directly for files ‘%(RecursiveDir)%(Filename)%(Extension)’.

If you create custom tasks, you can access custom metadata like the DirName above inside the task as follows:

           public ITaskItem[] Parameters { get; set; }
 public override bool Execute() {
//System.Diagnostics.Debugger.Launch();
 try {
	 if (Parameters != null) {
		 foreach (var item in Parameters) {
			var dirName = item.GetMetadata("DirName");
		 }
	 }
	return true;
 }
 catch(Exception exception)
 {
	 LogError(exception);
	 return false;
 }
 }

        

Conditions

All elements in MSBuild have a condition attribute that must be true before the MSBuild engine evaluates the element.

          <PropertyGroup>
	<DBPrefix></DBPrefix>
	<!-- Prefix DB  not local trunk; any where else prefix is applied. -->
	<DBPrefix Condition=" !( '$(BranchNameByFolder)' == 'trunk-webapp' AND '$(IsInTeamBuild)' != 'True') ">$(DeployServer)</DBPrefix>
</PropertyGroup>

        

Loops, Batching

Batching is the closest concept to loops. Batching executes the current statement for each item in the ItemGroup. Seeing ‘%’ without the projection notation ‘@’ ‘->’, should signal the use of batching in the script. The notation is: %(ItemGroupLabel.ItemMetaDataName).

          <ItemGroup>
  <Databases Include="CompanyABC">
    <Name>CompanyABC</Name>
    <DatabaseName>$(DBPrefix)CompanyABC</DatabaseName>
    <CanRestore>true</CanRestore>
    <CanBackup>true</CanBackup>
  </Databases>
  <Databases Include="CompanyABCAttachments">
    <Name>CompanyABCAttachments</Name>
    <DatabaseName>$(DBPrefix)CompanyABCAttachments</DatabaseName>
    <CanRestore>true</CanRestore>
    <CanBackup>true</CanBackup>
  </Databases>
  <Databases Include="CompanyABCTransactions">
    <Name>CompanyABCTransactions</Name>
    <DatabaseName>$(DBPrefix)CompanyABCTransactions</DatabaseName>
    <CanRestore>true</CanRestore>
    <CanBackup>true</CanBackup>
  </Databases>
</ItemGroup>



<Delete Condition=" '$(DbRestore)' != '' " Files="$(DatabaseFolder)\%(Databases.Name).bak" />
<Exec Condition=" '$(DbRestore)' != '' AND '%(Databases.CanRestore)' == 'true' " WorkingDirectory="$(ExternsMSBuildFolder)\ExecTasks\7-Zip\" Command="&quot;$(ExternsMSBuildFolder)\ExecTasks\7-Zip\7z.exe&quot; e &quot;$(DatabaseFolder)\%(Databases.Name).zip&quot; -o&quot;$(DatabaseFolder)&quot; *.* -r" />
<DBChangeManagement.MSBuildTask.MSSQLTask Condition=" '$(DbRestore)' != '' AND '%(Databases.CanRestore)' == 'true' "
  TaskConfiguration="$(TemporaryFolder)\MSBuild.config"
  EnvironmentName="$(DeployServer)"
  ConnectionStringKey="dbhReader"
  BackupLocation="$(DatabaseFolder)"
  IsBackupEnabled="false"
  RestoreFile="$(DatabaseFolder)\%(Databases.Name).bak"
  IsRestoreEnabled="true"
  IsRestoreEnabledAndRestoreFileEmptyCreateNewDatabase="true"
  RestoreDatabaseAs="%(Databases.DatabaseName)"
  UpdateScriptsLocation="$(DatabaseFolder)\%(Databases.Name)"
/>
<Delete Condition=" '$(DbRestore)' != '' " Files="$(DatabaseFolder)\%(Databases.Name).bak" />

        

Functions

To mimic functions in MSBuild it requires creating a separate file. I normally prefix this with an underscore. In this sample I have created a file ‘CrossConcern_Entlib.proj’ inside the MSBuild directory, checked into version control.

I then use batching to execute the MSBuild task for each element in the ItemGroup. Be aware that the MSBuild task executes the ‘CrossConcern_Entlib.proj’ outside the scope of the current build engine. So do not forget to pass required parameters using the Properties attribute on the MSBuild task.

          <Target Name="ModuleSync">
	<ItemGroup>
		<Modules Include="SecurityModule">
			<ModuleName>SecurityModule</ModuleName>
			<ServiceReferencesPath></ServiceReferencesPath>
			<WebConfigPath>$(Host)\Web.$(ConfigurationNameSuffix)</WebConfigPath>
			<SvcPath>$(Host)\Services</SvcPath>
		</Modules>
		<Modules Include="XyzModule">
			<ModuleName>XyzModule</ModuleName>
			<ServiceReferencesPath></ServiceReferencesPath>
			<WebConfigPath>$(Host)\Web.$(ConfigurationNameSuffix)</WebConfigPath>
			<SvcPath>$(Host)\Services</SvcPath>
		</Modules>
	</ItemGroup>

	<MSBuild Projects="$(ExternsMSBuildFolder)\Module.proj"
		Targets="ModuleSync"
		Properties="ModuleName=%(Modules.ModuleName);
		TemporaryFolder=$(TemporaryFolder);
		MyMSBuildFolder=$(MyMSBuildFolder);
		SvcPath=%(SvcPath);
		ServiceReferencesPath=%(ServiceReferencesPath);
		WebConfigPath=%(WebConfigPath);
		FlexibleConfigTaskDevIdName=DevId;
		FlexibleConfigTaskDevId=$(DevId);
		FlexibleConfigTaskDevEnvironmentName=DevEnvironment;
		FlexibleConfigTaskDevEnvironment=$(DevEnvironment);
		FlexibleConfigTaskDevTaskName=DevTask;
		FlexibleConfigTaskDevTask=$(DevTask)" />
</Target>

        

Scope

If you run into scope issues, then you probably need to move to implicit resolution of dependencies, rather than explicit with CallTargets.

The target IdentifyBuildVariables is reponsible for appending the ItemGroup FlexibleConfigParameters. Since it is called using DependsOnTargets from the Target UpdateAfterSvn, the SyncConfiguration will be able to use the appended values.

If it had been called using only a CallTarget task, then the ItemGroup FlexibleConfigParameters would not have been appended.

          <ItemGroup>
	<FlexibleConfigParameters Include="MSBuildStartupDirectory">
	  <Key>MSBuildStartupDirectory</Key>
	  <Value>$(MyMSBuildFolder)</Value>
	</FlexibleConfigParameters>
</ItemGroup>

<Target Name="IdentifyBuildVariables">
        <Microsoft.Sdc.Tasks.GetEnvironmentVariable Condition="'$(IsInTeamBuild)' == ''" Variable="RobustHavenDevId" Target="User">
            <Output PropertyName="RobustHavenDevId" TaskParameter="Value" />
	</Microsoft.Sdc.Tasks.GetEnvironmentVariable>
        <Microsoft.Sdc.Tasks.GetEnvironmentVariable Condition="'$(IsInTeamBuild)' == ''" Variable="RobustHavenDevLocation" Target="User">
            <Output PropertyName="RobustHavenDevLocation" TaskParameter="Value" />
	</Microsoft.Sdc.Tasks.GetEnvironmentVariable>
        <Microsoft.Sdc.Tasks.GetEnvironmentVariable Condition="'$(IsInTeamBuild)' == ''" Variable="RobustHavenDevTask" Target="User">
            <Output PropertyName="RobustHavenDevTask" TaskParameter="Value" />
	</Microsoft.Sdc.Tasks.GetEnvironmentVariable>

	<Message Text=" $(RobustHavenDevId) $(RobustHavenDevLocation) $(RobustHavenDevTask)" />

	<ItemGroup>
		<FlexibleConfigParameters Include="RobustHavenDevId">
		  <Key>RobustHavenDevId</Key>
		  <Value>$(RobustHavenDevId)</Value>
		</FlexibleConfigParameters>
		<FlexibleConfigParameters Include="RobustHavenDevLocation">
		  <Key>RobustHavenDevLocation</Key>
		  <Value>$(RobustHavenDevLocation)</Value>
		</FlexibleConfigParameters>
		<FlexibleConfigParameters Include="RobustHavenDevTask">
		  <Key>RobustHavenDevTask</Key>
		  <Value>$(RobustHavenDevTask)</Value>
		</FlexibleConfigParameters>
	</ItemGroup>
</Target>


<Target Name="UpdateAfterSvn" DependsOnTargets="SyncConfiguration; DatabaseChangesSync">
	<Exec Command="&quot;$(AltovaXmlTask)&quot;  /in &quot;%(WebSites.WebConfigFile)&quot; /xslt2 &quot;$(ExternsMSBuildFolder)\ExecTasks\AltovaXmlIdentity.xslt&quot; > &quot;%(WebSites.WebConfigFile)-tmp.config&quot;" />
</Target>

        

Exception Handling

ContinueOnError is a property on any Task Element. If an error occurs and ContinueOnError is true, then the build will not halt.

OnError is in case a Task error occurs.

          <Target Name="RollingBackup">

	<Delete ContinueOnError="true" Files="@(DeleteFiles)" />

	<!-- tasks that can fail -->

	<OnError ExecuteTargets="HandleErrors" />

</Target>

<!-- Handle errors -->
<Target Name="HandleErrors">
	<GetRHSettingValue
		TaskConfiguration="$(ConfigFileName)"
		Key="error.email.body">
		<Output TaskParameter="Value" PropertyName="ErrorEmailBody"/>
	</GetRHSettingValue>
	<GetRHSettingValue
		TaskConfiguration="$(ConfigFileName)"
		Key="error.email.to">
		<Output TaskParameter="Value" PropertyName="ErrorEmailTo"/>
	</GetRHSettingValue>
	<Email
		TaskConfiguration="$(ConfigFileName)"
		To="$(ErrorEmailTo)"
		Subject="Backup error"
		Body="$(ErrorEmailBody)"/>
</Target>

        

Logging

The bat file template provided above shows how logging parameters can be passed to the MSBuild.exe. You can also create custom loggers, and specify type and assembly, containing the class implementing the logging interface.

           /f1 /f1p:v=diag;logfile=run-after-update.log /f2 /f2p:v=diag;logfile=run-after-update.log /f3 /f3p:v=diag;logfile=run-after-update.log .. you get the picture

        

In a custom task, I recommend having the task implement your logging interface. So that you can avoid cluttering the code with MSBuild host specific interfaces.

          public class MSSQLTask : Task, ILogger
{
private readonly ILogger _logger;

public MSSQLTask()
{
	_logger = this;
	_fileSystem = new FileSystem();
}

public void LogErrorFromException(Exception exception)
{
	Log.LogErrorFromException(exception);
	Exception ex = exception.InnerException;
	for (int i = 1; (ex != null); ex = ex.InnerException, i++)
	{
		LogError(string.Format("{0}->{1}", "".PadLeft(i*2), ex.Message));
	}
}
}

        

Debugging

Using diagnostic logging, you can determine where the build failed. However, as with most scripting languages, I find myself using the Message task to debug values. In the “Popular Articles” I provide a link on how you can debug with Visual Studio. If you are writing a custom Task you should always use the following to launch Visual Studio when executing the task.

           System.Diagnostics.Debugger.Launch();

        

The could be wrapped with Preprocessor directives for DEBUG and RELEASE builds.

Common Tasks

On a day to day basis, I recommend first looking to see if your goal is described in the MSBuild Task Reference.

Publish website to an output folder.

          <Target Name="Package">
	<MSBuild Projects="%(WebSites.SolutionFile)"
		Properties="Configuration=Release;
		DeployOnBuild=true;
		PackageAsSingleFile=false;
		AutoParameterizationWebConfigConnectionStrings=false;
		outdir=$(OutputFolder)\WwwTmp\%(WebSites.ApplicationId)\"
		Targets="Clean;Rebuild" />


	<ItemGroup>
		<WebsiteFiles Include="$(OutputFolder)\WwwTmp\%(WebSites.ApplicationId)\_PublishedWebsites\%(WebSites.DeployName)\**\*.*">
			<DirName>%(WebSites.DeployName)</DirName>
		</WebsiteFiles>
		<WebsiteFiles Remove="$(OutputFolder)\WwwTmp\%(WebSites.ApplicationId)\_PublishedWebsites\%(WebSites.DeployName)\bin\**\*.pdb" />
		<WebsiteFiles Remove="$(OutputFolder)\WwwTmp\%(WebSites.ApplicationId)\_PublishedWebsites\%(WebSites.DeployName)\bin\**\*.xml" />
	</ItemGroup>
	<Copy
            SourceFiles="@(WebsiteFiles)"
            DestinationFiles="@(WebsiteFiles->'$(OutputFolder)\Www\%(DirName)\%(RecursiveDir)%(Filename)%(Extension)')"
        />
</Target>

        

If you are using MvcContrib portable areas, then you are going want to add this to the web application .csproj file.

            <Target Name="BeforeBuild">
    <ItemGroup>
      <EmbeddedResource Include="**\*.aspx;**\*.ascx;**\*.gif;**\*.png;**\*.jpg;**\*.css;**\*.js" />
    </ItemGroup>
  </Target>

        

Community Tools & Reference

Popular Articles