Archive for the ‘Code Coverage’ Tag

NAnt and Unit Testing: Create unit tests for NAnt projects – build files – custom tasks/functions

Since the project i’m working has a LOT of build NAnt scritps, and we are at a crucial time in our development, every change at the build scripts itself, custom made Functions and Tasks, could(read it can) break a existing piece of script, create a bug, harm or prevent a developer work, keep the QA team without a updated deploy etc.

To prevent this i thought: HEY, lets unit test our build process! 😀

We are trying to increase our Unit Tests coverage but one place that we did not had any was at the build process itself. I searched everywhere how to do this but did not found a proper answer. I downloaded the NAnt source code and after some investigation got to the following conclusion:

You can execute NAnt inside a #PLACE_YOUR_FAVORITE_UNIT_TESTING_FRAMEWORK_FIRST_LETTER_HERE#_Unit test or any .NET application just by doing the following:

* Choose your unit tests framework.
* Add a reference to your unit tests framework of choice assemblies.
* Add a reference to the NAnt.Core assembly (has all the important classes like Project, Task, etc)
* Add a reference to the NAnt.Win32Tasks (required because it uses the readregistry task inside this assembly to retrieve the .NET desired framework configuration at the registry).
* Add a reference to the additional assemblies you need (Your custom task assembly, other tasks you use on your project under task, etc…)

* Add to the App.config of your unit tests project the nant required configuration. I just got the corresponding framework (we here use the .NET 4.0) config from the nant.exe.config file that ships with the latest nant version. I added the other configuration sections also (log, assembly probe, etc).

<?xml version="1.0"?>
<configuration>
  <!-- Leave this alone. Sets up configsectionhandler section -->
  <configSections>
    <section name="nant" type="NAnt.Core.ConfigurationSection, NAnt.Core" />
    <section name="log4net" type="System.Configuration.IgnoreSectionHandler" />
  </configSections>
  <appSettings>
    <!-- Used to indicate the location of the cache folder for shadow files -->
    <add key="shadowfiles.path" value="%temp%\nunit20\ShadowCopyCache" />
    <!-- Used to indicate that NAnt should shadow copy files in a cache folder near the executable -->
    <add key="nant.shadowfiles" value="False" />
    <!-- Used to indicate if cached files should be deleted when done running-->
    <add key="nant.shadowfiles.cleanup" value="False" />
    <!-- To enable internal log4net logging, uncomment the next line -->
    <!-- <add key="log4net.Internal.Debug" value="true"/> -->
  </appSettings>
  <!-- nant config settings -->
  <nant>
    <frameworks>
      <platform name="win32" default="auto">
        <task-assemblies>
          <!-- include NAnt task assemblies -->
          <include name="*Tasks.dll" />
          <!-- include NAnt test assemblies -->
          <include name="*Tests.dll" />
          <!-- include framework-neutral assemblies -->
          <include name="extensions/common/neutral/**/*.dll" />
          <!-- exclude Microsoft.NET specific task assembly -->
          <exclude name="NAnt.MSNetTasks.dll" />
          <!-- exclude Microsoft.NET specific test assembly -->
          <exclude name="NAnt.MSNet.Tests.dll" />
        </task-assemblies>
        <framework
               name="net-4.0"
               family="net"
               version="4.0"
               description="Microsoft .NET Framework 4.0"
               sdkdirectory="${sdkInstallRoot}"
               frameworkdirectory="${path::combine(installRoot, 'v4.0.30319')}"
               frameworkassemblydirectory="${path::combine(installRoot, 'v4.0.30319')}"
               clrversion="4.0.30319"
               clrtype="Desktop"
               vendor="Microsoft"
                    >
          <runtime>
            <probing-paths>
              <directory name="lib/common/2.0" />
              <directory name="lib/common/neutral" />
            </probing-paths>
            <modes>
              <strict>
                <environment>
                  <variable name="COMPLUS_VERSION" value="v4.0.30319" />
                </environment>
              </strict>
            </modes>
          </runtime>
          <reference-assemblies basedir="${path::combine(installRoot, 'v4.0.30319')}">
            <include name="Accessibility.dll" />
            <include name="Microsoft.Build.Conversion.v4.0.dll" />
            <include name="Microsoft.Build.dll" />
            <include name="Microsoft.Build.Engine.dll" />
            <include name="Microsoft.Build.Framework.dll" />
            <include name="Microsoft.Build.Tasks.v4.0.dll" />
            <include name="Microsoft.Build.Utilities.v4.0.dll" />
            <include name="Microsoft.CSharp.dll" />
            <include name="Microsoft.Data.Entity.Build.Tasks.dll" />
            <include name="Microsoft.JScript.dll" />
            <include name="Microsoft.Transactions.Bridge.dll" />
            <include name="Microsoft.Transactions.Bridge.Dtc.dll" />
            <include name="Microsoft.VisualBasic.Activities.Compiler.dll" />
            <include name="Microsoft.VisualBasic.Compatibility.Data.dll" />
            <include name="Microsoft.VisualBasic.Compatibility.dll" />
            <include name="Microsoft.VisualBasic.dll" />
            <include name="Microsoft.VisualC.dll" />
            <include name="Microsoft.VisualC.STLCLR.dll" />
            <include name="mscorlib.dll" />
            <include name="System.Activities.Core.Presentation.dll" />
            <include name="System.Activities.dll" />
            <include name="System.Activities.DurableInstancing.dll" />
            <include name="System.Activities.Presentation.dll" />
            <include name="System.AddIn.Contract" />
            <include name="System.AddIn.dll" />
            <include name="System.ComponentModel.Composition.dll" />
            <include name="System.ComponentModel.DataAnnotations.dll" />
            <include name="System.Configuration.dll" />
            <include name="System.Configuration.Install.dll" />
            <include name="System.Core.dll" />
            <include name="System.Data.DataSetExtensions.dll" />
            <include name="System.Data.dll" />
            <include name="System.Data.Entity.Design.dll" />
            <include name="System.Data.Entity.dll" />
            <include name="System.Data.Linq.dll" />
            <include name="System.Data.OracleClient.dll" />
            <include name="System.Data.Services.Client.dll" />
            <include name="System.Data.Services.Design.dll" />
            <include name="System.Data.Services.dll" />
            <include name="System.Data.SqlXml.dll" />
            <include name="System.Deployment.dll" />
            <include name="System.Design.dll" />
            <include name="System.Device.dll" />
            <include name="System.DirectoryServices.dll" />
            <include name="System.DirectoryServices.Protocols.dll" />
            <include name="System.dll" />
            <include name="System.Drawing.Design.dll" />
            <include name="System.Drawing.dll" />
            <include name="System.Dynamic.dll" />
            <include name="System.EnterpriseServices.dll" />
            <include name="System.EnterpriseServices.Thunk.dll" />
            <include name="System.EnterpriseServices.Wrapper.dll" />
            <include name="System.IdentityModel.dll" />
            <include name="System.IdentityModel.Selectors.dll" />
            <include name="System.IO.Log.dll" />
            <include name="System.Management.dll" />
            <include name="System.Management.Instrumentation.dll" />
            <include name="System.Messaging.dll" />
            <include name="System.Net.dll" />
            <include name="System.Numerics.dll" />
            <include name="System.Runtime.Caching.dll" />
            <include name="System.Runtime.DurableInstancing.dll" />
            <include name="System.Runtime.Remoting.dll" />
            <include name="System.Runtime.Serialization.dll" />
            <include name="System.Runtime.Serialization.Formatters.Soap.dll" />
            <include name="System.Security.dll" />
            <include name="System.ServiceModel.Activation.dll" />
            <include name="System.ServiceModel.Activities.dll" />
            <include name="System.ServiceModel.Channels.dll" />
            <include name="System.ServiceModel.Discovery.dll" />
            <include name="System.ServiceModel.dll" />
            <include name="System.ServiceModel.Routing.dll" />
            <include name="System.ServiceModel.ServiceMoniker40.dll" />
            <include name="System.ServiceModel.WasHosting.dll" />
            <include name="System.ServiceModel.Web.dll" />
            <include name="System.ServiceProcess.dll" />
            <include name="System.Transactions.dll" />
            <include name="System.Web.Abstractions.dll" />
            <include name="System.Web.ApplicationServices.dll" />
            <include name="System.Web.DataVisualization.Design.dll" />
            <include name="System.Web.DataVisualization.dll" />
            <include name="System.Web.dll" />
            <include name="System.Web.DynamicData.Design.dll" />
            <include name="System.Web.DynamicData.dll" />
            <include name="System.Web.Entity.Design.dll" />
            <include name="System.Web.Entity.dll" />
            <include name="System.Web.Extensions.Design.dll" />
            <include name="System.Web.Extensions.dll" />
            <include name="System.Web.Mobile.dll" />
            <include name="System.Web.RegularExpressions.dll" />
            <include name="System.Web.Routing.dll" />
            <include name="System.Web.Services.dll" />
            <include name="System.Windows.Forms.DataVisualization.Design.dll" />
            <include name="System.Windows.Forms.DataVisualization.dll" />
            <include name="System.Windows.Forms.dll" />
            <include name="System.Workflow.Activities.dll" />
            <include name="System.Workflow.ComponentModel.dll" />
            <include name="System.Workflow.Runtime.dll" />
            <include name="System.WorkflowServices.dll" />
            <include name="System.Xaml.dll" />
            <include name="System.Xaml.Hosting.dll" />
            <include name="System.Xml.dll" />
            <include name="System.Xml.Linq.dll" />
          </reference-assemblies>
          <reference-assemblies basedir="${environment::get-folder-path('ProgramFiles')}/Reference Assemblies/Microsoft/Framework/.NETFramework/v4.0">
            <include name="Microsoft.Build.Conversion.v4.0.dll" />
            <include name="Microsoft.Build.dll" />
            <include name="Microsoft.Build.Engine.dll" />
            <include name="Microsoft.Build.Framework.dll" />
            <include name="Microsoft.Build.Tasks.v4.0.dll" />
            <include name="Microsoft.Build.Utilities.v4.0.dll" />
            <include name="Microsoft.CSharp.dll" />
            <include name="Microsoft.JScript.dll" />
            <include name="Microsoft.VisualBasic.Compatibility.Data.dll" />
            <include name="Microsoft.VisualBasic.Comptatibility.dll" />
            <include name="Microsoft.VisualBasic.dll" />
            <include name="Microsoft.VisualC.dll" />
            <include name="Microsoft.VisualC.STLCLR.dll" />
            <include name="mscorlib.dll" />
            <include name="PresentationBuildTasks.dll" />
            <include name="PresentationCore.dll" />
            <include name="PresentationFramework.Aero.dll" />
            <include name="PresentationFramework.Classic.dll" />
            <include name="PresentationFramework.Luna.dll" />
            <include name="PresentationFramework.Royale.dll" />
            <include name="ReachFramework.dll" />
            <include name="System.Activities.Core.Presentation.dll" />
            <include name="System.Activities.dll" />
            <include name="System.Activities.DurableInstancing.dll" />
            <include name="System.Activities.Presentation.dll" />
            <include name="System.AddIn.Contract.dll" />
            <include name="System.AddIn.dll" />
            <include name="System.ComponentModel.Composition.dll" />
            <include name="System.ComponentModel.DataAnnotations.dll" />
            <include name="System.Configuration.dll" />
            <include name="System.Core.dll" />
            <include name="System.Data.DataSetExtension.dll" />
            <include name="System.Data.dll" />
            <include name="System.Data.Entity.Design.dll" />
            <include name="System.Data.Entity.dll" />
            <include name="System.Data.Linq.dll" />
            <include name="System.Data.OracleClient.dll" />
            <include name="System.Data.Services.Client.dll" />
            <include name="System.Data.Services.Design.dll" />
            <include name="System.Data.Services.dll" />
            <include name="System.Data.SqlXml.dll" />
            <include name="System.Deployment.dll" />
            <include name="System.Design.dll" />
            <include name="System.Device.dll" />
            <include name="System.DirectoryServices.AccountManagement.dll" />
            <include name="System.DirectoryServices.dll" />
            <include name="System.DirectoryServices.Protocols.dll" />
            <include name="System.dll" />
            <include name="System.Drawing.Design.dll" />
            <include name="System.Drawing.dll" />
            <include name="System.EnterpriseServices.dll" />
            <include name="System.EnterpriseServices.Thunk.dll" />
            <include name="System.EnterpriseServices.Wrapper.dll" />
            <include name="System.IdentityModel.dll" />
            <include name="System.IdentityModel.Selectors.dll" />
            <include name="System.IO.Log.dll" />
            <include name="System.Management.dll" />
            <include name="System.Management.Instrumentation.dll" />
            <include name="System.Messaging.dll" />
            <include name="System.Net.dll" />
            <include name="System.Numerics.dll" />
            <include name="System.Printing.dll" />
            <include name="System.Runtime.Caching.dll" />
            <include name="System.Runtime.DurableInstancing.dll" />
            <include name="System.Runtime.Remoting.dll" />
            <include name="System.Runtime.Serialization.dll" />
            <include name="System.Runtime.Serialization.Formatters.Soap.dll" />
            <include name="System.Security.dll" />
            <include name="System.ServiceModel.Activation.dll" />
            <include name="System.ServiceModel.Activities.dll" />
            <include name="System.ServiceModel.Channels.dll" />
            <include name="System.ServiceModel.Discovery.dll" />
            <include name="System.ServiceModel.dll" />
            <include name="System.ServiceModel.Routing.dll" />
            <include name="System.ServiceModel.Web.dll" />
            <include name="System.ServiceProcess.dll" />
            <include name="System.Speech.dll" />
            <include name="System.Transactions.dll" />
            <include name="System.Web.Abstractions.dll" />
            <include name="System.Web.ApplicationServices.dll" />
            <include name="System.Web.DataVisualization.Design.dll" />
            <include name="System.Web.DataVisualization.dll" />
            <include name="System.Web.dll" />
            <include name="System.Web.DynamicData.Design.dll" />
            <include name="System.Web.DynamicData.dll" />
            <include name="System.Web.Entity.Design.dll" />
            <include name="System.Web.Entity.dll" />
            <include name="System.Web.Extensions.Design.dll" />
            <include name="System.Web.Extensions.dll" />
            <include name="System.Web.Mobile.dll" />
            <include name="System.Web.RegularExpressions.dll" />
            <include name="System.Web.Routing.dll" />
            <include name="System.Web.Services.dll" />
            <include name="System.Windows.Forms.DataVisualization.Design.dll" />
            <include name="System.Windows.Forms.DataVisualization.dll" />
            <include name="System.Windows.Forms.dll" />
            <include name="System.Windows.Input.Manipulations.dll" />
            <include name="System.Windows.Presentation.dll" />
            <include name="System.Workflow.Activities.dll" />
            <include name="System.Workflow.ComponentModel.dll" />
            <include name="System.Workflow.Runtime.dll" />
            <include name="System.WorkflowServices.dll" />
            <include name="System.Xaml.dll" />
            <include name="System.Xml.dll" />
            <include name="System.Xml.Linq.dll" />
            <include name="WindowsBase.dll" />
          </reference-assemblies>
          <task-assemblies>
            <!-- include MS.NET version-neutral assemblies -->
            <include name="extensions/net/neutral/**/*.dll" />
            <!-- include MS.NET 4.0 specific assemblies -->
            <include name="extensions/net/4.0/**/*.dll" />
            <!-- include MS.NET specific task assembly -->
            <include name="NAnt.MSNetTasks.dll" />
            <!-- include MS.NET specific test assembly -->
            <include name="NAnt.MSNet.Tests.dll" />
            <!-- include .NET 4.0 specific assemblies -->
            <include name="extensions/common/4.0/**/*.dll" />
          </task-assemblies>
          <tool-paths>
            <directory name="${sdkInstallRoot}"
                if="${property::exists('sdkInstallRoot')}" />
            <directory name="${path::combine(installRoot, 'v4.0.30319')}" />
          </tool-paths>
          <project>
            <readregistry
                property="installRoot"
                key="SOFTWARE\Microsoft\.NETFramework\InstallRoot"
                hive="LocalMachine" />
            <locatesdk property="sdkInstallRoot" minwinsdkver="v7.0A" minnetfxver="4.0" maxnetfxver="4.0.99999" failonerror="false" />
            <!--
                        <echo message="sdkInstallRoot=${sdkInstallRoot}" if="${property::exists('sdkInstallRoot')}" />
                        <readregistry
                            property="sdkInstallRoot"
                            key="SOFTWARE\Microsoft\Microsoft SDKs\Windows\v7.0A\WinSDK-NetFx40Tools\InstallationFolder"
                            hive="LocalMachine"
                            failonerror="false" />
                        <readregistry
                            property="sdkInstallRoot"
                            key="SOFTWARE\Microsoft\Microsoft SDKs\Windows\v7.0A\WinSDK-NetFx40Tools-x86\InstallationFolder"
                            hive="LocalMachine"
                            failonerror="false" />
                        -->
          </project>
          <tasks>
            <task name="csc">
              <attribute name="supportsnowarnlist">true</attribute>
              <attribute name="supportswarnaserrorlist">true</attribute>
              <attribute name="supportskeycontainer">true</attribute>
              <attribute name="supportskeyfile">true</attribute>
              <attribute name="supportsdelaysign">true</attribute>
              <attribute name="supportsplatform">true</attribute>
              <attribute name="supportslangversion">true</attribute>
            </task>
            <task name="vbc">
              <attribute name="supportsdocgeneration">true</attribute>
              <attribute name="supportsnostdlib">true</attribute>
              <attribute name="supportsnowarnlist">true</attribute>
              <attribute name="supportskeycontainer">true</attribute>
              <attribute name="supportskeyfile">true</attribute>
              <attribute name="supportsdelaysign">true</attribute>
              <attribute name="supportsplatform">true</attribute>
              <attribute name="supportswarnaserrorlist">true</attribute>
            </task>
            <task name="jsc">
              <attribute name="supportsplatform">true</attribute>
            </task>
            <task name="vjc">
              <attribute name="supportsnowarnlist">true</attribute>
              <attribute name="supportskeycontainer">true</attribute>
              <attribute name="supportskeyfile">true</attribute>
              <attribute name="supportsdelaysign">true</attribute>
            </task>
            <task name="resgen">
              <attribute name="supportsassemblyreferences">true</attribute>
              <attribute name="supportsexternalfilereferences">true</attribute>
            </task>
            <task name="delay-sign">
              <attribute name="exename">sn</attribute>
            </task>
            <task name="license">
              <attribute name="exename">lc</attribute>
              <attribute name="supportsassemblyreferences">true</attribute>
            </task>
          </tasks>
        </framework>
      </platform>
    </frameworks>
    <properties>
      <!-- properties defined here are accessible to all build files -->
      <!-- <property name="foo" value = "bar" readonly="false" /> -->
    </properties>
  </nant>
  <!--
        This section contains the log4net configuration settings.

        By default, no messages will be logged to the log4net logging infrastructure.

        To enable the internal logging, set the threshold attribute on the log4net element
        to "ALL".

        When internal logging is enabled, internal messages will be written to the 
        console.
    -->
  <log4net threshold="OFF">
    <appender name="ConsoleAppender" type="log4net.Appender.ConsoleAppender">
      <layout type="log4net.Layout.PatternLayout">
        <param name="ConversionPattern" value="[%c{2}:%m  - [%x] &lt;%X{auth}&gt;]%n" />
      </layout>
    </appender>
    <appender name="RollingLogFileAppender" type="log4net.Appender.RollingFileAppender">
      <param name="File" value="${APPDATA}\\NAnt\\NAnt.log" />
      <param name="AppendToFile" value="true" />
      <param name="MaxSizeRollBackups" value="2" />
      <param name="MaximumFileSize" value="500KB" />
      <param name="RollingStyle" value="Size" />
      <param name="StaticLogFileName" value="true" />
      <layout type="log4net.Layout.PatternLayout">
        <param name="ConversionPattern" value="[%c{2}:%m  - [%x] &lt;%X{auth}&gt;]%n" />
      </layout>
    </appender>
    <!-- Setup the root category, add the appenders and set the default level -->
    <root>
      <!-- Only log messages with severity ERROR (or higher) -->
      <level value="ERROR" />
      <!-- Log messages to the console -->
      <appender-ref ref="ConsoleAppender" />
      <!-- Uncomment the next line to enable logging messages to the NAnt.log file -->
      <!-- <appender-ref ref="RollingLogFileAppender" /> -->
    </root>
    <!-- Specify the priority for some specific categories -->
    <!--
        <logger name="NAnt.Core.TaskBuilderCollection">
            <level value="DEBUG" />
        </logger>
        <logger name="NAnt">
            <level value="INFO" />
        </logger>
        -->
  </log4net>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <probing privatePath="lib" />
    </assemblyBinding>
    <NetFx40_LegacySecurityPolicy enabled="true"/>
  </runtime>
  <startup useLegacyV2RuntimeActivationPolicy="true">
    <!-- .NET Framework 4.0 -->
    <supportedRuntime version="v4.0.30319" />
    <!-- .NET Framework 2.0 -->
    <supportedRuntime version="v2.0.50727" />
    <!-- .NET Framework 1.1 -->
    <supportedRuntime version="v1.1.4322" />
    <!-- .NET Framework 1.0 -->
    <supportedRuntime version="v1.0.3705" />
  </startup>
</configuration>

Wow, thats a lot… i don’t know if they are all required, but better safe than sorry. If you use other framework version, just check the nant.exe.config file because they have all.

Here is an example of a test using an existing build file and NUnit:

[Test]
public void TestNAntBuildFile()
{
    Project proj = new Project("PATH_TO_MY_NANT_PROJECT_FILE.build", Level.Debug, 0);
    proj.Run();

    proj.Execute("TestTarget");

    // Do some asserts
    Assert.AreEqual(.....
}

Here is how you can create the build file on the fly:

[Test]
public void TestCreatedNAntBuildFile()
{
    XmlDocument doc = new XmlDocument();
    doc.LoadXml("<project><target name=\"TestTarget\"><echo message=\"NAnt and Unit tests. A nice couple.\"/></target></project>");

    Project proj = new Project(doc, Level.Debug, 0);
    proj.Run();

    proj.Execute("TestTarget");

    // Do some asserts
    Assert.AreEqual(.....
}

Hope this helps a lot of automation lovers out there! 😀

Advertisements