Archive for the ‘.NET’ Tag

Easier automated builds and continuous integration with Naven: A .NET port of Maven

I love Apache Maven. I really do. It increased my productivity and quality on java projects so much! I stopped worring about the small and repetitive details about my build process.

But here in 4N1, we have a lot of projects in .NET, and we could not find a good, easy and free tool like Maven for the .NET world, so we decided to start our own.

Naven comes to the rescue!!!

Naven is a port of the Maven concept, meaning that you don’t need to specify a build process, you just need to inform how your application is, and Naven will “create” the build process for you. Implemented as a command line application (but with the real logic in a C# 4.0 API), Naven can be used as a build automation tool and executed inside a continuous integration server, in fact, here in our CI server(Jenkins) we use it already.

We also did Naven to help spread build automation and CI concepts, making it easier for someone to automate their build process.

Apache Maven has a POM (Project Object Model) to define the caracteristics of your projects. Since the .NET world already have a POM (the *.csproj and such files), Naven only needs a AOM (Application Object Model). The AOM is a file that contains the basic information about your application.

Components

Basically, a Naven application has components. The currently supported are:

  • Solution: References an existing .Net solution.
  • Project: References an existing .Net project.

Phases

In the Naven world, a build process is divided in phases. Phases are nothing more that steps of a build process. The currently supported are:

  • Initialize: Deserializes the AOM file and initialize all necessary objects.
  • Validate: Validates the AOM file looking for errors (EX: pointing to an invalid solution, a non existent project file, etc)
  • Resolve Dependencies: Since version 0.2.0, Naven integrates with Nuget, meaning that if you have projects that specify their dependencies in a Nuget file (package.config) it will install the packages for you.
  • Compile: Using MSBuild, executes the compilation of the components of your application.
  • Tests: Using NUnit, Executes the automated tests of your application.

Besides the “out-of-the-box” phases, you can extend the Naven build process using NAnt, on the “custom phases”. They happen before and after their respective phases:

  • AfterInit
  • BeforeValidate
  • AfterValidate
  • BeforeResolve
  • AfterResolve
  • BeforeCompile
  • AfterCompile
  • BeforeTest
  • AfterTest

Goals

A Goal is something that you want Naven to do with that component. The currently supported are:

  • Resolve: Tells Naven to resolve the dependencies of the component. In the current version you don’t need to specify this Goal because it will process all “packages.config” files found.
  • Compile: Tells Naven that the component should be compiled.
  • Test: Tells Naven that the component have tests that should be executed.

NOTE: In the current version you need to manually specify the Goals, but we are planning to apply the concept of Convention over Configuration, and try to figure it out what are the expected Goals for a particular component, for example, adding the Test goal automatically for all projects with the name ending with “.Tests”.

Here is an example of a concrete Naven AOM file:

<?xml version="1.0" encoding="utf-8"?>
<DotNetApplication xmlns="http://naven.4n1.pt/aom" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://naven.4n1.pt/aom http://naven.4n1.pt/0.3/aom.xsd">

  <!-- Application name -->
  <Name>Sample App Name</Name>

  <!-- Application description -->
  <Description>This is just a sample application.</Description>

  <!-- Path where Nuget packages will be installed -->
  <NugetPackagesFolder>packages</NugetPackagesFolder>

  <Components>

    <!-- Represents an existing solution -->
    <Solution Path="Solution.sln">
      <Goals>

        <!-- Tells naven to compile this solution. Normally this would be the solution that contains all the projects of your application. Here in <a href="http://www.4N1.pt">4N1</a>, we normally have a "main" solution, will all the projects. Its easier and faster to compile a lot of related projects. -->
        <Compile/>
      </Goals>
    </Solution>

    <!-- Represents an existing project -->
    <Project Path="SampleProject.Tests\SampleProject.Tests.csproj">
      <Goals>

        <!-- Tells naven to run the tests on this project. Naven will look on the .csproj file to find out
	where the compiled assembly is.-->
        <Test/>
      </Goals>
    </Project>

  </Components>
  <Phases>
    <AfterInit>
      <!-- Inside a "custom phase", you can use all NAnt tasks (as long as you include their assembly in the same folder Naven executable) -->
      <echo message="This is happening after Init"/>
      <echo message="This is also happening after Init"/>
      <property name="Environment" value="SomeValue"/>
      <echo message="This is the environment: ${Environment}"/>
    </AfterInit>
  </Phases>
</DotNetApplication>

How to use it?

Just grab the latest release(v 0.3.0) on the Naven repo, unzip it, create your Naven AOM file and execute the command line, like this:

  • nvn.exe PATH_TO_THE_AOM_FILE Resolve
  • – To resolve the dependencies of your application.
    or

  • nvn.exe PATH_TO_THE_AOM_FILE Compile
  • – To compile your application.
    or

  • nvn.exe PATH_TO_THE_AOM_FILE Test
  • – To compile and test your application.

Properties Support

When extending Naven native build process with NAnt, you may need to supply additional properties values when invoking Naven by command line. Just like NAnt /D:PropertyName=PropertyValue switch, you have a similar mechanism in Naven:

  • nvn PATH_TO_FILE GOAL /p=PropertyOne:ValueOne /p=PropertyTwo:ValueTwo

On the “custom phases” implementation, you reference those defined properties just like normal NAnt properties:

<?xml version="1.0" encoding="utf-8"?>
<DotNetApplication xmlns="http://naven.4n1.pt/aom" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://naven.4n1.pt/aom http://naven.4n1.pt/0.3/aom.xsd">
  <Name>NAnt Support Application.</Name>
  <Description>An application that shows the NAnt support.</Description>
  <Phases>
    <AfterInit>
      <echo message="This are properties references: ${PropertyOne}, ${PropertyTwo}"/>
    </AfterInit>
  </Phases>
</DotNetApplication>

What are the next planned features?

Naven has a lot way to go to get even just a little bit closer to the great tool Maven is. With that in mind, we have planned a few features already.

  • Convention support: By parsing the main solution file or searching for project files below the base folder.
  • Support other agents: An agent (for now) may be a DependencyResolver, a Compiler or a Tester. Naven code is made with dependency injection concepts, so we could change the default agents used. The ones available now are only:
    • MSBuild for compilation
    • Nuget for dependency resolution
    • NUnit for testing
  • Additional phases: We plan also to extend the Naven build process, to include some additional phases. Some of them may be:
    • Package: Creates a package of your projects/applications in Nuget format.
    • Install: Install the package in a package manager, probably a Nuget server for now.
    • Deploy: Deploy the application/project on a server.

For all this, we really would like to receive some help or opinions, specially because Naven is Open Source, and also because we hope it make your life easir. 😀 Feel free to check it out, download it, use it, and contribute. The 4N1 team thanks you.

The HUGE power of SSD’s while developing HUGE projects

A friend of mine (Jose Formiga, check out his blog) sent me the link to this article on the Coding Horror blog and i thought it was a nice opportunity to share the experience that we had while working with SSD’s.

Our development team uses a Dell Latitude E6510 Pentium core I7 4gb RAM with an HDD with 7200 RPM and we are working on a HUGE .NET 4.0 Application. Here are some stats:

  • Over 180 projects
  • Over 6 millions LOC’s (lines of code)
  • 8 Databases
  • Over 5 Gbs of source code

We have everything: ASP.NET MVC Front-Ends, Windows Services, WCF Services, JAVA Tools, you name it.

We also have our build and deploy processes fully automated. The steps are:

  1. Get the latest version from repository
  2. Execute the code-generation tool that we use
  3. Build all the projects that we have
  4. Create the database schema for all the databases
  5. Load the databases with some initial data
  6. Execute the unit tests
  7. Execute UI(Web) functional tests with another tool that we use
  8. Create the Deploy Package

The FULL execution (non-incremental, cleaning all compilation outputs, data, etc) of this build process takes about 25 minutes on the HDD.

When we bought a SSD (OCZ VERTEX 2) the same execution got down to less than 10 minutes.

I mean…. UOW!!! 😀

And not only that… Since most of the time we are using a LOT of programs simultaneously (Visual Studio 2010, Eclipse, SQL Management Studio, Web browsers, etc) we noticed that while on the HDD, even with a lot of free RAM memory, the computer would hang a few seconds from time to time (while changing between files on Visual Studio, creating a new tab on Chrome, etc.). With the SSD the response time of these applications dropped dramatically, and we all noticed a lot of improvement.

This build process its executed on average 5 times a day by every developer before they commit code because we don’t want to rely only on the CI server to detect possible problems but also encourage a more proactive error check. We believe this creates a better mood between the developers and reduce the time of instable code on the repository. With this in mind, here is some math:

WITHOUT SDD -> 5 * 25 = 125 minutes lost on build time by day.

WITH SSD -> 5 * 10 = 50 minutes lost on build time by day.

Again… UOW!!!

I just gained an extra hour a day to actually do something and not wait for the build process to finish.

No more excuses… (at least not that often)

I’m really sure that my Project Manager got quite happy about that.

If i consider that a month has 20 working days average, we could say that for each developer, we would gain around 20 hours a month. We have 20 developers so that makes 400 hours a month as a whole.

We spent around 2000 € to buy the same SSD (OCZ VERTEX 2) to everyone. Some would say that the speed of SSDs are not worth because of the lower life time, reliability, etc.

One developer of ours had his SSD completely burned (not literally, it just went dead and did not wake up :D). And other two that lost the MBR (Master Boot Record) while returning the laptop from hibernate. The hibernate problem can also happens to HDD’s, so we all just disabled it to prevent future loss and now only use the Sleep mode. There is a bunch of forums talking about that, just google it. We also disabled a lot of now unused services like Defrag, Indexing, etc. (Also, there are a lot of forums and articles about this because they lower the life time of an SSD because of unnecessary writes). On the other hand, the complete fail could not be explained and we had to send the SSD back. Thank god it was still on warranty. With this we have lost 3 days of those 3 developers total (1 day each) due to software installation on the new/reinstalled ones. We do have in mind that we have a READ and WRITE super intensive modus operandi and also always try to make commits as frequently as possible to avoid data loss.

We have been using the SSDs for almost 3 months. Again some more math:

3(Months) * 400(hours saved by the development team) = 1200

3(days of software installation because of the failed SSDs) * 8 (work hours per day) = 24

1200 – 24 = 1176 hours saved by the development team in 3 months.

20 (hours per month and per developer wasted on build time without SSD) * 20 (Average developer price per hour) = 400
SSD unit price = 200

Even if we bought two SSDs for developer each month, it would be cheaper for the employer!

Now you can imaginage how happy our Project Manager is :D.

We are not happy yet with the build time, and we are making some optimizations on it.

GO GO GO SSD’s!