Archive for the ‘Build’ Tag

Setup Jenkins on TomCat/Windows with Mercurial Repository in BitBucket over SSH

This is just the list of steps that I’ve made to get a Windows build machine execute a Jenkins Job (running inside TomCat) clone a Mercurial repository hosted on BitBucket with SSH authentication.

  • Install TomCat (I’ve used the 7)
  • Install Jenkins on TomCat
  • Create a Windows User for executing the TomCat (Lets reference it as CI_USER, but you can name it as you like)
  • Set the TomCat process to execute with the CI_USER
  • Log in on the machine with the CI_USER
    • Download putty zip so that you can create a SSH key.
      • Generate a key using “PUTTYGEN.EXE”
      • Copy the Public-key and add it as a Deployment keys on your Repository settings on BitBucket.
      • Save the private key without a passphrase.
    • Install TortoiseHG on the Jenkins Machine.
    • Create a user Mercurial settings file on C:\Documents and Settings\CI_USER\Mercurial.ini (Look here for references)
    • Add this code to the file

      [ui]
      ssh=”C:\PATH_TO_TORTOISE_HG_INSTALL_ROOT\TortoisePlink.exe” -batch -i “C:\PATH_TO_PRIVATE_KEY_FILE_CREATED_WITH_PUTTYGEN.ppk”

    • I’m not really sure if this step is really mandatory, but do it just in case:
      • Manually clone your repo using TortoiseHG. Make sure you use the ssh url from bitbucket as source and some temporary folder as destination.
      • Accept Bitbucket certificate and add it to the cache (Click yes if any window appears regarding that)
    • Install the Mercurial plugin on Jenkins.
    • Go to jenkins and configure your job to point to the ssh url
    • Happy CI
    Advertisements

The HUGE power of SSD’s while developing HUGE projects

A friend of mine (Jose Formiga, check out his blog) sent me the link to this article on the Coding Horror blog and i thought it was a nice opportunity to share the experience that we had while working with SSD’s.

Our development team uses a Dell Latitude E6510 Pentium core I7 4gb RAM with an HDD with 7200 RPM and we are working on a HUGE .NET 4.0 Application. Here are some stats:

  • Over 180 projects
  • Over 6 millions LOC’s (lines of code)
  • 8 Databases
  • Over 5 Gbs of source code

We have everything: ASP.NET MVC Front-Ends, Windows Services, WCF Services, JAVA Tools, you name it.

We also have our build and deploy processes fully automated. The steps are:

  1. Get the latest version from repository
  2. Execute the code-generation tool that we use
  3. Build all the projects that we have
  4. Create the database schema for all the databases
  5. Load the databases with some initial data
  6. Execute the unit tests
  7. Execute UI(Web) functional tests with another tool that we use
  8. Create the Deploy Package

The FULL execution (non-incremental, cleaning all compilation outputs, data, etc) of this build process takes about 25 minutes on the HDD.

When we bought a SSD (OCZ VERTEX 2) the same execution got down to less than 10 minutes.

I mean…. UOW!!! 😀

And not only that… Since most of the time we are using a LOT of programs simultaneously (Visual Studio 2010, Eclipse, SQL Management Studio, Web browsers, etc) we noticed that while on the HDD, even with a lot of free RAM memory, the computer would hang a few seconds from time to time (while changing between files on Visual Studio, creating a new tab on Chrome, etc.). With the SSD the response time of these applications dropped dramatically, and we all noticed a lot of improvement.

This build process its executed on average 5 times a day by every developer before they commit code because we don’t want to rely only on the CI server to detect possible problems but also encourage a more proactive error check. We believe this creates a better mood between the developers and reduce the time of instable code on the repository. With this in mind, here is some math:

WITHOUT SDD -> 5 * 25 = 125 minutes lost on build time by day.

WITH SSD -> 5 * 10 = 50 minutes lost on build time by day.

Again… UOW!!!

I just gained an extra hour a day to actually do something and not wait for the build process to finish.

No more excuses… (at least not that often)

I’m really sure that my Project Manager got quite happy about that.

If i consider that a month has 20 working days average, we could say that for each developer, we would gain around 20 hours a month. We have 20 developers so that makes 400 hours a month as a whole.

We spent around 2000 € to buy the same SSD (OCZ VERTEX 2) to everyone. Some would say that the speed of SSDs are not worth because of the lower life time, reliability, etc.

One developer of ours had his SSD completely burned (not literally, it just went dead and did not wake up :D). And other two that lost the MBR (Master Boot Record) while returning the laptop from hibernate. The hibernate problem can also happens to HDD’s, so we all just disabled it to prevent future loss and now only use the Sleep mode. There is a bunch of forums talking about that, just google it. We also disabled a lot of now unused services like Defrag, Indexing, etc. (Also, there are a lot of forums and articles about this because they lower the life time of an SSD because of unnecessary writes). On the other hand, the complete fail could not be explained and we had to send the SSD back. Thank god it was still on warranty. With this we have lost 3 days of those 3 developers total (1 day each) due to software installation on the new/reinstalled ones. We do have in mind that we have a READ and WRITE super intensive modus operandi and also always try to make commits as frequently as possible to avoid data loss.

We have been using the SSDs for almost 3 months. Again some more math:

3(Months) * 400(hours saved by the development team) = 1200

3(days of software installation because of the failed SSDs) * 8 (work hours per day) = 24

1200 – 24 = 1176 hours saved by the development team in 3 months.

20 (hours per month and per developer wasted on build time without SSD) * 20 (Average developer price per hour) = 400
SSD unit price = 200

Even if we bought two SSDs for developer each month, it would be cheaper for the employer!

Now you can imaginage how happy our Project Manager is :D.

We are not happy yet with the build time, and we are making some optimizations on it.

GO GO GO SSD’s!