Development Blog

 Thursday, May 08, 2008

Just wanted to quickly note that I tracked down the performance issue in Rhino.Mocks and patched it. I also updated the original post with the new numbers. Enjoy!

by Aaron on Thursday, May 08, 2008 9:56:48 PM (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [0]  |  Trackback
UPDATE: I tracked down the issue and committed a patch to Rhino.Mocks. Rhino.Mocks is now much more competetive performance wise, our CI build time nearly halved, and about 4 minutes out of 7 of our test time has disappeared. New numbers below.

I've complained before that Mocking is Slow but I never really dove further into it. Today I decided to actually compare Rhino.Mocks to other mock frameworks on a pure performance basis to see if it was a global problem. I timed 2000 unit tests across 100 classes with 20 tests each. The results were a bit surprising:

Framework TD.NET Time nunit-console Time
Rhino.Mocks old trunk 57.36s 28.82s
Rhino.Mocks new trunk 22.94s 7.59s
Moq trunk 18.30s 5.91s
TypeMock 4.2.3 Reflective Mocks 15.36s 9.35s
TypeMock 4.2.3 Natural Mocks 16.92s 9.56s

That's right, according to these tests, Rhino.Mocks is at least 3 times slower than the other frameworks when under heavy load in TD.NET and five times slower in the console according to these tests. It's also interesting to note that TypeMock is faster than Moq in TD.NET, but slower in the console runner.

While running the Rhino.Mocks tests it is very clear that there is a degrading performance issue. All the other frameworks executed tests with a near constant speed per test, but Rhino.Mocks slowed down noticeably about half way through.

Please feel free to try it yourself, grab the project here. You should be able to just run the 4 strategy .bat files (run-rhino, run-moq, run-tmock-reflective, run-tmock-natural). Let me know if you find anything interesting.

by Aaron on Thursday, May 08, 2008 7:52:30 AM (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [0]  |  Trackback

As some of you who follow me on twitter know, I've been working on Yet Another Context/Specification Framework as an experiment. Yeah, I know we already have NSpec and NBehave, and they're great and all, but MSpec takes things on from a slightly different angle, and it's just an experiment (for now). Here's a sample Description:

[Description]
public class Transferring_between_from_account_and_to_account
{
  static Account fromAccount;
  static Account toAccount;

  Context before_each =()=>
  {
    fromAccount = new Account {Balance = 1m};
    toAccount = new Account {Balance = 1m};
  };
  
  When the_transfer_is_made =()=>
  {
    fromAccount.Transfer(1m, toAccount);
  };
   
  It should_debit_the_from_account_by_the_amount_transferred =()=>
  {
    fromAccount.Balance.ShouldEqual(0m);
  };

  It should_credit_the_to_account_by_the_amount_transferred =()=>
  {
    toAccount.Balance.ShouldEqual(2m);
  };
}

And a TestDriven.NET run:

------ Test started: Assembly: Machine.Specifications.Example.dll ------

Transferring between from account and to account
  When the transfer is made
    * It should debit the from account by the amount transferred
    * It should credit the to account by the amount transferred


2 passed, 0 failed, 0 skipped, took 0.79 seconds.

Err, What?

Different eh? The idea was heavily inspired by Scott Bellware's SpecUnit.Net framework he showed at the ALT.NET conference. It also took heavy cues from RSpec and my insanity. I realize that the the code doesn't look much like C# code and I'm OK with that. Many have and will ask why I don't just use Boo or RSpec w/ IronRuby eventually or even one of the existing Context/Spec/BDD frameworks. Those are good questions, but my main motivations are tooling and syntax. I enjoy the tooling I get in C# and I personally like the syntax in this library considering the limitations imposed by C#.

How's it work?

The simplest way to describe it is to compare it to a normal *Unit style testing framework:

  • Description = TestContext
  • Context before_each = SetUp
  • Context before_all = SetUpFixture
  • Context after_each = TearDown
  • Context after_all = TearDownFixture
  • When = Also SetUp, but happens after Context before_each
  • It = Test

Rather than methods and attributes, MSpec uses named delegates and anonymous functions. The only reason for this is readability. You'll also notice that the fields used in the context are static. This is necessary so that the anonymous functions in the field initializers can access them. Probably the first thing you noticed is the =()=> construct. I won't mention the names that this was given on twitter, but I think it's an acceptable thing to have to deal with in exchange for the cleanliness of the rest of the syntax.

Ok, you're crazy, but how do I try it?

First, this is a very rough cut. Everything is subject to change as we experiment with the language. That said, here's how you play with it:

  1. Grab the drop here.
  2. Extract it somewhere. Put it somewhere semi-permanent because the TestDriven.NET runner will need a static location for the MSpec TDNet Runner.
  3. If you want TestDriven.NET support, run InstallTDNetRunner.bat
  4. Check out the example in Machine.Specifications.Example. Note that you can run with TD.NET.
  5. Create a project of your own. Just add Machine.Specifications.dll and get started.
  6. Send me feedback! Leave comments, email me, tweet me, whatever.

Also, this is part of Machine, so feel free to take a look at the code and/or submit patches. There's also a Gallio adapter in there, but I didn't include it in the release as it's not quite polished enough yet. If you're interested in it, talk to me. Special thanks to Scott Bellware, Jeff Brown and Jamie Cansdale for their help and support.

by Aaron on Wednesday, May 07, 2008 11:11:27 PM (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [8]  |  Trackback
 Sunday, April 13, 2008

Currently in Rhino.Mocks, making mocks fire events and ensuring that an event on your SUT was fired are both awkward and verbose at best. Here is an example of both things at once:

[Test]
public void ViewFiresBeginDrag_Always_FiresChangedEvent()
{
  IEventRaiser raiser;
  bool eventFired = false;
  using (Record)
  {
    The<ICardView>().BeginDrag += null;
    raiser = LastCall.IgnoreArguments().GetEventRaiser();

    Target.Changed += (x,y) => eventFired = true;
  }

  using (Playback)
  {
    raiser.Raise(The<ICardView>(), EventArgs.Empty);
    Assert.IsTrue(eventFired);
  }
}

Nice eh? First, is there a better way to do either of these things that I'm missing? Please tell me if so. Next, if not, what can we do to clean this up?

Well, Ayende and I discussed this in the past and Ayende spiked it and asked for feedback. The feedback was mixed and for one reason or another it never made it into Rhino.Mocks that I know of.

Well, today while I was working on Machine.Testing and another side project, I decided to give something else a shot. What I ended up with is this:

[Test]
public void ViewFiresBeginDrag_Always_FiresChangedEvent()
{
  using (Record)
  {
    PrimeEventFiringOn<ICardView>(x => x.BeginDrag += null);
    Target.Changed += NewEventFireExpectation<EventHandler>();
  }

  using (Playback)
  {
    FireEventOn<ICardView>(EventArgs.Empty);
  }
}

Better, but I still don't think it's perfect. Also, it probably requires some explanation, so let's pick it apart piece by piece:

    PrimeEventFiringOn<ICardView>(x => x.BeginDrag += null);

=> +=??? Ugly huh? I really wish we could just refer to an event somehow without having to do +=/-=. At least I'm not using vb though... Alas, we cannot access them easily so we're stuck hacking away like this. So this particular method will basically get the mock or stub ICardView.BeginDrag ready to be fired. This needs to be done during the record phase it seems. You can only prime one event at a time per mock, so if you need to do more than one you can revert to the normal Rhino.Mocks syntax.

    Target.Changed += NewEventFireExpectation<EventHandler>();

This was a fun method. This method actually creates a new DynamicMethod in the signature required by the event and creates a new delegate. The method tracks whether or not it was fired, and at the end of the Playback phase in my fixture it will assert that all of the events were actually fired.

    FireEventOn<ICardView>(EventArgs.Empty);

This actually fires the event we set up in the Prime call. It can only be called after you've primed an event.

So, the whole thing is kind of "magic", but it's less code if you can accept the magic. I think we can make things even better though, but it'd require changes to Rhino.Mocks and it's time for bed so maybe Ayende can swing by the Eleutian office while in Seattle and we can work on it. Here's what I'm thinking:

[Test]
public void ViewFiresBeginDrag_Always_FiresChangedEvent()
{
  using (Record)
  {
    Target.Changed += Mocks.CreateEventHandler<EventHandler>();
  }

  using (Playback)
  {
    EventRaiser.Raise(() => The().BeginDrag += null, The<ICardView>(), EventArgs.Empty);
  }
}

There's probably more you can do with the CreateEventHandler syntax like add more specific expectations, assert its not fired, assert that it's fired X times, etc. The EventRaiser syntax is ugly, but it doesn't involve strings or the fire from the right hand side like the syntax I mentioned at the beginning of the post.

You can get the source here, but I warn you it's first draft and there are hacky bits.

Oh, and here's some example tests using the TestsFor fixture:

[TestFixture]
public class CardPresenterTests : TestsFor<CardPresenter>
{
  private Card _card;

  public override void SetupContainer()
  {
    Override<ICardView>(With.Stub);
  }

  public override void BeforeEachTest()
  {
    _card = new Card(0);
  }

  [Test]
  public void OnBeginDrag_Always_SetsIsInFluxToTrue()
  {
    using (Record)
    {
      PrimeEventFiringOn<ICardView>(x => x.BeginDrag += null);
    }

    using (Playback)
    {
      FireEventOn(EventArgs.Empty);

      Assert.That(The<ICardView>().IsInFlux, Is.True);
    }
  }

  [Test]
  public void OnBeginDrag_Always_FiresChangedEvent()
  {
    using (Record)
    {
      PrimeEventFiringOn<ICardView>(x => x.BeginDrag += null);
      Target.Changed += NewEventFireExpectation<EventHandler>();
    }

    using (Playback)
    {
      FireEventOn<ICardView>(EventArgs.Empty);
    }
  }
}
by Aaron on Sunday, April 13, 2008 12:30:10 AM (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [4]  |  Trackback
 Thursday, April 03, 2008

Daniel Cazzulino, author of Moq posted a good comment on my last post where I suggested looking into a Mockito like syntax for .NET Mock Frameworks.

On the surface, Mockito's approach seems good. But if you do the "true" comparison, you'll see that stub(...) is exactly the same as mock.Expect(...) in Moq.

Then, when you do verify(...), you have to basically repeat the exact same expression you put in stub(...). This might work if you only have a couple calls to verify, but for anything else, it will be a lot of repeated code, I'm afraid.

I thought this too. See my comment here from a month ago. Szczepan made a good point and I've thought about it more since then.

When combined with my position on loose vs strict mocks (almost always use loose), I'd say that *most* of the time you are either stubbing or verifying. Meaning, if you're verifying you don't need to stub unless of course that method returns something that is critical to the flow of your test, in which case you don't really need to verify, because the flow would have verified. That's a mouthful, but does that make sense?

I haven't used mockito, and I know there are times I use Expect.Call with return values that matter (which would essentially require you to duplicate stub & verify), but maybe that's a smell? Maybe if you think you need that you can do state based testing or change your API?

Here's an example Test using Rhino.Mocks:

[Test]
public void SomeMethod_Always_CallsSendMail()
{
  IMailSender sender = mocks.DynamicMock();
  UnderTest underTest = new UnderTest(sender);

  using (mocks.Record())
  {
    Expect.Call(sender.SendMail()).Returns(true);
  }

  underTest.SomeMethod();

  mocks.Verify(sender);
}

And some code this is testing (obviously not test driven, but you get the idea):

public void SomeMethod()
{
  if (!_sender.SendMail())
  {
    throw new Exception("OH NOS");
  }
}

Notice that here we would need to stub and verify separately with Mockito like syntax. This would look something like this:

[Test]
public void SomeMethod_Always_CallsSendMail()
{
  IMailSender sender = mocks.DynamicMock();
  UnderTest underTest = new UnderTest(sender);

  Stub.That(() => sender.SendMail()).Returns(true);

  underTest.SomeMethod();

  Verify.That(() => sender.SendMail()).WasCalled();
}

This may violate DRY, but what if you designed your API differently? Maybe SendMail should throw an exception on failure instead of returning a boolean? This would make the return value unnecessary and remove the need for the Stub call. Clearly you can't always do this, especially with unwrapped legacy or API code, but it's something to think about.

Also, I think you shouldn't be verifying more than one method generally to go along with the one assert/test rule, so a single repeat would not be that horrendous. Heck, you could even do:

[Test]
public void SomeMethod_Always_CallsSendMail()
{
  IMailSender sender = mocks.DynamicMock();
  UnderTest underTest = new UnderTest(sender);

  Stub.That(var sendMail = () => sender.SendMail()).Returns(true);

  underTest.SomeMethod();

  Verify.That(sendMail).WasCalled();
}

I think the syntax would lead to better, more concise tests. But maybe it would just be too annoying? I wouldn't know until I tried it for a while I guess.

by Aaron on Thursday, April 03, 2008 8:44:29 AM (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [1]  |  Trackback
 Wednesday, April 02, 2008

(Note: I'm going to speak about .NET mock projects here for the most part, but most of them have Java quasi-equivalents.)

The original mocking frameworks like NMock required you to setup expectations by passing strings for method names. This was fragile and made refactoring more difficult.

A few mock frameworks now allow you to define expectations and mock results in a strongly typed manner. Rhino Mocks and TypeMock use a record/replay method to setup expectations. The record replay method is mostly necessary because the same calls are made on the same objects under two different scenarios. This leads to a few issues.

The first issue is confusion and barrier to entry. Many people have complained that the Record/Replay method is not straight forward and the whole paradigm is confusing. There are also complains about the naming, are you really recording and then replaying? It's just kind of a strange thing. Of course most of us learn to live with it, understand it, and accept it for what it is. Recently though, a few mock frameworks have popped up that do away with this model.

In the .NET world we have Moq. Moq gets rid of the need for record/replay because recordings have a very different syntax. They use lambdas instead of actual calls to the mock object. This allows the framework to know when you are recording an expectation and when you are fulfilling an expectation. It adds a bit of noise in the form of "() =>" but all in all it's not bad. Of course this requires C# 3.0, but it's good to keep looking ahead.

In the Java world we have Mockito. Mockito also does away with the record/replay model but it does it in a different way. At first I wasn't a fan, but thinking about it more, I like it. Mockito has two main apis, stub and verify. Stub is equivalent to SetupResult.For, and verify is equivalent to Expect.Call with a verify. The interesting bit is that the stubbing happens before the the class under test is invoked and the verifying (which includes describing the method to be verified) happens after the class under test is invoked. This is best shown with an example stolen from the Mockito site:

  
  //stubbing using built-in anyInt() argument matcher
  stub(mockedList.get(anyInt())).toReturn("element");
  
  //stubbing using hamcrest (let's say isValid() returns your own hamcrest matcher):
  stub(mockedList.contains(argThat(isValid()))).toReturn("element");
  
  //following prints "element"
  System.out.println(mockedList.get(999));
  
  //you can also verify using argument matcher
  verify(mockedList).get(anyInt());

Obviously it would take a bit of imagination to arrive at a .NET equivalent, but you get the idea. I like this because the normal test structure is Setup Stuff->Do Stuff to Test->Verify Stuff did what it should have. The normal record/replay model requires you to set up verifications before you actually Do Stuff (though you call VerifyAll afterwards). This is a bit less natural. I feel syntax like this (yeah, I like the new NUnit syntax) would be more intention revealing:

Assert.That(() => someMock.Foo(), Was.Called);

Or:

Verify.That(() => someMock.Foo()).WasCalled();

Then you would stub like this:

Stub.That(() => someMock.Bar()).Returns(3);

Note: No idea if this is feasible or makes sense or not, my lambda experience is limited to light reading, but you get the idea. I'm sure the syntax could also be prettier.

Rhino.Mocks is my current mock framework of choice. I'm used to it, I've lightly contributed to it, and I've been working with it for a while now. Despite that, I do think that there is definitely more to explore in the mocking arena especially with C# 3.0.

There are lots of other fun things to talk about too... like TypeMock's magic, but that's another day still...

by Aaron on Wednesday, April 02, 2008 8:13:18 PM (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [3]  |  Trackback
 Friday, March 07, 2008

Sean Chambers asked how we work with a published branch so I figured I'd post on the topic since it's a somewhat interesting one. It's not trivial and it took us a few tries to get it where it is now, and it's still not quite right.

The first step to using a published branch is to create the branch. You can do that like this:

svn cp https://yoursvnserver/svn/project/trunk \
https://yoursvnserver/svn/project/branches/published \
-m "Branching published"

After that, we actually check out the whole branch. It's very useful to have both the pubilshed branch and the trunk on one machine. Note that this isn't necessarily trivial, and its feasibility entirely depends on your build. Having a location agnostic build is a very important thing, and this is one of the reasons.

Now we have both trunk and published branches. We almost always do all of our work in trunk and then merge over to published. Jacob actually wrote a PowerShell script to make the merges easier (from merge.ps1):

param([int]$rev, [string]$reverse)

if (!$rev)
{
  echo "Merge.ps1 REV [REVERSE]"
  return;
}
if (!$reverse)
{
  $branch = "Published"
  $url = "https://yoursvnserver/svn/project/trunk"
}
else
{
  $branch = "Trunk"
  $url = "https://yoursvnserver/svn/project/published"
}

$previous = $rev - 1
$range = $previous.ToString() + ":" + $rev

pushd $branch
svn merge -r $range $url
popd

The reason for doing most of the work in trunk is that often times any issues we run into on the published site will still be issues in the trunk. It makes sense to apply the work there first and then merge it over. The only time we patch published is when we need to apply a hack to make something work so that we can fix it the right way on the trunk later, or maybe that feature is completely different on the trunk and the fix would not apply. Of course this is dangerous and you have to be sure to remember to fix the underlying issue in the trunk before publishing from trunk again. Bug trackers help with that.

That brings us to publishing from trunk again. Merging everything from trunk into the published branch is a giant pain, and just won't work if you've applied many hacks to the published branch. I strongly advise against this. Instead, just start over:

svn rm https://yoursvnserver/svn/project/branches/published \
-m "Deleteing published branch"
svn cp https://yoursvnserver/svn/project/trunk \
https://yoursvnserver/svn/project/branches/published \
-m "Branching published"

By nuking it and recopying it you can just svn up on your published branch and you'll have everything from trunk. For whatever reason, not removing it before copying it again caused us issues. I'd recommend this two phase approach.

Other things we learned in the course of this are the pros and cons of shared "stuff". We have at least 10 gigs of course content and a few other resources that don't need to be in the separate branches like our main trunk and published. We pulled those into their own repositories and keep them in shared directories so both installs can reference it. We also have a set of common build scripts between the two. This is both a good and a bad thing. It's good because it removes some duplication and it allows us to use a separate repo for these scripts (which is handy for some TC build configurations that only need the scripts, though I guess we could just checkout the scripts directory from trunk...) but it's bad because sometimes things will get out of sync. We'll make changes to the shared scripts, and fix the trunk, but it doesn't make sense or it's prohibitive to fix the published branch. You can see in the screen shot I posted our published branch is currently failing. This is likely why. I'd probably recommend keeping all build scripts branched so that you don't run into these sort of issues.

The next thing to worry about is the database. I mentioned that we use multiple databases on the build server. What I didn't mention is that we also use multiple databases on our dev machines. We usually have three. One for trunk, one for trunk tests and one for published. The trunk tests db is imported "light" so our tests run in 6 minutes instead of 14. The trunk db is a nearly full copy of the production db so we have data when we poke around the site on our dev machines. We have a ConnectionStrings.config that is generated from the database info you pass the build script. You can do something like: msbuild Default.msbuild /p:DatabaseName=published and it'd build with the appropriate connection string.

For web applications you also have to worry about IIS. We have two web applications configured in IIS. One for trunk and one for published. This allows us to easily switch between the two by just changing our port or vdir in our url. They have multiple virtual directories underneath them that point to our various shared directories.

I think that's most of the tips I can think of right now. Let me know if you have any questions about anything.

by Aaron on Friday, March 07, 2008 4:21:29 PM (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [8]  |  Trackback

projectsI've mentioned before how much I like TeamCity, but I didn't really talk about how we use it. Just recently Jacob completed some work on multiple build configurations that make our life much easier. I thought I'd go over them here to give you an idea of how we handle continuous integration here.

Continuous Integration configurations

  • CI - Trunk - This is the CI configuration everyone has. This watches the trunk of our SVN repository and builds whenever it changes it will also run our database migrations against the CI trunk database. It has all our built assemblies as artifacts. The artifacts take up a lot of space so we don't keep them around that long.
  • CI - Published - This is just like CI - Trunk, but it watches our published branch, which is where we put everything that we're about to publish to the live site. We keep two branches so that we can make quick fixes to the published site without having to publish new features we're working on. This has its own database that is migrated.
  • Nightly - Trunk - This runs daily rather than watching our source control. It migrates a database on our production database server that is a copy of our live database. It also builds and deploys the trunk to a test address on our production servers. This allows our team in Korea and our stakeholders to see changes every day in a safe environment. The Nightly is also a big part of our localization story, which I'll save for another post.

Database build configurations

  • Snapshot - This and the other db build configurations probably deserve their own post with more details, but I'll do my best to explain these briefly. The Snapshot build configuration takes a point in time snapshot of the database and packages it into a zip file. The zip file becomes a TeamCity artifact that other projects can depend on.
  • Nightly/CI Trunk/CI Published Baseline -

    These configurations import database snapshots into the database they refer to. The only time we need to do this is if a bad migration runs, or we want to "refresh" the data inside the database.

    It is important to note that we do not run CI - Trunk on a complete snapshot of the live database. When we do, it greatly increases the build time because our integration tests run significantly slower in a real database. Instead, we import a "light" database which contains all of the tables, but only the data from our static tables. The users, records, and anything else that grows as we get more and more users are just left empty. This means that we have zero sample data for these things during our integration tests, so we rely heavily on Fluent Fixtures to set up sample data.

    The other two databases do run nearly complete copies of the live database (we exclude log tables basically), so we still get to test our migrations and our site on real data.

Utility build configurations

  • StatSVN - Runs StatSVN on our codebase. Somewhat useful source statistics. Mostly use it to see growth and churn.
  • Duplicate Finder - Haven't really done this much to be honest, but TeamCity has build configurations whose sole purpose is finding duplicate code.

I'm completely enamored with this set of configurations. It makes so many things so painless. We still have room for improvement, managing all of the configuration differences in the sites is difficult. We also lack a one click live publish ability. We still follow a manual script for that, which is error prone and dangerous.

by Aaron on Friday, March 07, 2008 6:40:24 AM (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [1]  |  Trackback
 Thursday, January 10, 2008

stack-bar-chart

With Mingle 1.1 came several great new features, including date fields. Now you can track the date that you complete tickets by adding a Date Done field (along with a Complete transition that sets the Date Done field to today) and locking the Status field to be editable only by transition.

This works great for newly completed cards, but what about all the cards you already have in your system? Sure you could go without burn-up charts and date stats for those cards, but what's the fun in that? 


taskcomplete

Fortunately, Mingle tracks every card change in a history table. Each change in the history table has a date associated with it, so all we need to do is figure out how to extract the dates we care about. I must warn you, this is a little complicated, and I'm sure ThoughtWorks doesn't at all condone you mucking around in your Mingle database manually, so please please back up your database before attempting this and of course I'm not responsible for any damage you do. That said, here's how I was able to retroactively add dates for our two date fields: Date Done, and Date Published.

  1. Back up your Mingle DB. Really.
  2. You'll need to identify the field you use to track status. We use Status. We have Status of 'Not Done', 'Done', and 'Published'.
  3. Then connect to your Mingle database and find the name of your cards table and your card_versions table. Our project is called "eleutian_speakeng" so our tables are "eleutian_speakeng_cards" and "eleutian_speakeng_card_versions".
  4. Add the date fields you want to populate if you don't already have them. We have "Date Added", "Date Published" and "Date Done".
  5. Figure out which fields in the two tables you identified map to your status and dates (this is usually as easy as prepending cp_ and box_caring your names. For us we have cp_status, cp_date_published, cp_date_done, cp_date_added.
  6. Edit the following queries to add in your table/field names and values (these are the exact queries I ran, you'll need to change the red things):
    UPDATE eleutian_speakeng_cards SET cp_date_added = created_at
    
    UPDATE eleutian_speakeng_cards card SET card.cp_date_done = (
      SELECT MAX(b.updated_at) 
        FROM eleutian_speakeng_card_versions a
      INNER JOIN eleutian_speakeng_card_versions b ON a.number = b.number 
        AND b.version=a.version+1 
      WHERE card.number = a.number
      AND (a.cp_status='Not Done' OR a.cp_status IS NULL) AND b.cp_status='Done'
    ) WHERE card.cp_status = 'Done'
    
    
    UPDATE eleutian_speakeng_cards card SET card.cp_date_published = (
      SELECT MAX(b.updated_at) 
        FROM eleutian_speakeng_card_versions a
      INNER JOIN eleutian_speakeng_card_versions b ON a.number = b.number 
        AND b.version=a.version+1 
      WHERE card.number = a.number
        AND (a.cp_status IN ('Done', 'Not Done') OR a.cp_status IS NULL) 
        AND b.cp_status='Published'
    ) WHERE card.cp_status = 'Published'
    
  7. Once you've got that taken care of, go ahead and run the queries (you backed up right?) and you should be good to go.

So now that you've got your dates in your database, how do you get the cool burn up chart? Like this (mold to your project):

{{
   stack-bar-chart
   x-label-start: SELECT MIN('Date Added')
   x-label-end: Today
   x-label-step: 14
   chart-height: 600
   chart-width: 700
   plot-height: 500
   plot-width: 550
   conditions: Type IN (Bug, Story, Task)
   three-d: true
   cumulative: true
   series:
     - label: 'Not Done'
       color: #e6a800
       data: SELECT 'Date Added', COUNT(*)
       type: area
     - label: Done
       color: #1bcc00
       type: area
       data: SELECT 'Date Done', Count(*) WHERE 'Date Done' IS NOT NULL
     - label: Published
       color: #0079bf
       type: area
       data: SELECT 'Date Published', COUNT(*) WHERE 'Date Published' IS NOT NULL
}}
agile | development | mingle | tips | tools
by Aaron on Thursday, January 10, 2008 11:48:38 PM (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [0]  |  Trackback
 Saturday, December 22, 2007

One of the first things we did when we got a dedicated development server was set up a continuous integration server. The natural choice at the time was CruiseControl.NET, so Jacob set off to get CC.NET installed and configured. I can't speak to that experience first hand, but I know it wasn't a fun one. Lots of XML hell and other non-fun issues. The end result wasn't too bad--it'd email us when builds failed with a log, and we could review logs on our CC.NET site. There was really nothing WOW about it--it got the job done. It was frustrating at times because the logs it spat out were difficult to visually parse; it was basically just a verbose msbuild log with a failure mixed in somewhere... and that was if we were lucky enough to get a text log and not have to filter through the build xml log. I'm sure there are some things we could have done to filter that log down, apply xslt to our build.xml to make it more readable, or configure things to be nicer, but I'm also sure it would have involved hard work and more XML hell.

Enter the free Professional Edition of TeamCity 3.0. We'd tried out TC briefly in the past and it seriously lacked in the .NET support arena. Now they tout support for msbuild (2.0 and 3.5), sln files from 2003-2008, and even a nifty .NET duplicate code finder. They also have some pretty slick support for NUnit which I'll get to in a moment.

I decided to give it a shot again, so I set up the TC server on our Ubuntu VM. Why there? Well that's where most of our server tools are: SVN, Mingle, FogBugz and now TC. But wait, aren't we a .NET shop? Yep. The build doesn't happen on our Ubuntu VM. See, TC is set up to have a server + build agents. A build agent is basically a machine that will check out code, do the build when the server tells it to, and report back results and artifacts to the build server. This way you can have multiple Build Agents--different Operating Systems, system configurations, physical locations, etc.

Installing both the Server and the Agent were relatively straight forward. There were a few annoyances with the Agent's configuration (like I couldn't get it to use a network drive as a working directory and it took some tweaking to get things to work with our directory structure,) but after a while I got things up and running.

buildconfiguration Setting up a build configuration in Team City is an absolute breeze. You basically have a 7 step Q&A about your configuration:

  1. General Settings - Name, Description, a few options and the build number format (we use our SVN revision #)
  2. Version Control Settings - You can set up multiple repos to check out with a number of tools (SVN, CVS, Perforce, StarTeam, ClearCase). I'm sure it wouldn't be too difficult to write plugins for Mecurial or Git. You can also tell it where to check out relative to your working directory and whether or not you want it to check out to the agent or on the server and then have the agent download it.
  3. Build Runner - Mainly .NET and Java stuff. No Rake yet, but it supports command line so you could just use that. With MSBuild you just specify your MSBuild file, the target you want to run, and any command line args you need. It's here that you specify your build artifacts as well. Build artifacts are kept along side your completed builds and easily accessible from the TC site.
  4. Build Triggering - You can trigger based on VCS changes, schedule, other build configurations completing, or automatic retry.
  5. Artifact dependencies - This makes it really easy to say you want Build A to use Build B's artifacts. For example, an installer configuration that will build an installer based on the last successful CI build.
  6. Properties and environment variables - Self explanatory.
  7. Agent requirements - Not all your agents can run all your build configurations necessarily. This will let you set requirements. You can set these based on environment variables, machine names, etc.

configs

Once you get things set up, you get to see the really cool stuff. TeamCity is just packed full of nerdgasm features that just work out of the box. No need to scour the web for plugins, deal with XML, or anything like that. Things just work. For example, you get full status of the build while it is happening--you can see what MSBuild step the build is on, you can see the important messages in the Build Log (it filters them quite intelligently), you can even see what threads/processes are running. If you use their NUnit runner you can even see how many tests have passed/failed/been ignored.

testduration That's not all you can see with their test runner though. You also get a full history of each test--how long each test took each time it was run, and if one fails you can see in what build it first failed, if it has been fixed yet, and you can even click to open it in your IDE if you have the Visual Studio plugin failedtextinstalled. The Tests tab even orders your tests by Duration so you have an idea which tests you may need to optimize.

statisticsThen there's the Statistics tab. This tab is a one stop shop for build health. You can see how long the build takes, how often it succeeds, how many tests you have, how many fail and even how big the artifacts are. You can see our build #s jump when we switched from a counter to SVN revision.

If there aren't enough features for you, don't worry, like with most JetBrains products you can extend them to your hearts content. That is if you can stomach the lack of API documentation (surprisingly, normal usage documentation is pretty good). One of our guys is working on a Campfire Notification plugin at the moment so we can get better build notifications in our Campfire. That's another post though.

All in all, we're very happy with TeamCity. It's just as free as CC.NET for a team of our size, it's much much easier to set up, and it has way more features. How could you go wrong? I'm sure its only a matter of time before we start to see rake runners, MBUnit test runners, and many other things to make TC even better. JetBrains has a winner here.

by Aaron on Saturday, December 22, 2007 2:13:34 PM (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [5]  |  Trackback
 Saturday, December 15, 2007

I just profiled one of our test assemblies (Unit tests only, nothing hitting the db). Mocking accounts for 196 seconds out of 357 seconds. The culprit is System.Reflection.Emit.TypeBuilder.CreateType(). This doesn't come as that much of a surprise to me, but still... is there a faster way to do this? Oh and yeah, the results of CreateType are cached so its only the first time you create a mock of a type that you take this hit, but still... that can really slow down TDD.

And no, I'm not saying we should stop mocking... just raising awareness.

by Aaron on Saturday, December 15, 2007 11:17:03 AM (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [3]  |  Trackback
 Thursday, December 13, 2007

I'm going to be putting together a series of Vim Screencast Tutorials. My goal is to raise awareness in the world of just how powerful Vim is and then get everyone comfortable using it. Here's a short trailer that shows a few of the things that make me love Vim. This is my first Screencast of this kind and, yea, I know its cheesy.

You need to upgrade your Flash Player to at
least version 8 to view this video content.

Click here to download the latest version of Flash Player.

Click here to save

You can get Vim here, but I'd ultimately recommend ViEmu, a Vim emulator plugin for Visual Studio.

by Aaron on Thursday, December 13, 2007 2:23:32 AM (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [14]  |  Trackback
 Sunday, November 18, 2007

I haven't posted in a while and decided to write a post about one of the little side projects I've worked on recently. I've been lurking around on the ALT.NET mailing list and one of the discussions I take time to keep up on is on Dependency Injection. Something I worked on a few weeks ago has some applications in that area and so I decided to see what others thought.

I was talking with Aaron several months ago about how many problems constructors in static languages cause. Ruby people are fortunate in that their constructors are much more flexible than their C# counterparts. Want something to be a singleton? Just return the same instance from the constructor! Lifecycle management and dependencies can be the responsibility of the constructor itself, rather than a function of a container.

Over the weekend a few weeks ago I wrote a library and MsBuild task that factors constructor calls out of code and into a pluggable factory system. You write code like this:

public class EmailSender
{
  public void TellEveryone(string message)
  {
    TemplateRenderer renderer = new TemplateRenderer();
    Console.WriteLine(renderer.Render(message));
  }
}
public class EmailSenderFactory : IObjectFactory<EmailSender>
{
  public EmailSender Create()
  {
    return new EmailSender();
  } 
}
EmailSender emailer = new EmailSender();
emailer.TellEveryone("Hello, World!");

What the MsBuild task will do is this:

  1. Load the target Assemblies using Mono.Cecil
  2. Find implementations of IObjectFactory<T> and add them to the FactoryMap.
  3. Find all instances of new T() for each of those factories and replace them with a call to the Factories.Create<T>(), ignoring calls inside of factories themselves.
  4. Weave the FactoryMap (just the Factory Type and its Object Type) into the Assembly.

So during runtime we can use the FactoryMap that was serialized in step #4 to create the factories and use them to create instances as they're demanded. It is important to note that your objects need default constructors, even though they may never be called after the code is weaved. One can also replace the factory implementations with IoC backed versions, etc...

Why don't you just create a New.Instance<T> method and use that instead of new T() and save yourself the pain of weaving and having a task and all that, moron?

Good question! I mostly did this as an exercise in code weaving using Cecil. It's a great library, although testing code using it can be a little tedious. I'm tossing around some other projects that I'd love to give a shot and this was a good introduction. You could achieve the same thing using reflection to build your factory map and remember to use New.Instance<T>() instead of the new operator in your code. Granted, the manual approach plasters the concern all over the place and weaving keeps the original code free from that. Which may be an OK trade-off for the simplified process.

Another thing to consider, is tha