Home > Uncategorized > storyq-vs-slim-fitnesse

storyq-vs-slim-fitnesse

August 24th, 2010 Leave a comment Go to comments

Acceptance testing from user stories via StoryQ and Slim

Recently I was working with a team using user stories for acceptance testing (in .Net). As it turned out I had the opportunity to implement the acceptance tests in both Slim/fitnesse and StoryQ. To me, either framework is fine because they are both the same strategy: they create an abstraction that is understood by the customer that we can run against the system under test. Below is an extract of the user story that was discussed with the customer (and there was also a sketch of the GUI that I haven’t included) and what I’ll show is the StoryQ and Slim implementations up to the point of hooking into the system under test (SUT). The process for each is pretty similar of a two set process: (1) code up the test as the customer representation (2) create (C#) code that wraps hooking into the SUT

    
                  step one           step two
 ------------------------------------------------
  user story  ->  xUnit StoryQ    -> TestUniverse
  user story  ->  story test wiki -> Fixtures

User story

This story was written in plain and developed over three cycles over the course of an hour. There was actually about 7 scenarios in the end.

As a Account Manager
I want to cater for risks terminated in System1 after extraction to System2
So that I offer a renewal to the Broker

    Given I have a policy                                                                                                                                        
      And it has a current risk in System2                                                                                                                           
    When the risk is terminated in System1                                                                                                                        
    Then the risk is deleted in System2                    

    Given I have a policy                                                                                                                                      
    When a deleted risk later than the load date is added                                                                                                    
    Then it should be rerated      
                                                                                                                      
    Given I have a policy                                                                                                                                        
      And it has an current risk in System2                                                                                                                          
    When its status is ready to review                                                                                                                           
      And a risk is deleted in System1                                                                                                                            
    Then the policy should be rated                                                                                                                              
      And the policy should remain in the same status        

Example One: StoryQ

I took this user story and then converted it into StoryQ using the StoryQ converter that you can get with the library. StoryQ has a slight difference: (a) it uses the In order syntax and (b) I had to have a Scenario with each Given/When/Then. It took me in total about half an hour to download, add as a reference and convert the story.

Story to storyq via converter

When doing the conversion, the converter will error until it is in the correct format. Here’s what I had to rewrite to:

Story is Terminated risks after extractions
  In order to offer renewal to the Broker
  As a Account Manager
  I want to cater for risks terminated in System1 after extraction to System2

 With scenario risks become deleted in System2
   Given I have a policy                                                                                                                                        
     And it has a current risk in System2                                                                                                                           
  When the risk is terminated in System1                                                                                                                        
    Then the risk is deleted in System2                    

 With scenario newly deleted risks need to be rerated
   Given I have a policy                                                                                                                                      
   When a deleted risk later than the load date is added                                                                                                    
   Then it should be rerated      

 With scenario status being reviewed
   Given I have a policy                                                                                                                                        
     And it has an current risk in System2                                                                                                                          
   When its status is ready to review                                                                                                                           
     And a risk is deleted in System1                                                                                                                            
   Then the policy should be rated                                                                                                                              
     And the policy should remain in the same status            

The output of the converter in C# is:

new Story("Terminated risks after extractions")
       .InOrderTo("offer renewal to the Broker")
       .AsA("Account Manager")
       .IWant("to cater for risks terminated in System1 after extraction to System2")

       .WithScenario("risks become deleted in System2")
       .Given("I have a policy")
       .And("it has a current risk in System2")
       .When("the risk is terminated in System1")
       .Then("the risk is deleted in System2")

       .WithScenario("newly deleted risks need to be rerated")
       .Given("I have a policy")
       .When("a deleted risk later than the load date is added")
       .Then("it should be rerated")

       .WithScenario("status being reviewed")
       .Given("I have a policy")
       .And("it has an current risk in System2")
       .When("its status is ready to review")
       .And("a risk is deleted in System1")
       .Then("the policy should be rated")
       .And("the policy should remain in the same status")

StoryQ into xUnit

From this output you add it into a xUnit test framework – so far either MSTest or NUnit. We were using MSTest and it looks like this. We need to add

  • Testclass, Testmethod as per MSTest
  • I add indentation myself
  • .ExecuteWithReport(MethodBase.GetCurrentMethod()) at the end
using System.Reflection;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using StoryQ;

namespace Tests.System.Acceptance
{
    [TestClass]
    public class TerminatedRiskAfterExtractionTest
    {
        [TestMethod]
        public void Extraction()
        {
            new Story("Terminated risks after extractions")
                .InOrderTo("offer renewal to the Broker")
                .AsA("Account Manager")
                .IWant("to cater for risks terminated in System1 after extraction to System2")

                .WithScenario("risks become deleted in System2")
                    .Given("I have a policy")
                        .And("it has a current risk in System2")
                    .When("the risk is terminated in System1")
                    .Then("the risk is deleted in System2")

                .WithScenario("newly deleted risks need to be rerated")
                    .Given("I have a policy")
                    .When("a deleted risk later than the load date is added")
                    .Then("it should be rerated")

               .WithScenario("status being reviewed")
                   .Given("I have a policy")
                       .And("it has an current risk in System2")
                   .When("its status is ready to review")
                       .And("a risk is deleted in System1")
                   .Then("the policy should be rated")
                       .And("the policy should remain in the same status")

                .ExecuteWithReport(MethodBase.GetCurrentMethod());
        }
    }
}

Link to the SUT (TestUniverse)

StoryQ needs to link the user story test text to the SUT. One clean approach is to use a TestUniverse – and as you will see it replicates the idea of the fitnesse fixture. A quick summary as you look at the code:

  • add a class TestUniverse, have a reference to it available (_t_) and instatiate it in the Setup()
  • convert string text to methods eg ("I have a policy" to _t.IHaveAPolicy)
  • add methods to the TestUniverse (personally I use resharper and it does the work for me)
  • quite quickly you get reuse patterns eg (_t.IHaveAPolicy)
  • I tend to create methods only as I need them because it gives me a better sense of tests actually implemented
    using System.Reflection;
    using Microsoft.VisualStudio.TestTools.UnitTesting;
    using StoryQ;

    namespace Tests.System.Acceptance
    {
        [TestClass]
        public class TerminatedRiskInSystem1AfterExtractionTest
        {
            private TestUniverse _t;

            [TestInitialize]
            public void Setup()
            {
                _t = new TestUniverse();
            }

            [TestMethod]
            public void Extraction()
            {
                new Story("Terminated risks after extractions")
                    .InOrderTo("offer renewal to the Broker")
                    .AsA("Account Manager")
                    .IWant("to cater for risks terminated in System1 after extraction to System2")

                    .WithScenario("risks become deleted in System2")
                    .Given(_t.IHaveAPolicy)
                        .And("it has a current risk in System2")
                    .When("the risk is terminated in System1")
                    .Then("the risk is deleted in System2")

                    .WithScenario("newly deleted risks need to be rerated")
                    .Given(_t.IHaveAPolicy)
                    .When("a deleted risk later than the load date is added")
                    .Then("the policy should be rerated")

                    .WithScenario("status being reviewed")
                    .Given(_t.IHaveAPolicy)
                        .And("it has an current risk in System2")
                    .When(_t.ItsStatusIsReadyToReview)
                        .And("a risk is deleted in System1")
                    .Then("the policy should be rated")
                        .And("the policy should remain in the same status")

                    .ExecuteWithReport(MethodBase.GetCurrentMethod());
            }

        }

        public class TestUniverse
        {
            // private variables here if needed

            public void IHaveAPolicy()
            {
                // Hook up to SUT
            }

            public void ItsStatusIsReadyToReview()
            {
                // Hook up to SUT
            }

            public void ThePolicyShouldBeReRated()
            {
                // Hook up to SUT
                // Assertions too
            }

        }
    }   

Hopefully, you can see the TestUniverse helps us keep the two parts of the test separated – and of course, you could have them in separate files if you really wanted.

Now when you run this test through your favourite xUnit runner (MSTest, Resharper, TestDriven.Net, Gallio) you will get the results of the test. Or more appropriately, I have the tests ready and waiting. Now we implement the system before return to these after to hook up. Yes, I confess, these system tests are test-last! I explain that somewhere else.

Example Two: Slim

Having got the code in StoryQ to this point, we decided to investigate what the same user story would look like in Slim. This exercise turned out to take somewhat longer. In C#, there are no specific libraries for implementing the Given/When/Then although there is one in java (givenWenZen). So we given our stories we implemented them using a Script table. This turned out to the fruitful.

Slim story test wiki

Here are the wiki tables. In the actual, wiki we also included the original user story around the table which for this example I have included only the scenario.

With scenario risks become deleted in System2

|script|Risk Terminated                  |
|check |I have a policy                 ||
|check |it has a current risk in System2||
|terminate risk                          |
|ensure|risk is deleted                  |

With scenario newly deleted risks need to be rerated

|script|Risk Terminated                                |
|check |I have a policy                               ||
|check |deleted risk later than the load date is added||
|ensure|risk is rerated                                |

With scenario status being reviewed       

|script|Risk Terminated                                        |
|check |I have a policy                        |               |
|check |it has a current risk in System2       |               |
|check |status is in                           |ready to review|
|check |risk is deleted                        |               |
|ensure|policy is rerated                                      |
|check |policy should remain in the same status|ready to review|

My working knowledge of the check, ensure, reject took a while to get up to speed including the mechanism for reporting back values in empty cells (eg |check|I have a policy||). Other than that it took as about an hour to get on top of structure for the table in this case. Script table has worked okay but it did mean that we went from a user story to a story test that in the future we should go straight to. Having said that the user story Given/When/Then is nice for its natural language syntax and helps avoids sticking solely to GUI-type actions.

Slim fixture

The slim fixture is pretty straight forward. We hardcoded in our policy and risk to start with. Later on we would swap this out for the system.

namespace Tests.System.Acceptance
{
    public class RiskTerminated
    {

        public string IHaveAPolicy()
        {
            return "12-4567846-POF";
        }

        public string ItHasACurrentRiskInSystem2()
        {
            return "001";
        }

        public bool terminateRisk()
        {
            return true;
        }

        public bool riskIsDeleted()
        {
            return true;
        }

        public string deletedRiskLaterThanTheLoadDateIsAdded()
        {
            return string.Empty;
        }

        public bool RiskIsRerated()
        {
            return true;
        }

        public bool PolicyIsRerated()
        {
            return true;
        }

        public string statusisin()
        {
            return string.Empty;
        }

        public string policyShouldRemainInTheSameStatus()
        {
            return string.Empty;
        }

        public bool getAListOfCancelledRisksFromSystem1()
        {
            return true;
        }

        public bool theRiskIsNotAvailableInSystem2()
        {
            return true;
        }
    }
}

Now that this is all working we can run the tests in Fitnesse. We already had Fitnesse up and running with Slim build to the correct version. Don’t underestimate the time and expertise needed to get this all up and running – it’s not hard, after the first time that is! We could have also got our wiki running under a Generic Test in Visual Studio but we haven’t gone that far yet.

Some general comments

If you were attentive to the style of tests, you might have noticed that we passed in no values for the tests. In this testing, the setup is limited to fetching existing data and then making changes from there. Because we are interested in knowing which pieces of data were touched the tests do want, in this case, the policies reported back. This is not ideal but pragmatic for the state of the system. Both StoryQ and Slim can report back this type of information – in these examples, Slim has this implemented but not StoryQ.

  • StoryQ has the least barrier to entry and keeps the tests focussed at the developers
  • StoryQ is easy to get onto a build server
  • StoryQ has much better for refactoring because it is in C# in an IDE and is not using (magic) string matching – you could though
  • StoryQ is generally unknown and is harder to introduce particularly to devs that haven’t really grasped Fitnesse
  • Cucumber could just as easily be used (I just thought that I would throw that in from left field)
  • Given/When/Then may be limited as a rubric for testing – but those constraints are also benefits
  • Slim allows realistically allows more comments around tests
  • Wiki reporting is potentially richer and closer to the customer
  • Separate systems of a wiki and code is good separation
  • The wiki and code are nonetheless two different systems – that comes at a cost
  • If the wiki isn’t driving development then I’m not convinced it is cost effective
  • Slim required more knowledge around which table to use and its constraints – that’s (sort of) a one-time cost
  • I wouldn’t mind a Given/When/Then table for Slim – apparently UncleBob might be writing one?

Rock on having customers

A final note in this example. I most interested in the benefits of this type of testing rather than the specific implementation. Anything that allows us to have more conversations with our code base and the people who develop it the better. By writing the user stories down before development means we need to do the “outside in” thing first. By coming back to them, I have found that it flushes assumptions made in development and makes us remember our earlier discussions. It easy to forget that our customers are often good at helping us create abstractions in our code base that we loose sight of in the middle coding the solution. Those abstractions live in the user stories – to hook up the user stories to the SUT we often need to decouple our code in ways that is not required when our only client code of the SUT is the application/UI. So I say rock on having customers that require us to have proxies in the form of code to represent them!

Categories: Uncategorized Tags:
  1. Pradeep
    September 3rd, 2010 at 19:18 | #1

    Todd you are the best!

  1. No trackbacks yet.