Archive

Posts Tagged ‘jsspec’

Sharepoint & TDD: Getting started advice

July 1st, 2011 4 comments

I have a couple of people asking lately about starting on SharePoint. They’ve asked about how to move forward with unit and integration testing and stability. No one wants to go down the mocking route (typemock, pex and moles) and quite rightly. So here’s my road map:

The foundations: Hello World scripted and deployable without the GUI

  1. Get a Hello World SharePoint “app” – something that is packageable and deployable as a WSP
  2. Restructure the folders of the code away from the Microsoft project structure so that the root folder has src/, tools/, lib/ and scripts/ folders. All source and tests are in src/ folder. This lays the foundation for a layered code base. The layout looks like this sample application
  3. Make the compilation, packaging, installation (and configuration) all scripted. Learn to use psake for your build scripts and powershell more generally (particularly against the SharePoint 2010 API). The goal here is that devs can build and deploy through the command line. As such, so too can the build server. I have a suggestion here that still stands but I need to blog on improvements. Most notably, not splitting out tasks but rather keeping them in the same default.ps (because -docs works best). Rather than get reuse at the task level do it as functions (or cmdlets). Also, I am now moving away from the mix with msbuild that I blogged here and am moving them into powershell. There is no real advantage other than less files and reduced mix of techniques (and lib inclusions).
  4. Create a build server and link this build and deployment to it. I have been using TFS and TeamCity. I recommend TeamCity but TFS will suffice. If you haven’t created Build Definitions in TFS Workflow allow days-to-weeks to learn it. In the end, but only in the end, it is simple. Becareful with TFS, the paradigm here is that build server does tasks that devs don’t. It looks a nice approach. I don’t recommend it and there is nothing here by design that makes this inevitable. In TFS, you are going to need to build two build definitions: SharePointBuild.xaml and SharePointDeploy.xaml. The build is a compile, package and test. The deploy simply deploys to an environment – Dev, Test, Pre-prod and Prod. The challenge here is to work out a method for deploying into environments. In the end, I wrote a simple self-host windows workflow (xamlx) that did the deploying. Again, I haven’t had time to blog the sample. Alternatively, you can use psexec. The key is that for a SharePoint deployment you must be running on the local box and the most configurations have a specific service account for perms. So I run a service for deployment that runs under that service account.

Now that you can reliably and repeatably test and deploy, you are ready to write code!

Walking Skeleton

Next is to start writing code based on a layered strategy. What we have found is that we need to do two important things: (1) always keep our tests running on the build server and (2) attend to keeping the tests running quickly. This is difficult in SharePoint because a lot of code relates to integration and system tests (as defined by test automation pyramid). We find that integration tests that require setup/teardown of a site/features get brittle and slow very quickly. In this case, reduce setup and teardown in the the system tests. However, I am also had a case where the integration test showed that a redesigned object (that facaded SharePoint) would give better testability for little extra work.

  1. Create 6 more projects based on a DDD structure (Domain, Infrastructure, Application, Tests.Unit, Tests.Integration & Tests.System). Also rename your SharePoint project to UI-[Your App], this avoids naming conflicts on a SharePoint installation. We want to create a port-and-adapters application around SharePoint. For example, we can wrap property bags with repository pattern. This means that we create domain models (in Domain) and return them with repositories (in Infrastructure) and can test with integration tests.
  2. System tests: I have used StoryQ with the team to write tests because it allows for a setup/teardown and then multiple test scenario. I could use SpecFlow or nBehave just as easily.
  3. Integration tests: these are written classical TDD style.
  4. Unit tests: these are written also classical TDD/BDD style
  5. Javascript tests: we write all javascript code using a jQuery plugin style (aka Object Literal) – in this case, we use JSSpec (but I would now use Jasmine) – we put all tests in Tests.Unit but the actual javascript is still in the UI-SharePoint project. You will need two sorts of tests: Example for exploratory testing and Specs for the jasmine specs. I haven’t blogged about this and need to but is based on my work for writing jQuery plugins with tests.
  6. Deployment tests: these are tests that run once that application is deployed. You can go to an ATOM feed which returns the results of a series of tests that run against the current system. For example, we have the standard set with tells us the binary versions and which migrations (see below) have been applied. Others check whether a certain wsp has been deployed, different endpoints are listening, etc. I haven’t blogged this code and mean to – this has been great for testers to see if the current system is running as expected. We also get the build server to pass/fail a build based on these results.

We don’t use Pex and Moles. We use exploratory testings to ensure that something actually works on the page

Other bits you’ll need to sort out

  • Migrations: if you have manual configurations for each environment then you’ll want to script/automate this. Otherwise, you aren’t going to be one-click deployments. Furthermore, you’ll need to assume that each environment is in a different state/version. We use migratordotnet with a SharePoint adapter that I wrote – it is here for SharePoint 2010 – there is also a powershell runner in the source to adapt – you’ll need to download the source and compile. Migrations as an approach works extremely well for feature activation and publishing.
  • Application Configuration: we use domain models for configuration and then instantiate via an infrastructure factory – certain configs require SharePoint knowledge
  • Logging: you’ll need to sort of that Service Locator because in tests you’ll swap it out for Console.Logger
  • WebParts: can’t be in a strongly typed binary (we found we needed another project!)
  • Extension Methods to Wrap SharePoint API: we also found that we wrapped a lot of SharePoint material with extension methods

Other advice: stay simple

For SharePoint developers not used to object oriented programming, I would stay simple. In this case, I wouldn’t create code with abstractions that allowed you to unit test like this. I found in the end the complexity and testability outweighed the simplicity and maintainability.

Microsoft itself has recommended the Repository Pattern to facade the SharePoint API (sorry I can’t for the life of me find the link). This has been effective. It is so effective we have found that we can facade most SharePoint calls in two ways: a repository that returns/works with a domain concept or a Configurator (which has the single public method Process()).Anymore than that it was really working against the grain. All cool, very possible but not very desirable for a team which rotates people.

jQuery and testing — JSUnit, QUnit, JsSpec [Part 2]

November 15th, 2008 No comments

Trying QUnit with jQuery

In the JSUnit entry, one of the main problems was with the sequencing of calls. Let’s see how QUnit handles this. QUnit has a simple implementation to this problem: stop() and start() commands to synchronise sequences. The basic approach is that it calls a test function. With the stop() command, it will not call the next test until you say start(). But it does complete the rest of the current test.

The code that I ended up with was just what I wanted compared to JsUnit. Previously, I had said that basically I wanted to call my function and then process the result that it was in fact what I wanted. Here is the main javascript code (it’s nice and concise).

One point about the code at this stage. The tests had to be inside the success function to be run. I wonder if this is going to create a code smell in the long run. Plus there is no setup/teardown cycles, again I wonder what that will mean in the long run. Perhaps not?

module("XML to object");

test("Check the xml to object conversion without showing the tree", function() {
  
  expect( 5 )
  stop();
  
   $().storyq({
        url: 'data/results-01.xml', 
        load: '',
        success: function(feed) {
          ok( feed, "is an object" )
          ok( !$.isFunction(feed), "is not a function" )
          ok( feed.version, "has a version: " + feed.version )
          ok( feed.items, "has items")
          same( feed, reference, "is the same as the reference object in data/results-01.js")
          start();
        }
    });

});

Honestly, it is that easy.

Here’s a couple of features that took me half an hour to work out restated from above. (1) stop() and start() almost work as you expect – but I had to put some alerts in to check the order of execution. Basically, a stop() halts any new tests from executing but it keeps the current test executing. The effect of this is that the asynchronise call can be completed. start() then tells the tests to start running again. If you don’t have a start() then you will find that your test runner halts altogether. There is another option and that is to put a timer on the stop and then you don’t need a start. I prefer to keep the tests running as quickly as possible.

Just another note. I decided to do a same() comparison. I have saved and preloaded an object from a file for ease. This kept the test easy to read – my reference object here is quite long. You can see the insertion of this file in the entire code below <script type="text/javascript" src="data/results-01.js"/>

&lt;html>
&lt;head>
  &lt;title>Unit tests for StoryQ Results viewer&lt;/title>
  &lt;link rel="stylesheet" href="../../lib/qunit/testsuite.css" type="text/css" media="screen" />

  &lt;link rel="stylesheet" href="../../lib/treeview/jquery.treeview.css" />
  &lt;link rel="stylesheet" href="../../src/css/storyq.treeview.css" />
  &lt;link rel="stylesheet" href="../../src/css/storyq.screen.css" />

  &lt;script src="../../lib/jquery/jquery.js">&lt;/script>
  &lt;script src="../../lib/jquery/jquery.cookie.js" type="text/javascript">&lt;/script>
  &lt;script src="../../lib/treeview/jquery.treeview.js" type="text/javascript">&lt;/script>
  &lt;script src="../../src/storyq.js" type="text/javascript">&lt;/script>
  &lt;script src="../../src/storyq.treeview.js" type="text/javascript">&lt;/script>
  &lt;script src="../../src/storyq.xml.js" type="text/javascript">&lt;/script>
  &lt;script src="../../src/storyqitem.js" type="text/javascript">&lt;/script>
  &lt;script src="../../src/storyqresults.js" type="text/javascript">&lt;/script> 

  &lt;script type="text/javascript" src="../../lib/qunit/testrunner.js">&lt;/script> 
  &lt;script type="text/javascript" src="data/results-01.js">&lt;/script>  
  
  &lt;script type="text/javascript">
    module("XML");

    test("Check the xml to object conversion without showing the tree", function() {

      expect( 5 )
      stop();

       $().storyq({
            url: 'data/results-01.xml', 
            load: '',
            success: function(feed) {
              ok( feed, "is an object" )
              ok( !$.isFunction(feed), "is not a function" )
              ok( feed.version, "has a version: " + feed.version )
              ok( feed.items, "has items")
              same( feed, reference, "is the same as the refefence object in data/results-01.js")
              start();
            }
        });

    });
  &lt;/script>
  
&lt;/head>
&lt;body>
  
 &lt;h1>QUnit tests&lt;/h1>
 &lt;h2 id="banner">&lt;/h2>
 &lt;h2 id="userAgent">&lt;/h2>
 &lt;ol id="tests">&lt;/ol>

 &lt;div id="main">&lt;/div>

&lt;/div>
&lt;/div>

&lt;/body>
&lt;/html>

The output results from QUnit are nice to look at and easy to read. I didn’t have a couple of errors that weren’t the easiest to debug given the output. Partly though because I was new to it, I was taking too big a step at times!

I’m happy with QUnit – and there are plenty of examples in the JQuery test suite. I can see that I would do TDD with this.

Being a BDD type of guy, I’m now off to see what JsSpec has to offer.

jQuery and testing – JSUnit, QUnit, JsSpec [Part 1]

November 15th, 2008 No comments

Trying JsUnit with JQuery

I have started first with JSUnit because it is tried and true (and to tell the truth I thought it would be fine and didn’t bother with a quick search for alternatives).

For the impatient, I won’t be going with JSUnit and here are some reasons:

  • the problem is that the integration of a setup (ie onload – pausing) to load the data doesn’t integrate well with jQuery. JSUnit has its own setup and document loader but I am still wondering how to do this transparently (ie I didn’t actually get the test to work – I wasn’t exhaustive but then again I don’t think that I should have needed to be to get this test going)
  • Firefox 3.0 on mac doesn’t integrate well (ie it doesn’t work), but safari does! Unfortunately, I am a little bound to firebug for development.
  • JSUnit doesn’t report tests well either

I went and had a look at how JSUnit does it. (Remember that this framework has been around a lot longer than jQuery.) Here is the extract from the code/test samples. The basic setup is to hook into an existing testManager that existing within a frame and then get the data from there. Furthermore, you need to manage your own flag that the process has been complete. JSUnit then looks through all functions that start with test in this case testDocumentGetElementsByTagName checks expected data. Here I assume that the tests are run in a particular frame (buffer()) that testManager gives us access to.

var uri = 'tests/data/data.html';

function setUpPage() {
    setUpPageStatus = 'running';
    top.testManager.documentLoader.callback = setUpPageComplete;
    top.testManager.documentLoader.load(uri);
}

function setUpPageComplete() {
    if (setUpPageStatus == 'running')
        setUpPageStatus = 'complete';
}

function testDocumentGetElementsByTagName() {
    assertEquals(setUpPageStatus, 'complete');
    var buffer = top.testManager.documentLoader.buffer();
    var elms = buffer.document.getElementsByTagName('P');
    assert('getElementsByTagName("P") returned is null', elms != null);
    assert('getElementsByTagName("P") is empty', elms.length > 0);
}

Below is the rewritten code to exercise my code. Here’s a couple of the features:

  • for setup, pass in the correct xml file via uri variable (obviously)
  • to test, I have written a test testXML2Object.

    p. There is one major design problem with the code itself that didn’t allow me to use my own data loader. You will see the line var feed = new StoryQResults(buffer);. Where did that come from? It is nothing close to the code I said that I wanted to exercise. It is infact from within the code I want to exercise. The major issue I found here is that to load and test data I had to use the testManager rather than use my own ().storyq() call.

The other problem was it wouldn’t return the result that I wanted either. I was expecting my feed variable to be an object of the results. Instead I was getting a reference to function StoryQResults – given now that it wasn’t running in Firefox and I didn’t have Firebug life was getting a little hard.

var uri = '../../xml/results-01.xml';

function setUpPage() {
    setUpPageStatus = 'running';
    top.testManager.documentLoader.callback = setUpPageComplete;
    top.testManager.documentLoader.load(uri);
}

function setUpPageComplete() {
    if (setUpPageStatus == 'running')
        setUpPageStatus = 'complete';
}

function testXML2Object() {
    assertEquals(setUpPageStatus, 'complete');
    var buffer = top.testManager.documentLoader.buffer();
    
    var feed = new StoryQResults(buffer);               

    assertEquals(feed.version, '0.1')
    assertEquals("Number of stories", $(feed).size(), 1)
    $.each(feed, function(){
      alert(this)               
    })

}

Even though I know that I getting a function returned instead of an object I am still going to see if I can invoke my own loading function within JSUnit. Here’s what the code would look like below. I wouldn’t recommend running it – just take a look at it. The code to me is a mixtures of styles that start to bloat the code. On the one hand, JSUnit has this setup phase with explicit flags and no anonymous functions. On the other hand, because I am using JQuery conventions, I encapsulate alot of that logic. For example, jQuery(function(){}) waits for the page to be loaded before executing ("#tree").story(), Then I have the callback function inline. It looks good from the outside, but it doesn’t work.

The order of calls is: loading, in test and then loaded. Indicating that my JQuery function runs after the test has been run. The order should have been loading, loaded and then in test. In this sense, while setUpPage runs within its own setup/test/teardown cycle. But my JQuery call isn’t linked into this. JQuery waits is waiting on a document flag rather than a custom flag (within testManager). At this point, I do not wish to dig into these libraries to work them out to get it to all play nicely. It wasn’t designed to work this way. Let’s find one that was.

var data = '';

function setUpPage() {
    setUpPageStatus = 'running';
    alert('loading')
    jQuery(function() {
      $("#tree").storyq({
          url: '../../xml/results-01.xml', 
          success: null, 
          load: function(feed) {
            data = feed;
            alert('loaded')
            setUpPageComplete()
            }
          });
    });
}

function setUpPageComplete() {
    if (setUpPageStatus == 'running')
        setUpPageStatus = 'complete';
}

function testXML2Object() {
    alert('in test')
    assertEquals(setUpPageStatus, 'complete');

    assertEquals(data.version, '0.1')
    assertEquals("Number of stories", $(feed).size(), 1)
}

I’m invoking the two-feet principle: I’m moving onto the next framework (after a quick search): QUnit

jQuery and testing – JSUnit, QUnit, JsSpec [Part 3]

November 14th, 2008 No comments

Trying JsSpec with jQuery

This is part three of three. The previous two have been focussed around managing the problem of timing: JSUnit got too hard and QUnit is easy but you still have to manage timings yourself. With JsSpec there is no problem because it is all managed for you. Nice work! Here’s the code I had to write.

A couple of things to writing it. I had to dig into the code to find out the setup/teardown lifecycle keywords. There turns out to be setup/teardown per test (eg before) and per test suite (eg before all). I also had to dig around to find then comparators (eg should_be, should_not_be_null). I couldn’t find any documentation.

describe('I need to read the xml and convert into object', {
  'before all': function() {
    target = {};
    $().storyq({
        url: 'data/results-01.xml', 
        load: '',
        success: function(feed) {
          target = feed
      }
    })
   
  },
  
  'should return an object': function() {
    value_of(target).should_not_be_null()
  },
  
  'should not be a function': function() {
    value_of(typeof target).should_not_be(typeof Function )
  },
  
  'should have a version': function(){
    value_of(target.version).should_be('0.1')
  },
  
  'should have items': function(){
    value_of(target.items).should_not_be_empty()
  },
  
  'should have the same value as the reference object in data/results-01.js': function(){
    value_of(reference).should_not_be_undefined()
    value_of(target).should_be(reference)
  },
  
})

Ihe output looks nice too ;-) Here’s the overall code. Notice that I have also used the technique of a reference object in results-01.js:

&lt;html>
&lt;head>
&lt;title>JSSpec results&lt;/title>
&lt;link rel="stylesheet" type="text/css" href="../lib/jsspec/JSSpec.css" />
&lt;script type="text/javascript" src="../lib/jsspec/JSSpec.js"/>

&lt;script src="../lib/jquery/jquery.js"/>
&lt;script src="../lib/jquery/jquery.cookie.js" type="text/javascript"/>
&lt;script src="../lib/treeview/jquery.treeview.js" type="text/javascript"/>
&lt;script src="../build/dist/jquery.storyq.js" type="text/javascript"/>

&lt;script type="text/javascript" src="data/results-01.js"/>
&lt;script type="text/javascript" src="specs/treeview.js"/>  

&lt;script type="text/javascript">

  describe('I need to read the xml and convert into object', {
    'before all': function() {
      target = {};
      $().storyq({
          url: 'data/results-01.xml', 
          load: '',
          success: function(feed) {
            target = feed
        }
      })

    },

    'should return an object': function() {
      value_of(target).should_not_be_null()
    },

    'should not be a function': function() {
      value_of(typeof target).should_not_be(typeof Function )
    },

    'should have a version': function(){
      value_of(target.version).should_be('0.1')
    },

    'should have items': function(){
      value_of(target.items).should_not_be_empty()
    },

    'should have the same value as the reference object in data/results-01.js': function(){
      value_of(reference).should_not_be_undefined()
      value_of(target).should_be(reference)
    },

  })
&lt;/script>

&lt;/head>
    &lt;body>
      &lt;div style="display:none;">&lt;p>A&lt;/p>&lt;p>B&lt;/p>&lt;/div>
    &lt;/body>
&lt;/html>

JSSpec isn’t written using JQuery. So there are a couple of issues that I can’t pin down. When I get errors it stops the tests completely. I suspect that this is because these tests are using callbacks and they don’t return an (JQuery) object. JQuery does alot of object chaining and JsSpec isn’t cut out for it (I think).

Well, that’s it.

jQuery and testing – JSUnit, QUnit, JsSpec [Introduction]

November 12th, 2008 No comments

I had been writing a jQuery parser and then realised once I had spiked it that I hadn’t actually written any tests. So, these are some results from a spike in unit testing a jQuery plugin.

Some background, the plugin is a results viewer from an xml feed from storyq. So, I have run some tests and have results. Now I want to see them in html format. The plugin merely transforms the xml to be displayed using the treeview plugin. I wanted to avoid handing in a json feed formatted specifically for the treeview. I wanted all this to happen client side.

The tests have two aspects:

  • xml loading and parsing into an object
  • rendering the object into a tree (at that point treeview takes over)

    In short, I want to test the code underlying this call that returns the feed before populating an <ul id="tree"> element:
$('#tree').storyq({
    url: 'tests/data/results-01.xml',   
    success: function(feed) {

      $("#tree").treeview(); //populate the tree
  
  }    
});

Problem for any framework: sequencing

Let’s take a look at what I want as test code. In this code, I want to populate data with the feed variable returned in the success callback. The test can then check for values. Take a look at the code below. When I run the code, I should (ideally) see the sequence of alerts: loaded, start test, end test. Of course, I don’t. I see start start test, end test, loaded as the sequence. That should be obvious that the callback success hasn’t been called as yet: javascript is run sequentially. Okay, nothing here is surprising. I laboured this point because any of the frameworks must deal with this problem.

var data = {};

jQuery(function() {
  $().storyq({
      url: '../../xml/results-01.xml', 
      success: function(feed) {
        data = feed;
        alert('loaded')       
        }
      });
});

function testXML2Object() {
  alert('start test')
  assertEquals(data, {"result"}, "Assume that result is a real/correct object");
  alert('end test')     
}

Frameworks looked at

I looked at three frameworks for xml loading and parsing:

Conclusions

  • In short, QUnit and JsSpec are both as easy as the other. JSUnit seems now to be too heavy given what we now have.
  • QUnit is a TDD framework is used by jQuery itself. I suspect it might survive longer. There are no setup/teardown phases.
  • JsSpec is a BDD framework and doesn’t use jQuery at all but can easily be used with JQuery plugins. There are good setup/teardown phases for tests and suites.
  • Your choice between the two is likely to be your preference between TDD or BDD. It probably depends upon which boundaries you want to test and how you want to go about specifying.

What I didn’t do:

  • integration with a CI system
  • cross-browser testing
  • test with selenium or watir