the tar pit » jquery

Having Jasmine tests results in TeamCity via node.js (on windows) invoked from powershell

todd — Mon, 26 Sep 2011 03:47:52 +0000

I test my javascript code via jasmine on a windows machine. I primarily write jquery-plugin-style code. Now I need to get this onto CI. A colleague I worked with took my test strategy in jasmine and wrote a library to run it very quickly on node.js. There are other places showing how to integrate with TeamCity and Jasmine with JsTestDriver or Qunit and also the documentation on how to easily integrate service messages with TeamCity.

One caveat: I am not wanting to test cross-browser functionality. Therefore, I don’t need or want a browser or the associated slowness and the brittleness of cross-process orchestration. (Note: I have tried to stabilise these types of tests using NUnit and MSTest runners invoking selenium and/or watin – it gets unstable quickly and there is too much wiring up.)

So this approach is simple and blindly fast thanks to Andrew McKenzie’s jasmine-node-dom which is an extension of jasmine-dom. He wrote his for linux and his example is with Jenkins so I have forked a version for windows which has a node.exe binary which is available form node.js.

Anyway, this blog covers the powershell script to invoke the jasmine-node-dom and publish it to TeamCity.

Here’s the script:

build.ps1 (or directly in TeamCity)

	
	$node_dir = "node-jasmine-dom\bin"	

	& "$node_dir\node.exe" "$node_dir\jasmine-dom" `   
				--config tests.yaml `
				--format junit `
				--output javascript-results.xml 

	write-host "##teamcity[importData type='junit' path='javascript-results.xml']"

An explanation if it isn’t obvious. First let’s start with files that are needed. I have the windows node-jasmine-dom installed in its own directory. I then call node.exe with jasmine-dom. That should just work all out-of-the-box. I then tell it where the manifest is that knows about the tests (tests.yaml – see below for example) and then I give it the results file. jasmine-node-dom is great because it reads the SpecRunner.html and reconstructs the DOM enough that the tests are valid.

Finally, I tell teamcity to read the results out of junit. This is very easy and I recommend that you find out what else you need to do.

tests.yaml

	---
	  test_one:
	    name: Example test one
	    runner: ./tests/SpecRunner.html

This yaml file points to the Jasmine runner.

Other points:

* All my jasmine tests are invoked from their own SpecRunner.html file by convention
* I will write a script that will automatically generate the yaml file
* I always put all my powershell scripts into psake scripts (then they can be run by the dev or the build machine)
* my code isn’t quite filed as above

Summary Instructions:

# download jasmine-node-dom and install in tools\ directory
# add new Task to build scripts (I use psake)
# add a new test.yaml manifest (or build one each time)
# Add new jasmine tests via SpecRunner.html with your javascript
# Ensure that the build script is run via a build step in a configuration from TeamCity
# Now you can inspect the Tests in TeamCity

Sharepoint & TDD: Getting started advice

todd — Thu, 30 Jun 2011 23:19:44 +0000

I have a couple of people asking lately about starting on SharePoint. They’ve asked about how to move forward with unit and integration testing and stability. No one wants to go down the mocking route (typemock, pex and moles) and quite rightly. So here’s my road map:

The foundations: Hello World scripted and deployable without the GUI

Get a Hello World SharePoint “app” – something that is packageable and deployable as a WSP
Restructure the folders of the code away from the Microsoft project structure so that the root folder has src/, tools/, lib/ and scripts/ folders. All source and tests are in src/ folder. This lays the foundation for a layered code base. The layout looks like this sample application
Make the compilation, packaging, installation (and configuration) all scripted. Learn to use psake for your build scripts and powershell more generally (particularly against the SharePoint 2010 API). The goal here is that devs can build and deploy through the command line. As such, so too can the build server. I have a suggestion here that still stands but I need to blog on improvements. Most notably, not splitting out tasks but rather keeping them in the same default.ps (because -docs works best). Rather than get reuse at the task level do it as functions (or cmdlets). Also, I am now moving away from the mix with msbuild that I blogged here and am moving them into powershell. There is no real advantage other than less files and reduced mix of techniques (and lib inclusions).
Create a build server and link this build and deployment to it. I have been using TFS and TeamCity. I recommend TeamCity but TFS will suffice. If you haven’t created Build Definitions in TFS Workflow allow days-to-weeks to learn it. In the end, but only in the end, it is simple. Becareful with TFS, the paradigm here is that build server does tasks that devs don’t. It looks a nice approach. I don’t recommend it and there is nothing here by design that makes this inevitable. In TFS, you are going to need to build two build definitions: SharePointBuild.xaml and SharePointDeploy.xaml. The build is a compile, package and test. The deploy simply deploys to an environment – Dev, Test, Pre-prod and Prod. The challenge here is to work out a method for deploying into environments. In the end, I wrote a simple self-host windows workflow (xamlx) that did the deploying. Again, I haven’t had time to blog the sample. Alternatively, you can use psexec. The key is that for a SharePoint deployment you must be running on the local box and the most configurations have a specific service account for perms. So I run a service for deployment that runs under that service account.

Now that you can reliably and repeatably test and deploy, you are ready to write code!

Walking Skeleton

Next is to start writing code based on a layered strategy. What we have found is that we need to do two important things: (1) always keep our tests running on the build server and (2) attend to keeping the tests running quickly. This is difficult in SharePoint because a lot of code relates to integration and system tests (as defined by test automation pyramid). We find that integration tests that require setup/teardown of a site/features get brittle and slow very quickly. In this case, reduce setup and teardown in the the system tests. However, I am also had a case where the integration test showed that a redesigned object (that facaded SharePoint) would give better testability for little extra work.

Create 6 more projects based on a DDD structure (Domain, Infrastructure, Application, Tests.Unit, Tests.Integration & Tests.System). Also rename your SharePoint project to UI-[Your App], this avoids naming conflicts on a SharePoint installation. We want to create a port-and-adapters application around SharePoint. For example, we can wrap property bags with repository pattern. This means that we create domain models (in Domain) and return them with repositories (in Infrastructure) and can test with integration tests.
System tests: I have used StoryQ with the team to write tests because it allows for a setup/teardown and then multiple test scenario. I could use SpecFlow or nBehave just as easily.
Integration tests: these are written classical TDD style.
Unit tests: these are written also classical TDD/BDD style
Javascript tests: we write all javascript code using a jQuery plugin style (aka Object Literal) – in this case, we use JSSpec (but I would now use Jasmine) – we put all tests in Tests.Unit but the actual javascript is still in the UI-SharePoint project. You will need two sorts of tests: Example for exploratory testing and Specs for the jasmine specs. I haven’t blogged about this and need to but is based on my work for writing jQuery plugins with tests.
Deployment tests: these are tests that run once that application is deployed. You can go to an ATOM feed which returns the results of a series of tests that run against the current system. For example, we have the standard set with tells us the binary versions and which migrations (see below) have been applied. Others check whether a certain wsp has been deployed, different endpoints are listening, etc. I haven’t blogged this code and mean to – this has been great for testers to see if the current system is running as expected. We also get the build server to pass/fail a build based on these results.

We don’t use Pex and Moles. We use exploratory testings to ensure that something actually works on the page

Other bits you’ll need to sort out

Migrations: if you have manual configurations for each environment then you’ll want to script/automate this. Otherwise, you aren’t going to be one-click deployments. Furthermore, you’ll need to assume that each environment is in a different state/version. We use migratordotnet with a SharePoint adapter that I wrote – it is here for SharePoint 2010 – there is also a powershell runner in the source to adapt – you’ll need to download the source and compile. Migrations as an approach works extremely well for feature activation and publishing.
Application Configuration: we use domain models for configuration and then instantiate via an infrastructure factory – certain configs require SharePoint knowledge
Logging: you’ll need to sort of that Service Locator because in tests you’ll swap it out for Console.Logger
WebParts: can’t be in a strongly typed binary (we found we needed another project!)
Extension Methods to Wrap SharePoint API: we also found that we wrapped a lot of SharePoint material with extension methods

Other advice: stay simple

For SharePoint developers not used to object oriented programming, I would stay simple. In this case, I wouldn’t create code with abstractions that allowed you to unit test like this. I found in the end the complexity and testability outweighed the simplicity and maintainability.

Microsoft itself has recommended the Repository Pattern to facade the SharePoint API (sorry I can’t for the life of me find the link). This has been effective. It is so effective we have found that we can facade most SharePoint calls in two ways: a repository that returns/works with a domain concept or a Configurator (which has the single public method Process()).Anymore than that it was really working against the grain. All cool, very possible but not very desirable for a team which rotates people.

jquery-bdd

todd — Sun, 01 Aug 2010 02:06:04 +0000

Notes from a jQuery session

Structure of session:

Jump into an example
Build some new functionality
Come back and see the major concepts
Look at what is actually needed to treat javascript as a first-class citizen
Different testing libraries

Quick why a library in javascript?

cross-browser abstraction (dynduo circa 1998 was still hard!)
jQuery, MooTools, Extjs, Prototype, GWT, YUI, Dojo, …
I want to work with DOM with some UI abstractions – with a little general purpose
simple HTML traversing, event handling
also functional, inline style
I want plugin type architecture

Story plugin demo

Viewer of StoryQ results
StoryQ produces XML, this widget gives the XML a pretty viewer

A screenshot and demo

What would your acceptance criteria be? What do you think some of the behaviours of this page are?

Acceptance:

eg should display the PROJECT at the top with the number of tests

Behaviours:

think in terms of themes: data, display, events

The tests … what does the application do?

run the tests and see the categories
data loading: xml
display: traversing xml and creating html
events: click handlers

Let’s build some new functionality!

Goal: add “Expand All | Contract All | Toggle” functionality to the page

Acceptance:

The user should be able to expand, collapse or toggle the tree

Specs

Display:
* should show “Expand All | Contract All | Toggle”
Events:
* should show all results when clicking expand all
* should show only top class when clicked contract all
* should toggle between all and one when clicking on toggle

Coding: Add acceptance

Add Display specs

Add Event specs

Return back to completing the Acceptance

Major aspects we covered

HTML traversing

I want to program akin to how I look at the page
I may look for: an element, a style, some content or a relationship
then perform an action

$(‘div > p:first’)
$(‘#mylist’).append(“

another item

“)

Event handling

I want to look at page and add event at that point
I want to load data (ie xml or json)

$(‘div’).click(function(){
alert(“div clicked”)
})
$(‘div’).bind(‘drag’, function(){
$(this).addClass(‘dragged’)
})
$(‘#results’).load(‘result.html’)
$.get(‘result.xml’, function(xml){
$(“user”, xml).each(function(){
$(“

“).text($(this).text() .appendTo(“#mylist”))
})
})

Functional style

almost everything in jQuery are JQuery objects
that returns an object
every method can call a jQuery object
that means you can chain
plus I want it to be short code

$(‘‘)
.addClass(‘indent’)
.addClass((idx == 4) ? ‘scenario’ : ”)
.text($(this).attr(‘Prefix’) + ‘ ‘ + $(this).attr(‘Text’))
.append($(‘‘).text(“a child piece of text”)
.click(function(){ $(this).addClass(‘click’)}))
.appendTo(results)

Plugin architecture

drop in a widget (including my own)
then combine, extend
help understand customisation
basically just work

$(‘#tree’).treeview();
$.ajax({
url: ‘/update’,
data: name,
type: ‘put’,
success: function(xml){
$(‘#flash’).text(“successful update”).addClass(‘success’)
}
})

With power and simplicity … comes responsibility

the need to follow conventions
– plugins return an array
– plugins accept parameters but have clear defaults
– respect namespace
the need for structure
– test data
– min & pack
– releases
the need to avoid mundane, time consuming tasks
– downloading jquery latest
– download jQuery UI
– building and releasing packages
needs tests
– I use jsspec

Sounds like real software development?

Treat javascript as a first-class citizen

Give your plugin a directory structure:

/src
/css
/images
query.plugin.js
/test
spec_plugin.js
acceptance_plugin.js
specs.html
acceptance.html
/lib
/jsspec
jquery.min.js
jquery-ui.js
/themes
/base
example.html
Rakefile
History.txt
README.txt

Generate your plugin boilerplate code

jQuery Plugin Generator
* gem install jquery-plugin-generator

(function($) {
$.jquery.test = {
VERSION: “0.0.1″,
defaults: {
key: ‘value’
}
};

$.fn.extend({
jquery.test: function(settings) {
settings = $.extend({}, $.jquery.test.defaults, settings);
return this.each( function(){
self = this;
// your plugin

})
}
})
})(jQuery);

Use a build tool to do the … ah … building

rake acceptance # Run acceptance test in browser
rake bundles:tm # Install TextMate bundles from SVN for jQuery and…
rake clean # Remove any temporary products.
rake clobber # Remove any generated file.
rake clobber_compile # Remove compile products
rake clobber_package # Remove package products
rake compile # Build all the packages
rake example # Show example
rake first_time # First time run to demonstrate that pages are wor…
rake jquery:add # Add latest jquery core, ui and themes to lib
rake jquery:add_core # Add latest jQuery to library
rake jquery:add_themes # Add all themes to libary
rake jquery:add_ui # Add latest jQueryUI (without theme) to library
rake jquery:add_version # Add specific version of jQuery library: see with…
rake jquery:packages # List all packages for core and ui
rake jquery:packages_core # List versions of released packages
rake jquery:packages_ui # List versions of released packages
rake jquery:versions # List all versions for core and ui
rake jquery:versions_core # List jQuery packages available
rake jquery:versions_ui # List jQuery UI packages available
rake merge # Merge js files into one
rake pack # Compress js files to min
rake package # Build all the packages
rake recompile # Force a rebuild of the package files
rake repackage # Force a rebuild of the package files
rake show # Show all browser examples and tests
rake specs # Run spec tests in browser

Testing … acceptance and specs

specs: BDD style – DOM traversing and events
acceptance: tasks on the GUI with the packaged version (minified or packed)
these both server as good documentation as well
plus, you have demo page baked in!

Compiling and packaging

Compiling javascript … hhh? … yes, if you minify or pack your code
gzip compression with caching and header control is probably easier though
packed code is HARD to debug if it doesn’t work

Now you are ready for some development

Different ways to test Javascript

Understanding JavaScript Testing

testing for cross-browser issues – so this is useful if you are building javascript frameworks
Unit – QUnit, JsUnit, FireUnit,
Behvaiour – Screw.Unit, JSSpec, YUITest,
Functional with browser launching – Selenium (IDE & RC & HQ), Watir/n, JSTestDriver, WebDriver
Server-side: Crosscheck, env.js, blueridge
Distributed: Selenium Grid, TestSwarm

Reference

jQuery CheatSheet

slide 48: great explanation of DOM manipulation – append, prepend, after, bfore, wrap, replace …

jQuery and testing — JSUnit, QUnit, JsSpec [Part 2]

todd — Sat, 15 Nov 2008 10:34:14 +0000

Trying QUnit with jQuery

In the JSUnit entry, one of the main problems was with the sequencing of calls. Let’s see how QUnit handles this. QUnit has a simple implementation to this problem: stop() and start() commands to synchronise sequences. The basic approach is that it calls a test function. With the stop() command, it will not call the next test until you say start(). But it does complete the rest of the current test.

The code that I ended up with was just what I wanted compared to JsUnit. Previously, I had said that basically I wanted to call my function and then process the result that it was in fact what I wanted. Here is the main javascript code (it’s nice and concise).

One point about the code at this stage. The tests had to be inside the success function to be run. I wonder if this is going to create a code smell in the long run. Plus there is no setup/teardown cycles, again I wonder what that will mean in the long run. Perhaps not?

module("XML to object");

test("Check the xml to object conversion without showing the tree", function() {
  
  expect( 5 )
  stop();
  
   $().storyq({
        url: 'data/results-01.xml', 
        load: '',
        success: function(feed) {
          ok( feed, "is an object" )
          ok( !$.isFunction(feed), "is not a function" )
          ok( feed.version, "has a version: " + feed.version )
          ok( feed.items, "has items")
          same( feed, reference, "is the same as the reference object in data/results-01.js")
          start();
        }
    });

});

Honestly, it is that easy.

Here’s a couple of features that took me half an hour to work out restated from above. (1) stop() and start() almost work as you expect – but I had to put some alerts in to check the order of execution. Basically, a stop() halts any new tests from executing but it keeps the current test executing. The effect of this is that the asynchronise call can be completed. start() then tells the tests to start running again. If you don’t have a start() then you will find that your test runner halts altogether. There is another option and that is to put a timer on the stop and then you don’t need a start. I prefer to keep the tests running as quickly as possible.

Just another note. I decided to do a same() comparison. I have saved and preloaded an object from a file for ease. This kept the test easy to read – my reference object here is quite long. You can see the insertion of this file in the entire code below