I'm working on a project that uses a certain amount of JavaScript to build a dynamic UI, the project team were of the opinion that we used barely any JavaScript and were flummoxed as to why the integration tests using Selenium with the HtmlUnit Driver were so slow.
I finally got time a couple of weeks ago to investigate the problem and because I tend to avoid JavaScript like the plague, it took me a little while to garner the necessary skills and utilities to diagnose the problems.
I thought I would share the lessons I learnt in case anyone else comes across similar problems.
The first thing I did was to upgrade all the libraries to their latest versions this often fixes performance bugs and can highlight problems with the code base. In my case it only garnered a minor speed up but highlighted the fact that the old version of Selenium we were using allowed interaction with hidden and disabled HTML elements and a number of our tests were badly written.
The second thing I discovered is that the Selenium / HtmlUnit event model means that SELECT element change events were only fired when focus changed to another element in the page.
Seeing no major speed increase I instead reverted to profiling the Java processes. We don't have access to a commercial profiler but a remarkable amount can be achieved using VisualVM which is included as part of the Sun JDK since about version 6.012.
The first thing you should always do when installing this utility is to increase the memory available using the VisualVM options file in the JDK configuration folders.
I was very careful to limit what I profiled to only a certain subset of the classes involved as even with the tweaked memory settings VisualVM is a little flaky when profiling large, long-running processes.
I used the CPU profiling capability and focused in on the projects test classes, the Selenium classes and the HtmlUnit classes.
After a certain amount of faffing around I discovered that a large amount of time was spent in the JavaScript and particularly around event bubbling and DOM change listening.
Unfortunately there is a certain level of indirection involved in the JavaScript implementation and could not discover precisely which JavaScript elements were involved even when using Java debugging.
I instead reverted to using FireBug and FireQuery to profile and navigate through the JavaScript within Firefox. I quickly discovered some obvious JavaScript performance problems and fixed them. Equally rapidly I discovered that the performance characteristics of the Firefox JavaScript engine is very different to those of the HtmlUnit JavaScript engine - Mozilla Rhino.
FireBug and FireQuery were excellent for helping me to understand what was executing and would be perfect for profiling browser performance issues, I would have to look elsewhere for the root cause of the HtmlUnit performance problems.
I resorted to old-school - comment stuff out until it broke or performed faster.
The problem was rapidly resolved to being the use of JQuery live() methods. They are designed to handle events highly dynamic DOMs by listening to events that have bubbled all the way up to the root and then matching the source of the event to a selector before calling the method intended to handle the event.
I replaced them with JQuery bind() and delegate() methods and saw a massive performance increase: A UI test suite that took 26 minutes to run completed in 10.
I learnt that highly dynamic JQuery based scripting could cause severe test performance problems and learnt a lot about how it worked. Longer term I'm going to move to statically bound JavaScript as much as possible particularly when I need to be able to test it.
A framework like JQuery is a little like an iceberg, it may look like you're barely using any JavaScript but under the surface...
- Posted using BlogPress from my iPad
