I'm working on a project that uses a certain amount of JavaScript to build a dynamic UI, the project team were of the opinion that we used barely any JavaScript and were flummoxed as to why the integration tests using Selenium with the HtmlUnit Driver were so slow.
I finally got time a couple of weeks ago to investigate the problem and because I tend to avoid JavaScript like the plague, it took me a little while to garner the necessary skills and utilities to diagnose the problems.
I thought I would share the lessons I learnt in case anyone else comes across similar problems.
The first thing I did was to upgrade all the libraries to their latest versions this often fixes performance bugs and can highlight problems with the code base. In my case it only garnered a minor speed up but highlighted the fact that the old version of Selenium we were using allowed interaction with hidden and disabled HTML elements and a number of our tests were badly written.
The second thing I discovered is that the Selenium / HtmlUnit event model means that SELECT element change events were only fired when focus changed to another element in the page.
Seeing no major speed increase I instead reverted to profiling the Java processes. We don't have access to a commercial profiler but a remarkable amount can be achieved using VisualVM which is included as part of the Sun JDK since about version 6.012.
The first thing you should always do when installing this utility is to increase the memory available using the VisualVM options file in the JDK configuration folders.
I was very careful to limit what I profiled to only a certain subset of the classes involved as even with the tweaked memory settings VisualVM is a little flaky when profiling large, long-running processes.
I used the CPU profiling capability and focused in on the projects test classes, the Selenium classes and the HtmlUnit classes.
After a certain amount of faffing around I discovered that a large amount of time was spent in the JavaScript and particularly around event bubbling and DOM change listening.
Unfortunately there is a certain level of indirection involved in the JavaScript implementation and could not discover precisely which JavaScript elements were involved even when using Java debugging.
I instead reverted to using FireBug and FireQuery to profile and navigate through the JavaScript within Firefox. I quickly discovered some obvious JavaScript performance problems and fixed them. Equally rapidly I discovered that the performance characteristics of the Firefox JavaScript engine is very different to those of the HtmlUnit JavaScript engine - Mozilla Rhino.
FireBug and FireQuery were excellent for helping me to understand what was executing and would be perfect for profiling browser performance issues, I would have to look elsewhere for the root cause of the HtmlUnit performance problems.
I resorted to old-school - comment stuff out until it broke or performed faster.
The problem was rapidly resolved to being the use of JQuery live() methods. They are designed to handle events highly dynamic DOMs by listening to events that have bubbled all the way up to the root and then matching the source of the event to a selector before calling the method intended to handle the event.
I replaced them with JQuery bind() and delegate() methods and saw a massive performance increase: A UI test suite that took 26 minutes to run completed in 10.
I learnt that highly dynamic JQuery based scripting could cause severe test performance problems and learnt a lot about how it worked. Longer term I'm going to move to statically bound JavaScript as much as possible particularly when I need to be able to test it.
A framework like JQuery is a little like an iceberg, it may look like you're barely using any JavaScript but under the surface...
- Posted using BlogPress from my iPad
1 comment:
Hi Bob,
A while back I needed to simulate user behaviour in the browser but needed to run the scripts in a background process. I initially resorted to JRuby and using Celerity. After some on-going problems with memory usage I decided to look at the underlying code and see whether I could improve.
Celerity is a JRuby wrapper around HtmlUnit so thought I may as well look at using this directly instead of adding another layer into the mix.
Couple of lessons I learnt quickly (bearing in mind I have no control over the quality of the JS on the remote site!)
1) I started using version 2.9 of HtmlUnit and again ran into excessive memory problems (approx 80% usage, stable but still high). I decided to downgrade after reading some issues on the forums, and chose v.2.8. Instantly our memory problems were resolved and now the suite runs comfortably at around 18% memory usage on the server
2) We had all sorts of problems with the validity of the JS on the remote site to the point where even filling in a box and clicking a button was causing exceptions to be thrown from HtmlUnit. We got round this by setting the following:
webClient.setThrowExceptionOnScriptError(false);
3) Some of the methods (i.e. click()) were failing the first time even though I could see them in the HtmlPage object. I even tried adding a waitForId method but this passed (element was obtainable) but the click even t still do not perform correctly. I resolved simply by waiting for 0.5second and running the click event again.
Just some additional pointers some may find useful.
Post a Comment