QA and Javascript Libraries
Thursday, May 25th, 2006Javascript libraries have become increasingly common, there’s a whole plethora to choose from, with a different range of feature sets. They allow you to rapidly prototype something, but I will argue below that they shouldn’t yet be used in a real environment. If you’ve followed my usenet discussions, or got me on the topic over a beer or on IRC, you’ll know everything below, the themes have been knocking around my head for a long time, Andrea Campi finally triggered the post.
The first complaint is code size, and it’s not particularly important to me, compared to the second, but I’m only going to address two, and depressingly this is the one other people seem to care about more. Code size, Dean Edwards noted this problem with respect to Yahoo’s library, a tree menu in 250kb (or maybe 100kb of compressed script) [*] Jeff Rafter recently wrote a treeview, it’s 6k or so. I have no idea on the quality, but it’s definately better than the yahoo one in that it degrades to HTML, and has drag’n'drop moving - it was just one I’d recently seen for the example.
For most projects, the javascript is basically incidental, it’s a little effect, or a little piece of user interface like a tree view, that improves the page a little, these are simple, they can be written quickly - re-using code from “a copy and paste library” - and do just what you want and no more, which means they are a fraction of the size for the user to download. With so many of these libraries waiting until onload before they can do anything, this is particularly relevant - the user spends there whole time waiting to interact with the page until everything has downloaded, it may work great when using it locally or on your business adsl, but it’s a nightmare on a broadband connection that’s saturated with p2p data.
QA, or the bigger problem
For me, the much bigger problem is QA’ing sites that are based on libraries, let’s use a hypothetical example. You’re creating a large site, 100 pages, you’re using a library, it does 90% of what you want to do and is well supported, with frequent an irregular releases involving new features and bug fixes.
First up is what do you do about the 10%, to keep with the advantages of a library, you really need to integrate your new features into the library, which might be fine if the library developers accepted your changes into the core library, but we’ll assume they don’t (which isn’t unreasonable, you’d probably not want the cost of making the code universal enough for a library anyway). This means you’ve now got a patched library to the core one available on the website.
Your code arrives at QA, hopefully an internal QA resource, but it could just be the public if you follow the no QA, who cares policy of web 2.0 companies. They find the inevitable bugs, you fix them, feeding the fixes back to the library developer and launch.
Three months later, and want to add a couple of new features to one of your pages, the library has had some changes, this is where the problems arrive. The 10% it doesn’t do is now 5%, but they’ve solved it differently to you did, the bugs you fixed are integrated, and so are a number of others. What do you do? If you change the library to the new one, every one of your 100 pages needs to go through QA again, the impact of changing the library code has an immense cost in time to retest everything, this is of course especially relevant if your fixing a security flaw. Yahoo have already shown they can’t write javascript without security flaws.
Of course if you’ve written the script yourself, you’ll still have issues, but, the script will be smaller, it will do only what’s necessary, the scripts will only touch the pages they’re used in, so if you need to fix a flaw in your tree menu, the tree menu script will only impact those pages with tree-menus on, and there won’t be any external dependency.
This concern mostly comes down to the fact that I always see QA as one of the hardest areas in a web project, QA is often underfunded, but even when not, recruiting good testers is tough, there’s not that many about, and even ones who are good, often want to move into other areas - probably because of the underfunding. Libraries may reduce your development costs a little (although I’m not even convinced of this), but they certainly harm your testing cost.
A unit testing framework for the libraries could certainly change this, however the animation and CSS related issues that so often occure are very difficult to test, browser DOM’s are not deterministic, so can’t be stubbed out, equally often the DOM makes sense, but a bug elsewhere causes the rendering or interaction to fail, these sort of things mean you need humans involved, and humans cost money.
So use the libraries for prototyping, or quick and dirty short term projects, but stay away on the larger project unless your QA resoures are massive.
The size is made worse of course by the fact yahoo’s developer resources aren’t cacheable, particularly bad for a company that employees the author of the cacheability engine.
206a