QA and Javascript Libraries
Javascript libraries have become increasingly common, there’s a whole plethora to choose from, with a different range of feature sets. They allow you to rapidly prototype something, but I will argue below that they shouldn’t yet be used in a real environment. If you’ve followed my usenet discussions, or got me on the topic over a beer or on IRC, you’ll know everything below, the themes have been knocking around my head for a long time, Andrea Campi finally triggered the post.
The first complaint is code size, and it’s not particularly important to me, compared to the second, but I’m only going to address two, and depressingly this is the one other people seem to care about more. Code size, Dean Edwards noted this problem with respect to Yahoo’s library, a tree menu in 250kb (or maybe 100kb of compressed script) [*] Jeff Rafter recently wrote a treeview, it’s 6k or so. I have no idea on the quality, but it’s definately better than the yahoo one in that it degrades to HTML, and has drag’n'drop moving - it was just one I’d recently seen for the example.
For most projects, the javascript is basically incidental, it’s a little effect, or a little piece of user interface like a tree view, that improves the page a little, these are simple, they can be written quickly - re-using code from “a copy and paste library” - and do just what you want and no more, which means they are a fraction of the size for the user to download. With so many of these libraries waiting until onload before they can do anything, this is particularly relevant - the user spends there whole time waiting to interact with the page until everything has downloaded, it may work great when using it locally or on your business adsl, but it’s a nightmare on a broadband connection that’s saturated with p2p data.
QA, or the bigger problem
For me, the much bigger problem is QA’ing sites that are based on libraries, let’s use a hypothetical example. You’re creating a large site, 100 pages, you’re using a library, it does 90% of what you want to do and is well supported, with frequent an irregular releases involving new features and bug fixes.
First up is what do you do about the 10%, to keep with the advantages of a library, you really need to integrate your new features into the library, which might be fine if the library developers accepted your changes into the core library, but we’ll assume they don’t (which isn’t unreasonable, you’d probably not want the cost of making the code universal enough for a library anyway). This means you’ve now got a patched library to the core one available on the website.
Your code arrives at QA, hopefully an internal QA resource, but it could just be the public if you follow the no QA, who cares policy of web 2.0 companies. They find the inevitable bugs, you fix them, feeding the fixes back to the library developer and launch.
Three months later, and want to add a couple of new features to one of your pages, the library has had some changes, this is where the problems arrive. The 10% it doesn’t do is now 5%, but they’ve solved it differently to you did, the bugs you fixed are integrated, and so are a number of others. What do you do? If you change the library to the new one, every one of your 100 pages needs to go through QA again, the impact of changing the library code has an immense cost in time to retest everything, this is of course especially relevant if your fixing a security flaw. Yahoo have already shown they can’t write javascript without security flaws.
Of course if you’ve written the script yourself, you’ll still have issues, but, the script will be smaller, it will do only what’s necessary, the scripts will only touch the pages they’re used in, so if you need to fix a flaw in your tree menu, the tree menu script will only impact those pages with tree-menus on, and there won’t be any external dependency.
This concern mostly comes down to the fact that I always see QA as one of the hardest areas in a web project, QA is often underfunded, but even when not, recruiting good testers is tough, there’s not that many about, and even ones who are good, often want to move into other areas - probably because of the underfunding. Libraries may reduce your development costs a little (although I’m not even convinced of this), but they certainly harm your testing cost.
A unit testing framework for the libraries could certainly change this, however the animation and CSS related issues that so often occure are very difficult to test, browser DOM’s are not deterministic, so can’t be stubbed out, equally often the DOM makes sense, but a bug elsewhere causes the rendering or interaction to fail, these sort of things mean you need humans involved, and humans cost money.
So use the libraries for prototyping, or quick and dirty short term projects, but stay away on the larger project unless your QA resoures are massive.
The size is made worse of course by the fact yahoo’s developer resources aren’t cacheable, particularly bad for a company that employees the author of the cacheability engine.
May 25th, 2006 at 3:57 pm
I hope what you’re saying is that the JavaScript frameworks (and possibly browser JS engine capabilities in terms of library/module loading) are not mature enough yet, because I still firmly believe that we should eventually be moving up the development stack in the web development community. Libraries and frameworks are a sign that a language is starting to mature enough that people want to do even more complex things in the language.
Have you considered intellectual property aspects? If I custom-develop some JavaScript widgets for Company A, I can’t reuse what I’ve learned for Company B because Company A owns that code. Standardizing on libraries or frameworks helps can help in that respect.
Anyway, if I had to write/debug/test my own hashmap or stack in C++/Java every time I started a new project, productivity would dip and I’d go insane. It’s great that I can rely on the STL/Boost or the Java SDK for this.
May 25th, 2006 at 5:21 pm
yahoo’s developer resources are cacheable. the production releases all reside under the yimg.com domain. yahoo is guilty of a lot, but cache-friendliness is not one of them. compare their expires headers, division of delivery to honor http limitations, use of akamai, etc. this is not excusing bloated or buggy code though.
May 25th, 2006 at 5:38 pm
grumpY! Check the link to the cacheability engine, the Yahoo developer resources there are not cacheable.
May 25th, 2006 at 6:58 pm
jim, lets pull a random yahoo JS lib and put this issue to rest:
http://us.js2.yimg.com/us.js.yimg.com/lib/common/utils/2/connection_2.0.0-b4.js
a beta version of one of their files
this is being served via a domain that has akamai coverage
this has expiration headers in 2016
the url contains no gibberish or unique metadata that might impair caching
if your tool says this cannot or is not cached, your tool is wrong
May 25th, 2006 at 8:06 pm
grumpY! - the URL I get from the tree menu page is http://com1.devnet.scd.yahoo.com/yui/build/connection/connection.js not, one of the yimg ones, and it’s not cacheable, it’s not my tool, it’s the excellent cacheability engine from Mark Nottingham which says it’s not cacheable, but any simple inspection shows it’s not cacheable. I dunno what pages you’re looking at, but I’m looking at http://com2.devnet.scd.yahoo.com/yui/examples/treeview/default.html?mode=dist which is the url the search engines provide for it, if alternatively I use http://developer.yahoo.com/yui/treeview/ which appears to be the same http://www.web-caching.com/cgi-web-caching/cacheability.py?query=http%3A%2F%2Fdeveloper.yahoo.com%2Fyui%2Ftreeview%2F&descend=on then I get similar results.
So whilst I agree the yahoo site itself is very good at organising caching, the developer site doesn’t appear to be at all, where did you get the link above from?
May 25th, 2006 at 9:36 pm
first of all i did not mean to insult the author of the caching tool. if that was perceived i apologize.
the link was one i pulled internally from the development site for these libs. i don’t mind posting it here because i don’t believe that it is considered “secret”, these new versions are set to be delivered shortly, and the yimg urls are used internally (as they are used on any non .html files on yahoo’s sites, or should be)
anyhow, i now see where you are getting your file, from the “official” ydn site. for example
http://developer.yahoo.com/yui/build/yahoo/yahoo.js
which does NOT have correct caching, as you point out. you are right and i am wrong.
the only consolation i can provide is that the ydn site appears to strongly suggest that users serve these files from their own site….where maybe they can implement proper caching.
May 25th, 2006 at 9:55 pm
Yeah, the developer site is not a fair example of the use in the wild, it’s bloated because of comments too, which you’d strip as part of a release to testing hopefully. I didn’t really mean to pick on yahoo’s library, it’s certainly very good, it was just easiest as Dean Edwards had already done the effort on the size of the library.
So don’t think I’m criticising the yahoo library in particular, it applies to all libraries, the yahoo one is one of the better libraries itself, my advise is do no more than demos and prototypes with it though.
May 25th, 2006 at 10:11 pm
Jeff,
Libraries and frameworks are a sign that a language is starting to mature enough that people want to do even more complex things in the language.
That depends, it doesn’t actually mean you should be doing more complex things in the language, it may be that the language isn’t right for doing these more complicated things - like writing an operating system in VB, it’s not going to help no matter how many libraries you bring to the party.
Licensing of code snippets isn’t a problem I think, if your snippets are large enough to carry any IPR, then you’re not following my maxim of minimum code required to the specific task you have, or you are doing very, very similar tasks each time - if that’s the case then certainly try to get the code opened up so you can licence it to each client, but I don’t see the premise that starting with a library and building your 10% actually makes any difference, you’ll still be building your same 10% for the next customer, and the customer will have all the same size and QA issues.
Certainly changing javascript into some object orientated language with packages would help in some ways, but it doesn’t solve every issue.
May 26th, 2006 at 12:19 pm
One other thing that I may have missed in your post: I was never suggesting referencing the library from my site to the Yahoo site. I would only ever consider using a JavaScript library/framework by keeping a local copy on my site so the version is frozen and I have control over it. Doing anything else sounds insane.
>> Libraries and frameworks are a sign that a language
>> is starting to mature enough that people want to do
>> even more complex things in the language.
> That depends, it doesn’t actually mean you should be doing more complex things in the language, it may be that the language isn’t right for doing these more complicated things
Jim, that’s a rather obvious statement. But are you suggesting that JavaScript is not an appropriate language for handling events, creating user interface widgets, animations? C’mon, I think the last year or two have proven that false…
May 26th, 2006 at 12:52 pm
The frozen version is part of the QA issue Jeff, what do you do when you’ve got bugs that need fixing - fix it in your local frozen copy - diverging your copy from the release copy, or do you install the new version - the new version entails the entire site QAing.
But are you suggesting that JavaScript is not an appropriate language for handling events, creating user interface widgets, animations?
Nope, I’m suggesting javascript is not an appropriate language for creating a framework that does those things, I’m saying the correct boundary is the browser API, and not an API authored in javascript.
May 26th, 2006 at 2:19 pm
> The frozen version is part of the QA issue Jeff, what do you do when you’ve got bugs that need fixing - fix it in your local frozen copy - diverging your copy from the release copy, or do you install the new version - the new version entails the entire site QAing.
It’s just like ANY application dependency. If it’s under some open-source type license, you can fix the bugs yourself and submit patches back to the library owner. Another option is to submit a bug and wait for a new version and perform testing. This happens with Boost, with cairo, with OpenGL, with SDL, and millions of other dependencies. Should Mozilla have implemented their own vector graphics library or should they work closely with the cairo team and perform regular “uplifts”?
Sorry, it may be I’m just being being dense, but I don’t see the difference just because the application is delivered over the web via a browser vs. a standalone application that runs on the operating system. Is it because of the nature of JavaScript relatively slow or interpreted or …? Again, I’m assuming that, as an application developer, you are using a “frozen” JavaScript library/framework here and not something cross-domain linked.
Also, can you clarify what you mean by the “browser API” in your second comment?
May 26th, 2006 at 3:54 pm
Jeff, it’s not like any other application dependency though so comparing it to Boost or cairo is crazy, they’re completely different levels. the javascript is not and cannot be isolated from its environment, both the other script, but more importantly the mark-up, so there’s no isolation of these things. You keep saying “perform testing” as if it was nothing, performing testing on a Bosst Regex library is easy, you check it passes all the unit tests, you can’t do that with a framework in a browser. With a scrit library testing is a slow, very labourious process that has to revisit every page on your site, if you’re updating a script library, you have to do that, as the script touches every page, even if your bug was only in a single page. Without the library that is the only one you’d touch the code of, just like in a C++ situation the only things that need testing is the interface to the library - it’s also simpler to test as you don’t need to test the interface between the library and every execution evironment you have.
The “browser API”, is the point to divide the development, the external dependencies is the browser, not some library, the library is just something that increases cost, without any benefit other than development time (and that is more than sacrificied in QA time, or often quality as people don’t do QA)
May 28th, 2006 at 1:35 am
“Three months later, when you want to add a couple of new features to one of your pages…”
Well if it is not broken, don’t fix it. Unless “a couple of features” is really a substantial upgrade, then I simply don’t see the need to upgrade the library as well. Unless, of course, new features exist in the library that will cut down the amount of work required to implement your new features. If that’s the case, you would still have to factor in the time it will take to test the new library.
Whilst I agree with you that coding against the browser API has its advantages, I disagree that frameworks in general have such an affect on QA that you claim. Don’t forget that since so many other developers are using the same framework, problems are more likely to have been found and solved, than with your own, built-from-scratch solution.
I think your post does highlight a programmer’s dilemma in using framework libraries with any language, but I think the answer to the problem lies in knowing when and when not to upgrade.
May 28th, 2006 at 10:11 am
Adriand - so in your example, you code the new features in script that exists purely in the pages you want to update? So losing lots of the benefits of the library. Basically just using the library for the initial stuff - I don’t see the value in that, you’re not really getting the benefit of the library.
The current frameworks are certainly massively in need of QA - these aren’t isolated library’s, they all depend on what else is in the document, both script, html and css, all can easily be broken by any one of those things - a library that coped with every such change would be so large as to be useless - not that someone is going to write one. And of course the library supposedly does so much more than your single solution, so you’re relying there not being interactions with all of that.
May 29th, 2006 at 6:11 am
“Adriand - so in your example, you code the new features in script that exists purely in the pages you want to update? So losing lots of the benefits of the library”
Well, yeah, if you are tasked with updating certain pages of the site then yes I would apply new script purely to those pages, working within the confines of the libray that already exists across the site. Why does that mean I’m not getting the benefit of the library? Why can’t I use the existing library when updating these pages? If the library was good enough to build 100 pages around, then it’s likely going to be good enough when updating a few pages too. If not, then a) bypass the library, b) apply a newer version of the library to these particluar pages only, c) upgrade the entire site to the new library and factor in the time to QA the entire site.