You are reading a MIX Online Opinion. In which we speak our minds. Joshua Allen Meet Joshua Arrow

Opinions

18Comment Retweet

Web Standards Gone Wild

Feb 18, 2009 By Joshua Allen

I recently attended the excellent “Web Directions North” conference, and the opening keynote by Dan Connolly of the W3C really brought into clarity something that’s been gestating in my mind for
awhile now. Dan was editor of the original HTML 1.0 specification and began his keynote by explaining that he is a tinkerer who likes to look at
specs and read BNF — he then asked for a show of hands of people who can read BNF. Shockingly, only a few hands went up.

At first glance, Dan’s statements could be seen as a sort of boastful way of pointing out how “elite” he is. But I didn’t take it that way, and
he quickly put that idea to rest by describing how the W3C didn’t want to take on JavaScript as a spec, since “it wasn’t in our core competence”.
Dan’s comments highlighted an important trend in our industry — that normal people are no longer comfortable reading specs. This is a tragedy,
IMO, and the blame rests squarely on the shoulders of specification authors.

The truth is, specs used to be a lot simpler, shorter, and cleaner. I implemented my own IP stack by reading RFC 791, and I implemented an HTTP
server shortly after RFC 1945 was first drafted. These are two of the most fundamental technologies upon which the modern Web is built,yet any
determined amateur can implement them by reading the simple specifications. In fact,I wrote my own implementations of most of the important
technologies in the early years of the web:

Now contrast this with the HTML 5 specification, which is nowhere near being done, but
already more than 268,000 words! This is nearly 30x larger than the original HTML specification! The bar for someone to read the spec, let alone
implement it, is massively higher. Is it any wonder that people don’t bother reading specs these days?

This is a real tragedy, because it practically guarantees that only large organizations will have the resources to maintain software based on
these new specifications. It turns the democratic spirit of the web upside-down. As Dan said in his keynote, “Hixie sent out an e-mail and 500
people joined the HTML5 working group; a level of participation we’ve never had before”, and then went on to explain that many people drop off
over time after their initial enthusiasm. This anecdote illustrates the sea change perfectly. In the past, we had only 11 people on a working
group, but 500 people could quickly implement the spec. Now we have 500 people on the working group, but very few can implement the spec. Of
course, orders of magnitude more people can claim to have participated in a W3C working group, but is this really what we should be optimizing
for?

And this, I think, gets to one of the root causes of this sea change. The people writing and implementing the early web specifications were
generally very practical people unconcerned with politics or egos. I can remember times that I e-mailed Jon Postel with questions about one or another of his specs, and he would reply quickly and to
the point — he was always gracious and pragmatic on mailing list exchanges, too. Yes, we all felt that “this Internet thing is going to be
BIG!”, but we didn’t have proof back then. People were working to solve technical challenges and bootstrap adoption, not to make personal names
for themselves or push an ideological viewpoint. This isn’t to say that spec authors today are all egomaniacs, but people today implicitly
understand that they might be the next Jon Postel, whereas Jon Postel didn’t ever entertain such thoughts, since he was in an unproven field.

Dave Winer made a similar point in a blog post yesterday. For
the most part, the OAuth specs are some of the cleanest, simplest specs to emerge in the past couple of years. But Dave makes an interesting
observation: “OAuth isn’t so hard once you got it working, but there are too many docs, too heavy on theory, and there is no validator … I
offered to help get the process systematized by remembering what it was like to be a newbie, which I still am, totally.”
On the one hand,
OAuth is head and shoulders above other recent specs, because a determined newbie actually CAN implement the thing. But the “too heavy on theory”
part is right on, and other spec authors would do well to pay attention to this. Read any of the specs in the bulleted list above, and see if you
can find any signs of “theory”, “manifestos”, or other such nonsense. When someone reads your spec, they care about only 2 things: 1) What can it
do for me? and 2) How do I implement it?

Now, I wouldn’t go so far as to say that the standards process has never been politicized. Realistically, the standards bodies are just one
additional mechanism via which companies compete with one another. And I’ve seen plenty of politics play out in the standards bodies. But you
cannot argue with the Spartan clarity of the early specs, versus what comes out these days. Specs used to be clean and crisp — documents that
the enthusiastic newbie could use to build real software to solve real problems.

Finally, I completely understand that there are sometimes legitimate reasons for bloat. But we should look very critically at bloat and be aware
of what it’s costing us versus what we gain, before agreeing that it’s a good thing. The HTML 1.0 spec, at 10,000 words, is the HTML spec that
changed the world. No additional HTML spec is going to ever have that level of global impact again. At best, we can hope for some incremental
improvements. We were faced with a similar choice when XML 1.1 was proposed — we were being asked to add complexity for implementers in favor of
an incrementally improved standard, and we finally all realized “this is stupid, XML 1.0 is plenty good enough”. We killed XML 1.1, and it turns
out that nobody really complained. A proud moment for the web, if you ask me.

What do you think? Agree or disagree? We would love to see your opinions here in the comments section. And don’t forget to follow our twitter feed to be notified when new lab projects, articles, or opinions are posted.

Follow the Conversation

18 comments so far. You should leave one, too.

oliver said on Feb 18, 2009

It''s worth considering that perhaps the brevity of some of the older specs was due to their ambiguity and failure to fully document behaviour -- certainly HTML5 is a much bigger spec, but a fairly substantial amount of it is spent going into great detail over the exact behaviour in all circumstances, covering faliure cases, etc, etc

Joshua Allen said on Feb 18, 2009

Thanks Oliver; that is indeed the only real justification for bloat. But I''m skeptical that it''s justification enough. IETF''s philosophy was always "loose consensus and running code", and this worked very well historically. And times where it didn''t work almost never were due to spec ambiguity; they were due to other factors. Does anyone really believe that IE6''s standards support would have been better "if only the specs were more detailed"? The vast majority of interop problems were not due to insufficient detail in the specs. And we''ve historically seen the best interop results by providing detailed test suites and hosting interopathons; not by writing specs that are accessible only to large organizations.

I''ll try to keep an open mind, but I just don''t see it yet.

Andrew Dupont said on Feb 18, 2009

Josh, these are all well-reasoned arguments, and large parts of my brain would agree with you on those occasional days when I can''t seem to get the simplest markup to render the way I want.

Still, the potential of HTML5 overwhelms my skepticism. You see a markup language that balloons in complexity with each new version; I see a language that is bursting with good ideas after a decade of stagnation. (HTML 4.01 was published nine years ago; XHTML wasn''t a "feature" release, unless you consider more forward slashes a "feature.")

You''re also comparing apples and oranges, though it''s the WHAT-WG''s fault for putting them all in the same bowl. HTML5 is so gargantuan for two main reasons: (a) it aims to define new JavaScript/browser APIs (the "DOM5" stuff); (b) it prescribes, for the first time, how a user-agent should behave when faced with markup that is not valid or well-formed.

I am not sold on the monolithic spec. Experience tells us that just because it''s one spec doesn''t mean it''ll be implemented in all-or-nothing fashion. On the other hand, I would rather chew off my forearm at the elbow than repeat the CSS3 debacle — i.e., divide the task into "modules" that don''t fit together as part of one system, then watch them all wilt from the diffusion of responsibility and urgency.

Anyway, I''ll reiterate: I am just as hesitant as you about the size of the HTML5 spec. But let me raise a few more points.

First: wouldn''t you agree that the hardest part of building a web browser from scratch in 2009 is reverse-engineering the arbitrary, undocumented, and downright perplexing rendering decisions made by browsers past? And isn''t that what specs are supposed to prevent? Better a bloated spec than no spec at all.

Second: remember the adage — "A complex system that works is invariably found to have evolved from a simple system that worked." If we control for the expanded scope of the HTML5 spec, I think the progression from HTML 1.0 to HTML 5.0 reflects a pattern of slow growth toward a complex system that works.

Third: the implied comparison between XML 1.0/1.1 and HTML 4/5 is... a stretch. It quietly assumes that HTML4 is "good enough," and that HTML5 would deliver only "incremental" improvement. I''m sure a few people out there believe this, just as there are people who believe in centaurs; but I must conclude that they do not see the same Web that I do.

Boris said on Feb 18, 2009

I can''t speak to IE6, which was mostly not bothering to implement specifications at all, but I can tell you right now that Gecko''s, Presto''s''s and Webkit''s standards support would be much better today if they hadn''t had to spend years reverse-engineering IE''s handling of the parser and the DOM. IE was in a similar situation with respect to Netscape 4 at one point too.

On a separate note the last several words of each comment are cut off on the right. This happens in Firefox, Safari, and Opera, so I suspect the problem is in your stylesheet...

Boris said on Feb 18, 2009

Oh, and the right-hand side of the "leave a comment" form is cut off too, so it''s impossible to see what one has typed... hence the typos above.

karl Dubost said on Feb 19, 2009

If you really want to compare magnitude of orders. You have to put the right things into place. XHTML 1.0 is not implementable without reading HTML 4.01. Because the full semantics of the specification is in fact in HTML 4.01.

So to compare the number of words of HTML 5, you have to take this equation HTML 4.01 + XHTML 1.0 + Dom Level 2 + Some Javascript APIs here and there. Then you will get a fair comparison. It is still too gigantic, and I''m pretty sure even doing that HTML 5 is bigger.

Compare the growth of HTML 2.0 to HTML 3.2, and the growth of HTML 3.2 to HTML 4.01. When a technology evolves it becomes more complex.

100 years ago a mechanics amateur could create a car in his own garage. He had to be passionate and have money but he could do it. Creating a today''s car is very hard, costs money, etc. It is not accessible anymore to a passionate soul.

Some pieces of HTML 5 are easily implementable by amateurs. For example, the parsing algorithm has been implemented in html5lib by James Graham who was not a specialist of Web browser development. A lot more complex to implement (and not in HTML 5) is CSS 2.1, the rendering part.

Ian Hickson said on Feb 19, 2009

Another key point when comparing HTML4 to HTML5 is that the size of the specs doesn''t, for the reasons given in some of the earlier comments, correlate to the amount of effort required to implement the specs.

To implement HTML4+DOM2 HTML+XHTML1 you have to read the specs, and then fill in the blanks yourself, typically by reverse-engineering other browsers. This is a massive amount of additional work -- I''ve heard estimates from browser vendors that they spend literally in the order of 50% of their time reverse engineer each other instead of implementing what the specs say.

To implement HTML5, assuming I did my job right, you only have to read the spec. All the reverse-engineering has been done for you.

This actually means that implementing HTML5 is *easier* than implementing the previous generation, relative to the number of features in the spec.

Now, having said that, HTML5 have more features than HTML4. In fact, the Web platform as a whole (the set of specs a Web browser has to implement) has more features than it did ten years ago. This is a natural state of affairs -- as technology matures, it usually grows features -- and it is what enables your site to have fancy colours, it''s what enables Google Calendar to exist, and so on.

Back in the early 90s, you could write a browser with just HTTP, HTML, and URIs. Now, you have to have HTTP, HTML, CSS, URIs, IRIs, IDNA, DOM Core, DOM HTML, DOM Style, DOM Events, JS, SVG, PNG, GIF, JPEG, XHR, XHR2, CORS, Selectors API, Geolocation, etc etc etc.

We could propose to the world that we go back to just HTTP, HTML, and URIs, but I don''t think anyone but us geeks would agree.

Ian Hickson said on Feb 19, 2009

Oh, while I''m at it, one other minor thing: there is work afoot to shrink HTML5 by taking some of its more self-contained component parts and moving them to separate specifications. Is that an improvement?

It doesn''t reduce the implementation burden. It just means that instead of implementing "HTML5", browsers will implement HTML5 + Local Storage + Database + Workers + Event Source + Origin + Web Socket API + Web Socket Protocol + URL Fixes + Content-Type sniffing. It''s still the same set of requirements, the same resulting code.

Joshua Allen said on Feb 19, 2009

@Andrew, @Boris; thanks for the comments. You raise some excellent points. Regarding the DOM and JavaScript conformance problems, I would still argue that the bulk are not due to spec deficiencies. For example, the Microsoft JScript team wrote a document describing all of the conformance issues they know about [0] but haven''t yet fixed. Obviously, the lack of conformance isn''t due to ignorance or lack of understanding the spec.

Like you, I look forward to seeing new features in the platform, but I''m trying to be realistic about the risks. There is a bit of wishful thinking in the idea that a monolithic spec will drop and all the vendors will implement (and deploy) it lockstep. It would be great if it happened, but when it doesn''t, it means 5 more years of pain for developers -- 5 years of pain in which proprietary plugins like Gears, Flash, or Silverlight start to look more attractive. And really that''s my main concern about the process. It feels like there are certainly more people clamoring around asking for features, but the actual implementation is becoming more exclusive. Maybe it was inevitable, as you say, and maybe I''m just being anachronistic. But it''s a lot different from the past, when any university could write their own software to implement the standards.

[0] http://wiki.ecmascript.org/lib/exe/fetch.php?id=resources%3Aresources&cache=cache&media=resources:jscriptdeviationsfromes3.pdf

Joshua Allen said on Feb 19, 2009

@Karl -- The example of James Graham demonstrates that it is still *possible* to write simple and modular specs that are accessible to hobbyists. For the most part, the XML specs were like that. XML, Infoset, and XSLT were all pretty clean. When it''s all just data structures and code, people can usually figure it out. But really good point about CSS.

Joshua Allen said on Feb 19, 2009

@Ian, I really appreciate the response. Regarding modularization, I''m glad to hear you''re thinking about it. Obviously, modularizing just because some people like me complain about size of the spec, is no more reasonable than making the spec monolithic out of hopes that it will all be implemented and deployed lockstep. As Andrew points out, modularization can cause more problems than it solves; and conversely there are cases in the specs where modularization was very successful. IMO, modularization should follow closely the way that the implementation would be modularized by a typical engineering team. Or looking at it another way, things like geolocation and web workers were initially developed as plugins, which enforces some level of modularity, which ends up being good design. Without that constraint, if the designers had full access to the browser internals, it''s easy to imagine that the solution chosen would have been a lot more complex.

Regarding spec ambiguities, you know as well as anyone that many substantial advances in interop come from outside the spec. A spec doesn''t absolve vendors of the necessity to work together, nor does it automatically motivate them to do so. Acid 2 clearly helped push things along, and didn''t require big modifications to the specs. Maybe just personal preference, but I believe all of the errata, supplements, and test suites should be decoupled from the main spec. Again, as a personal preference, I would prefer to see the test suites developed cooperatively by all the vendors. But the fact that tests like Acid 2 or Acid 3 can be developed outside of W3C and have a visible impact in driving interop, suggests that the interoperability problems of the past weren''t primarily due to spec ambiguities.

Good point about all of the new features being added. There''s always a danger of trying to do too much at once, but it seems there is some anxiety with some people who are afraid that HTML will lose relevance if it doesn''t evolve quickly. IMO, this is a silly worry -- the web as a platform is ubiquitous largely *because* people could count on it being the same.

Ian Hickson said on Feb 19, 2009

The splitting out of things from HTML5 (not really modularisation) is driven primarily by their nature -- we''re splitting out self-contained sections (timers, XHR, WebSocket, Selectors API) and things that affect more than just HTML5 (Content-Type sniffing, Origin, URL error handling). The main reason is just to keep the HTML5 spec limited to things that are interconnected. Unfortunately, a lot of the Web platform is interconnected in pretty fundamental ways, e.g. HTML parsing, document.write(), and the Window object are all so closely related that it''s hard enough to spec them in different sections, let alone different specs.

BTW we''re not just thinking about splitting things out -- we''re doing it. XHR was taken out a few years ago, Selectors API -- at the time just a note in HTML5 that we should do it -- a few months later. Last month we started work on Content-Type sniffing, the WebSocket protocol and API specs, and the Origin spec. One of the big problems is that editing two specs is significantly more work for me than editing one, even if the content of the one is bigger than the content of the two (basically, the per-spec overhead is huge -- mostly because I''ve had years to reduce the per-spec overhead of working on HTML5). So splitting things out usually requires a volunteer to edit the split out section, and that''s often hard to find.

Acid tests (in theory at least) don''t test subtleties that the specs don''t require. I don''t think it makes sense to leave things undefined and hope for the best. That kind of model might work when innovating, when there''s not much legacy content, but in a mature system, it''s a cost. Of course, having a spec doesn''t mean the browsers are bug-free. Having a spec, however, does mean that we can have tests (like the Acid tests) to ensure that browsers all follow the same rules. It also makes it cheaper for browser vendors to converge on doing the same thing (they don''t all have to reverse engineer each other, each getting it slightly wrong).

I''m not really sure what you mean by "errata, supplements, and test suites" -- could you point to a bit of HTML5 that you think should be decoupled? (Other than the sections we''re already planning on splitting out into self-contained specs, that is.)

As far as adding too much at once -- if you think HTML5 adds too much, you should see what we _didn''t_ add! (Search for "v2" in the spec source, e.g.) I don''t think anyone can seriously say that HTML5 is going too quickly, either. :-)

Joshua Allen said on Feb 19, 2009

Yeah, you''re right about people saying "not moving fast enough", when it comes to the spec at least. Jeff Croft had a classic post about that last year. OTOH a few features of HTML5 have already been implemented in all the major browsers, including IE (and many more in the other browsers). I thought we all swore off implementing unfinished specs after the WD-XSL fiasco, but it appears that having a signed-off spec is no longer as important.

Really great to know the spec is being broken apart on those lines. That''s the right breakdown, IMO, since those pieces can be, and often already were, implemented independently. I know it adds to the editing overhead, but I really think it will make a difference in approachability. A 260,000 word spec is just frightening and would repel a lot of people from the start.

Regarding "errata and supplements", I am thinking of examples like XSD, where the core spec was rather focused, and then the errata document (31,000 words) came out after we made progress on implementing and discovered inconsistencies among implementations. Or XQuery, where there was a whole separate "use cases" supplement. They were very valuable, but supplemental.

I may just be overreacting, but it seems like a very common justification for HTML5 is that "it documents the differences in implementation among browsers". IMO, that stuff should be published as errata, and the new features published separately. The fact is, even if HTML 5 is only new features, and even if half the browsers implement it 2 years before it''s signed off, there will still be browser inconsistencies and mistakes that need to be corrected. In other words, I don''t think it''s realistic to promise a spec with no ambiguities, and it feels like that''s what is being promised when people say "HTML5 is going to correct the ambiguities in the previous HTML specs". And the task of correcting ambiguities of the past seems orthogonal to adding new features, and adds considerably to the size and complexity of the spec.

Ian Hickson said on Feb 19, 2009

The problem with speccing the ambiguities separately from the new features is that the new features depend on the ambiguities.

For example, take the new pushState() method in HTML5 (it lets you add stuff to the back button without having to navigate, so that you can do AJAX apps without breaking the back button).

It depends on the inner workings of the browsing context session history. But no spec before HTML5 ever even _mentioned_ the _existence_ of the session history, let alone defined it in serious detail.

So HTML5 has both the old DOM Level 0 session history API with all its horrible craziness, and the new pushState() method added to that API. Separating the two would mean that implementors would have to read two specs side by side instead of having one coherent document.

Incidentally, note that HTML5 doesn''t actually document the _differences_ in implementation among browsers, it documents one required behavior that all browsers are supposed to converge on. And sure, it''ll take years for all browsers to implement it all correctly -- has any browser implemented any spec completely right ever? -- but that''s fine, we''re in this for the long haul, and the spec will continue to be fixed as implementors find problems with it. IMHO specifications that are widely used are never "done" (or "signed off"), they''re either living documents or abandoned. HTML5 will continue to be updated until such time as HTML6 is ready to take over being the HTML spec.

rtpHarry said on Feb 20, 2009

sorry, I know you wont like this but I couldnt be bothered to read the whole post lol.

I think that it is good to keep it succinct but on the flip side a lot of the browser incompatibilities around today are because of developers reading brief specs and implementing their interpretation of it.

Joshua Allen said on Feb 20, 2009

@rtpHarry: You got me there; nice answer :-)

Air Jordan 13 Retro said on Apr 17, 2012

One more new write-up with powerful points, I've been a lurker below for any brief time but wish to become a great deal much more engaged inside long term.

Jordan Retro 5 said on May 22, 2012

This blog site is great. How did you come up witht he idea?