Sunday, May 15, 2011

Myths and Misunderstandings of Chromebooks

When the iPod was launched, pundits immediately compared it to the existing mp3 player market. It was too big, too expensive, not powerful enough, and lacked the features other players had.

Then, when the iPhone was announced, a similarly sounding chorus declared its deficiencies, some of which I detailed here in 2007 It had no MMS, no swappable battery, no 3g, etc. Oh sure, there were other smartphones on the market that had a much longer list of features and were even cheaper, but it didn't matter. Paul Buchheit covers this "more features = better" philosophy here.

Now, Paul is somewhat pessimistic about Chrome OS, which I disagree with, but what's interesting, is that some of his arguments against Chromebooks I actually used myself when the iPad came around -- If already have an iPhone, and a Macbook Air, why do I need this tablet thingy? It sucks for content creation, and if I already lug around my notebook, why would I lug this thing around in addition? I was wrong, because there are times of the day where you need to create content, and times when you simply need to consume it, and tablets are perfect for the latter. More on that divide later.

The parallels today are appearing again. At $349, Chromebooks are too expensive compared to Netbooks at the same price level! Netbooks had been tried before, and failed! With Windows 7 basic or whatever, on a Netbook, you have far more flexibility and features! If someone already has a tablet, and a notebook, why would they want a Chromebook! And so on. There may be some merit to those arguments, but at iPad launch, the idea that a third kind of device wasn't needed also had merit. After all, tablets had been tried before and failed. We now know there was a market need.

Content creation vs Content Consumption

Where I disagree with Paul and others, is that even a souped up tablet is not a good stand-in for a work device. Perhaps with a large external monitor, and external keyboard and mouse, but as tablets constructed today, I would not want to write code on them, or even long blog articles like this. Perhaps it's my generation and young people won't have such hangups, but my generation is still a large market.

Thus, I assert, the need for a traditional WIMP device: physical keyboard, mouse, connection for monitor. I don't think enterprise IT would really find this controversial. So the next question is, why would a Web-only device be better than one with "native apps".

Myths about Chrome

In Paul's message on Chrome OS, he rightly sees that Android is becoming more "webby", but does not comment on the fact that the Web is becoming more "natively". What is the difference between an offline, cached, Chrome Web Application, that uses file system APIs, and WebGL, and a Android app, other than the fact that one is written in Java and uses Java APIs, the other is written in JavaScript and uses Browser APIs? Is Angry Birds Chrome really that much different than Angry Birds Android?

The principle difference between Web, Flash, and Android, is merely the difference in virtual machine, programming language, and API flavor. With Chrome, you have V8 as the VM, JavaScript as the source language, and HTML5 as the API to the VM. With Flash, you've got Flash VM + Actionscript3 + Flex APIs. With Android, you have Dalvik, Java, and Android APIs. All three have APIs for UI, 2d/3d, sound, network, etc. All three platforms have "web-like" install/update models, and all three have webby sandbox security models.

Well, you say, Android has NDK. To that I answer, Chrome has Native Client.

Well, you say, Android is Linux. To that I answer, so is Chrome OS.

Well, you say, Android doesn't force every app be a "document". To that, I point to Bespin and Angry Birds.

The conceptual gulf just isn't that big folks.

Other Myths overheard

  • If I'm not online, I can't do anything!

    Not true, Web apps can be offline. Chromebooks have local storage which can act as a sync repository/cache. It will depend on how apps are written. It's really no different than iOS/Android.

  • Everything is in the cloud, Google can read everything.

    Conditionally true, depending on the app, data can be encrypted on the server with only the client app able to view it

  • I can't access some critical windows apps

    True in some cases, false in others, see: Citrix, VNC, X

So if they're the same, why not just merge the platforms?

That may very well happen someday, who knows. But I do want to take issue with a point Paul made about the webby-ness of iOS/Android apps, because I think it is only true in theory, much like (not not as severe as) Web apps being offline capable.

Assertion: Web apps encourage linkability and searchability more than native apps

While it is true that you can deep link into iOS or Android apps, how many times have you followed a link from Flipboard directly into a story in the HuffingtonPost app? Web apps by their nature, and by years of convention, make their location and state known. By contrast, if native apps have URL schemes to trigger deep links, they are not omnipresent in an Address bar, but typically custom and hidden and not easy to find. Web apps are composeable by links because of their relative transparency, and it is more difficult to achieve in a native app ecosystem. Such conventions may evolve later, but it requires vigilence by developers for the gradient is not towards such transparency.

This concerns me, because if every website becomes a custom app, the web-of-links will be broken, and even Page Rank will become less relevant. Granted, this is not an argument against merging, but it is in argument to preserving the current model of publishing information via a standardized document format as opposed to proprietary per-website native applications.

So why then, do we need Chrome OS?

Because 90% of what people seem to do this days, outside of games, can be done on the Web. If a user is spending almost all of their time running Web apps and NOT running native apps, why not construct a device that is stripped down and simplified to streamline exactly what he needs?

I hear you say, "but why not include Android as well?" But this gets back to "more features == better". Remember, the postulated user is one who spends most of his time browsing Sure, it sounds good to have all of these extra features and access to the wealth of Android Apps, but non-iPod MP3 players also sounded like a better value proposition compared to the iPod originally.

In particular, for old fuddie-duddie Enterprise users, a locked down device, centrally managed, with cloud backup, and no expensive IT department needed, sounds very viable.

The idea of producing a netbook, which boots up with Android OS and contains a *full, not chopped down* version of the latest Chrome, also sounds like a viable SKU for content-creation activities. It may very well be the winning formula as Paul indicates!

But Google's trying Chrome OS first and experiments are good. I don't think power users who want to run Crysis on high-powered windows notebooks, can dismiss it, nor can it be dismissed just because it doesn't include every feature and the kitchen sink.

Sometimes I just need a fast, convenient browser.

p.s. I'm also not sure, a future in which every website is a native app (all newspapers, all blogs, etc) that I will be able to find information as easily, or to extract, mashup, and link information as easily. The Web has its problems, but let's not throw it out because of smooth animations and sexy devices.

p.p.s. The separate evolution of Chrome as an OS in which you can do everything and to which apps can be targeted will be good for a merged Chrome OS/Android OS too, because it ensures the Chrome team will have to make Chrome as great as possible before that merge happens. If the merge happened too early (say, Chrome 1.0), then Dalvik would have been a crutch, and they probably wouldn't have improved it as much as they have. Right now, the idea of Web as an app platform is a forcing function for improvement.

Friday, May 13, 2011

The problems with HTML5 <Audio>

I've been having a Twitter back and forth discussion with Giorgio Sardo, Microsoft's HTML5/IE evangelist on Audio, but I find 140 characters too limiting to explain the issues, and Giorgio seems more interested in snark and attacking Chrome, than attacking the root of the problem.


This all started because after the release of Angry Birds at Google I/O, people noticed that it was requesting Flash. Angry Birds is written in GWT and uses a GWT library written by Fred Sauer called GWT-voices. This library not only supports HTML5 audio, but has fallbacks to flash and even <bgsound> on IE6!

There was speculation the Flash requirement was done for nefarious purposes (to block iOS) or because Chrome includes Flash, but the reality is, it was done *both* because Chrome has some bugs, and HTML5 <audio> just isn't good for games or professional music applications.

I first noticed the shortcomings of the audio tag last year when we ported Quake2 to HTML5 (GwtQuake) shown at last years I/O, where I also demoed a Commodore 64 SID music emulator. There are two issues with using HTML5 Audio, which was originally designed to support applications like streaming music players.

It is missing functionality

The HTML5 audio element permits operations like seeking, looping, and volume control, which are great for jukebox applications, but you cannot synthesize sound on the fly, retrieve sound samples, process sound samples, apply environmental effects, or even do basic stereo panning. Quake2 required 3D sound based OpenAL's inverse distance damping method as well as stereo panning, I did my best, and implemented distance damping with the volume control, but had no ability to position sounds left or right.

For sound synthesis, there is no official way to play back dynamically created buffers. The workaround is to use Javascript to encode sample buffers into PCM or OGG in realtime, convert them to data URLs and use those as the source for an audio element, which is very computationally expensive and chews up browser memory. For developers wishing to create even basic music visualizers, it creates huge difficulties.

Problem #2: Latency

Audio applications require low latency. Studies have shown human beings can perceive audio latency down to the millisecond, but in general, lower than 7ms is considered good enough. This means in some circumstances, you need to schedule sounds within 7ms of one another, for example, if you need to simultaneously start two sounds, one on the left ear, and one on the right ear, or if you need to concatenate several sounds together in series.

Giorgio has a neat demo here of playing piano notes in sequence, and hats off to Microsoft for providing a great <audio> implementation. It's a cool demo, but I still hear latency variation in playback between notes and occasional glitches. No one's going to build something even 1/10th as good as Garage Band on iPad using this technique. That's because the one way you can schedule audio in HTML5 is via the browser's event-loop using setInterval or setTimeout, and that's problematic for several reasons.

First, it's unreliable. Over the years, setInterval/Timeout has been clamped to different minimal resolutions, depending on the browser and operating system. On some systems, it was tied to vertical refresh and would clamp to 16ms, then vendors started clamping to 10ms, and now they clamp as low as 4ms. But 4ms isn't a guarantee, it's a request. Many things can stand in the way of that request, for example, by just mousing over the page, user interface events can trigger Javascript handlers, CSS rules which force a relayout, and excessive Javascript work can trigger garbage collection.

Secondly, aggressive setInterval periods can delay response to user input, making the browser feel sluggish. If the user tabs to another window, the browser must decide whether or not to clamp timeouts to a much higher value (say 1 second), to avoid needlessly burning CPU which could harm background playback. Unlike requestAnimationFrame which solves this problem for graphics, there's no "requestSoundEvent".

Music Apps and Games sometime require playback of short-buffers

Some of the sounds in Quake2, for example, the hyper-blaster are sample buffers as small as 300 bytes. At 44khz, this is a hard deadline of 8ms to schedule the playback of the next sound in the sequence. With all of the other stuff going on within a frame, processing physics, AI, rendering, it is highly unlikely to be consistent, and do we really want JS performing this scheduling task.

Especially on mobile

Remember, mobile devices are HTML5 devices as well, and are continually getting better at HTML5, but they are much more resource constrained, and Javascript is even slower. Here, native scheduling is even more beneficial, and intensive Javascript scheduling of playback would be difficult, and waste battery.

That's why the Web Audio APIis important, because it permits complex audio schedule tasks, application of environmental effects, convolutions, etc to be natively accelerated without involving the Javascript engine in many cases. This takes pressure off the CPU, off of memory and the garbage collector, and makes timing overall more consistent. Here's a neat demo recently shown at Google I/O

Microsoft deserves credit

They made massive improvements in support of HTML5 from IE8 and IE9 especially in <canvas> and <audio>, and they deserve the right to feel proud and evangelize them. We celebrate that. It's why Angry Birds works, to some people's shock on other browsers, and it's not by accident. We built in fallbacks in our core library for 2d canvas, and tested on non-WebGL capable browsers like IE9 which have excellent GPU accelerated 2d support.

Angry Birds was not an attempt to make non-Chrome browsers look bad, but to make HTML5 look good, because when developers start realizing that professionally developed and polished games and applications can be done in HTML5, we all win.

But now is not the time to rest on our laurels. HTML5 is not done. There are many things incomplete and broken in the spec. I am sad to see Microsoft trying to talk down the experimentation that is going on in Firefox and Chrome, vis-a-vis WebGL and new Audio APIs, just because they are on a slower release cycle and do not have these bleeding edge features.

Giorgio seems to be suggesting in his tweets that the basic HTML5 <audio> tag is "good enough" and that the current IE9 implementation covers use cases sufficiently, and I disagree with that strongly.

We need 3d on the web. We need high quality, low latency, audio. We need to be able to do the things that OpenAL and DirectX can do with sound on the Web. And we're not going to get there by sticking our head in the sand and declaring premature victory.

Labels: , , , , ,