September 08, 2008

<video> and Flash

The availability of a Firefox 3.1 alpha (and Safari 3.1) with video tag support has prompted a few people to proclaim that Flash will soon be replaced as the primary medium for playing video on the web. I think predictions of Flash’s demise are both premature and inaccurate, and also think there’s a possibility that the introduction of the new tag could cause more problems than it was intended to solve. Why? – codecs…

Firefox (and Opera) will support one embedded codec in the first instance, Theora. Theora is completely open-source but based on an older-generation codec, On2 VP3. Flash currently uses VP6 and H.264, Quicktime Player supports H.264 and Silverlight adds support for VC-1, one of the newest kids on the video block. All are newer, higher-quality formats than Theora (typical comparison here). That’s not to say Theora is a bad codec, far from it, but in the world of video codecs it is at least a generation old, maybe more, and as such doesn’t represent the current best of breed in terms of video quality/performance.

Secondly, in order to take over from Flash, Theora needs to be on all or enough browsers to work as a standard. The HTML5 specification doesn’t advocate or recommend any codecs, and as the specification notes, this is problematic:

It would be helpful for interoperability if all browsers could support the same codecs. However, there are no known codecs that satisfy all the current players: we need a codec that is known to not require per-unit or per-distributor licensing, that is compatible with the open source development model, that is of sufficient quality as to be usable, and that is not an additional submarine patent risk for large companies. This is an ongoing issue and this section will be updated once more information is available.

So, if one browser advocates Theora, will every other browser follow? After all it’s a fairly trivial (and zero-licencing cost) thing to include support for it. Firefox and Opera combined currently have around 25-30% share of the browser market; while 30% (and growing) is an excellent starting point to build from, IE’s and Flash’s dominance in terms of installed base will make it difficult forTheora to overtake the current popular formats unless they support it too.

We’re not starting with a blank sheet of paper here either; many video services and sites that use video have spent a considerable amount of time and investment in encoding to a specific platform, one that is currently supported by enough browsers/platforms to make it worthwhile. At best a move to Theora would have to be transitional, and likely to take several years, as people migrate content over to the new format. IE is the most common browser by a considerable margin, and Microsoft has its own favoured formats; WMV via Windows Media Player and/or VC-1 via Silverlight, plus the formats supported by Flash Player. It’s hard to see these not being with us for a while yet, unless IE starts supporting Theora at the expense of Windows Media formats (unlikely); if Theora isn’t included in IE, it simply isn’t going to get traction as quickly as it needs to to become the standard. Similarly unless Webkit (as the core engine of Chrome and Safari) or YouTube also add native support for it (AFAIU latest Safari builds that support the video tag currently support Quicktime’s supported formats, which doesn’t include Theora as standard), it’s hard to see it taking over. While users will be able to add new codecs manually, assuming they will do so in order to view a new video format could be risky. Then there’s streaming support via RTSP/RTMP, etc.

My point is we are going to have to live with multiple formats whether we like it or not, and that Theora is a technically a backward step; it will only take one browser vendor to ignore Theora and it will become void as a standard, yet HTML5 isn’t forcing or even recommending a standard to follow. At this point video on the web will be in danger of becoming a mess as people find they don’t have the required codec and have to install support for it. I wouldn’t go as far as to suggest the video tag is going to create chaos where relative calm currently exists, but it might. I can’t see a good technical reason why the current model of using Flash as a wrapper for video and codec support is broken, or why it needs to be replaced with what may become a mess of format support where we move from two or three dominant formats to five, six or maybe more competing ones, plus multiple downloads for users. In my experience proprietary formats tend to be better, that’s why they cost, and neither do I buy into the ‘everything has to be open-source’ argument, especially considering that the dominant audio format is MP3, which isn’t an open-source format at all (even AAC codecs require patent licence).

Most importantly though, while all this takes place a solution already exists; Flash Player. It supports H.264, VP6 and Sorenson, most people already have it and it works on all the browsers mentioned above, across Windows, Linux and Mac. “But Flash is a terrible resource hog!” I hear you say, but when you consider that Flash is decoding and rendering video, in software, it’s worth noting that this takes CPU cycles and is a processor/memory intensive task. Even if you could run fully-hardware accelerated video (which Flash is moving towards), video is still relatively intensive work for the average desktop/laptop. In the first instance, Theora decoding in Firefox is going to run in software, just like Flash, and presumably consume CPU cycles in a similar way.

With all this in mind it would seem to me that the only reason for wanting to take Flash off its video pedestal is that it is owned by a commercial entity, ignoring the fact that Flash Player has been a key enabler and driver of the dramatic increase in use of video on the web, without any of the ‘payback’ people seem to fear. Without advocacy from W3C, the reality is that browsers are going to be free to implement their own choice of favoured codecs, but those choices are likely to be driven by different criteria, not necessarily whether they are open-source. It could be about to get messy.

Alternatively, we could all start using Mike Chambers’ workaround for getting Flash to display video wherever the video tag is used, and everyone could just carry on. ;-)


- 3 comments by 1 or more people Not publicly viewable

  1. John Dowdell

    Hi Steven, thanks for being a voice of sanity in an otherwise wacky world…. ;-)

    (There’s the codec supply issue, but also the video production issue… getting some VP3 encoders, support for media servers, camera-to-editing-to-browswer production workflows… the VIDEO tag discussion has been lengthy, but I haven’t seen the basics really considered yet.)

    jd/adobe

    08 Sep 2008, 17:07

  2. Chris May

    It’s fairly clear that WHATWG have dug themselves something of a hole here. They’re committed to a spec which is freely implementable, but the only freely-implementable video codec is, to put it mildly, not very good.

    I can see two possible positive outcomes; one good-but-improbable, one OK-ish.

    The OK-ish outcome is that the spec remains as-is, but that browser manufacturers agree a de facto set of codecs that they’ll actually ship. Kind of like how ubuntu out-of-the box is entirely free, but the first thing everyone does is to add in the non-free repositories so they can get the video/wifi drivers they need.

    The good-but-improbable one is that, whilst this is going on, someone fixes theora and it becomes a viable competitor to H264. If this ever happens, then it wouldn’t take much for sites to start transcoding new video to whatever this new format turns out to be, and since browsers already have the legacy support for older codecs, there’s no need to reformat all of youtube’s existing 10 gazillion hours of video. Then the spec can be updated to mandate the new format*, and off we go.

    * this may be more difficult than it sounds. IIRC the arguments against mandating theora in HTML5 were largely political, driven by the lack of DRM support in theora, rather than the quality of the codec per se.

    10 Sep 2008, 11:10

  3. Hampton

    Xiph should change the name of the codec to VP7 so people like you will feel more comfortable with the Theora software maturity. Listen, the number in the product name is not a simple way to determine how good it is. The Xiph guys have been working diligently to improve the encoding and decoding of Theora, ever since they acquired it from On2. Your argument is based on the idea that VP2 is not as good as VP6, and this disregards all the work Xiph and others have done and continue to do with the VP2-based Theora codec.

    27 Oct 2008, 16:22


Add a comment

You are not allowed to comment on this entry as it has restricted commenting permissions.

Trackbacks

  1. Silverlight to support H.264, AAC

    Microsoft has announced that Silverlight will be supporting AAC audio and H.264 video. With regard to my previous entry about the video tag, this will make it even harder for Theora and can only consolidate H.264 as the current codec of choice. The announcem&hellip;

    [Ux] - 09 Sep 2008, 21:13

Search:

MXNA link

Tweets



    Tags

    Other blogs I like...

    Black Pepper Software Go to 'Black Pepper Software'

    Eismann-sf Go to 'eismann-sf'

    Ted On Flex Go to 'Ted On Flash'

    Galleries

    Meetups:

    Not signed in
    Sign in

    Powered by BlogBuilder
    © MMXIV