All 103 entries tagged Rich Media
September 08, 2008
The availability of a Firefox 3.1 alpha (and Safari 3.1) with video tag support has prompted a few people to proclaim that Flash will soon be replaced as the primary medium for playing video on the web. I think predictions of Flash’s demise are both premature and inaccurate, and also think there’s a possibility that the introduction of the new tag could cause more problems than it was intended to solve. Why? – codecs…
Firefox (and Opera) will support one embedded codec in the first instance, Theora. Theora is completely open-source but based on an older-generation codec, On2 VP3. Flash currently uses VP6 and H.264, Quicktime Player supports H.264 and Silverlight adds support for VC-1, one of the newest kids on the video block. All are newer, higher-quality formats than Theora (typical comparison here). That’s not to say Theora is a bad codec, far from it, but in the world of video codecs it is at least a generation old, maybe more, and as such doesn’t represent the current best of breed in terms of video quality/performance.
Secondly, in order to take over from Flash, Theora needs to be on all or enough browsers to work as a standard. The HTML5 specification doesn’t advocate or recommend any codecs, and as the specification notes, this is problematic:
It would be helpful for interoperability if all browsers could support the same codecs. However, there are no known codecs that satisfy all the current players: we need a codec that is known to not require per-unit or per-distributor licensing, that is compatible with the open source development model, that is of sufficient quality as to be usable, and that is not an additional submarine patent risk for large companies. This is an ongoing issue and this section will be updated once more information is available.
So, if one browser advocates Theora, will every other browser follow? After all it’s a fairly trivial (and zero-licencing cost) thing to include support for it. Firefox and Opera combined currently have around 25-30% share of the browser market; while 30% (and growing) is an excellent starting point to build from, IE’s and Flash’s dominance in terms of installed base will make it difficult forTheora to overtake the current popular formats unless they support it too.
We’re not starting with a blank sheet of paper here either; many video services and sites that use video have spent a considerable amount of time and investment in encoding to a specific platform, one that is currently supported by enough browsers/platforms to make it worthwhile. At best a move to Theora would have to be transitional, and likely to take several years, as people migrate content over to the new format. IE is the most common browser by a considerable margin, and Microsoft has its own favoured formats; WMV via Windows Media Player and/or VC-1 via Silverlight, plus the formats supported by Flash Player. It’s hard to see these not being with us for a while yet, unless IE starts supporting Theora at the expense of Windows Media formats (unlikely); if Theora isn’t included in IE, it simply isn’t going to get traction as quickly as it needs to to become the standard. Similarly unless Webkit (as the core engine of Chrome and Safari) or YouTube also add native support for it (AFAIU latest Safari builds that support the video tag currently support Quicktime’s supported formats, which doesn’t include Theora as standard), it’s hard to see it taking over. While users will be able to add new codecs manually, assuming they will do so in order to view a new video format could be risky. Then there’s streaming support via RTSP/RTMP, etc.
My point is we are going to have to live with multiple formats whether we like it or not, and that Theora is a technically a backward step; it will only take one browser vendor to ignore Theora and it will become void as a standard, yet HTML5 isn’t forcing or even recommending a standard to follow. At this point video on the web will be in danger of becoming a mess as people find they don’t have the required codec and have to install support for it. I wouldn’t go as far as to suggest the video tag is going to create chaos where relative calm currently exists, but it might. I can’t see a good technical reason why the current model of using Flash as a wrapper for video and codec support is broken, or why it needs to be replaced with what may become a mess of format support where we move from two or three dominant formats to five, six or maybe more competing ones, plus multiple downloads for users. In my experience proprietary formats tend to be better, that’s why they cost, and neither do I buy into the ‘everything has to be open-source’ argument, especially considering that the dominant audio format is MP3, which isn’t an open-source format at all (even AAC codecs require patent licence).
Most importantly though, while all this takes place a solution already exists; Flash Player. It supports H.264, VP6 and Sorenson, most people already have it and it works on all the browsers mentioned above, across Windows, Linux and Mac. “But Flash is a terrible resource hog!” I hear you say, but when you consider that Flash is decoding and rendering video, in software, it’s worth noting that this takes CPU cycles and is a processor/memory intensive task. Even if you could run fully-hardware accelerated video (which Flash is moving towards), video is still relatively intensive work for the average desktop/laptop. In the first instance, Theora decoding in Firefox is going to run in software, just like Flash, and presumably consume CPU cycles in a similar way.
With all this in mind it would seem to me that the only reason for wanting to take Flash off its video pedestal is that it is owned by a commercial entity, ignoring the fact that Flash Player has been a key enabler and driver of the dramatic increase in use of video on the web, without any of the ‘payback’ people seem to fear. Without advocacy from W3C, the reality is that browsers are going to be free to implement their own choice of favoured codecs, but those choices are likely to be driven by different criteria, not necessarily whether they are open-source. It could be about to get messy.
Alternatively, we could all start using Mike Chambers’ workaround for getting Flash to display video wherever the video tag is used, and everyone could just carry on. ;-)
August 28, 2008
David Tucker recently posted an article on the Adobe Developer Centre citing 10 common mistakes with developing AIR applications and how to avoid them. I thought I’d quickly compare David’s points against my current AIR project, a remote file-system/transfer manager:
1. Making an application platform specific
David primarily refers here to the UI/UX differences between platforms, but I also found it essential because I encountered significant performance differences with some file system operations (icon getters, mostly), which were fixed with the help of the Flex Profiler. At the moment we can’t test it on the Linux Alpha, but hopefully we’ll be able to try that soon.
2. Not including update capability in an application
We included this as soon as the framework became available and it works very well – whenever the application starts it can check for a newer version of itself, and the user has control over whether to update or not.
3. Changing the application ID after an app has been released
Oops – I ran into this early on; changing the application ID means the update framework will break, amongst other things. Changing the name of your application halfway through the project also risks confusion.
4. Not planning for offline support
Our application relies on a live connection to work, but it does use AIR’s network monitoring APIs to check for a valid connection at startup and then continually monitor connection status, and warns the user if connectivity is lost.
5. Not thinking in AIR
I didn’t find this too difficult – some things were relevant and others weren’t. There have often been moments where I’ve discovered a capability and thought “I didn’t know it could do that”; we do use the application storage directory and the user’s temp directory for file transfers though, and the File , FileStream and FileReference classes were the key to making it work.
6. Using custom chrome to create confusing interfaces
We used the standard chrome, but most of the AIR applications I’ve seen that use custom chrome have done so pretty effectively. One of the most powerful aspects of AIR is that you have very fine control over the UI. As a counter to this though, in addition to the native window chrome there are valid use-cases for having access to native system controls, like toolbars, buttons etc. – platform UI differences can make this even more acute, so I’d like the ability to use standard UI elements where appropriate.
7. Not using the seamless install badge
We implemented this early on – having an install badge makes installation a snap for most people and like the update framework it works well – installation of the AIR runtime can be managed automatically and so far the whole thing has worked fine, except when I broke it myself by not updating the right fields in the updater XML file.
8. Not encrypting sensitive data
Not relevant for this application (yet) – we don’t store any information other than the last file-space used, using a standard SharedObject. AIR has an EncryptedLocalStore for this kind of thing though.
9. Not preserving native interaction
As yet we haven’t got a complete set of keyboard interactions in place (e.g. cut, copy and paste) but some are there. The core interaction type is dragging and dropping, and Flex/AIR gives you control over the process by splitting this action into discreet event-driven stages, so providing visual feedback about whether a drag/drop is allowed can be controlled via your own logic. Something I’ve not yet overcome is how to work with internal and external drag handling at the same time – I may be wrong here but so far it looks like the external drag management only knows about drag in/out operations at the application level, not at the component level, so I need to work on how to use the external drag management to allow items to be dragged into components with the same level of control as the internal drag manager.
A more difficult problem is when a standard component doesn’t quite mimic the operation of a native system control – take the Tree for instance, which will close when its DP is refreshed – in situations like these its nearly always possible to closely replicate the native behaviour by extending or over-riding the component default, but there can be some work involved when finding out what to do. At this point I’ll thank Peter Ent and the excellent Flex Examples Blog for their invaluable resources; they saved me a lot of time.
10. Assuming performance doesn’t matter outside of the browser
In this case, the performance issues highlighted when checking across platforms also highlighted the importance of using the Flex Profiler, the net result being a five-fold increase in speed on OSX and smaller but useful increases on Windows, plus reduced memory usage. The original performance on OSX was bad enough to almost make the application unusable, but after identifying and fixing/working around the problems, the application performs similarly in Windows and OSX.
Thanks to David for writing about what to avoid – fortunately most of them we’d already come across and fixed, so that’s good!
August 19, 2008
July 18, 2008
I’m currently writing a few dialogue and interaction menus for my current AIR project, and the thought had occurred to me that having established a reasonable methodology for handling and displaying modal dialogues within my Cairngorm-based app, I was perhaps using them almost by default, without thinking too carefully about whether a modal dialogue was the most appropriate means of interaction. By modal in this context we mean “A state of a dialogue that requires the user to interact with the dialogue before interacting with other parts of the application or with other applications”.
At the same time, Chris and I have been talking about metadata recently (another entry to come, but the premise was that persuading users to input metadata about assets is hard to incentivise). Related to that, Chris sent me this great link to an entry by Jeff Attwood that in turns talks about an entry by Eric Lippert on how dialogue boxes are perceived by users:
* Dialog boxes are modal. But users do not think of them as “modal”, they think of them as “preventing me from getting any work done until I get rid of them.”
- Dialog boxes almost always go away when you click the leftmost or rightmost button.
- Dialog boxes usually say “If you want to tech the tech, you need to tech the tech with the teching tech tech. Tech the tech? Yes / No”
- If you press one of those buttons, something happens. If you press the other one, nothing happens. Very few users want nothing to happen—in the majority of cases, whatever happens is what the user wanted to happen. Only in rare cases does something bad happen.
In short, from a user perspective, dialog boxes are impediments to productivity which provide no information. It’s like giving shocks or food pellets to monkeys when they press buttons—primates very quickly learn what gives them the good stuff and avoids the bad.
I liked that, especially the bit about “Teching the tech” – while it’s quite funny it’s also a pretty accurate reflection of my experience as a user.
This is also related closely to what Chris and I were discussing about metadata; expecting the user to fill in information that has no obvious purpose and slows down the primary task of upload/publish or whatever it is that they are trying to do, is likely to be ignored. If those fields/dialogues are modal or conditional, it’s worth thinking carefully about whether there are alternative ways to complete the operation or gather the infomation. That’s harder to do of course, and there are cases where modal dialogues should be considered appropriate, e.g. where the application is about to do something destructive like deleting or overwriting a file, but there are alternatives, like how IE and Firefox avoid breaking the flow of interaction when blocking certain actions.
July 01, 2008
Kudos to Adobe, Google and Yahoo for creating the mechanism for Flash content to be indexed on search engines. With one or two reservations (like how to distinguish between application content and a site) I think this is another significant move towards maintaining the ubiquity of Flash. It seems as though Adobe is steadily, but impressively quickly, removing piece by piece the most-cited drawbacks of Flash. Some of the most significant announcements (in no particular order):
- H.264 video support
- Open-sourced Flex SDK, BlazeDS
- Opened access to Flash Player APIs
- 3D support (thanks to Papervision, Away3D etc.)
In addition to the technology itself, Adobe has provided the means to develop and deploy it effectively, with the Flex SDK and FlexBuilder. Personally I have no objection to proprietary technologies when they a) work, b) don’t break anything and c) positively drive change and allow people to do things that standards-based technologies often take much longer to enable (and often not quite as well). Flash and Flex won’t be the standard, they will peacefully co-exist with other technologies (along with man, and fish); a single unified standard just isn’t possible in a competitive world, nor is it always desirable. Someone has to innovate, and attacking Flash (or Apple come to to think of it) for being proprietary is like attacking Ferrari for making a better sportscar (and charging for it). If it enables you to go faster, better, and (similar to Java and JS) is on 90-something% of desktops, who can blame Adobe for adding features and functionality that will maintain or increase edge and adoption? And at the same time if it is making key components of its platforms open, regardless of motivation, it’s A Good Thing*. So long as the standards do catch up, it’s fine.
There is I think, one thing left to do at the moment, the final hurdle as I see it – accessibility. It’s kind of in there, but if Adobe could make Flash and Flex as accessible as a typical web page, or at least as easy to make accessible as a web page, it would remove the one last stick with which it gets beaten. In fact and to bring this full circle, the same mechanism by which search indexing now works may also prove the key to unlocking accessibility, so maybe that’s already possible?
*None of these arguments apply to Microsoft, especially the Ferrari analogy. Silverlight is neither better or faster.