HTML and Flash Thoughts

Kevin Lynch blogged today in response to a number of issues that have cropped up recently and does a great job of laying out both Adobe’s vision for Flash and for web tools. That and a couple of other posts got me thinking a bit. The first is Jeffrey Zeldman’s post. He contrasts Flash with some of the benefits of standards. Most of the time I see “HTML5 is going to kill Flash!!” without any kind of rational conversation on why. Jeffrey does a better job than most. His intro paragraph is a perfect example:

Lack of Flash in the iPad (and before that, in the iPhone) is a win for accessible, standards-based design. Not because Flash is bad, but because the increasing popularity of devices that don’t support Flash is going to force recalcitrant web developers to build the semantic HTML layer first. Additional layers of Flash UX can then be optionally added in, just as, in proper, accessible, standards-based development, JavaScript UX enhancements are added only after we verify that the site works without them.

My Current Problems with Flash

There are a couple of things I hate about Flash inside the browser. Both of them were covered very well by Richard Leggett in his Flash/HTML5 post. The first is browser integration. One of the things I think a lot of users find annoying about Flash is that it feels so alien in the browser. We have basically a single API, ExternalInterface, that developers can use to connect HTML and Flash. But it’s very awkward and it makes both the development experience and end user experience feel very different. Flash has become the “black box” of the browser. If you take a look at AIR, you see how great those two worlds can be. Flash and JavaScript can call each other’s APIs, Flash can access the DOM, and JavaScript can call Flash only when it needs to, for things like playing sound or performing graphical tricks. SVG and Canvas add a layer of complexity to that, but I don’t think that’s an insurmountable hurdle. In fact, I think those two technologies, when combined with Flash, would make a very interesting combination.

But that leads to the other problem. Flash is horrible when it comes to the semantic web. And this causes some other issues, like deep-linking or search engine optimization that we’ve worked on, but haven’t perfected yet. As Richard says, and Jeffrey notes, the current solutions aren’t entirely bad. Flash works very well with a CMS like Drupal so that you can have the semantic web layer and a Flash layer. And largely it depends on your project. In some cases that semantic layer isn’t going to be as important. It’s also important for developers to use Flash inside HTML where appropriate. Another thing I’d like to see is making it easier to create Flex applications that don’t take up the whole page, but work within an HTML context. Think a bit about what I said above combined with some kind of Dreamweaver/Flash Builder integration so that the developer can unify the HTML and Flash experience at a tooling level.

Adobe Is a Web Tools Company

People seem to think that Adobe has eschewed HTML5 in favor of Flash. A lot of innovation goes into Flash. We have a number of features that our customers want and we’re able to add those to the runtime and create tooling around them because we can move quickly and innovate. HTML5 is obviously more consensus driven. As a result, some of the details that would be required to add tooling support haven’t been fully nailed down yet. But we’ve moved ahead to do what we can. Dreamweaver continues to have great HTML and JavaScript framework support. We’ve shown sneaks of BrowserLab that will help HTML/JS developers see how their site looks in various browsers. We’ve included the latest versions of WebKit in Adobe AIR so that HTML developers can take advantage of a lot of HTML5 features in a desktop context, and in ColdFusion we’ve got support for creating ExtJS 3.0 components with ColdFusion tags. We even sneaked a feature at MAX that used Flash and Illustrator to display animated vector content using the canvas tag. So Adobe is supporting HTML5 in a number of different ways already and experimenting with even more. When the spec is nailed down, you’re going to see a lot of Adobe tools that support it.

Flash Is Driven By Customers

It’s important to remember that Flash isn’t some isolated plug-in that we maliciously deployed on 98% of web browsers and that consistently hits 80% penetration for new versions in 6 months. Flash is driven by customers. Both developers and end users. Developers still want content that runs the same way across browsers (and now devices). They still want web content that provides innovation around things like video, sockets, animation, data push, web camera, 3D transformations, and works across multiple platforms and 98% of the people on the web. There are some developers who won’t ever use Flash, and that’s fine. There are others who want to do things that HTML5 just doesn’t have an option for now. Flash is there to fill that gap. Part of the reason we can innovate with Flash is because we control the source code. While we’ve worked hard to be more open, ultimately it’s our customers demand for innovation that drives us. And I wouldn’t want Flash to open up and lose that ability to innovate. It wouldn’t be fair to our customers.

I’d love to see Flash do a better job of integrating with the browser and the semantic web. And I hope HTML5 pushes us more in that direction. I disagree that the era of plug-ins is coming to a close because I think there will always be web developers who want to do a little bit more and have the same experience across devices and platforms. Adobe can move at that speed while still offering tools for web developers of all stripes because both HTML and Flash are baked into our DNA. I genuinely wish for a more open dialogue between standards organizations and the Flash community. Unfortunately it seems like a “my way or the highway” attitude when it comes to web standards. I can understand that to some degree, but I think the web would be a much better place of everyone took a deep breath and took another look at Flash’s deficiencies and its strengths. With that as a starting point, I think there could be some very valuable conversations about how Flash can do more to support standards while still catering to customers who want new features.

  • http://blog.paveljacko.com Pavel

    I think, biggest issue with flash is just one and is very simple. it’s SPPED. nothing else. This is only thing which restrict anything else. “Give me power and I’ll change the world” applies here. You can develop/hack nearly everything else. 3D, robust frameworks, advanced UI, proper video, all that could be build but can’t. Also this is major reason why people really hate it. They don’t mind flash advertisement but they hate when it slows their computer(yes that’s fault of advert-flash idiots who don’t care about optimizations but also by fact that flash is simply slow) and I can’t blame them…Flash don’t give us that true “power”. I hope one day(soon) it will…otherwise I don’t see flash future very bright and that would be big shame

  • JulesLt

    Part of the problem is that the loudest voices are those who take sides – the vast majority of us out there are choosing tools on their suitability and reach.

    In the mobile browser space HTML 5 currently has better reach than Flash, so is a more logical choice for certain types on app/content that have traditionally been done in Flash.

    In the desktop browser space – well, you have to use Flash to render SVG content in Internet Explorer. Even if IE9 implements all the standards MS promise it will, it will still be years after that before we can stop supporting IE.

  • Andy

    You missed one of the biggest problems I have with Flash — inconsistent performance across platforms. It’s ironic, since Flash used to be the environment of choice to avoid cross-browser issues, however as modern browsers continue to adopt standards, Flash has fallen behind. As it stands now, I cannot develop a flash application in Windows and expect it to perform the same in Linux or Mac.
    The most popular bug on the flash player bug tracker is FP-890, “Poor performance on Mac OS X”, and it was created in November 2008! In that bug you’ll see comparisons of flash player performance on Windows vs Mac. A flash player video running on a Mac takes 61% of the CPU. The same video, running in VMWare, with the entire virtualization stack, Windows, etc. takes 8%.
    This kind of performance disparity is unacceptable for reasonable cross-platform development. I can, today, develop a site using HTML5 video and be confident it will perform the same across all modern browsers — something I can’t do with Flash.
    Until I stop hearing from my customers on non-windows platforms about issues running my Flash applications, I will continue to turn to HTML5 whenever possible.

  • Simon

    Flash Player 10.1 is going to a good step in the right direction with regards to increased performance. HTML5 still lacks a lot of functionality that Flash possess and has a mature set of development tools.

    HTML5 will get there eventually, and then its up to the developer to choose which technology suites the situation.

  • ryanstewart

    @Pavel what do you want when you say “true power”? I think we can make some optimizations to Flash and make it more performant but I don’t know which area you want to see.

    @JulesLt, yeah, I think we got behind on mobile. And frankly, it’s a good example of why a real standard (like HTML) will always continue to push forward. With Flash Player 10.1, hopefully that changes.

  • http://hat6.com Dusty

    Flash/Flex may not have been built for integrating with the latest JS libraries, but that doesn’t mean that it can’t. Check out the presentation I recently did for SeaFlexUG, showing how to use Flash/Flex for efficient push/long-poll communications for HTML applications

    http://hat6.com/2009/12/09/using-flex-for-pushing-data-from-blazeds/

  • http://blog.paveljacko.com Pavel

    @ryanstewart “true power”?…I simply mean that if you trying to build supercar from trabant to run a race you needs to be mechanical genius…surely you can optimize its engine,wheel,brakes…and on the end you have much better ehm….trabant…it will be surely fast but in comparison with other trabants. However if you try to build supercar from let’s say mustang you don’t need to be such a “genius”. even normal human can win the race without spending so much time in the garage. With this weird comparison I just want to point out that Flash Player dev team done absolutely amazing job with it…but it looks like they try to make supercar from trabant. Great job…but bad foundation…I’m not fp engineer…I’m not sure what are the exact issues BUT I already seen things performing way faster (Unity3D,O3D)…sadly I seen even Silverlight performing faster in some crucial cases and looks like matter of time when it will outperform flash if nothing changes…why?…because maybe they don’t have mustang but they have opel(or something)…while rubbish it’s still little bit better than trabi. But most worrying thing is that while native apps and browsers getting faster and faster as machines become faster, flash player is still more and more behing and “genius” optimizations are just not enough. Technically speaking stuff like byte level access speed, array iterations, hash tables, vectors, frame animation…all those very basic things should perform much much faster, also having stuff like threading, better gc, rendering time, even having possibility to somehow switch to native execution instead of JIT…all those things are something I consider as “foundation” and “power”. Flash community is already potent enough to build rest of the stuff on top of that. Just my opinion based what I know and seen so far.

  • http://hat6.com Dusty

    @Pavel
    I find it difficult to believe that you could consider anything running in the browser to have ‘true power’. When you compare any browser-based performance to c++ performance, the difference between Unity3D and Flash is negligible. Also keep in mind that Unity has completely different optimizations than Flash does… Unity and other plugins suffer from the same downfall that Ryan was talking about, specifically difficult integration with the rest of the web page.

    I really don’t know what you’d use ‘true power’ for in a web browser, but judging from the quantity of flash games vs. Unity3D games, I’d suggest that Flash’s comparative lack of power is not hurting it as much as you believe. I’m sure there are things it could do better, but 99% of the flash/flex apps out there don’t push the limits of performance. (Excluding video, though 10.1 significantly increases video performance)

  • http://blog.paveljacko.com Pavel

    @Dusty Obviously comparing native code with some plugin is just nonsense…I was pointing to the gap between flash performance and native app performace over the decade. While 5 yers ago difference was just reasonably big now it’s enormous. And yes you have games done with flash and there are many but those games are “flash games”. Simple, easily accessible on the net. what if flash would be somewhere on unity3D level for example? well in my opinion this would revolutionize game industry and not just that. applications like google earth, online 3D IDEs, games like Second Life, scientific 3d virtualizations, virtual tours, whole online casino/gambling industry..all that reachable on 98% desktops…you can’t really compare “flash games” industry with all this. This is just different category/scale of business. Another thing is reputation and image. Flash has very bad image. I work with different technologies and no other technology give me people’s reactions like “err..flash?!..hmm I see” in very ironic way. In these days image means a lot (look at Apple) you can sell shit covered in paper and people will buy it and same on other hand, sell best product and people won’t. And I think flash is still in second category. Most recognizable it is now with current highly debated launch of iFamous. Strange part of all that flame-ware was that it wasn’t just Apple who didn’t want to see flash player on their device but also actual users. And except iFanatics that means that they have pretty much horrible experience with it to be willing sacrifice most of the web experience and don’t bother about flash. People still think that flash is some annoying advert,intro or recently slow video (hopefully HTML5 will catch on quickly and that advert plaque will move there) Can you blame them? Flash user experience is not great in comparison of expectations of today’s 2010 aware user (forget for now 1999’s intros and flash menus) In many cases not entirely because of developers, not because of UI but just because of that often “jerkiness” which destroy all that nice feeling. 99% of flex apps outhere are quite simple, you’re absolutely right. And did you thought about the reason? Have you ever tried to do complex ui in flex? It’s hard, very hard and not because of complexity. I had few projects where flex/air was great candidate but then just because of performance limitations I went to .NET solution. Yes you can optimize but then we’re back at trabants. Air been here for while. Did you seen some more serious apps out there? And reason? Flex and Especially Flex 4 makes creating windows like applications piece of cake so why nobody use it? Well, I can tell you my reason…just the performance. Flex 4 is almost perfect for such a things but in most cases I just can’t get most of it potential and trust me this has potential to even knock down WPF model ;) . And to make it clear, I’m not trying bitching. Adobe done amazing job so far and their situation is not easy. Flash Player has lot of heritage to carry on, loads of devices to support, loads of crap code to deal with, loads of different cpu speeds and I know that and I highly respect engineers who have to deal with all that. I’m just giving my thoughts. If I would know answers I would call Adobe and tell them what to do:D. And yes there are obviously other issues with fp including browser integration, proper seo model, better IDEs, more mature language and others. But they are not as major and influential as this one is. Because on this most of the rest is based on. So again, do I really think that having those entire possibilities means “power”? Yes I certainly do.

  • JulesLt

    As Ryan’s not said it – and bearing in mind that I’m a Mac user, and I loathe what bad Flash content does to my battery – not all of the issues around Flash performance are with Adobe. For instance, the Flash dev team have been quite clear about the way that OS X doesn’t offer them an API to allow hardware acceleration of Flash video on OS X.

    There’s also some interesting benchmarks that show 2D drawing on OS X is far slower than on Windows XP. The flipside of this is that you rarely see the kind of ‘artefacts’ you often get on Windows XP where drawing operations fail while dragging a window around while the system is busy. Part of this is down to the approach where rather than drawing directly to the screen memory, drawing ‘off-screen’ and then compositing what’s shown to the user (although Vista, Win 7 and many desktop Linux also use the same approach). Other issues, I suspect relate to things like sub-pixel anti-aliasing (which can make vector drawing look a lot better, and requires you to draw to a higher resolution that you display).

    What was interesting was that some of the higher level APIs, on the other hand, were far faster. CoreAnimation, for instance, often uses GPU acceleration (presumably sacrificing sub-pixel accuracy for speed). What I don’t know is how Flash interacts with these layers, and to what degree it implements it’s own drawing libraries vs being dependent on the ones in the OS.

    To be honest, the biggest problem I have with Flash performance often comes from very simple content – I’ve seen banner adverts with a static image using up 100% CPU, and I’ve seen complex 3D Papervision demos using 13% or so – presumably because the people doing the latter had to focus on performance to achieve their demo (and therefore on a high-spec machine it will use little CPU) whereas the banner ad guys have done something stupid, like set a timer loop to 0 msecs, because they don’t think about performance at all. Another question is how much of this content uses the new AS3 VM, which is much faster and better optimised, and how much of it is done using earlier versions.

    (Maybe the Flash tooling needs improvement here, doing some static analysis on the code and warning of stupid things?? I do think that the likes of Yahoo should monitor and ban adverts that use excessive CPU – what is the carbon footprint of every Yahoo Mail user losing 1 hour of battery life per charge cycle?).

    Pavel – I think some of the reasons why 3D may be faster possibly relate to the issues I touch on above – the 3D plugins tend to be wrappers over OpenGL and therefore GPU acceleration – whereas 2D drawing tends to be over Quartz, and video over Quicktime (which again, is hardware accelerated for the formats Quicktime natively supports).

    In terms of the basic model, I don’t think there is a whole lot of difference between HTML’s Canvas and SVG combined with JavaScript, and Flash’s stage+ActionScript, and from what I’ve seen of Canvas and SVG demos they do not significantly less CPU than Flash (a few percentage points variance). Where people do anything with a complex UI in the browser (let’s say using Capuccino or GWT or one of the other complex JavaScript frameworks) the time to load and memory usage of the app flies right up.

    Maybe it’s just a category problem – i.e. there are a lot of people out there who want the browser or Flash to be a universal operating system, when it’s not. Let’s face it, there’s a reason why an app runtime like .NET or Cocoa is vastly bigger than a web browser (and the browser often hugely bigger than Flash) – and that’s before you add in the size of the OS underneath them all, or relative performance. (Hell, there’s still people out there who won’t use C# due to the performance impact).

    And my view on cross-platform development is quite strong – you need to start by understanding why your customer is using the platform, rather than starting from the point of view of minimising your costs. You certainly need to understand platform conventions where they exist, and how they differ. Again, considering the iPad, it’s evident that Apple haven’t used the same ‘Mail’ program on the desktop, iPhone and iPad. I’m not advocating writing everything for every platform – a game running full screen on a 3.5” touch screen is going to be the same on any equivalent hardware – just that the idea of universal software is a pipe dream.

    Well, at least so long as some of us have something different from a Trabant . . .

  • http://polyGeek.com polyGeek

    Anyone who thinks the era of browser plugins is drawing to a close is delusional. Flash, and other plugins, will only become more ubiquitous as time goes on. That’s because these plugins can innovate/evolve far faster than HTML will ever dream of.

    Truth be told HTML is a browser plugin itself. Just one that ships with the browser application. Because of that we *think* of HTML as the natural developer language of the web when in fact it just landed that way for now. Doesn’t mean it will always be that way. Nor should it.

  • http://blog.paveljacko.com Pavel

    @JulesLt I Agree with you. I talked about 3D as result. High CPU usage is caused with even slightly more complicated banner. And because CPU must do so much work just for simple stuff performance is affected obviously. Term performance covers huge spectrum of other topics, it’s not one entity so it’s hard to discuss about that directly. Also you have point about “universal software”. That’s just utopia. But world is not black and white, so none of extremes is good. One platform for every single programs neither unique program for every different platform. The middle is best most of the times. You must count costs, user experience, maintainability and lot of other factors and I think that’s the reason flash is here. I think it’s great presentation layer. But project generally has more that just that.(backend,db,soa..) That’s lot of different languages and platforms. For example I like .NET for its capabilities in backend area but don’t like its front end presentation architecture. That’s why I prefer Flash.NET :D …I think having one consistent presentation layer base is great idea but currently is still in infancy. If application can resize according to user request It can also retransform to different device request. But again, platform evolution is not mature enough to allow you do it seamlessly. I just think that Flash(flex including) has much more potential here and on application field. It doesn’t need to replace them. I don’t need AS3 server-side language, I don’t need Flash Oracle, I don’t need Flash Halo 3… I don’t need flash to do everything, just do what it do best, user experience only do it better because I think there is definitely room for that.

  • http://blog.paveljacko.com Pavel

    @polyGeek I think that HTML5 will be actually beneficial for plug-ins. It will take all that dirty work on its own shoulders (video, adverts, porn:D..) and plug-in will do what should be their purpose. Only threat to plug-in is its own penetration I guess and its own quality. So Adobe has few years to perfect their auto update because it’s little bit PIA now:)

  • JulesLt

    Pavel – but the interesting thing is that point about seeing Papervision demos that use less CPU than a banner advert – I think there’s a mix of things here – performance of the Flash player, and performance of the code produced by the Flash authoring tools.

    We’ve had issues (memory leaks, high CPU usage) with our Flex apps, and used the Flash debugger to eliminate them.

    I’d wager that the average banner ad designer never does that, or even looks at the performance of what they’ve done.

    Performance of the Flash player should be improved with 10.1, as that is the version that shares the same codebase with the mobile/Open Screen edition Adobe have been working on – i.e. moving away from having a separate desktop and mobile edition, and dealing with having to deploy the same plugin across 4 desktop OS and at least 4 mobile OS, plus games consoles and set-top boxes using 3 distinct CPU architectures.

    And that, I think, is the problem anyone introducing a new web plugin will face. It’s no longer 1997, with over 95% of the things connecting to the web being Windows PCs running Netscape or IE. Hence the Open Screen Project.

    Taking a look at my installed plugins, I’m trying to work out why we will see new ones.

    What new types of content may need new plugins?

    Video & Audio : DivX, VLC (Google video/Ogg), Veoh, Quicktime (which itself has various plug-ins for codecs, inc. Quartz Composer for OpenGL animations).

    App runtimes : Curl, Java, Silverlight, Flash, Unity

    Curl’s interesting but pretty much dead outside the Intranet, and inside the Intranet you may as well use .NET and an auto-update framework for your clients.

    Java – ditto. If you’re going to use it, may as well do so out of the browser, or use WebStart rather than as a plugin.

    Unity – it’s an interesting one, because what it offers (3D graphics) is on the roadmap of both Flash and web standards. On the other hand, like Adobe, Unity’s focus is on development tools – they’re happy to compile down to iPhone apps as well as their plugin.

    My gut feel is that 3D will come to the browser before Unity is widely installed – even if Facebook games started pushing it out, it’s one of those things (like media plugins) that will meet the might of the IT department.

    Other : Google Earth, iPhoto Photocast, emusic

    Never seen the first two used in any site, and these days I use the emusic standalone app (built using the Mozilla codebase) because the plug-in was always flaky. URL integration to local apps seems far more common than plugins.

  • JulesLt

    Pavel – “I think having one consistent presentation layer base is great idea but currently is still in infancy”

    I think it’s one of those things that is attractive, but very hard to do, especially when we consider the variety of presentation layer technologies out there.

    I think there are some obvious common patterns emerging – for instance, declarative layout (MXML, XAML, NIB/XIB files, HTML/CSS), and the use of bindings to link UI elements to data.

    It’s taken 20 years for the declarative layout idea to become widespread.

    There are some good patterns that are less universal – like creating UI elements using ‘recipes’ (vector graphics, masks) rather than as bitmaps.