I think it’s one of those things that is attractive, but very hard to do, especially when we consider the variety of presentation layer technologies out there.

I think there are some obvious common patterns emerging – for instance, declarative layout (MXML, XAML, NIB/XIB files, HTML/CSS), and the use of bindings to link UI elements to data.

It’s taken 20 years for the declarative layout idea to become widespread.

There are some good patterns that are less universal – like creating UI elements using ‘recipes’ (vector graphics, masks) rather than as bitmaps.

We’ve had issues (memory leaks, high CPU usage) with our Flex apps, and used the Flash debugger to eliminate them.

I’d wager that the average banner ad designer never does that, or even looks at the performance of what they’ve done.

Performance of the Flash player should be improved with 10.1, as that is the version that shares the same codebase with the mobile/Open Screen edition Adobe have been working on – i.e. moving away from having a separate desktop and mobile edition, and dealing with having to deploy the same plugin across 4 desktop OS and at least 4 mobile OS, plus games consoles and set-top boxes using 3 distinct CPU architectures.

And that, I think, is the problem anyone introducing a new web plugin will face. It’s no longer 1997, with over 95% of the things connecting to the web being Windows PCs running Netscape or IE. Hence the Open Screen Project.

Taking a look at my installed plugins, I’m trying to work out why we will see new ones.

What new types of content may need new plugins?

Video & Audio : DivX, VLC (Google video/Ogg), Veoh, Quicktime (which itself has various plug-ins for codecs, inc. Quartz Composer for OpenGL animations).

App runtimes : Curl, Java, Silverlight, Flash, Unity

Curl’s interesting but pretty much dead outside the Intranet, and inside the Intranet you may as well use .NET and an auto-update framework for your clients.

Java – ditto. If you’re going to use it, may as well do so out of the browser, or use WebStart rather than as a plugin.

Unity – it’s an interesting one, because what it offers (3D graphics) is on the roadmap of both Flash and web standards. On the other hand, like Adobe, Unity’s focus is on development tools – they’re happy to compile down to iPhone apps as well as their plugin.

My gut feel is that 3D will come to the browser before Unity is widely installed – even if Facebook games started pushing it out, it’s one of those things (like media plugins) that will meet the might of the IT department.

Other : Google Earth, iPhoto Photocast, emusic

Never seen the first two used in any site, and these days I use the emusic standalone app (built using the Mozilla codebase) because the plug-in was always flaky. URL integration to local apps seems far more common than plugins.

Truth be told HTML is a browser plugin itself. Just one that ships with the browser application. Because of that we *think* of HTML as the natural developer language of the web when in fact it just landed that way for now. Doesn’t mean it will always be that way. Nor should it.

There’s also some interesting benchmarks that show 2D drawing on OS X is far slower than on Windows XP. The flipside of this is that you rarely see the kind of ‘artefacts’ you often get on Windows XP where drawing operations fail while dragging a window around while the system is busy. Part of this is down to the approach where rather than drawing directly to the screen memory, drawing ‘off-screen’ and then compositing what’s shown to the user (although Vista, Win 7 and many desktop Linux also use the same approach). Other issues, I suspect relate to things like sub-pixel anti-aliasing (which can make vector drawing look a lot better, and requires you to draw to a higher resolution that you display).

What was interesting was that some of the higher level APIs, on the other hand, were far faster. CoreAnimation, for instance, often uses GPU acceleration (presumably sacrificing sub-pixel accuracy for speed). What I don’t know is how Flash interacts with these layers, and to what degree it implements it’s own drawing libraries vs being dependent on the ones in the OS.

To be honest, the biggest problem I have with Flash performance often comes from very simple content – I’ve seen banner adverts with a static image using up 100% CPU, and I’ve seen complex 3D Papervision demos using 13% or so – presumably because the people doing the latter had to focus on performance to achieve their demo (and therefore on a high-spec machine it will use little CPU) whereas the banner ad guys have done something stupid, like set a timer loop to 0 msecs, because they don’t think about performance at all. Another question is how much of this content uses the new AS3 VM, which is much faster and better optimised, and how much of it is done using earlier versions.

(Maybe the Flash tooling needs improvement here, doing some static analysis on the code and warning of stupid things?? I do think that the likes of Yahoo should monitor and ban adverts that use excessive CPU – what is the carbon footprint of every Yahoo Mail user losing 1 hour of battery life per charge cycle?).

Pavel – I think some of the reasons why 3D may be faster possibly relate to the issues I touch on above – the 3D plugins tend to be wrappers over OpenGL and therefore GPU acceleration – whereas 2D drawing tends to be over Quartz, and video over Quicktime (which again, is hardware accelerated for the formats Quicktime natively supports).

In terms of the basic model, I don’t think there is a whole lot of difference between HTML’s Canvas and SVG combined with JavaScript, and Flash’s stage+ActionScript, and from what I’ve seen of Canvas and SVG demos they do not significantly less CPU than Flash (a few percentage points variance). Where people do anything with a complex UI in the browser (let’s say using Capuccino or GWT or one of the other complex JavaScript frameworks) the time to load and memory usage of the app flies right up.

Maybe it’s just a category problem – i.e. there are a lot of people out there who want the browser or Flash to be a universal operating system, when it’s not. Let’s face it, there’s a reason why an app runtime like .NET or Cocoa is vastly bigger than a web browser (and the browser often hugely bigger than Flash) – and that’s before you add in the size of the OS underneath them all, or relative performance. (Hell, there’s still people out there who won’t use C# due to the performance impact).

And my view on cross-platform development is quite strong – you need to start by understanding why your customer is using the platform, rather than starting from the point of view of minimising your costs. You certainly need to understand platform conventions where they exist, and how they differ. Again, considering the iPad, it’s evident that Apple haven’t used the same ‘Mail’ program on the desktop, iPhone and iPad. I’m not advocating writing everything for every platform – a game running full screen on a 3.5” touch screen is going to be the same on any equivalent hardware – just that the idea of universal software is a pipe dream.

Well, at least so long as some of us have something different from a Trabant . . .

I really don’t know what you’d use ‘true power’ for in a web browser, but judging from the quantity of flash games vs. Unity3D games, I’d suggest that Flash’s comparative lack of power is not hurting it as much as you believe. I’m sure there are things it could do better, but 99% of the flash/flex apps out there don’t push the limits of performance. (Excluding video, though 10.1 significantly increases video performance)

http://hat6.com/2009/12/09/using-flex-for-pushing-data-from-blazeds/