Interview with Scott Petersen on the C/C++ in Flash Player Sneak

Scott PetersenScott Petersen presented a project he’s been working on that takes C/C++ code and turns it into ActionScript that is runnable by the Flash Player. A lot of people wanted more info so I sent him some questions. You can take a look at the demo over on Peter Elst’s site.

Ryan Stewart: So who are you? What do you do at Adobe?

Scott Petersen: I’ve been at Adobe almost 10 years. I worked all but the last 5 months or so in the Acrobat Engineering Organization. My current role involves doing research for the Document and Application Technology Group (DATG). I’m doing research into development tools and methodologies for Adobe’s platforms (Flash, Flex, AIR.)

RS: And you blew people away during Sneaks at MAX right? What exactly did you show?

SP: At MAX I showed a demonstration of some experimental tools I’ve been developing that allow cross compiling C and C++ code into ActionScript that can then be used in Flash, Flex, or AIR apps.

RS: Now before we get started this is just a side project, right? Do we have plans to release this?

SP: It’s currently a side project. However, the considerable excitement around the demos will hopefully fuel my own efforts to eventually have a releasable piece of technology. No promises, but I personally would love to see what cool things the community could do with this kind of tooling.

RS: You showed Quake running in the Flash Player right? So how exactly does it work?

SP: I showed Quake running as an AIR application, though the AIR application was only made up of the application descriptor, a single SWF and Quake’s data file. While the SWF would run in shipping Flash Player 9, I cheated a little and optimized access to ByteArray objects in my AIR runtime. This did not entail any new APIs, though. The Quake SWF will happily run in your Flash 9-enabled browser though the performance won’t be quite what you saw at MAX.

The technology itself is essentially a backend for the LLVM compiler infrastructure that emits ActionScript instead of platform assembly code.

RS: Can you explain a little bit what the LLVM compiler is for those that might not know?

SP: The LLVM compiler infrastructure is an amazing set of programs that provides all of the common command line C and C++ development tools like a compiler, assembler, linker, archiver, etc. However, instead of generating and manipulating platform specific assembly code and object code, these tools operate on a platform neutral “assembly language” and byte code.

RS: So you’ve added/tweaked the LLVM compiler so that your C or C++ code compiles down to ActionScript?

SP: Right. I’ve created an LLVM backend that uses the same underlying mechanism as the platform specific assembly language generators for x86, ARM, PowerPC, etc. But instead of generating “real” assembly language, it generates low level ActionScript.

RS: You mentioned in sneaks that you can get some “faux-multithreading” using asynchronous code in ActionScript? Are there other performance or functionality enhancements that you get by converting C/C++ to ActionScript?

SP: One of the big differences between typical ActionScript and typical C and C++ is asynchronous vs. synchronous code. Most ActionScript code is asynchronous meaning that it responds to events. Most C and C++ code is synchronous meaning that there’s a “main” function that runs for the duration of the program. Naive conversion of C or C++ programs to ActionScript would result in a “main” type function that would execute continuously and not give the Flash Player a chance to draw, respond to keyboard or mouse input, etc. To get around that problem, the backend generates an ActionScript class for every C or C++ function. An instantiation of one of these classes more or less corresponds to a C or C++ function invocation. However, unlike the function invocation in C or C++, the corresponding object instance doesn’t do its processing all at once. It has a “work” method that, when called, does a little bit of work and returns. This allows the generated ActionScript to be effectively suspended and resumed to bubble all the way back out to the Flash player so it can draw, etc. If you imagine several instances of these types of objects, one could call the “work” method on each object in turn simulating multiple threads of execution.

RS: So can you take any C/C++ code and compile it to ActionScript right now using your compiler?

SP: Yes, but there are caveats. Any C or C++ code can be converted to ActionScript, but library calls used by any given piece of C or C++ code may or may not be supported by the tools. For many library calls, if a given call isn’t currently supported, it could be by either writing equivalent calls in pure ActionScript or by porting C or C++ implementations of the call. However, some library calls do things the Flash player / AIR runtime can’t do or won’t do (say, for security reasons). In those cases, the Flash player or AIR runtime would need to be extended.

RS: So will you be able to support other languages besides C/C++?

SP: I mentioned at MAX that C and C++ support effectively means some minimal support for any language that has a C or C++ based interpreter. Ruby, Python, PHP, Squeak and others fall into this category. A non-JITting JVM could go this way as well. As a first pass, just port the interpreter/VM. This doesn’t magically expose the Flash Player / AIR runtime’s object model, but it’s a start. A pretty obvious second step is to map that object model into the scripting environment/VM. There is probably a clever way to do this pretty transparently in many cases. Longer term, direct conversion could be developed as well for optimal performance.

LLVM uses the GCC frontend for C and C++ so pretty much any C or C++ GCC can deal with can be dealt with by the tool. One might imagine that other languages supported by GCC such as Java, Objective-C, etc. could make their way into LLVM as well. The way LLVM is architected, it should require very little work (or none at all) to have ActionScript support for those languages after LLVM brings support.

RS: Since people are excited, how can the community help? Are you looking for feedback on anything?

SP: Primarily, I’m interested in what people see using this technology for. That could be C/C++ libraries (Adobe code or open source), desiring to write Flash/AIR apps from scratch with C/C++ or interest in other languages this tech could enable (like Ruby, Java, etc.)

RS: Is there anything else you’re working on?

SP: I spend a fair bit of my time these days trying to decide which of the various uses of this technology to pursue first. Looking beyond enabling various languages on the Flash/AIR platforms, one could imagine something even crazier like conversion of, say, x86 machine code to ActionScript. Perhaps even dynamically. This is by no means right around the corner, but I’d love to some day be able to “free” code trapped in native libraries that have no corresponding source for use with Flash and AIR.

RS: Awesome! Thanks a lot for taking the time to do this Scott. The community was stoked about the demo, so thanks for giving us some insight into how it works.

[tags]Flash Player, C++, C, ActionScript, Scott Petersen[/tags]

The MTV/Adobe AIR Contest

The MTV/Adobe AIR ContestWe announced this at MAX and I wanted to post it so since Rey mentioned the Adobe AIR MTV contest on Ajaxian, I thought now would be a good time to post about it as well. So yup, we’re partnering with MTV to see what kinds of cool AIR apps the kids can come up with. The prizes are pretty impressive (and such witty names!). It’s not just Adobe stuff, it’s trips to New York City, VIP tickets and MP3 Players.

Me? I hope someone creates an AIR app that gets MTV to play music videos again. -ZING- :)

[tags]MTV, Adobe, AIR[/tags]

Generating PDF Directly from a Flex Application

AlivePDFI want to give a huge shout out to Thibault Imbert because he’s created AlivePDF, a very powerful library that allows you to generate a PDF directly from Flex/ActionScript. It was a big part of my presentation at MAX and I think it deserves it’s own post.

In the past creating a PDF from Flex or Flash consisted of taking a Bitmap screenshot of the page/application and then turning that into a PDF. AlivePDF is much more powerful. Using his libraries you can actually create text elements, picture elements, etc and lay them out on the page exactly how you want. One of my (very simple) examples in the MAX preso is a drag and drop WYSIWYG interface for creating PDFs. We’ve got some great server-side PDF technology, but this is a cool library if you need to generate PDF on the client side. He also has a hook for AIR that allows you to save the PDF directly to the hard drive.

Also want to say thanks to Marc Hughes (grand prize MAX winner) for this post. It helped me get started with AlivePDF.

[tags]Flex, PDF, AIR, AlivePDF[/tags]

Leveraging PDF and AIR

At long last here are the slides and examples from my MAX talk: Leveraging PDF and AIR. Before you dig in, I’m going to warn you that the title of the session is kind of misleading. There isn’t a lot of true integration yet with PDF and AIR. Right now the extent is that you can render a PDF inside of your AIR application but you can’t go beyond that. Luckily PDF is pretty powerful, so being able to render PDFs inside of your AIR application gives you a lot of new options.

One of my favorite parts of PDF and AIR is the script bridging. Because there is a really good script bridge between JavaScript and Flash, you can use that to your advantage in PDF. This web example is fully enabled with PDF and AIR so that you can create some really great user interfaces in Flex/Flash and then go in and write things directly to the PDF DOM. Hopefully you guys will do some interesting stuff with it.

Slides are here (and embedded below) and the examples are downloadable from here.

Update: I can’t embed the Share document because I’m having feed validation issues when I do. I also added the link to the web example. Sorry ’bout that.
[tags]PDF, AIR, MAX[/tags]

Adobe at SAP TechEd

You may have thought all the excitement was at MAX Barcelona but there are a lot of the cool kids up in Munich at SAP’s TechEd and Adobe’s own Matthias Zeller is up there presenting Adobe’s platform. I got to participate in a meeting with SAP and I can tell you that there’s some exciting stuff going on in the partnership with Adobe and SAP. SAP really gets user experience and they’ve been showing how great a platform like Adobe’s can be when used in an enterprise environment. Thomas Otter liveblogged some of Adobe’s session and has a good slant on the enterprise/SAP/Adobe RIA relationship. Props to Matthias for helping spread the word.

[tags]SAP TechEd, Adobe, SAP[/tags]

Google Analytics Rolling Out Tracking for Flash/Ajax

Google Analytics is rolling out some new features that will be of interest to Flash/Flex developers. As part of the new Google Analytics Event Tracking you will be able to expose your Flash events to Google so that you can track how people are interacting with your site. At first I thought this was like Nitobi’s RobotReplay for Flash but after talking to Andre, we came to the conclusion that it’s not quite that cool. However there’s still some interesting stuff here. The line that everyone keeps repeating is this:

These new reports are designed to help you understand how people use and interact with Ajax, Flash and multimedia on your site without artificially increasing your pageview metrics. In order to provide a way for you to define and track a wide variety of applications and interactions, there will be a new tracking module called ga.js.

But I wasn’t really sure what that meant. So after discussion I think you can actually expose your Flex/Flash events to this JavaScript file using ExternalInterface so that they show up on your Google Analytics page. That means anything in the Flex/Flash event model could potentially be reported on. A drag operation, a button click, even a custom event. This could be a really interesting way to track Flash and Flex apps so if anyone has a demo or an account, I’d love to hear what you think.

[tags]Google, Flex, Flash, Web Analytics[/tags]

Better Rich Internet Applications with Mind Reading

Mind Reading Rich Internet ApplicationsI think one of the primary benefits of RIAs is a better UI. I think web-centric designer/developers create better UIs which is one reason I like Adobe. But I’m also a huge fan of Vista and Office 2007 for the same reason – better, more expressive user interfaces. With that in mind, how cool is this!?

Microsoft has filed a patent for a device that would let them read the minds of people as they’re using software. The device could be used to create much better user experiences because as the article says, people are bad reporters of their experiences. By seeing exactly what they’re feeling/thinking while they are moving through an application we could create some REALLY slick and usable interfaces. We’d also have a great data set to work with.

[tags] User Interface, Microsoft, Rich Internet Applications[/tags]

Adobe AIR as the Wii

Abbas (couldn’t find his last name) has a very interesting comparison to Adobe AIR. The Wii. Even though he says he’s not sure it will succeed, I think he’s on to something. Nintendo opened up gaming to a new market becuase as he says, they went back to basics. They focused on a good, fun experience and the people followed. In some ways I think we’re doing the same thing with AIR. We’re opening up desktop development to a new group of people – web developers, and we’ve seen some really cool applications as a result.

Abbas thinks it won’t work because of the move to the web. But there’s still a lot to the desktop that people like. I’m betting on the tactic that combines the two so you can mix and match code and assets as your users require different assets.

[tags]Wii, Adobe, AIR[/tags]

Adobe at NewTeeVee’s Live Conference

Just caught the schedule for NewTeeVee Live, a conference that the OmPire is putting on as part of their NewTeeVee blog which focuses on video on the web. It looks like an interesting conference and Adobe’s Jim Guerard is going to be thre taking about New Media Platforms. He’s a VP in the Dynamic Media group (tons of cool stuff like Flash, AMP, Premiere, FMS and others). I’m not sure if he’ll have any new news, but we’re obviously doing a ton with video, so there’s a lot to talk about.

Robert Scoble also has some really good thoughts about what the conference could be. Seriously, go look at that list. He’s a freaking machine being able to rattle off 40 good suggestions for this conference. I especially like the idea of a platform/codec conversation with all the companies out there; DivX, Adobe, Microsoft, Move, etc.