It bummed me out, and still does. I thought AIR was an okay solution to the problem, but by the time AIR came around it was pretty clear that “the web” had come to mean HTML/JS, and I’m fine with that. So as PhoneGap started getting traction, and then Adobe took a major interest in the project, I was excited about the prospect of working on it as an Adobe evangelist, and more importantly, working with the teams behind it to see what else they had up their sleeves as the web moves forward.

But another side benefit is that it’s put me on what I’d consider the “right” side of the web argument. Two things got me thinking more about this. One was a very good post by a VC named Mark Suster, who while not telling his companies to focus exclusively on the web, has told them to make it a big part of what they do. So many startups nowadays are thinking completely mobile-first while ignoring the web, I’d say at their peril. When I think of my own usage, I’m still using a lot of native apps (for reasons I can’t quite figure out) but the ones I enjoy most have a web component that is well done and part of the overall experience. Yelp comes to mind. Another is Untappd. I can do almost anything I need to on the website version of Untappd so it’s not as though I’m getting a watered down experience. It more easily lets me move between contexts and devices while still using the service. I contrast that to something like Foursquare or Path. Both are mobile-centric, and with Foursquare I can’t check in because it wants to be sure that I am where I say I am (using GPS) but it still makes the web side of it less useful for me. Path is unusable when you’re not on a mobile device as far as I can tell. When I log in all I get is “download the app”. Which I hate. Mobile is clearly important but the web can’t be ignored.

If this is what I see when I log into your site, you're too mobile-first.

The second thing that got me thinking more about this was something Brian Leroux pointed to on the Cordova mailing list, a post by Tim Berners-Lee about Installable Web Apps. This is a model I would love to see take hold. As Tim notes, there are a few things that users need to have when they’re installing web apps, and some trust/permissions issues that need to be figured out. Right now, I think PhoneGap is closest to this model, but a huge, huge, part of me wishes PhoneGap didn’t need to exist. If we could somehow skip the native shim and just take for granted that every platform supported, and at its core used, installable web apps. Maybe something like the WebOS model. But we’re not there yet. So for now, I’m glad I get to work with PhoneGap and build apps with web technologies. Eventually though I think PhoneGap can be used as inspiration for installable web apps. This is kind of how the standards world moves, as more and more people adopt something, people find ways to bring that something back into the standards. I think some of PhoneGap’s APIs and methodologies would make a great start at the idea of installable web apps. And I think the guys behind PhoneGap will be at the forefront of making those things happen, which means Adobe is going to be a really cool place to be over the next few years. It feels like there is a lot of potential to change the world and while I miss spending time with Flash, I feel like the HTML/JS/CSS work I’ve been doing and that Adobe is investing in, will make a similar impact on the web down the road.

Translating Canvas Data Into a Binary Format

Canvas has a getImageData() method that gives you an ImageData object. Within that ImageData object is a data property, which has the actual array data of the image. Normally I would have been able to stop right there because that would (in theory) have the information I needed. But what I had to get at was the ArrayBuffer. The way the spec has been implemented you can’t do anything with the ArrayBuffer. Instead you have to use an ArrayBufferView, which takes the form of TypedArrays in JavaScript. Luckily, to get the actual buffer you can just call the buffer property on any of the typed arrays and do what you want with it. But Canvas (at least in Chrome) is slightly different. The object you get from the ArrayBuffer of ImageData.data is something called CanvasPixelArray. Currently CanvasPixelArray doesn’t behave like a regular typed array, it looks like it will become a Uint8ClampedArray but the way it works in Chrome right now the CanvasPixelArray doesn’t provide access to a buffer property so you can’t send/access the ArrayBuffer data. Luckily getting that data into a Uint8Array, which you can get the buffer data from is pretty easy:

     imagedata = context.getImageData(0, 0, imagewidth,imageheight);
 
     var canvaspixelarray = imagedata.data;
 
 
     var canvaspixellen = canvaspixelarray.length;
     var bytearray = new Uint8Array(canvaspixellen);
 
     for (var i=0;i<canvaspixellen;++i) {
          bytearray[i] = canvaspixelarray[i];
     }

With that new Uint8Array all that we have to do is grab the buffer property and we can send it across the wire (beyond this post, but I’m planning on writing up the binary WebSocket info).

Reassembling the pieces

Now what I wanted to do was take that binary data from my Canvas and render it as a PNG file on the screen. The first step is to use a Canvas object to render out a PNG. But before that we have to get the data into a Canvas. In theory, you should be able to just do what happened above in reverse. But it’s not quite that simple. Once you get back the binary data from somewhere (WebSocket say), you’ve got an ArrayBuffer that you have to deal with. There is a putImageData() that takes an ImageData object, and we can create an ImageData object a few different ways, but you can’t set the data property of it. It’s read only. So we can’t take our data from the ArrayBuffer and just put it into our Canvas. We have to manually loop through the data property and line-by-line change the data.

          var bytearray = new Uint8Array(event.data);
 
 
          var tempcanvas = document.createElement('canvas');
               tempcanvas.height = imageheight;
               tempcanvas.width = imagewidth;
          var tempcontext = tempcanvas.getContext('2d');
 
          var imgdata = tempcontext.getImageData(0,0,imagewidth,imageheight);
 
          var imgdatalen = imgdata.data.length;
 
          for(var i=8;i<imgdatalen;i++)
          {
               imgdata.data[i] = bytearray[i];
          }
 
          tempcontext.putImageData(imgdata,0,0);

If the above isn’t clear, basically I’m just creating a new Uint8Array with the data from the server, then creating the temporary Canvas so I can get image data from it, and when I have that, I’m looping through the data property and inserting my own data from the Uint8Array.

Rendering it as a PNG

So now we have a Canvas (not being displayed) that has all of the data from our server, so it’s an exact graphical copy of the info we received. Turning that into a PNG is actually pretty easy because HTML is awesome. Canvas has a toDataURL() method that will take whatever is in the Canvas and create a string that can be put into the src property of an image. Then putting that image somewhere on the DOM will display the data as an image.

          var img = document.createElement('img');
               img.height = imageheight;
               img.width = imagewidth;
               img.src = tempcanvas.toDataURL();

Fin

Now that I’ve gotten my head around binary data a lot more, I’m kind of excited about JavaScript typed arrays. Looking through the list it looks like the typed arrays will help a lot with byte manipulation because of the different types. It also looks like they’re pretty fast (at least the fastest option at the time of that post).

In fact, I’d argue they still aren’t. But we’ve taken that as something that comes with the web. It’s always evolving, always moving, and while things will start to coalesce more and more, in the end, you have to get in and be ready to move. That’s kind of the approach we’ve taken with Adobe Edge. We just released Preview 4 of Edge which incorporates a lot of features that people have been asking for. Some of it I’m not even sure if it was on the original roadmap. But the Edge team made a conscious decision to be very agile, to build Edge in such a way that it could incorporate customer feedback quickly, and then getting product management on board to do lots of versions very quickly. I think it’s worked out very well and despite being on the earlier side of Grant’s curves, I think Edge will be a very helpful tool for a lot of people because of it.

Developer tools are a bit of a different story in terms of both ecosystem and readiness in my mind. There isn’t one, big, HTML IDE that people seem to like (akin to Flash Builder, Eclipse or Visual Studio). Instead people seem to be using a lot of different things and experimenting. What actually seems to be most popular right now are the basic text editors like TextMate or Sublime (my favorite). These seem to be focused on helping smart people work faster. Lots of shortcuts, lots of snippets, but not a lot in terms of helping along the learning process. And I think that’s just where we are now as far as HTML/JS/CSS tooling. But I’m excited to see that evolution as well and see what happens when frameworks get a bit more standardized and more general web developers start jumping into JS more and more. Will those people need a more robust HTML/CSS/JS editor that’s still developer centric? And I think the answer is yes, but I think it’s also tough to really see what that would look like until the JS/HTML/CSS stack is a bit more solid. But I’m excited to watch it and find out. And from what I’ve seen of Adobe’s HTML tooling side, we’re taking a good approach and I’m excited to see what people think as the PMs share more and more of it.

It’s fascinating to see some of the anti-Flash attitudes manifest themselves in the new world of a very rich HTML5. And I think it goes to show that a lot of the animosity towards Flash was rooted in a very strict interpretation of what is “good” when it comes to UX and UI. I don’t even think that’s entirely wrong, but I do think that some of it misses the point. As Jeff noted, you have to have experiments like this that show off the technology. This kind of thing inspires people to think beyond the gimmick and potentially implement some of these ideas in a more usable and meaningful way. It also does a disservice to the fact that things like this make the web a more beautiful place. This isn’t “design” in the traditional sense. But it’s still art, and art should be celebrated. Most of the things I’ve seen on Twitter were pretty positive about the site and appreciated it, but the opposite reaction from a lot of the web standardistas was, as Jeff said, disappointing.

My plea for the potential creators of things like this is to not heed the mindset that because you’re using open standards you need to adhere to the unwritten rules of UX/UI design and what’s “good”. The open web is at the beginning of what is going to be an amazingly creative period. With technologies like CSS3 and Canvas, the open web has the building blocks in place to harness a lot of creative energy. That is going to take many forms and some of it won’t seem as useful as other bits. But it’s all part of a process, one that we saw with Flash, that ultimately leads to better ideas, better implementations, and a better web.

This example is part of an app that’s using Backbone.js so I won’t post a working example because the Backbone.js infrastructure makes it kind of tough to strip out just the toggle button part. I’ll post the whole thing later, but wanted to show the general process.

The Look and Feel

To set it up I wrapped a div tag around a select tag with a couple of options. The div tag provided the border so it ended up being the track that the sider went along. This makes it pretty easy to customize the look and feel while still being very nicely degraded into a regular HTML select box.

                         <div id="toggle" class="toggleoff">
                         <select name="activetoggle" id="activetoggle" class="selectoff">
                              <option value="all">All</option>
                              <option value="active">Active</option>
                         </select>
                         </div>

Initially both the div and the select start with “off” classes. At this point the CSS starts to take over and create a more slider-looking button. There is a cool -webkit-appearance property that can be used on the slider to give it some instantaneous look/feel. But for the most part I rely on -webkit-border-image to customize the look and feel of the slider button. The CSS for the toggle button itself, the div tag, and the off states give us a basic looking toggle with no real toggle functionality because it behaves just like a regular HTML select box.

Toggle button off state

select {
     -webkit-appearance: push-button;
     margin-top: 0px;
     margin-bottom: 0px;
     height: 50px;
     width: 155px;
     border-style: solid;
     border-color: #cdcdcd;
     border-width: 1px;
     color: #555;
       font-size: 25px;
       overflow: hidden;
     padding-top: 2px;
       padding-bottom: 1px;
     white-space: nowrap;
     text-align: center;
 
}
 
.selectoff {
     -webkit-border-image: url( ../assets/button.up.png ) 4 4 4 4 stretch stretch;
     padding-left: 60px;
 
}
 
#toggle {
     margin-left: 5px;
     margin-right: 5px;
     margin-top: 5px;
     height: 50px;
     width: 312px;
     -webkit-border-radius: 2px;
     -webkit-box-shadow: 0px 1px 3px rgba(0, 0, 0, 0.1);    
     border: 2px solid #cdcdcd;    
}
 
.toggleoff {
     padding-left: 158px;
     background-image: -webkit-linear-gradient(#ffffff, #e7e7e7);
}

I have the benefit of using absolute positioning because I’m building this for an iPhone, but the CSS is pretty straightforward. I’ve got some padding-left properties so I can control how the text looks and where the actual toggle button appears within the div tag. This is why this particular strategy works. When I move between states I’m just going to change the position of the toggle button, I’m not actually doing anything with the select object itself other than changing how it looks and where it is. Here are the “on” classes.

 
.selecton {
     -webkit-border-image: url(../assets/button.down.png) 4 4 4 4 stretch stretch;
     padding-left: 30px;
}
 
.toggleon {
     padding-left: 0px;
     background-image:-webkit-linear-gradient(#bccead, #cbdfba);
}

Just a couple of small changes to gradients and backgrounds so that when our button is on or active, it looks like this:

Toggle button on state

Functionality

Now that the CSS classes are set up I used jQuery to first add/remove the classes I wanted and finally set the value of the select box programatically. As I mentioned, this is using Backbone.js so the code may look a bit odd.

          onMouseDown:function(event){
               event.preventDefault();
               if(this.displayingAll) {
                    $("#activetoggle").removeClass("selectoff");
                    $("#activetoggle").addClass("selecton");
                    $("#toggle").removeClass("toggleoff");
                    $("#toggle").addClass("toggleon");
                    $("#activetoggle").val("active");         
               } else {
                    $("#activetoggle").removeClass("selecton");
                    $("#activetoggle").addClass("selectoff");
                    $("#toggle").removeClass("toggleon");
                    $("#toggle").addClass("toggleoff");    
                    $("#activetoggle").val("all");                             
               }

My Backbone view has a variable, displayingAll, which I set to either true or false depending on what is currently visible. It helps track which classes need to be added/removed. But the first, and probably most important thing, is that I’m capturing the event and keeping it from doing anything. I don’t want the default select behavior at all so I can trap it and make sure it doesn’t keep doing the rest of its work. That means no pop up where the user can select values. Instead I go through and swap out the on/off classes to change the look of my button. Finally I use the val() method to set the value of the select so that what the screen is showing is the actual value of the select input.

So now the slider is working but we haven’t done any animation. So right now the slider just jumps back and forth. It works but it isn’t all that pretty. Luckily CSS3 is pretty awesome and we can leverage the very powerful -webkit-transition to help us out.

Animation

Animation for this example is insanely simple. Using the -webkit-transition CSS property we can listen for changes in the DOM on a specific property and whenever that property changes we can play a specific transition. The way I have set this up is that I’m changing the padding-left property between on/off classes so that’s the one we have to listen for.

Quick sidenote: I struggled with this for a bit. At first I was using the align property of the div to move the toggle button back and forth. That had the benefit of being able to scale better for multiple pixel-widths. The button would move back and forth just fine but there wasn’t a way to animate the align property that I found, which makes sense. So to do animation I had to use a value that could be animated and padding-left ended up being a decent option.

In order to get the animation to trigger we have to add a single CSS property to the #toggle div tag:

     -webkit-transition: padding-left 250ms ease-out;

So whenever the padding-left property changes (on a Webkit browser) a quarter of a second ease-out transition will play and move the toggle button back and forth. Under the hood we’re changing the value of the select so that if we have to rely on that value it’s all in sync while still creating a very specific look-and-feel for our toggle button.

I realize this isn’t entirely helpful without a working demo and I’ll work on getting a jsFiddle or something up to show it but I’m working on some other bits of the project and want to be able to share the whole thing when it’s done. One of the main things I learned: CSS3 transitions aren’t entirely intuitive when coming from the Flash world, but they’re very powerful.

JavaScript: The Good Parts

I was absolutely one of those people who looked at JavaScript as a runty kid brother when I compared it to ActionScript 3. I started off wanting a lot of what ActionScript 3 had in JavaScript. This book completely changed my opinion because it did a fantastic job of laying out how some of the seemingly less-well designed parts of JavaScript end up being pretty powerful. It also highlights some of the areas (like Scope) where JavaScript uses C-like syntax but doesn’t implement it in the same way and may trip up developers. Finally there are a couple of chapters on the bad and awful parts of JavaScript to be aware of.

In a lot of ways it’s the perfect book for someone coming from AS3 to JavaScript. I think it will give you some new found respect for JavaScript and help with some of the parts of JS that seem a little contrary.

The Animation

Building the animation itself was quite simple. I took Victor’s graphics, made them transparent PNGs, and imported them into Edge. At that point it was just a matter of lining everything up and using the timeline to tweak the animation. I didn’t sit down and do the math on the circle, so the elements aren’t aligned perfectly (apologies) but it’s basically just six elements that all pop in from behind the plus button. With Edge it was pretty easy to add the bounce effect. I just set the easing property to easeOutBack and that gave me the bounce.

The project loaded in Adobe Edge

Interaction

While the animation was easy, it took me a bit to get the interaction down correctly. Basically whenever the red button was clicked I wanted to play the timeline animation. When it was clicked again I wanted to reverse it. Luckily Edge has a decent API that makes working with the code it generates pretty straightforward. The biggest issue I ran into was that the playReverse() API can only be run on a symbol and I was having trouble figuring out how to get a symbol instance from within my click event. Running play() on the Composition (kind of the main Edge element, and the one Edge uses by default) runs off an entire set of other operations that made it tougher to do what I wanted.

So instead I used comp.getStage() to get an instance of the “stage” symbol where I could play and reverse the timeline at will. The result is pretty close to what Victor created and what the Path app uses. To do it all, I replaced the Adobe Edge DOM Ready Event Handler code in my project_name_edge.js file with the following:

/**
 * Adobe Edge DOM Ready Event Handler
 */
$(window).ready(function() {
     comp = new Edge.Composition(compId, {stage: "." + compId}, {});
        /**
 * Adobe Edge Timeline Launch
 */
     comp.ready(function() {
         var symb = comp.getStage();
 
         $('#stage_add_button').click(function(e){
               if(played)
               {
                    symb.playReverse();
                    played = false;
               } else {
                    symb.play();
                    played = true;
               }
 
         });
     });
});

Downsides

One downside is that once I make a change in Edge it overwrites all of the changes I’ve made. I’m not sure if there’s a safe place to make JavaScript edits like the one above but I’m going to check with the team. The other downside is that it’s all JavaScript, which makes it kind of heavy and you won’t get the hardware transforms that some browsers use for CSS3. I’m not much of a designer but after this little proof of concept I’m excited to see how Edge could potentially be used to add some fine-tuned interactions like the Path menu.

You can see a working demo here and the source for the project is up on GitHub.

The setup of my pages is as follows. My first page has a list of breweries, and each brewery page has a list of the beers that brewery uses. When you click on any of the beers, it goes to a form that can be used to rate the beers and provide some extra information about them. My goal was to prepopulate some of those forms with the beer data I already had. For instance I know the beer name, brewery, and style of each beer, so I should be able to prepopulate those, but they change for every beer so it has to be dynamic.

Within each page I defined a hidden div with two span elements with ids that represent the brewery name and brewery location (since they’re the same for every beer). Then within the beer list I have a hidden div that contains the beer name and the beer style. Here’s an example:

<div data-role="page" id="aleasylum" data-theme="e" data-add-back-btn="true">
     <div data-role="header">
          <h1>Ale Asylum</h1>
     </div>
     <div style="display:none">
          <span id="brewernameinfo">Ale Asylum</span>
          <span id="brewerlocationinfo">Madison, WI</span>
     </div>    
     <div data-role="content">    
          <ul data-role="listview">
               <li data-role="list-divider">Year Round Beers</li>
               <li>
                    <a href="#beerdetails">
                         <img src="images/hopalicious-thumb.gif" />
                         <h3>Hopalicious</h3>
                         <p>5.8% abv. Eleven separate additions of cascade hops give this American pale ale its lush citrus aroma and bold hop flavor without crazy bitterness. Hopalicious is available year round in six packs and on tap throughout the Madison and Milwaukee regions.</p>
                         <div style="display:none">
                              <span id="beernameinfo">Hopalicious</span>
                              <span id="beerstyleinfo">IPA</span>
                         </div>
                    </a>
               </li>
               <li>
                    <a href="#beerdetails">
                         <img src="images/madtown-nutbrown-thumb.gif" />
                         <h3>Madtown Nutbrown</h3>
                         <p>5.5% abv. Our nutbrown ale is velvety smooth with a rich caramel aroma. We blend seven different malts for just the right touch of sweetness and a creamy finish youÔøΩll really dig. Madtown Nutbrown is available year round in six packs and on tap throughout the Madison and Milwaukee regions.</p>
                         <div id="info" style="display:none">
                              <span id="beernameinfo">Madtown Nutbrown</span>
                              <span id="beerstyle">IPA</span>
                         </div>                        
                    </a>
               </li>

So in order to grab that, I just added an event listener to the pagebeforeshow method that uses selectors to grab the data. One of the critical parts is that jQuery selectors have an optional context property that can be set so that jQuery only selects from that context. In this case, since my pages are all using the same id names for the values, I need to set the context so that the selectors are only pulling from the previous page and not the entire document.

 $('#beerdetails').on('pagebeforeshow',function(e,data){
     var beername = $('.ui-btn-hover-e #beernameinfo',data.prevPage).text();
     var brewername = $('#brewernameinfo',data.prevPage).text();
     var brewerlocation = $('#brewerlocationinfo',data.prevPage).text();
     var beerstyle = $('.ui-btn-hover-e #beerstyleinfo',data.prevPage).text();
     $('#beername').val(beername);
     $('#brewername').val(brewername);
     $('#brewerlocation').val(brewerlocation);
     $('#beerstyle').val(beerstyle);
 });

The code above executes before anything gets displayed on the new page, grabs the values from the first page with the hidden div tags, and then sets some of the form fields to those new values.

It strikes me as a bit odd that there isn’t an easier way to do this, so it could be that I’m just not googling the correct thing and jQuery Mobile has something built-in to help with this issue. It does seem like this can be done by using URL variables, but what I like about this approach is that it’s a bit more semantic and there’s no required parsing of the URL string.

                         $('#beerform').submit(function(e){
                              var beer_obj = new Object();
                              $('#beerform :input').each(function(){
                                   beer_obj[this.name] = this.value;
                              });
                              beers.all(function(arrBeers){
 
                                   var beer_db_obj = {beername:beer_obj.beername,brewername:beer_obj.brewername,brewerlocation:beer_obj.brewerlocation
                         ,beerstyle:beer_obj.beerstyle,quantity:beer_obj.quantity,purchasedate:beer_obj.purchasedate,price:beer_obj.price
                         ,cellardate:beer_obj.cellardate,cellartemp:beer_obj.cellartemp,brewdate:beer_obj.brewdate};
                                   console.log(beer_db_obj);
                                   beers.save({key:arrBeers.length.toString(),value:beer_db_obj});
                              });

There’s kind of a lot of nastiness going on there, but basically I’m just getting all of the elements from the form that has been submitted, putting them into an object and then preparing them for the database and saving them using Lawnchair. The problem was that I was putting it in the same place I put all of my other JavaScript events, as a separate function inside my app.js file. But when I’d load the page and submit the form, nothing happened. It took me a couple of minutes to figure out what was going on.

Since Mustache doesn’t load things into the DOM right away, I had to make sure the above code was only being added after the Mustache template had loaded. By putting the code within the callback function for the original get() method, which retrieves the template with Ajax, and making sure the template had been injected into the DOM, it worked.

               $.get('templates/beer_add.mustache',
                    function(data){
                         template = data;
                         var renderedhtml = Mustache.to_html(template);
                         $('#content').html(renderedhtml);
                         // the template is now in the DOM
                         $('#beerform').submit(function(e){
                              var beer_obj = new Object();
                              $('#beerform :input').each(function(){
                                   beer_obj[this.name] = this.value;
                              });
                              beers.all(function(arrBeers){
 
                                   var beer_db_obj = {beername:beer_obj.beername,brewername:beer_obj.brewername,brewerlocation:beer_obj.brewerlocation
                         ,beerstyle:beer_obj.beerstyle,quantity:beer_obj.quantity,purchasedate:beer_obj.purchasedate,price:beer_obj.price
                         ,cellardate:beer_obj.cellardate,cellartemp:beer_obj.cellartemp,brewdate:beer_obj.brewdate};
                                   console.log(beer_db_obj);
                                   beers.save({key:arrBeers.length.toString(),value:beer_db_obj});
                              });
 
                              console.log(beer_obj);
                    });
 
               });

Fairly obvious once I figured it out, but I’m still having to remind myself of just how the DOM operates so I don’t run into issues like this.