We have just hit a bit of an issue with AppCache whilst we were deploying a new version of a client site. It’s not really a bug, but more a lack of clarity in the current documentation and different implementations in the browsers.

It has taken me sometime to understand how the NETWORK: section of the AppCache actually works. In the end, I had to build a series of AppCache tests to figure it out.

The story

We setup the NETWORK: section of the AppCache to point at a rest API

NETWORK:
//example.com/api/*

# This code is an incorrect use of AppCache

Breaking AppCache

As we deployed our site we wanted to run the API that is on another domain under SSL. So we changed the URL in the NETWORK: section so it started with https and added the certificate to the site. i.e.

NETWORK:
https//example.com/api/*

# This code is an incorrect use of AppCache

At this point Chrome stopped making API requests, we were only initially testing with Chrome.

First mistake – putting the * wildcard in the wrong place

Our first mistake is that we wrongly added the * wildcard to the end of the URL. Each entry in the NETWORK: section can be one of three types. These entries are usually added as a new line directly under the NETWORK: section header. The entry types are:

• * wildcard on its own
• relative or absolute URL
• URL “Prefix match”

Examples of the correct use of AppCache NETWORK: section

NETWORK:
/data/api/
https//example.com/api/
*

“Prefix match” URL

A “Prefix match” is a strange concept – it’s a URL that is used as a “starting with” matching pattern. If your API has many endpoints but they all live in the path http://example.com/api/ then that’s all you need to add to the NETWORK: section. The * wildcard can only be used on its own and means any URL.

Second mistake – URLs should have the same protocol and domain as the manifest

There are other rules that effect the use of URLs in the NETWORK: section. All URLs have to use the same protocol scheme i.e. http or https and be from the same domain as the manifest.

Browser implementations of these rules do differ, Firefox is strict and insists on the same domain, where as other browsers only insist on the same protocol scheme. See the test examples I have built to demonstrate this.

In effect, that means to get good support across the major browsers you can only use URLs in the NETWORK: section if they are to the same domain as the manifest.

The fix is to use the * wildcard and not URLs

The vast majority of sites sidestep the complexities of URLs, by just applying the * wildcard on its own i.e.

NETWORK:
*

This will work with the manifest on one scheme (http) and the API on another (https). The wildcard does not have the same rules as URLs.

You have to ask why the hell the authors of the specification added all this complexity if all that happens is that everyone applies the * wildcard.

Thanks to Jake Archibald for some pointers to the answers as I waded my way through this.

I went to the Responsive Day Out event this Friday. Rather than write about each speaker I want to try and pull out the major themes of the day.

There were a number of good practical take aways from Richard Rutter, Josh Emerson and Andy Hume talks on web fonts. Josh’s talk had a couple of neat ways of progressively enhancing content with resolution independent icons stored as a web font. I particularly liked the use of a data attribute in the HTML and content attribute in the CSS in his code examples

// CCS
[data-icon]:before{
   content: attr(data-icon);
   font-family: 'Cleardings';
   speak: none;
}

// HTML
<a data-icon="&#xe012;" href="http://twitter.com/clearleft">
@clearleft
</a>
});

I also thought the use of ligatures to allow the replacement of words in your text with single icons was good. Take a look at the forecast.is example site Josh put together to illustrate the approaches he talked about.

Part of Andy’s talk on “The Anatomy of Responsive Page Load” covered the method the Guardian mobile site is using to load and cache its custom web fonts. It is a form of progressive enhancement, where only browsers that pass the following tests display the custom font:

• Cuts the mustard (BBC phrase for modern standards support)
• Supports WOFF
• Supports localStorage

The server then sends a base64-encoded font so it can be cached client-side in localStorage for reuse on further page requests.

Patterns of navigation

David Bushell took an in depth look at UI patterns use for navigation. He divided the common patterns found in current responsive design into five :

• Minor break points (gloople.com)
• Multiple tiers (microsoft.com)
• Overlay (sony.com)
• Go off-canvas (dbushell.com)
• Jump to “big footer” (iso.org)

His talk finished with a useful list of considerations you should be focusing on when making responsive design choices about navigation.

For me one of the strongest points David made was all the ways someone can now interact with navigation i.e. mouse, keyboard, touchpad, touchscreen, stylus, voice, movement, remote and games-pads. True device independence is not just about screen size, a point massively reinforced in Anna Debenham’s talk. Her love of game consoles always heartens me and is a good antidote to those who consider the web a web-kit monoculture. Take a look at her console site its a great resource.

Progressive enhancement

Both Andy Hume and Tom Maslen have been involved in building large-scale responsive sites for the Guardian and BBC respectively. Their talks both focused around the practical use of progressive enhancement. Tom laid out the much talked about “Cutting the Mustard” concept from the BBC i.e. dividing user agents by modern functionality support.

The BBC’s core experience of HTML4 is delivered to all browsers, but an enhanced JavaScript experience is loaded onto any browser that supports the following:

• querySelector
• localStorage
• addEventListener

I was impressed with the script loading optimisation the Guardian is using. Somehow the new async and defer properties for a script tag had passed me by.

<script src="app.js" async defer></script>

This allows a script to load directly after the HTML at the same time the CSS is loading. They fallback to use appendChild(script) on browsers without support for these new properties. This support detection has to be done server-side.

Media queries (the future or not)

More than one speaker said that they expected their use of CSS media queries to reduce as we move through the next couple of years. I got the feeling that these comments are a mixture of a response to both an over fixation on this area of CSS, but also a belief that other elements of CSS will play a much larger part in creating fluid layouts in the future.

So it was interesting that Bruce Lawson’s talk about future standards development centered so heavily around new media queries that can target device types like touch and remote. This felt like a mixed message.

Things that where un-said

There are two topics I would really liked to have heard about. These are the two difficult subjects of display advertising and web apps vs responsive design

I thought the approach my friend Jeremy Keith took in sidelining the subjects and the people asking about them was not as productive as maybe getting speakers to hit the subject full on. To be fair these issues were most likely out of the scope of this event, but they will be the anchor around responsive design’s neck and they deserve a honest straight-up engagement.

Death of Photoshop and winging it

Sarah Parmenter started the day by being honest and saying she often feels she is just winging it while creating responsive web designs. Her comfortable and well-homed design processes of the past had been lost in the move to responsive design.

A few of the speakers also made reference to the fall from favor of the Photoshop centered design workflow. With some contempt being levelled at the idea of the ‘deliverable’, a Photoshop layout given to a client as if it were the end point in its own right.

I think both points are rooted in issues of client communication.

Photoshop has not been removed from our toolkit, just its output can no longer be the crutch by which we dumb down the way we communicate complex design problems to our clients.

This new world of thousands of device formats and usage contexts means we have to draw clients more fully into the design process with all the subtle and complex trade offs involved in resolving a responsive design.

I am sure Sarah is not winging it; just like most of us feeling the uncomfortable uncertainty that always comes with change.

Thanks

The event was a thought provoking day of responsive web design. Even if I have not mentioned all the speakers, they all did a great job. Thanks to Jeremy and Clearleft for putting on the event. The day was split into groups of 3 speakers all doing a 20 minutes slot with a small joint Q&A session together. A good format I hope they will use again.

Speaker’s slides

• David Bushell: Responsive Navigation
• Josh Emerson: Asset Fonts
• Andy Hume: The Anatomy of a Responsive Page Load
• Paul Lloyd: The Edge of the Web

Speaker’s notes

• Elliot Jay Stocks: RWD — The War Has Not Yet Been Won

Audio recordings of talks

• Sarah Parmenter: The Responsive Workflow
• David Bushell: Responsive Navigation
• Tom Maslen: Cutting the Mustard
• Jeremy chatting with Sarah, David, and Tom
• Richard Rutter: Responsive Web Fonts
• Josh Emerson: Asset Fonts
• Laura Kalbag: Design Systems
• Elliot Jay Stocks: RWD — The War Has Not Yet Been Won
• Jeremy chatting with Richard, Josh, Laura, and Elliot
• Anna Debenham: Playing with Game Console Browsers
• Andy Hume: The Anatomy of a Responsive Page Load
• Bruce Lawson: What’€™s Next in StandardsLand
• Jeremy chatting with Anna, Andy, and Bruce
• Owen Gregory: Antiphonal Geometry
• Paul Lloyd: The Edge of the Web
• Mark Boulton: In Between
• Jeremy chatting with Owen, Paul, and Mark

Write ups by other people:

• Orde Saunders: Liveblogging Responsive Conf
• Morgan Jones: Notes from Responsive Conf
• Dan Davies: Beefcheeks in Brighton
• Responsive Day Out in Brighton
• Lee Griffin: Responsive Day Out
• Ashley Nolan: Responsively winging it together
• James Young: – Responsive Day Out – food for thought
• David Bushell: A Responsive Day Out
• Jeremy Kieth: Oh, what a Responsive Day Out that was!

Demo API
http://microformat2-node.jit.su/

Try the API with:
http://the-pastry-box-project.net/
http://microformats.org/

New life in the semantic web

A lot has changed in the last couple of years; the search engines have started to use semantic mark-up to improve their listings. Google’s rich snippets feature has created a secret army of SEO people who are quietly marking up big parts of the web with semantic data. Not that there are not already billions of microformats on the web.

HTML5 created a third standard of semantic web mark-up to add to the mix of RDFa and microformats. Then the search engines clubbed together and brought us schema.org. After a few catfights between the standards supporter clubs, these events have brought us a small rebirth of the semantic web.

microformats 2

The microformats community has revisited it’s standard and come up with “microformats version 2”. At first, I thought why! I don’t like change unless it gives me something worthwhile. After reviewing the work, I think the wholesale change to a new version of microformats is worthwhile because:

1. The authoring patterns have been simplified even more and they are based on real life use cases e.g.: <a class="h-card" href="http://glennJones.net">Glenn Jones<a/> is a valid microformat




2. microformats 2 addresses one of the biggest problems in maintaining microformats in real sites. The class names are now prefixed i.e. class=”fn” is now class=”p-name”. The prefixes like “h-*” and “p-*” tells you a class is a microformat property and helps make sure classes are not moved or deleted by mistake




3. Like microdata, microformats 2 now has a full specification for a JSON API. This is important as it means the parsers should now have the same output and also browsers could implement this API.

New parser and test suite

Most importantly for me I wanted to help move forward microformats, I liked the fact that the API design aligned microformats 2 a little closer to microdata and RDFa. So I have invested a couple of months of my time to building a brand new JavaScript parsing engine and a comprehensive test suite.

microformat-node GitHub https://github.com/glennjones/microformat-node
Test Suite GitHub https://github.com/microformats/tests

I have just open sourced a new version of microformat-node using this parsing engine. Soon I will create a browser compatible version of the engine and update the microformat-shiv library which powers browser plug-ins etc.

The test suite took a long time to develop, but should provide an excellent starting point for anyone else who wants to develop a new microformats 2 parser. I know Barnaby Walters has already started working with it for his php-mf2 parser.

I hope this work will help microformats and the concept of the semantic web move forward another little step. Enjoy

Links to the high-res versions (.mov, .mp4 or .ogv) of the video

Hackdays are funny, sometimes everything comes together, other times small things just trip you up. Unfortunately Saturday was a day when nothing seemed to work for me, mainly because I made the mistake of loading the wrong version of a USB driver.

On the upside that day I bumped into Aral getting coffee at Taylor St who pointed me to a project he did using Kinect with processing.

So after a couple of hours of hacking after the event, I could wave my hands and fly a drone. Once I had the right USB driver installed.

Thanks to everyone who made the day possible, I had a great time. I am glad with others, Madgex was able to sponsor the event.

You can download the project from github kinect-drone, enjoy

Features of new version

• About 8x faster at parsing
• Will now load onto Windows based hosting solutions correctly
• Inbuilt cache system, which can be customised
• Upgraded logging and trace options
• Upgraded unit test system
• Added support for JavaScript promise

Example: http://microformat-node.jit.su/
Code: https://github.com/glennjones/microformat-node

I have changed how the method calls work so if you are using the last version you may have to update your code. The parse methods parseHtml and parseUrl now follow the standard pattern used in node.js and return an error and data object rather than just the data object.

with URL

var microformat = require("microformat-node");

microformat.parseUrl('http://glennjones.net/about', function(err, data){
    // do something with data
});

or with raw html

var microformat = require('microformat-node');

var html = '<p class="vcard"><a class="fn url" href="http://glennjones.net">Glenn Jones</a></p>';
microformat.parseHtml(html, function(err, data){
    // do something with data
});

using a promise

var microformat = require("microformat-node");

microformat.parseUrl('http://glennjones.net/about').then( function(err, data){
    // do something with data
}));

Source code: https://github.com/glennjones/microformat-node
Test server : http://microformat-node.jit.su

Install

npm install microformat-node

or

git clone http://github.com/glennjones/microformat-node.git
cd microformat-node
npm link

Use

with URL

var shiv = require("microformat-node");

shiv.parseUrl('http://glennjones.net/about', {}, function(data){
    // do something with data
});

or with raw html

var shiv = require('microformat-node');

var html = '<p class="vcard"><a class="fn url" href="http://glennjones.net">Glenn Jones</a></p>';
shiv.parseHtml(html, {}, function(data){
    // do something with data
});

with URL for a single format

var shiv = require("microformat-node");

shiv.parseUrl('http://glennjones.net/about', {'format': 'XFN'}, function(data){
    // do something with data
});

Supported formats

Currently microformat-node supports the following formats:
hCard, XFN, hReview, hCalendar, hAtom, hResume, geo, adr and tag. Its important to use the right case when specifying the format query string parameter.

Response

This will return JSON. This is example of two geo microformats found in a page.

{
    "microformats": {
        "geo": [{
            "latitude": 37.77,
            "longitude": -122.41
        }, {
            "latitude": 37.77,
            "longitude": -122.41
        }]
    },
    "parser-information": {
        "name": "Microformat Shiv",
        "version": "0.2.4",
        "page-title": "geo 1 - extracting singular and paired values test",
        "time": "-140ms",
        "page-http-status": 200,
        "page-url": "http://ufxtract.com/testsuite/geo/geo1.htm"
    }
}

Querying demo server

Start the server binary:

$ bin/microformat-node

Then visit the server URL

http://localhost:8888/

Using the server API

You need to provide the url of the web page and the format(s) you wish to parse as a single value or a comma delimited list:

GET http://localhost:8888/?url=http%3A%2F%2Fufxtract.com%2Ftestsuite%2Fhcard%2Fhcard1.htm&format=hCard

You can also use the hash # fragment element of a url to target only part of a HTML page. The hash is used to target the HTML element with the same id.

Viewing the unit tests

The module inculdes a page which runs the ufxtract microfomats unit test suite.

http://localhost:8888/unit-tests/

Notes for Windows install

microformat-node using a module called ‘jsdom’ which in turn uses ‘contextify’ that requires native code build.

There are a couple of things you normally need to do to compile node code on Windows.

1. Install python 2.6 or 2.7, as the build scripts use it
2. Run npm inside a Visual Studio shell, so for me, Start->Programs->Microsoft Visual Studio 2010->Visual Studio Tools->Visual Studio Command Prompt

If you have the standard release of node it will probably be x86 rather than x64, for x64 there is a different Visual Studio shell but usally in same place.

This has left me in the lucky position of being able to follow my passions. I want to try to take the product research and design I have done commercially for years and mix it with my interests in open web and standards development.

In the next few months I want to research and build projects in a number of areas:

• Web Intents/Activities
• Personnel data stores using services like Dropbox
• Possibility of semantic data reuse
• Mobile web apps – the right way

I am sure within a few months I will want to work with teams again as I always want to design and build products that impact people’s lives, that usually means lots of calibrations, but at first I want some time to open my mind to new people and ideas.

No sitting on beaches for me. Watch this space I am about to turn up the volume…

I missed Javier Usobiaga talk on Responsive Web Design, which is a shame, but managed to go to the session by Ibon Tolosana on CocoonJS. It takes HTML5 canvas based games and boosts their performance by using an OpenGL ES execution environment. Through the day a recurring theme seemed to be that the current performance of Canvas on phones is just under what the gaming community are looking for. Although like petrol-heads I feel they will always want just a few extra frames per a second. The best example of this was Miguel Ángel Pastor’s talk on cross compiling JavaScript from C++. Not an approach I would take, but interesting.

I hope people found my talk “Beyond the page” on API’s of interest. The Web Intents demo did not work, although I am in good company as Paul Kinlan had the same trouble when he visited Barcelona. Maybe adding that one last demo at 1am the night before was not a good idea! Next time I am going to have backup screencasts.

The PeopleStore HTML5 app I showed on the day is unfortunately not online, but the codebits area of my site does have demos/code of some of the API’s it uses.

• Building a virtual file and creating a download link
• Creating a virtual file that can be dragged off the browser
• Copy and paste data between pages
• Custom protocol handlers

Sorry the video does not show the demo’s there is an earlier screencast of some of these demo’s from a previous blog post.

I would like to thank Ideateca for inviting me to speak and putting on such a good event, the live translation services and high quality video which was streamed live added to the events success.

“Dear Google, Fuck you. Signed, the people who actually use your APIs”

The API provides two main features the first of which lists our distributed identities across the web. So if I give it the URL of my Twitter profile it returns a list of profiles I have on other sites. I wrote an in depth List Apart article about how this feature works. The second feature tries to find links to profiles of people you have listed as friends/followers on social media sites.

Let’s be pragmatic about its true value

Brad Fitzpatrick built this API as a Google 20% project and it has never really lost its experimental roots. From the outset I was not a fan of the social graph friends listing. It’s too problematic, a lot of friend’s data is private and it’s too complex to mark-up and extract from web pages well. I personally wrote off using the social graph element of the API from the beginning. Evan Prodromou also made a good point that developers want to get authentication and social graph data together. I think lanyrd.com is a good example of this approach.

The identity aggregation element of the API was impressive if not a little too raw to be use on commercial sites. The results needed a degree of post processing to increase the quality. Although I would love to say increasing the quality of the results could be completely done by parsing open standards like hCard or FOAF, you do need to connect to some sites proprietary APIs to get profile data.

Google never tackled the quality issue or put the API on a commercial footing, both of which help stop most people using it beyond experimental hacks.

Panda, schema.org and identity based authority in search

In the last couple of years I have lost my faith a little in the ideas which gave birth to the SGAPI. Development of the semantic web and distributed open web, which seem to have drifted with the growth of monolithic services like Facebook, but things are changing.

Google’s recent changes to search have breathed new life into the semantic web concept. As Google tries to increase its search quality it is moving towards identifying entities (the blocks of structured information within a page). The SEO industry is now adding vast amounts of semantic mark-up to the web. This is being done not because it is the right thing to do, but because of the enhanced click through rates providing the right commercial motivation.

More importantly part of Panda’s new ethos is the promotion of identity authority. We can see this, both in Google’s search listing that displays recommendations from your friends and authorship profiles. They are attempting to link people to other entities such as articles by using mark-up like rel=me and rel=author. Profiles and how they are interlinked is a small part of the Panda concept but still important.

The future – Getting the food chain right

Today’s web apps are often more about building ecosystems of service relationships than technology. These relationships are often chained together and always need each actor to be rewarded for their part.

Web authors or at least the SEO wing of our industry are now seeing a real return for adding semantic structures for entities such as products and reviews. The mark-up of people and organisations entities still have rogue claim issues and still may not become a strong part of Googles search listing. Let’s hope Google continues to support rel=me for identity authority and it drives them to resolve these issues. At the moment them seems to be moving towards a wall garden approach.

Unfortunately, although Google may have the ability now to build a much better API based on its latest developments parsing semantic information for search, it unlikely it will be created. Google is now bringing together its services into a coherent whole and focusing on building its own monolithic social network Google+. It just does not make commercial sense for it to support the open web without financial return.

Other companies have started to provide successful services in this area. Qwerly.com by Max Niederhofer was one of the most impressive identity aggregation APIs I have seen, it’s now part of fliptop.com. Products like rapportive.com and thesocialcv.com use the same technique under the hood. These companies are providing the next generation pay-as-you-go APIs blending together the semantic web and snowflake APIs.

Let’s hope we see on-going development of this new generation of APIs.

So on 20th April I will have a drink and say goodbye to SGAPI

I would like to thank Brad for giving us the SGAPI and everyone else who has worked on it. Although I can understand the commercial rationale that has driven Google to murder my friend, I am not sure I can forgive them for it.

So, for the past couple of months I have been working with two of Brighton’s leading UX people Danny Hope and Andy Dennis to organise a UX design event. We are trying a slightly new format that we’ve coined a “design push”. The idea is to take a current technology or topic and focus a group of UX designers on a day of open collaboration with the aim of positively adding to its traction by the wider community.

Web Intents Design Push, 25 February 2012 – Brighton, UK

The first “design push” will be on Web Intents, an idea about how to standardise buttons such as Tweet, Like, Share, etc. on the web. My last few posts will give you some background. I have also created a brief screencast introduction to Web Intents.

I had some fun creating the screencast, I built a presentation using HTML for the first time. It’s worth a play as I built the demo’s of Web Intents directly into the presentation. Best use Chrome if you want to play with the demo’s.