this is aaronland

the year of inodes

the calm before the SQUEEEEEEE

Warning: This blog post may have more typos and grammatical errors than usual. That's because I've been SKYBLOGGING it and as I write this I can see the sun rising out the window on the other side of the continent I started this journey from. I'm going to hit the publish button shortly errors be damned and clean them up when I've gotten a little bit of sleep.

Last weekend I had the pleasure of attending the fifth annual State of the Map conference, in Denver, and the privilege of speaking on the last day of the event. The title of my talk was Using OpenStreetMap As a Divining Rod although there was a typo in the schedule so at first glance everyone thought I was talking about how OSM relates to diving rods.

As usual, these are not the verbatim notes that I wrote or used for the talk. It is a smushing together of what I wrote before the talk, what I actually said and what I wanted (or meant) to say after the fact. I don't know if that's bad form but it's my party so there you go. Also, I'm pretty sure Myles would do the same thing...

This is what I said:

Good morning. I know many of you were out late last night so I'd first like to thank you all for choosing me over your hangovers.

I am a founding member and co-Director of Revolutionary Technologies for an organization called the Spinny Bar Historical Society. The Society was born here in Denver, in 2010! Although the Society has deep roots here in the city it is still not the sort of thing that you will hear about in the tour guides or most cultural histories of the city so I'd like to start changing that.

The Society traces its origins to Indianapolis, during the 2009 Museums and the Web conference. Not only did we discover the wonders of the spinny bar at the top of the Hyatt Regency Hotel but we were fortunate enough to both see the "switch" that controls the bar and also receive a guided tour of its inner workings.

The Society was not officially founded until the following year during the 2010 Museums and the Web conference when we met at the bar in the Hotel VQ, next to Mile High Stadium, here in Denver. I should warn you, however, that the bar at the top of the hotel is neither spinny nor entirely round having been halved some years back to make room for a kitchen.

That night we formed a non-profit organization dedicated to the promotion and preservation of spinny (or rotating) bars and restaurants all over the world and one day we hope to have a spinny bar classified as a UNESCO World Heritage site.

There are some members of the Wikipedia community who might have you believe that we are a joke but I assure you we are not.

Before I came here to State of the Map I made sure to update the tagging data for the Hotel VQ to accurately represent its true nature. Since then I've been told that the proper, or correct, tagging nomenclature is "motion=static" but I have chosen the more familiar "spinny=no".

I will come back to buildings in a little while.

The rest of the time, I work at Stamen Design in San Francisco. As the name suggests we are design and technology studio and we do a wide variety of work, with a range of clients, visualizing different kinds of data but at the end of the day most of us are map nerds at heart.

We love everything about maps and we love OpenStreetMap and everything it represents.

Recently, we finished our first iPad application: A proper 3D spinny globe (I kid you not) for the National Geographic Society that can display all of the amazing maps, physical and political, that they've produced over the years.

After its release the author Warren Ellis wrote a blog post complimenting our work with the comment that it was nice to see the studio working on maps that actually work as maps.

I will be honest and tell you that I was irked the first time I read that because I imagined that the remark was a sideways jab about most of the work I’m going to show you today.

Since then I've decided it's all good. It's a big world and we don't have to pretend to like everything about it. I've come to appreciate Warren's comments for the brevity and succinctness with which they capture some of the debate around the challenges and the opportunities of creating maps in 2011.

Parts of that debate are hardly new at all, principally the one that asks whether something can be considered a map and how its utility is measured. J.B. Post's essay Ruminations on the Borderlands of Cartography is a worthwhile read on the subject.

This image and the text in this slide are taken from a project, and essay, by James Bridle called Rorschmap. We've already got our hands full today without diving in to the territory James is exploring but I would recommend taking the time to read it.

(That link on the screen is to Post's essay.)

Meanwhile, the world of GIS has been on a 250-year tear in pursuit of relentless precision. This is understandable when you remember that “way-finding” used to mean being guided by little more than the stars. It is doubly understandable when you consider the scope the OSM's project goals to create a map for the entire planet, from scratch.

Before we go much further I want to make one thing clear: I am not here to suggest to anyone that they stop improving the map or that there isn't enormous value in the work that's already been done. The work that has been accomplished so far, whether it is the map itself, the community or the tools that have been built to support both are nothing short of amazing.

But I am here to suggest that as the map grows and improves one of the most exciting aspects of that process is the way that it makes possible entirely new and unexpected creations. The map itself becomes the seeds for a wildflower garden of entirely new maps.

I don’t know about you but I do not want to live in a world where the best thing we can accomplish are driving directions. We can and should take pride in the skill and craft required to make that happen but like I said it's a big world and it has more to offer.

If, for example, we held language to the same standard of absolute utility that we some times hold maps to then we would not have literature, we would not have humour, we would not have philosophy.

We've heard a few people talk about tagging for the renderer this weekend. For those who don't know the expression refers to the practice of deliberately tagging a node or way with incorrect metadata so that the map renderer will draw something that it otherwise wouldn’t.

Patrick, from MapQuest, talked earlier this weekend about how TomTom community map users would deliberately tag the speed limit of the roads around their houses with a value of 0 so that they wouldn’t be included in driving directions.

The most famous example is when the large hadron collider at CERN was tagged as a road so that it would appear on the map. This is actually listed on the OSM wiki as a kind of example of what not to do, but it's hard not to think it's being talked about in a way that's really saying: Don't do this unless you can manage to do something better than this, in which case it's awesome!

I am actually heartened by both of these examples. Not because I think that OSM necessarily benefits from people gaming the system but because it means that OSM – the community, the tools and the map – are mature enough and robust enough that someone would even think that its worth bending it in a new way to see what happens. That's usually a good sign.

There's an expression that I've always enjoyed which likens building successful community projects to that of "building small tools for self-organization". That is, you assume that you don't know how something will be used and that, equally, those unknowns will be the very things that breath life in to your project.

Lately, I've been making an argument that we should start to think about how we interpret data the same way that people design patterns for textiles and work with it the way they might approach a bolt of fabric to fashion any number of different objects – from a bag to a dress to a wall-hanging – out of it.

So what would it mean to think of the map – and in particular all the pieces of data that make up the OSM – as a bolt of fabric instead of a finished product?

Okay, on to the pictures.

This isn’t even OpenStreetMap data. It’s actually an incomplete rendering of the Flickr alpha shapes dataset. If you don't know about the Flickr alpha shapes they are the shapes of places (neighbourhoods, cities and so on) whose contours are generated using nothing but the geotagged photos of those places, on Flickr.

I’m showing you this picture because it was, in many ways, the spark for the rest of the work I’m going to show.

The short version is that, one day, I needed a dataset to test a browser-based polygon renderer and knew that we had a database with the Flickr alpha shapes. It turned out that the database was missing any flag that I could use to filter by place type (a state, a county, etc.) but since it was a just a test I threw caution to wind and decided to see what would come out the other end if I loaded all the shapes at once.

The reason the image is incomplete is that the renderer blew its brains out trying to draw that many shapes but not before producing this image. I had never seen anything like this before and it was a beautiful and fascinating way to consider both the raw data itself but also the geographies that they represent.

So I started pulling the thread and this happened.

This is a proper slippy map of all those shapes rendered all together between zoom levels 2 and 10. This is zoom level 2. It's easy to get caught up in the very dense areas of coverage but one of the things I love, in particular, are the outlines of Northern Canada and of Russia.

Neither one reflects an accurate physical geography; their contours are generated according to the edges of where Flickr users have taken (geotagged) photos. But that's actually a useful way to understand how people (in this case photographers who share their work on Flickr) actually interact with the world around them.

This is the same map zoomed in to the UK and France. You can easily pick out both London and Paris simply by the clustering of neighbourhoods. Part of the reason that I am so interested in these shapes is that before joining Stamen I worked at Flickr (for a thousand years) and had a hand in almost anything to do with geotagged photos, including these shapes.

One of the most frustrating things about working on geo and photos back in 2006 and 2007 is that we were limited to a single map that had to serve as not only the means by which people geotagged their photos (the driving directions, so to speak) but also the context in which that photo was displayed.

Is this the map that we should have used to display geotagged photos? Probably not but we should have had something other than a telephone book of street names and highway shields.

Here's a variation on the same idea but this time limited only to shapes for neighbourhoods and cities, from Flickr. The large orange shapes are "urban areas" as defined by the Natural Earth project. The areas are derived by post-processing satellite imagery and tracking clusterings of built or man-made environments, like cities and suburbs and the industrial-commercial netherzones that flow between them.

This is not a map that you'd want to use to bomb your enemies but I would argue that it is a not an inaccurate representation of the US East Coast.

So, that's what was going on when this happened while we doing some client work around the idea of street intersections using OSM data for New York City.

There is a beauty and a mystery in an image like this. In seeing the shadow a city casts when it is seen through the prism of a single facet, in this case streets. But also by the density of its coverage as a telling of the history of a city's relationship with OSM.

This particular image was just a rough sketch, something to work through the conceptual and technical issues of a problem but you can probably see where this is headed.

In August of last year, the studio released prettymaps which was, and is, very much an experimental map and a sort of synthesis of everything I've talked about so far.

The maps are made up of three data sources: The pink are blue-ish shapes are Flickr cities and neighbourhoods, respectively; The orange shapes are the urban areas from Natural Earth; The roads and foot paths and green spaces are from Open Street Map!

This is Beijing.

I should note that prettymaps was rendered using a very old copy of the planet (OSM) so that if these were rendered today many of these images would be more detailed.

Hong Kong.

These maps are a bit weird, it's true. I probably wouldn't give one to somebody who was visiting from out of town but like the earlier map of the Eastern United States they are not altogether inaccurate representations, spatially or conceptually.

One of the nicest thing anyone has said about prettymaps, I think, is that although they don't really make any sense at first glance the instant you see a prettymap of the city you live in everything snaps it to place.

Washington D.C. and Baltimore.

One of the things to note about these maps is that they don't follow the traditional convention of scaling or pruning data sources as the map zooms in or out. In particular, highways are drawn as the sort of big friendly puppy dog features that always demand your attention not unlike highways in real life.

As you zoom all the way out the roads cluster together to create these intense hot spots on the map, a kind of cluster-based analysis without any of the fancy math but just sheer numbers and a constant presence.

This is somewhere around zoom level 3 and you can see how the highways in North America and the urban areas in Western dominate their respective landscapes.

If you look carefully you can also see the ghosts of an almost transparent outline of Canada rendered in white. This is the same shape I was talking about before.

While we were making these maps the project had a cheeky code name which was "Isola", after the Finnish textiles designer Maija Isola. While I was reading about her life I also discovered that the name was used as the setting for a series of books, by the author Ed McBain, that everyone understood to be New York City.

I love this quote from a reviewer who says that McBain's Isola exists not simply as a reflection of New York but as a fully formed place of its own.

It's a useful prism to think about maps through.

More recently, the author China Mieville published a novel titled The City and the City. Ostensibly a detective story the book's setting is a physical city that has been divided in half not just politically but through a deliberate and enforced "unsee-ing" of one another.

That is two people, each from separate cities, might be standing next to each other physically but would not only refuse to acknowledge one another but also actively *disbelieve* that the other was even present.

We're starting to stray actively in to the territory of human geography so I'm just going to toss that story out there in the hopes that it will come back to haunt you, even just as a rhetorical exercise, when you are thinking about this presentation.

prettymaps does not contain any residential streets. This was a deliberate decision because there are so many of them and I wanted to leave space, literally and figuratively, for prettymaps to be used in other contexts.

The comic book writer Scott McCloud talks about the value of the "gutter" in comix, the space between the drawn panels, because it allows each reader to fill in the action in a story with their own narrative.

We don't often do that with maps and this is a not particularly successful attempt that I made last year trying to fill in the gaps in the city (Montreal) with two weeks worth of GPS traces taken as I went about my daily life on bicycles.

I think what I enjoy the most about this map is the way St. Helen’s Island, over there on the left, hangs off the Jaques Cartier bridge like an anatomical heart.

Here's another version of the same idea, also in Montreal.

This is a (Mapnik) map style that the studio released earlier this year as part of the Dotspotting project, funded by the Knight News Grant. It is called "toner" and was designed by Geraldine Sarmiento.

The "toner" map tiles were produced specifically to act as a better neutral ground for displaying a variety of heterogeneous datasources than any of the big map providers offer, today.

I don't think it's any surprise that so many Google Maps applications use the terrain style. It is an elegant design and most importantly it doesn't jump up and down demanding the viewer's attention the way that conventional road maps do.

I know that pubs are considered sacred in OSM but, honestly, their inclusion was the last possible thing we wanted for our maps.

All of the Mapnik goodness required to create "toner" maps has been published on Github under a GPL license.

And it's not just for cities. This is the south of England rendered using "toner".

As you zoom out of the cities the distinction between a country's road networks and its rivers and streams become increasingly difficult to discern. I think that's an interesting and telling view on the country.

Moscow.

Mostly just because it's so beautiful but also because it's a good example of the ripple-effect cast by the many ring-roads that surround a city.

Baghdad.

The architect LeCorbusier used the phrase "the voice from above" when writing about the earliest airplanes flown in around Paris at the turn of the 20th century.

It was a prescient expression given how many of us spend as much time as we do in airplanes looking down on the landscape. Maps have always been a voice from above that allow us to see "through" a terrain to which we are literally attached. And they still are but I don't think it's crazy to suggest that prolonged time spent in the sky looking down has given us a whole new appreciation of the physical world and land features that can only be seen from above.

Why wouldn't we expect to see that sensibility start to make its way in to our maps?

This is somewhere in Houston.

Instead of the Tigris river winding its way through Baghdad these are motorways that cut through, and the on/off ramps that connect them to, the city.

Different but the same.

On/off ramps were originally included in prettymaps because of the way they mimicked the cartilage in x-ray images. But then you zoom all the way in and you can see how they create this amazing lattice work of intent and relationships.

Or put another way: a giant bag of thread to pull on.

This is a screenshot from a project called Airport City, a slippy map made up of nothing but ways in OSM tagged "aeroway=*" or "trunk=*"

(The airport codes were pulled in and merged separately.)

This is the area around Newark airport in New Jersey and although Airport City is not an entirely successful project I think that has more to do with the execution than the idea itself.

The map tiles themselves are transparent which means that we can set and update the background colour along a gradient range as the user zooms in and out.

One of the reasons I don't think the project was as good as it could have been is that the colour palette I chose was too uniform which make it difficult to distinguish the different elements in the map.

(Unless, like me, you've spent hours staring at the map and have trained yourself to recognize the subtle differences.)

This is the Airport City map zoomed out to the continental United States.

Remember, there are only two features being drawn in this map: airports and on/off ramps.

Rome.

As you zoom in the background colour shifts from a pale-grey to meat-y blue. The hope was the colour pairings would cause the elements to play and hide-and-seek with the viewer's eye as the background changed depending on the zoom level.

Aside from the choice of palette it might have been more successful had I rendered the map down to zoom level 14 but zoom level 13 is already where baking (pre-rendering) map tiles start to be painful and did I mention that the world is a big place?

Anyway, I like to look at the airport runways, rendered in isolation, and imagine them lumbering across the landscape like tankers or cargo ships at sea. Or as new pieces in a very large and very slow-moving chess game that is still looking for its arc.

Speaking of airports, here's a photo my friend Josh took of the Terminal 5 building at New York's JFK airport.

I am on the advisory board for a project called the Built Works Registry which aims to create an authoritative database of buildings for art historians and scholars. The reason I was asked to participate is that they are also interested in understanding how the registry might be opened up in such a way that the general public and amateur enthusiasts might contribute.

One of the things that I think it's important for any project like the BWR to be cogniscent of is that there are already a number of parallel registries out there.

The largest these days is foursquare with its reported 15 million venue listings. Many of those listings won't be of interest to a BWR (whether or not they should be is an entirely other question) but to ignore projects like foursquare and the role they play in shaping people's understanding of space and place would be short-sighted.

When the board held its first meeting I made a point of adding the building where we were assembled to OSM. I added four nodes, created a way and tagged it "building=yes".

This got me wondering: How many ways are there in OSM tagged "building=yes" ? It turns out that as of late January 2011 there are 26 million of them!

So I made a website and gave each building its own page.

Although the vast majority of those buildings often don't have any metadata associated with them they all have accurate geographic footprints.

And those that do have metadata are often full of delightful surprises like the London Eye which is tagged "attraction=big_wheel". Tagging for the renderer is bad, right?

Originally I had wanted to use the "toner" map tiles I showed you earlier for this project but at the time we hadn't finished instrumenting a tile-serving setup for use with other projects.

Instead I used aerial tiles from [A VERY BIG COMPANY] that have been dithered, a kind of half-toning process for digital images. I've used this technique for other projects and it produces a really lovely ground, or base, map for framing other features without overwhelming them.

This is the airport in San Francisco and the reason the building at the tip of passenger complex, Terminal 2, is not highlighted with the others is not because it hasn't been added to OSM but because this is a search results page (for buildings at SFO) and I am paginating the results. Skip to page 2 and that building will be highlighted but none of the others will be.

Which got me wondering whether a more suitable base map for this project would have been one that rendered all of the 26 million buildings on top of these dithered tiles or even a flat neutral background.

That way it might have been clearer that even though a building wasn't contained by a particular grouping or selection that it was still part of larger whole.

I'm going to come back to this idea, shortly.

You may have heard Jason Wilson talk, on the first day of the conference, about a project that the studio did for MapQuest this earlier this year.

It is called "map=yes" and during the early, exploratory days of the project we went down the rabbit-hole of trying to aggregate all of the data contained by the area in a single map tile and then representing it visually in that tile's pixel space.

Because the studio is located in San Francisco we tend to use the West Coast of the United States as the patient zero for all of our projects.

The problem with our idea was simply that at low and mid zoom levels most of the tiles contain at least a little bit of land which means they become candidates for drawing data on top of.

This has the unwanted side-effect of obliterating the actual coastline and with it any sense of place.

So, we sub-divided each map tile into a number of cells to try and even things out, a little. But by increasing the resolution of the tiles we also increased our levels of uncertainty surrounding the data we were trying to visualize.

If a given sub-cell was rendered without any data was that because there was actually no data in that quadrant (itself a potentially useful fact) or was it because the area was nothing but water?

To account for this we pre-filtered each cell by checking its bounding box against the OSM coastline to first ask whether it intersected any land features.

Internally, we started referring to these as "boolean tiles" and they weren't originally meant to be anything more than an odd-shaped widget in our toolbox.

And then one morning while I was thinking about the boolean tiles I wondered what would happen if you applied the same principle to other aspects of OSM.

For example, what if you made boolean tiles where the filter was highways instead of the coastline? Sort of like this.

It is an idea that owes at the very least a debt of inspiration to Brett Camper's "8-bit Cities" project which itself is built using OSM data. It is worth checking out.

What if instead of highways you filtered on buildings? What if instead of buildings you filtered on anything you could add as a tag to OSM?

Aside from all the other great work that MapQuest has done to support the OSM community one of the things they've done is commit to hosting a publicly accessible instance of the OSM extended API (or XAPI) which provides both a geographic and tag-based query interface to OSM.

That's actually super exciting because there's a treasure trove of data in all the tags that have been added to OSM and until now there's not been an easy way for people who don't live and breathe the OSM toolchain to access them.

The black squares in this screenshot are parts of San Francisco where there's at least one way in OSM with a tag whose key is "building".

Aside from its aesthetic qualities it's actually a pretty useful device for any organization, like MapQuest, that's trying to foster participation in OSM and encouraging people to start updating the map.

Using "boolean tiles" you could create a website that displays reasonably sized chunks of a city (for example) that are missing a particular type of feature, let's say buildings, that users could claim and go out and map with a Walking Papers style guide.

The "boolean tiles" are still very slow to render. Each map tile is cut in to (n) to the power of two tiles and then the XAPI endpoint is queried for each cell. There's a lot of room for improvement to the way we've done things. The boolean tiles are, as the name implies, boolean: A tile with a single building with be true (filled in) whether it has a single building in its bounding box or a hundred.

So we decided to see what things would look like if we actually drew the buildings too. That's how the "footprint" tiles, and from them the map=yes project, were born.

For map=yes we focused principally the "footprint" tiles and the ability to query for arbitrary tags using the MapQuest XAPI endpoint. We produced a series of slippy maps for 20 cities between zoom levels 10 and 15 that consisted solely of ways tagged "building", "leisure" or "trunk".

For the most part all "footprint" tiles are drawn as closed polygons filled with a single colour but you can also configure the code to draw (OSM) ways as line strings.

Here's an example. All those spaghetti lines on the top and left hand sides are, like prettymaps and airport city, on/off ramps. They don't look like this in "real" life except that unless you're driving a car they kind of do. When you're out walking around a city on foot most people's experience of highways exchanges is pretty much like this: They're big and consume an awful lot of the space around them and they are one big mess of directionality.

map=yes is a statically rendered map site but all of the code, including both the boolean and footprint tile providers, used to produce the tiles is open source and available on the website and, shortly, Github.

This is Pleasure Bay in South Boston. It is full of, or at least buffered by, "leisure".

map=yes is a bit of a love song for the English comic book artist Dave McKean and in particular for a graphic novel he illustrated (and authored) during the 1990s called Cages. With few exceptions the only colours in Cages are the pale slate blues that shadow some of the most beautiful pen and ink line work I've ever seen. It is a wonderful example of the old adage about doing more with less.

Like airport city we revisited the idea of a dynamic background colour ranging from black to grey (dubbed "sunrise"), grey to black (dubbed "sunset") and entirely black ("midnight").

In the first two the highway exchanges are drawn using the same colour as the background at its most zoomed out which means they don't appear until you begin to zoom in.

Occasionally, during all of this I've asked myself: Is this really any different than Google's "map styling" tools? Is this all just window dressing?

I think the answer is "no".

It's true that with Google's maps you can achieve some impressive results by limiting the appearance of map features and by adjusting their visual properties.

But you can only choose to exclude items from a pre-defined list and you can't actually add any new ones.

This is the strength of OSM.

The tagging database is not something that's gotten a lot of attention until relatively recently but combined with tools like XAPI and a (generally) consensus driven tagging process that can take a joke it is possible not simply to imagine new kinds of maps but to mechanize them and make it possible to create them on an properly global scale.

We're still at the possible-not-easy stage of things but that's okay. It's never been this easy and it will only get easier as people begin to experiment and develop and adapt the tools to meet their needs.

I'm going to come crashing back down to Earth just long enough to talk about a project that was released this week by my colleagues Mike Migurski, Nate Kelso and Geraldine Sarmiento.

It is a Mapnik stylesheet called "High Road" and is meant to, in its own words, "be the last OpenStreetMap road query you will ever need" for rendering highways at different zoom levels. See the way the highway is raised and is placed over all the other roads except where it is suddenly superseded by a series of overpasses? That's because the highway is actually beneath street level in this part of Oakland and "High Road" takes care of all the head scratching required to draw this accurately.

I want to end this talk with a project like this because I want to repeat something I said at the beginning: That the diligence with which people add features to the map and the care they take in tending them and seriousness with which they craft a finished product is hugely important.

It is that work, ultimately, makes everything I've shown you today possible.

But we shouldn't forget that the world is full of delight like the wiggly chaos that guides buses in and out of New York City's Port Authority terminal.

Maps are not a binary proposition and if they ever were they needn't be anymore.

We have spent the last 250 years chasing down the dragons that previously haunted all the maps we made before we discovered the super-powers of community and computers and the network. But it's not as though we'll ever run out of unknown territories to map so what does it mean to use these new tools we've fashioned to look for new dragons?


Some closing thoughts (that have nothing to do with my talk at State of the Map):

In October I will be attending NACIS for the first time and holy-shit-excited to be participating on a panel about art and cartography organized by.Tim Wallace.

The title of this blog post refers to a Twitter message I sent out, while I was in New York in July, just before I left my hotel to head uptown to the MoMA for the opening of the Talk To Me exhibition which includes a giant wall of prettymaps. There's even a postcard, too!

There are a lot of things one could say about this kind of good fortune but, believe it or not, I am often at a loss for words and usually just sum it up with SQUEEEEEE!!!!!