to good effect

Towards the end of the so that it may be remembered blog post I wrote:

Android devices even have public APIs for developers to use host card emulation (HCE) which allows the device to be programmed to act as though it were an NFC tag. This is widely understood to be how Apple Pay works, for example, but it is functionality only available to applications that Apple controls. HCE in a museum context offers a number of interesting opportunities. I probably wouldn't recommend installing a fleet of Android phones in the walls and gallery cases of a museum but, for a unit cost of 100$, there are a number of places where a tiny computer with a screen, an internet connection, sophisiticated graphics and rendering capabilities and the ability to programatically broadcast itself (HCE) would be ripe with possibility.

In both the bring your own pen device and so that it may be remembered blog posts I also wrote:

The phrase "NFC tag" should be understood to mean "any equivalent technology". Everything I've described could be implemented using QR codes and camera-based applications. QR codes introduce their own aesthetic and design challenges given the limited space on museum wall labels but when weighed against the cost and complexity of deploying embedded NFC tags they might be a better choice for some museums.

Take SFO Museum, for example, where most of the wall labels are mounted on slanted plates behind glass. Reading NFC tags can be tricky even when you can touch them. NFC tags that are six to ten centimeters away behind glass and positioned at an angle from that glass are basically invisible to an NFC reader.

After those three blog posts I wrote about working with e-ink displays and Raspberry Pi computers. In that blog post, titled just how easy, I said:

The Raspberry Pi Zero ships without any network interfaces (ethernet or WiFi) and requires that you solder on the connectors for the (e-ink) display HAT to plug in to but for 14$ you can buy a Raspberry Pi Zero HW which comes with a pre-soldered connector and both WiFi and Bluetooth chips.

There is also the Raspberry Pi Zero W which is the same as a the HW but without a pre-soldered connector. It retails for 10$ What I didn't say in that blog post but what I've been thinking about for a little while now is: What if the ID of each object was broadcast as a Bluetooth Low Energy (BLE) advertisement as it was being displayed?

This isn't a new idea. It's what Bluetooth beacons are designed to do. By and large, I have not been a huge fan of either Bluetooth or beacons in a museum setting. Bluetooth is a complicated technology that is made more difficult by the complexities of working with radio frequencies in uncontrolled environments. Bluetooth opens the door to whole universe of security concerns. Bluetooth beacons have, so far, been notoriously expensive and challenging to program, update and maintain.

It is possible to use beacons to good effect but never without a dedicated, costly and concerted long-term effort. Matt Tarr's Location, location, location! The proliferation of indoor positioning and what it means and doesn’t mean for museums paper about the American Museum of Natural History's (AMNH) Explorer application and Shelley Bernstein's The Realities of Installing iBeacon to Scale blog post about the experience at Brooklyn Museum are good discussions of the benefits and the challenges of deploying Bluetooth beacons in a museum.

The cost to develop AMNH's Explorer application, inclusive of the 800 beacons installed in the museum and the native mobile applications that were built to use them, is said to have been about $2.75 million. As a practical matter this makes it an approach far outside the reach of most museums or cultural heritage organizations.

In the just how easy blog post, writing about the use of Raspberry Pi computers, I said:

It is the combination of pluggable (low-cost) hardware and high-level expressive programming languages for operating that hardware and being able to do so in a traditional Unix operating system that makes all of this exciting.

The Raspberry Pi Zero W is a 10$ computer with a 1GHz CPU, 512MB of RAM with the ability to connect to the internet over Wifi, the ability to communicate over Bluetooth and broadcast over Bluetooth Low Energy (BLE) that runs a modern (and updated) Unix operating system with support for most programming languages in use today. The principal drawback, compared to commercial BLE beacons, is that they require a power source which means that what might have begun as a technology problem is also very much an operations and a physical architecture problem.

Being a general purpose computer the Zero doesn't come with any default tooling for configuring itself as a BLE beacon so developing that software becomes an additional burden. Greg Turner's An Internet-of-Things strategy for ACMI blog post about how they are maintinaing a fleet of 800–1000 Raspberry Pi computers that have been installed in the galleries at the Australian Centre for the Moving Image is a good example of how the configuration challenge can be managed.

There are tradeoffs with any technology choice and they are influenced by the circumstances in each institution. As much as I might like to install NFC tags in all the gallery spaces at SFO Museum the choices that were made to meet the demands of exhibiting objects in a busy airport make NFC tags impractical. Could AMNH install 800 Raspberry Pi Zero W computers in discrete locations throughout their galleries in a way that visitors wouldn't see them? Perhaps but it's easy to imagine scenarios where that would be difficult or impossible.

One advantage that standalone and battery powered BLE beacons have is that they can be easily moved to another location. In order to do the same with a Raspberry Pi and institution would need to also design and build a modular approach for running dozens of power sources to different locations both on the floor and in the walls in any given gallery. That's not something most museums can do, easily, right now. This difficulty suggests it is something that any museum looking to do renovations should make a priority.

I haven't been successful in programming a Raspberry Pi Zero to act as a BLE beacon yet. I am confident it's possible and the documentation appears to be meticulous and detailed; the official Bluetooth 4.0 specification is 3,256 pages long, not including subsequent addenda. Perversely, that specificity absent any simple I just want to... cookbook-style examples serves as a barrier for what is still a mornings-and-weekends project.

So far I have been dependent on other people's software implementations of the Bluetooth specification and I have only been able to make one of three packages, Paypal's gatt library written in Go, actually broadcast anything. Unfortunately while gatt is able to broadcast messages that can be seen by an iOS device that device doesn't think the messages contain any data. The gatt codebase hasn't been updated in five years so it is unclear whether the problem is that it doesn't conform to the current Bluetooth specification or something else. Maybe when I've had a chance to read the 400 pages outlining the Low Energy Controller in the documentation I will better understand what's going on.

In the interest of testing my original idea — What if the ID of each object was broadcast as a Bluetooth Low Energy (BLE) advertisement as it was being displayed? — I opted instead to teach the wunderkammer application, first described in the bring your own pen device blog post, not only to scan for BLE advertisements but to broadcast them as well.

Here are some screenshots and a video to demonstrate what that looks like:

There are now two broadcast buttons in the wunderkammer navigation bar. The original antenna.radiowaves.left.and.right icon used to scan NFC tags is now used to start and stop BLE advertising (broadcasting) in the application. The tag-scanning activity is now represented by the badge.plus.radiowaves.right icon which has been flipped horizontally to better signal the idea that scanning happens outside the device. In this screenshot the broadcasting button is disabled because the device's Bluetooth functionality has been turned off.

This is what things look like when Bluetooth is enabled. If a device hasn't enabled Bluetooth or doesn't support scanning NFC tags both buttons would be disabled.

When the broadcasting button is pressed it turns red to signal that it is advertising the object ID that the wunderkammer application is displaying.

The scan button has been updated to present a modal dialog when the device is able to scan for object tags using different technologies.

Here's a video of two iOS devices, each with a copy of the wunderkammer application, where the device on the left is broadcasting BLE advertisements of the object it is showing and the device on the right is scanning for those advertisements and using the data contained in them to show the same object.

As with NFC tags the device on the left is broadcasting an object ID like sfom://id/12345 and the device on the right is resolving that identifier in to a fully qualified URL that it can use to retrieve an object image and metadata from the internet or a local database.

Unlike NFC tags the device on the left updates the data it is broadcasting (the object identifier) when the object it is displaying changes and the device on the right continues to listen for those updates. This feels like a significantly different and novel interaction model, from what's currently available to visitors, that it could be used to good effect in a museum setting.

In both cases, encoding data in NFC tags or broadcasting it as BLE advertisements, there still needs to be an application that consumes the data and does something with it. Development and maintenance costs, as well as the awareness and adoption, of bespoke applications to work with these signals continue to be challenges that the cultural heritage sector struggles with.

Shelley Bernstein's Building is easy, but launching is hard and Sara Devine's Messaging is Harder blog posts about raising awareness, adoption and use of the Ask Brooklyn mobile device at Brooklyn Museum illustrate the problems well. It's generally worth reading anything on the Brooklyn Museum tech blog but especially any post written about the Ask Brooklyn initiative.

There is also Seb Chan and Luke Dearnley's 2012 Museums and the Web (MW) paper Using QR codes, mobile apps and wifi tracking data to understand visitor behaviour in exhibitions about the difficulties, and ultimately failure, of getting visitors to download an application once they've set foot in the museum. So far I have only been able to find a placeholder for the paper on the MW website but not the actual content of the paper itself. Maybe when Seb or Luke have a chance to read this they can point me to the paper or their slides.

I don't have a simple answer to these problems but earlier this year in the Geotagging Photos at SFO Museum, Part 1 – Setting the Stage blog post I wrote:

The larger motivation for doing things this way is to ... address an evergreen subject in the cultural heritage sector: The lack of common software tools, and the challenges of integrating and maintaining those infrastructures, across multiple institutions.

My own feeling is that many past efforts have failed because they tried to do too much. It’s a bit of a simplification but a helpful way to think about these tools is to imagine they consist of three steps:

• Data comes from somewhere. For example, an image comes out of a database and an asset management system.
• Something happens to that data. For example, an image is geotagged.
• The data then goes somewhere else. For example, the geotagging data goes in to a database.

The problem with many tools, developed by and for the sector, has been that they spend a lot of time and effort to abstract the first and the last points and ultimately fail. The nuances, details and limitations, not to mention the vast inequalities, in all the different technical infrastructures within the cultural heritage sector are legion. Developing an abstraction layer for the retrieval and publishing of cultural heritage materials that attempts to integrate and interface directly with an institution’s technical scaffolding is going to be a challenge at best and a fool’s errand at worst.

The cultural heritage sector needs as many ... small, focused, tools as it can produce. It needs them in the long-term to finally reach the goal of a common infrastructure that can be employed sector-wide. It needs them in the short-term to develop the skill and the practice required to make those tools successful. We need to learn how to scope the purpose of and our expectations of any single tool so that we can be generous of, and learn from, the inevitable missteps and false starts that will occur along the way.

I mention that for two reasons.

The first is to argue that confusing the value of making the objects in our collections addressable, by publishing stable and permanent identifiers be they on wall labels, NFC tags or BLE advertisements, with the difficulties of building, maintaining and promoting a museum's own application of that data is a mistake. To me the purpose of broadcasting an object's identifier, of announcing its presence on and participation with the internet, is to foster as many applications of that data as possible.

That does not, and should not, preclude an institution from having its own application and editorial viewpoint around these objects but nor should the success or failure of those endeavours be the measure by which the value of making our collections accessible in the first place is gauged. The effort to do these things is, I believe, worth it on its merits alone. The day-to-day work becomes ensuring that these efforts are done in a way, both technically and financially, that they outlast people's initial reluctance to embrace them.

The second reason is to solicit help in building, documenting and distributing the simplest and dumbest tool to make broadcasting an object identifier as a BLE advertisement on a Raspberry Pi computer possible. Version 1 could be as simple as a command-line tool that only accepts a single parameter, the object identifier to broadcast. For example:

$> ble-tool sfom://id/6789

It could be as complicated as:

$> ble-tool -service-id {CUSTOM_BLE_SERVICE} -characteristic-id {CUSTOM_BLE_CHARACTERISTIC} sfom://id/6789

The Bluetooth specification is over 3,000 pages long and outlines a symphony of functionality and possible uses none of which are necessary for this tool. Version 2 of my imagined tool should allow the object identifier to be updated dynamically, to stream notifications to a consuming device like I demonstrated in the video, but after that I don't think it needs to do anything else. By design.

This is what I mean when I talk about small, focused tools. The details of why or where object identifiers might be broadcast as BLE notifications will vary and those details should be left to individual institutions. The commonality in those applications is the act of broadcasting and those are the tools and tasks that we should make as simple and inexpensive as possible.

It should not be a requirement to have to use a vendor specific device in order to do these things. I built this functionality in to the wunderkammer application because Apple made it easier to get things working than doing the same on a Raspberry Pi. In the end it wasn't even easy but it was still easier than the alternative. As a sector we should endeavour to fix that disparity. There are good reasons why deploying iOS devices in to a production setting makes sense but we will all benefit from being able to do the same with a 10$ Raspberry Pi computer.

I will keep working on this problem because I want it to integrate it with the wunderkammer-server tool I mentioned at the end of the just how easy blog post but I would welcome some help. In the meantime I have merged all the Bluetooth-specific code in to the main branch of the ios-wunderkammer application if anyone is curious to see how it can be done in iOS devices.