This is the story I want to remember

see also: py-wsclustr, testing py-wsclustr and selflesh's photostream

This is the story I want to remember

Untitled Intimacies

A few years ago, the now defunct oreillynet.com website ran a series of articles called something like your ~/bin directory. The idea was to get people to write about the quick and dirty scripts they wrote for themselves to solve the variety of problems and hurdles they encountered every day. They often weren't very elegant but they worked — they just fucking did it — at least for that person. It was a lovely series because it was an especially confortable window on which to sit and see how a person solved a problem and, just as importantly, which issues they simply chose to side-step.

Everything is easier when you're just writing for yourself, right?

I was mentioning all this to Seth one day, during a meeting when I'd showed him the Untitled Intimacies Twitter-posts-on-a-map pictures I'd been creating and uploading to Flickr. These are selected Twitter messages that have been geotagged and plotted on a stylized map with a big honking pinwin containing a cropped image of Twitter's own big honking pinwin, generated using a rasterized version of the site's HTML. I could have also used the API to draw the text of a post but I like the idea of including the actual markup specific to a time and in place both the user and site's history.

Layers of context.

You can draw a pretty straight line from the early Net::Flickr::Geo maps to the pinwins and then from history box to history box to history box to these and then back again to delicious maps and, carrying on the grand tradition of one name stupider than the next, bucket maps. There's a lot to talk about in all of that; I'm pretty sure that's at least one end to the thread with...no end, that I mentioned above. These days, it mostly just comes out in the form of drunken exhaustion, angry ranting and borderline character assasinations.

But that's a story for another day.

What I was talking about, that day, was trying to do a your ~/bin directory style post about the code that generates the Twitter maps because I figured someone else might want to geotag their Twitter posts and I thought it would be nice to share. In that way that you want to share where you don't want any of the burden of maintaining the code or making it work anywhere but your own setup.

Enter GitHub.

I come slowly to revision control systems in part because they go in and out of fashion at roughly the same rate as boy-boy and girly-girl bands. I believe that some, like Git, can have real advantages over others or have been tailored to fit a specific class of projects but they are not the ones I work on. My needs are pretty pedestrian and while there may be perfectly good reasons that git commit does both exactly what it sounds like and not at all what you'd normally expect, given the semantics of all the version control systems that came before it, the disconnect has always made me a little wary of making the effort.

There's also the part where it seems like every single introductory HOWTO on the subject of Git fails the Hello World test by requiring what feels like four times as many steps to do the most basic of things, like this:

	cvs checkout 
       	cd ; touch bar
       	cvs add bar
       	cvs commit -m ur mum uses svn bar

Git feels like the Java of version control which is fine since Java's good at a lot of things but generally seems to come at the cost of making simple things a nuisance. But hey, it also seems ideal for your ~/bin directory style projects! That's what Paul Hammond did when he released his MiniMuni webapp. He just slapped it up on GitHub and said:

As the about page says, if you live exactly 6 minutes from Sunset Tunnel East Portal, 8 minutes from Duboce and Church, and 10 minutes from Church Station you may find it useful too.

Within a day, two or three people had found it useful enough to fork and to tweak to work within whatever (n) minute walk from the MUNI they lived near. I like this. I like this because it's fast and cheap and no one asked Paul to make his code any more complicated than it needed it to be for him or fall in to the rabbit hole of abstraction; first another MUNI stop, then another transportation system, then the moon.

Simon Wistow is not wrong in his takedown of the Gitastic habit of forking first and asking questions later but I do think there is a place for the sort of rapid cloning of simple projects described above and GitHub seems to make it easier than most.

Which is what I chose to hear Seth say to me that day, even if he didn't. So, here it is:

http://github.com/straup/untitled-intimacies/

It's mostly written in Perl and will require you to install stuff using the scary CPAN. It currently only works on a Mac because it uses webkit2png and requires PyObjC. It shells out to Python. Twice. If you have a custom background you will need to patch the crop_tweet.py script which has about as much grace as a land mine. Depending on how your Mac is set up it shells out to Python twice, to two different versions of Python; if you can get PyObjC to build out of MacPorts, more power to you. It effectively shells out three times if you count the call to the ModestMaps ws-pinwin server running on localhost (and which is not started automatically).

Actually, if your Twitter stream is not public it will probably work on a (not a Mac) because the part where you could pass your (Twitter) credentials in a GET request suddenly stopped working the other day and webkit2png, as it is written, doesn't read from a cookies file so I modified the code to work with a screen grab.

	# Usage (simple)
                            
	$> perl ./map_post.pl -c your.cfg -l '45.123,-37.939' -u http://twitter.com/you/status/1234

        # Use a screenshot instead of webkit2png (usually, because Twitter is doing
        # weird auth-y stuff they won't explain to anyone...)

        $> perl ./map_post.pl -c your.cfg -l '45.123,-37.939' -u /path/to/screenshot

I'd like to think that it would be as easy to fix as fetching a Twitter page using curl -c and rendering it as an image using webkit2png on a local file. But given how completely hosed Twitter's persistent cookies have been in the last month a more likely scenario will involve using WWW::Mechanize and pretending to actually log in to the site itself before writing an HTML file for the Twitter message in question to disk.

Update. Or like this which hasn't yet been plugged in to the rest of the code : http://github.com/.../fetch_tweet.pl. I love Perl.

But otherwise it just works. I have pictures, and everything, to prove it.

There are a lot of moving parts. Most are things that I use, or prefer to use, for a variety of related tasks: Python for maps and images, Perl for Flickr, Java for barcodes and so on. The important part in that is: preferred. I briefly considered rewriting map_post.pl in Python so that it could load the ModestMaps code natively but then realized that I had already written an HTTP wrapper in Perl, I would have to rewrite all the Flickr code in Python and mostly I just wanted something that worked now and didn't really care about building a shiny tower of idealized beautiful code. I can live with that.

All in all, I've probably forgotten some of grunt work in setting things up but, then again, these days almost everything installs out of one package manager or another so it's mostly a question of typing install x over and over while you read the paper for a little while.

Welcome to my ~/bin directory.

No training for trouble-shooting

There's also a PDF version available here, for safe-keeping.