For a while now I’ve been thinking about the fact that we spend so much time adding content to hosted services like Facebook, Twitter, Flickr et al, without thinking much about what could happen to this data. If the demise of Magnolia and Geocities has taught us anything, it’s that hosted services don’t last forever, and so, unless we are careful, so does all our data in these services.

There has been some high profile debate about this lately (Zeldman vs Tantek for example, and Jeremey Keith’s links section), about whether all this data needs to be saved, or whether the nature of it all means it should be disposable.

Now, I’m not suggesting for one second that my tweets or Instagram photos are very interesting to anyone but me, and even less so when they are years out of date, but I’d like to have the choice of whether that data is available to at least me, in the future.

There is a very real possibility that some of the services we use every day just now will disappear in the next couple of years. Yahoo clearly don’t want to bother with Delicious any more, and what if no one buys it? It could go the way of Geocities. Instragram is a great service, and I’m not suggesting for one minute it will disappear tomorrow, but do we really know what sort of strategy they have for dealing with problems of scale, or a massive server failure? Do you have all the Instagram photos you’ve taken, either on your phone or computer? Would the world suffer as a result of all Instagram photos disappearing? No. Would it be a real shame? Absolutely. Is it our duty to try and archive all this content for future generations, or our future selves, to decide what’s important? Definitely. As many people said when Geocities was switched off, websites and web content are now part of our culture. as with culture of any era, that needs to be archived. Yes, there is much, much more content than there used to be, but now we have the tools necessary to filter that information effectively.

There seems to be an opportunity for someone to create a self hosted app for the (techy) masses that allows us to post conetent to our own site, which in turn is posted to Twitter, Delicious, Flickr et al, like an advanced version of Tantek’s Falcon. This would allow us to own our content, and have the copies on our social networks, for the social benefits and interaction. Yet, if these services disappear or go offline for a time, we still have all our content online and easily searchable. Also, importantly, the interaction possible with this content on the relevant service, would also be possible from the original on our own site (retweets, likes, comment etc).

That is the ideal (for me anyway) but for the moment it is not a reality. I’ve been experimenting with a few options for archiving my data (for me, not really for anyone else). Tweet Nest is a self hosted app which archives your tweets and favourites from Twitter. It’s simple to install and does a good job as a ‘set up and forget’ app. You can search your tweets which can be very handy for finding that link you posted last year that you didn’t bookmark for yourself. ThinkUp is another self hosted app by some very smart people, which also archives your tweets as well as Facebook activity. You can see graphs of your follower count over time if you really want (this seems to be aimed more at businesses and organisations) and other handy things. I’ve been running both of these service for now, to see which one is easier to maintain and is more handy.

I’ve also set up a WordPress install with the FeedWordpress plugin. FeedWordpress allows you to add an RSS feed as a source, and new items in the feed will be automatically added as posts in WordPress. So now every new piece of content I post of any social network (I need to remember to add the feed obviously) is also posted as a post in a section of my personal site. It’s an archive of all my public activity online, whether the services the content came from are online or not. I can filter, search, see what I was doing on a certain day or week from a while back. It’s useful (for me) and it’s nice to sometimes look back at a kind of diary of my (online) life. It’s like Momento app but web based and self hosted. For now It’s crude, it uses a default WordPress theme and the language and format from the different feeds doesn’t gel very well. I’ll be working on cleaning that up soon. It was more important to start archiving data now and worry about presentation later. With this system I have MySQL database that’s backed up by my host daily, and by me weekly. It’s in a format that is exportable to another type of software if necessary in the future.

This system won’t work for everyone, and it’s certainly not syncing data the optimum way. Creating originals on my site and copies on social networks would be more effective, but my developer skills don’t stretch to anything approaching that, sadly. It’s a start, and it’s got me thinking more and more owning and hosting our own data, and backing things up.

I’m a geek. I think about things like archiving and backups, both online and on my devices. But we are the Facebook generation where an awful lot of people don’t think about this stuff. They don’t think about the fact that those photos they post could disappear, and if they haven’t moved them to a computer or a new phone, they will disappear forever.

I’d love to hear whether you think all this fuss is justified, and also if you have any better systems in place for your own data archiving. Hit me up on Twitter if you do.


Steve Wozniak’s open letter to the FCC

SoundCloud Icon Download

Add a Twitter @anywhere hovercard to any text or image

Moving On