Designing a Data Lifeboat

About the project

You have probably been affected by web services that go dark or disappear, often with little or no warning. Just think about how many services have been removed or “sunsetted” since the web began, and how much of our content and culture has just disappeared. It’s all at risk—though Flickr is certainly not in imminent danger thanks to a world-leading infrastructure—and that’s why the foundation has been set up, to explore this risk, and prepare for the worst. 

Today, the Flickr holds “tens of billions of images” documenting our planet from the first days of photography to just a moment ago. SmugMug + Flickr believes this collection, and photography in general, is an incredibly important visual record of our history, and just like any responsible ship builder, acknowledges that there is a risk—however small— that the ship could go down. Simply admitting this might happen and preparing for it is a form of preservation. We want to shore up our this vast and unique visual commons so it’s still accessible in 100 years.

What if it was possible to make an archival copy of your Flickr presence? Or a collection of your friends’ pictures? Or your taekwondo club? Or, amazing photographs of life during the COVID pandemic, for the record? Not just the pictures, but their metadata, and also their social context.

We call this a Data Lifeboat.

A Data Lifeboat is an archival piece of Flickr, not all of the 50 billion images and their metadata. We envision an archival sliver richer than a mere folder of JPGs: one where you can navigate the content to explore and understand its networked context. Even better, an archival sliver that is updated if things change at flickr.com. (But that’s really hard, so we’re not sure if we can do it yet.)

This year (2024)

We are developing the Data Lifeboat idea initially using smaller and openly-licensed subsets of imagery held within the Flickr Commons. Using this collection as our baseline, we will explore the edges of what’s required to create a Data Lifeboat that’s transportable, buoyant, and robust.

Our 2024 goals are to create several rough prototypes of the software, develop a reasonably detailed understanding of the main technical challenges, prepare a survey of critical ongoing legal issues, and establish a robust design direction for further product development. We are also investigating sustainable software environments and business models.

You can follow our progress on the blog: Data Lifeboat Archives. 

Funding

Our Data Lifeboat has received two grants to date.

1. We received a Digital Humanities Advancement Grant from the National Endowment for the Humanities to support prototyping the software and accompanying policies, with a working group of advisors.
2. We received a Public Knowledge Grant from the Mellon Foundation, to take the prototypes into our community to demonstrate the idea, and move it forward together.