Wellcome Collection is a free museum and library. Our goal is to foster great ideas by connecting medicine, health and art through our public programmes, library collections, research development and online experiences.
We have been digitising our library collections for at least 20 years. In 2010 we secured an internal funding stream that transformed our digitisation prospects, allowing us to increase the scale of digitisation, build a digital library system, and create an online experience far beyond what we had before. Our goal is to provide unrestricted online access to collections and content, focusing on content that will provide greatest impact to our users, or requires urgent digital preservation.
We currently digitise and make freely available 3 to 4 million images per year from our books, pamphlets and journals holdings, over 300,000 images per year from our archives, and tens of thousands of images per year from our manuscripts and visual culture collections. We also digitise and preserve audiovisual materials, and we acquire and preserve born-digital archives and published works.
Our digitisation work, funded separately to our operational budgeting process, was carried out largely as a bolt-on to the operational working of the museum and library, even as it was becoming increasingly relevant to our long-term strategy to provide access and inspirational experiences to our users. We were committed to preserving and delivering our digital media in perpetuity, our management systems and technical capabilities have evolved to accommodate the increase in our digital estate, and we didn’t want to stop digitising our material at some arbitrary point when the pot of funding ran out. In short, we needed to fully operationalise the whole endeavour.
Embedding digitisation into our operational structure
In 2015 Wellcome Collection was restructured, and the Digital Engagement Department was born. This department has 5 key responsibilities:
- Develop and maintain our digital preservation and access infrastructure
- Manage our data and create innovate solutions for search and discovery
- Digitise our collections
- Create compelling online editorial content, including high quality on-brand imagery
- Provide user-centric design through extensive user testing and research
These goals have become recognised as central to Wellcome Collection’s overall strategy and integral to its success as a modern museum and library.
We had to acknowledge and prepare for an increase in our operational costs to support this activity long-term. We worked hard to streamline our processes, sharpen our focus, and identify areas where we could reduce our costs and make our work more efficient.
Resourcing and team structure
Although technology and systems greatly help us optimise our workflows, our success in managing large-scale digitisation is really down to the teams that we have built, and the way we treat our digitisation efforts as “pipelines” rather than specific projects. A pipeline consists of an ongoing standardised workflow that accommodates the needs of similar physical formats. This continuity helps us make the most efficient use of staff availability, equipment set-ups, contractual relationships, etc.
We subscribe to the “self-organising” principle of team management. Within a framework of collaborate objective-setting on a quarterly basis and clarity on boundaries, teams can experiment with different approaches to how they work.
As an example, the Digitisation Support Team, a small team of 4 to 5 staff members who carry out nearly all the digitisation scheduling and preparation of physical materials across all our workflows, has experimented with a regular rotation of responsibilities. They’ve tested monthly, bi-monthly and mixed-length rotation, depending on how fast individuals want to shift their focus to something new. This has worked well, but they are now ready to try a quarterly rotation that fits in more closely with the quarterly plan. This regular refresh has improved resilience. The team members are all able to backfill each other if needed, they get some variety in pace and nature of the work improving motivation, and with their different perspectives they bring different improvements and enhancements to the work as they move around. At the same time, they are focussed on tasks long enough to see real progress and embed any new learning.
We use contractors for the larger-scale projects, benefitting from economies of scale and a general lack of inflation in digitisation costs over the past several years. It is a buyer’s market when it comes to procuring digitisation services. Digitisation management of all strategic digitisation streams, and our Digitisation Service, is largely centralised within a single team, including contract management, scheduling, and the end-to-end workflow. This team works closely with conservation, photographers, and collections staff who can provide specialist support.
We implemented Quickbase a few years ago, an online subscription database that can be easily customised by the team, for the team. This program allows us to collect information on and track sample surveys, assessment activities, retrievals, locations and movement of items, and batching of deliveries without resorting to countless Excel spreadsheets. This has greatly improved project management and team efficiency, allowing us to increase the number of projects we can handle at any given time. We can store, capture and analyse data that helps us to work through large numbers of items in the most efficient ways. Members of other teams involved in supporting digitisation can access the data too, so there is transparency across all teams involved in the digitisation effort.
Figure 1. Screenshot of Quickbase digitisation dashboard
Systems development and automation
We brought software development and product management expertise in-house, greatly reducing our overall costs. We can now make wider use of open source software, saving on annual support costs which can run well into the tens of thousands. We have been able to start reducing our reliance on external developers, bringing in consultants only for specialised support.
We’ve automated our ingest workflows extensively using Goobi, an open source workflow system specifically designed for digitisation, to reduce unnecessary manual labour and improve workflow management. We used to require at least 2 full-time staff to ingest digital content at a far lower rate than today, with up to 4 FTEs regularly required to handle peak periods. Now, we can handle a much-increased rate of ingest with less than 1.5 members of staff even during the most busy periods.
Scalability of storage and systems has always been an issue for us, and as part of a re-think of our infrastructure strategy, we are making moves to alleviate this bottleneck and save on costs by moving our digital assets and systems from local servers to the cloud. We are two years into this move, and on the cusp of working entirely in the cloud – anticipated in late 2019.
We use Amazon Web Services (AWS) as our primary cloud storage provider. This allows us to meet spikes in storage and computation requirements in a very cost-effective way. Other major benefits include a standards-based approach to file fixity (BagIt), development of fully open source Storage Services, replication to multiple locations (supplier and geographical), ease of processing and re-processing content (format conversion, etc.), and increased capacity to extract additional information about our assets.
One practical benefit of this is that we can scale up our audiovisual digitisation workflow without the painfully slow and error-prone file movement to, from, and within our local storage. We were severely limited in our ability to cope with quantity or quality, and it impacted our choice of master file format. Once we are fully on the cloud, our off-site vendor can upload large master files directly to S3 (our AWS storage area) where they can be processed, converted to derivative formats as needed using MediaConvert or other online tools, ingested to the preservation system, and disseminated, all without downloading to local servers. We are now able to digitise our unique film materials to 2K, and processing and ingest speed is only limited by our budget for scanning services and staff resources to support the digitisation.
Conclusion
To sum up, we have realised major benefits from transforming our ways of working, triggered largely by the scale of our digitisation ambitions. Key takeaways include:
- Bringing core activities in house, outsourcing to specialists as necessary
- Transition to cloud infrastructure and off-site storage
- Automation and continuous improvement
- Integration of open source software
- Dedicated, cross-functional teams empowered to self-organise and act
- Flexible pipelines of work, versus project-based planning (a pull vs. push model)
Christy Henshaw, Digital Production Manager, Wellcome Collection
