How do I get started as a small archive in terms of technology?

This paraphrases a series of questions that we’ve been asked a few times over the past year, and this really came to light in last year’s Australian Society of Archives conference where I was involved in a few workshops around archival software, standards and just ways forward.  One of the things I wrote about back then was that I’ll do an outline for small archives, and this is part of that approach.

If you’re reading this outside of Australia, please be aware this is written with a quite parochial Australian approach – so if you like, imagine me wearing a broad-brimmed hat with corks tied around it to keep the flies off!

It’s not all about size…

Firstly, it’s pretty hard to come up with a definition that suits for “small” but here I’m thinking about just not the size of the collection (or the building you’re in), but more about the resources that you have available to you.

The size of the archive (physically, and digitally) does play a role but it’s not actually that important (at least from a technology standpoint) for what you have to do.  If you have 10,000 records or 1,000,000 records, that’s not too big a deal.  However, digital storage does have an impact – storage can be expensive when you consider it needs to also be backed up and replicated for maximum redundancy.

So, ask yourself a few questions:

• Do you have your own servers and/or archival management or digital preservation software?
• Do you have support for your archival software? and
• Do you have a role defined in the orgnanisation that has at least some time dedicated to archival management and digital preservation?

If you answered No to one or more of these things, you’re probably a small archive…

Roadmap

Before you set off in a car for a long drive – and this could well be a long drive depending on where you’re heading – then you need to check that the vehicle is ready for that trip.  You would check the tyre pressure, oil, water, fuel, appropriate sugary treats in the centre console, road trip music, and then, appropriately fuelled with caffeine, you head off.

The first step is to look at what the health of the technology is around your archive.  For smaller groups, we do this relatively quickly with a conversation and some email exchanges, just to see what you’ve done already.  Often, we just need to go around and kick the tyres, but usually we’ll ask you to show us what you’ve done through a quick call and a screen share or two.

Once we’re sure of where we’re starting from, then we define the destination, and we do that through a workshop which spells out the destination and the way to get there.  I don’t like doing these workshops remotely, so I prefer to get people together in the same room and work through these sorts of issues and see where everyone wants to go.

Some of the questions I’ve asked include:

• How busy are you?
• Have you got anything already in place?
• Have you tried anything new yet?
• Are you hoping to open up the history and stories in your archives to external parties/the public or to be an internal archive?
• Do you have much digital ‘stuff’ to add to the archive?

These questions are trying to elicit more backstory and context than what we are often provided in emails and requests for quotes and proposals.  For example, sometimes we get someone coming to say “I need software package X” but what they actually have done is heard someone mention software package X and think that it’s a silver bullet – which it won’t be.

There aren’t any silver bullets here, sorry.

We’ve done a few of these roadmapping workshops with our Collections clients (some of our projects are listed on our projects page) and while it can be challenging to find the budget to do this for smaller organisations, I still think well worth the investment.

Archival Management

Usually, it’s the Archival Management side of things that kicks off first, because you want people to know what is in the Archive as a starting point.  There’s one critical part to decide on here, before we even start looking at technology, and that’s which data standard you are going to use.  If you are in Australia, and you haven’t already, pick up this publication:

Describing Archives in Context: A Guide to Australasian Practice (Australian Society of Archivists Committee on Descriptive Standards) 

This is what I’ve been using as my guide to “how things are done” in Australia, Archives wise.  It’s something I’ve been referring to regularly in our work with Archives, and the concepts are pretty important to understand.  I’ve seen a few situations where this standard isn’t followed in implementation or has been misinterpreted, and the headaches this leads to.  If you are unclear about this standard, find an Archivist who isn’t – look to your local State Archive or the Australian Society of Archivists – and get to grips with this.

Next up – how do we implement this standard and start describing our archive?

Here’s the thing – the Australian Series System isn’t something supported natively by most of the software I’ll discuss in the next section.  And if you wonder why, it’s because this wonderful standard has been developed for a small part of the world, which is a very small part of the global Archival market.  Groups like us are working towards including it into the open source packages, but that will take time and funding, although some of the planned standards like Records in Context are more inline with the Australian Series System.

There’s a whole raft of ways to implement an Archival Management System, but I’d take on board the various options at their merits, depending on where you are at in your organisational roadmap.

• Got nothing digital at all? Try Microsoft Excel to start with. Set up a single file that is structured to meet the Australian Series System and start documenting your collection.  The pedant in me really doesn’t like this option, but it does mean you can get started on capturing what’s in the archive, learning about how the Standard should be implemented (and what data you need to start collating to be compliant), and you can do it at very little cost .
• Got some sort of database, like an Access database, or one of the Archival “systems” built on that? Perhaps you’re intending to move as you can’t find support, or your own internal developer has moved on to something else, or left. Then perhaps it’s time to move into something like AtoM, or ArchivesSpace**.  While they don’t quite meet the Australian Series System out of the box, they can approximate it if you think hard about it (like we’ve helped the State Records of Western Australia to do with AtoM).
• Got your own Archival Management System already? Then why are you here? If it’s heading to its end of life (10 years is a pretty good run for a piece of software), then perhaps it’s time for a refresh? Then AtoM or ArchivesSpace might well be the choice for you – although a few other Australian organisations are also looking at CollectiveAccess.  Anyway, there’s a few choices out there that can meet your requirements, and in our experience AtoM and ArchivesSpace are both decent Archival Management System to look at, and that could be where many of the small Archive road maps end up at (you might go through the steps above first, but end up here).

Let me wax lyrical about open source here as well, because some of you might be thinking that’s all I’m looking at.  And you’d be right, for some very good reasons.  Open source is still seen as a “bad thing” in some areas of the Archival community – have a read of our previous blog I wrote addressing some of the things we commonly hear, but here’s some reminders;

The first is budget – commercial licensing comes at a cost that is an ongoing licence for using the software and getting updates.  For the open source archiving and digital preservation software I’m thinking about, this licence cost is zero, but that’s not to say the whole thing is free – you pay organisations like us to support you, to help you implement it, to keep it up to date, and you can sponsor the development of new functionality.  Effectively, everything we do for you is under a service based engagement.

The second is that open source and Archives go together better than piping hot buttered toast and Vegemite (if you’re not Australian, insert something locally appropriate).  Open source means that the source code is available to all.  If you are thinking about “forever” then this has to be an advantage – you’re not tied to a vendor (and look at us – ten years ago we didn’t do any work in Archives at all!), and you can get in there and amend the code base as you need to.

Thirdly; as we’re often used to seeing now, you can get started right now just with your own resources (possibly).  Download the open source software and give it a red hot go.  Don’t be afraid to try the demonstration sites, and reach out to others that are using it, or groups like us that support it.  The open source community is building in Australia, and the more, the merrier!

Open source software is all around us – many of the systems you use already are based on it

Oh, and don’t forget, you also need to make sure that people can use the systems you’ve put in – even the most simplistic ones – so make sure you accommodate training for your team somewhere in here (and in the next section as well!).

Digital Preservation

This is the area that trips up a lot of the Archival professionals, which is why I tend to push it back in our approach for small archives.  It’s definitely the hardest part to deal with well.

A great starting point is Ricky Erway’s “You’ve Got to Walk Before You Can Run: First Steps for Managing Born-Digital Content Received on Physical Media”.  This is a really consumable 3 page guide to getting started with digital preservation and is well worth a read to see some simple ways forward.

In our work, we’ve found vastly different views of what digital preservation means to different people.  The reference above is one that encompasses what many people think is digital preservation, but that’s just the first step.  You still have to deal with keeping the preserved copies “preserved”, and you need to be able to provide access to these as well.

Both of these are a real challenge for any archive (big or small) and again, there’s no silver bullet.  We have been implementing Archivematica for clients, and supporting them with that, but digital preservation is complicated, and there are a range of decisions that you need to make as an Archivist before you can get started with a digital preservation tool like Archivematica – and you still need to consider the digital repository as well.

In terms of the digital repository, you need to think about how much storage you’ll be needing, where you’re going to put it (e.g. Glacier), what to put on the top to manage it (e.g. DSpace), how you are going to do proactive preservation across it, and these are all things that take time and consideration.

To try to give you a complete list of things to think about regarding digital preservation (and to do so in a consumable fashion for this blog) is well beyond the scope of what I was trying to write here, so let’s just say that this is going to be a difficult part of the roadmap we have proposed, and will require considerable work up front before you dive in.  But Ricky Erway’s document is a good starting point to get you going down this path.

Ongoing

You’ve got a roadmap defined; an Archival Management System in place and in use, and you’re down the Digital Preservation path – hooray we’re done! Right?

Nope. No silver bullets, remember?

The one aspect that a lot of technology clients forget (not just for Archives) is that you need to think about how to keep the whole thing ticking along smoothly in the future, and so a big part of the roadmap discussions we have are about how this will occur after the “implementation” phases.

Apart from looking at things like providing or leasing the servers that you are using (hosting), you also need to look at how to keep the hardware and software up to date (maintenance) and what you can do when it all goes pear shaped (support).

In terms of hosting, we’ve moved a lot of our hosted services to the cloud using systems like Amazon Web Services, which give us a scalable means of providing the underlying infrastructure at a low cost.  While this might appear to be expensive when when you start looking at large volumes of data, the true cost of running your own servers to do the same is actually far greater – think of the staff costs to manage those systems, upgrade and renewal regimes, rotating backups, and the like.  For small archives, it’s hard to beat from a price/value review.

Maintenance and Support are two things that we tend to bundle together into packages for our clients, using either Service Level Agreements (SLAs), Support Blocks or Time and Materials packages.  For most small Archives, SLAs seem to be an overkill – you get us on call, faster support times and other benefits, but at a higher cost.  In terms of small Archives, this is often a stretch too far.

So most small archives turn to the Support Block approach; pre-purchasing a 40 hour block of time that can then be used across support and maintenance.  This helps fix the budget for support for the organisation, but still retains a high degree of flexibility.  Time and Materials approaches also work for very small groups, but that does usually present challenges in terms of not having fixed budgets.

Often groups forget about the Ongoing part of technology projects (especially in the app space, but that’s for another time) and that is what often causes us not only technical issues down the line, but also is very demoralising for our team – spending so much personal time and effort on projects to see them slowly decay isn’t a great way to have job satisfaction (software engineers *are* human too).

A way forward

Our way forward for smaller archives can be summarised in the four steps:

• Roadmap – define the destination and how you will get there,
• Archival Management – start documenting what’s in the Archive,
• Digital Preservation – deal with the ‘digital tsunami’ in stages, and
• Ongoing – don’t forget to support the investment you’ve just made into the future.

The last area I’d suggest you also look at is to keep an eye on the future – keeping your digital archive plans up-to-date so you can migrate to future software versions and adopt new and improved features.  When your scope is “forever” you really do need to be thinking longer term than just the latest software packages.

Each of these steps is very complex, and probably worthy of a separate blog in itself, but for now, I hope this helps to provide something for you to grasp as a starting point.  I’d be keen to hear your feedback on this*, so please get in touch via email or start a conversation with us on our  Facebook, Twitter or LinkedIn feeds.

Piers

* I’m still hoping for more people to reach out from my last call for action and help about mapping where Archives are using open source software! https://archive.gaiaresources.com.au/mapping-archives-open-source/

** I added in ArchivesSpace here after I first published this post as I’ve been talking to James from Hudson Molonglo about it since then, and I think it’s actually a pretty good option.  There might be others out there too – let me know if you have something else we don’t know about!