We have all heard and probably used the colloquialism “how long is a piece of string,” but in a similar vein we often get asked “how long should we archive data for?” This is a really difficult question and the only answer we can give is “for as long as you’re likely to need it.” At some point, as your data grows you will have to put together a business continuity plan that dictates how long you keep certain types of data. Namely, a data archive. If your business does not have a lot of data then it doesn’t become a problem, but as this grows to Terabytes it does become an issue for your IT Department.
Traditional data storage is not expensive, and as time moves on, it actually becomes cheaper. So a good question to ask is why should I bother about archiving data? However, if you have production data which requires fast access, then this data is not cheap. Therefore it makes more sense to move it to cheaper data storage to free up more room for the data that does require faster access. This then provides its own problems. If you just move it because no-one has accessed it for 6 months or so and someone does need access to it and can’t find it, you will have no end of support calls. So if data is moved you need to have some kind of archiving solution that leaves a tag behind to redirect the user to the new data store.
Retention periods for data storage can be a highly emotive subject and guaranteed to get people arguing. In some cases it can be simple, i.e. for legal reasons financial data may be kept for seven years, but HR data may only need to be kept for three years. Every company is different and has different compliance regulations – hence the arguing!
However, as most IT Directors and Managers have no legal training, they will have to develop a data archiving strategy.
The first step is to analyse all of your data and assign a retention period to each different data set. You may need to speak to your corporate lawyers in the first instance to determine any legal or compliance issues you need to adhere to. The next step, will be to speak to department managers and ask them to determine own retention periods. One thing for IT to bear in mind, if there is not retention period or no-one is willing to tell you then as a general rule of thumb – keep it. The next step is where to keep it and that is often the main issue. If you speak to a general user, they need access to all the data they have ever created and used all the time, even the stuff that is years old. As already stated, if you are not short of disk storage, you don’t need to worry about archiving but if you are running out of expensive storage is can be a great way of saving money.
Data archiving benefits
- Reduced costs: move older data to cheaper forms of storage to free up space on your production data storage.
- Reduced backup window: removing data from expensive storage also means less backup and reduced time to complete your backup.
- Compliance: This is often the main reason for implementing a data archiving strategy.
- Knowledge retention: In the era of ‘big data’ knowledge is king and the more data you have the more you can report on it. By removing old data, you can report on the important data and up to date reporting.
- Improved performance: by reducing your production data, you can now access data quicker and give an overall performance boost to your data store.
Application-specific archiving products are tailored to deliver these benefits to specific environments. Examples include SAP, email and Oracle applications. Application-specific products are designed to know the ins and outs of the application so they can prune or separate data in a manner that optimizes the application without endangering referential integrity. General-purpose archivers aren’t usually smart enough to do this. An application-specific tool may be all that’s needed when data volumes don’t justify a system-wide implementation, the major pain point relates to a specific application or a general-purpose product won’t adequately address a given application.
There are many software archiving products on the market but only a few have the intelligence to actually aid the IT Department and reduce the workload. Other manual solution will increase your workload and therefore not prove to be good value for money.
Unleashed are happy to carry out a data storage review and help you to put together a data archiving strategy. For more information, visit us at Unleashed.