As businesses continue to produce, store and manage mushrooming volumes of data, many are struggling to balance the requirements of timely file retrieval and the cost of storing data so that it is easily accessible.  Floyd Christofferson, product marketing director at SGI discusses how companies can use intelligent ‘Hierarchical Storage Management’ (HSM) technology to help achieve this balance.

When trying to cope with the overwhelming quantity of regularly accessed business information, companies are faced with the problem of how to best manage their required files.  Compliance demands from government and industry regulations must also be considered as these compel organisations to retain information for longer periods of time, especially when files may contain diverse data such as purchase orders and contracts, personnel information, medical images and banking records.

The problem is, retaining files in an “on-line” state typically adds significant infrastructure and operating costs.  Moving the files to an “off-line” tape library reduces infrastructure and operating costs, but increases the risk and difficulty for users to access and manage the files.

A valuable tool for IT managers to establish a balanced relationship between cost and data accessibility is to implement a Hierarchical Storage Management (HSM) solution to virtualise multiple tiers of storage into one large virtual environment.  This has the effect of creating a virtualised online view of all storage tiers, whether data is on disk or tape, even though in reality, only a relatively small portion of the data might be on tier one storage.

An effective HSM strategy transparently migrates the most critical or timely data to higher performance, more expensive storage media, while less critical data is automatically moved to less expensive, lower performance storage media such as a VTL (virtual tape library) or tape. 

This migration is usually policy based, and is determined by business rules of the particular data types or workflow required.  This can be managed based on manual, semi-automatic or policy-based automatic procedures.  Policies can also be created through user-defined tags to provide a high degree of control on how a class of files is managed.  Furthermore, administrators can define high and low free space thresholds which trigger data migration to lower tiers and increase free space on selected volumes.

When such data management strategies are successful, they should be practically invisible to the end user while providing the IT manager with extremely granular control over the backend infrastructure.  By enabling “offline” archive content to appear to the file system as “online” data, HSMs allow the most efficient infrastructure from a management perspective, while helping businesses to dramatically reduce storage and storage management costs. 

The use of tape storage or VTL in an HSM solution not only has considerable financial benefits on infrastructure costs, but data managers should also be aware of the inherently green nature of tape for backup, archiving and storage., Studies from The Clipper Group have shown that tape can cost 23 times less than SATA disk and tape libraries use up to 290 times less energy.  Furthermore, because the storage tiers are virtualised and transparent to the user layer, additional capacity and/or performance can be added incrementally to the system when needed, without interrupting user access to data.  This has the added advantage of “future-proofing” the infrastructure so that new storage technologies can be added later into the tiers as they become available.

Data migration solutions are capable of managing many Petabytes of data, hundreds of millions of files and can run storage devices at their maximum rated speed by creating virtualised storage pools across Fibre Channel, SAS RAID, SATA RAID, MAID or VTL solutions and tape.  When files are accessed, they are automatically retrieved without the need for operator intervention and, when files are migrated, there is no interruption or degradation of normal service. 

HSM data migration solutions can also be implemented in parallel with a backup and/or disaster recovery plans to protect against the risk of operator error or data corruption.   The HSM can be configured to automatically create and manage multiple copies of files on disk or tape during the migration process.  This can be managed with web-based administration tools, simplifying the management of HSM operations and assisting with regulatory compliance.