Ravi Chalaka, senior director, product and solutions marketing, BlueArc looks at the development of pNFS The Network File System (NFS) protocol is a widely recognised standard for High Performance Computing (HPC) applications and is commonly associated with scientific research and the oil & gas industry. Unfortunately NFS does not scale well, making it impractical for larger systems with greater I/O needs. Up to now organisations requiring scale have addressed this challenge by investing in custom-built parallel file systems and multiple I/O servers. Such solutions are complex, take years to build and are not easy to replicate. The lack of open, industry standards-based solutions has been holding back parallel file systems from expanding into broader commercial environments.  At BlueArc we believe 2011 will be the breakthrough year for standards-based Parallel NFS (pNFS). Once this happens, HPC technologies will at last be viable for a wide variety of enterprise applications.

Standards
Part of the NFS v4.1 standard, pNFS benefits from considerable private industry expertise and experience as well as input from the open source community.  Through pNFS, organisations gain increased storage performance and scalability as well as investment protection and the ability to interact with best-of-breed solutions. The general availability of pNFS-based storage solutions will fuel the adoption of parallel file systems within a broad range of data-intensive industries. As demand for parallel file system standard solutions builds, vendors will be presented with new opportunities to solve technical challenges – such as how to support increasingly mixed workloads - through innovation. Vendors will also give their pNFS standard systems enhanced reliability and availability to protect valuable data and ensure continuity for important cluster applications. In time healthy competition between pNFS-based HPC products will spell the end for the vendor lock-in and technology experiments we see today. We are entering an age of simpler, better HPC solutions for customers. 

Implications for vendors
Since the mid-2000s BlueArc has been working with vendors like Panasas, EMC, NetApp, Oracle and Microsoft on pNFS development.  Contemporaneously open source initiatives like Lustre have sought to integrate pNFS clients into Linux distributions (although these efforts remain complex, requiring considerable expertise to manage).  The vendors see pNFS as a way to offer customers a standard interface to their own technologies. Layering pNFS on top of their own backend solutions allows them to be transparent to the end user. The result will bring more scalable I/O throughput solutions – and in some cases HPC in general – to whole new categories of users.

We see vendors competing to develop differentiated offerings that optimise performance for specific applications. pNFS therefore will not short-circuit or diminish existing intellectual property or competitive advantage. It is a tool for bringing proprietary advantages to more end users without forcing them to abandon the standards path.

Impact on users
Many experienced users of parallel file systems will appreciate simplified and standardised implementation and management pNFS brings. We believe pNFS could even become a key technology for addressing the advanced technology gap thought to be hindering innovation amount small and medium-size manufacturers - fewer than 8% are using HPC technologies today.
A big customer advantage for pNFS is that IT administrators already know the environment inside out. pNFS will therefore have a broad end-user appeal for those wanting greater data management scalability. Its ability to work with proprietary backend file systems to project an NFS environment means pNFS can be easily integrated into most product offerings.  For example, we may very soon see Windows-based solutions that allow new users to scale their I/O infrastructures without a custom-built file system and without introducing Linux into predominately Windows IT environments.

We also expect pNFS to introduce new use cases for parallel file systems. One example is for scratch space - where high-performance disk is serving up active data for an application that is currently running. pNFS is perfect for shared work environments where scratch space needs to deliver balanced scalability - either delivering bandwidth for a relatively few large files or managing throughput for relatively large number of small files.  Since pNFS will be more familiar for administrators to manage, some users will look to bring the features of parallel environments – such as greater scalability and a shared namespace – to adjacent areas such as home directories or shared repositories.

In summary, pNFS looks set to have a big impact on commercial organisations that want greater performance within a standards-based IT environment. It will also be a cost-effective alternative for organisations, scientific research institutions for example, that may have already invested in expensive custom-built ways of extending existing distributed file systems beyond their limits. pNFS will also open up opportunities for new groups of users to harness the benefits of HPC where it was too costly or impractical before.