How did the media giant get a grip on its data and storage management? DCM found out how

 Business Objects reports are now running over 100 times faster thanks to data warehouse appliance technology from Netezza. Business users are free to spend more time analysing data, rather than waiting for queries to complete. So, now more users are performing more detailed analysis than ever before.

Virgin Media is a huge operation. It has almost 10 million customers, and offers a range of services including the UK's only quad-play of TV, broadband, phone and mobile, plus the most advanced TV on demand service available, the UK's first high definition TV service and V+, its high specification personal video recorder.

As Paul Froggatt, information management Manager at Virgin Media recalls: “There was pressure from the business about the performance of existing systems. They were not able to run the queries they wanted to, and had to cut the queries into small, more manageable chunks. The time it took to turn around new developments or enhancements to existing systems was a barrier too, and the complexity of building something in Oracle was paralysing the Business Intelligence (BI).”

At the time, the company outsourced the analysis of NTL telephony data, while Telewest was a long-term user of Business Objects Desktop Intelligence for data analysis. This allows the company to access and analyse data both online and offline – and share it within and beyond the enterprise. In October 2006, the company started to look for a solution that would allow it to scale up its existing data warehouse to incorporate NTL data and deliver better performance than its existing Oracle Unix platform. Even before the merger with NTL and Telewest, the data warehouse had been struggling to perform.

As Howard Watson, chief technology and information officer of Virgin Media, explains, “The project started at Telewest, where we had a data warehouse in place to capture CDRs (call detail records) for regulatory compliance, network routing and telephony high usage analysis. But, owing to the performance issues with the older architecture, we were restricted to querying data specific to international, mobile and premium rate calls for high usage analysis.”

With the integration of the three businesses together, it would mean more than doubling existing data volumes, and it was clear the existing warehouse technology would be inadequate. The team therefore began researching the market to find suitable new solutions. They arrived at a shortlist of four vendors, and conducted off-site ‘proof of performance’ exercises with each of them. The intention had been to produce a short-list of two and then hold further tests on-site, but one of the candidates, the Netezza Performance Server (NPS) appliance, performed so far ahead of the rest that the choice was made immediately. “Netezza scored highly on all the counts that matter - scalability, performance, simplicity and total cost of ownership,” says Froggatt.

A prime requirement of the project, which was sponsored by the ‘revenue assurance’ (RA) part of the business, was that it should be seamless to the business users. It was extremely important for Virgin Media to be able to take its existing BI tools, along with what had been built in those tools, and port them across to the new Netezza platform. “Moving the Business Objects BI tools from Oracle to Netezza did not require a lot of effort,” says Froggatt. “We took just a couple of days to convert quite a complex universe. It was a painless experience. We simply cloned the users’ existing universe, implemented the required syntactical changes, and then instructed the users to re-point their reports to the new universe. It was a one-step change that had a limited impact on the business users.”
 
The Four-Week Data Warehouse
The project had always worked to ambitious deadlines, and when it started in early October 2006, the team anticipated it would take a couple of months to complete. But, timescales became further squeezed by another IT project, forcing the deadline to come forward to November 1, just four weeks away.

“We had four IT people involved, and a representative from the business undertaking user acceptance testing,” says Froggatt. “In just four weeks, we managed to migrate the existing data warehouse, all seven terabytes of it, to the Netezza appliance as well as converting a suite of a dozen extract and summary routines, two Business Objects universes and an Informatica Data Analyzer dashboard.”
 
The benefits – 252 times as fast and simple to manage
The results have been dramatic. With Netezza as the enterprise data warehouse of choice, and the world’s leading BI software, Business Objects as the query tool, the biggest single benefit has been in increased performance. Business Objects solutions are designed to help organisations gain better insight into their business, improve decision-making and enterprise performance. This clearer ‘vision’ powered by the near real-time delivery from NPS systems makes a powerful combination.

“The speed and accuracy of the Netezza/Business Objects set-up is a massive step change, and performance is consistent. So, no matter how large or complex the query – the results are always fast, very fast,” says Moly McMillan, head of Revenue Assurance at Virgin Media. “No matter what the question is, the answer is always just seconds away. Before Netezza we could never be sure how long a query would take. Now, no matter how much data there is to be analysed – be it three, six or nine months’ worth – the time frame for results is entirely predictable.”

Compared with the old Oracle-based system, the new one is, incredibly, 252 times faster on average. This means it will support any type of query without a problem, whereas with the previous set-up, users were restricted in the size and complexity of query they could run. Maintaining the data warehouse is also much easier. For instance, the company is now able to do things that would have been impossible before – such as implementing changes to the structure of a set of tables containing 32 billion rows of data, which was achieved in just four hours.

Paul Froggatt says the simplicity of approach in the Netezza system is fundamental both for the IT department and for users. “The new approach has cut out the complexity of traditional databases implementations, and the data warehouse appliances are delivering on that promise. Because we don’t have to think about the layout of data, we are reaping productivity gains with new solutions.”

In addition, he says the use of the database administration team has dropped dramatically and is now very minimal. “The NPS system requires only a basic skill set to manage,” he says. Previously they would have had to educate the business users on how to write new queries efficiently. “If the queries were complex, that process could take up to a week, or if it involved a lot of data - say six months’ worth, they would have to run it several times, first testing it over smaller date ranges. Now, all of these constraints are gone, and we don’t have to think about these issues,” he says. “The chains are off and business users are free to run the queries they want.”
 
Multiple uses
The Netezza/Business Objects platform is used across the organisation by revenue assurance, credit services, product marketing and product pricing teams. Each of the departments has super users who know the system intimately, and there are around 100 users on the data warehouse specifically.

In the revenue assurance department, the investment in the 8450z and 8050z Netezza Performance Server (NPS) systems has effectively been recovered within three months, by helping them to identify revenue leakage and plugging any gaps in the revenue chain.

In credit services, they use the system to spot high telephony usage and proactively manage any potential credit problems before they arise. This usually involves contacting customers to ensure that they are aware of the high usage. This protects customers from fraud, and helps them best manage their repayments. The team has made significant savings on bad debt write-off and reduced customer ‘churn’. Before the introduction of Netezza, staff had to rely on a summary report three times a day to spot high usage. With the new systems, the data warehouse is updated in real-time, enabling the credit services staff to spot anomalies early and act quickly.

And, in the telephony product marketing team they use the system to pro-actively plan and forecast the effect of telephony tariff changes, and to plan how to best react to competitors’ offerings.  They are able to compare usage profiles of customers before and after such changes, enabling them to measure their success.