Analysing dirty data is bad business

Business intelligence (BI) is undoubtedly flavour of the year and is even challenging security for mind share among CIOs according to vendors and analysts. But can companies really trust the data and information on which they are basing key strategic business decisions?

  • E-Mail
By  Colin Edwards Published  May 8, 2006

|~||~||~|Business intelligence (BI) is undoubtedly flavour of the year and is even challenging security for mind share among CIOs according to vendors and analysts. Among the trends driving spending on BI, according to Gartner, is business performance management, where financial and operational data is analysed against key performance metrics to guide business strategy.

Now all this is a great concept except for that universal truth that IT has proved time and time again since its inception: GIGO - garbage in, garbage out.

But surely, I can hear you exclaim, the whole development of BI over the past five years or so has been made on the premise that the data quality and integration issues on which successful BI depends had been conquered? After all, how on earth could all these companies be achieving that 360 degree Nirvana view of their operations without underlying data quality?

The fact is most companies have neither secured data quality nor the promised Nirvana, though no one can deny the business case for BI continues to be very compelling and, despite having feet of clay, some of its successes to date have been remarkable. But among the reasons for many of these successes is that BI has been implemented in easily managed pilots and departmental projects where data sources have been limited and simple to control and verify. Cleansing names and addresses for a direct mail campaign is a bit different to using multiple-sourced information to formulate a critical business strategy.

BI is moving into and across the enterprise, where data sources are numerous and complex, user needs are becoming more sophisticated, and the user base is expanding beyond the traditional senior management, knowledge specialist and IT user.

Despite denying that BI might not have been able to deliver total data quality in the past to meet enterprise wide BI needs, vendors are beginning to acknowledge that if BI is going to deliver on its promises, then data quality has to be addressed.

Research organisation, Forrester points out in an information quality report that BI needs clean data and argues that the closer data cleansing gets to the actual creation of data, or the movement of data into transactional systems or data warehouses, the more valuable those solutions become.

Two vendors that are currently upping their involvement in data quality are Informatica and Business Objects, the latter of which last week announced new products, services, and partnerships for enterprise information management (EIM). EIM provides customers with a strategy to ensure that their information is accurate and current.

"The first step in building a high performance organization is to make sure that you can trust your data," said John Schwarz, CEO of Business Objects. "A complete EIM approach will ensure that every decision maker - from the boardroom to the shop floor - is using correct, consistent data.”

The company’s move into EIM follows its US$69 million acquisition a few months ago of Firstlogic, an information quality solutions developer. The move enables Business Objects to offer the product components of an EIM strategy - data integration, metadata management, and data quality products.

Forrester’s Keith Gile and J Paul Kirby in their Quick Take report on Business Objects’ acquisition of Firstlogic and Informatica’s acquisition of Similarity Systems, wrote: “Incorporating a native cleansing mechanism into the Business Objects BI stack to help clean up the data makes sense and should add value to Business Objects’ BI platform.”

They add that both vendors’ acquisitions send out a message to the leading platform vendors such as IBM, Microsoft and Oracle that they are no longer willing to cede information quality and data management to them.

Among Forrester’s recommendations are, that while information quality may not be the biggest thing in BI, it’s one of the more important issues and customers should: “acknowledge that reporting and analysing dirty data is bad business."

Organisations need to add information quality to their BI strategies to enhance reporting and analysis.


Add a Comment

Your display name This field is mandatory

Your e-mail address This field is mandatory (Your e-mail address won't be published)

Security code