Your guide to data warehouses

Understanding the requirements and strategies of data warehousing is the key to a successful deployment, according to Abhijit Pendse, Senior Engagement Manager, Cedar Management Consulting International.

Tags: Cedar Consulting (www.cedar-consulting.com)Data warehouse
  • E-Mail
Your guide to data warehouses A true enterprise data warehouse deployment requires careful consideration, says Pendse.
By  Abhijit Pendse Published  June 21, 2010 Network Middle East Logo

Understanding the requirements and strategies of data warehousing is the key to a successful deployment, according to Abhijit Pendse, Senior Engagement Manager, Cedar Management Consulting International.

Business Intelligence has been a popular buzzword among the C-Suite executives in the past couple of decades. The business folks always look towards it as a panacea for all analytics, but for the CIOs it has been a perennial headache. Balancing the business expectations and the technology possibilities within an acceptable timeframe is a game which not many CIOs have won in the complex maze of dimensions and cubes of data warehouses.

Data Warehouse (DW) projects are huge and typically the implementations run over a couple of years costing 5-10 million dollars. However these projects also have a very high failure rate. As a result choosing the right architecture and the implementation approach becomes critical before venturing into a DW project.

Even before start of the project, the first decision the CIOs have to take is whether to implement an enterprise data warehouse (EDW) or to go with a tactical solution. The business requirements and the existing technology architecture play an important part in this decision. If the business requirements are operational in nature, then in majority of the cases, a tactical solution with a Business Intelligence tool deployed on top of an operational database should suffice.

However if such a tactical solution is not going to serve the purpose and an enterprise data warehouse has to be implemented, then a completely new set of attributes have to be considered.

DW design attributes

The first critical attribute in the design of a DW is the architecture. In a distributed architecture, the data is stored between diverse, heterogeneous and loosely related data stores. In a centralized architecture, data is stored centrally in a hub and spoke model. A centralized architecture is the preferred choice of the majority due to ease of maintenance, data consistency and optimized interfaces.

The second attribute is the design methodology. In the Bill Inmon ‘top down’ design approach, the EDW is created first and the dimensional data marts are carved out of the EDW. The ‘bottom up’ approach proposed by Ralph Kimball recommends creating the data marts firsts which are later joined to create the EDW. While there are pros and cons associated with both the approaches, the majority of implementations have used the Inmon top down approach.

Third attribute is deciding the refresh rate. The first generation data warehouses of the 1990’s usually had a monthly refresh rate. Over the years, there has been a gradual shift from monthly to a daily refresh. With new requirements like fraud detection, risk management and dynamic dashboards taking business priority, the need has arisen for real time or near real time refresh. A real time refresh requires higher storage, bandwidth and faster server infrastructure. While the industry standard at this moment is the daily refresh, a real time refresh rate is worth considering as the analytical possibilities and applications are endless in this option.

Add a Comment

Your display name This field is mandatory

Your e-mail address This field is mandatory (Your e-mail address won't be published)

Security code