Data driven decisions

Dubai Statistics Center is leveraging a powerful business intelligence solution to collate and report on a huge volume of data for the Emirate

Tags: Dubai Statistics CentreIBM (www.ibm.com)
  • E-Mail
Data driven decisions Ahmad Ali Al Dashti, director of IT & Central Statistical Systems Department, Dubai Statistics Center. (ITP Images)
By  Mark Sutton Published  June 18, 2014

In any decision making process, the more valid and relevant data that can be factored in to the decision, the better the results that will come out of it. At the highest level, for governments making strategic decisions, having access to the right data is critical, and being able to get accurate information in a timely fashion is a vital tool to planning and execution of leader’s visions.

For the government of Dubai, the role of handling vital statistical data, including collection, management and dissemination, is the responsibility of Dubai Statistics Center. Founded in 2007, DSC is responsible for collecting statistical data from across a wide range of sources, including government departments, the private sector and individuals. The data collected by DSC covers everything from demographic, economic and social statistics, and is used by high level decision makers in both the public and private sectors.

In its early stages, DSC was reliant on a lot of manual processes in gathering and publishing its statistics. Many surveys were conducted with ‘pen and paper’, and data was gathered from different sources via fax or email. Data was held in many different spreadsheets in a variety of formats, and creating reports was a complex and time consuming process. The organisation published all of the data it held just once per year, in a hard copy format. With Dubai’s ambitious plans in e-government and smart government, and the booming economy in the emirate, DSC recognised that it needed to lift its processes to a new level and to develop a more connected, electronic approach to how it managed its data.

Ahmad Ali Dashti, director of IT & Central Statistical Systems Department, DSC, said that in line with Dubai’s Strategic Plan 2021 and the drive for integrated government, DSC chose to look for a comprehensive data management and analytics solution.

“We felt that the e-services model would enable us to offer similar benefits to our audience of citizens, business leaders and senior government employees — especially 24/7 access and self-service functionality,” he said.

DSC wanted to create a highly integrated, fully online system to handle data all through the process of collection, analysis and dissemination, but meeting all of its requirements with one solution proved to be difficult.

“It was a new approach, not only for us in Dubai Statistics Centre, but worldwide,” Al Dashti commented. “We benchmarked many different statistical centres around the world, but we could not find a comprehensive, end-to-end electronic statistical system.”

In the end, DSC decided to work with IBM, and to use its Cognos Business Intelligence and Infosphere as the basis of a new statistical platform, the Smart Statistics System. The system would need to tackle three main areas of activities — gathering of data, data mining and data cleansing, and publication of data — all of which needed to be integrated together. The solution would need to account for the different data coming from different departments based on different technology, and to handle terabyte volumes of data.

Along with consultants from IBM, DSC worked with the partner organisations that would supply data and consume the final reports, to check their requirements, what data they would supply and in what format. One of the reasons for selecting IBM Cognos was the flexibility of the solution, in enabling DSC to integrate a wide source of data through a variety of different means.

DSC takes data from a wide number of sources, across the government and business sector, and from sampling the public, with different degrees of readiness to integrate and provide electronic data, so it was essential that the solution would be able to accept all sources of data, cleanse it and integrated it into a single format in a single repository.

“Data comes in different formats from different sources,” Al Dashti explained. “Some were ready to integrate, some not, so we had to be flexible, to have direct database integration, XMLs, data entry windows - all has to be turned into unified structured format and unified standard definition.”

The key to collating all of the various data was IBM’s InfoSphere DataStage software, an extract, transform, and load (ETL) tool that enabled DSC to integrate the data easily.

Hassan Machmouchi, IT Expert, Information Technology Department explained: “We didn’t try to force our formats on other government entities, they are using different formats, different technologies, different meta data, different definitions, and this is where IBM DataStage allowed us to integrate all these formats, build the definitions, unifying it.”

Add a Comment

Your display name This field is mandatory

Your e-mail address This field is mandatory (Your e-mail address won't be published)

Security code