Small steps to big data adoption

Big data analytics solutions are starting to see initial adoption among some sectors in the Gulf region

Tags: Banking and financeBig dataTeradata (
  • E-Mail
Small steps to big data adoption Brobst: Data discovery platforms are the tools that enable data scientists.
By  Mark Sutton Published  May 9, 2014

Big data uptake is in its early stages almost everywhere apart from Silicon Valley, according to Stephen Brobst, chief technology officer of Teradata, but there is growing curiosity from organisations in the Gulf. Brobst, an expert in data warehousing, warned that there is considerable hype around big data, but that a few specific vertical sectors are leading regional interest.

“In banking, telco and government, they are now in the proof of value state — they have done some tire kicking, gone to a couple of conferences, bought a Gartner report, that kind of thing, probably spent a total of $5,000 on big data in the last 18 months, and now they are trying to find out ‘can I make something real of it?’” Brobst said

The tipping point for the leading organisations is that interest in big data has spread from the IT department to the business, Brobst pointed out. While IT teams have been experimenting with solutions such as Hadoop, this has not translated into business value. In sectors where there is a desire for competitive advantage however, the business side of organisations have now become involved, and are looking for potential deployments.

Another shift in big data towards business usage is in the development of data discovery platforms, Brobst said. IT-driven big data investigations have tended towards to attempting to store all data, forever, at a low cost, an approach he calls ‘data lakes’.

“Storing data doesn’t add value unless you do something with it, so in addition to the data lake, they will have a data discovery platform. This is not for the business analyst — they are still on traditional data warehouses — the data scientists are the target for this.”

Brobst draws the distinction between business analysts, who get answers to business questions, and data scientists, who are more engaged with seeing patterns and relationships in data to know what questions to ask to be able to gain new insights into data. While these data scientists are few in number, they are very sophisticated in their use of data, and look to tools such as data discovery platforms to enable them to extract value from data.

“In a traditional data warehouse the lingua franca is SQL; in the discovery platform you use SQL but you will use other things as well, like MapReduce programming framework, maybe graph processing, text processing, things that go beyond traditional relational data manipulation. You will tend to use a lot more data visualtisation and data mining as opposed to reporting tools,” Brobst said.

“This data discovery infrastructure is a relatively new thing, and the advantage of it if is done well is it provides accessibility to big data analytics to people who are not computer scientists.”

The areas where organisations are typically seeing value from data is in customer interaction, Brobst said, where data discovery is enabling companies to analyse not just the immediate customer transaction, but other activities around the transaction, such as how long it took the customer to make a payment on a website, and to extend this to areas such as the customer’s social media interactions.

Leveraging big data is relatively expensive he warned, not so much for the technology, but because of the cost of the skills around it. At present, with very few data scientists available, they are the key commodity. Teradata has invested in programs with universities to help develop these skillsets, including institutions in Egypt and Saudi Arabia, and the company also has its own data scientists with vertical expertise to support customer deployments and knowledge transfer.

Although big data is still in its early stages, Brobst said that there is already some shift from projects that provide value to the business, to those that focus on the consumer, by empowering them with apps that use analysis of data such as personal health or driving habits to help them make better decisions.

“I think there is a big shift under foot, to start focusing on consumer intelligence, in other words, use the data for the benefit of the consumer. Financial services was an early adopter of this, there is consumer intelligence in healthcare, retail, in energy consumption, automobiles, there is all kinds of data people are providing back.”

Add a Comment

Your display name This field is mandatory

Your e-mail address This field is mandatory (Your e-mail address won't be published)

Security code