For once, it may be time to believe the hype. Artificial intelligence (AI) really does have the potential to change everything: the way we work, the way we live, how we make decisions and even how we understand human nature. Everything.

In various forms, AI technologies have existed for several decades, but it’s the explosion of data – the raw material that fuels AI – that has allowed these technologies to advance with incredible speed in recent years.


Within the next ten years, the Internet of Things (IoT) is expected to grow to 150 billion networked sensors. Those sensors will be found in home appliances, vehicles, in industrial robots that manufacture the products we consume, and even on and inside our own bodies, in the form of fitness trackers and medical devices. All these sensors will generate data, constantly, feeding AI systems designed to create better services, products and customer experiences.


Organisations know this is a new battlefront they must master. Those that can harness the explosion of data will be able to make smarter, faster decisions to offer more relevant and compelling propositions, and transform the way their business operates in a modern digital world.


But there’s a catch to all this: AI on its own is not the silver bullet many believe it to be. Much like humans, it relies on reference points, experience and education to develop its intelligence. Similarly, the benefits that we expect AI to deliver depend implicitly on the data that feeds it. If the data used to train AI systems for decision-making isn’t complete and of high quality, then investments made in AI will fail to realise expected returns.


It’s a pretty sobering point when you consider that businesses worldwide were expected to spend $19.1bn on AI systems in 2018 – a figure that is only set to rise this year.

When AI goes rogue

Unlike flash drives and mobile apps, AI is not standalone, plug-and-play technology. It needs careful planning and good data for it to be smart, because an AI algorithm can only be as accurate as the data used to train it. In other words, with biased data, you get biased decision-making. With flawed datasets, you get flawed decisions.


Take, for example, an AI-driven recruitment engine at Amazon that was used to review CVs from job applicants: it was discovered that this system was not rating candidates for software development jobs in a gender-neutral way, because it had been trained by observing patterns in resumes primarily from male candidates.


Likewise, the 2017 shelving of a planned oncology advisor tool developed by IBM’s Watson Health was attributed, in part, to problems in the data intended to feed it: difficult-to-decipher, handwritten notes from doctors; inconsistent use of electronic medical records; widespread use of acronyms across both analogue and digital records.


If firms are really serious about becoming insights-driven, they must build strong data foundations. The quality of data management, as well as an organisation’s R&D and business culture, are critical foundations for any successful AI initiative.

The case for data management 

The AI promise is for data to be better analysed and significantly improve human decision-making. But in reality, many of the organisations that need to take advantage of AI are simply not geared toward managing the sheer complexity and volatility of today’s data. In turn this will inhibit their AI deployments from being truly intelligent.


Only now are IT leaders realising that new technologies like AI require hard work. The forward-looking organisations have worked out how to use people and technology combined, to make their AI successful and leverage its full potential value. New data-focused roles and processes are needed to take full advantage of AI. It’s not a case of just shifting from old architectures. It requires redesigning entire operating models to suit the new wave of technology.

“It’s not a case of just shifting from old architectures. It requires redesigning entire operating models to suit the new wave of technology.”

From a technology standpoint, what’s clear is that a solid data management strategy is a key prerequisite to effective AI. For AI to work it’s not the more media-hyped frontend, such as a virtual assistant, that is critical to its success but instead with the backend data – the foundation of any digital change.


Good data feeds good insights. AI can help to make sure that that the journey of all data is tracked, recorded and visible across the organisations to feed into any transformational activity.


This places a number of demands on data management teams. They must learn how to simplify access to data from a wide range of both established and emerging data sources; hone their data cleansing techniques to eliminate data that is invalid or redundant from key datasets; and re-evaluate their approach to metadata, so that it is able to provide answers to any questions that arise over the provenance and quality of data used to make decisions.


To achieve this they need a data management platform that can leverage AI implicitly in the data engineering level.

Trifecta of data experts needed to activate insights

Organisations also need to reimagine today’s data and analytic roles to activate AI insights. Even before you design or choose your AI, you need to understand the possibilities, limitations, biases and gaps in your data.


This very human process requires data teams to work with internal subject-matter experts to understand how the business works. It includes all the knowledge and the variables the experts may no longer think about, because they have become routine and there is no longer a need to articulate them.


This fact-finding requires a data team with deep-data capabilities and includes data scientists and data engineers, ideally led by a chief data officer – but these are skilled people who are in short supply and whose skills come at a premium. The chasm between demand and available talent has led to almost 50% of European companies that are thought to be struggling to fill their data scientist positions.

“To fully optimise AI, more people are needed, not fewer.”

In short, to fully optimise AI, more people are needed, not fewer. Data scientists are needed for their advanced mathematics and statistical analysis of the data that feeds AI and the business insights that it produces. While data engineers are essential to build, test and maintain the infrastructure and architecture for the data generation that underpins effective AI. These are the hot skill sets that organisations must scramble to recruit.


After all, for businesses seeking competitive edge, AI will be the answer.


It will redefine and advance the customer relationship, providing opportunity to deepen knowledge on customers and develop services and products that directly address their needs. This new customer contract, integral to the generational shift we call Data 3.0, will define the next era of a trusted business and may prove to be a bigger differentiator than price. To realise this potential, however, requires more commitment to data than has ever been needed before.

Share this article