Gartner's Top 10 data analytics trends

Using augmented analytics, banks found that younger clients (aged 20 to 35) are actually more likely to transition into wealth management - a clear example of how relying on business users to find patterns, and on data scientists to build models manually, may result in bias and incorrect conclusions.

Augmented analytics is just one of the top 10 technologies Gartner has identified with the potential to address these and other major data and analytics challenges in the next three to five years.

Digital transformation has put data at the centre of every organisation. Businesses are awash with data. They struggle to identify what is most important and what actions to take (or avoid).

Act now on emerging trends

Rita Sallam, distinguished vice president analyst, Gartner, says organisations need formal mechanisms to identify technology trends and prioritise those with the biggest potential impact.

“Data and analytics leaders should actively monitor, experiment with or deploy emerging technologies. Don’t just react to trends as they mature,” Sallam says. “Use this list to educate and engage with other leaders about business priorities and where data and analytics can build competitive advantage.”

Gartner’s list of top technology trends in data and analytics does not include trends that are less than three years away from mainstream adoption (such as self-service analytics and BI) or more than five years out (such as quantum computing). Nor does it include non-technology trends such as data literacy, storytelling or data ethics that are also critical to success.

Trend No. 1: Augmented analytics

Augmented analytics automates finding and surfacing the most important insights or changes in the business to optimise decision making. It does this in a fraction of the time compared to manual approaches.

Augmented analytics makes insights available to all business roles. While it reduces reliance on analytics, data science and machine learning experts, it will require increased data literacy across the organisation.

By 2020, augmented analytics will be a dominant driver of new purchases of analytics and business intelligence as well as data science and machine learning platforms.

Trend No. 2: Augmented data management

With technical skills in short supply and data growing exponentially, organizations need to automate data management tasks. Vendors are adding machine learning and artificial intelligence (AI) capabilities to make data management processes self-configuring and self-tuning so that highly skilled technical staff can focus on higher-value tasks.

This trend is impacting all enterprise data management categories, including data quality, metadata management, master data management, data integration and databases.

Trend No. 3: Natural language processing (NLP) and conversational analytics

Just as search interfaces like Google made the internet accessible to everyday consumers, NLP gives business people an easier way to ask questions about data and to receive an explanation of the insights. Conversational analytics takes the concept of NLP a step further by enabling such questions to be posed and answered verbally rather than through text.

By 2021, NLP and conversational analytics will boost analytics and business intelligence adoption from 35% of employees to over 50%, including new classes of users, particularly front-office workers.

Trend No. 4: Graph analytics

Business users are asking increasingly complex questions across structured and unstructured data, often blending data from multiple applications, and increasingly, external data. Analysing this level of data complexity at scale is not practical, or in some cases possible, using traditional query tools or query languages such as SQL.

The application of graph processing and graph databases will grow at 100% annually

Graph analytics is a set of analytic techniques that shows how entities such as people, places and things are related to each other. Applications of the technology range from fraud detection, traffic route optimisation and social network analysis to genome research.

Gartner predicts that the application of graph processing and graph databases will grow at 100% annually over the next few years to accelerate data preparation and enable more complex and adaptive data science.

Trend No. 5: Commercial AI and machine learning

Open-source platforms currently dominate artificial intelligence (AI) and machine learning and have been the primary source of innovation in algorithms and development environments. Commercial vendors were slow to respond, but now provide connectors into the open-source ecosystem. They also offer enterprise features necessary to scale AI and ML, such as project and model management, reuse, transparency and integration — capabilities that open-source platforms currently lack.

Increased use of commercial AI and ML will help to accelerate the deployment of models in production, which will drive business value from these investments.

Trend No. 6: Data fabric

Deriving value from analytics investments depends on having an agile and trusted data fabric. A data fabric is generally a custom-made design that provides reusable data services, pipelines, semantic tiers or APIs via a combination of data integration approaches in an orchestrated fashion. It enables frictionless access and sharing of data in a distributed data environment.

Trend No. 7: Explainable AI

Explainable AI increases the transparency and trustworthiness of AI solutions and outcomes, reducing regulatory and reputational risk. Explainable AI is the set of capabilities that describes a model, highlights its strengths and weaknesses, predicts its likely behaviour and identifies any potential biases.

Without acceptable explanation, autogenerated insights or “black-box” approaches to AI can cause concerns about regulation, reputation, accountability and model bias.

Trend No. 8: Blockchain in data and analytics

Blockchain technologies address two challenges in data and analytics. First, blockchain provides the lineage of assets and transactions. Second, it provides transparency for complex networks of participants.

However, blockchain is not a stand-alone data store and it has limited data management capabilities. A blockchain-based system can’t serve as a system of record, meaning a huge integration effort involving data, applications and business processes. Realistically, the technology hasn’t yet matured to real-world, production-level scalability for use cases beyond cryptocurrency.

Trend No. 9: Continuous intelligence

Organisations have long sought real-time intelligence, and systems are available to do this for a limited set of tasks. Now it is finally practical to implement these systems — what Gartner calls continuous intelligence — on a much broader scale because of the cloud, advances in streaming software and growth data from sensors in the Internet of Things (IoT).

By 2022, more than half of major new business systems will incorporate continuous intelligence that uses real-time context data to improve decisions.

Trend No. 10: Persistent memory servers

Most database management systems (DBMS) make use of in-memory database structures, but with data volumes growing rapidly, memory size can be restrictive. New server workloads are demanding not just faster processor performance, but also massive memory and faster storage.

Persistent memory technology will help businesses extract more actionable insights from data. Many DBMS vendors are experimenting with persistent memory, although it may take several years to modify their software to take advantage of it.