Big Data and AI Executive Survey 2020
Big Data and AI Executive Survey 2020
www.newvantage.com
Copyright 2020
We described the problem this way in the Foreword to last year’s survey:
Yet we would guess with high confidence that the great majority of spending on big data
and AI goes for technology and its development. We hear little about initiatives devoted to
changing human attitudes and behaviors around data. Unless the focus shifts to these types
of activities, we are likely to see the same problem areas in the future that we’ve observed
year after year in this survey.
Unfortunately, our prediction was correct. Companies continue to focus on the supply side for data
and technology, instead of increasing demand for them by business executives and employees. It’s
a technology push rather than a pull from humans who want to make more data-based decisions,
develop more intelligent business processes, or embed data and analytics into more products and
services.
In case you don’t plan to read the body of the report, a little evidence of the problem may be useful.
Spending on big data and AI continues to increase, albeit less rapidly. More than two-thirds of
these large organizations are investing $50MM or more in these technologies, and many spend
half a billion or more. Yet minorities of those surveyed report that their business is data-driven,
and even smaller minorities that their cultures emphasize data. About three quarters of
respondents say that business adoption of big data and AI is a challenge. Over 90% report that the
challenges to becoming data-driven are in people, process, and culture—not technology. Perhaps
not surprisingly, only 15% have deployed AI broadly into production—because that is where people
and process issues come into play.
Continuing with the same supply-driven approach is not going to change this situation. What will
change it is concerted efforts to educate senior and middle managers about the power of big data
and AI, to provide experiences that make them more comfortable with technology, to redesign jobs
to accommodate smart machines, and to upskill employees at every level to work with analytics
and AI.
Perhaps IT and data executives feel that business leaders in their organizations are too old to learn
new tricks. Indeed, in a recent global survey of executives sponsored by the analytics vendor
Splunk, 73% felt that data skills were harder to learn than other business skills, and 53% believed
they are too old to learn such skills. This speaks either to the need for a concerted effort to change
their minds, or to a need for generational change.
But continuing in the current vein, or giving up on the objective of data-driven organizations and
cultures, isn’t really a viable option. That would mean the eventual demise of legacy organizations
in favor of digital native firms—a prospect that many executives have feared in previous surveys.
While human change is almost always more difficult to accomplish than technical change, it is
hardly impossible. The companies that change the paradigm of data management and exploitation
in a more human direction will, we believe, be the leading firms of the future.
January 2020
Introduction
Leading companies are certainly well on their way with their investments in Big Data and Artificial
Intelligence (AI) capabilities with 98.8% reporting active investment underway.
What has been less understood is the extent to which these companies are generating business
results and measurable outcomes. While 73.3% of the survey participants indicate that they are
achieving measurable results, the precise nature of these results has been less explored.
How is data-driven business transformation driving better business outcomes, and what are the
obstacles to that transformation? These are the core themes of NewVantage Partners 8th annual
executive survey.
The key question we sought to answer this year is how Big Data and AI investments are connected
to tangible business outcomes. With this objective in mind, we asked executives to tell us:
▪ What is the principal driver of Data and AI investment within your firm?
▪ How widespread is AI adoption within your firm?
▪ What is the overall state of data and analytics within your firm?
▪ What is the state of your corporate data environment?
▪ What prevents your firm from achieving data-driven transformation?
▪ How successful has your organization been in establishing the CDAO function?
This survey was first conducted in 2012 -- in response to Fortune 1000 business and technology
C-Executives who sought to understand the potential impact of Big Data.
Last year, the 2019 version of NewVantage Partners Executive Survey revealed some compelling
and perhaps disheartening findings:
Clearly, while progress has been made, much work remains to be done. Need drives opportunity.
Data and AI are a journey. We are in the midst of it.
We are very grateful to report this year that over 70 Fortune 1000 or industry leading firms are
represented in our 2020 survey, our highest rate of participation and representation to date.
C-executive participation has never been higher – 98.8% of participants identify as C-executives
or the equivalent. This year’s results and findings paint a picture of businesses and industries in
transition.
As always, a special thank you to all the executives who participated. We are very appreciative of
your time and thought.
Financial services firms once again comprise the majority. Healthcare firms are increasingly
represented for the 2nd consecutive year – now representing over one quarter (26.4%) of
participating companies. Here are the 2020 participants:
And so it has remained. This is our 8th survey of senior corporate officers.
This year, C-executive decision-makers comprise 98.8% of survey participants, up modestly from
2019. This group is once again dominated by Chief Data Officer (CDO) and Chief Data & Analytics
Officer (CDAO) respondents, the CDO and CDAO being a role and function which has emerged
over the course of the past 8 years as the principal advocates for data within the corporate
enterprise. The evolution of the CDO/CDAO role is a focus area of this survey and is discussed in
detail in the Leadership section of this report. As noted last year, many organizations have merged
the Chief Analytics Officer (CAO) role with the CDO role to create the combined CDAO function.
A shrinking percentage of firms are represented in the survey by the CAO function.
The following represents a summary of the 2020 executive survey participants by responsibility.
Overall participation by role and responsibilities remains largely consistent in recent years.
Here are some of the major findings and highlights from NewVantage Partners 2020 Big Data
and AI Executive Survey:
Data and AI investment are up, but the pace of investment is slowing
The percentage of firms investing greater than $50MM is up to 64.8% in 2020 from just 39.7% in
2018, with a total of 98.8% of firms investing in Big Data and AI initiatives. However, the pace of
investment is leveling off, as only 51.9% of firms are accelerating their rate of investment, in stark
contrast to the 91.6% who were accelerating their pace of investment in 2019.
Firms report ongoing interest and an active embrace of AI technologies and solutions, with 91.5%
of firms reporting ongoing investment in AI. On the side of caution, however, only 14.6% of firms
report that they have deployed AI capabilities into widespread production, and the percentage of
firms naming AI as the most disruptive technology declined from 80.0% in 2o19 to 69.5% in
2020 -- still a sizable lead over Cloud Computing at a distant 11.0%.
Business adoption of Big Data continues to be a struggle, with 73.4% of firms citing this as an
ongoing challenge. Only 37.8% report that they have thus far created a data-driven organization.
Even fewer – only 26.8% -- report success at building a data culture within their firms. Firms are
still struggling to compete on analytics – only 45.1%, a slight decline from 47.6% in 2019. Most
firms cite people and process challenges – 90.9% -- as the biggest barriers to becoming data-
driven organizations.
The Chief Data Officer role remains unsettled and in early maturity for most firms
Firms continue to struggle with the role and function of the Chief Data Officer (CDO/CDAO)–
72.1% of firms report that the CDO/CDAO function remains an unsettled role, while only 27.9%
describe the role as being successful and established. Firms are favoring outside change agents
in the CDO/CDAO role – 48.5% in 2020, up from 38.2% in 2019, and are turning away from
insider veterans – down to 15.8% in 2020 from 32.4% in 2019.
Many organizations continue to point to no single point of accountability for data within their
firms, with 26.8% pointing to no single owner, and only 40.2% identifying the CDO/CDAO as
primary executive with data responsibility.
Investment
In recent years, it became evident that investment in Big Data and AI initiatives has become nearly
universal among surveyed firms. That trend continues unabated in 2020, as the percentage of
firms investing in Big Data and AI continues to climb – now reaching 98.8%.
What is noteworthy, however, is that the degree of urgency associated with last year’s investments
in Big Data and AI has appeared to ease up considerably.
While 91.6% of executives reported that Big Data and AI investments were accelerating in 2019,
nearly half – 46.9% -- of those executives now report that these investments are now being
undertaken at a steadier pace.
This may be a case of digesting what firms have already invested in, or may signal a maturing of
Big Data and AI functions within many organizations.
Investment in Big Data and AI initiatives continues to increase. Notably, the percentage of firms
investing greater than $50MM has increased steadily in recent years – from 39.7% in 2018 to
55.0% in 2019, and up sharply to 64.8% in 2020.
The percentage of firms making modest investments under $50MM continues to decline, from
nearly 60.3% in 2018 to a considerably lower 35.2% in 2020.
One clear finding from respondents over the past two years is that Big Data and AI investments
are being driven by “offensive” factors – transformation, innovation, and competitive advantage -
- that are intended to drive revenue growth and business advantage.
Transformation 53.7%
Innovation 26.8%
Competition 8.5%
Cost-Savings 6.1%
Regulation 4.9%
AI/Machine Learning continue to be seen as the most disruptive technology, although it is down
from 96.4% in 2019 to 91.5% in 2020, perhaps echoing the point about AI investment entering into
a more mature phase.
Cloud and Digital investments continue strong, while investment in Blockchain seems to have
faded, down from peak investment levels of 53.4% to less than half – 25.6% two years later.
Overall, the industry appears to be entering a period of maturity and technology absorption.
Business Results
The theme of this year’s survey is “Data-Driven Transformation: Connecting Data/AI to Business
Outcomes”. So, what does the data tell us about the state of the field in 2020?
First, it suggests that firms are realizing measurable results from their Big Data and AI
investments, a pattern that has been holding steady in recent years, after a decrease from 2018 to
2019. Clearly, firms have established business use cases that are demonstrating value.
Second, the data underscores what remains obvious to many – that even with progress in achieving
measurable results, a vast majority of organizations – 73.4% -- still experience business adoption
of Big Data and AI initiatives as a challenge.
Third, organizations are still struggling with making the transformation to becoming Data-Driven.
Note these eye-popping metrics:
▪ Only 26.8% of firms have forged a data culture; 73.2% have yet to achieve this
▪ Only 37.8% of firms have created a data-driven organization; 62.2% have not
▪ Only 45.1% of firms are competing on data and analytics; 54.9% are not
▪ Only half of firms are managing data as a business asset; half are not.
What is most concerning is that these numbers are largely unchanged from 2019 – progress in
these areas remains elusive.
In particular, the trend toward data-driven business transformation has shown no real
improvement over the past 4 years, with the 2020 number being a virtual flat line since 2017.
The principal challenges to becoming data-driven continue to be cultural – people and business
process related – and not about technology (only 9.1%). The greatest barriers to success continue
to be organizational.
Why are so many organizations struggling to successfully tackle the human factors? Perhaps
behavior is difficult to change, or they are not focusing on it as much as technology.
There are causes for optimism – 65.8% report AI in some degree of production; 92.9% see data
& analytics improving or in excellent shape.
Legacy systems still abound in 93.9% of firms -- though they are transitioning – 81.7%. Only 6%
of firms have modern data architectures; there is much work to be done in this regard.
Leadership
As noted in recent years, the role and function of the Chief Data Officer continues to evolve and
expand.
Most notable has been the step by many firms to combine the Chief Data and Chief Analytics
functions into a single role as the Chief Data and Analytics Officer.
We see further evolution in the CDO/CDAO role, as management reporting relationships continue
to evolve, and CDO/CDAO’s seek to gain traction and firmly establish this function.
For purposes of this report, we use the terms Chief Data Officer and Chief Data & Analytics Officer
interchangeably.
For the first time in recent years, we note a decline in the percentage of firms that have appointed
a CDO/CDAO, down from 67.9% in 2019 to 57.3% in 2020, the lowest number in 3 years.
One reason for this discrepancy may be the greater representation of life sciences firms in this
year’s survey. While 63.6% of financial services firms have appointed a CDO/CDAO, only 47.6%
of life sciences firms have filled this role.
Given that life sciences firms are more nascent in their development of mature data management
capabilities and data governance, this should not be surprising. We expect that this number will
increase in the next few years.
We asked this year whether CDO/CDAO’s are focused on what we characterize as “offensive”
activities (revenue generation and innovation) or on “defensive” activities (regulatory and
compliance reporting).
Since that the CDO role originated in the 2009-2012 period for many firms--notably leading banks-
-due to regulatory reporting demands, it is interesting to note the progression from defense to
offense, over half of firms – 54.6% -- now reporting that the CDO/CDAO has shifted to a more
offensive and innovative function.
Adding analytics to the CDO role may have helped with that transition.
Even with this offensive trend, the percentage of firms reporting that the CDO/CDAO has assumed
revenue responsibility is low, at 12.3%, suggesting that “data monetization” remains elusive.
A significant 26.8% of firms report there is still no single point of accountability for data within
their company, perhaps reflecting decentralized ownership of data within lines of business.
This number has not shown much change during the past 3 years, and the percent of CDO/CDAO’s
with primary responsibility for data declined from 48.1% in 2019 to 40.2% in 2020.
Companies continue to struggle to calibrate their expectations and establish a profile that reflects
their vision of a qualified CDO/CDAO, although an increasing percentage of organizations are
coalescing around the perspective on the CDO/CDAO as an external change agent – up from 38.2%
in 2019 to 48.5% in 2020.
The percentage of firms looking to company insiders appears to have fallen sharply from 32.4% in
2019 to just 15.8% in 2020, while the number that believe the CDO/CDAO should own business
results has risen from 13.2% in 2019 to 21.4% in 2020, to represent the 2 nd most sought after
CDO/CDAO qualification.
In contrast to previous years, firms seem to be accepting that the CDO/CDAO should report to
the executive committee – 50.7% -- and not actually sit on the committee – 37.9%.
A minority – 11.4% -- continue to maintain the view that the CDO/CDAO is an interim or
unnecessary role that should be phased out over time.
Finally, we thought it was time to ask firms whether they feel that the CDO/CDAO role has been
a success or not.
In recent years, we have observed anecdotally what appears to be a high rate of turnover in the
CDO/CDAO role, particularly among the leading banks that first established this role.
• a failure on the part of firms to establish clear and achievable expectations for the
CDO/CDAO role;
• the evolving and changing nature of the role and different requirements for it at different
stages of organization evolution and maturity;
• a lack of preparation and necessary skills on the part of the CDO/CDAO;
• a high market demand for CDO/CDAO’s that results in short tenures.
It is worth noting that nearly half of firms – 49.1% -- view the role as being nascent and evolving,
which seems to be consistent with the broader set of answers provided in this survey.
Just over one quarter of firms – 27.9% -- report that the CDO/CDAO is successful and established,
with just under one quarter – 23.0% -- reporting that they are struggling with turnover.
The bottom line based on this year’s survey is that for 72.1% of firms, the CDO/CDAO function
remains an unsettled role -- reflective of the dynamic and disruptive changes that Big Data and
AI have driven over the course of the past decade. The transformation continues.
We serve as trusted advisors to our clients, guiding and assisting them in the processes of:
Since 2001, NewVantage Partners has helped a blue-chip roster of Fortune 1000 companies and
industry leaders leverage data and analytics to drive innovation and business transformation.
Our clients include leading financial services and life sciences firms, and data-intensive firms
across many industries.
NewVantage is headquartered in Boston with offices in New York, San Francisco, Austin, and
Raleigh.
Thought Leadership
Forbes
Demystifying Artificial Intelligence in the Corporation. October 13, 2019.
A Long View on How Data and AI Have Transformed Business Culture. September 25, 2019.
Chief Data Officers Struggle to Make a Business Impact. June 24, 2019.
Contact Information
Randy Bean | CEO | NewVantage Partners LLC | 800 Boylston Street | PO 990465 | Boston, MA 02199
Cell/Direct: 781.789.8794 | Office: 857.991.1404 | [email protected] | www.newvantage.com