The State Of Generative AI, 2024

TRENDS REPORT

The State Of Generative AI, 2024

January 25, 2024

ZK Zeid Khater

with Aaron Katz, Srividya Sridharan, Brian Hopkins, Rowan Curran, J. P. Gownder, Evan Megan

Summary

Generative AI (genAI) technology promises to transform how enterprises operate, how employees do
their jobs, and how consumers interact with brands. We’re in the early stages of genAI technology, and
there are many questions around its use cases, risks, and real-world impact. Enterprises are balancing
opportunity with apprehension; most are experimenting with genAI in their workflows, and many are
developing production systems based on solutions from tech companies and service providers. This
report explores the current state of generative AI from the demand and supply angles, how companies
are adopting it, and what factors enterprises should consider when preparing to implement it.

Topics

The State Of Generative AI: … Generative AI Use Cases Ar… Prepare For Generative AI …

The State Of Generative AI: Wave Of Disruption Coming

Generative AI is shifting the paradigm of digital technology away from experiences based on
pages, sites, and search terms and toward experiences based purely on natural language. It
promises to transform how we interact with technology itself. Forrester expects that genAI will add
convenience to and remove friction from a variety of experiences, reshape jobs in ways we are only
beginning to contemplate, and disrupt organizations and industries.

However, it’s still early days. Broad confusion about and misunderstanding of the technology
coexists with widespread experimentation; near-daily announcements of new features,
investments, and partnerships; and hype from tech vendors about capabilities that may be months
or years away. And there are still meaningful obstacles to genAI adoption for enterprises,
particularly for externally oriented (i.e., customer-facing) use cases. These include questions about
the quality of training data; the use of copyrighted material in training and outputs; model and data
bias; frequent model “hallucinations” and inaccuracies; and data security concerns. Even so,
enterprises are moving aggressively from education and exploration to experimentation and
adoption, requiring new approaches to assessing business value and strong governance to ensure
ethical and safe use.

A Small Number Of Key Players Dominate LLMs

Large language models (LLMs) like OpenAI’s GPT series have been at the center of the generative
AI discourse (see Figure 1). Although LLMs are only one segment of a broad spectrum of generative
 models, their impact has been undeniably groundbreaking. While LLMs are just the beginning of
The State Of Generative AI, 2024
the genAI transformation, they dominate the conversation today. A growing set of open source
players and service providers also have their own proprietary models. The supply side of genAI is
characterized by:

A small group of leading tech companies. Foundation LLMs require almost unimaginable
volumes of data (GPT-4 is trained on more than a petabyte of data), years of development, and
millions of dollars of infrastructure, which limits the number of competitors. As a result, the LLM
space is dominated by a handful of vendors that have made long-term investments in a
foundation model, including Open AI’s GPT models, Meta’s LLaMa, and Google’s Bard for text
and Stable Diffusion, Open AI’s Dall-E, and Midjourney for images. Some foundation models
(e.g., LLaMa) are open source, so developers have access to the code and can modify and
extend it; others (e.g., GPT-4) can be licensed, but the source code is not publicly available.

Generative AI service providers. Large service providers such as Accenture, Deloitte,

McKinsey, and PwC have built proprietary genAI solutions to serve enterprises. These firms
typically leverage public LLMs, open source models, and additional data and technologies to
build broader AI solutions that meet the specific needs of clients. They act as intermediaries,
transforming high-level AI research into practical customized applications for enterprises.

A broad group of vendors offering some genAI features. Tech vendors of all stripes are finding
ways to integrate LLM capabilities into their products and services. Most vendors’ genAI
features are a thin wrapper empowering more efficient input and output in software,
leveraging the technology’s ability to summarize, translate, and create content. Vendors of call
center and customer experience software use genAI to provide more natural interactions with
chatbots; creators of software development tools use it to write and QA code; and creative
industries use genAI-based tools to produce text and images at scale.

Figure 1

Three Broad Tiers Of Generative AI Suppliers

Plans To Adopt Generative AI Are Nearly Universal

According to Forrester’s 2023 data, over 90% of global enterprise AI decision-makers have
concrete plans to implement generative AI for internal and customer-facing use cases (see Figures
2 and 3). While production use cases are limited to a few sophisticated organizations, enterprises
expect broad value from genAI, with productivity as the leading benefit (47%) and innovation and
cost efficiencies close behind (see Figure 4). Most enterprises have yet to realize bottom-line
benefits from genAI; the technology is too new to have many proven cases where genAI in
 production workflows is definitively linked to growth. Given the complexity, spotty explainability,
The State Of Generative AI, 2024
and unpredictability of genAI models, firms are taking a cautious approach that:

Starts with internal use cases. Using generative AI internally for employee productivity and
workflow optimization allows firms to test and refine their models in a controlled environment.
The top three use cases — employee productivity, knowledge management, and software
development — all focus on internal improvements, where knowledge management bots can
accelerate workflows, automate tasks, and generate new ideas and innovation by providing
inspiration at scale and TuringBots can vastly improve developer productivity.

Moves slowly into customer-facing and other external applications. Once companies are
successfully using genAI internally and understand how to manage and govern genAI
applications, they slowly push outward to test new ways to enhance customer experiences via
conversational AI, chatbots, and virtual assistants. Most of these external deployments employ
heavy human-in-the-loop management and validation — at least for now.

Figure 2

Most Organizations Are Using Or Planning To Use Generative AI

Figure 3

Organizations Follow Multiple Approaches To Generative AI Adoption

Figure 4

Users Of Generative AI See A Variety Of Benefits

The State Of Generative AI, 2024

Risks Are Still Top Of Mind For Many AI Skeptics

Widespread enterprise adoption of generative AI still faces significant obstacles. Among AI

decision-makers, 30% say that a lack of technical skills to use genAI is one of their biggest barriers
to adopting it; 28% cite difficulty integrating genAI with existing infrastructure. About as many cite
data privacy and security concerns, governance and risk challenges, and (lack of) employee
readiness. Moreover, many enterprises are waiting for more mature regulatory frameworks and
clarity on the relevance of foundation models to specific industry verticals or corporate functions
before putting genAI applications into production. While generative AI promises huge benefits, the
technology still has challenges of:

Hallucination, error, and bias. Despite dramatic improvements in accuracy over the past year,
public LLMs regularly produce results that are irrelevant or flat wrong. Because their training
data sets are weighted toward publicly available internet data, their outputs will reflect biases
and misinformation common on the internet. Moreover, LLMs and other generative models are
probabilistic, not deterministic. They don’t reason or fact-check; they use millions of
parameters and billions of data points to predict what word is most likely to come next based
on a prompt. Fortunately, new tools and techniques are emerging to help reduce the
occurrence of problematic outputs considerably.

Weak or absent explainability. Enterprises need explainability to be able to troubleshoot when

a model produces undesirable results or because regulators require it. With most traditional AI
models, it’s possible to reverse-engineer how the model produced an output or build an
explainable version of an opaque model. This is not the case with generative models; even
experts still don’t know exactly how they produce results. No one can perfectly predict what
results a generative model will produce or explain why it produced a given result. It’s uncertain
whether explainability is even possible for genAI applications.

Privacy and regulatory concerns. Firms have legitimate questions about the incorporation of
their data (or even their prompts) into LLM training data sets and how proprietary company or
customer data is treated when fed into an LLM. Some large enterprises, particularly those in
heavily regulated industries, are exercising extreme caution to protect company and customer
data; many ban tools like ChatGPT outright over concerns about data protection and
regulatory backlash. Others are leveraging genAI in data-safe ways (e.g., using private
enterprise instances of ChatGPT) as more options become available.

IP and copyright uncertainty. The legal landscape around generative AI is marked by

uncertainty, particularly concerning copyright infringement and the ownership of AI-generated
content. Recent lawsuits against major firms like Microsoft and OpenAI have intensified these
concerns. Courts and law firms are navigating these challenges, but no clear consensus has
emerged on how intellectual property (IP) laws apply to genAI.
The State Of Generative AI, 2024
Generative AI Use Cases Are Diverse
Early adopters report they are implementing generative AI in diverse use cases across operations,
customer engagement, and product development (see Figure 5). The areas of biggest immediate
impact are employee productivity, customer support, and coding, where genAI’s ability to
streamline workflows with natural language answers to common questions and to automate
repetitive tasks and uncover prior art (particularly in application development) can be used with less
risk (see Figure 6).

Figure 5

Internal And External Use Cases

Figure 6

Generative AI Heat Map

Prepare For Generative AI With Governance And

Experimentation
To prepare for generative AI, leaders need to take several steps now, even if the enterprise has no
immediate plans to build custom AI applications. AI is coming into your organization whether you
know it or not. Your employees have access to free AI tools that they may be using in their work,
and your IT and software partners are building genAI capabilities into the applications deployed in
your on-premises and cloud environments. This is the time to start building in-house skills: Forrester
sees genAI as a key enabler of most future enterprise applications, so experimenting with internal
use cases and building your AI muscles will ensure that you aren’t blindsided by rapid
developments and disruptions. The first steps toward generative AI success are:
 Setting AI,
The State Of Generative governance
2024 guidelines for BYOAI. Anyone can log into a free version of ChatGPT and
copy data into the prompt window, raising potential risks ranging from inaccurate output to a
data privacy breach. Leaders need to immediately establish clear guidelines around the use of
bring your own AI (BYOAI), including limiting access to various tools; specifying a preferred
company or enterprise toolset; or defining the circumstances under which using genAI is fair
game, what data may be used in a prompt window, what is permissible on employee-owned or
company-owned devices, and how the enterprise will track and report usage, costs, and
outputs. Well-defined BYOAI policies will empower employees to innovate while ensuring
proper oversight for security, ethics, and compliance.

Establishing standards for evaluating genAI in vendor solutions. Most of the AI operating in
your organization wasn’t built by you — it’s embedded in third-party software and solutions.
You should evaluate and govern AI that is part of a vendor’s capabilities, just as you would
with a homegrown application. The risks — such as hallucinations, bias, and data privacy
breaches — are the same regardless of the source. When assessing a vendor’s genAI
products, know what the must-haves are for your enterprise. You may need to examine the
vendor’s data sourcing and content curation practices, data privacy protections, human review
and training processes, model performance and oversight, and ability to customize training
data sets and fine-tune models responsibly. Ensure that vendors comply with your enterprise
standards before you sign that contract.

Updating your AI strategy with standards and guardrails for implementing genAI. As
generative AI advances and enterprises tackle more complex and risky use cases, leaders
must refresh their AI strategy and governance to incorporate appropriate standards,
principles, and guardrails for ethical and responsible use. Address bias, misinformation, and IP
protection via policies and controls tailored to generative models. Many firms have
enterprisewide guidelines to manage hidden or “shadow” AI, which is the use of AI
applications by employees to enhance or accelerate their work without their employer’s
knowledge. Accepting this as reality — or at least a strong possibility — should spur you to
create guidelines for safe usage. Guidelines that have surfaced so far provide whitelists of
approved applications, blacklists of data types to avoid introducing to the models, and
limitations on work functions that need to stay in the realm of “human intelligence” until
governance can catch up.

Identifying use cases that can deliver value. Given the pace of change with generative AI,
there are many unproven use cases. Prudent leaders will focus on high-value applications with
demonstrable impact rather than deploying unproven use cases. Prioritizing solutions that
improve core workflows and provide clear productivity gains will ensure judicious adoption
amid great hype.

© 2024, Forrester Research, Inc. and/or its subsidiaries. All rights reserved.