Integrating Responsible
AI Practices Into LLMOps.
AUTHORS
Debmalya Biswas, Dipta Chakraborty, Bhargav Kumar Mitra,
Pushpa Ramachandran M
Table Of Contents
Introduction 03
MLOps vs. LLMOps 04
Responsible LLMOps 05
LLMOps Architecture Patterns 05
Black-Box LLM APIs 05
Embedded LLM Apps 06
Fine-tuned LLMs / Domain-specific SLMs 06
Retrieval-Augmented Generation (RAG) 08
A Responsible AI Framework For LLMs 09
Data Quality (Reliability) 12
Model Performance (Reproducibility) 12
Model Explainability 14
Data Privacy (Security) 15
Conclusion 15
Introduction
While Large Language Models (LLMs) have great Here, we introduce LLMOps and explore the
potential to impact enterprise value chains, they relevant LLMOps architectural patterns. We
have so far been mostly limited to consider the relevant dimensions of
proof-of-concept (POCs). To truly scale GenAI use Responsible AI (such as fairness, data
cases, enterprises need to establish scalable privacy, and explainability) and explore how
LLM platforms with LLMOps capabilities and organizations can achieve an integrated
appropriate LLM governance. LLMOps capability governed by Responsible
AI practices.
Responsible AI will be a key aspect of LLMOps.
Without the right guardrails in place, LLMs can
lead to biased, misguided, and even toxic
outputs.
Integrating Responsible AI Practices Into LLMOps | 03
MLOps VS. LLMOps
In an enterprise context, MLOps provides the
tooling and frameworks to scale AI/ML use
cases. MLOps manages the end-to-end ML Data leakage
lifecycle at scale by enabling technology for the
development, deployment, monitoring, and The generative nature of LLMs means that new
ongoing management of ML models. However, content is generated in real-time based on
LLMOps (MLOps for LLMs) is very different from user prompts/responses. These prompts and
MLOps and poses many challenges that are responses can then be used to further train
difficult to address by legacy MLOps LLMs. However, enterprises need to ask: Who
capabilities, such as: owns these conversations, and what privacy
parameters need to be in place? When trained
on conversations that contain sensitive data,
LLMs may run the risk of leaking that data in
the future.
Unstructured data
Supervised (predictive) ML primarily deals
with structured data (in the form of labeled
relational data, time series data, etc.) on
which models are trained. In the case of LLMs,
Hallucination
we are mostly dealing with unstructured data, Enterprises also need to verify that
documents, and multi-modal data consisting proper guardrails are in place to filter
of text, images, audio, and video files. incorrect responses (hallucination)
produced by LLMs.
Pre-trained foundational LLMs Unsupervised LLM programs
Instead of training ML models from scratch, Human feedback loops are an essential part
the most prevalent scenario is to fine-tune of training LLMs and continuously improving
foundational LLMs trained on a large corpus of the quality of LLM responses. Reinforcement
generic data. For teams running LLMOps, learning helps to perform this improvement
experience with traditional ML models does in a targeted fashion by leveraging a rewards
not necessarily translate to success strategy (e.g., Proximal Policy Optimization)
fine-tuning LLMs. to fine-tune the responses.
Integrating Responsible AI Practices Into LLMOps | 04
Responsible LLMOps
GenAI solutions are varied in scope, and we From a Responsible AI perspective, we then
expect them to enter the enterprise landscape identify the key challenges in integrating
via different applications and platforms. Responsible AI dimensions (such as fairness,
Enterprise users may engage with LLM reliability, explainability, privacy, and security)
based-application directly, as in the case of into the outlined GenAI architectural patterns.
ChatGPT. On the other hand, LLMs will also be Explainability, for example, becomes more
embedded within SaaS products and enterprise challenging given the large corpus of (generic)
platform (e.g., Salesforce and ServiceNow). Or training data involved in the case of LLMs. Novel
enterprises might leverage foundational models privacy challenges arise given that user inputs
fine-tuned with enterprise data for specific can potentially be used as training data to
strategic use cases. fine-tune the LLMs. Hallucinations remain the
key issue to be addressed given the generative
Recognizing the diversity of LLM architectures, nature of LLMs.
this report first identifies and outlines the five
most prevalent Gen AI architectural patterns: To address the above challenges, we propose a
black-box LLM APIs, embedded LLM apps, consolidated Responsible AI framework for LLM
fine-tuned LLMs and domain-specific LLMs, platforms, taking into consideration best
Retrieval Augmented Generation (RAG), and AI practices and design patterns that integrate
agents . necessary governance and guardrails at various
stages of the LLMOps pipeline.
LLMOps Architecture Patterns
Black-Box LLM APIs
This is the classic ChatGPT scenario, where Prompts are the primary interaction mechanism
enterprises gain black-box access to a LLM here and can encompass user queries and tasks.
API/UI. Similar LLM APIs can be considered For this reason, prompt engineering is becoming
for other Natural Language Processing (NLP) a full-scale professional discipline. Prompt
core tasks, such as knowledge retrieval, engineers can systematically perform trials,
summarization, auto-correct, translation, record their findings, and arrive at prompts that
and Natural Language Generation (NLG). elicit the optimal contextual responses.
Mobile / Web UI
Prompts
Users
End user Apps Tasks /
Training dataset Large Language Queries
(Public/proprietary) Model (LLM) LLM API
LLM Provider
Fig: LLM APIs
Integrating Responsible AI Practices Into LLMOps | 05
Embedded LLM Apps
LLMs can be embedded within enterprise systems, and especially so for LLMs, which are
platforms such as Salesforce, SAP, and often trained on public datasets. The data usage
ServiceNow. They also underpin the rights for this AI/ML training are often not well
ready-to-use enterprise apps created by LLM defined and will evolve in the future as data
providers such as OpenAI. originators seek to secure and often monetize
their data ownership. For example, Reddit
Enterprise LLM apps have the potential to recently announced that it will begin charging
accelerate LLM adoption by providing an enterprise AI/ML models to train on its massive
enterprise-ready solution. However, enterprises library of user-generated content.
need to approach these applications with the
same caution that should be exercised when Data ownership issues go beyond training data.
using a third-party ML model. That means With embedded LLM apps, enterprises will also
validating LLM/training data ownership, IP, and need to clarify the ownership of input data,
liability clauses and considering potential risks output data, and other generated data. Further, it
like bias and hallucination. will be important to understand and assess how
the enterprise app provider will be capturing data
Data ownership is a particularly important related to user interactions with the application.
dimension. Data is critical for supervised AI/ML
LLM App Store
Mobile / Web UI
Prompts
Users
Training dataset Large Language LLM API Tasks /
(Public/proprietary) Model (LLM) Queries
Enterprise Apps
LLM Provider Enterprise App
Provider
Fig: LLM apps embedded within Enterprise Apps / Platforms
Fine-tuned LLMs / Domain-specific SLMs
LLMs are generic in nature. To realize the full This contextualization is achieved in most cases
potential of language models, enterprises need by fine-tuning a Large Language Model (LLM)
to further train these models with with enterprise data, or by creating a
enterprise-specific knowledge (data flowing domain-specific Small Language Model (SLM).
through the business in the form of documents,
wikis, business processes, etc.).
Integrating Responsible AI Practices Into LLMOps | 06
Select pre-trained Large Language Model (LLM):
• OpenAl • Research
• Google • Hugging Face
• Meta Al • Anthropic
• Microsoft
Specify NLP task:
• QA/Chatbots • Auto-correct
• Extraction • Translation
• Summarization • Classification
Prep enterprise training data:
• Prioritize use-case(s)
• Gather training & validation data
• Annotate data
Select contextualization strategy:
• Fine-tuning
• Retrieval Augmented Generation (RAG)
• Reinforcement Learning with Human Feedback (RLHF)
Deploy enterprise model:
• Taking enterprise data into context
applying selected fine-tuning strategy.
Fig: Enterprise LLM contextualization strategy
Integrating Responsible AI Practices Into LLMOps | 07
Fine-tuning re-trains a pre-trained Large On this front, open-source pre-trained LLMs
Language Model (LLM) on a (smaller) set of present a promising opportunity for many
enterprise data. On a technical level, this means enterprises. Meta, for example, recently
amplifying and regularly updating the weights of open-sourced its LLaMA LLM. The Stanford
the enterprise-specific data layers while Alpaca project showed that it is possible to
freezing the weights of the trained neural fine-tune LLaMA for a mere $600, resulting in a
network’s base data layers. This fine-tuning model performance comparable with ChatGPT. It
yields a LLM more capable of performing is becoming clear that fine-tuning a LLM does not
enterprise-specific tasks. necessarily need to be complex or expensive.
Enterprise
data
Data Processing Knowledge Graphs /
Pipelines Embeddings
(Vector Stores)
Public data
Mobile / Web UI
Supervised fine-tuning
/ Few-shot Learning Prompts
Open Source Pre- Context-specific LLM / Tasks / Users
SLM API
trained LLM Small Language Model (SLM) Queries
End user Apps
Ethical AI Safeguards
Reinforcement Learning from
Model Model Model Human Feedback (RLHF)
Versioning Caching Monitoring
Fig: LLM Fine-tuning with Enterprise data
Integrating Responsible AI Practices Into LLMOps | 08
Retrieval-Augmented Generation (RAG)
Fine-tuning is not always feasible, as many LLMs that are first retrieved using an indexed
do not accommodate fine-tuning. RAG provides a document or vector search, and then provided as
viable alternative to fine-tuning by providing context with the prompts to limit the responses.
additional context via prompts, grounding the Most LLM platforms now allow prompts to be
retrieval/response process to a given context. rather extensive, so it is possible to embed this
This can be in the form of a set of documents enterprise context as part of the prompt.
Training dataset Large Language
LLM API
(Public/proprietary) Model (LLM)
LLM Provider
Mobile / Web UI
Grounded
Prompts
Queries
with relevant
context / data Enterprise Document
End user Apps Users
data Search
Fig: Retrieval-Augmented Generation (RAG)
Integrating Responsible AI Practices Into LLMOps | 09
A Responsible AI Framework For LLMs
The growing adoption of Generative AI, especially organizations to classify how the capabilities of a
LLMs, has reignited the discussion around AI non-deterministic system will impact users and
regulations. Regulators are seeking to ensure systems that might interact with it in the future.
that AI/ML systems are responsibly trained and
deployed. Unfortunately, this effort is Responsible LLMOps aims to ensure ethical AI
complicated by different governmental deployment that avoids harmful applications,
organizations and regulatory bodies releasing mitigates biases to promote fairness, and
their own guidelines and policies with very little maintains transparency about model
agreement on the definitions of terms. Often, development and usage. Data privacy is a priority,
regulatory language is (sometimes intentionally) with strict adherence to regulations to protect
kept at such an abstract level that it is difficult user information. Continuous monitoring of AI
to map to an implementation/enforcement models ensures they perform accurately and
mechanism. For example, the EU AI Act ethically, while user feedback is integrated to
mandates best practices and restrictions based refine and improve the models, ensuring they
on the “risk level” of an AI application. However, meet user needs and expectations. The table
quantifying the risk level of an AI application is below summarizes the key imperatives in
easier said than done, as it depends on multiple implementing Responsible AI for the different
parameters. Quantifying risk requires GenAI architectural patterns.
Responsible AI Fine-tuned LLMs with
Factors LLM APIs
Dimensions LLMs RAG
Reliability Data Achieve consistency Provide consistent Provide consistent
Consistency through few-shot and balanced fetched data from
prompting or other training data. the vector
prompt instructions. database
according to the
use case and
prompting.
Bias/Fairness Avoid training Fine-tune LLMs Provide RAGs with
prebuilt LLMs on with unbiased unbiased data to
data that might data and reduce the chance
perpetuate or generate of biased
amplify harmful business-context responses.
biases. ual synthetic
data using LLMs
to minimize the
risk of biased
responses.
Integrating Responsible AI Practices Into LLMOps | 10
Ensure that
hallucinations are
minimized with
few-shot prompting
Via RAG
and other prompt Reduce
architecture, reduce
instructions. hallucinations to
the hallucination by
Hallucination Because the model some extent by
providing relevant
generates responses re-training the
data specific to use
based on previous model with curated
case and business
Reliability training data, an enterprise data.
domain.
enterprise’s ability
to mitigate
hallucinations is
inherently limited.
Humans should be in the loop while training LLMs (or during the
build phase) to ensure that the output of the model can be
Accountability
verified before deployment. Human feedback can then be
leveraged for continuous improvement of the model.
Evaluate LLM performance either through manual testing
Evaluation
(keeping humans in the loop) or using statistical measures. There
during
are different statistical measures available to evaluate the
Training
performance of LLMs: Perplexity, BLEU, ROGUE, etc.
Reproducibility
Measure LLM performance while handling incoming requests,
Inference using evaluations such as completed requests per minute, time to
Evaluation first token (TTFT), inter-token latency (ITL), and end-to-end
latency.
RAG plus CoT
Chain of Thought Chain of Thought prompting can
(CoT) prompting can (CoT) prompting can provide the logic to
Explainability Evidence be used to ascertain be used to ascertain the LLM response. It
the logic behind the the logic behind the can also provide the
LLM response. LLM response. evidence link with
the response.
Leverage secure hyperscaler cloud platforms (e.g., Azure, AWS,
Data GCP) to train and deploy LLMs. These platforms provide the
Guardrails necessary landing zones, infrastructure, and tooling to securely
host both data and models.
Security
Configure context-aware access controls for the data used by
Access
LLMs as per the use case. Cloud platforms can provide a default
Control
standard protocol to access the data used in LLMs.
Below, we expand on how each of these four dimensions of Responsible AI governance can be implemented
in the LLMOps pipeline.
Integrating Responsible AI Practices Into LLMOps | 11
Data Quality (Reliability)
With respect to data quality/reliability, engineering, data reliability is of utter
enterprises need to consider how various importance. Data reliability is the foundation of
dimensions of data quality can enhance the trustworthy LLM outputs. The following data
reliability of the overall LLM ecosystem. Whether quality checks should be considered when
training a LLM or actively using a LLM via prompt seeking to use or train LLMs:
Data Consistency
To ensure accurate and precise LLM outputs, the data used for training (and
particularly fine-tuning) a LLM should be relevant to the specific use case.
For example, if the use case is to generate a summary of a medical
prescription, the LLM should only be trained on medical prescriptions and
corresponding summarizations of the prescriptions, not unrelated diagnostic
information. Extra caution should be exercised when incorporating
time-related data, ensuring that data is temporally relevant and refreshed at
an appropriate frequency. Often, data pipelines will need to be created to
automatically ingest data and feed that data to LLMs. In such scenarios, extra
caution needs to be exercised while consuming running text fields, as these
fields may contain a significant amount of inconsistent and incorrect data.
Bias/Fairness
In terms of model performance and reliability, controlling undesired biases
in black-box LLMs is challenging. However, this can be mitigated to some
extent by using fair and unbiased data for fine-tuning and/or generating
business-contextual synthetic data with LLMs to reduce the likelihood of
biased responses. Additionally, contextualizing the LLMs within a RAG
architecture can further help manage biases.
Hallucination
When using LLM APIs or orchestrating multiple LLMs, hallucination likelihood
increases due to a lack of guardrails. The right prompts can help, but only to a
limited extent. To further limit hallucinations, LLMs need to be fine-tuned
with curated data and/or limited to relevant and recent enterprise data.
Accountability
To make LLMs more reliable, we recommend that humans validate
LLMs outputs. Involving a human ensures that if LLMs hallucinate or
provide wrong response, a human can evaluate it and make it correct.
Integrating Responsible AI Practices Into LLMOps | 12
Data Quality Case Study
Working with an airport client, Wipro created a Initially, the baseline LLM tended to provide
chatbot that provides real-time information on wrong (i.e. hallucinated) flight numbers and
flight status and ticket availability. User can statuses. Following a RAG implementation that
check their flight status as well as their ticket automatically embedded the PNR number,
status based on their Passenger Name Record passenger name, and flight number in the prompt
(PNR). In this context, it is extremely important process, the LLM was able to consistently
to provide correct feedback to the Users. provide accurate flight information based on
natural language prompts from users.
Model Performance (Reproducibility)
With respect to model both the training and the inferencing phases to
performance/reproducibility, it is important to evaluate model performance.
measure the performance of the model during
Model Evaluation during Training
To ensure the performance of the model, organizations should always
measure the performance of the model during the training phase before
deploying it to the production. There are several metrics that organizations
can use to statistically quantify model performance, including:
Perplexity
This quantifies how well a model predicts the next piece of text
in a sequence. The lower the score, better the model.
BLEU (Bilingual Evaluation Understudy)
BLEU is a metric commonly used in machine translation tasks. It
compares the generated output with one or more reference
translations and measures the similarity between them. The
higher the score, better the model.
ROUGE (Recall-Oriented Understudy for Gissing
Evaluation
ROUGE is a set of metrics used to evaluate the quality of
summaries. It compares the generated summary with one or more
reference summaries and calculates precision, recall, and F1
score. The higher the F1 score, the better the model.
Human Evaluation
Last but not least, human evaluation or manual testing of the
model is essential to check the performance of the model with different
scenarios.
Integrating Responsible AI Practices Into LLMOps | 13
Model Performance during Inferencing
Monitoring model performance during live run is essential for ensuring
reliability of the model. Model performance can be monitored in two ways:
1) measuring the quality of the model outcome and 2) checking model
performance in handling request.
Regarding quality of model outcomes
Regular monitoring of the model outcome is essential to make the
model more robust and reproducible. Evaluations like Perplexity,
BLEU, ROGUE, and others can be used to measure the model
performance. If there is any significant drift in model
performance, the model will need to be retrained from the last
verified checkpoint. In cases of drift, the model may have
forgotten what it learned previously and be over-indexing toward
the most recent training data.
Checking Model Performance in Handling
Incoming Request
Model performance during production (i.e. handling request/handling
incoming request) is the second critical aspect of model robustness and
reproducibility. Metrics that can be used to evaluate model performance
include completed requests per minute, time to first token (TTFT),
inter-token latency (ITL), and end-to-end latency (end-to-end latency
should be approximately the same as the average output length of tokens
multiplied by the inter-token latency).
Model Performance Case Study
User Input
A food and beverages company
What are the general audience’s thoughts on the
aimed to refine its product
taste and texture of our new chocolate candy?
development process for its
chocolate candy line by leveraging
customer feedback from social LLM Output
media. Wipro developed a solution
that utilizes a Large Language Model For your new chocolate candy, users are particularly
(LLM) to analyze and summarize user impressed with its rich, creamy texture and the
responses, blending these insights balance of sweetness. However, some feedback
with product features to guide future suggests that the candy could benefit from a wider
product enhancements. Consistently variety of flavors and a more eco-friendly packaging.
high BLEU and ROUGE scores To enhance your product, consider introducing new
reflected the accuracy and relevance flavor combinations and using sustainable
of the generated summaries. packaging materials. Additionally, conducting taste
tests and gathering direct feedback from customers
can provide more targeted insights for product
improvements.
Integrating Responsible AI Practices Into LLMOps | 14
Model Explainability
Explainability is an umbrella term for a range of models, enterprises should prioritize them highly.
tools, algorithms, and methods that accompany An enterprise should be able to explain, for
AI model inferences with explanations. While example, why a ML model categorized a user in a
explainability and transparency are particular user segment for advertising purposes.
characterized as “non-functional” aspects of AI
Chain of Thought (CoT)
is a framework that can be leveraged to understand how a LLM is solving a problem. CoT can be implemented
in two ways: user prompting and automated prompting.
User Prompting
Here, a user manually prompts the model to provide the logic about how it
approaches a certain problem. The LLM has the ability to understand the
instruction and provide the output as per the logic provided in the prompt.
Automated Chain of Thought Prompting
Manually handcrafting CoT can be time consuming and, in the long
term, is a sub-optimal solution. Automatic Chain of Thought
(Auto-CoT) can be leveraged to generate the reasoning chains
automatically, thus eliminating the human intervention. An
Auto-CoT process can be built by combining tactics such as
question clustering (clustering the questions of a given dataset)
and demonstration sampling (selecting a representative question
from each cluster and generating a reasoning chain). Auto-CoT
works well for LLMs having approximately 100B parameters or
more, but is not as accurate for small-scale LLMs.
Model Explainability Case Study
User Input
In the context of medical insurance,
What is the status of claim ID 123456789?
it is critical to ensure that LLM
outputs are consistent with legally
binding member agreements. Wipro
created a chatbot for a medical LLM Output
insurance company that allows user
to check the status of a claim by Upon checking, we can confirm that your claim is
providing the claim ID. The chatbot approved and 80% of the claim amount is finalized
returns not only the status of the for payment in accordance to the condition
claim, but also a verifiable reference mentioned in the policy terms section 1.5.
to the relevant policy language, which
ensures explainability.
Integrating Responsible AI Practices Into LLMOps | 15
Data Privacy (Security)
With respect to data privacy, enterprises need to Cloud providers and hyperscalers providing LLMs
consider the privacy aspects of both enterprise data and enabling their fine-tuning on enterprise data
used to fine-tune the LLMs and enterprise data can provide the necessary setup/landing zone for
provided as context (RAGs). In addition, one novel data privacy and access controls access to the
privacy aspect of GenAI is the need to consider the data for specific use cases.
privacy risks of data (prompts) provided voluntarily
by the end users, which can potentially be used as
training data to re-train/fine-tune LLMs.
Conclusion
Gen AI is a disruptive technology, and we are We have proposed a LLMOps framework that
seeing it evolve faster than any technology optimizes tooling and frameworks to deliver
innovation curve we have experienced before. As responsible, governed LLM capabilities. We
such, it is critical for organizations to believe that this framework will accelerate LLM
understand the key architectural patterns adoption and enable enterprises to scale GenAI
entailed in scaling GenAI proofs-of-concept into use cases in a responsible fashion. Responsible
production. A mature LLMOps strategy will be LLMOps effectively future-proofs GenAI
critical to both accelerating innovation and investments and ensures that LLM platforms can
mitigating risk. Large enterprises will cope with a GenAI landscape that will bring new
increasingly need to manage the mix of LLMs and new training/deployment architectures
proprietary, open-source, and fine-tuned LLMs. at a rapid pace.
Across all GenAI architectures, LLMOps should
ensure that LLMs are being used efficiently and
responsibly.
Integrating Responsible AI Practices Into LLMOps | 16
Wipro Limited Wipro Limited (NYSE: WIT, clients realize their boldest
Doddakannelli BSE: 507685, NSE: WIPRO) is a ambitions and build future-ready,
Sarjapur Road leading technology services and sustainable businesses. With over
Bengaluru – 560 035 consulting company focused on 230,000 employees and business
India building innovative solutions that partners across 65 countries, we
address clients’ most complex deliver on the promise of helping our
Tel: +91 (80) 2844 0011 digital transformation needs. clients, colleagues, and communities
Fax: +91 (80) 2844 0256 thrive in an ever-changing world.
[Link] Leveraging our holistic portfolio of
capabilities in consulting, design, For additional information,
engineering, and operations, we help visit us at [Link]
GEO/AGENCY CODE/VALIDITY
