ARTIFICIAL INTELLIGENCE 101 SERIES:

ChatGPT
for
Internal
Auditors
Use cases, sample prompts, and key
considerations when using Natural
Language Processing tools.

Check out our

“Artificial Intelligence 101
for Internal Auditors” for a
primer on AI technology
 Table of Contents
Introduction....................................................................................................2

What Is Natural Language Processing?......................................................................3

• Definition..................................................................................................3
• Use in the Marketplace..................................................................................3
• ChatGPT Alternatives....................................................................................3

Common Risks Associated With the Use of Publicly Available AI Tools................................ 4

Use Cases.......................................................................................................5
• Example 1: General Audit Plan............................................................................5
• Example 2: General Audit Plan.........................................................................10
• Example 3: Structure Unstructured Data.............................................................13
• Example 4: Write Your Audit Report.................................................................. 14

Introduction
By November 2023, over 100 million people can internal auditors. This AI 101 guide will
globally were regularly using ChatGPT. provide novice AI users with use cases and
Businesses and individuals have praised the recommendations for how they can
AI tool for its ability to save time spent on incorporate ChatGPT-style tools into their
manual, time-consuming activities, using practice.
it for everything from writing emails and However, as with any technology, there are
summarizing documents to developing code, both risks and rewards — and the potential
Excel shortcuts, and PowerPoint presentations. risks associated with publicly available tools
Because of these and other benefits, cannot be overlooked. As always, internal
organizations across all professions and auditors should remain vigilant about the
industries are using ChatGPT and other inherent risks
Natural Language Processing (NLP) tools — and diligent about the controls in place to avoid,
and so share, accept, or mitigate those risks.

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 2

What Is Natural Language Processing?
Natural Language Processing (NLP) is a type of
artificial intelligence that gives a machine the ability to How are NLP tools like ChatGPT
understand written and spoken words in the same way used today?
that humans do. While each specific type of NLP may
Natural Language Processing-based tools are already
be programmed to focus on different skills, in general
integrated into many aspects of our everyday lives.
this type of AI is capable of:
For example, NLP is used in:
• Speech recognition – identifying when a human
is speaking and what words are being said • Google Translate
• Grammatical tagging – identifying parts of • Email filters (Gmail’s primary, social, or
speech, such as nouns, verbs, or adjectives promotions categories)
• Sentiment analysis – determining themes in • Chatbots and virtual agents like Apple Siri
data, such as whether customer responses are and Amazon Alexa
positive, negative, or neutral
• Search results and predictive text
• Text summarization – reducing the numbers
• Speech recognition software (voice-to-text)
of words in data without changing its
meaning, or providing a synopsis of other
data (a video, for example)
Do alternatives to OpenAI’s
• Natural language generation – putting
ChatGPT exist?
information into new text or speech, such as the The use cases in this resource relied on ChatGPT
content produced by ChatGPT or Apple Siri 4.0. However, there are numerous alternatives to
ChatGPT, each with unique features and
capabilities. Below is a small sample of commonly
used ones:

In addition to being a 1. Google BARD – BARD is a conversational AI

type of NLP, ChatGPT is chatbot that ties into the popular search
more specifically a engine.
Generative Pre-trained 2. Microsoft Bing AI – Bing’s AI-slash-search
Transformer (GPT). It is engine integrates with and has support for
designed to handle other Microsoft applications, allowing it to
sequential data, such as integrate items such as search and chat
language, and will histories into its responses.
generate text based on the input it receives. GPT
3. Amazon Lex – This chatbot by Amazon Web
models don’t only copy and paste — they can
Services allows users to build conversational
create brand new content. This content is based
interfaces into applications using voice and
on all the data it was initially trained on, as well
text. It is adept at automatic speech
as new data it received. This means that the
recognition (ASR)
model will learn from — and “keep” — any
for converting speech to text and natural language
information you give it. Internal auditors and the
understanding (NLU) to recognize user intent.
businesses they represent must be cautious not
to provide proprietary data in “conversations” 4. IBM Watson Assistant – This tool is designed to
with NLP models like ChatGPT. understand and respond to customer inquiries.
It can be trained to provide information specific
Some organizations are beginning to develop to an industry or company.
private, or closed, GPT models. In those cases,
the information entered into the system does not 5. Jasper – This AI is used often to develop
become publicly available. marketing and communications content,
including blogs, social media profiles, and
websites.

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 3

 6. CopyAI – CopyAI is another tool often
used for developing marketing and
communications content for
organizations.

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 4

Common Risks Associated With the
Use of Publicly Available AI Tools
RISK EXPLANATION SAMPLE MITIGATION STEPS
Data is inaccurate due to a biased The algorithm has been trained on data
algorithm that was collected, classified, or otherwise
subjected to biases
Don’t accept AI-generated
Data has been “poisoned” An AI tool has been deliberately given responses as completely
inaccurate, biased, or otherwise bad accurate; apply professional
information, which it then uses in developing skepticism and verify all
responses information
Users place too much reliance Internal auditors or other staff blindly “copy
on AI-generated content and paste” information from the tool
Confidential information is Users enter information into the chat that Remove any potential
provided in the prompts is not meant for public consumption identifying information from
prompts

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 5

USE CASES:

Examples of How to Safely

Incorporate ChatGPT-Style Tools in
Your Practice
EXAMPLE 1: Use it to help you think through your general audit
plan
are clear delineations of roles and
Prompt: Given The Institute of Internal Auditors’ responsibilities concerning AI development,
existing standards for internal auditing, what would you implementation, and monitoring.
recommend a Chief Audit Executive include in an audit
plan designed to audit the appropriate use of artificial
intelligence within their organization?

PRO TIP! Ask generalized questions.

Never give identifiable or proprietary
information to an NLP tool available for
public use. If you think it could be used
to identify your company, remove it.

Actual ChatGPT 4.0 Response: Auditing the appropriate

use of artificial intelligence (AI) within an organization is
a relatively new but increasingly important frontier.
Given The IIA’s standards for internal auditing, a Chief
Audit Executive (CAE) should consider the following
components when developing an audit plan for AI:
1. Risk Assessment:
• Identification of AI Systems: Catalog all AI-
driven systems and applications in the
organization.
• Risk Prioritization: Evaluate the criticality
and potential impact of each AI system on
the organization. Systems with higher
potential impact (financial, operational,
reputational) should receive priority.

2. Governance and Oversight:

• AI Strategy: Review the organization’s AI
strategy and ensure alignment with
organizational goals.
• Roles and Responsibilities: Check that there
Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 6
3. Data Integrity and Quality:
• Source Data: Assess the accuracy,
completeness, and timeliness of the data
feeding into the AI models.
• Data Privacy and Security: Ensure that the
AI systems comply with data protection laws
and that sensitive data is appropriately
protected.
• Bias Detection: Audit for biases in the data
that might result in unfair or discriminatory
outcomes.

4. AI Model Validation:
• Model Development and Training: Assess the
methodologies used to develop and train AI
models for appropriateness and scientific
validity.
• Testing and Calibration: Ensure rigorous
testing and calibration of models before
deployment and periodically thereafter.
• Model Documentation: Review documentation
for thoroughness — including assumptions
made, data sources, algorithms used, and
decision rationales.

5. Transparency and Explainability:

• Interpretability: Check if the AI model’s
decisions can be explained and understood
by stakeholders. This is especially crucial for
AI systems used in decision-making that
affects individuals, like hiring or loan
approvals.

PRO TIP! Throughout the entire

engagement, make sure you are
scrutinizing the responses and applying
reasonable skepticism. The responses
provided should be used as a general guide and
to help provoke your own thoughts; they should
not be used as authoritative text.

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 7

 • Reporting: Review reporting mechanisms for 11. Third-Party Management (if
AI decisions, ensuring they are transparent the AI system is outsourced or built in
and provide appropriate information to partnership):
relevant stakeholders.
• Vendor Due Diligence: Audit third-party
6. Operational Controls: providers for their AI capabilities, data
management practices, and compliance with
• Change Management: Assess the processes
relevant standards.
for updating or altering AI models and
systems. • Contractual Safeguards: Ensure contracts
with third parties have clear terms regarding
• Incident Management: Ensure there are
data usage, AI model transparency, and other
mechanisms for capturing, reporting,
critical factors.
and addressing anomalies or errors in AI-
driven decisions. Regularly updating the audit plan to reflect advances in
AI technology and methodologies will also be crucial.
7. Ethical Considerations: The rapid evolution of AI means that what is considered
• Ethical Guidelines: Ensure AI deployments a best practice now might be outdated in just a few
align with ethical guidelines and principles, years.
both internally set and recognized globally.
• Stakeholder Impact Analysis: Analyze potential
adverse impacts of AI decisions on Prompt: You suggest that the audit plan should audit
stakeholders and ensure that there are for biases in data that might result in unfair or
mitigation strategies. discriminatory outcomes. How might the Chief Audit
Executive audit for such biases? What specific steps
8. Performance Monitoring: should the auditor take?
• Ongoing Monitoring: Audit processes
for continuous monitoring of AI
systems’ performance against defined
metrics.
PRO TIP! Ask a clarifying question or
request a more detailed response based
• Feedback Loops: Ensure that there are upon the information given (or not
mechanisms for learning from inaccuracies given).
or failures and that these insights are used
to improve AI systems.
Actual ChatGPT 4.0 Response: Auditing for biases in
9. Regulatory and Compliance: data and AI models is a challenging yet essential task,
especially as businesses and organizations increasingly
• Legal Requirements: Ensure AI
rely on AI for decision-making. Biased decisions can lead
applications are in compliance with any
to significant financial, reputational, and legal risks.
sector-specific or jurisdiction-specific
Here’s a step-by-step approach a Chief Audit Executive
regulations.
(CAE) might take:
• External Reporting: Review processes for
1. Define What Bias Looks Like:
external reporting, if required, ensuring
accuracy and timeliness. • Understand the business and regulatory
context. Different industries or sectors might
10.Stakeholder Communication: have specific definitions or concerns related to
bias.
• Training and Awareness: Assess training
programs for employees and stakeholders • Consult with stakeholders, including legal,
affected by AI decisions to ensure ethical, and diversity and inclusion experts, to
understanding and effective use. define clear benchmarks or indicators of bias.
• Feedback Channels: Ensure stakeholders
2. Identify High-Risk Areas:
have clear channels to provide feedback or
raise concerns about AI-driven decisions. • Prioritize AI systems and data sets that have a
more significant impact on individuals or

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 8

 groups, such as hiring systems, lending
algorithms, or healthcare diagnostics.

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 9

 • Understand the historical context: Some 8. Evaluate Mitigation Measures:
industries or sectors have historical biases • Review any fairness interventions or corrective
that might seep into AI models if not measures the organization has already
corrected. implemented in the AI model.

3. Assess Data Collection Methods:

• Review the sources of the data. Is it from a
diverse and representative sample, or does
it disproportionately represent certain
groups?
• Examine how data was cleaned and
pre- processed. Data cleaning can
sometimes unintentionally introduce
biases.

4. Conduct Statistical Analysis:

• Use statistical tools to identify patterns in
the data. For instance, disproportionate
outcomes for certain groups might indicate
bias.
• Consider using fairness-enhancing interventions
and techniques to mitigate identified biases in
data.

5. Test the Model:

• Use adversarial testing: Intentionally input
data that challenges the AI model to see if it
produces biased outcomes.
• Compare model decisions across different
groups to identify any disproportionate
impacts. For instance, compare approval rates
between different ethnic or gender groups.

6. Review Model Training and Development:

• Understand the objectives and optimization
criteria of the AI model. If a model is
optimized for profit maximization without
fairness constraints, it may produce biased
outcomes.
• Examine the model’s feedback loops. If a
model is continually learning from biased
decisions, it can reinforce and exacerbate
those biases.

7. Engage External Experts:

• Consider bringing in third-party experts
specializing in AI fairness and bias to review
the model and its outcomes. This can
provide an objective perspective and
validate internal findings.

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 10

 measures are valuable, they must be combined with a
contextual
understanding of the AI system’s impact on individuals
and groups to effectively audit for biases.

• Check if these measures align with best

practices and industry standards.

9. Review Documentation and Transparency:

• Ensure that there’s comprehensive
documentation about the AI model’s
development, training, objectives, and any
known limitations. This can help provide
context and might highlight areas where
bias was introduced.
• Check if explanations for AI decisions
(especially those impacting individuals)
are clear, understandable, and free
from bias.

10. Stakeholder Feedback:

• Engage with end-users and those
affected by the AI decisions. Feedback
from these groups can provide valuable
insights into perceived and actual
biases.

11. Continuous Monitoring:

• Bias detection is not a one-off activity.
Regularly audit and monitor the AI
model’s decisions to ensure that biases
do not creep in over time.

12. Report and Recommend:

• Document findings, potential risks, and
provide recommendations for
rectification. Ensure that senior
management is aware of any significant
risks related to bias.
• Advocate for training and awareness
programs within the organization to
ensure everyone understands the
importance of unbiased AI systems.

Remember, while statistical and technical

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 11
 Steps to Take
Prompt: What questions should I ask or what steps
should I take to analyze the data for inherent biases? 1. Statistical Analysis:
• Conduct statistical tests to identify any
disparities in data distribution across different
groups.
PRO TIP! Continue to ask • Look for correlations that might indicate
clarifying and/or refining biases, especially involving sensitive
questions. attributes.

2. Data Segmentation:
Actual ChatGPT 4.0 Response: Analyzing data for
• Segment the data based on various
inherent biases is a crucial step in ensuring fairness and
demographics and analyze each segment
ethical use of AI systems. Here are questions to ask and
to identify disparities.
steps to take for a thorough analysis:
• This helps in understanding how different
Questions to Ask groups are represented and treated in the
1. Is the Data Representative? dataset.
• Does the dataset adequately represent all groups,
3. Bias Detection Algorithms:
especially those that are historically marginalized
or underrepresented? • Use specialized algorithms designed to
detect biases in datasets.
• Are there any groups that are overrepresented
or underrepresented? • Tools and frameworks for fairness assessment
(like IBM’s AI Fairness 360, Google’s What-If
2. Are There Proxy Variables? Tool) can be helpful.
• Does the dataset contain variables that
4. Consultation With Subject Matter Experts:
could act as proxies for sensitive attributes
(like race, gender, age)? • Engage with domain experts and data
scientists to understand potential sources of
• How might these proxies influence the bias.
outcomes of the AI system?
• Experts can provide insights into nuances of
3. How Was the Data Collected? data that might not be evident through
statistical analysis alone.
• What was the process of data collection?
• Could the collection method have
introduced any biases (e.g., self-selection
bias, sampling bias)?

4. What Is the Historical Context?

• Does the historical data contain biases or
reflect past discriminatory practices?
• How might historical biases affect current
and future AI decisions?

5. Data Labeling:
• Who labeled the data, and what criteria
were used?
• Could there be subjective biases in how data
was categorized or labeled?

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 12

 5. Review Data Collection and 8. Historical Data Review:
Curation Processes:
• Critically evaluate historical data for biases
• Examine how data was collected, processed, and consider how historical trends might
and curated. influence current data.
• Look for any stages in the data pipeline
where biases could have been introduced. 9. Impact Assessment:
• Conduct impact assessments to understand
6. External Benchmarks and Standards: how biases in data might affect various
• Compare your data with external benchmarks groups.
or standards to identify any deviations. • Evaluate the potential consequences of
• This can help in understanding if the dataset biased decisions on different communities.
is skewed compared to real-world
distributions. 10. Document Findings and
Recommendations:
7. Feedback Loops: • Thoroughly document the analysis
• Analyze whether the AI system’s outputs process, findings, and any
are creating feedback loops that reinforce recommendations for addressing
biases. identified biases.
• Assess how data generated by the AI system is • This documentation is crucial for
fed back into the system, potentially transparency and accountability.
amplifying initial biases.
By asking these questions and following these steps,
you can conduct a comprehensive analysis of your
data for inherent biases, which is essential for
building fair and ethical AI systems.

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 13

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 14
EXAMPLE 2: Use it to help you think through your general audit
plan
• Scope:
Prompt: Using The Institute of Internal Auditors’ • Examination of the online content library.
existing standards for internal auditing, draft an audit
• Assessment of the current taxonomy
program with the objective of validating whether an
and tagging processes.
organization’s website search functionality and the
processes supporting website search capabilities are • Evaluation of the website’s
effective. The scope of the audit includes the search functionality.
organization’s online content library, its current
taxonomy, tagging processes, and website. The 3. Risk Assessment:
organization in question is a small, for-profit • Conduct a risk assessment to identify potential
organization specializing in the sale of pet products. Its issues that could impact the effectiveness of
website governance process is defined; the entire the website’s search functionality.
organization has standardized and documented its
processes; and cross- team collaboration thrives. 4. Develop Audit Criteria:
• Benchmarking: Compare the
organization’s practices against best
PRO TIP! Be specific in your practices in website search functionality.
questions. The more nuanced you are • Compliance: Ensure adherence to any
(without giving out private relevant laws, regulations, and internal
information!), the better your policies.
responses will be.
5. Conducting the Audit:
Actual ChatGPT 4.0 Response: Creating an audit
• Test Website Search Functionality:
program using The Institute of Internal Auditors’ existing
Conduct tests to assess the accuracy,
standards for internal auditing to evaluate the
speed, and relevance of search results.
effectiveness of an organization’s website search
functionality and related processes involves several • Evaluate Taxonomy and Tagging Processes:
steps. Here’s a draft audit program tailored to a small, Review how content is categorized and
for-profit organization specializing in pet products: tagged, and how this influences search
results.
1. Preliminary Planning:
• Assess Content Library: Determine if the
• Understand the Business: Gain an in- content library is up to date, relevant, and
depth understanding of the organization’s adequately indexed for search.
business model, focusing on online sales
and content dissemination strategies. • Website Technical Evaluation: Examine
technical aspects like website coding,
• Review Existing Documentation: Examine metadata, and search algorithms.
the organization’s website governance
process, standardized documentation, and • Stakeholder Interviews: Engage with
collaboration protocols. team members across different
departments to understand their role in
• Identify Key Stakeholders: Determine who will maintaining and improving search
be involved in the audit (e.g., IT, marketing, functionality.
content management teams).
• Document Review: Analyze the
2. Define Audit Objective and Scope: standardization and documentation of
processes related to website search.
• Objective: To validate the effectiveness of
the organization’s website search functionality
6. Reporting:
and the processes supporting it.
• Draft Report: Prepare a draft audit report
detailing findings, risks identified, and areas
Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 15
 for improvement.

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 16

 • Management Review: Present the draft report enough to handle complex queries or filter
to management for feedback. results effectively.
• Final Report: Issue a final report
with recommendations and an action
plan.

7. Follow-Up:
• Action Plan Monitoring: Monitor the
implementation of
recommendations.
• Follow-Up Audit: Plan a follow-up audit, if
necessary, to evaluate the implementation of
the action plan.

8.Continuous Improvement:
• Feedback Loop: Establish a feedback
mechanism to continuously improve website
search functionality and related processes.

This audit program is designed to comprehensively

evaluate the effectiveness of the organization’s website
search functionality, ensuring it aligns with its objectives
and the needs of its customers in the pet products
industry.

Prompt: Given the information already provided about

the organization’s size and type, as well as the
maturity of the website governance process, what
risks should I be looking for that could impact the
effectiveness of the website’s search functionality?

PRO TIP! Biased questions or data

will give biased answers. Try to
remove all bias in your prompts.

Actual ChatGPT 4.0 Response: Given the nature of the

small, for-profit organization specializing in pet products,
with a well-defined website governance process and
standardized, documented processes, several specific
risks could impact the effectiveness of the website’s
search functionality. Here are some key risks to consider:
1. Technological Limitations:
• Outdated Technology: The website might
be running on outdated technology that
limits search functionality capabilities.
• Inadequate Search Algorithms: The algorithms
used for search might not be sophisticated

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 17

2. Content Management Risks:
• Poorly Managed Taxonomy and Tagging:
Inaccurate or inconsistent taxonomy and
tagging can lead to irrelevant search results.
• Outdated or Irrelevant Content: If the
content library is not regularly updated or
curated, it can result in outdated or irrelevant
search results.

3. User Experience Issues:

• Ineffective Search Interface: A non-intuitive
or complicated search interface can hinder
user experience and reduce the effectiveness
of searches.
• Lack of Customization: Inability to tailor
search results to individual user preferences
or search history.

4. Data and Analytics Challenges:

• Inadequate Data Analysis: Failure to
effectively analyze user search patterns and
behaviors to improve search functionality.
• Limited Feedback Mechanisms: Lack of
effective channels for receiving user feedback
on search functionality.

5. Security and Privacy Concerns:

• Data Breaches: Risks associated with
the security of user data and search
histories.
• Compliance Issues: Non-compliance with
data protection regulations (like GDPR) in
handling user data.

6. Operational and Governance Risks:

• Insufficient Cross-Team Collaboration: Even
with thriving collaboration, there could be
gaps in communication or understanding
between different teams (like IT, content,
marketing) impacting search efficiency.
• Change Management Issues: Challenges in
implementing changes or updates to the
search functionality due to rigid governance
structures.

7. Resource Constraints:
• Limited IT Resources: Being a small
organization, there might be constraints in IT
support
and resources for ongoing maintenance
and improvement of search functionality.
Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 18
 • Budget Limitations: Financial constraints • Personalization Features: Develop capabilities
may limit the ability to invest in advanced for customized search experiences based on
search technology or hire specialized talent. user preferences.

8. Scalability and Growth Challenges: 4. Data and Analytics Challenges:

• Scalability of Search Function: As the • Data Analysis Tools: Use advanced data
business grows, the search functionality may analytics tools to monitor and analyze search
not be scalable to handle increased traffic or patterns.
expanded content.
• Feedback Mechanisms: Establish channels for
Identifying and addressing these risks during the audit collecting and analyzing user feedback on
will help ensure that the organization’s website search search functionality.
functionality remains effective and aligned with its
business objectives and user needs. 5. Security and Privacy Concerns:
• Data Security Protocols: Implement robust
data security measures, including encryption
Prompt: For each of the risks identified above, and access controls.
what are some potential internal controls that could • Compliance Audits: Regularly audit data
help the handling practices for compliance with relevant
organization avoid, share, accept, or mitigate those risks? data protection laws.

6. Operational and Governance Risks:

Actual ChatGPT 4.0 Response: For each of the risks
identified in the effectiveness of the website’s search • Cross-Functional Teams: Form dedicated
functionality, implementing specific internal controls teams comprising members from IT, content,
can help the organization manage these risks effectively. and marketing to oversee search functionality.
Here’s a breakdown of potential controls for each risk • Change Management Processes: Develop a
category: structured approach for implementing changes
1. Technological Limitations: to the search system.
• Regular Technology Audits: Conduct 7. Resource Constraints:
periodic reviews of the website’s
technology stack to ensure it’s up to date. • Resource Allocation Plan: Develop a
strategic plan for allocating IT resources
• Investment in Advanced Search efficiently.
Technologies: Allocate budget for upgrading
search algorithms and software. • Budget Planning: Set aside funds specifically
for maintaining and upgrading search
2. Content Management Risks: functionality.
• Standardized Tagging Procedures: 8. Scalability and Growth Challenges:
Implement and enforce strict guidelines for
tagging and categorizing content. • Scalability Testing: Regularly test the search
system for scalability as part of the
• Regular Content Reviews: Schedule periodic ongoing maintenance routine.
audits of the content library to ensure
relevance and accuracy. • Growth Planning: Incorporate scalability
considerations in long-term planning for
3. User Experience Issues: the website’s infrastructure.
• User Interface Testing: Regularly test and By implementing these controls, the organization can
update the search interface based on user effectively manage the risks associated with its website
feedback and usability studies. search functionality. It’s important to periodically review and
update these controls to ensure they remain effective as the
organization and its online presence grow and evolve.

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 19

EXAMPLE 3: Use It to Help You Structure Unstructured Data
• Date: May 18, 2023
Prompt: Access the attached, publicly available
• Contract Name: Procurement of Directors
minutes of the New York City Municipal Water Finance
and Officers Liability Insurance from AIG,
Authority Board of Directors for 2023. Based on the
Zurich American Insurance Company, Liberty
information listed in the files, identify all of the times
Mutual Insurance Company, and Aspen
that the Board of Directors discussed contracts. List the
Insurance
Date, Contract Name, and Status of each contract.
• Status: Approved

• Date: August 3, 2023

PRO TIP! ChatGPT-style tools can • Contract Name: Rate Consultant Contracts
structure many types of unstructured with Amawalk Consulting Group LLC and
data, but avoid uploading any Carollo Engineers
proprietary information into a public
tool. For • Status: Approved
example, an organization would not want
• Date: September 22, 2023
minutes from a private meeting, accounts
payable reports, or bank statements uploaded • Contract Name: Ratification of Rate
into a public tool. Consultant Contracts with Amawalk
Consulting Group LLC and Carollo Engineers
Actual ChatGPT 4.0 Response: Based on the information
• Status: Ratified
from the minutes of the New York City Municipal Water
Finance Authority Board of Directors meetings for Prompt: Put the above information into a CSV file.
2023,
here are the times the Board discussed contracts:
• Date: February 2, 2023
• Contract Name: Amendment of Consulting Actual ChatGPT 4.0 Response: The information
Engineer Contracts with AECOM and Macan about the contracts discussed by the New York City
Deve Municipal Water Finance Authority Board of Directors
in 2023 has been structured into a CSV file.
• Status: Approved

• Date: February 24, 2023

PRO TIP! Be skeptical and verify.
• Contract Name: Approval of Minutes of
The responses a NLP tool give you
Previous Meeting and Approval of
should be fact-checked and used to
Supplemental Resolution No. 180 Authorizing
provoke further thinking, but should
the Issuance of Fiscal 2023 Series DD Water
not be
and Sewer System Second General Resolution
treated as an authoritative statement.
Revenue Bonds
• Status: Approved

• Date: March 28, 2023

• Contract Name: Extension of Bond Counsel
Contracts with Nixon Peabody LLP, Hardwick
LLC, and Orrick Herrington & Sutcliffe LLP
• Status: Approved

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 20

EXAMPLE 4: Use It to Help You Write Your Audit Report
governance plan. This response fails to address critical
Prompt: Make the following information more concise aspects of
and action-oriented:

PRO TIP! ChatGPT-type tools excel

at producing short, concise, and
compelling text. However, there are
significant limitations with report
writing, as
internal auditors should not include any
proprietary information about their organization in
their prompts. Remember: ChatGPT stores
everything that is typed into it.

• Expanded Background Information: The concept

of website governance is an essential best practice
that is crucial for ensuring that this digital asset
significantly supports the organization’s overarching
strategic
goals. It is instrumental in delivering the anticipated
value from the substantial investment placed in the
website’s creation and ongoing maintenance. This
comprehensive approach includes a well-defined set
of policies, processes, roles, and responsibilities.
Moreover, it comprehensively addresses all aspects
of website maintenance and operation, ensuring that
every facet of the website’s performance and
functionality aligns seamlessly with the
organization’s objectives and expectations.
• Detailed Observations: Currently, there is a
notable absence of a structured governance plan
specifically targeting the operation and
maintenance of the website. This gap was
somewhat addressed in October 2022, when the
Web Team presented key performance indicators
(KPIs) for the website, along with a plan outlining
potential future improvements. This presentation
was a commendable initial step towards
establishing a framework for website governance.
However, the situation has evolved since then due
to a turnover in leadership, which
has somewhat stalled these initial efforts. When the
Board of Directors recently inquired about the state
of website governance, the response provided was
rather simplistic and insufficient. It was stated that
“the
Web Team owns the website,” which, while
factually correct, does not constitute a robust

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 21

 website governance, such as compliance and other
key operational components, which may not fall
under the direct purview of the Web Team.
• Enhanced Recommendations: It is imperative
that the management takes a proactive stance in
this matter. The first step should be to establish,
document, and effectively communicate a
comprehensive plan. This plan must explicitly
address the strategic direction
of the website, ensuring that it aligns with and
supports the organization’s broader strategic
goals. In addition to setting the strategic
direction, it is equally important to delineate
clear policies, processes, and
roles/responsibilities. These should be geared
towards ensuring the continued operation and
maintenance of the website at an optimal level.
Establishing such
a governance framework will not only provide
clear guidelines for current operations, but will
also lay a solid foundation for future growth and
adaptation of the website in alignment with the
evolving objectives of the organization.
Actual ChatGPT 4.0 Response: Concise Summary
and Action Plan
• Issue: The organization lacks a structured
website governance plan, essential for
aligning the website with strategic goals.
Initial efforts stalled due to leadership
changes, and the current understanding of
website governance is inadequate.
• Immediate Action: Management must
establish and document a comprehensive
website governance plan. This plan should:
• Clearly define the strategic
direction of the website, ensuring
alignment with the organization’s
broader goals.
• Detail policies, processes, roles, and
responsibilities for ongoing website
operation and maintenance.
• Address all aspects of website
performance and functionality.

• Long-Term Benefits: Implementing this

governance framework will provide clear
operational guidelines, support the
website’s optimal functioning, and facilitate
future growth and alignment with
organizational objectives.

Artificial Intelligence 101 Series: ChatGPT for Internal Auditors 22

About The IIA
The Institute of Internal Auditors (IIA) is the internal audit profession’s most widely recognized
advocate, educator, and provider of standards, guidance, and certifications. Established in 1941, The
IIA today serves more than 230,000 members from more than 170 countries and territories. The
association’s global headquarters is in Lake Mary, Fla., USA. For more information, visit
www.theiia.org.

Disclaimer
The IIA publishes this document for informational and educational purposes. This material is not
intended to provide definitive answers to specific individual circumstances and as such is only
intended to be used as a guide. The IIA recommends seeking independent expert advice relating
directly to any specific situation. The IIA accepts no responsibility for anyone placing sole reliance on
this material.

Copyright
Copyright © 2023 The Institute of Internal Auditors, Inc. All rights reserved. For permission to
reproduce, please contact [email protected].

IIA Headquarters
1035 Greenwood Blvd., Suite
401 Lake Mary, FL 32746 USA