Prompt Engineering Lecture Elvis
Prompt Engineering Lecture Elvis
A lecture by DAIR.AI
Elvis Saravia
Prerequisites & Objectives
• Prerequisites:
• Python
• Knowledge of language models
• Basic understanding of deep learning / ML concepts
• Objectives
• Present an introduction to prompt engineering
• Present an overview of the latest prompting techniques
• Provide demonstrations and exercises to practice different
prompting techniques
Agenda
• Introduction to Prompt Engineering
• Advanced Techniques for Prompt Engineering
• Applications & Tools
• Conclusion & Future Directions
Part 1
Source: https://jobs.lever.co/Anthropic/e3cde481-d446-460f-b576-93cab67bd1ed
The sky is
blue
Model: “text-davinci-003”
temperature: 0.7
top-p: 1
Elements of a Prompt
• A prompt is composed with the following components:
• Instructions
• Context
• Input data
• Output indicator
Sentiment:
Text Summarization
Answer: Mice.
Sentiment: Neutral
Role Playing
“””
Table departments, columns = [DepartmentId,
DepartmentName]
Table students, columns = [DepartmentId, StudentId,
StudentName]
Create a MySQL query for all students in the Computer
Science Department
“””
Demo
Part 2
Few-shot Prompts
• Few-shot prompting allows us to provide exemplars in
prompts to steer the model towards better performance
Zero-Shot CoT
• Involves adding "Let's think step by step" to the original
prompt
11 apples
35
Self-Consistency Example
Q: There are 15 trees in the grove. Grove workers will plant trees in the
grove today. After they are done, there will be 21 trees. How many trees did
the grove workers plant today?
A: We start with 15 trees. Later we have 21 trees. The difference must be the
number of trees they planted. So, they must have planted 21 - 15 = 6 trees.
The answer is 6.
Q: If there are 3 cars in the parking lot and 2 more cars arrive, how many
cars are in the parking lot?
A: There are 3 cars in the parking lot already. 2 more arrive. Now there are
3 + 2 = 5 cars. The answer is 5.
...
Q: Olivia has $23. She bought five bagels for $3 each. How much money does
she have left?
A: She bought 5 bagels for $3 each. This means she spent 5
Q: When I was 6 my sister was half my age. Now I’m 70 how old is my sister?
A:
Output 1
When I was 6 my sister was half my age, so she was 3. Now I am 70, so she
is 70 - 3 = 67. The answer is 67.
Output 2
When the narrator was 6, his sister was half his age, which is 3. Now
that the narrator is 70, his sister would be 70 - 3 = 67 years old. The
answer is 67.
Output 3
When the narrator was 6, his sister was half his age, which is 3. Now
that the narrator is 70, his sister would be 70 - 3 = 67 years old. The
answer is 67.
Generate Knowledge Prompting
• This technique involves using additional knowledge
provided as part of the context to improve results on
complex tasks such as commonsense reasoning
• The knowledge used in the context is generated by a
model and used in the prompt to make a prediction
• Highest-confidence prediction is used
...
Input: Part of golf is trying to get a higher point total than others.
Knowledge:
Knowledge 1
The objective of golf is to play a set of holes in the least number of strokes. A
round of golf typically consists of 18 holes. Each hole is played once in the
round on a standard golf course. Each stroke is counted as one point, and the
total number of strokes is used to determine the winner of the game.
Knowledge 2
Explain and Answer: No, the objective of golf is not to get a higher
point total than others. Rather, the objective is to play a set of holes
in the least number of strokes. The total number of strokes is used to
determine the winner of the game, not the total number of points.
High-confidence prediction
Question: Part of golf is trying to get a higher point total than others.
Yes or No?
Explain and Answer: Yes, part of golf is trying to get a higher point
total than others. Each player tries to complete the course with the
lowest score, which is calculated by adding up the total number of
strokes taken on each hole. The player with the lowest score wins the
game.
Low-confidence prediction
PAL
ReAct
Demo
Part 3
Applications
• Advanced prompting techniques are now allowing all sorts
of advanced applications with LLMs
• LLMs and external tools/APIs
• Data-augmented Generation
• QA with sources
• Summarization using sources
Data-Augmented Generation
• For many real-world applications there is a need to
augment the generation of a model by incorporating
external data
• Steps involved:
• Fetching relevant data
• Augmenting prompts with the retrieved data as context
• External data can include:
• Document stores
• APIs
• Databases
• User provided data
• …
Demo
Part 4
Model Safety
• Prompt engineering can be used not only to improve
performance but also the reliability of response from a
safety perspective
• Prompt engineering can help identify risky behaviours of LLMs
which can help to reduce harmful behaviours and risks that may
arise from language models
• There is also a part of the community performing prompt
injection to understand the vulnerability of LLMs
Prompt Injections
• It turns out that building with LLMs, like any other systems,
comes with challenges that includes safety considerations.
• Prompt injections aim to find vulnerabilities in LLMs
• Some common issues include:
• Prompt Injection
• Prompt Leaking
• Jailbreaking
Prompt Injection
• Prompt injection is used to hijack an LM’s output by
injecting an untrusted command that overrides instruction
of a prompt
• This could easily happen if you just concatenate your
prompt with another user generated prompt
Prompt Leaking
• Prompt leaking aims to force the model to spit out
information about its own prompt.
• This can lead to leaking of either sensitive, private or
information that’s confidential
Jailbreaking
• Jailbreaking is another form of prompt injection where the
goal is to bypass safety and moderation features
• LLMs provided via APIs might be coupled with safety
features or content moderation which can be bypassed
with harmful prompts/attacks
• This might sound like a difficult task but it’s not because
the model is usually served static and might have these
vulnerabilities due to many factors such as the data it was
trained on, etc.
Jailbreaking examples
RLHF
• Reinforcement Learning from Human Feedback (RLHF) is
now being used to train LLMs that fit human preference
• RLHF involves collecting high-quality prompt datasets
• Popular examples:
• Claude (Anthropic)
• ChatGPT (OpenAI)
Future Directions
• Augmented LMs
• Emergent ability of LMs
• Acting / Planning - Reinforcement Learning
• Multimodal Prompting
• Graph Prompting
What’s next?
• This is the first lecture in a series of lectures
• Subscribe to the channel
• Follow us on Twitter (@dair_ai and @omarsar0) for
announcements on upcoming special lectures covering a
variety of topics in language models
https://github.com/dair-ai/Prompt-Engineering-Guide