Unit 5

Planning – planning problem – Simple planning agent – Blocks

world problem – Mean Ends analysis Learning - Machine
learning - Learning concepts, methods and models
Introduction to expert system – architecture of expert systems.
 Planning
• The task of coming up with a sequence of actions that will achieve
a goal is called planning.
• Planning Environments
1. Classical Planning Environments
• Fully observable, deterministic, finite, static and discrete.
2. Non classical Planning Environments
• Partially observable, stochastic with different algorithms and
agent designs.
 Planning Problem
• The planning problem is actualy the question how to go to next state or the goal
state from the current state. It involves two things 'how' and 'when'.
• The planning problem is defined with:
1. Domain model
• The domain model defines the actions along with the objects. It is necessary to
specify the operators too that actually describe the action. Along with this,
information about actions and state constraints while acting should also be given.
This entirely formulates the domain model.
2. Initial state
• The initial state is the state where any action is yet to take place (the stage when the
exam schedule is put up!).
3. Goal state (next state)
• The final state or the goal state is the state which the plan is intended to achieve.
Basic Planning Representation
• Planning representation
involves
1) State Representation
2) Goal Representation
3) Action
Representation
 State Representation
• A state is represented with a conjunction of positive
literals using
• Logical Propositions: Poor Λ Unknown
• FOL literals: At(Plane1,OMA) Λ At(Plane2,JFK)
• FOL literals must be ground & function-free
• Not allowed: At(x,y) or At(Father(Fred),Sydney)
• Closed World Assumption
• What is not stated are assumed false
 Goal Representation
• Goal is a partially specified state
• A proposition satisfies a goal if it contains all the atoms of the
goal and possibly others.
• Example: Rich ^ Famous ^ Miserable satisfies the goal Rich ^
Famous
Ac tion Representation
 Simple planning agent
• A simple planning agent is very similar to problem-solving agents in
that it constructs plans that achieve its goals, and then executes them.
• The limitations of the problem- solving approach motivates the design
of planning systems.
• To solve a planning problem using a state-space search approach we
would let the:
• initial state = initial situation
• goal-test predicate = goal state description
• successor function computed from the set of operators
• once a goal is found, solution plan is the sequence of operators in the
path from the start node to the goal node
 Algorithm of a simple planning agent
1.Generate a goal to achieve
2. Construct a plan to achieve goal from current state
3. Execute plan until finished
4. Begin again with new goal
• The agent first generates a goal to achieve, and then constructs a
plan to achieve it from the current state.
• Once it has a plan, it keeps executing it until the plan is finished,
then begins again with a new goal.
 Languages for Planning Problems
• STRIPS
– Stanford Research Institute Problem Solver
– Historically important
• ADL
– Action Description Languages
• PDDL
– Planning Domain Definition Language
 Stanford Research Institute Problem Solver
(STRIPS)
• Makes use of the first order
predicates.
• STRIPS allows function-free
literals.
• Example of a robot: The
example involves a robot, a
cup tea, guest and two rooms.
We want the robot to get the
tea and give it to the guest.
 Action Description Language (ADL)
• STRIPS language lacks the expressive power.
• It can be extended very well to overcome some of the limitations
and ADL does that.
• The properties of ADL are as follows:
1. It allows negative literals.
2. It makes use of quantified variables along with the disjunctions
and the conjunctions.
3. Conditional post-conditions are allowed.
4. Variables with different types at the same time are allowed and
also equality property is available.
 Planning Domain Description
DEFINITION Language
(PDDL)
• It is a standardization of the planning languages.
• It is defined by the researchers as a standard language.
• We can say it is a superset of STRIPS and ADL that allows features
like
1. Objects can have type specifications.
2. It can have negative pre-conditions.
3. The add and delete lists can be conditional.
4. In some cases, it also allows numeric values.
 Block World

• There are ‘N’ number of Blocks resting on table with specified

sequence.
• Goal is to arrange in desired sequence.
• Available moves
– Put block on table
– Put a block on another block top
• State is represented using sequence of blocks in current pos.
Block World
 Means - Ends Analysis
• Means-Ends Analysis is problem-solving techniques used in Artificial
intelligence for limiting search in AI programs.
• It is a mixture of Backward and forward search technique.
• The means -ends analysis process centers around finding the
difference between current state and goal state.
• Search strategies are either reason forward of backward.
Steps
1. First, evaluate the difference between Initial State and final State.
2. Select the various operators which can be applied for each
difference.
3. Apply the operator at each difference, which reduces the difference
between the current state and goal state.
 MEA Algorithm
Step 1: Compare CURRENT to GOAL, if there are no differences between both then
return Success and Exit.
• Step 2: Else, select the most significant difference and reduce it by doing the
following steps until the success or failure occurs.
• A) Select a new operator O which is applicable for the current difference, and if there
is no such operator, then signal failure.
• B) Attempt to apply operator O to CURRENT. Make a description of two states.
i) O-Start, a state in which O?s preconditions are satisfied.
ii) O-Result, the state that would result if O were applied In O-start.
• C) If
(First-Part <------ MEA (CURRENT, O-START)
And
(LAST-Part <----- MEA (O-Result, GOAL), are successful, then signal
Success and return the result of combining FIRST-PART, O, and LASTPART.
Example of Mean-Ends Analysis:
Example of Mean-Ends Analysis:
 Non-linear planning
• A plan that consists of sub-problems, which
are solved simultaneously is said non-linear
plan.
• In case of the goal stack planning there are
some problems. To achieve any goal, it could
have an impact on the one that has been
achieved.
There is a concept of constraint posting that
comes with non-linear planning. The constraint
posting states that the plan can be built by
• 1. Addition of operators or suggesting
operators
• 2. Ordering them
• 3. Binding the variables to the operators
 Conditional Planning
• Conditional planning has to work regardless of the outcome of an action.
• The outcome of actions cannot be determined so the environment is said to
be nondeterministic.
• It’s a way to deal with uncertainty by checking what is actually happening in
the environment at predetermined points in the plan. (Conditional Steps)
Example:
• Check whether SFO airport (San Francisco International Airport) is oper
Three kind of Environments
• Fully Observable - The agent always knows the current state
• Partially Observable - The agent knows only a certain amount about the
actual state. (much more common in real world)
• Unknown - The agent knows nothing about the current state.
 Conditional Planning in Fully Observable
Environments
• Agent used conditional steps to check the state of the environment
to decide what to do next.
• Plan information stores in a library
Ex:

• Syntax: If then plan_A else plan_B

 Reactive Planning
• Reactive planning is planning under uncertainty.
• Makes use of the if-then rules.
• The reactive planners are based on the concept that they should be able to handle
an unknown situation too. So, the reaction rules are used that help them in doing so.
• A rule selection is based on the priority and a holding condition that maximises the
priority.
• The rule which is at present in execution is said to be active whereas the ones with
holding priority (we can call them possible competitors) are pre-active others are
inactive.
• A B-tree structure is used in reactive planning, where the things are algorithm
selects the rule. Sometimes, no rule can be selected. In such a case, dependent on
the algorithm implementation for rule selection.
 Machine Learning
• Machine learning is
building and exploring of
methods for
programming computer
to make them Learn.
 Machine Learning
• Machine Learning Approach • Scope of ML
 Machine Learning
Goals of machine learning
• To produce learning algorithms with practical value.
• Development and enhancement of computer algorithms and models to
meet the decision making requirements in practical scenarios.
• To facilitate in building Intelligent System (IS) that can be used in
solving real time problems.
Challenges of Machine Learning
• Availability of limited learning data and unknown perspectives.
• Acquiring Accurate , compact and precise knowledge building.
• Require large working memory to store data.
• Focusing Too Much on Algorithms and Theories
• Monitoring and maintenance
 Machine Learning
Types of Machine Learning
• Rote learning
Computational structure used in
Machine learning: • Learning from observations
• Learning from agents
1. Functions. • Inductive learning
2. Logic programs and rule sets • Decision tree learning
3. Finite state machines • Supervised learning
4. Grammars • Unsupervised learning
5. Problem solving system • Semi-supervised learning
• Ensemble learning
• Discovery based learning
• Learning by problem solving
 ML - Rote Learning
• Rote learning is a rudimentary form of learning, which focusses on
memorization.
• Here, the values are stored so that these are not re-computed.
• When computations are more expensive than re-computing from
scratch, this strategy can save a significant amount of time.
• Selective paging and caching are some of the examples of rote
learning.
 ML - Learning from agents
•An agent is defined the computational entity which is capable of
receiving the environment and can act based on the situation.
•The agent is composed of learning element performance element
and a curiosity element.
•Based on the coordination between these elements the outcome
of the agent’s behavior is measured.
 ML - Inductive Learning
• Inductive learning involves learning generalized rules from specific
examples (can think of this as the “inverse” of deduction)
• Main task: given a set of examples, each classified as positive or
negative produce a concept description that matches exactly the
positive examples.
• The examples are coded in some representation language, e.g. they are
coded by a finite set of real-valued features.
• The concept description is in a certain language that is presumably a
superset of the language of possible example encodings.
• A “correct” concept description is one that classifies correctly ALL
possible examples, not just those given in the training set.
 ML - Decision Tree Learning
• The learned function is represented by a decision tree.
• In terms of programming it is also represented as if then rules.
• Decision tree depicts the simple learning from the observation,
method.
• Based on the observation, at every node, decision is taken.
 ML - Supervised learning
• Uses labelled dataset.
• Supervised learning is the learning algorithm that is provided with
the set of training data and the algorithm further induces the
classifier to classify the unseen or new data.
• A line (hyperplane) which is generated after learning separating
two classes class A and class the in two parts the classifier and
the decision-making engine minimize the false positives and false
negative.
 ML - Unsupervised Learning
• Use unlabeled dataset.
• Learning is more based on similarities and differences which are
visible. These differences and similarities are mathematically
represented in unsupervised learning
• Grouping and categorization of the objects is based on the
understanding of similarities and visualization of their relations
• Unsupervised learning performs hierarchical clustering.
 ML - Semi supervised Learning
• Semi supervised learning is developed to
cope up with the issues of learning in
supervised or unsupervised mode in
isolation.
• Semi-supervised learning tries to learn from
the labelled as well as unlabeled data.
• Let U be a set of unlabeled data and L be a
set of labelled data.
• As the learning process the learning
approach identifies the unlabeled data U with
reference to a labelled data L and keeps on
labelling the unlabeled data.
• This method is also called as self training in
semi supervised learning.
ML - Ensemble Learning
• Ensemble learning method is the one where multiple
learners or learning algorithms are trained.
• In ensemble learning method the whole collection or
ensemble of hypothesis is selected from the hypothesis
space and their predictions are combined.
• In this approach, the learners or referred to as base learners.
• The most commonly used ensemble learning methods are
1.Boosting:
• Boosting can probably be defined as the method for
generating accurate predictions by combining the rules that
are comparatively inaccurate.
• Boosting works on the weighted training sets. The weights of
the training example reflects the importance of training
examples.
2.Bagging:
• In Bagging, the training data is resampled. This is referred to
as bootstrap sampling, where the training data with
replacement is taken in the learning approaches.
 ML
Discovery-based Learning
• In case of discovery learning the learning, takes place without any help of teacher or
supervisor.
• It is called unsupervised.
• It can be said to be inquiry-based learning, where the learner on the basis of past
experience and knowledge tries to discover the outcomes.
Learning by Problem Solving
• In learning from problem solving, various parameters related to solution and problem are
considered.
• These parameters are used and effectively desirability of a particular outcome or decision
is determined.
• It is typically based on the way a problem is solved and the outcomes at every step.
• Even in pattern-based decision- making, the correct category is determined based on the
previous results.
 ML - Learning Methods

• Artificial neural network-based learning - Back propagation

• Support vector machines
• Reinforcement learning
• Adaptive learning
• Multi agent based learning
• Distributed learning
• Speedup learning
 Artificial neural network based learning
• An Artificial Neural Network (ANN) is a computational model
inspired by the human brain’s neural structure.
• It consists of interconnected nodes (neurons) organized into
layers.
• Information flows through these nodes, and the network adjusts
the connection strengths (weights) during training to learn from
data, enabling it to recognize patterns, make predictions, and
solve various tasks in machine learning and artificial intelligence.
 ANN - Back propagation Algorithm
• Back-propagation is the
essence of neural net
training.
• It is the method of fine-tuning
the weights of a neural net
based on the error rate
obtained in the previous
epoch (i.e., iteration).
• Proper tuning of the weights
allows you to reduce error
rates and to make the model
reliable by increasing its
generalization.
 ANN - Back propagation Algorithm
1.Inputs X, arrive through the preconnected path
2. Input is modeled using real weights W. The weights are usually
randomly selected.
3. Calculate the output for every neuron from the input layer, to the
hidden layers, to the output layer.
4. Calculate the error in the outputs.
ErrorB= Actual Output – Desired Output
5. Travel back from the output layer to the hidden layer to adjust the
weights such that the error is decreased.
6. Keep repeating the process until the desired output is achieved.
 Support Vector Machine (SVM)
• Support vector machines (SVM)
are supervised learning models
with associated learning
algorithms that analyze data used
for classification and regression
analysis.
• It is a machine learning approach.
• They analyze the large amount of
data to identify patterns from them.
• SVMs are based on the idea of
finding a hyperplane that best
divides a dataset into two classes,
as shown in the image below.
 Hyperplane
• As a simple example, for a classification task
with only two features, hyperplane is a line
that linearly separates and classifies a set of
data.
• Intuitively, the further from the hyperplane
our data points lie, the more confident we
are that they have been correctly classified.
• We therefore want our data points to be as
far away from the hyperplane as possible,
while still being on the correct side of it.
• So when new testing data are added,
whatever side of the hyperplane it lands will
decide the class that we assign to it.
 Identify the right hyperplane (Scenario-1):
• Here, we have three hyperplanes (A, B and C). Now, identify the
right hyperplane to classify star and circle.

Hyperplane “B” has excellently performed this job.

 Identify the right hyperplane (Scenario-2):
• Here, we have three hyperplanes (A, B and C) and all are
segregating the classes well. Now, how can we identify the right
hyperplane?
• Here, maximizing the distances between nearest
data point (either class) and hyperplane will help
us to decide the right hyperplane.
• This distance is called as Margin. Let’s look at
the second fig
• We can see that the margin for hyperplane C is
high as compared to both A and B. Hence, we
name the right hyperplane as C.
• Another lightning reason for selecting the
hyperplane with higher margin is robustness. If
we select a hyperplane having low margin then
there is high chance of misclassification.
 Reinforcement Learning
• Learning from interaction with an environment to achieve some
long-term goal that is related to the state of the environment
• The goal is defined by reward signal, which must be maximised.
• Agent must be able to partially/fully sense the environment state
and take actions to influence the environment state
• The state is typically described with a feature vector
 Reinforcement Learning Systems
• Reinforcement learning systems have 4 main elements:
1.Policy-A policy is a mapping from the perceived states of the environment to
actions to be taken when in those states
• A reinforcement learning agent uses a policy to select actions given the
current environment state
• An on-policy agent learns only about the policy that it is executing
• An off-policy agent learns about a policy or policies different from the one
that it is executing.
2. Reward signal
• The reward signal defines the goal
• On each time step, the environment sends a single number called the reward
to the reinforcement learning agent,
• The agent’s objective is to maximize the total reward that it receives over the
long run
• The reward signal is used to alter the policy
 Reinforcement Learning Systems
3.Value function
• The value of a state is the total amount of reward an agent can
expect to accumulate over the future, starting in that state.
• Values must continually be re-estimated from the sequence of
observations that an agent makes over its lifetime.
4. Optional model of the environment
• A model of the environment allows inferences to be made about
how the environment will behave.
• Example: Given a state and an action to be taken while in that
state, the model could predict the next state and the next reward.
• Models are used for planning, which means deciding on a course
of action by considering possible future situations before they are
experienced
 Adaptive learning
• Adaptive machine learning algorithms are the machine learning
models, where the changes in the environment help in selecting the
algorithm or learning method.
• The adaptive learning solves some of the complex problems for
which a single learning method is not enough.
• This method is even more appropriate when the environment is
continuously changing and real time response is expected.
 Multi-Agent System
• A system with multiple autonomous
entities, with distributed information,
computational ability, and possibly
divergent interests.
• A multiagent system is one that
consists of a number of agents, which
interact with one-another
• In the most general case, agents will be
acting on behalf of users with different
goals and motivations
• To successfully interact, they will
require the ability to cooperate,
coordinate, and negotiate with each
other, much as people do
 Distributed learning
• In distributed learning the task of learning is distributed.
• Need for distributed learning - Arises due to large data sets and
time constraints.
• More than one agent in different parts of the data set.
• There will be distributed learning algorithms taken part in each
partition to get the desired outcome, which would then be
combined.
• Multi agent systems can be thought of as a subset of distributed
learning.
 Speedup learning
• Speedup learning typically deals with speeding up problem solving by
effective use of problem solving experience.
• Hence, prior problem solving experience is an input for speedup learning.
• In this learning,
1. There is no interaction with the environment.
2. New problems cannot be solved.
• So, speedup learning accelerates the process based on the previous
experiences and prior observations.
 Expert System
• An expert system is a computer program that is designed to solve
complex problems and to provide decision-making ability like a
human expert.
• It performs this by extracting knowledge from its knowledge base
using the reasoning and inference rules according to the user
queries.
 Characteristics of Expert System

• High Performance: The expert system provides high performance for

solving any type of complex problem of a specific domain with high
efficiency and accuracy.
• Understandable: It responds in a way that can be easily
understandable by the user. It can take input in human language and
provides the output in the same way.
• Reliable: It is much reliable for generating an efficient and accurate
output.
• Highly responsive: ES provides the result for any complex query
within a very short period of time.
Components of Expert System
 Components of Expert System
1. User Interface
• With the help of a user interface, the expert system interacts with the user, takes queries as
an input in a readable format, and passes it to the inference engine.
• After getting the response from the inference engine, it displays the output to the user.
• In other words, it is an interface that helps a non-expert user to communicate with the expert
system to find a solution.
2. Inference Engine(Rules of Engine)
• The inference engine is known as the brain of the expert system as it is the main processing
unit of the system.
• It applies inference rules to the knowledge base to derive a conclusion or deduce new
information.
• It helps in deriving an error-free solution of queries asked by the user.
3. Knowledge Base
• The knowledgebase is a type of storage that stores knowledge acquired from the different experts
of the particular domain.
• It is considered as big storage of knowledge.
• The more the knowledge base, the more precise will be the Expert System.
Architecture of Expert System
 Architecture of Expert System
Explanation Facility
Knowledge acquisition facility
• Allows a user to understand how the
expert system arrived at certain • Provide convenient and efficient means of
conclusions or results. capturing and storing all the components
• For example: it allows a doctor to find out of the knowledge base.
the logic or rationale of the diagnosis • Acts as an interface between experts and
made by a medical expert system.
Inference Engine the knowledge base.
User Interface
• Seeks information and relationships from
the knowledge base and provides • Specialized user interface software
answers, predictions, and suggestions the employed for designing, creating,
way a human expert would.
updating, and using expert systems.
Knowledge Base
• The main purpose of the user interface is
• Stores all relevant information, data, to make the development and use of an
rules, cases, and relationships used by
the expert system. expert system easier for users and
• It uses rules, If-then statements, fuzzy decision makers
logic
Advantages and Limitations of Expert
Systems