ARTIFICIAL

INTELLIGENCE Lecture #3

CS-414
• Reflex-based models in AI refer to
systems that operate based on predefined
rules or reactions to specific stimuli, rather
than making decisions through complex
reasoning or learning.
• These models are typically characterized
by their simplicity and efficiency in
environments where the rules are well-
defined and do not change frequently.
key aspects of reflex-based models:

1. Rule-Based Systems:
•Reflex-based AI often relies on a set of if-then rules. For
example, in a game like chess, a reflex-based AI might
have rules for capturing pieces or moving to certain
positions based on the opponent's moves.
2. Reactive Behavior:
•These models respond to specific inputs with predefined
outputs. They don't analyze or learn from past experiences
but instead execute a direct response based on the current
state of the system.
3. Applications:
•Reflex-based models are often used in scenarios where
decisions need to be made rapidly and where the
Applications and Limitations
•Applications:
• Reflex models are commonly used in simple systems
where complexity is minimal, and rapid response is
crucial. They are often found in embedded systems, basic
robots, and simple control systems.
•Limitations:
• Reflex models are limited in their ability to handle
complex environments or tasks that require planning,
learning, or adaptation. They can only handle situations
they have explicitly been programmed to manage and
may struggle with unforeseen or novel situations.
In summary, reflex models are useful for straightforward,
reactive tasks where immediate responses are needed, but
they are not suitable for more complex scenarios that require
• Modeling takes messy real world problems
and packages them into neat formal
mathematical objects called models

• Which can be subject to rigorous analysis

and can be operated on by computers.
However, modeling is lossy: not all of the
richness of the real world can be captured,
and therefore there is an art of modeling

• what does one keep versus ignore?

• As an example, suppose we’re trying to

have an AI that can navigate through a
busy city.
• We might formulate this as a graph where
nodes represent points in the city, edges
represent the roads, and the cost of an
An artificial intelligence system must learn
from data and improve their performance over
time. There are several key concepts involved:
1. Machine Learning (ML): This is a subset of AI
where algorithms learn patterns from data. It's often
divided into:
•Supervised Learning: Models learn from labeled
data, where both the input and the correct output are
provided.
•Unsupervised Learning: Models work with
unlabeled data and try to find patterns or groupings
on their own.
2.Neural Networks: Inspired by the human brain, these are composed of layers
of nodes (neurons) and are used in many deep learning applications. They can
model complex patterns and are particularly effective in tasks like image and
speech recognition.

3.Training: This involves feeding data into a model and adjusting its parameters
to minimize the error between the model’s predictions and the actual outcomes.
Techniques like gradient descent are commonly used for this.

4.Overfitting and Underfitting: Overfitting occurs when a model learns the

training data too well and performs poorly on new data. Underfitting happens
when the model is too simple to capture the underlying patterns in the data.

5.Evaluation Metrics: To assess how well a model is performing, various

metrics are used depending on the task, such as accuracy, precision, recall, F1
score for classification tasks, or mean squared error for regression.
6. Hyperparameter Tuning: This involves optimizing the parameters that
control the training process (like learning rate or the number of layers in a
neural network) to improve model performance.

7.Transfer Learning: This is a technique where a model trained on one task is

adapted for a different but related task, leveraging the knowledge gained from
the initial task.

8.Generative Models: These models, like Generative Adversarial Networks

(GANs), learn to generate new data samples that resemble the training data.
They are used in applications like creating realistic images or synthesizing new
content.

Understanding and effectively implementing these

aspects of modeling-learning in AI can help create
systems that are more accurate, efficient, and capable of
handling complex tasks.
•The inference paradigm in AI refers to the
methods and processes by which an AI
system makes predictions or decisions
based on learned models and new input
data.
•It involves using a trained model to infer
outcomes from unseen data.
•Given a model, the task of inference is to
answer questions with respect to the
model.
•Given the model of the city, one could ask
questions such as: what is the shortest
path? what is the cheapest path?
• The focus of inference is usually on
efficient algorithms that can answer these
questions. For some models, computational
complexity can be a concern (games such
as Go), and usually approximations are
 OPTIMIZATIO
• N improving models and algorithms to
AI involves
achieve better performance and efficiency
• It's a critical aspect of training and deploying AI
systems, as it helps in finding the best
parameters, minimizing errors, and ensuring
models generalize well to new data.
• Discrete optimization
• Continuous
Optimization optimization
Objectives
•Loss Function: This measures the discrepancy between the
model’s predictions and the actual outcomes. Common loss
functions include Mean Squared Error (MSE) for regression tasks
and Cross-Entropy Loss for classification tasks.
Discrete Optimization
Discrete optimization deals with problems where the
variables take on discrete values, such as integers or
categorical choices.
1.Combinatorial Problems: where the goal is to find
the best solution from a finite set of possible
solutions.

1.Search and Planning: Tasks such as pathfinding

in robotics or game AI often involve discrete
choices, like navigating a grid or selecting actions
from a finite set.
1.Graph-Based Problems:
1.Network Optimization: Algorithms like the
Traveling Salesman Problem (TSP) and Minimum
Spanning Tree (MST) are used in routing and network
design.
2.Inference in Graphical Models: Discrete variables
are common in models such as Bayesian networks
and Markov Random Fields, where inference involves
discrete state spaces.
2.Optimization in Machine Learning:
1.Hyperparameter Tuning: Often involves discrete
choices (e.g., number of layers in a neural network,
types of kernels in SVM). Techniques like grid search
and random search are used.
Techniques:
•Integer Programming: Solves problems with integer
constraints.
•Dynamic Programming: Efficiently solves problems by
breaking them into simpler subproblems.
•Branch and Bound: A method for solving integer
programming problems by exploring possible solutions
systematically.
•Heuristic and Metaheuristic Algorithms: Such as Genetic
Algorithms (GA) and Simulated Annealing (SA), which can
handle complex discrete problems with large search spaces.
Challenges:
•Computational Complexity: Discrete optimization problems
can be NP-hard, making exact solutions computationally
infeasible for large instances.
Classic examples of discrete optimization problems, along with
their descriptions and applications:

1. Traveling Salesman Problem (TSP)

Description: Given a set of cities and the distances between
each pair of cities, the goal is to find the shortest possible
route that visits each city exactly once and returns to the
starting city.

Applications:
•Route planning for logistics and delivery services.
•Circuit design in electronics.
•Vehicle routing in transportation and logistics.
Challenges:
•The problem is NP-hard, meaning that finding the exact
optimal solution can be computationally infeasible for large
2. Knapsack Problem

Description: Given a set of items, each with a weight and a

value, and a knapsack with a maximum weight capacity, the
goal is to select a subset of items that maximizes the total
value without exceeding the weight capacity.

Applications:
•Resource allocation and budget management.
•Investment decisions and portfolio optimization.
•Cutting stock problems in manufacturing.

Challenges:
•The problem can be solved exactly using dynamic
programming for small instances, but becomes
Graph Coloring
Description: Given a graph, the task is to assign
colors to the vertices such that no two adjacent
vertices have the same color, using the minimum
number of colors.

Applications:
•Frequency assignment in wireless networks to avoid
interference.
•Register allocation in compilers to manage CPU
registers.
•Scheduling problems where conflicts need to be
avoided.
Constraint Satisfaction Problems (CSPs)
Description: Involves finding values for variables
such that all constraints are satisfied. Constraints can
be unary (involving a single variable) or binary
(involving pairs of variables), and may be more
complex.

Applications:
•Sudoku and other puzzles.
•Scheduling and assignment problems.
•Configuration problems in systems design.
Challenges:
•CSPs can be solved using backtracking algorithms,
CONTINUOUS OPTIMIZATION
• Continuous optimization in AI focuses on problems
where the decision variables can take any value
within a continuous range.

• This is in contrast to discrete optimization, where

variables are restricted to specific, often integer,
values.

• Continuous optimization is particularly important

in machine learning, control systems, and various
other AI applications where smooth, real-valued
parameters are involved.
 Techniques for Continuous Optimization
1.Gradient-Based Methods:
1.Gradient Descent: Iteratively adjusts parameters in the
direction of the negative gradient of the objective function.
Variants include:
1.Stochastic Gradient Descent (SGD): Uses random
subsets of data to update parameters, improving
efficiency.
2.Mini-Batch
Evolutionary Gradient Descent:
and Metaheuristic Uses small batches of
Methods:
data to balance between SGD and Batch Gradient
•Genetic Algorithms: Inspired by natural evolution,
Descent.
they use operations such as mutation and crossover
to explore the solution space.
•Simulated Annealing: Mimics the annealing process
in metallurgy to escape local minima and find a global
Applications in AI
1.Machine Learning:
1.Training Neural Networks: Optimization
algorithms adjust weights and biases to minimize
loss functions, improving model performance.
2.Hyperparameter Tuning: Optimizing
continuous hyperparameters like learning rates or
regularization parameters.
3. Reinforcement Learning:
•Policy Optimization: Optimizing continuous action
spaces to find the best policy for decision-making tasks.
Challenges in Continuous Optimization

1.Local Minima:
1.Continuous optimization algorithms may get
stuck in local minima, especially in non-convex
problems.
2.Computational Complexity:
1.High-dimensional problems or problems with
complex constraints can be computationally
intensive.
3.Scalability:
1.Ensuring that optimization algorithms scale
efficiently with problem size and complexity.