Module - 3 (Search in Complex environments)

Adversarial search - Games, Optimal decisions in games, The Minimax algorithm, Alpha-Beta
pruning. Constraint Satisfaction Problems – Defining CSP, Constraint Propagation- inference in
CSPs, Backtracking search for CSPs, Structure of CSP problems.

Games

➢ In multiagent environments, each agent needs to consider the actions of other agents and
how they affect its own welfare.

➢ Adversarial search is a method applied to a situation where you are planning while another
actor prepares against you.(game theory).

➢ Adversarial search problems typically exist in two-player games where the players’ actions
alternate.

➢ Examples are chess, checkers, and tic-tac-toe.

➢ The agents can be cooperative or competitive.

➢ In competitive environments, the agents’ goals are in conflict, giving rise to Adversarial
Search problems—often known as GAMES.

➢ In mathematical game theory, a multiagent environment is treated as a game, the impact of

each agent (economy) on the others is significant, regardless of whether the agents are
cooperative or competitive.

➢ In AI, the most common games are deterministic, turn-taking, two-player, zero-sum games of
perfect information (such as chess).

•In deterministic and fully observable environments in two agents act alternately and in which
the utility values at the end of the game are always equal and opposite.
•If one player wins a game of chess, the other player necessarily loses.
•It is this opposition between the agents’ utility functions that makes the situation
adversarial.

➢ Searches in which two or more players with conflicting goals are trying to explore the same
search space for the solution, are called adversarial searches, often known as Games.
➢ Games are modeled as a Search problem and heuristic evaluation function, and these are the
two main factors which help to model and solve games in AI.

o Perfect information: A game with the perfect information is that in which agents can look into
the complete board. Agents have all the information about the game, and they can see each other
moves also. Examples are Chess, Checkers, Go, etc.
o Imperfect information: If in a game agents do not have all information about the game and not
aware with what's going on, such type of games are called the game with imperfect information,
such as tic-tac-toe, Battleship, blind, Bridge, etc.
o Deterministic games: Deterministic games are those games which follow a strict pattern and
set of rules for the games, and there is no randomness associated with them. Examples are chess,
Checkers, Go, tic-tac-toe, etc.
o Non-deterministic games: Non-deterministic are those games which have various
unpredictable events and has a factor of chance or luck. This factor of chance or luck is
introduced by either dice or cards. These are random, and each action response is not fixed. Such
games are also called as stochastic games.
Example: Backgammon, Monopoly, Poker, etc.
Games as Search Problem

A game can be formally defined as a kind of search problem with the following elements.

•S0: The initial state, which specifies how the game is set up at the start.
•PLAYER(s): Defines which player has the move in a state.
•ACTIONS(s): Returns the set of legal moves in a state.
•RESULT(s, a): The transition model, which defines the result of a move.
•TERMINAL-TEST(s): A terminal test, which is true when the game is over and falseotherwise.
States where the game has ended are called terminal states.
•UTILITY(s, p): A utility function (also called objective function or payoff function),defines the
final numeric value for a game that ends in terminal state s for a player p. In

•A zero-sum game is (constant-sum is a better term) defined as one where the total payoff to all
players is the same for every instance of the game.

TECHNIQUES REQUIRED TO GET THE BEST OPTIMAL SOLUTION

• There is always a need to choose those algorithms which provide the best optimal solution in a
limited time. The following techniques are used to fulfill our requirements:

1)Pruning: A technique which allows ignoring the unwanted portions of a search tree which
make no difference in its final result.
2)Heuristic Evaluation Function: It allows to approximate the cost value at each level of the
search tree, before reaching the goal node.

•We have two players: MAX and MIN.

•MAX (our player) moves first in the game.
•Regardless of size of game tree, MAX’s (our player’s) job to search for a good move in search tree.
•A search algorithm does not explore the full game tree, and examines enough nodes to allow a player
to determine what move to make.
 Types of algorithms in Adversarial search
• In a normal search, we follow a sequence of actions to reach the goal or to finish the game optimally.
But in an adversarial search, the result depends on the players which will decide the result of the
game.
• It is also obvious that the solution for the goal state will be an optimal solution because the player will
try to win the game with the shortest path and under limited time.
• There are following types of adversarial search:
1. Minmax Algorithm
2. Alpha-beta Pruning
MIN MAX ALGORITHM

• Min-max algorithm is a recursive or backtracking algorithm which is used in decision-making and

game theory. It provides an optimal move for the player assuming that opponent is also playing
optimally.

• Min-Max algorithm uses recursion to search through the game- tree.

• Min-Max algorithm is mostly used for game playing in AI, such as Chess, Checkers, tic-tac-toe, go
etc.

• This Algorithm computes the min-max decision for the current state.
• In this algorithm two players play the game, one is called MAX and other is called MIN.
• MAX will select the maximized value
• MIN will select the minimized value.
• The min-max algorithm performs a depth-first search algorithm for the exploration of the complete
game tree.

• The min-max algorithm proceeds all the way down to the terminal node of the tree, then backtrack
the tree as the recursion.

Minimax search procedure

• Consider games with 2 players. The opponents in a game are referred to as MIN and MAX.
• MAX represents the player trying to win or to MAXimize his advantage.
• MIN is the opponent who attempts to MINimize MAX’s score.
• MAX moves first, and then they take turns moving until the game is over.
• At the end of the game, points are awarded to the winning player and penalties are given to the
loser.
Optimal Decisions in Games
• Optimal Solution in a search problem is a sequence of actions leading to a goal state (a terminal
state that is a win).

•MAX must find a contingent strategy, which specifies MAX’s move in the initial state, then MAX’s
moves in the states resulting from every possible response by MIN.

•Given a game tree, optimal strategy can be determined from minimax value of each node n, and it is
called as MINIMAX(n).

•The minimax value of a node is the utility (for MAX) of being in the corresponding state, assuming
that both players play optimally from there to the end of the game.

•The minimax value of a terminal state is just its utility.

•MAX prefers to move to a state of maximum value, whereas MIN prefers a state of minimum value.

MINIMAX (A twoply game tree)

•Δ nodes are “MAX nodes,” in which it is MAX’s turn to move, and ∇ nodes are “MIN nodes.”

•Terminal nodes show the utility values for MAX; other nodes are labeled with their minimax values.

•MAX’s best move at the root is a1, because it leads to the state with the highest minimax value, and
MIN’s best reply is b1, because it leads to the state with the lowest minimax value.
Example:
Step-1: In the first step, the algorithm generates the entire game-tree and apply the utility function to
get the utility values for the terminal states. In the below tree diagram, let's take A is the initial state of
the tree. Suppose maximizer takes first turn which has worst-case initial value =- infinity, and
minimizer will take next turn which has worst-case initial value = +infinity.

Step 2: Now, first we find the utilities value for the Maximizer, its initial value is -∞, so we will
compare each value in terminal state with initial value of Maximizer and determines the higher nodes
values. It will find the maximum among the all.
• For node D max(-1,- -∞) => max(-1,4)= 4
• For Node E max(2, -∞) => max(2, 6)= 6
• For Node F max(-3, -∞) => max(-3,-5) = -3
• For node G max(0, -∞) = max(0, 7) = 7

Step 4: Now it's a turn for Maximizer, and it will again choose the maximum of all nodes value and
find the maximum value for the root node. In this game tree, there are only 4 layers, hence we reach
immediately to the root node, but in real games, there will be more than 4 layers.
• For node A max(4, -3)= 4
Properties of Mini-Max algorithm:

➢ Complete- Min-Max algorithm is Complete. It will definitely find a solution (if exist), in the
finite search tree.
➢ Optimal- Min-Max algorithm is optimal if both opponents are playing optimally.
➢ Time complexity- As it performs DFS for the game-tree, so the time complexity of Min-Max
algorithm is O(bm), where b is branching factor of the game-tree, and m is the maximum
depth of the tree.
➢ Space Complexity- Space complexity of Mini-max algorithm is also similar to DFS which is
O(bm).

Limitation

➢ It requires to visit every node in the tree. So space complexity is too much.

Working of Alpha-Beta Pruning:

Let's take an example of two-player search tree to understand the working of Alpha- beta pruning
Step 1: At the first step the, Max player will start first move from node A where α= -∞ and β= +∞,
these value of alpha and beta passed down to node B where again α= -∞ and β= +∞, and Node B
passes the same value to its child D.
Step 2: At Node D, the value of α will be calculated as its turn for Max. The value of α is compared
with firstly 2 and then 3, and the max (2, 3) = 3 will be the value of α at node D and node value will
also 3.
Step 3: Now algorithm backtrack to node B, where the value of β will change as this is a turn of Min,
Now β= +∞, will compare with the available subsequent nodes value,
i.e. min (∞, 3) = 3, hence at node B now α= -∞, and β= 3.

In the next step, algorithm traverse the next successor of Node B which is node E, and the values
of α= -∞, and β= 3 will also be passed.
Step 4: At node E, Max will take its turn, and the value of alpha will change. The current value of alpha
will be compared with 5, so max (-∞, 5) = 5, hence at node E α= 5 and β= 3, where α>=β, so the right
successor of E will be pruned, and algorithm will not traverse it, and the value at
node E will be 5.

Step 5: At next step, algorithm again backtrack the tree, from node B to node A. At node A, the value
of alpha will be changed the maximum available value is 3 as max (-∞, 3)= 3, and β= +∞, these two
values now passes to right successor of A which is Node C.
At node C, α=3 and β= +∞, and the same values will be passed on to node F.
Step 6: At node F, again the value of α will be compared with left child which is 0, and max(3,0)= 3,
and then compared with right child which is 1, and max(3,1)= 3 still α remains 3, but the node value of
F will become 1.

Step 7: Node F returns the node value 1 to node C, at C α= 3 and β= +∞, here the value of beta
will be changed, it will compare with 1 so min (∞, 1) = 1. Now at C, α=3 and β= 1, and again it satisfies
the condition α>=β, so the next child of C which is G will be pruned, and the algorithm will not compute
the entire sub-tree G.

Step 8: C now returns the value of 1 to A here the best value for A is max (3, 1) = 3. Following is the
final game tree which is the showing the nodes which are computed and nodes which has never
computed. Hence the optimal value for the maximizer is 3 for this example.
Constraint satisfaction Problem
➢ Constraint satisfaction is a technique where a problem is solved when its values satisfy certain
constraints or rules of the problem.
➢ Constraint satisfaction is a problem-solving technique in which the values of a problem
satisfy certain constraints or rules of the problem.

Popular Problems with CSP

• CryptArithmetic (Coding alphabets to numbers.)
• n-Queen (In an n-queen problem, n queens should be placed in an n×n matrix such that no
queen shares the same row, column or diagonal.)
• Map Coloring (coloring different regions of map, ensuring no adjacent regions have the same
color)
• Crossword (everyday puzzles appearing in newspapers)
• Sudoku (a number grid)

➢ A CSP consists of 3 components:

➢ Each domain Di consists of a set of allowable values, {v1, …, vk} for variable Xi.

Assignment:

➢ Each state in a CSP is defined by an assignment of values to some of the variables, {X i=vi,

Xj=vj, …};

➢ An assignment that does not violate any constraints is called a consistent or legal assignment;
 ➢ A complete assignment is one in which every variable is assigned;

➢ A solution to a CSP is a consistent, complete assignment;

➢ A partial assignment is one that assigns values to only some of the variables.

Solving Constraint Satisfaction Problems

The requirements to solve a constraint satisfaction problem (CSP) is:

• A state-space
• The notion of the solution.

A state in state-space is defined by assigning values to some or all variables such as

{X1=v1, X2=v2, and so on…}.

A State-space

➢ The state-space is a set of all possible configurations of variable assignments, each of which is
a potential solution to the problem.
➢ The state space can be searched using various algorithms.

The Notion of the Solution

➢ The notion of a solution in CSP depends on the specific problem being solved.
➢ In general, a solution is a complete assignment of values to all the variables in a way that
satisfies all the constraints.

Domains

Two tyoes of domains which are used by variables are:

• Boolean: {true, false} or {0,1} (binary).

• Finite: a limited number of possible values for each variable.
• Discrete: The domain is countably infinite.(eg:positive integers{1,2,3…}
• Continuous: real numbers.
• A set of linear constraints on continuous domains can be solved in polynomial time by linear
programming.

Varieties of constraints

➢ Unary constraints involve a single variable. q e.g. SA ≠ green

➢ Binary constraints involve pairs of variables. q e.g. SA ≠ WA
➢ Global constraints involve an arbitrary number of variables.
➢ Preference (soft constraints) e.g. red is better than green often representable by a cost for each
variable assignment;

Example: Map-Coloring

•We are given the task of coloring each region of Australia either red, green, or bluein such a way that
no neighboring regions have the same color.

Variables: Each region is a variable:X = {WA,NT,Q,NSW, V,SA, T} .

Domains: The domain of each variable is the set Di= {red, green, blue}.
Constraintsrequire neighboring regions to have distinct colors.
•Since there are nine places where regions border, there are nine constraints.

C = {SA≠WA, SA ≠NT, SA≠Q, SA≠NSW, SA≠V,

WA≠NT, NT≠Q,Q ≠NSW, NSW≠V } .

Example: Map-Coloring

Variables :WA, NT, Q, NSW, V , SA, T

Domains : Di= {red, green, blue}

Constraints:
•adjacent regions must have different colors.

•e.g., WA≠NT (if the language allows this), or (WA,NT) ε{(red,green), (red,blue),(green,red),…}

Constraint graph

Binary CSP: each constraint relates two variables .

Constraint graph: nodes are variables, edges are constraints.

Constraint Propagation- inference in CSPs
There are several different types of local consistency: node consistency, arc consistency, path
consistency, k-consistency

Arc consistency
AC-3 algorithm (Arc consistency algorithm)

Path consistency
K consistency

Backtracking search
Structure of CSP problems
The structure of constraint graph

The structure of the problem as represented by the constraint graph can be used to find solution
quickly.

e.g. The map coloring problem can be decomposed into 2 independent subproblems: Coloring T and
coloring the mainland.

• Tree: A constraint graph is a tree when any two variable are connected by only one path.
• Directed arc consistency (DAC): A CSP is defined to be directed arc-consistent under an
ordering of variables X1, X2, … , Xn if and only if every Xi is arc-consistent with each Xj for j>i.
• By using DAC, any tree-structured CSP can be solved in time linear in the number of
variables.
How to solve a tree-structure CSP:
Pick any variable to be the root of the tree;
Choose an ordering of the variable such that each variable appears after its parent in the tree.
(topological sort)
Any tree with n nodes has n-1 arcs, so we can make this graph directed arc-consistent in O(n) steps,
each of which must compare up to d possible domain values for 2 variables, for a total time of O(nd 2).
Once we have a directed arc-consistent graph, we can just march down the list of variables and choose
any remaining value.
Since each link from a parent to its child is arc consistent, we won’t have to backtrack, and can move
linearly through the variables.