Lecture Notes

Course Name- Artificial Intelligence

Course Code- PEC-IT501B

Lecture No. 2
Module: Introduction
Topic: AI technique, Tic - Tac - Toe problem.

Outline of the Lecture

1.1 Introductory Problem-Tic Tac Toe

Beyond the Curriculum:

Topic:
1.1 Java Graphics Tutorial
 Lecture Notes
Course Name- Artificial Intelligence
Course Code- PEC-IT501B

Introductory Problem-Tic Tac Toe

The game Tic Tac Toe is also known as Noughts and Crosses or Xs and Os ,the player needs to take turns marking
the spaces in a 3x3 grid with their own marks, if 3 consecutive marks (Horizontal, Vertical, Diagonal) are formed
then the player who owns these moves get won.

Assume ,a

Player 1 - X
Player 2 - O

Lecture 2: Topic: Ai Technique, Tic - Tac – Toe

Prepared by Prof. Tapashri Sur
2
 Lecture Notes
Course Name- Artificial Intelligence
Course Code- PEC-IT501B

So, a player who gets 3 consecutive marks first, they will win the game .

Let's have a discussion about how a board's data structure looks and how the Tic Tac Toe algorithm works.

Board's Data Structure:

The cells could be represent as Center square,Corner,Edge as like below

Number each square starts from 1 to 9 like following image

Consider a Board having nine elements vector. Each element will contain
• 0 for blank
• 1 indicating X player move
• 2 indicating O player move
Computer may play as X or O player. First player is always X.

Lecture 2: Topic: Ai Technique, Tic - Tac – Toe

Prepared by Prof. Tapashri Sur
3
 Lecture Notes
Course Name- Artificial Intelligence
Course Code- PEC-IT501B

Move Table
It is a vector of 3^9 elements, each element of which is a nine-element vector representing board position.
Total of 3^9(19683) elements in move table
Move Table
Index Current Board Position New Board position
0 000000000 000010000
1 000000001 020000001
2 000000002 000100002
3 000000010 002000010
.
.
Algorithm
To make a move, do the following:
1. View the vector (board) as a ternary number and convert it to its corresponding decimal number.
2. Use the computed number as an index into the move table and access the vector stored there.
3. The vector selected in step 2 represents the way the board will look after the move that should be made.
So set board equal to that vector.
Let's start with empty board

Step 1: Now our board looks like 000 000 000 (ternary number) convert it into decimal no. The decimal no is 0
Step 2: Use the computed number ie 0 as an index into the move table and access the vector stored in
New Board Position.
The new board position is 000 010 000
Step 3: The vector selected in step 2(000 010 000 ) represents the way the board will look after the move that should
be made. So set board equal to that vector.
After complete the 3rd step your board looks like.

Lecture 2: Topic: Ai Technique, Tic - Tac – Toe

Prepared by Prof. Tapashri Sur
4
 Lecture Notes
Course Name- Artificial Intelligence
Course Code- PEC-IT501B

This process continues until the player get win or tie.

Flowchart:

Lecture 2: Topic: Ai Technique, Tic - Tac – Toe

Prepared by Prof. Tapashri Sur
5
 Lecture Notes
Course Name- Artificial Intelligence
Course Code- PEC-IT501B

Java Graphics Tutorial

Case Study on Tic-Tac-Toe: With AI
1. Playing Against Computer with AI (Advanced)

Click the image to run the demo for the various AI strategies (under the "Options" menu):

Tic-tac-toe seems dumb, but it actually requires you to lookahead one opponent's move to ensure that you will
not loss. That is, you need to consider your opponent's move after your next move.
For example, suppose that the computer uses 'O'. At (D), the computer did not consider the opponent's next move
and place at the corner (which is preferred over the side). At (E), the opponent was able to block the computer's
winning move and create a fork.

X| | X| | X| | X| | X| |X
----------- ----------- ----------- ----------- -----------
| | |O| |O| |O| |O|
----------- ----------- ----------- ----------- -----------
| | | | | |X O| |X O| |X
(A) (B) (C) (D) (E)

1.1 Implementing the AI Player

To test the various AI strategies, an abstract superclass called AI Player is defined, which takes the Board as an
argument in its constructor (because you need the board position to compute the next move). An abstract method
called move() is defined, which shall be implemented in subclasses with the chosen strategy.
1 /**
2 * Abstract superclass for all AI players with different strategies.
3 * To construct an AI player:
4 * 1. Construct an instance (of its subclass) with the game Board
5 * 2. Call setSeed() to set the computer's seed
6 * 3. Call move() which returns the next move in an int[2] array of {row, col}.
7 *
8 * The implementation subclasses need to override abstract method move().
9 * They shall not modify Cell[][], i.e., no side effect expected.
10 * Assume that next move is available, i.e., not game-over yet.
11 */
12 public abstract class AIPlayer {
13 protected int ROWS = GameMain.ROWS; // number of rows
14 protected int COLS = GameMain.COLS; // number of columns
15

Lecture 2: Topic: Ai Technique, Tic - Tac – Toe

Prepared by Prof. Tapashri Sur
6
 Lecture Notes
Course Name- Artificial Intelligence
Course Code- PEC-IT501B

16 protected Cell[][] cells; // the board's ROWS-by-COLS array of Cells

17 protected Seed mySeed; // computer's seed
18 protected Seed oppSeed; // opponent's seed
19
20 /** Constructor with reference to game board */
21 public AIPlayer(Board board) {
22 cells = board.cells;
23 }
24
25 /** Set/change the seed used by computer and opponent */
26 public void setSeed(Seed seed) {
27 this.mySeed = seed;
28 oppSeed = (mySeed == Seed.CROSS) ? Seed.NOUGHT : Seed.CROSS;
29 }
30
31 /** Abstract method to get next move. Return int[2] of {row, col} */
32 abstract int[] move(); // to be implemented by subclasses
33 }

1.2 Simplest Strategy – Heuristic Preferences via Table Lookup

The simplest computer strategy is to place the seed on the first empty cell in this order: the centre, one of the four
corners, one of the four sides. This dumb strategy, of course, does not work. It merely gets you started
programming the computer play.
For example,

1 /**
2 * Computer move based on simple table lookup of preferences
3 */
4 public class AIPlayerTableLookup extends AIPlayer {
5
6 // Moves {row, col} in order of preferences. {0, 0} at top-left corner
7 private int[][] preferredMoves = {
8 {1, 1}, {0, 0}, {0, 2}, {2, 0}, {2, 2},
9 {0, 1}, {1, 0}, {1, 2}, {2, 1}};
10
11 /** constructor */
12 public AIPlayerTableLookup(Board board) {
13 super(board);
14 }
15
16 /** Search for the first empty cell, according to the preferences
17 * Assume that next move is available, i.e., not gameover
18 * @return int[2] of {row, col}
19 */
20 @Override
21 public int[] move() {
Lecture 2: Topic: Ai Technique, Tic - Tac – Toe
Prepared by Prof. Tapashri Sur
7
 Lecture Notes
Course Name- Artificial Intelligence
Course Code- PEC-IT501B

22 for (int[] move : preferredMoves) {

23 if (cells[move[0]][move[1]].content == Seed.EMPTY) {
24 return move;
25 }
26 }
27 assert false : "No empty cell?!";
28 return null;
29 }
30 }
1.3 Heuristic Board Evaluation Function
In this strategy, we need to formula a heuristic evaluation function, which returns a relative score, e.g., +∞ for
computer-win, -∞ for opponent-win, 0 for neutral, and a number in between to indicate the relative advantage of
the computer vs. the opponent.
In Tic-Tac-Toe, a possible heuristic evaluation function for the current board position is:
• +100 for EACH 3-in-a-line for computer.
• +10 for EACH two-in-a-line (with a empty cell) for computer.
• +1 for EACH one-in-a-line (with two empty cells) for computer.
• Negative scores for opponent, i.e., -100, -10, -1 for EACH opponent's 3-in-a-line, 2-in-a-line and 1-
in-a-line.
• 0 otherwise (empty lines or lines with both computer's and opponent's seeds).
For Tic-Tac-Toe, compute the scores for each of the 8 lines (3 rows, 3 columns and 2 diagonals) and obtain the
sum.
For an Othello (Reversi), the heuristic evaluation function could be the difference of computer's seeds over
opponent's seeds.
To implement this strategy, you need to compute the score for all the valid moves, and place the seed at the
position with the highest score. This strategy does not work in Tic-Tac-Toe (and in most of the board game)
because it does not lookahead opponent's next move.
1.4 Rule-based Strategy
For Tic-tac-toe, the rules, in the order of importance, are:
• Rule 1: If I have a winning move, take it.
• Rule 2: If the opponent has a winning move, block it.
• Rule 3: If I can create a fork (two winning ways) after this move, do it.
• Rule 4: Do not let the opponent creating a fork after my move. (Opponent may block your winning
move and create a fork.)
• Rule 5: Place in the position such as I may win in the most number of possible ways.
Rule 1 and 2 can be programmed easily. Rule 3 is harder. Rule 4 is even harder because you need to lookahead
one opponent move, after your move. For rule 5, you need to count the number of possible winning ways.
Rule-based strategy is only applicable for simple game such as Tic-tac-toe and Othello.
1.5 Minimax Search Algorithm

Lecture 2: Topic: Ai Technique, Tic - Tac – Toe

Prepared by Prof. Tapashri Sur
8
 Lecture Notes
Course Name- Artificial Intelligence
Course Code- PEC-IT501B

First, decide on a heuristic board evaluation function (see above section).

For Tic-Tac-Toe, the function could be as simple as returning +1 if the computer wins, -1 if the player wins, or 0
otherwise. However, simple evaluation function may require deeper search.
A better evaluation function for Tic-Tac-Toe is:
• +100 for EACH 3-in-a-line for computer.
• +10 for EACH 2-in-a-line (with a empty cell) for computer.
• +1 for EACH 1-in-a-line (with two empty cells) for computer.
• Negative scores for opponent, i.e., -100, -10, -1 for EACH opponent's 3-in-a-line, 2-in-a-line and 1-
in-a-line.
• 0 otherwise (empty lines or lines with both computer's and opponent's seed).
Compute the scores for each of the 8 lines (3 rows, 3 columns and 2 diagonals) and obtain the sum.
The principle of minimax is to minimize the maximum possible loss.
As an illustration, suppose that there are only one or two possible moves per player in each turn. Furthermore, an
evaluation function has been defined, which returns +∞ if the computer wins, -∞ if the computer loses, and a
score in between to reflect the relative advantage of the computer. Computer (or the maximizing player) is
represented by circle. The opponent (or the minimizing player) is represented by square. We limit the lookahead
to four moves.
The algorithm evaluates the leaf nodes (terminating "gameover" nodes or at maximum depth of 4) using the
heuristic evaluation function, obtaining the values shown. At level 3, the minimizing player will choose, for each
node, the minimum of its children. In level 2, the maximizing player chooses the maximum of the children. The
algorithm continues evaluating the maximum and minimum values of the child nodes alternately until it reaches
the root node, where it chooses the move with the maximum value. This is the move that the player should make
in order to minimize the maximum possible loss.

Minimax is recursive. The algorithm is as follows:

minimax(level, player) // player may be "computer" or "opponent"
if (gameover || level == 0)
return score
children = all legal moves for this player
if (player is computer, i.e., maximizing player)
// find max
bestScore = -inf

Lecture 2: Topic: Ai Technique, Tic - Tac – Toe

Prepared by Prof. Tapashri Sur
9
 Lecture Notes
Course Name- Artificial Intelligence
Course Code- PEC-IT501B

for each child

score = minimax(level - 1, opponent)
if (score > bestScore) bestScore = score
return bestScore
else (player is opponent, i.e., minimizing player)
// find min
bestScore = +inf
for each child
score = minimax(level - 1, computer)
if (score < bestScore) bestScore = score
return bestScore

// Initial Call
minimax(2, computer)

1 import java.util.*;
2 /** AIPlayer using Minimax algorithm */
3 public class AIPlayerMinimax extends AIPlayer {
4
5 /** Constructor with the given game board */
6 public AIPlayerMinimax(Board board) {
7 super(board);
8 }
9
10 /** Get next best move for computer. Return int[2] of {row, col} */
11 @Override
12 int[] move() {
13 int[] result = minimax(2, mySeed); // depth, max turn
14 return new int[] {result[1], result[2]}; // row, col
15 }
16
17 /** Recursive minimax at level of depth for either maximizing or minimizing player.
18 Return int[3] of {score, row, col} */
19 private int[] minimax(int depth, Seed player) {
20 // Generate possible next moves in a List of int[2] of {row, col}.
21 List<int[]> nextMoves = generateMoves();
22
23 // mySeed is maximizing; while oppSeed is minimizing
24 int bestScore = (player == mySeed) ? Integer.MIN_VALUE : Integer.MAX_VALUE;
25 int currentScore;
26 int bestRow = -1;
27 int bestCol = -1;
28
29 if (nextMoves.isEmpty() || depth == 0) {
30 // Gameover or depth reached, evaluate score
31 bestScore = evaluate();
32 } else {

Lecture 2: Topic: Ai Technique, Tic - Tac – Toe

Prepared by Prof. Tapashri Sur
10
 Lecture Notes
Course Name- Artificial Intelligence
Course Code- PEC-IT501B

33 for (int[] move : nextMoves) {

34 // Try this move for the current "player"
35 cells[move[0]][move[1]].content = player;
36 if (player == mySeed) { // mySeed (computer) is maximizing player
37 currentScore = minimax(depth - 1, oppSeed)[0];
38 if (currentScore > bestScore) {
39 bestScore = currentScore;
40 bestRow = move[0];
41 bestCol = move[1];
42 }
43 } else { // oppSeed is minimizing player
44 currentScore = minimax(depth - 1, mySeed)[0];
45 if (currentScore < bestScore) {
46 bestScore = currentScore;
47 bestRow = move[0];
48 bestCol = move[1];
49 }
50 }
51 // Undo move
52 cells[move[0]][move[1]].content = Seed.EMPTY;
53 }
54 }
55 return new int[] {bestScore, bestRow, bestCol};
56 }
57
58 /** Find all valid next moves.
59 Return List of moves in int[2] of {row, col} or empty list if gameover */
60 private List<int[]> generateMoves() {
61 List<int[]> nextMoves = new ArrayList<int[]>(); // allocate List
62
63 // If gameover, i.e., no next move
64 if (hasWon(mySeed) || hasWon(oppSeed)) {
65 return nextMoves; // return empty list
66 }
67
68 // Search for empty cells and add to the List
69 for (int row = 0; row < ROWS; ++row) {
70 for (int col = 0; col < COLS; ++col) {
71 if (cells[row][col].content == Seed.EMPTY) {
72 nextMoves.add(new int[] {row, col});
73 }
74 }
75 }
76 return nextMoves;
77 }
78
79 /** The heuristic evaluation function for the current board

Lecture 2: Topic: Ai Technique, Tic - Tac – Toe

Prepared by Prof. Tapashri Sur
11
 Lecture Notes
Course Name- Artificial Intelligence
Course Code- PEC-IT501B

80 @Return +100, +10, +1 for EACH 3-, 2-, 1-in-a-line for computer.
81 -100, -10, -1 for EACH 3-, 2-, 1-in-a-line for opponent.
82 0 otherwise */
83 private int evaluate() {
84 int score = 0;
85 // Evaluate score for each of the 8 lines (3 rows, 3 columns, 2 diagonals)
86 score += evaluateLine(0, 0, 0, 1, 0, 2); // row 0
87 score += evaluateLine(1, 0, 1, 1, 1, 2); // row 1
88 score += evaluateLine(2, 0, 2, 1, 2, 2); // row 2
89 score += evaluateLine(0, 0, 1, 0, 2, 0); // col 0
90 score += evaluateLine(0, 1, 1, 1, 2, 1); // col 1
91 score += evaluateLine(0, 2, 1, 2, 2, 2); // col 2
92 score += evaluateLine(0, 0, 1, 1, 2, 2); // diagonal
93 score += evaluateLine(0, 2, 1, 1, 2, 0); // alternate diagonal
94 return score;
95 }
96
97 /** The heuristic evaluation function for the given line of 3 cells
98 @Return +100, +10, +1 for 3-, 2-, 1-in-a-line for computer.
99 -100, -10, -1 for 3-, 2-, 1-in-a-line for opponent.
100 0 otherwise */
101 private int evaluateLine(int row1, int col1, int row2, int col2, int row3, int col3) {
102 int score = 0;
103
104 // First cell
105 if (cells[row1][col1].content == mySeed) {
106 score = 1;
107 } else if (cells[row1][col1].content == oppSeed) {
108 score = -1;
109 }
110
111 // Second cell
112 if (cells[row2][col2].content == mySeed) {
113 if (score == 1) { // cell1 is mySeed
114 score = 10;
115 } else if (score == -1) { // cell1 is oppSeed
116 return 0;
117 } else { // cell1 is empty
118 score = 1;
119 }
120 } else if (cells[row2][col2].content == oppSeed) {
121 if (score == -1) { // cell1 is oppSeed
122 score = -10;
123 } else if (score == 1) { // cell1 is mySeed
124 return 0;
125 } else { // cell1 is empty
126 score = -1;

Lecture 2: Topic: Ai Technique, Tic - Tac – Toe

Prepared by Prof. Tapashri Sur
12
 Lecture Notes
Course Name- Artificial Intelligence
Course Code- PEC-IT501B

127 }
128 }
129
130 // Third cell
131 if (cells[row3][col3].content == mySeed) {
132 if (score > 0) { // cell1 and/or cell2 is mySeed
133 score *= 10;
134 } else if (score < 0) { // cell1 and/or cell2 is oppSeed
135 return 0;
136 } else { // cell1 and cell2 are empty
137 score = 1;
138 }
139 } else if (cells[row3][col3].content == oppSeed) {
140 if (score < 0) { // cell1 and/or cell2 is oppSeed
141 score *= 10;
142 } else if (score > 1) { // cell1 and/or cell2 is mySeed
143 return 0;
144 } else { // cell1 and cell2 are empty
145 score = -1;
146 }
147 }
148 return score;
149 }
150
151 private int[] winningPatterns = {
152 0b111000000, 0b000111000, 0b000000111, // rows
153 0b100100100, 0b010010010, 0b001001001, // cols
154 0b100010001, 0b001010100 // diagonals
155 };
156
157 /** Returns true if thePlayer wins */
158 private boolean hasWon(Seed thePlayer) {
159 int pattern = 0b000000000; // 9-bit pattern for the 9 cells
160 for (int row = 0; row < ROWS; ++row) {
161 for (int col = 0; col < COLS; ++col) {
162 if (cells[row][col].content == thePlayer) {
163 pattern |= (1 << (row * COLS + col));
164 }
165 }
166 }
167 for (int winningPattern : winningPatterns) {
168 if ((pattern & winningPattern) == winningPattern) return true;
169 }
170 return false;
171 }
172 }

Lecture 2: Topic: Ai Technique, Tic - Tac – Toe

Prepared by Prof. Tapashri Sur
13
 Lecture Notes
Course Name- Artificial Intelligence
Course Code- PEC-IT501B

Note: The pseudocode presented in Wiki "minimax" is known as "negamax", which is very hard to understand,
and even harder to program and debug.
1.6 Minimax with Alpha-beta Pruning
Reference: Wiki "Alpha-beta pruning".
Alpha-beta pruning seeks to reduce the number of nodes that needs to be evaluated in the search tree by the
minimax algorithm. For example, in the alpha cut-off, since node D returns 1, node C (MIN) cannot be more than
1. But node B is 4. There is no need to search the other children of node C, as node A will certainly pick node B
over node C.

In the algorithm, two parameters are needed: an alpha value which holds the best MAX value found for MAX
node; and a beta value which holds the best MIN value found for MIN node. As illustrated, the remaining children
can be aborted if alpha ≥ beta, for both the alpha cut-off and beta cut-off. Alpha and beta are initialized to -∞ and
+∞ respectively.
The recursive algorithm for "minimax with alpha-beta pruning" is as follows:

minimax(level, player, alpha, beta) // player may be "computer" or "opponent"

if (gameover || level == 0)
return score
children = all valid moves for this "player"
if (player is computer, i.e., max's turn)
// Find max and store in alpha
for each child
score = minimax(level - 1, opponent, alpha, beta)
if (score > alpha) alpha = score
if (alpha >= beta) break; // beta cut-off
return alpha
else (player is opponent, i.e., min's turn)
// Find min and store in beta
for each child
score = minimax(level - 1, computer, alpha, beta)
if (score < beta) beta = score
if (alpha >= beta) break; // alpha cut-off
return beta

// Initial call with alpha=-inf and beta=inf

minimax(2, computer, -inf, +inf)
The relevant changes (over the AIPlayerMinimax.java) are:
/** Get next best move for computer. Return int[2] of {row, col} */
@Override

Lecture 2: Topic: Ai Technique, Tic - Tac – Toe

Prepared by Prof. Tapashri Sur
14
 Lecture Notes
Course Name- Artificial Intelligence
Course Code- PEC-IT501B

int[] move() {
int[] result = minimax(2, mySeed, Integer.MIN_VALUE, Integer.MAX_VALUE);
// depth, max-turn, alpha, beta
return new int[] {result[1], result[2]}; // row, col
}

/** Minimax (recursive) at level of depth for maximizing or minimizing player

with alpha-beta cut-off. Return int[3] of {score, row, col} */
private int[] minimax(int depth, Seed player, int alpha, int beta) {
// Generate possible next moves in a list of int[2] of {row, col}.
List<int[]> nextMoves = generateMoves();

// mySeed is maximizing; while oppSeed is minimizing

int score;
int bestRow = -1;
int bestCol = -1;

if (nextMoves.isEmpty() || depth == 0) {
// Gameover or depth reached, evaluate score
score = evaluate();
return new int[] {score, bestRow, bestCol};
} else {
for (int[] move : nextMoves) {
// try this move for the current "player"
cells[move[0]][move[1]].content = player;
if (player == mySeed) { // mySeed (computer) is maximizing player
score = minimax(depth - 1, oppSeed, alpha, beta)[0];
if (score > alpha) {
alpha = score;
bestRow = move[0];
bestCol = move[1];
}
} else { // oppSeed is minimizing player
score = minimax(depth - 1, mySeed, alpha, beta)[0];
if (score < beta) {
beta = score;
bestRow = move[0];
bestCol = move[1];
}
}
// undo move
cells[move[0]][move[1]].content = Seed.EMPTY;
// cut-off
if (alpha >= beta) break;
}
return new int[] {(player == mySeed) ? alpha : beta, bestRow, bestCol};
}
}

Lecture 2: Topic: Ai Technique, Tic - Tac – Toe

Prepared by Prof. Tapashri Sur
15
 Lecture Notes
Course Name- Artificial Intelligence
Course Code- PEC-IT501B

Outline of the Next Lecture

Lecture No. 3
Module: Introduction

Assignment
(Previous year question paper discussion from MAKAUT and GATE)

Lecture 2: Topic: Ai Technique, Tic - Tac – Toe

Prepared by Prof. Tapashri Sur
16