Artificial Intelligence

Lecture 7

Bicol University
1st Semester 2023-2024
Constraint Satisfaction Problem
(CSP)
Constraint satisfaction
Constraint problems
satisfaction problems (CSPs)
●
Standard search problem:
– state is a "black box“ – any data structure that supports successor
function, heuristic function, and goal test
●
CSP:
– state is defined by variables Xi with values from domain Di
– goal test is a set of constraints specifying allowable combinations
of values for subsets of variables

●
Simple example of a formal representation language

●
Allows useful general-purpose algorithms with more power
than standard search algorithms
Constraint satisfaction
Constraint Satisfaction Problems problems
●
What is a CSP? (Problem Formulation)
– Finite set of variables X1, X2, …, Xn
– Nonempty domain of possible values for each variable
D1, D2, …, Dn
– Finite set of constraints C1, C2, …, Cm
●
Each constraint Ci limits the values that variables can take,
●
e.g., X1 ≠ X2
– Each constraint Ci is a pair <scope, relation>
●
Scope = Tuple of variables that participate in the constraint.
●
Relation = List of allowed combinations of variable values.
– May be an explicit list of allowed combinations,e.g.⟨(X1,X2 ),
{(3,1),(3,2),(2,1)}⟩
– May be an abstract relation allowing membership testing and
listing,e.g. ⟨(X1 ,X2 ),X1 > X2⟩
CSP example: map coloring
CSP example: map coloring

 Variables: WA, NT, Q, NSW, V, SA, T

 Domains: Di={red,green,blue}
 Constraints:adjacent regions must have different colors.
 E.g. WA NT (if the language allows this)
WA≠NT
 E.g. (WA,NT) {(red,green),(red,blue),(green,red),…}
CSPs --- what is a solution?

●
A state is an assignment of values to some or all variables.
– An assignment is complete when every variable has a value.
– An assignment is partial when some variables have no values.

●
Consistent assignment
– assignment does not violate the constraints

●
A solution to a CSP is a complete and consistent
assignment.

●
Some CSPs require a solution that maximizes an objective
function.
CSP example: map coloring

 Solutions are assignments satisfying all constraints, e.g.

{WA=red,NT=green,Q=red,NSW=green,V=red,SA=blue,T=gre
en}
Constraint graphs
Constraint graphs

• Constraint graph:

• nodes are variables

• arcs are binary constraints

• Graph can be used to simplify search

e.g. Tasmania is an independent subproblem
Varieties of CSPs
● Discrete variables

Finite domains; size d 
O(dn) complete assignments.
- E.g. Boolean CSPs, include. Boolean satisfiability (NP-
complete).
•
Infinite domains (integers, strings, etc.)
- E.g. job scheduling, variables are start/end days for each job
- Need a constraint language e.g StartJob1 +5 ≤ StartJob3.
- Linear constraints solvable, nonlinear undecidable.
● Continuous variables

e.g. start/end times for Hubble Telescope observations.

Linear constraints solvable in poly time by LP methods.
Varieties
Varieties ofof constraints
constraints

●
Unary constraints involve a single variable.
– e.g. SA  green
– initial specification of the domain of a variable can also be seen as a unary constraint.

●
Binary constraints involve pairs of variables.
– e.g. SA  WA

●
Higher-order constraints involve 3 or more
variables.
– Professors A, B,and C cannot be on a committee together
– Can always be represented by multiple binary constraints

●
Preference (soft constraints)
– e.g. red is better than green often can be represented by a cost for each variable assignment
– combination of optimization with CSPs
CSP Example: Cryptharithmetic puzzle
Each letter stands for a distinct digit; the aim
is to find a substitution of digits for letters
such that the resulting sum is arithmetically
correct, with the added restriction that no
leading zeroes are allowed.
CSP
CSPExample: Cryptharithmetic
Example: Cryptharithmetic puzzle puzzle
constraint
hypergraph

hypernodes (the squares)-

represent n-ary constraints
(constraints involving n
variables)
Real-world CSPs

●
Assignment problems
– e.g., who teaches what class
●
Timetabling problems
– e.g., which class is offered when and where?
●
Transportation scheduling
●
Factory scheduling

●
Notice that many real-world problems
involve real-valued variables
CSP as a standard search problem
●
A CSP can easily be expressed as a standard search
problem.

●
Incremental formulation

– Initial State: the empty assignment {}

– Successor function: Assign a value to an unassigned variable provided
that it does not violate a constraint
– Goal test: the current assignment is complete
(by construction it is consistent)

– Path cost: constant cost for every step (not really relevant)

●
Can also use complete-state formulation
– Local search techniques
CSP as a standard search problem
●
This is the same for all CSP’s !!!

●
Solution is found at depth n (if there are n variables).

●
Hence depth first search can be used.

●
Path is irrelevant, so complete state representation can also be used.

●
Branching factor b at the top level is nd.

●
b=(n-l)d at depth l, hence n!dn leaves (only dn complete assignments).
Commutativity

CSPs are commutative.

● The order of any given set of actions has no effect on the
outcome.

● Example: choose colors for Australian territories one at a

time

- [WA=red then NT=green] same as [NT=green then WA=red]

- All CSP search algorithms consider a single variable
assignment at a time there are dn leaves.
Backtracking search

 Cfr. Depth-first search

 Chooses values for one variable at a time

and backtracks when a variable has no legal
values left to assign.

 Uninformed algorithm
No good general performance
Backtracking search

function BACKTRACKING-SEARCH(csp) return a solution or failure

return RECURSIVE-BACKTRACKING({} , csp)

function RECURSIVE-BACKTRACKING(assignment, csp) return a solution or failure

if assignment is complete then return assignment
var ←SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp],assignment,csp)
for each value in ORDER-DOMAIN-VALUES(var, assignment, csp) do
if value is consistent with assignment according to CONSTRAINTS[csp] then
add {var=value} to assignment
result ← RECURSIVE-BACTRACKING(assignment, csp)
if result = failure then return result
remove {var=value} from assignment
return failure
Backtracking example
Backtracking example
Backtracking example
Backtracking example
Idea 2: Improving backtracking efficiency

 General-purpose methods & heuristics can

give huge gains in speed, on average
 Heuristics:
Q: Which variable should be assigned next?
1. Most constrained variable
2. Most constraining variable

Q: In what order should that variable’s values be tried?

3. Least constraining value

Q: Can we detect inevitable failure early?

4. Forward checking
Heuristic 1: Most constrained variable

var SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp],assignment,csp)

 a.k.a. Minimum remaining values (MRV)

 Rule: choose variable with the fewest legal moves
 Which variable shall we try first?
Heuristic 2: Most constraining variable

 a.k.a. degree heuristic

 Rule: select variable that is involved in the largest number of
constraints on other unassigned variables.
 Degree heuristic is very useful as a tie breaker.
 In what order should its values be tried?
Heuristic 3: Least constraining value

 Least constraining value heuristic

 Rule: given a variable choose the least constraing value i.e.
the one that leaves the maximum flexibility for subsequent
variable assignments.
Heuristic 4: Forward checking

 Can we detect inevitable failure early?

And avoid it later?
 Forward checking idea: keep track of remaining legal values
for unassigned variables.
 Terminate search when any variable has no legal values.
Forward checking

 Assign {WA=red}
 Effects on other variables connected by constraints with WA
NT can no longer be red
SA can no longer be red
Forward checking

 Assign {Q=green}
 Effects on other variables connected by constraints with WA
NT can no longer be green
NSW can no longer be green
SA can no longer be green

 MRV heuristic will automatically select NT and SA next, why?

Forward checking

 If V is assigned blue
 Effects on other variables connected by constraints with WA
SA is empty
NSW can no longer be blue
 FC has detected that partial assignment is inconsistent with the
constraints and backtracking can occur.

 A Step toward AC-3: The most efficient algorithm

Example: 4-Queens Problem
Place 4 Queens on a chess board of 4x4 such
that no two queens reside in the same row,
column or diagonal.
1 2 3 4
• Variables: ?
• Domains: ?
1
• Constraints:?
2
3
4

[4-Queens slides copied from B.J. Dorr CMSC 421 course on AI]
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}

[4-Queens slides copied from B.J. Dorr CMSC 421 course on AI]
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , ,3,4}
1
2
3
4
X3 X4
{ ,2, ,4} { ,2,3, }
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , ,3,4}
1
2
3
4
X3 X4
{ ,2, ,4} { ,2,3, }
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , ,3,4}
1
2
3
4
X3 X4
Backtrack!!!
{ , , , } { ,2,3, }
Example: 4-Queens Problem
Picking up a little later after two steps of
backtracking....

X1 X2
1 2 3 4 { ,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, ,3, } {1, ,3,4}
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, ,3, } {1, ,3,4}
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } {1, ,3, }
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } {1, ,3, }
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } { , ,3, }
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } { , ,3, }
Towards Constraint propagation

 Solving CSPs with combination of heuristics plus forward

checking is more efficient than either approach alone.
 FC checking propagates information from assigned to
unassigned variables but does not provide detection for all
failures.
NT and SA cannot be blue!
 Constraint propagation is the general term for propagating the
implications of a constraint on one variable onto other
variables
Idea 3 (big idea): Inference in CSPs
 CSP solvers combine search and inference
– Search

●
assigning a value to a variable
– Constraint propagation (inference)
●
Eliminates possible values for a variable if the value
would violate local consistency
– Can do inference first, or intertwine it with search
 Local consistency
• Node consistency: satisfies unary constraints
—This is trivial!
• Arc consistency : satisfies binary constraints
—Xi is arc-consistent w.r.t. Xj if for every value v in Di , there is some
value w in Dj that satisfies the binary constraint on the arc between Xi
and Xj .
Example: Sudoku

•Variables: 81 slots

•Domains =
{1,2,3,4,5,6,7,8,9}

•Constraints:
•27 not-equal

Constraint
propagation
23
426

• Each row, column and major block must be alldifferent

• “Well posed” if it has unique solution
Arc consistency

 X -->Y is consistent iff

for every value x of X there is some allowed y
 SA -->NSW is consistent iff
SA=blue and NSW=red
Arc consistency

 X -->Y is consistent iff

for every value x of X there is some allowed y
 NSW --> SA is consistent iff
NSW=red and SA=blue
NSW=blue and SA=???
Arc can be made consistent by removing blue from NSW
Arc consistency

 Arc can be made consistent by removing blue from NSW

 RECHECK neighbours !!
Remove red from V
Arc consistency

 Arc can be made consistent by removing blue from NSW

 RECHECK neighbours !!
Remove red from V
 Arc consistency detects failure earlier than FC
 Can be run as a preprocessor or after each assignment.
Repeated until no inconsistency remains

 Arc consistency does not detect all inconsistencies:

Partial assignment {WA=red, NSW=red} is inconsistent
Arc consistency

 Ideally, if no solutions, would remove all values from the domain.

 Isn’t always effective
Arc consistency algorithm
function AC-3(csp) return the CSP, possibly with reduced domains
inputs: csp, a binary csp with variables {X1, X2, …, Xn}
local variables: queue, a queue of arcs initially the arcs in csp

while queue is not empty do

(Xi, Xj) REMOVE-FIRST(queue)
if REMOVE-INCONSISTENT-VALUES(Xi, Xj) then
for each Xk in NEIGHBORS[Xi ] do
add (Xk, Xi) to queue

function REMOVE-INCONSISTENT-VALUES(Xi, Xj) return true iff we remove a value

removed false
for each x in DOMAIN[Xi] do
if no value y in DOMAIN[Xj] allows (x,y) to satisfy the constraints between Xi and Xj
then delete x from DOMAIN[Xi]; removed true
return removed
 CSP using ARC CONSISTENCY
(AC-3 Algorithm)
Three employees in a small company, Alice, Rob and Ian
must contact their respective customers as quickly as possible.
The company has one telephone, one fax and one computer (for
email), with independent lines. Alice needs to contact two
customers: one who only has a telephone and the second one
has both a telephone and a fax. Rob must contact a customer
who has also has a telephone and a fax. Ian's customer can be
reached by fax or by computer. Suppose that Alice contacts her
customers one after the other while, during the same time, Rob
and Ian are communicating with their customers.
A) Model each problem as a CSP by specifying clearly the
variables, their domains and constraints.
B) Draw the constraint graph for each CSP.
C) Apply AC-3 to the problem.
 CSP using ARC CONSISTENCY
(AC-3 Algorithm)
CSP FORMULATION:
CONSTRAINT GRAPH
Variables: {A1, A2, R, I} where
A1: Alice's 1st customer
A2: Alice's 2nd customer
R: Rob's customer
I: Ian's customer
Domain: A1 = {T} ,
A2 & R = {T, F} ,
I = {F, C} where
T: telephone
F: fax
C: computer
Constraints:
(1) Alldiff(A1, R, I)
(2) Alldiff(A2, R, I)
CSP using ARC CONSISTENCY
(AC-3 Algorithm)
CSP using ARC CONSISTENCY
(AC-3 Algorithm)
CSP using ARC CONSISTENCY
(AC-3 Algorithm)
Summary

X Red
green
blue
X: red
X • Search
backtracking,
Y Z Red Red
variable/value Y: blue
green green heuristics

Y
blue blue

Z •Inference Z: green
Coloring Consistency
Constraint Graph
Problem enforcement,
forward checking

Variables
Problem Values CSP
Statement Algorithm Solution
Constraints

CSP Representation
End