CS:4420 Artificial Intelligence

Spring 2019

Constraint Satisfaction Problems

Cesare Tinelli

The University of Iowa

Copyright 2004–19, Cesare Tinelli and Stuart Russell a

a These notes were originally developed by Stuart Russell and are used with permission. They are
copyrighted material and may not be used in other course settings outside of the University of Iowa in their
current or modified form without the express written consent of the copyright holders.

CS:4420 Spring 2019 – p.1/34

 Readings

• Chap. 6 of [Russell and Norvig, 3rd Edition]

CS:4420 Spring 2019 – p.2/34

Constraint Satisfaction Problems (CSPs)
Standard search problem:
state is a “black box”—any old data structure
that supports goal test, eval, successor

CSP:
state is defined by variables Xi with values from domain Di
goal test is a set of constraints specifying
allowable combinations of values for subsets of variables

Simple example of a formal representation language

Allows useful general-purpose algorithms with more power than
standard search algorithms

CS:4420 Spring 2019 – p.3/34

 Example: Map coloring

Northern
Territory
Western Queensland
Australia

South
Australia
New South Wales

Victoria

Tasmania

Variables: MA, NT , Q, NSW , V , SA, T

Domains: Di = {r(ed), g(reen), b(lue)}
Constraints: adjacent regions must have different colors
e.g., WA 6= NT (if the language allows this), or
(WA, NT ) ∈ {(r, g), (r, b), (g, r), (g, b), . . .}
CS:4420 Spring 2019 – p.4/34
 Example: Map coloring contd.

Northern
Territory
Western Queensland
Australia

South
Australia
New South Wales

Victoria

Tasmania

Solutions are assignments satisfying all constraints,

e.g., {WA = r, NT = g, Q = r, NSW = g, V = r, SA = b, T = g}

CS:4420 Spring 2019 – p.5/34

 Constraint graph
Binary CSP: each constraint relates at most two variables
Constraint graph: nodes are variables, arcs show constraints

NT
Northern Q
Territory
Western Queensland
Australia WA
South
Australia
SA NSW
New South Wales

Victoria
V
Victoria

Tasmania

General-purpose CSP methods use the graph structure to speed up

search
e.g., Tasmania is an independent subproblem!

CS:4420 Spring 2019 – p.6/34

 Varieties of CSPs
Discrete variables
finite domains (size d)
• e.g., Boolean CSPs, incl. Boolean SAT (NP-complete)
• O(dn ) complete assignments
infinite domains (integers, strings, etc.)
• e.g., job scheduling, variables are start/end days for each job
• need a constraint language,
e.g., startJob 1 + 5 ≤ startJob 3
• linear constraints solvable, nonlinear undecidable

Continuous variables
• e.g., start/end times for Hubble Telescope observations
• linear constraints solvable in polynolmial time by linear
programming methods
CS:4420 Spring 2019 – p.7/34
 Varieties of constraints
Unary constraints involve a single variable
e.g., SA 6= g

Binary constraints involve pairs of variables

e.g., SA 6= WA

Higher-order constraints involve 3 or more variables

e.g., cryptarithmetic column constraints

Preferences are soft constraints

e.g., red is better than green
often representable by a cost for each variable assignment
→ constrained optimization problems

CS:4420 Spring 2019 – p.8/34

 Example: Cryptarithmetic

T WO F T U W R O
+ T WO
F O U R
X3 X2 X1

Variables: F, T, U, W, R, O, X1 , X2 , X3
Domain: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
Constraints: alldiff (F, T, U, W, R, O)
O + O = R + 10 · X1
...

CS:4420 Spring 2019 – p.9/34

 Example: 4-Queens as a CSP
Assume one queen in each column. Which row does each one go in?

Q1 = 1 Q2 = 3
Variables Q1 , Q2 , Q3 , Q4
Domains Di = {1, 2, 3, 4}
Constraints Qi 6= Qj (cannot be in same row)
|Qi − Qj | 6= |i − j| (cannot be on same diagonal)

Translate each constraint into set of allowable values for its variables
E.g., values for (Q1 , Q2 ) are (1, 3) (1, 4) (2, 4) (3, 1) (4, 1) (4, 2)
CS:4420 Spring 2019 – p.10/34
 Real-world CSPs
Assignment problems
e.g., who teaches what class

Timetabling problems
e.g., which class is offered when and where?

Hardware configuration

Transportation scheduling

Factory scheduling

Floorplanning

Notice that many real-world problems involve real-valued variables

CS:4420 Spring 2019 – p.11/34

Standard search formulation (incremental)
Let’s start with a basic, naive approach and then improve it

CS:4420 Spring 2019 – p.12/34

Standard search formulation (incremental)
Let’s start with a basic, naive approach and then improve it
States are defined by the values assigned so far
Initial state: the empty assignment, { }
Successor function: assign a value to an unassigned variable
that does not conflict with current assignment.
Fail if no legal assignments (not fixable!)
Goal test: the current assignment is complete

CS:4420 Spring 2019 – p.12/34

Standard search formulation (incremental)
Let’s start with a basic, naive approach and then improve it
States are defined by the values assigned so far
Initial state: the empty assignment, { }
Successor function: assign a value to an unassigned variable
that does not conflict with current assignment.
Fail if no legal assignments (not fixable!)
Goal test: the current assignment is complete
Note:
1. This is the same for all CSPs!
2. Every solution appears at depth n with n variables =⇒ use
depth-first search
3. Path is irrelevant, so can also use complete-state formulation
4. However, with domain of size d, branching factor b = (n − ℓ)d at
depth ℓ, hence n!dn leaves!
CS:4420 Spring 2019 – p.12/34
 Backtracking search
Order of variables in variable assignments is irrelevan
i.e., (WA = r, NT = g ) same as (NT = g , WA = r)

Only need to consider assignments to a single variable at each node

=⇒ b = d and there are dn leaves

Depth-first search for CSPs with single-variable assignments is called

backtracking search

Backtracking search is the basic uninformed algorithm for CSPs

Can solve n-queens for n ≈ 25

CS:4420 Spring 2019 – p.13/34

 Backtracking search
function Backtracking-Search(csp) returns a solution or failure
return Backtrack({}, csp)

function Backtrack(assignment, csp) returns a solution, or failure

if assignment is complete then return assignment
var ← Select-Unassignment-Variable(csp)
for each value in Order-Domain-Values(var, assignment, csp) do
if value is consistent with assignment then
add {var = value } to assignment
inferences ← Inference(csp, var, value) then
if inferences 6= failure then
add inferences to assignment
result ← Backtrack(assignment, csp)
if result 6= failure then return result
remove {var = value } and inferences from assignment
return failure

CS:4420 Spring 2019 – p.14/34

Backtracking example

CS:4420 Spring 2019 – p.15/34

Backtracking example

CS:4420 Spring 2019 – p.15/34

Backtracking example

CS:4420 Spring 2019 – p.15/34

Backtracking example

CS:4420 Spring 2019 – p.15/34

 Improving backtracking efficiency
General-purpose heuristics can yield huge gains in speed:
1. Which variable should be assigned next?
2. In what order should its values be tried?
3. Can we detect inevitable failure early?
4. Can we take advantage of problem structure?

CS:4420 Spring 2019 – p.16/34

 Variable choice heuristics
Minimum remaining values (MRV):
choose the variable with the fewest legal values

Degree heuristic:
choose the variable with the most constraints on remaining vars

Latter ofter used as a tie-breaker for former

CS:4420 Spring 2019 – p.17/34
 Value choice heuristics
Least constraining value:
For a given a variable, choose the least constraining value: the
one that rules out the fewest values in the remaining variables

Allows 1 value for SA

Allows 0 values for SA

Combining these heuristics makes 1000-queens feasible

Northern
Territory
Western Queensland
Australia

South
Australia
New South Wales

Victoria

CS:4420 Spring 2019 – p.18/34

Tasmania
 Forward checking
Idea: Keep track of remaining legal values for unassigned variables
Terminate search when any variable has no legal values

WA NT Q NSW V SA T

Northern
Territory
Western Queensland
Australia

South
Australia
New South Wales

Victoria

Tasmania

CS:4420 Spring 2019 – p.19/34

 Forward checking
Idea: Keep track of remaining legal values for unassigned variables
Terminate search when any variable has no legal values

WA NT Q NSW V SA T

Northern
Territory
Western Queensland
Australia

South
Australia
New South Wales

Victoria

Tasmania

CS:4420 Spring 2019 – p.19/34

 Forward checking
Idea: Keep track of remaining legal values for unassigned variables
Terminate search when any variable has no legal values

WA NT Q NSW V SA T

Northern
Territory
Western Queensland
Australia

South
Australia
New South Wales

Victoria

Tasmania

CS:4420 Spring 2019 – p.19/34

 Forward checking
Idea: Keep track of remaining legal values for unassigned variables
Terminate search when any variable has no legal values

WA NT Q NSW V SA T

Northern
Territory
Western Queensland
Australia

South
Australia
New South Wales

Victoria

CS:4420 Spring 2019 – p.19/34

Tasmania
 Constraint propagation
Forward checking propagates information from assigned to unassigned
variables, but doesn’t provide early detection for all failures:

WA NT Q NSW V SA T

NT and SA cannot both be blue!

Constraint propagation repeatedly enforces constraints locally

CS:4420 Spring 2019 – p.20/34

 Arc consistency
Simplest form of propagation, makes each arc consistent
Arc X → Y is consistent iff
for every value x of X there is some allowed value y for Y

WA NT Q NSW V SA T

CS:4420 Spring 2019 – p.21/34

 Arc consistency
Simplest form of propagation, makes each arc consistent
Arc X → Y is consistent iff
for every value x of X there is some allowed value y for Y

WA NT Q NSW V SA T

CS:4420 Spring 2019 – p.21/34

 Arc consistency
Simplest form of propagation, makes each arc consistent
Arc X → Y is consistent iff
for every value x of X there is some allowed value y for Y

WA NT Q NSW V SA T

If X loses a value, neighbors of X need to be rechecked

CS:4420 Spring 2019 – p.21/34

 Arc consistency
Simplest form of propagation, makes each arc consistent
Arc X → Y is consistent iff
for every value x of X there is some allowed value y for Y

WA NT Q NSW V SA T

If X loses a value, neighbors of X need to be rechecked

Arc consistency detects failure earlier than forward checking
Can be run as a preprocessor and/or after each assignment

CS:4420 Spring 2019 – p.21/34

 Arc consistency algorithm
function AC-3( csp) returns false if inconsistency found and true otherwise
inputs: csp, a binary CSP with components (X, D, C) and |C| = c
local vars: queue, a queue of arcs, initially all the arcs in csp

while queue is not empty do

(Xi , Xj ) ← Remove-First(queue)
if Revise(csp, Xi , Xj ) then
if size of Di = 0 then return false
for each Xk in Xi .Neighbors − {Xi } do add (Xk , Xi ) to queue

function Revise( Xi , Xj ) returns true iff Xi ’s domain is revised

revised ← false
for each x in Di do
if no value y in Dj allows (x,y) to satisfy the constraint between Xi and Xj then
delete x from Di
revised ← true
return revised

O(cd3 ), can be reduced to O(cd2 ) (detecting all inconsistencies is NP-hard)

CS:4420 Spring 2019 – p.22/34
 Further notions of consistency
Node consistency: A single variable X is node-consistent if all the
values in X ’s domain D(X) satisfy the unary constraints on X

Ex.
D(X) = {1, 2, 3} C1 = (X > 0) X node-consist. with C1
D(X) = {1, 2, 3} C2 = (X > 5) X not node-consist. with C2

CS:4420 Spring 2019 – p.23/34

 Further notions of consistency
Arc-consistency for n-constraints

Generalized arc consistency: A variable Xi is generalized

arc-consistent wrt an n-ary constraint C(X1 , . . . , Xi , . . . , Xn ) if,
for every v ∈ D(Xi ),
there is a (v1 , . . . , v, . . . , vn ) ∈ D(X1 ) × · · · × D(Xi ) × · · · × D(Xn )
that satisfies C

Ex.
D(X) = D(Y ) = D(Z) = {1, 2, 3}

C1 = (X + Y > Z) Y generalized arc-consist. with C1

C2 = (X + Y < Z) Z not generalized arc-consist. with C2

CS:4420 Spring 2019 – p.24/34

 Further notions of consistency
Chained arc-consistency

Path consistency: A two-variable set {X, Z} is path-consistent wrt a

third variable Y if,
for every assignment satisfying the constraints on {X, Z},
there is an assignment to Y that satisfies the constraints on {X, Y }
and {Y, Z}

Ex.
D(X) = D(Y ) = D(Z) = {1, 2, 3, 4}

{X > 2 · Z, X > Y, Y = Z + 1} {X, Z} path-consistent wrt Y

{X > 2 · Z, X < Y, Y = Z + 1} not {X, Z} path-consistent wrt Y

CS:4420 Spring 2019 – p.25/34

 Problem structure

NT
Q
WA

SA NSW

V
Victoria

Tasmania and mainland are independent subproblems

Identifiable as connected components of constraint graph

CS:4420 Spring 2019 – p.26/34

 Problem structure
Suppose each subproblem has c variables out of n total
n
Worst-case solution cost is c · dc , which is linear in n

E.g., n = 80, d = 2, c = 20

280 = 4 billion years at 10 million nodes/sec

4 · 220 = 0.4 seconds at 10 million nodes/sec

CS:4420 Spring 2019 – p.27/34

 Tree-structured CSPs
A E
B D
C F

Theorem: If the constraint graph has no loops, the CSP can be solved
in O(n d2 ) time

Compare to general CSPs, where worst-case time is O(dn )

This property also applies to logical and probabilistic reasoning:

an important example of the relation between
• syntactic restrictions and
• the complexity of reasoning
CS:4420 Spring 2019 – p.28/34
 Algorithm for tree-structured CSPs
1. Choose a variable as root, order variables from root to leaves
so that every node’s parent precedes it in the ordering

A E
B D A B C D E F
C F

2. For j from n down to 2, apply

MakeArcConsistent(P arent(Xj ), Xj )

3. For j from 1 to n, assign Xj consistently with P arent(Xj )

CS:4420 Spring 2019 – p.29/34

 Nearly tree-structured CSPs
Conditioning: instantiate a variable, prune its neighbors’ domains

NT NT
Q Q
WA WA

SA NSW NSW

V
Victoria V
Victoria

T T

Cutset conditioning: instantiate (in all ways) a set of variables

so that the remaining constraint graph is a tree

Cutset size c =⇒ runtime O(dc · (n − c)d2 ), very fast for small c

CS:4420 Spring 2019 – p.30/34

 Further Optimizations
• Tree decomposition

• Symmetry breaking

CS:4420 Spring 2019 – p.31/34

 Iterative algorithms for CSPs
Hill-climbing, simulated annealing typically work with
complete states, i.e., with all variables assigned

To apply to CSPs:
allow states with unsatisfied constraints
operators reassign variable values

Variable selection: randomly select any conflicted variable

Value selection by min-conflicts heuristic:

choose value that violates the fewest constraints
i.e., hillclimb with h(n) = total number of violated constraints

CS:4420 Spring 2019 – p.32/34

 Example: 4-Queens
States: 4 queens in 4 columns (44 = 256 states)
Operators: move queen in column
Goal test: no attacks
Evaluation: h(n) = number of attacks

h=5 h=2 h=0

CS:4420 Spring 2019 – p.33/34

 Performance of min-conflicts
Given random initial state, can solve n-queens in almost constant time
for arbitrary n with high probability (e.g., n = 10,000,000)
The same appears to be true for any randomly-generated CSP
except in a narrow range of the ratio

number of constraints
R=
number of variables
CPU
time

R
critical
ratio

The critical ration corresponds to a phase transition for the problems,

from satisfiable to unsatisfiable
CS:4420 Spring 2019 – p.34/34