CONSTRAINT SATISFACTION PROBLEMS

Fabrizio Santini | COMP 131A

VERSION 1.1

1
  Constraint Satisfaction Problems
 Solving CSPs
 Filtering
 Variable ordering
 Value ordering
 Smart backtracking
TODAY ON AI  Problem structure
 Questions?

2
Constraint Satisfaction Problems

SECTION 01

3
ARTIFICIAL INTELLIGENCE Constraint Satisfaction problems (CSPs) 101

Constraint satisfaction problems (or CSPs) belong to a class of

problems for which the goal itself is the most important part, not the
path used to reach it.

EXAMPLES
 Map coloring!  Assignment problems
 Sudokus  Transportation scheduling
 Crossword puzzles  Fault diagnosis
 Job scheduling  More…
 Cryptarithmetic puzzles
 N-Queens problems
 Hardware configuration

4
ARTIFICIAL INTELLIGENCE Constraint Satisfaction problems (CSPs) 102

The state of a CSP is defined by variables with values from domain :

 Discrete variables:
• Domains can be finite: a finite of size set of values or things (means complete
assignments). Examples are: Boolean values, specific meaningful numbers, set of
colors, etc.
• or infinite: integers or strings. Examples are: strings for a crossword puzzle, duration of
jobs in seconds, etc.

 Continuous variables:
• Domains are infinite. Examples are: start/end times for Hubble Telescope observations
as they obey to astronomical time laws

5
ARTIFICIAL INTELLIGENCE Constraint Satisfaction problems (CSPs) 103

 The goal test is a set of constraints that specifies allowable combinations

of values for subsets of variables:
• Constraints can be explicit (explicitly enumerated)
• or implicit (a formula describes it)
• Constraints can be: unary, binary, global, alldiff

 Constraints are generally represented with a graph, called hypergraph, that

shows the relationship between the variables

 Soft constraints represent preferences about some values of the variables.

They usually come with a cost value that expresses the strength of the
preference

6
ARTIFICIAL INTELLIGENCE Constraint Satisfaction problems (CSPs) 104
EXAMPLE: COLORING AUSTRALIA

 VARIABLES
 Solutions are assignments that
WA, NT, Q, NSW, V, SA, T
satisfying all constraints:
 DOMAINS
{ } WA NT Q
NSW V SA
 CONSTRAINTS
NORTHERN Adjacent regions must T
TERRITORY have different colors:
QUEENSLAND
WESTERN
AUSTRALIA Implicit: WA SA,
SOUTH
AUSTRALIA WA NT, NT SA,
NEW SOUTH NT Q, Q SA, Q NSW,
WALES
NSW SA, NSW V, V
VICTORIA SA
Explicit: , ∈
,
etc.
TASMANIA

7
ARTIFICIAL INTELLIGENCE Constraint Satisfaction problems (CSPs) 105
EXAMPLES: SUDOKU

 VARIABLES
Open squares

 DOMAINS
{1, 2, 3, … 9}

 CONSTRAINTS

9-way alldiff for each column

9-way alldiff for each row
9-way alldiff for each region

 RULES

Fill all empty squares so that the numbers 1 to

9 appear once in each row, column and 3x3
box.

8
Solving CSPs

SECTION 2

9
ARTIFICIAL INTELLIGENCE Solving CSPs 201
STANDARD SEARCH FORMULATION

The idea is to use standard search algorithms (DFS and BFS) to find a solution
that satisfies all the constraints.

 STATES
The variables assigned with values so far

 INITIAL STATE
All variable assignments are empty

 SUCCESSOR FUNCTION
Assign a value to an unassigned variable

 GOAL TEST
The current assignment is complete and satisfies all constraints

It’s a naïve approach, but it’s a start.

10
ARTIFICIAL INTELLIGENCE Solving CSPs 202
CHRONOLOGICAL BACKTRACKING SEARCH

Chronological backtracking search is an uninformed searching algorithm

based on the Depth-first searching algorithm with some improvements
related to CSPs.

 IMPROVEMENT 1
Each step considers only one assignment at the time

 IMPROVEMENT 2
Check constraints as the search continues. Consider only new assignments
which do not conflict previous assignments (incremental goal test)

11
ARTIFICIAL INTELLIGENCE Backtracking search 203
COLORING EXAMPLE

12
ARTIFICIAL INTELLIGENCE Backtracking search 204
PSEUDO-CODE

1 function Backtracking-search(csp) returns SOLUTION, or FAILURE

2 return Recursive-backtracking({}, csp)
3
function Recursive-backtracking(assignment, csp) returns SOLUTION, or FAILURE
4
if assignment is complete then
5 return assignment
6
7 variable = Select-unassigned-variable(variables[csp], assignment, csp)
8 for each value in Order-Domain-Value(variable, assignment, csp) do
9 if value is consistent with assignment given Constraints[csp] then
10 add {variable = value} to assignment
11 result = Recursive-backtracking(assignment, csp)
12
if result failure then
13 return result
14 remove {variable = value} from assignment
15
16 return FAILURE

13
ARTIFICIAL INTELLIGENCE Backtracking search 205
IMPROVING BY BAILING OUT EARLIER

We can improve backtracking even more with some additional improvements:

 IMPROVEMENT 1
Taking divination class: filter out inevitable failures as early as possible

 IMPROVEMENT 2
Do not choose poorly: choose carefully which variable for assignment

 IMPROVEMENT 3
You should never, never doubt something that no one is sure of: choose judicially
what value to use

 IMPROVEMENT 4
Where we're going we don't need... roads: choose judicially where to backtrack to

 IMPROVEMENT 5
See the whole board: use the topology of the problem, or its structure, to assign
variables

Professor Sybill Trelawney: Professor of divination

Indiana Jones: Templar knight
Charlie and the chocolate factory: Roald Dahl
Back to the future: Dr. Emmet Brown
West wing: President Bartlet

14
Filtering

SECTION 03

15
ARTIFICIAL INTELLIGENCE Filtering
FORWARD CHECKING

 Forward checking keeps track of the domains for the unassigned variables
and remove possible bad options right away

16
ARTIFICIAL INTELLIGENCE Filtering
ARC CONSISTENCY

Arc consistency is one form of constraint propagation that tries to

prune illegal assignment before they happen
 While evaluating , an arc → from a neighbor is consistent if and
only if every ∈ there is some ∈ which could be assigned without
violating a constraint:

Arc consistency on is also triggered

if the domain of ( that is) changes.

17
ARTIFICIAL INTELLIGENCE Filtering
ARC CONSISTENCY: THE ALGORITHM

1 function AC-3(csp) returns SOLUTION

2 push all arcs in queue
3 while queue is not empty do
4 pop arc( , ) from queue
5
6 if Remove-Inconsistent-Values( , ) then
7 for each in Neighbors( ) do
8 add( , ) to queue
9
10 function Remove-Inconsistent-Values( , )
11 removed = false
12 for each x in Domain( ) do
13 if no value y in Domain( )
14 allow (x, y) to satisfy the constraint ↔ then
15 delete x from Domain( )
16 removed = true
17 return removed

18
ARTIFICIAL INTELLIGENCE Filtering
ARC CONSISTENCY CANNOT RUN ALONE

 Arc consistency has to run inside a backtracking search because:

• There might still be one or more solutions left
• There might be no solution left

1 SOLUTION ? SOLUTIONS ? SOLUTIONS

19
Variable ordering

SECTION 04

20
ARTIFICIAL INTELLIGENCE Ordering
PICKING A VARIABLE

In order to prune even more illegal assignments, we also have to consider how
we choose the variables:

Minimum Remaining Values (MRV): Degree Heuristic: choose the variable

choose the variable with the fewest legal with the highest number of constraints
values in its domain

Why do we choose the minimum rather than the maximum?

21
Value ordering

SECTION 05

22
ARTIFICIAL INTELLIGENCE Ordering 501
VALUE ORDERING: PICKING THE VALUE

Least Constraining Value (LCV): Once the variable is selected, choose the
value that rules out the fewest choices for the neighbors

Why do we choose the minimum rather than the maximum?

23
Smart backtracking

SECTION 06

24
ARTIFICIAL INTELLIGENCE Intelligent backtracking 601
LOOKING BACKWARD

 The minimal form of backtracking is known as chronological backtracking

 Backjumping: A smarter approach is to jump back to the most recent
conflict using the idea of conflict sets

VARIABLE ORDER
Q, NSW, V, T, SA, WA, NT

CONFLICT SET CONFLICT SET CONFLICT SET CONFLICT SET

1: Q = 2: NSW = 3: V = 3: V =
1: Q = 2: NSW = 2: NSW =
1: Q = 1: Q =

25
ARTIFICIAL INTELLIGENCE Intelligent backtracking 602
CONFLICT-DIRECTED BACKJUMPING

In conflict-directed backjumping the conflict sets migrate from one

variable to another:

CONFLICT SET
1: WA = CONFLICT SET SA CONFLICT SET
2: NT = 4: Q =
3: NT =
1. WA = CONFLICT SET 1: NSW =
2: NSW =
2. NSW = 1: WA =
3. T=
4. NT =
CONFLICT SET Q CONFLICT SET CONFLICT SET
5. Q= 2: NT = 3: NT =
1: NSW = 2: NSW =
CONFLICT SET
1: WA =
4: Q =
SA = ? 3: NT =
2: NSW =
1: WA =
NT CONFLICT SET CONFLICT SET
1: WA = 2: NSW =
1: WA =

NSW

26
Problem structure

SECTION 07

27
ARTIFICIAL INTELLIGENCE Problem structure 701
DIVIDE AND CONQUER

Independent sub-problems can make life much

easier:
 The worst-case complexity of a solution search is
normally . For a problem with 60 and 2 (a
binary domain), and assuming 1M node/s evaluation, the
search takes 36,558 years

 In the case the problem can be broken into smaller

problems with variables, worst-case complexity is
. For the same problem above, with 20, the
search would only be 3s

 Independent sub-problems are identifiable as

connected components of the constraint graph

28
ARTIFICIAL INTELLIGENCE Problem structure 702
TREE-STRUCTURED CSPS

If the constraint graph has no loops, the CSP can be solved in .

1. Remove backward: Apply arc consistency from the deepest leaf to its
parent. After this phase, all arcs are consistent.
2. Assign forward: Assign a value to the variable consistent with its parent.
Forward assignment will never backtrack.

29
ARTIFICIAL INTELLIGENCE Problem structure 703
NEARLY TREE-STRUCTURED CSPS

Sometimes is possible to find one or more variables that, if instantiated,

transform the constraint graph into a tree. This process is called cutset
conditioning.

With a cutset of size , complexity of nearly tree-structured CSPs is

The process requires to instantiate the variables of the cutset and prune its
neighbors’ domain.

30
ARTIFICIAL INTELLIGENCE Problem structure 704
NEARLY TREE-STRUCTURED CSPS: EXAMPLE

INSTANTIATE THE CUTSET

PRUNE THE REMAINING DOMAINS

SOLVE THE RESIDUAL CSPS

31
QUESTIONS ?

32