Assignment 3

NAME: Navya Sanjay Sharma SUBJECT: Artificial Intelligence - LAB

ID: 191071049 BRANCH: Computer Science

SUBMISSION: 05/09/2021 SEMESTER: V

AIM:
A. Write a program to randomly generate k-SAT problems. The program must accept values for k, m the
number of clauses and n the number of variables. Each clause of length k must contain distinct variables
or their negations. Instances generated by this algorithm belong to the fixed clause length model of SAT
and are known as the Uniform Random k-SAT problem.

B. Write programs to solve a set of Uniform Random 3-SAT problems for different combinations of m and n
and compare their performance. Try the
1. Hill Climbing algorithm,
2. Beam Search with beam width of 3 and 4 ,
3. Variable Neighbourhood Descent with 3 Neighbourhood functions and
4. Tabu Search with neighbourhood functions changing 2 bits at a time.

THEORY:
The Boolean satisfiability (SAT) problem is defined as follows:
Given a Boolean formula, check whether an assignment of Boolean values to the propositional variables in the
formula exists, such that the formula evaluates to true.

If such an assignment exists, the formula is said to be satisfiable; otherwise, it is unsatisfiable. For a formula
with m variables, there are 2m possible truth assignments. The conjunctive normal form (CNF) (X1 ∨ X2) ∧
(X3 ∨ X4) ∧ · · · ∧ (Xn−1 ∨ Xn) is most frequently used for representing Boolean formulas, where ¬∀Xi are
independent. In CNF, the variables of the formula appear in literals (e.g., x) or their negation (e.g., ¬x (logical
NOT ¬)). Literals are grouped into clauses, which represent a disjunction (logical OR ∨) of the literals they
contain. A single literal can appear in any number of clauses. The conjunction (logical AND ∧) of all clauses
represents a formula. Several algorithms are known for solving the 2 - satisfiability problem; the most efficient
of them take linear time.

The K-satisfiability (Ksat) problem deals with an ensemble of N Boolean variables, submitted to M constraints.
Each constraint is in the form of an ’OR’ function of K variables in the ensemble (or their negations), and the
problem is to know whether there exists one configuration of the variables (among the 2N possible ones) which
satisfies all constraints.

The Ksat problem for K ≥ 3 is a central problem in combinatorial optimization: it was the first problem to be
shown NP-complete, and an efficient algorithm for solving Ksat in its worst case instances would immediately
lead to other algorithms for solving efficiently thousands of different hard combinatorial problems.
1
The study of random Ksat problems, where the clauses are chosen randomly, is also interesting from the
viewpoint of optimization. In practice, algorithms which are used to solve real-world NP-complete problems
display a huge variability of running times, ranging from linear to exponential, when the parameters (e.g. the
number of clauses) are changed.

Hill Climbing Algorithm

● Hill climbing algorithm is a local search algorithm which continuously moves in the direction of
increasing elevation/value to find the peak of the mountain or best solution to the problem. It
terminates when it reaches a peak value where no neighbor has a higher value i.e. it returns the local
maximum. Hill climbing algorithm is a technique which is used for optimizing mathematical problems.
● It is also called greedy local search as it only looks to its good immediate neighbor state and not beyond
that. A node of hill climbing algorithm has two components which are state and value. Hill Climbing is
mostly used when a good heuristic is available. In this algorithm, we don't need to maintain and handle
the search tree or graph as it only keeps a single current state.
● It is an iterative algorithm that starts with an arbitrary solution to a problem and attempts to find a
better solution by changing a single element of the solution incrementally. If the change produces a
better solution, an incremental change is taken as a new solution. This process is repeated until there
are no further improvements. It returns a state that is a local maximum.

Beam Search
● The Beam Search algorithm holds k number of states at any given time. It has a hyper parameter named
beam size. At the start, these states are generated randomly. The successors of these k states are
computed with the help of objective function. If any of these successors is the maximum value of the
objective function, then the algorithm stops.
● Otherwise the (initial k states and k number of successors of the states = 2k) states are placed in a pool.
The pool is then sorted numerically. The highest k states are selected as new initial states. This process
continues until a maximum value is reached.

Variable Neighbourhood Descent

● Variable neighbourhood descent (VND) is a variant of VNS that explores neighbourhoods in a
deterministic way. In general, VND explores small neighborhoods until a local optimum is encountered.
At that point, the search process switches to a different (typically larger) neighborhood that might allow
further progress.

Tabu Search
● Tabu search is a metaheuristic local heuristic search method used for mathematical optimization and to
explore the solution space beyond local optimality. Local search methods have the tendency to be stuck
in suboptimal regions but TS enhances the performance of these techniques by prohibiting already
visited solutions. Its use of adaptive memory creates a more flexible search behavior. The master loop

2
 deals with all the integer variables using TS, and the inner loop minimizes each NLP sub problems using
a gradient-based method.
● TS generates a series of different sets of integer variables which are called candidates. These candidates
differ by one or more bits from the current best solution and are not included in the tabu list. NLP
subproblems are then solved for each candidate using the gradient-based method. Among all the new
candidates, the one with the best objective value is selected and treated as a seed to generate the next
generation of candidates. To prevent being trapped into local optima, a tabu list is built to forbid the
selection of already visited solutions and their neighborhoods.

CODE:
Part A:-
#include<bits/stdc++.h>
#include <random>
using namespace std;

void generateProblem(int k, int m, int n){

string choices[2*n];
string ans="";
int i,j;
for(i=0;i<n;i++){
choices[i]='a'+i;
choices[2*n-i-1] = "~";
choices[2*n-i-1] += 'a'+i;
}
int ind; // v1 in the range 0 to 2n-1
for ( i = 0; i < m; i++){
ans+="(";
for(j=0;j<k;j++){
random_device dev;
mt19937 rng(dev());
uniform_int_distribution<mt19937::result_type> dist(0,2*n-1); // distribution
in range [1, 2n]
ind = dist(rng);
ans+=choices[ind];
if(j!=k-1)
ans+=" | ";
}
ans+=" )";
if(i!=m-1)
ans+=" & ";
}
cout<<ans<<endl;
}
int main(){
int k,m,n,p;

3
 cout<<"\n-----Random k-SAT Problem Generator----\n\n";
cout<<"Enter the length of clause k = ";
cin>>k;
cout<<"Enter the number of clauses m = ";
cin>>m;
cout<<"Enter the number of variables n (max 26) = ";
cin>>n;
cout<<"Enter the number of problems to be generated = ";
cin>>p;

for(int i=0;i<p;i++){
cout<<"\nProblem "<<i+1<<endl;
generateProblem(k,m,n);

OUTPUT:

4
5
CODE:
Part B:-

kSAT.h
#include<bits/stdc++.h>
using namespace std;

#ifndef KSAT_H
#define KSAT_H

class kSAT{
private:
vector<int> createClause();
void printClause(vector<int> a);
public:
int k, n, m;
vector<vector<int> >kSATformula;
6
 kSAT(int k, int n, int m);
void createFormula();
void printkSAT();
vector<int> generateAssignment();
int calScore(vector<int> a);
};
#endif

kSAT.cpp
#include<bits/stdc++.h>
#include"kSAT.h"
using namespace std;

vector<int> kSAT:: createClause(){

vector<int> kSAT;
vector<int> arr;
for (int j = 1; j <= n; ++j)
arr.push_back(j);
random_shuffle(arr.begin(), arr.end());
for(int i=0;i<k;i++){
kSAT.push_back(arr[i]);
float random = (float) rand()/RAND_MAX;
if(random>0.5)
kSAT[i] = -kSAT[i];
}
return kSAT;
}

void kSAT:: printClause(vector<int> a){

cout<<"(";
for(int i=0;i<k-1;i++){
if(a[i]<0)
cout<<"~";
cout<<"a"<<abs(a[i])<<" | ";
}
if(a[k-1]<0)
cout<<"~";
cout<<"a"<<abs(a[k-1])<<")";
}

kSAT::kSAT(int k, int n, int m){

this->k = k;
this->n = n;
this->m = m;
}

7
void kSAT:: createFormula(){
vector<int> a;
for(int i=0;i<m;i++){
a = createClause();
kSATformula.push_back(a);
}
}

void kSAT:: printkSAT(){

cout<<"Randomly generated uniform "<<k<<"-SAT formula: \n";
for(int i=0;i<m;i++){
vector<int>clause = kSATformula[i];
printClause(clause);
if(i!=m-1)
cout<<" & ";
}
cout<<endl;
}

vector<int> kSAT::generateAssignment(){
vector<int> a;
for(int i=0; i<n;i++){
float random = (float) rand()/RAND_MAX;
if(random>0.5)
a.push_back(1);
else
a.push_back(0);
}
return a;
}

int kSAT::calScore(vector<int> a){

int score = 0;
for(int i=0;i<m;i++){
bool flag = false;
for(int j=0;j<k;j++){
int x = kSATformula[i][j];
if(x<0){
if(a[abs(x)-1] == 0){
flag = true;
break;
}
}
if(x>0){
if(a[abs(x)-1] == 1){
flag = true;
8
 break;
}
}
}
if(flag==true)
score++;
}
return score;
}

algorithms.h
#include<bits/stdc++.h>
#include"kSAT.h"
using namespace std;

#ifndef ALGORITHMS_H
#define ALGORITHMS_H

void hillClimbing(kSAT kSATi);

void beamSearch(kSAT kSATi, int BEAM_WIDTH);
void variableNeighborhoodDescent(kSAT kSATi, int neighborhoods = 3);
void tabuSearch(kSAT kSATi, int tabuTenure = 3);

#endif

algorithms.cpp
#include<bits/stdc++.h>
#include"kSAT.h"
#include"algorithms.h"
#include"assignment.h"
using namespace std;

void hillClimbing(kSAT kSATi){

int m = kSATi.m;
vector<int> best = kSATi.generateAssignment();
int bestScore = kSATi.calScore(best);
//cout<<"Assignment: ";
//printVector(best);
cout<<"Score: "<<bestScore<<"\n";
while(bestScore!=m){
pair<vector<int>, int> child = getBestNeighbor(kSATi, best);
int currScore = child.second;
if(currScore>bestScore){
best = child.first;
9
 bestScore = currScore;
//cout<<"Assignment: ";
//printVector(best);
cout<<"Score: "<<bestScore<<"\n";
}
else break;
}
cout<<"\n";
if(bestScore == m){
cout<<"A satifiable assignment for the formula is found: \n";
cout<<"Assignment: ";
printVector(best);
cout<<"Score: "<<bestScore<<"\n";
}
else{
cout<<"Satisfiable assignment not found!\n";
cout<<"Best Assignment found: ";
printVector(best);
cout<<"Score: "<<bestScore<<"\n";
}
}

void beamSearch(kSAT kSATi, int BEAM_WIDTH){

int m = kSATi.m;
vector<int> best = kSATi.generateAssignment();
int bestScore = kSATi.calScore(best);
//cout<<"Assignment: ";
//printVector(best);
cout<<"Score: "<<bestScore<<"\n";
queue<pair<vector<int>, int> > open = bestneighbors(bitChange(best,1), kSATi,
BEAM_WIDTH);
while(bestScore<m){
for(int i=0;i<open.size();i++){
pair<vector<int>, int> peek = open.front();
//cout<<"Score: "<<peek.second<<"\n";
if(peek.second > bestScore){
bestScore = peek.second;
best = peek.first;
cout<<"Score: "<<bestScore<<"\n";
}
open.pop();
pair<vector<int>, int> child = getBestNeighbor(kSATi, peek.first);
if(child.second<peek.second)
continue;
else
open.push(child);
10
 }
if(open.size()==0)
break;
}
cout<<"\n";
if(bestScore == m){
cout<<"A satifiable assignment for the formula is found: \n";
cout<<"Assignment: ";
printVector(best);
cout<<"Score: "<<bestScore<<"\n";
}
else{
cout<<"Satisfiable assignment not found!\n";
cout<<"Best Assignment found: ";
printVector(best);
cout<<"Score: "<<bestScore<<"\n";
}
}

void variableNeighborhoodDescent(kSAT kSATi, int neighborhoods ){

int m = kSATi.m;
int b = 1;
cout<<"Neighborhood Function "<<b<<": \n";
vector<int> best = kSATi.generateAssignment();
int bestScore = kSATi.calScore(best);
cout<<"Score: "<<bestScore<<"\n";
while(bestScore!=m){
if(b>neighborhoods)
break;
pair<vector<int>, int> child = getBestFrom(kSATi, bitChange(best, b));
int currScore = child.second;
if(currScore>bestScore){
best = child.first;
bestScore = currScore;
//cout<<"Assignment: ";
//printVector(best);
cout<<"Score: "<<bestScore<<"\n";
}
else {
b++;
cout<<"Neighborhood Function "<<b<<" \n";
}
}
cout<<"\n";
if(bestScore == m){
cout<<"A satifiable assignment for the formula is found: \n";
11
 cout<<"Assignment: ";
printVector(best);
cout<<"Score: "<<bestScore<<"\n";
}
else{
cout<<"Satisfiable assignment not found!\n";
cout<<"Best Assignment found: ";
printVector(best);
cout<<"Score: "<<bestScore<<"\n";
}
}

void tabuSearch(kSAT kSATi, int tabuTenure){

int m = kSATi.m;
int n = kSATi.n;
vector<int> best = kSATi.generateAssignment();
int bestScore = kSATi.calScore(best);
cout<<"Score: "<<bestScore<<"\n";
int tabuPositions[n] = {0};
while(bestScore!=m){
/*cout<<"Tabu Positions: ";
for(int i=0;i<n;i++)
cout<<tabuPositions[i]<<" ";
cout<<"\n";*/
pair<vector<int>, int> child = getBestNonTabuNeighbor(kSATi, best, tabuPositions,
tabuTenure);
int currScore = child.second;
if(currScore>bestScore){
best = child.first;
bestScore = currScore;
//cout<<"Assignment: ";
//printVector(best);
cout<<"Score: "<<bestScore<<"\n";
for(int i=0;i<n;i++){
if(tabuPositions[i]>0)
tabuPositions[i]--;
}
}
else break;
}
cout<<"\n";
if(bestScore == m){
cout<<"A satifiable assignment for the formula is found: \n";
cout<<"Assignment: ";
printVector(best);
cout<<"Score: "<<bestScore<<"\n";
12
 }
else{
cout<<"Satisfiable assignment not found!\n";
cout<<"Best Assignment found: ";
printVector(best);
cout<<"Score: "<<bestScore<<"\n";
}
}

assignment.h
#include<bits/stdc++.h>
#include"kSAT.h"
using namespace std;

#ifndef ASSIGNMENT_H
#define ASSIGNMENT_H

void printVector(vector<int> a);

queue<pair<vector<int>, int> >bestneighbors(vector<vector<int> >unsorted, kSAT kSATi, int
BEAM_WIDTH);
pair<vector<int>, int> getBestNeighbor(kSAT kSATi, vector<int> a);
vector<vector<int> > bitChange(vector<int> a, int number, int *toChange= NULL);
pair<vector<int>, int> getBestFrom(kSAT kSATi, vector<vector<int> >a);
pair<vector<int>, int> getBestNonTabuNeighbor(kSAT kSATi, vector<int> a,int
tabuPositions[], int tabuTenure);
#endif

assignment.cpp
#include<bits/stdc++.h>
#include"kSAT.h"
#include"assignment.h"
using namespace std;
void printVector(vector<int> a){
for(int i=0;i<a.size();i++)
cout<<a[i]<<" ";
cout<<"\n";
}

bool compare(const pair<vector<int>, int> &a, const pair<vector<int>,int> &b){

return a.second>b.second;
}

queue<pair<vector<int>, int> >bestneighbors(vector<vector<int> >unsorted, kSAT kSATi, int

BEAM_WIDTH){
vector<pair<vector<int>, int> > sorted;
13
 for(int i=0;i<unsorted.size();i++){
sorted.push_back(make_pair(unsorted[i], kSATi.calScore(unsorted[i])));
}
sort(sorted.begin(), sorted.end(), compare);
queue<pair<vector<int>, int> > open;
for(int i=0;i<BEAM_WIDTH;i++)
open.push(sorted[i]);
return open;
}

pair<vector<int>, int> getBestNeighbor(kSAT kSATi, vector<int> a){

vector<int> bestNeighbor ;
int bestScore = -1;
int n = a.size();
for(int i=1;i<n;i++){
vector<int> copy(a);
if(copy[i] == 0)
copy[i] = 1;
else
copy[i] = 0;
int currScore = kSATi.calScore(copy);
if(bestScore < currScore){
bestNeighbor = copy;
bestScore = currScore;
}
}
return make_pair(bestNeighbor, bestScore);
}

vector<vector<int> > bitChange(vector<int> a, int number, int *toChange ){

vector<vector<int> >neighbors;
int n = a.size();
if(toChange == NULL){
int temp[n] = {1};
toChange = temp;
}
for(int i=0;i<n;i++){
if(toChange[i]){
vector<int> copy(a);
if(copy[i] == 0)
copy[i] = 1;
else
copy[i] = 0;
if(number>1){
toChange[i] = 0;
vector<vector<int> >temp = bitChange(copy, number-1, toChange);
14
 neighbors.insert(neighbors.end(), temp.begin(), temp.end());
toChange[i] = 1;
}
else
neighbors.push_back(copy);
}
}
return neighbors;
}

pair<vector<int>, int> getBestFrom(kSAT kSATi, vector<vector<int> >a){

int bestScore = -1e9;
vector<int> best;
for(int i=0;i<a.size();i++){
int score = kSATi.calScore(a[i]);
if(bestScore<score){
bestScore = score;
best = a[i];
}
}
return make_pair(best, bestScore);
}
pair<vector<int>, int> getBestNonTabuNeighbor(kSAT kSATi, vector<int> a,int
tabuPositions[], int tabuTenure){
vector<int> bestNeighbor ;
int bestScore = -1;
int bestIndex = -1;
int n = a.size();
for(int i=1;i<n;i++){
if(tabuPositions[i]<=0){
vector<int> copy(a);
if(copy[i] == 0)
copy[i] = 1;
else
copy[i] = 0;
int currScore = kSATi.calScore(copy);
if(bestScore < currScore){
bestNeighbor = copy;
bestScore = currScore;
bestIndex = i;
}
}
}
tabuPositions[bestIndex] = tabuTenure;
return make_pair(bestNeighbor, bestScore);
}
15
main.cpp
#include<bits/stdc++.h>
#include"kSAT.h"
#include"algorithms.h"
#include"assignment.h"
using namespace std;
int main(){
srand( (unsigned)time( NULL ) );
int k=3, n, m;
cout<<"\n-----Random 3-SAT Problem Solving using Algorithms----\n\n";
cout<<"Enter the number of clauses m = ";
cin>>m;
cout<<"Enter the number of variables n = ";
cin>>n;

kSAT kSATi(k, n, m);

kSATi.createFormula();
kSATi.printkSAT();

cout<<"\n\n--------------------------------\n\nUsing Hill Climbing Algorithm:\n";

hillClimbing(kSATi);

cout<<"\n\n-----------------------------------------------\n\nUsing Beam Search Algorithm

(Beam width = 3):\n";
beamSearch(kSATi, 3);

cout<<"\n------------------------------------------------\n\n\nUsing Beam Search Algorithm

(Beam width = 4):\n";
beamSearch(kSATi, 4);

cout<<"\n-----------------------------------------------\n\n\nUsing Variable Neighborhood

Descent Algorithm:\n";
variableNeighborhoodDescent(kSATi, 2);

cout<<"\n-----------------------------\n\n\nUsing Tabu Search Algorithm:\n";

tabuSearch(kSATi);

return 0;
}

16
OUTPUT:

17
18
19
CONCLUSION:
In this assignment, I implemented a uniform random k-SAT generator. I also understood the algorithms
hill climbing, Variable Neighborhood descent, beam search and tabu search in a better way by learning
to implement them practically to solve the random 3-SAT problem, and also by comparing their
performance on the basis of scores.

20