zee Algorithms and Problem Solving Syllabus Algorithm : The Role of Algorithms in Computing - What are algorithms, Algorithms as technology, Evolution of Algorithms, Design of Algorithm, Need of Correciness of Algorithm, Confirming correctness of Algorithm - sample examples, Iterative algorithm design issues. Problem solving Principles : Classification of problem, problem solving strategies, classification of time complexities (linear, logarithmic etc.) Contents 1.1 The Role of Algorithms in Computing April-18, 19, March-20, ---- Marks 5 1.2 Evolution of Algorithms vce Aprile, : Marks 5 1.3. Design of Algorithm . 6 DOC209, ee Marks 6 1.4 Need of Correctness of Algorithm . .. Dec.-18, seetees ss Marks 7 1.5 Confirming Correctness of Algorithm - Sample Examples April-18, May-19, March-20, . Marks 5 1.6 Iterative Algorithm Design Issues . May-19, March-20, Marks 6 1.7 Problem Solving Principles 1.8 Classification of Problem 1.9 Problem Solving Strategies 1.10 Classification of Time ComplexitiesSL in and Analysis of Algorithms eee eco Algonti and Problem Solving EERE What are Algorithms ? In this section we will first understand “What is algorithm?” and “When it is required?” Definition of Algorithm : The algorithm is defined as a cullection of unambiguous instructions occurring in some specific sequence and such an algorithm should produce output for given set of input in finite amount of time. This definition of algorithm is represented in Fig, 1.1.1. Problem to be solved After understanding the problem | statement we haveto create an Algorithm created for algorithm carefully for the given performing particular problem. The algorithm is then Seaton era converted into some programming : ae language and then given to some Input Computer Output computing device (computer). The computer then tes this Ervors if any algorithm which is actually Hanis UNotion|el elgoriim submitted in the form of source program. During the process of execution it requires certain set of input, With the help of algorithm (in the form of program) and input set, the result is produced as an output. If the given input is invalid then it should raise appropriate error message ; otherwise ict result will be produced as an output, Properties of Algorithm Simply writing the sequence of instructions as an algorithm is not sufficient to accomplish certain task. It is necessary to have following properties associated with an algo: 1. Non-ambiguity : Each step in an algorithm should be non-ambiguous. That means each instruction should be clear and precise. The instruction in an algorithm should not denote any conflicting meaning. This property also indicate the effectiveness of algorithm. 2. Range of input : The range of input should be specified. This is because normally the algorithm is input driven and if the range of the input is not been specified then algorithm can go in an infinite state. — © _ ar knculedESTGH ANG ATAIYSIS OF AGO ST TINT SOTO 3. Multiplicity : The same algorithm can be represented in several different ways. That means we can write in simple English the sequence of instructions or we can write it in the form of pseudo code. Similarly for solving the same problem we can write several different algorithms. For instance : for searching a number from the given list we can use sequential search or a binary search method. Here “searching” is a task and use of either a “sequential search method” or “binary search method” is an algorithm. 4. Speed : The algorithms are written using some specific ideas (which is popularly known as logic of algorithm). But such algorithms should be efficient and should produce the output with fast speed 5, Finiteness : The algorithm should be finite. That means after performing required operations it should terminate. ERE® Algorithms as Technology « Algorithms are used as a tool to utilize reasonable amount of execution time and memory space © Different algorithms can be used to same problem. Each algorithm may také different amount of time. « For example - For sorting the elements using bubble sort algorithm requires more time than that of merge sort or quick sort algorithm. «Execution of algorithm depends upon two factors - 1, Selection of efficient algorithm and 2. Fast hardware upon which the algorithm is to be executed. « For example if there are two algorithms insertion sort(less efficient sorting algorithm) and merge sort(more efficient sorting algorithm) and there are two computers - computer A and computer B. The computer B is more efficient than computer A. If the merge sort is exectited on computer B then this execution is efficient than executing insertion sort on computer B. Thus hardware technology plays an important role in algorithm execution. « Algorithms are core or fundamental factor for following technologies - © Advanced computer architectures © Object oriented systems ing applications that can be wired or wireless at use Graphical User Interfaces(GUI) © Fast networkit © Applications th. © Integrated web technologies TECHNICAL PUBLICATIONS® - an up-thrust for knowledgeDesign and Analysis of Algorithms 1-4 Algorithms and Problem Solving * Ever increasing storage space of computers also affect the processing of algorithm. That means the larger problems, or the problems handling large amount of data can be executed efficiently on increased storage space of the computer. 1. Explain characteristics of good algorithm ? List out the problems solved by algorithm. 2. Write a short note on algorithm as a technology with example. SO 3. What are algorithms ? Explain algorithm as technology with example. Ce ed EEG Evolution of Aigorithms * Introduction to Concept of Algorithm : The first algorithm is developed by famous Indian sanskrit grammarian Panini in the text Maheshwar Sutra. This text consists of compact rules and algorithms for Phonetics and syntax of Sanskrit language. He proposed formal language theory almost parallel to modern theory and gave concepts parallel to modern mathematical functions. * Structured Programming : In 1936 Alan Turing introduced the concept of procedures due to which the approach for systematic coding is developed. This led to concept of structured programming, * Correctness of Algorithm : The concep! of proving the correctness of algorithm is developed. * Analysis of Algorithm : More and more efficient algorithms are then getting developed for solving the problems. Prof. Donald Knuth introduced. the concept of analysis of algorithms * Complexity Theory : Later on the concept of complexity theory is introduced in which the problems were divided into two classes - tractable and non-tractable. The tractable problems is a class of problems which are solvable in reasonable amount of time while non-tractable problems cannot be solved in reasonable amount of time. The concept of lower bound on running time is emerged while solving the problems using complexity theory. Parallel Algorithmic Development : Later on it is found that finding the proof of correctness is very difficult because some proofs can be lengthy or there is no guarantee that the proof will be correct. This led to two developments. i) Automatic proof assistants - Programs which try to generate the detailed proof for @ given algorithm with some assistance from a human analyst, TECHNICAL PUBLICATIONS® - an up-thrust for knowledgeTOMMY ATT TOOT? i and ANAS Ol ii) Parallel development : The idea that an algorithm should be developed in parallel, along with its proof of correctness. + Formal Method of Specification : A new format popularly known as formal methods of specification is introduced to define the proof of correctness. + Time Space-trade off : While searching for effective algorithms researchers found the concept of time-space trade-off. The time space tradeoff is a concept which states that for making algorithm time efficient, the space is utilized more or for making the algorithm space efficient, the space is consumed a lot by the algorithm. * Approximation Algorithm : There are lot of problems which are non-tractable. In such a case, the approximation algorithms are developed to solve such problems. Approximation algorithms are those algorithms which gives the expected result within a time limit which is a small multiple of the theoretical limit. * Randomness : There are some algorithms that make use of random number generations for computing the result. For example genetic algorithms makes use of random number generation. Libido) 1, Write short note om evolution of algorithm LTC | EE¥ Design of Algorithm ‘SPPU: :,Dec.-09, ‘Marks 6 Algorithm is basically a sequence Saale Algorithm Heading of instructions written in simple iC consists of name of algorithm, problem English language. The algorithm is description, input and output broadly divided into two sections as shown in Fig. 1.3.1 Algorithm Body 't consists of logical body of the algorithm writing the algorithm, by making use of various programming Constructs and assignment statement. Let us understand some rules for 1. Algorithm is a procedure consisting of heading and body. The heading consists of keyword Algorithm and name of the algorithm and parameter list. The syntax is, Fig. 1.3.1 Structure of algorithm Algorithm name (p1,p2,, pn) This keyword should Here write Write be written first thename of parameters an algorithm (if any) TECHNICAL PURI ICATIONS® ~ an un-thrust for knowledeeAlgorithms and Problem Solving Then in the heading section we should write following, things //Probiem Description , : //nput //Output Then body of an algorithm is written, in which various programming, constructs like if, for, while or some assignment statements may be written. 4. The compound statements should be enclosed within { and } brackets 5. Single line comments are written using // as beginning of comment, 6. The identifier should begin by letter and not by digit. An identifier can be a combination of alphanumeric string. It is not necessary to write data types explicitly for identifiers. It will be represented by the context itself. Basic data types used are integer, float, char, Boolean and so on. The pointer type is also used to point memory location. The compound data type such as structure or record can also be used. 7. Using assignment operator < an assignment statement can be given For instance : Variable « expression 8. There are other types of operators such as Boolean operators such as true or false. Logical operator: such as and, or, not. And re ational operators such as <, <#, >, >=, =, # 9. The array indices are stored with in square brackets '{' ‘J. The index of array usually start at zero, ‘The multidimensional arrays can also be used in algorithm, 10. The inputting and outputting, can be done using read and write, For example write("This message will be displayed on console"); read(val); LL The conditional statements such as ifthen or if-then-else are written in following form if (condition) then statement if (condition) then statement else statement If the if-then statement is of compound type then ( and } should be used for enclosing, block. 12. while statement can be written as while (condition) do af statement 1 statement 2 statement 1 3° hvust fo TECHNICAL PUBLICATIONS® «an up-thiustWhile the condition is tru le the block se vi statement after } will be executed. ae oxeceted chhenviee 13. The general form for writing for loop is : for variable « value, to value, do { statement 1 statement 2 statement n } Here value, is initialization condition and value, is a terminating condition. The s! ates tep indicates the increments or decrements in value, for executing the for loop. Sometime a keyword step is used to denote increament or decreament the value of variable for example for i to n step 1 «| Here variable i is { incremented by 1 Write (i) at each iteration } 14, The repeat until statement can be written as repeat statement 1 statement 2 statement n until (condition) 15. The break statement is used to exit from inner loop. The return statement is used to return control from one point to another. Generally used while exiting from function: Note that statements in an algorithm executes in sequential order ie. in the same order as they appear-one after the other. ise uses CEEERERD write ar algorithm to count the sum of n manbers, Solution : Algorithm sum (1, n) //problem Desoription: This algorithm is for finding the //sum of given n numbers //input: 1 to n numbers TECHNICAL PUBLICATIONS® - an up-thrust for knowledgegn and Analysis of Algorithms 1-8 Algorithms and Problem Solving Output: The sum of n numbers result — 0 fori ltondoici+l result < result+i return result CEES) wvite an algorithm to check whether given number is even or odd. Solution : Algorithm eventest (val) Problem Description: This algorithm test whether given number is even or odd aput: the number to he tested ie. val Output t: Appropriate messages indicating even or oddness =0) then if (va write (“Given number is even” 6 write(“Given number is odd") GSES Solution : Algorithm sort (a.n} Problem Description: sorting the elements in ascending order sut'An array a in which the elements are stored and n umber of elements in the array Output: The sorted array fori -—itondo for) < i+lton-ido on algorithm for sorting the elements. if(a|i]>aijl) then { temp = ali} 1 ali} al) — temp + } write (“List is sorted") isu Write at Solution Algorithm fact (n) //Problem Description: This algorithm finds the factorial //of given number n /Anput: The number n of which the factorial is to be /calculated. leorithm to find factorial of n number TECHNICAL PUBLICA Tions®. - an _up-thrust for knowledgeTT PODIOAY SONTAG AUGOTINTANE A | /Output:factorial value of given n number if(a — 1) then return 1 else return n*fact(n-1) fexample 1.3.5] Write an algorithm: to perform multiplication of ee matrices Solution : Algorithm Mul(A.B.2) j/Problem Description: This algorithm is for computing ‘multiplication of two matrices /Imput:The two matrices A.B and order of then as n | //Output:The multiplication result will be in matrix c | fori itondo | for} 1tondo } Chi] -—0 | for k 1 tondo | | Chig] — Ch +ALKIBES) | Write tr’ algoriihin to cbmpuke GED of Feo given integers: | Solution + | Algorithm GCD_Euclid(a.b) // Problem Description: This algorithm computes the GCD of | //two numbers @ and b using Euclid's method. /(Input: two integers 4 and b. {/Ourput: GCD value of a and b. while (b + 0)do { c¢ —amod b aeb bec return a Easter sunday is i principle the fi sunday after the full incon “this 7 ffi ecis be called an algorithm 2 Justy nox. Is this rule sufficient Hy precise t be call uinox. Is t fi ations. By lunar age and by Spring answer. Solution : There are two types of day calcul pectin we ai i. | lunar age starts at 1 and increase Wo 29 of 30 and then 3B ee 39 or 30 makes a lunar month Thus every alternate MON © || year, if there are 12 months then. , x 29 + 4 4 (emonths ave 30 day’Design and Analysis of Algorithms 1-10 Algorithms and Problem Solving The s 22 more days in solar year than lunar age. To match both solar and lunar years we must any number of days to lunar calender. After 19 years there will be 209 days extra (As 19 X 11 = 209) in solar year than lunar year. If we perform 209 mod 30 then still we get 29 days exact, A cycle of lunar age is repeated after 30 days, hence by considering this issue, a sequence number called Golden Number is calculated as - GN = Number of years mod 1941. Each year is associated with Epact. sla year is 11 days longer than the lunar age. Hence after 2 years there will be add thos Fpact is nothing but the number of excess days in solar year. This epact is associated with the golden number as follows. Epact = (11 x (GN ~ 1) mod 30. ‘Thus there are some more calculations required to compute the epact of each year. ‘The date of full moon is obtained for each epact and the first sunday following the full moon day of corresponding epact. This sunday is actually an ester sunday. Thus computations are involved in this ester sunday calculations. More over these calculations must be performed in some specific sequence. Hence this computation can be represented by algorithm. {ERM Noed of Correctness of Algorithm 1. To ensure that the algorithm is developed to correctly satisfy the functional requirements, there is a need for correctness of algorithm. 2. Tu ensure that the solution for the given problem is valid we need to check the correctness of algorithm. 3. To check the given problem has finite or infinite solution it is essential to check the correctness of algorithm 4, The need for covrectness of algorithm is for efficient execution of given task on computers. 5. To ensure that all the cases in the problem statements are covered, it is necessary to check the correctness of algorithm. 6. For some application correctness of algorithm means safety of human life or very costly equipment, Hence it is necessary to ensure that the algorithm is correct. Basic steps in algorithmic correctness Following are the basic steps that need to be followed for checking the correctness of algorithm - . 1. Identification of the properties of input data. These properties of data are called preconditions. 2, Identification of the properties which must be satisfied by the output data. These properties are called postconditions. { ! f é § 6 & 6 & & & & &= tOTITIS BIE POTTET SOVTIG | [— Dasign and Anays's of Algonth 3. Proving that starting from the preconditions and executing each step specified in the algorithms one obtains the postconditions. SERETERES Compare between definiteness and effectiveness of an algorithm with example, c ize Solution : Definiteness : 1) The algorithm must always terminate after some fit number of states. 2) Each step of algorithm must be precisely defined. Also, the actions y defined. These actions has to be carried out rigorously and be pre guowsly basic Effectiveness : 1) All operations specified in the algorithm must be sui y that thet can be done exactly as per specification. 2) Each operation in the algorithmic steps must have finite length =O Fiv(0) ®OO a) @©@O) ©}|©OO||®OO||@@ Fin?) Fey Fin) ®©OO}|OOO)}|@OO}|®OO||©@©Oeet Algorithms and Problem Solving 1 1.5 ] Confirming Correctness of Algorithm - Sample Examples One simple method of confirming the correctness of algorithm is making use of assertions. An assertion is a logical statement or expression which is true. For example Example : Consider an algorithm for finding the smallest value among the three numbers. The assertions are denoted in curly brackets Algorithm Find_Min(int aint b,int c) 4 if a — K Case, Worst case . and averago 180 officlanclos teasing > 7 space ) complexity _ T Analysis framework Measuring input size x _—— ” We ~ /~ Moasuting Measuring time running, complexity time Computing order of growth of algorithms Fig. 2.1.1 Analysis framework The efficiency of an algorithm can be decided by measuring the performance of an algorithm. We can measure the performance of an algorithm by computing two factors. 1, Amount of time required by an algorithm to execute. 2. Amount of storage required by an algorithm, This is popularly known as time complexity and space complexity of an algorithm Eee Space Complexity the space complexity can be defined as amount of memory required by an algorithm to run To compute the space complexity we use two factors ; Constant and instance characteristics. The space requirement S(p) can be given as Sip) = C+ Sp where C 1s a constant i.e. fixed part and it denotes the space of inputs and outputs. This space is an amount of space taken by instruction, variables and identifiers. And Sp 18 a space dependent upon instance characteristics. This is a variable part whose space requirement depends on particular problem instance. There are two types of components that contribute to the space complexity - Fixed Part and variable part TECHNICAL PUBLICATIONSThe fixed part includes space for « Instructions * Variables * Array size « Space for constants The variable part includes space for The variables whose size is dependent upon the particular problei m instance being solved. The control statements (such as for, do, while, choice) are used to solve such instances | * Recursion stack for handling recursive call. | | Consider three examples of algorithms to compute the space complexity. | | Example 1: Compute the space required by following algorithm. Algorithm Add (a,b,c) /Problem Description : This algorithm computes the addition /of three elements /Input: a,b, and c are of floating type /Output: The addition is retumed return atb+c The space requirement for algorithm given in Example 1 is S(p) = ¢ © (Sp) = 0 If we assume that a, b and c occupy one word size then total size comes to be 3. | Example 2 : Compute the space needed by the following algorithms justify your answer Algorithm Sum(a.n) s:=0.0; | Fori: = 1tondo I s:=s+talil; 1] return s {| In the given code we require space for s:=0 «© O() Fori:=1ton © O(n) s:= 8 + ali]; O(n) returns ; <— O(1) Hence the space complexity of given algorithm can be denoted in terms of big-oh notation. It is O(n). Example 3 : Compute the space required by following, algorithm, Algorithm Add (x,n) //Problem Description: This is a recursive algorithm which TECHNICAL PUBLICATIONS® - an up-thrust for knowledge//computes addition of all the clements in an array x1] /Mnput: xli] is of floating type, total number of elements //in an array //Output: retums addition of n elements of an array return Add (x,1-1)+xin] The space req ment is - S(p) 2 3(n + 1) The internal stack used for recursion includes space for formal parameters, Jocal Variables and return address. The space required by each call to function Add requires atleast three words (space for n values + space for return address + pointer to x { |). The depth of recursion in n+1 (n times call to function and one return call). The recursion stack space will be = 3(n#1) Era Time Com plexity The time complexity of an algorithm is the amount of computer time required by an algorithm to run to completion. Its difficult to compute the time complexity in terms of physically clocked time For instance in multiuser system, executing time depends on many factors such as - * System load * Number of other programs running * Speed of underlying hardware The Ure complexity is therefore given in terms of frequency count quency count is a count denoting number of times of execution of statement Definition : The freaque, is Definition : The frequency count is @ count that denotes how many times p statement is executed, Fre articular For Example : Consider following, code for counting the frequency count void fun() 0; Printh aa), : } 1 The froqu CY COUNE Of above Provram ig 98 eG os 6 & ~VVvVduvbeeue UU BULLE Dasign and Anaya “Analysis of Algorithms and Complexity Theory (Seu ee Solution : YB Obtain the frequency count for the following code. void fun() int a: a=0, oH A forli=O;1< nit +) a n+1 { a = ati; qa en } printl#d" a); ...... Peer) ) The frequency count of above code is 2n +3. The for loop in above given fragment of code is executed n times when the condition is true and one more time when the condition becomes false. Hence for the for loop the frequency count is n+1. The statement inside the for loop will be executed only when the condition inside the for loop is true. Therefore this statement will be executed for n times. The jast printf statement will be executed for once. CEE, obtain the frequency count for the following code Solution : void fun(int afj{),int bit) int c13)(3); for(i=O;s no. Similarly c is some constant such that c > 0. We can write f(n) gtr) ifn = 2 then, fn) = 202)+2 = 6 atm) = (2)? gin) = 4 : Consider function f(n) = 2n gin). As f(n) ce gin) fin) f(n) € O(gin)) Fig. 2.2.1 Big oh representation 2 and gin) = n*. Then we have to find some + 2 and gin) = n? then we find ¢ for oro nnn @ anil ll ale le me ee LTP PPP ef re rr rpPporeges a> r» 7and Analysis of Aigonthms ‘ fin) > gin) If n = 3 then, f(n) = 23) +2 =8 gin) = (3)? gin) = 9 ie f(n) < g(n) is true. Hence we can conclude that for n > 2, we obtain ffn) < g(n) ‘Thus always upper bound of existing time is obtained by big oh notation. E28 omega Notation Omega notation is denoted by ‘Q. This notation is used to represent the lower bound nortest amount of of algorithm's running time. Using omega notation we can denote time taken by algorithm Definition A function f(n) is said to be in 2 (gin)) if {(n) is bounded below by some positive constant multiple of gin) such that f(r) 2 c¥ gin) For all n 27 It is denoted as f(n) = @ (g(n)). Following graph illustrates the curve for Q notation Example "%q fin) € Q(gin)) Fig. 2.2.2 Omega representation TECHNICAL PUBLICATIONS” - an up-thrust for knowledge 20 ae ne ae If:— Bnd Analysis of Algorithms 2-10 Consider f(n) = 2n? +5 and gn) = In Then if n=0 f(n) = 2(0)2 45 =5 gin) = 7 (0) =0 ie, f(n) > g(n) But if n=l fin) = 2702 45 =7 gin) = 7(1) 7 ie. f(m) = gin) If n = 3 then, fn) = 2(3)245 = 18+5 = 23 gin) = 703) = 2i ie. fn) > gi) Thus for n > 3 we get fin) > ¢ * gin). It can be represented as, Qn? +5 € Q(n) ‘ | | | i Li Theta notation f(n) © Elia 2.2.3 Theta representation : ——————— Analysis of Algorithms and Complexity Theory “0 (a0)Design and Analysis of Algorithms EE] © Notation The theta notation is denoted by @. By this method the running, time is | upper bound and lower bound. of Algorithms and Compleritly Thawy petweer Definition Let f(m) and g(n) be two non negative functions. ‘There are two positive constants namely c, and cp such that, e28(n) S$ f(n) Qin?) Properties of Order of Growth 1. If {,(n) is order of y j(n) and {)(n) is order of g y(n), then FQN) + f(n) © OCmax(y y(n),2 9(n)). N Polynomials of deg, me @(n'"), That means maximum degree is considered from the polynomial, For example ian‘ +agn? +a ne has the order of growth @(n ) 8 OG) © O(log, n) < Om) < O(n?) < O12"). 1. Exponential functions a" have different orders of growth for difter nt values of a. Key Points i) O(g(n)) is a class of functions f(n) that grows less fast than Pay possess the time complexity which is always slower than the that gfn) have that means f(n) time complexities ii) © (g(n)) is a class of functions f(n) that grows at erate as gin). iii) (AC) 18 a class Of functions f(n) that grows faster than or atk rast as fast as y That means {(n) is greater than (y(n). IS x(n).ray State whether the following functions are CORRECT or INCORRECT and justify your answer, 2 Ss2 = Oln) i) 100046 = Ofn) iti) 10n? +4042 = Oln?) Raa ere Le eee Solution : The function f(n) is said to be in O(gin)) ie. f(n) = O(g(n) or f(n) € O(gin)), if it is bounded above by some constant multi In order words if, ffm) S¢- gin) For n> no Consider i) fn) =3n42 gin) =n This assertion can be CORRECT if ¢ = 5 and n 21 ie. f(n) = 3ng +2 = 31) +2 When n = 1 =5 c-g(n) = 5x1 =5 f(n) < c-g(n) is true Ifn = 2 and c =5 then ffm) = 3n+2=32)+2=8 gin) = 20 c- gin) = 5(2) = 10 Again f(n) < g(n) is true Thus we can find that f(n) S g(n) remains true for n21whenc = 5. But n = 0 then f(n) 3n+2=3(0)+2=2 Bin) = Oe: gin) =5- (0) Then f(n) > ¢ - gin). Hence 3n + 2 = O(n) is correct for n 2landce=5 3n + 2 = O(n) is INCORRECT with n = 0 and any constant value. ple of g(n) for all large n. PyDesign anid Analysis of Algorithms 2-14 Analysis of Algorithms and Complexity Theory ii) f(n) = 100. + 6 gin)=n f(n) = O(n) is CORRECT if we consider ¢ = 106 and n 2 1. Case 1: If ng=1 then f(m) = 100n +6 100(1) + 6 = 106 a(n) = 1 c+ pin) = 106- (1) 106 Thus f(n) < c+ gin) is true. Case 2: If ny=2 f(n) = 100n + 6 = 10002) +6 = 206 ain) c-g(n) = 106 (2) = 22 Again fn) < ¢ a(n) Thus if f(n) < c+ g(n) for n 2 1 then 100n + 6 = Ofn) is CORRECT. But for n = 0 fin) = 100n + 6 = 100(0) +6 =6 gin) = 0 c:gin) = ¢-(0)=0 Thus f(n) > egin) Hence 100n + 6 = O(n) is SORRECT for n = 0, TECHNICAL PuBLicaTioNs® an upethrust for knowledgeiii) f(n) = 100n? +4n+2 a(n) = Consider c¢ = 106. The values for n can be Ca: se 1: Ifn=1 then f(n) = 100n? +4n+2 = 100(1)? +4(1) +2 = 100+4+2 = 106 gin) = n? c+ g(n) = 106(1) = 106 f(n) = c- gin) is true. Case 2: Ifn = 2 then f(n) = 100n? +4n+2 = 100(2)? +4(2) +2 = 4004842 = 410 c+ gin) = 106 (2)? = 106(4) = 424. f(n) 1 Hence 100n? +4n +2 = O(n?) is CORRECT for n 21 and c = 106 But forn=0, 100n2+4n42 # o(n?) TECHNICAL PUBLICATIONS® - an up-thrust for knowledgescons sclymantstmendccmbtlcn aniseed 2-16 Anatysis of Algorithms and Complexity Theory CREED) Prove that : fin fir) = O(n") Ag A +.40jn+a9 then Solution : Let, wal) m-2 x m “1 f(n) = agn™ +a, jn™!+ay_on tag If we treate a}, as a constant then the equation (1) becomes where Ais If equation (2) is f(n) = A Ie Alleteds +1) If equation (2) is Kn) = ADnaAlrs2434 n} =A-0(n?) = A(n? If equation (2) is a f(s) = AD = Ao(n’) = ae fn) = Aya=A0(n™) i=O ‘Thus neglecting the constant term we will have, PaesDET STATS OT ANISTON Solution : i) 2n? +0(n) = @(n?) f(n) = © (g(n)) is true when exg(n) $ f(a) $cyg(n) for all nn and where c, and cy are some positive constants. Let, LHS = 2"+0(n) and RHS = @(n?) We can rewrite it as 2° = @(n?)-0(n) 2" = @(n?-n) f(a) = 2" gin) = nt-n Now if we have cy = 1 and c; = 2 then forn > = 2 We get 2" = O(n? . n) to be CORRECT. It is illustrated as follows - n=2,c,=1,c, =2 ¢2-8(n) $ f(n) < c1g(n) ie. 1(n? =n) 52" $2-(n?-n) 1(2? -2)s2! Cm -2) | 2<284 is true | 3,c, 21,0, =2 \ e(n?-n)$2° =grow growth rate of f(r) ate of f(n) in)cagrowth rate of f(n) th rate of fin) "g(n) is @(f(n))" basically means growth rate of g(n)== "g{n) is o(f(n))" basically means growth rate of g(n)< growth t Lichdccaiee amcbontid 1 Define the asymptotic notations )Q io ite a ae Define asymptotic notations. Explain their significance in analyzing algorithm: SPPU': May-12, 14, Marke 5 Define the following i) Big “oh ii) Theta iii) Omega. SPPU ::Dec.-12, Marks 6 xample the notations Big-O and Omega EU Explain all three asymptotic notations with 2 examples of each EUS Re in with 4. Expl Explain Big Oh(O), Omega(Q) and Theta(@) notation in detail along with suitable examples 6. CS 7. Explain asymptotic notations with example. CSC 8. Define asymptotic notation. What is their significance in anlyzing algorithms ? Explain Big Oh, Omega and Theta notations. Rdysis of Algorithms 2 - 20 Anolysis of Algorithins and Complexity Theo’ ee Best Case, Worst Case, Average Case If an algorithm takes minimum amount of time to run to completion for a specific sot of input then it is called best case time complexity lement by using sequential search we eci best case time complexity. n for a specific set For example : While searching a particular el get the desired element at first place itself then it is call If an algorithm takes maximum amount of time to run to completio of iniput then it is called worst case time complexity, For example : While searching an element by using linear searching method if desired element is placed at the end of the list then we get worst time complexity. The time complexity that we get for certain set of inputs is as a average same. ‘Then for corresponding input such a time complexity is called average case time complexity. Consider the following algorithm Algorithm Seq_search(X[0 ... n-1 |, key) // Problem Description: This algorithm is for searching the //key element from an array X[0...n-1] sequentially, /finput: An array X|0...n-1] and search key //Output: Returns the index of X where key value is present for i — Oto n-] do if(Kli] =key)tven return i Best case time complexity Best caw time complexity is a time complexity when an algorithm runs for short tame, In above searching algorithm the element key is searched from the list of n elements. If the Key element is present at first location in the list(X{0...n-1]) then higorithm ron for a very short time and thereby we will get the best case time complexity. We can denote the best case time complexity as Coes: 1 Worst case time complexity Worst case time complexity is a time complexity when algorithm runs for a longest in above searching algorithm the element key is searched from ihe list of n ; St eee ara ae rats, If the key clement is present at n"™ location then clearly the algorithm will run t time and thereby we will get the worst case time complexity. We can denote ame. eleme’ for longest ite worst case ume complexity as Cworst = 7 rithm guarante at for any instance 7 sigorithm guarantees that for any instance of input which is of size n, the wil not exceed Cyoq(lt). Hence the worst case time complexily giv out the efficiency of algorithm. plexily gives runing, time yeortant information alTARGA AAV RTE HT ATH Average case time complexity algorithm on the behaviour of an are required for mation about ome terminologies that This type of complexity gives into; specific or random input Let us understands computing average case time complentty: al search and P be a probability of getting successful Lot the algoritim is tor sequent h. nis the total number of elements in the list 1 The first match of the element will occur at i® location, Hence probability of occurring first match is P/n for, every i" element. ful search is (1 - P) The probability of getting unsucc ‘ase time complexity Cay! Now, we can find average ¢ (n) as Cag) Probability of successful search (for elements t to nin the list) ul search Probability of unsuccess 7 Ply n(-P) may be 8 at which re eleme of ‘not ge chance element’ are possible. Cava 1 for computing average ease BATE complexity. anned the Ye fist then in can obtain the generat! toramul p=0 that me and still we arch ie, we have element i = no successful s d the desired Thus we ans there Suppose if do not foun entire list of elements such a situation ( 2+ 0) Caged = Ot 27nd n | ton. ei) = city becomes ea time comple’ PUN Thus the average 68° — 7 knowied TECHNICAL PUBLICATIONS® - an u-tnrust (OF Amowiedge" ul in Dasign and Analysis of Algorithms 2-22 Analysis of byorttvns ana Complarty thasey Suppose if P= 1 i.e, we get a successful search then Cavgln) © 1 (n+ 1)/2 +n (1 ~1) Covg(n) = (n+ 1)/2 That means the algorithm scans about half of the elements from the list For calculating average case time complexity we have to consider probability of getting success of the operation. And any operation in the algorithm is heavily dependent on input elements, Thus computing average case time complevity is difficult than computing worst case and best case time complexities. GEARED Colcutate the worst case time complexity of fon) = 6n(n* —n)+9n using the running time complexity, Solution : While finding the worst case analysis we calculate upper bound on running time of an algorithm. The upper bound is calculated using big oh notation. Let fin) = 6n(n?-n)+9n ie, fin) = 6n4-6n? +9n We will find f(n) ¢ ¢* gin) For f(n), Let n = 1. 6(1)4 —6(1)? +91) = 6-649 9 For ¢ 20 and g(n) = 9n4 We get f(n) $c * gin). ie. 6n(n?=n)+9ns9n4 The worst case time complexity is O(n4). To sort following set of the data, if we are having quick sort and merge sort, ‘available, which algorithm do you select for the following instance of the data to sort it in. ascending order. Set-l : 1, 2, 3,45, 6,7,8,9,10 Set-2 £10, 9, 8,7, 6,5, 4,3, 21,0. 2 :Design and Analysis of Algonitims 5 next page) ymic function is the slowest and grows rapidly with Sol an increase in number of comparisons. However, if data are in ascending or descending order then merge sort becomes faster. For set 1 and set 2 the merge sort algorithm will be chosen which results in (O(n log: n) performance. For set 3 the quicksort algorithm performance, be chosen which results in O(n log n) 1. What are the basic components that contribute tothe space compleity ? ime of an algrit. 2. Explain the different zaays of measuring the algorithm with one example 13, Explain the best case, worst ease and average case complexity of a EZ] crowth Rate of an algorithm in relation with the input size a is called ‘Measuring the performance order of growth. For example, the order of growth for varying input size of nis as given blow ; ss log n log n n? 2" = 0 o 1 2 2 1 2 4 4 i i : 7 “e = ig, 241 Rate of growth of common computing te function 8 3 a4 64 256 1s ‘ a 2 a Table 2.4.1 Order of growth ee e TECHNIGAL PUBLICATIONS® - an upsEBX Polynomial and Non-polynamial Problems gorithms that take palyn il, binary search O{logn) mulkiplcationDesign and Analysis of Aigonthms 2-26 Analysis of Algonthins and Gomplarty Theery For example - Consider an example of sudo-ku game. In order to solve this entire puzzle, the algorithm would have to check each 3 x 3 matrix to see which nuribers are missing, then each row, then each column and then make sure there are no repetitions of any digit from 0 - 9. This becomes more complex because the number of digits that are missing is inconsistent in each row, column and matrix. Solving, this probler is not possible in polynomial time. But once we get the solution, then verifying, this solution is possible within very less time. Thus verification of solution of this problern is possible ir polynomial time. a | NP complete probierrs are ne problems that NP class Fig. 2.6.2 P, NP, NP complete and NP hard class problems further categorized into NP-complete and NP hard @ The NP class problems can be problems A problem D is called NP-complete if - i) It belongs to class NP. in NP can also be solved in polynomial time tblem can be solved in polynomial time then ‘olved in polynomial time. are NP-hard but all NP-hard proble ui) Every problem 1 a * If an NP-hard pro! problems can also be s = All NP-complete problems NP-complete 6 cannot be P class NP - complete Fig. 2.6.3 Class of com putational complexity _[Design and Analysis of Algorithms ns 2-27 NE nee intersection of NP class problems and NP hard « The NP complete problems are problems. © Normally the NP complete problems are decision problems(For example - is a graph Euler graph or not). blems (For example - shortest path The NP hard problenis are optimization pro problem). ity - The computational problems is an infinite collection of * Computational comple: instances with a solution for every instance. In computational complexity the solution to the problem can be "yes" or "no" type. Such type of problems are called decision problem. The computational problems can be function problem. The function problem is a computational problem where single output is expected for every input. The output of such type of problems is more complex than the decision problem. The computational problems also consists of a class of problems, whose output can be obtained in polynomial time * Complexity classes - The complexity classes is a set of problems of related complexity. It includes function problems, P classes, NP classes, optimization problem. Intractability - Problems that can be solved by taking a long time for their solutions is known as intractable problems. If NP is not same as P then NP complete problems are called intractable problems. © The P class problems are considered as tractable problems. EXXED Example of P Class Problem Kruskal’s algorithm : In Kruskal’s algorithm the minimum weight is obtained. In this algorithm also the circuit should not be formed. Each time the edge of minimum weight has to be selected, from the graph. It is not necessary in this algorithm to have edges of minimum weights to be adjacent. Let us solve one example by Kruskal’s algorithm. Example : Find the minimum spanning tree for the following figure using Kruskal’s algorithm. TECHNICAL PUBLICATIONS® - an up-thrust for knowledge trav: dete that retuiAnalysis of Algorithms and Con molexity Theory Design and Analysis of Algonthms S iy 1 cover all the vertices In Kruskal algorithm, we will start with some vertex and will cover all th with minimum weight. The vertices need not be adjacent. | Example of NP Class Problem Travelling Salesman's Problem (TSP) This problem can be stated as" Given a set of cities and cost to travel between each pair of cities, determine whether there is a path that visits every city once and \ returns to the first city. Such that \ the cost travelled is less” For example : Refer Fig. 2.6.6 The tour path will be a-b-d-e-c-a and total cost of tour will be 16.ana Anaiys’s of Algorithm — igoritins 3 - Z 2-28 ‘Analysis of Algorithms end complexity Theory This problem is NP_ probes eee oe a Problem as there may exist some path with shortest distance ieee the ws vou get the solution by applying certain algorithm then Travelling Svlesman problem is NP Complete Problem. If we get no solution at all by applying an tgorith on the trave algonithoy then the travelling salesman problem belongs to NP hard class. EXSEY itteronco between P and NP Class Problems P class problems NP class problems Aa aigenitin tn which for given input the ‘An algorithm is called non deterministically serine output gets generated is called polynomial time algorithm (NP class) when for nial time algorithm(? class) given input there are more than one paths that the algorithm can follow. Due to which one can not determine which path is to be followed | after particular stage Atak Y class problems are basically All the NP class problems are basically non deterministic © deterministic, | Every problem which is a P class'is also in NP Every problem which is in NP is not the P class _ class problem. NP class problems can not be solved efficiently as efficiently as P class problems problems can be solved off Examples i Knapsack problem, traveling salesperson problem. GREG List tluve problems that have’ polynomial time algorithms, Justify your answer, - Solution: The problems that can be solved in polynomial time are called P - class problems. For example - 1. Binary search - In searching an element using binary search method, the list is amply divided at the mid and either left or right sublist is searched for key element. This process is carried out in O(logn). >. Evaluation of polynomial - In a polynomial evaluation we make out the Bli+1)) then fail();// not sorted elements write(B[1:n});// print the sorted list of elements success(); The time required by a non-deterministic algorithm with some input set is minimum number of steps needed to reach to a successful completion if there are multiple choices from input set leading to successful completion. In short, a non-deterministic algorithm is of complexity O(f(n)) where n is the total size of input : Nondeterministic Algorithm for 0/1 Knapsack Problem ‘The 0/1 Knapsack Decision Problem is to determine, for a given profit P, whether it as possible to load the knapsack so as to keep the total weight no greater than m making the total profit at least equal to Pt. This problem has the same 0-1 Knapsack Optimization Problem plus the additional parameter Pt. while Parameters as the ‘The non deterministic algorithm for 0/1 knapsack problem is as given below Algorithm Non_DKnaps\) /fp{im] is a profit each object i where 1 m) or (Profit operations such as ORG), AND (-) and NOT. There (means implies) and <> (means if and only 11)