adi

1,612 1 6 27
Arec Barrwin
10k 6 16 22
18 Answers
What is a plain English explanation of Big O? With as little formal definition as possible and simple
mathematics.
algorithm complexity-theory computer-science big-o time-complexity
edited May 20 at 19:22 asked Jan 28 '09 at 11:10
11 Summary: The upper bound of the complexity of an algorithm. See also the similar question Big O, how do
you calculate/approximate it? for a good explaination. Kosi2801 Jan 28 '09 at 11:18
1 The other answers are quite good, just one detail to understand it: O(log n) or similar means, that it depends
on the "length" or "size" of the input, not on the value itself. This could be hard to understand, but is very
important. For example, this happens when your algorithm is splitting things in two in each iteration.
Harald Schilly Jan 28 '09 at 11:23
1 Gotta agree, this question isn't the same. cletus Jan 28 '09 at 11:55
17 It'd have be an 8 year old child prodigy to understand both functions/methods and big o successfully
Arec Barrwin Jan 28 '09 at 12:23
6 Big O is an estimate of the worst case performance of a function assuming the algorithm will perform the
maximum number of iterations. Paul Sweatte Aug 28 '12 at 0:16
show 7 more comments
Quick note, this is almost certainly confusing Big O notation (which is an upper bound) with Theta notation
(which is a two-side bound). In my experience this is actually typical of discussions in non-academic settings.
Apologies for any confusion caused.
The simplest definition I can give for Big-O notation is this:
Big-O notation is a relative representation of the complexity of an algorithm.
There are some important and deliberately chosen words in that sentence:
relative: you can only compare apples to apples. You can't compare an algorithm to do arithmetic
multiplication to an algorithm that sorts a list of integers. But two algorithms that do arithmetic
operations (one multiplication, one addition) will tell you something meaningful;
representation: Big-O (in its simplest form) reduces the comparison between algorithms to a single
variable. That variable is chosen based on observations or assumptions. For example, sorting
algorithms are typically compared based on comparison operations (comparing two nodes to determine
their relative ordering). This assumes that comparison is expensive. But what if comparison is cheap but
swapping is expensive? It changes the comparison; and
complexity: if it takes me one second to sort 10,000 elements how long will it take me to sort one
million? Complexity in this instance is a relative measure to something else.
Come back and reread the above when you've read the rest.
The best example of Big-O I can think of is doing arithmetic. Take two numbers (123456 and 789012). The
basic arithmetic operations we learnt in school were:
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no
registration required.
Plain English explanation of Big O
algorithm - Plain English explanation of Big O - Stack Overflow http://stackoverflow.com/questions/487258/plain-english-explanation-of-...
1 of 15 10/16/2013 2:52 PM
addition;
subtraction;
multiplication; and
division.
Each of these is an operation or a problem. A method of solving these is called an algorithm.
Addition is the simplest. You line the numbers up (to the right) and add the digits in a column writing the last
number of that addition in the result. The 'tens' part of that number is carried over to the next column.
Let's assume that the addition of these numbers is the most expensive operation in this algorithm. It stands
to reason that to add these two numbers together we have to add together 6 digits (and possibly carry a 7th).
If we add two 100 digit numbers together we have to do 100 additions. If we add two 10,000 digit numbers
we have to do 10,000 additions.
See the pattern? The complexity (being the number of operations) is directly proportional to the number of
digits n in the larger number. We call this O(n) or linear complexity.
Subtraction is similar (except you may need to borrow instead of carry).
Multiplication is different. You line the numbers up, take the first digit in the bottom number and multiply it in
turn against each digit in the top number and so on through each digit. So to multiply our two 6 digit numbers
we must do 36 multiplications. We may need to do as many as 10 or 11 column adds to get the end result
too.
If we have two 100-digit numbers we need to do 10,000 multiplications and 200 adds. For two one million
digit numbers we need to do one trillion (10 ) multiplications and two million adds.
As the algorithm scales with n-squared, this is O(n ) or quadratic complexity. This is a good time to
introduce another important concept:
We only care about the most significant portion of complexity.
The astute may have realized that we could express the number of operations as: n + 2n. But as you saw
from our example with two numbers of a million digits apiece, the second term (2n) becomes insignificant
(accounting for 0.0002% of the total operations by that stage).
One can notice that we've assumed the worst case scenario here. While multiplying 6 digit numbers if one of
them is 4 digit and the other one is 6 digit, then we only have 24 multiplications. Still we calculate the worst
case scenario for that 'n', i.e when both are 6 digit numbers. Hence Big-O notation is about the Worst-case
scenario of an algorithm
The Telephone Book
The next best example I can think of is the telephone book, normally called the White Pages or similar but it'll
vary from country to country. But I'm talking about the one that lists people by surname and then initials or
first name, possibly address and then telephone numbers.
Now if you were instructing a computer to look up the phone number for "John Smith" in a telephone book
that contains 1,000,000 names, what would you do? Ignoring the fact that you could guess how far in the S's
started (let's assume you can't), what would you do?
A typical implementation might be to open up to the middle, take the 500,000 and compare it to "Smith". If it
happens to be "Smith, John", we just got real lucky. Far more likely is that "John Smith" will be before or after
that name. If it's after we then divide the last half of the phone book in half and repeat. If it's before then we
divide the first half of the phone book in half and repeat. And so on.
This is called a binary search and is used every day in programming whether you realize it or not.
So if you want to find a name in a phone book of a million names you can actually find any name by doing
this at most 20 times. In comparing search algorithms we decide that this comparison is our 'n'.
For a phone book of 3 names it takes 2 comparisons (at most).
For 7 it takes at most 3.
For 15 it takes 4.

For 1,000,000 it takes 20.

That is staggeringly good isn't it?
In Big-O terms this is O(log n) or logarithmic complexity. Now the logarithm in question could be ln (base
e), log , log or some other base. It doesn't matter it's still O(log n) just like O(2n ) and O(100n ) are still
both O(n ).
It's worthwhile at this point to explain that Big O can be used to determine three cases with an algorithm:
Best Case: In the telephone book search, the best case is that we find the name in one comparison.
This is O(1) or constant complexity;
12
2
2
th
10 2
2 2
2
algorithm - Plain English explanation of Big O - Stack Overflow http://stackoverflow.com/questions/487258/plain-english-explanation-of-...
2 of 15 10/16/2013 2:52 PM
community wiki
26 revs, 19 users 74%
cletus
Expected Case: As discussed above this is O(log n); and
Worst Case: This is also O(log n).
Normally we don't care about the best case. We're interested in the expected and worst case. Sometimes
one or the other of these will be more important.
Back to the telephone book.
What if you have a phone number and want to find a name? The police have a reverse phone book but such
lookups are denied to the general public. Or are they? Technically you can reverse lookup a number in an
ordinary phone book. How?
You start at the first name and compare the number. If it's a match, great, if not, you move on to the next. You
have to do it this way because the phone book is unordered (by phone number anyway).
So to find a name:
Best Case: O(1);
Expected Case: O(n) (for 500,000); and
Worst Case: O(n) (for 1,000,000).
The Travelling Salesman
This is quite a famous problem in computer science and deserves a mention. In this problem you have N
towns. Each of those towns is linked to 1 or more other towns by a road of a certain distance. The Travelling
Salesman problem is to find the shortest tour that visits every town.
Sounds simple? Think again.
If you have 3 towns A, B and C with roads between all pairs then you could go:
A B C
A C B
B C A
B A C
C A B
C B A
Well actually there's less than that because some of these are equivalent (A B C and C B A are
equivalent, for example, because they use the same roads, just in reverse).
In actuality there are 3 possibilities.
Take this to 4 towns and you have (iirc) 12 possibilities.
With 5 it's 60.
6 becomes 360.
This is a function of a mathematical operation called a factorial. Basically:
5! = 5 4 3 2 1 = 120
6! = 6 5 4 3 2 1 = 720
7! = 7 6 5 4 3 2 1 = 5040

25! = 25 24 2 1 = 15,511,210,043,330,985,984,000,000

50! = 50 49 2 1 = 3.04140932 10
So the Big-O of the Travelling Salesman problem is O(n!) or factorial or combinatorial complexity.
By the time you get to 200 towns there isn't enough time left in the universe to solve the problem
with traditional computers.
Something to think about.
Polynomial Time
Another point I wanted to make quick mention of is that any algorithm that has a complexity of O(n ) is said
to have polynomial complexity or is solvable in polynomial time.
Traditional computers can solve polynomial-time problems. Certain things are used in the world because of
this. Public Key Cryptography is a prime example. It is computationally hard to find two prime factors of a
very large number. If it wasn't, we couldn't use the public key systems we use.
Anyway, that's it for my (hopefully plain English) explanation of Big O (revised).
edited Jul 24 at 8:00
64
a
algorithm - Plain English explanation of Big O - Stack Overflow http://stackoverflow.com/questions/487258/plain-english-explanation-of-...
3 of 15 10/16/2013 2:52 PM
Ray Hidayat
6,965 3 17 31
210 While the other answers focus on explaining the differences between O(1), O(n^2) et al.... yours is the one
which details how algorithms can get classified into n^2, nlog(n) etc. +1 for a good answer that helped me
understand Big O notation as well Yew Long Jan 28 '09 at 11:42
7 one might want to add that big-O represents an upper bound (given by an algorithm), big-Omega give a
lower bound (usually given as a proof independent from a specific algorithm) and big-Theta means that an
"optimal" algorithm reaching that lower bound is known. mdm Feb 2 '09 at 19:16
9 This is good if you're looking for the longest answer, but not for the answer that best explains Big-O in a
simple manner. kirk.burleson Jul 16 '10 at 18:21
72 -1: This is blatantly wrong: _"BigOh is relative representation of complexity of algorithm". No. BigOh is an
asymptotic upper bound and exists quite well independent of computer science. O(n) is linear. No, you are
confusing BigOh with theta. log n is O(n). 1 is O(n). The number of upvotes to this answer (and the
comments), which makes the basic mistake of confusing Theta with BigOh is quite embarassing...
Aryabhatta May 24 '11 at 4:44
18 "By the time you get to 200 towns there isn't enough time left in the universe to solve the problem with
traditional computers." When the universe is going to end? Isaac Jun 18 '12 at 10:43
show 20 more comments
It shows how an algorithm scales.
O(n^2):
1 item: 1 second
10 items: 100 seconds
100 items: 10000 seconds
Notice that the number of items increases by a factor of 10, but the time increases by a factor of 10^2.
Basically, n=10 and so O(n^2) gives us the scaling factor n^2 which is 10^2.
O(n):
1 item: 1 second
10 items: 10 seconds
100 items: 100 seconds
This time the number of items increases by a factor of 10, and so does the time. n=10 and so O(n)'s scaling
factor is 10.
O(1):
1 item: 1 second
10 items: 1 second
100 items: 1 second
The number of items is still increasing by a factor of 10, but the scaling factor of O(1) is always 1.
That's the gist of it. They reduce the maths down so it might not be exactly n^2 or whatever they say it is, but
that'll be the dominating factor in the scaling.
answered Jan 28 '09 at 11:28
3 what does this definition mean exactly? (The number of items is still increasing by a factor of 10, but the
scaling factor of O(1) is always 1.) HollerTrain Mar 25 '10 at 22:10
15 Not seconds, operations. Also, you missed out on factorial and logarithmic time. Chris Charabaruk Jul 17
'10 at 1:27
41 Because the question said "plain English" a lot of answers assumed they could rant for pages and pages.
Thanks for not doing that. MarcH Aug 9 '11 at 16:19
2 This doesn't explain very well that O(n^2) could be describing an algorithm that runs in precisely .01*n^2 +
999999*n + 999999. It's important to know that algorithms are compared using this scale, and that the
comparison works when n is 'sufficiently large'. Python's timsort actually uses insertion sort (worst/average
case O(n^2)) for small arrays due to the fact that it has a small overhead. Darthfett May 21 '12 at 23:14
algorithm - Plain English explanation of Big O - Stack Overflow http://stackoverflow.com/questions/487258/plain-english-explanation-of-...
4 of 15 10/16/2013 2:52 PM
Jon Skeet
613k 223 3634 5085
This answer also confuses big O notation and Theta notation. The function of n that returns 1 for all its inputs
(usually simply written as 1) is actually in O(n^2) (even though it is also in O(1)). Similarly, an algorithm that
only has to do one step which takes a constant amount of time is also considered to be an O(1) algorithm, but
also to be an O(n) and an O(n^2) algorithm. But maybe mathematicians and computer scientists don't agree
on the definition :-/. Jacob Akkerboom Aug 8 at 11:11
EDIT: Quick note, this is almost certainly confusing Big O notation (which is an upper bound) with Theta
notation (which is both an upper and lower bound). In my experience this is actually typical of discussions in
non-academic settings. Apologies for any confusion caused.
In one sentence: As the size of your job goes up, how much longer does it take to complete it?
Obviously that's only using "size" as the input and "time taken" as the output the same idea applies if you
want to talk about memory usage etc.
Here's an example where we have N T-shirts which we want to dry. We'll assume it's incredibly quick to get
them in the drying position (i.e. the human interaction is negligible). That's not the case in real life, of
course...
Using a washing line outside: assuming you have an infinitely large back yard, washing dries in O(1)
time. However much you have of it, it'll get the same sun and fresh air, so the size doesn't affect the
drying time.
Using a tumble dryer: you put 10 shirts in each load, and then they're done an hour later. (Ignore the
actual numbers here they're irrelevant.) So drying 50 shirts takes about 5 times as long as drying 10
shirts.
Putting everything in an airing cupboard: If we put everything in one big pile and just let general warmth
do it, it will take a long time for the middle shirts to get dry. I wouldn't like to guess at the detail, but I
suspect this is at least O(N^2) as you increase the wash load, the drying time increases faster.
One important aspect of "big O" notation is that it doesn't say which algorithm will be faster for a given size.
Take a hashtable (string key, integer value) vs an array of pairs (string, integer). Is it faster to find a key in the
hashtable or an element in the array, based on a string? (i.e. for the array, "find the first element where the
string part matches the given key.") Hashtables are generally amortised (~= "on average") O(1) once
they're set up, it should take about the same time to find an entry in a 100 entry table as in a 1,000,000 entry
table. Finding an element in an array (based on content rather than index) is linear, i.e. O(N) on average,
you're going to have to look at half the entries.
Does this make a hashtable faster than an array for lookups? Not necessarily. If you've got a very small
collection of entries, an array may well be faster you may be able to check all the strings in the time that it
takes to just calculate the hashcode of the one you're looking at. As the data set grows larger, however, the
hashtable will eventually beat the array.
edited Nov 8 '11 at 6:15 answered Jan 28 '09 at 11:16
4 A hashtable requires an algorithm to run to calculate the index of the actual array ( depending on the
implementation ). And an array just have O(1) because it's just an adress. But this has nothing to do with the
question, just an observation :) Filip Ekberg Jan 28 '09 at 11:29
2 jon's explanation has very much todo with the question i think. it's exactly how one could explain it to some
mum, and she would eventually understand it i think :) i like the clothes example (in particular the last, where it
explains the exponential growth of complexity) Johannes Schaub - litb Jan 28 '09 at 11:32
Oh i don't mean the whole answer, just the hashtable lookup and that it can, actually, Never be as fast as a
direct adressing :) Filip Ekberg Jan 28 '09 at 11:34
Filip: I'm not talking about address an array by index, I'm talking about finding a matching entry in an array.
Could you reread the answer and see if that's still unclear? Jon Skeet Jan 28 '09 at 11:35
Ekberg, oh i'm sorry. you're right you didn't say anything about the answer as a whole. next time i read your
comment twice, mate :) Johannes Schaub - litb Jan 28 '09 at 11:38
show 2 more comments
Big-O notation (also called "asymptotic growth" notation) is what functions "look like" when you ignore
constant factors and stuff near the origin. We use it to talk about how thing scale.
Basics
algorithm - Plain English explanation of Big O - Stack Overflow http://stackoverflow.com/questions/487258/plain-english-explanation-of-...
5 of 15 10/16/2013 2:52 PM
for "sufficiently" large inputs...
f(x) O(upperbound) means f "grows no faster than" upperbound
f(x) (justlikethis) mean f "grows exactly like" justlikethis
f(x) (lowerbound) means f "grows no slower than" lowerbound
big-O notation doesn't care about constant factors: the function 9x is said to "grow exactly like" 10x .
Neither does big-O asymptotic notation care about non-asymptotic stuff ("stuff near the origin" or "what
happens when the problem size is small"): the function 10x is said to "grow exactly like" 10x - x + 2 .
Why would you want to ignore the smaller parts of the equation? Because they become completely dwarfed
by the big parts of the equation as you consider larger and larger scales; their contribution becomes dwarfed
and irrelevant. (See example section.)
Put another way, it's all about the ratio. If you divide the actual time it takes by the O(...) , you will get a
constant factor in the limit of large inputs. Intuitively this makes sense: functions "scale like" one another if
you can multiply one to get the other. That is, when we say...
actualAlgorithmTime(N) O(bound(N))
e.g. "time to mergesort N elements
is O(N log(N))"
... this means that for "large enough" problem sizes N (if we ignore stuff near the origin), there exists
some constant (e.g. 2.5, completely made up) such that:
actualAlgorithmTime(N) e.g. "mergesort_duration(N) "
< constant < 2.5
bound(N) N log(N)
There are many choices of constant; often the "best" choice is known as the "constant factor" of the
algorithm... but we often ignore it like we ignore non-largest terms (see Constant Factors section for why they
don't usually matter). You can also think of the above equation as a bound, saying "In the worst-case
scenario, the time it takes will never be worse than roughly N*log(N) , within a factor of 2.5 (a constant
factor we don't care much about)".
In general, O(...) is the most useful one because we often care about worst-case behavior. If f(x)
represents something "bad" like processor or memory usage, then " f(x) O(upperbound) " means
" upperbound is the worse-case scenario of processor/memory usage".
Intuition
This lets us make statements like...
"For large enough inputsize=N, and a constant
factor of 1, if I double the input size...
... I double the time it takes." ( O(N) )
... I quadruple the time it takes." ( O(N) )
... I add 1 to the time it takes." ( O(log(N)) )
... I don't change the time it takes." ( O(1) )
(with credit to http://stackoverflow.com/a/487292/711085 )
Applications
As a purely mathematical construct, big-O notation is not limited to talking about processing time and
memory. You can use it to discuss the asymptotics of anything where scaling is meaningful, such as:
the number of possibly handshakes among N people at a party ((N), specifically N(N-1)/2, but what
matters is that it "scales like" N)
probabilistic expected number of people who have seen some viral marketing as a function of time
how website latency scales with the number of processing units in a CPU or GPU or computer cluster
how heat output scales on CPU dies as a function of transistor count, voltage, etc.
Example
For the handshake example, #handshakes (N) . The number of handshakes is exactly n-choose-2 or
(N-N)/2 (each of N people shakes the hands of N-1 other people, but this double-counts handshakes so
divide by 2). However, for very large numbers of people, the linear term N is dwarfed and effectively
contributes 0 to the ratio. Therefore the scaling behavior is order N , or the number of handshakes "grows
like N".
#handshakes(N)
algorithm - Plain English explanation of Big O - Stack Overflow http://stackoverflow.com/questions/487258/plain-english-explanation-of-...
6 of 15 10/16/2013 2:52 PM
1/2
N
If you wanted to prove this to yourself, you could perform some simple algebra on the ratio to split it up into
multiple terms ( lim means "considered in the limit of", you can ignore it if it makes you feel better):
N/2 - N/2 (N)/2 N/2 1/2
lim = lim ( - ) = lim = 1/2
N N N N N N 1

this is 0 in the limit of N:
graph it, or plug in a really large number for N
Constant factors
Usually we don't care what the specific constant factors are, because they don't affect the way the function
grows. For example, two algorithm may both take O(N) time to complete, but one may be twice as slow as
the other. We usually don't care too much unless the factor is very large, since optimizing is tricky business (
When is optimisation premature? ); also the mere act of picking an algorithm with a better big-O will often
improve performance by orders of magnitude.
Some asymptotically superior algorithms (e.g. a non-comparison O(N log(log(N))) sort) can have so
large a constant factor (e.g. 100000*N log(log(N)) ), or overhead that is relatively large like O(N
log(log(N))) with a hidden + 100*N , that they are rarely worth using even on "big data".
Why O(N) is sometimes the best you can do, i.e. why we need datastructures
O(N) algorithms are in some sense the "best" algorithms if you need to read all your data. The very act of
reading a bunch of data is an O(N) operation. Loading it into memory is usually O(N) (or faster if you
have hardware support, or no time at all if you've already read the data). However if you touch or even look
at every piece of data (or even every other piece of data), your algorithm will take O(N) time to perform this
looking. Nomatter how long your actual algorithm takes, it will be at least O(N) because it spent that time
looking at all the data.
The same can be said for the very act of writing. For example, all algorithms which print out all
permutations of a number N are O(N!) because the output is at least that long.
This motivates the use of data structures: a data structure requires reading the data only once (usually
O(N) time), plus some arbitrary amount of preprocessing (e.g. O(N) or O(N log(N)) or O(N) ) which
we try to keep small. Thereafter, modifying the data structure (insertions / deletions / etc.) and making
queries on the data take very little time, such as O(1) or O(log(N)) . You then proceed to make a large
number of queries! In general, the more work you're willing to do ahead of time, the less work you'll have to
do later on.
For example, say you had the latitude and longitude coordinates of millions of roads segments, and wanted
to find all street intersections.
Naive method: If you had the coordinates of a street intersection, and wanted to examine nearby
streets, you would have to go through the millions of segments each time, and check each one for
adjacency.
If you only needed to do this once, it would not be a problem to have to do the naive method of O(N)
work only once, but if you want to do it many times (in this case, N times, once for each segment), we'd
have to do O(N) work, or 1000000=1000000000000 operations. Not good (a modern computer can
perform about a billion operations per second).
If we use a simple structure called a hash table (an instant-speed lookup table, also known as a
hashmap or dictionary), we pay a small cost by preprocessing everything in O(N) time. Thereafter, it
only takes constant time on average to look up something by its key (in this case, our key is the latitude
and longitude coordinates, rounded into a grid; we search the adjacent gridspaces of which there are
only 9, which is a constant).
Our task went from an infeasible O(N) to a manageable O(N), and all we had to do was pay a minor
cost to make a hash table.
The moral of the story: a data structure lets us speed up operations. Even more advanced data structures
can let you combine, delay, or even ignore operations in incredibly clever ways, like leaving the equivalent of
"to-do" notes at junctions in a tree.
Amortized / average-case complexity
There is also the concept of "amortized" or "average case". This is no more than using big-O notation for the
expected value of a function, rather than the function itself. For example, some data structures may have a
worse-case complexity of O(N) for a single operation, but guarantee that if you do many of these
operations, the average-case complexity will be O(1) .
algorithm - Plain English explanation of Big O - Stack Overflow http://stackoverflow.com/questions/487258/plain-english-explanation-of-...
7 of 15 10/16/2013 2:52 PM
ninjagecko
21.2k 3 42 55
Multidimensional big-O
Most of the time, people don't realize that there's more than one variable at work. For example, in a string-
search algorithm, your algorithm may take time O([length of text] + [length of query]) , i.e. it is
linear in two variables like O(N+M) . Other more naive algorithms may be O([length of text]*[length
of query]) or O(N*M) . Ignoring multiple variables is one of the most common oversights I see in
algorithm analysis, and can handicap you when designing an algorithm.
The whole story
Keep in mind that big-O is not the whole story. You can drastically speed up some algorithms by using
caching, making them cache-oblivious, avoiding bottlenecks by working with RAM instead of disk, using
parallelization, or doing work ahead of time -- these techniques are often independent of the order-of-growth
"big-O" notation, though you will often see the number of cores in the big-O notation of parallel algorithms.
Also keep in mind that due to hidden constraints of your program, you might not really care about asymptotic
behavior. You may be working with a bounded number of values, for example:
If you're sorting something like 5 elements, you don't want to use the speedy O(N log(N)) quicksort;
you want to use insertion sort, which happens to perform well on small inputs. These situations often
comes up in divide-and-conquer algorithms, where you split up the problem into smaller and smaller
subproblems, such as recursive sorting, fast Fourier transforms, or matrix multiplication.
If some values are effectively bounded due to some hidden fact (e.g. the average human name is softly
bounded at perhaps 40 letters, and human age is softly bounded at around 150). You can also impose
bounds on your input to effectively make terms constant.
In practice, even among algorithms which have the same or similar asymptotic performance, their relative
merit may actually be driven by other things, such as: other performance factors (quicksort and mergesort
are both O(N log(N)) , but quicksort takes advantage of CPU caches); non-performance considerations,
like ease of implementation; whether a library is available, and how reputable and maintained the library is.
Many things can implicitly contribute to the running time's constant factor, such as whether you run your
algorithm on a 500MHz computer vs 2GHz computer, whether your programming language is interpreted or
using a JIT compiler, whether you are doing a constant amount of extra work in a critical section of code, etc.
The effect may be small (e.g. 0.9x speed) or large (e.g. 0.01x speed) compared to a different implementation
and/or environment. Do you switch languages to eek out that little extra constant factor of work? That literally
depends on a hundred other reasons (necessity, skills, coworkers, programmer productivity, the monetary
value of your time, familiarity, workarounds, why not assembly or GPU, etc...), which may be more important
than performance.
The above issues, like programming language, are almost never considered as part of the constant factor
(nor should they be); yet one should be aware of them, because sometimes (though rarely) they may not be
constant. For example in cpython, the native priority queue implementation is asymptotically non-optimal
( O(log(N)) rather than O(1) for your choice of insertion or find-min); do you use another
implementation? Probably not, since the C implementation is probably faster, and there are probably other
similar issues elsewhere. There are tradeoffs; sometimes they matter and sometimes they don't.
Math addenda
For completeness, the precise definition of big-O notation is as follows: f(x) O(g(x)) means that "f is
asymptotically upper-bounded by const*g": ignoring everything below some finite value of x, there exists a
constant such that |f(x)| const * |g(x)| . (The other symbols are as follows: just like O means ,
means . There are lowercase variants: o means <, and means >.) f(x) (g(x)) means both
f(x) O(g(x)) and f(x) (g(x)) (upper- and lower-bounded by g): there exists some constants
such that f will always lie in the "band" between const1*g(x) and const2*g(x) . It is the strongest
asymptotic statement you can make and roughly equivalent to == . (Sorry, I elected to delay the mention of
the absolute-value symbols until now, for clarity's sake; especially because I have never seen negative
values come up in a computer science context.)
People will often use = O(...) . It is technically more correct to use O(...) . means "is an element
of". O(N) is actually an equivalence class, that is, it is a set of things which we consider to be the same. In
this particular case, O(N) contains elements like { 2 N , 3 N , 1/2 N , 2 N + log(N) , - N +
N^1.9 , ...} and is infinitely large, but it's still a set. People will know what you mean if you use = however.
Additionally, it is often the case that in a casual setting, people will say O(...) when they mean (...) ;
this is technically true since the set of things (exactlyThis) is a subset of O(noGreaterThanThis) ...
and it's easier to type. ;-)
edited May 15 '12 at 17:22 answered Jul 8 '11 at 4:46
algorithm - Plain English explanation of Big O - Stack Overflow http://stackoverflow.com/questions/487258/plain-english-explanation-of-...
8 of 15 10/16/2013 2:52 PM
starblue
26.5k 3 38 84
2 -1: I think this answer is much too verbose; the OP clearly just wants the gist of it and not a dump of
somebodys homework. Frerich Raabe Jan 9 at 8:08
10 This is absolutely not "Plain English" in anyway!! vidya sagar Mar 11 at 12:22
5 Is it only me, or is this the new plain english? saji89 Mar 30 at 10:14
1 It doesn't seem to be what OP expected or asked for, but I like it. maxdec May 29 at 11:08
1 An excellent mathematical answer, but the OP asked for a plain English answer. This level of mathematical
description isn't required to understand the answer, though for people particularly mathematically minded it
may be a lot simpler to understand than "plain English". However the OP asked for the latter. El Zorko Jul 2
at 22:19
Big O describes an upper limit on the growth behaviour of a function, for example the runtime of a program,
when inputs become large.
Examples:
O(n): If I double the input size the runtime doubles
O(n ): If the input size doubles the runtime quadruples
O(log n): If the input size doubles the runtime increases by one
O(2 ): If the input size increases by one, the runtime doubles
The input size is usually the space in bits needed to represent the input.
edited Jan 28 '09 at 11:51 answered Jan 28 '09 at 11:23
4 incorrect! for example O(n): If I double the input size the runtime will multiply to finite non zero constant. I mean
O(n) = O(n + n) arena-ru May 16 '10 at 11:33
5 I'm talking about the f in f(n) = O(g(n)), not the g as you seem to understand. starblue Aug 6 '10 at 12:30
I upvoted, but the last sentence doesn't contribute much I feel. We don't often talk about "bits" when discussing
or measuring Big(O). cdiggins Sep 5 '11 at 16:41
You should add an example for O(n log n). Christoffer Hammarstrm Sep 22 '11 at 15:50
That's not so clear, essentially it behaves a little worse than O(n). So if n doubles, the runtime is multiplied by a
factor somewhat larger than 2. starblue Sep 23 '11 at 6:44
Big O notation is most commonly used by programmers as an approximate measure of how long a
computation (algorithm) will take to complete expressed as a function of the size of the input set.
Big O is useful to compare how well two algorithms will scale up as the number of inputs is increased.
More precisely Big O notation is used to express the asymptotic behavior of a function. That means how the
function behaves as it approaches infinity.
In many cases the "O" of an algorithm will fall into one of the following cases:
O(1) - Time to complete is the same regardless of the size of input set. An example is accessing an
array element by index.
O(Log N) - Time to complete increases roughly in line with the log2(n). For example 1024 items takes
roughly twice as long as 32 items, because Log2(1024) = 10 and Log2(32) = 5. An example is finding
an item in a binary search tree (BST).
O(N) - Time to complete that scales linearly with the size of the input set. In other words if you double
the number of items in the input set, the algorithm takes roughly twice as long. An example is counting
the number of items in a linked list.
O(N Log N) - Time to complete increases by the number of items times the result of Log2(N). An
example of this is heap sort and quick sort.
O(N^2) - Time to complete is roughly equal to the square of the number of items. An example of this is
bubble sort.
O(N!) - Time to complete is the factorial of the input set. An example of this is the traveling salesman
problem brute-force solution.
Big O ignores factors that do not contribute in a meaningful way to the growth curve of a function as the input
2
n
algorithm - Plain English explanation of Big O - Stack Overflow http://stackoverflow.com/questions/487258/plain-english-explanation-of-...
9 of 15 10/16/2013 2:52 PM
HodofHod
511 5 18
cdiggins
4,807 22 37
Filip Ekberg
20.1k 11 56 132
size increases towards infinity. This means that constants that are added to or multiplied by the function are
simply ignored.
edited Sep 13 '11 at 3:08 answered Sep 5 '11 at 16:31
Big O is just a way to "Express" yourself in a common way, "How much time / space does it take to run my
code?".
You may often see O(n), O(n^2), O(nlogn) and so forth, all these are just ways to show; How does an
algorithm change?
O(n) means Big O is n, and now you might think, "What is n!?" Well "n" is the amount of elements. Imaging
you want to search for an Item in an Array. You would have to look on Each element and as "Are you the
correct element/item?" in the worst case, the item is at the last index, which means that it took as much time
as there are items in the list, so to be generic, we say "oh hey, n is a fair given amount of values!".
So then you might understand what "n^2" means, but to be even more specific, play with the thought you
have a simple, the simpliest of the sorting algorithms; bubblesort. This algorithm needs to look through the
whole list, for each item.
My list
1 1.
6 2.
3 3.
The flow here would be:
Compare 1 and 6, which is biggest? Ok 6 is in the right position, moving forward!
Compare 6 and 3, oh, 3 is less! Let's move that, Ok the list changed, we need to start from the begining
now!
This is O n^2 because, you need to look at all items in the list there are "n" items. For each item, you look at
all items once more, for comparing, this is also "n", so for every item, you look "n" times meaning n*n = n^2
I hope this is as simple as you want it.
But remember, Big O is just a way to experss yourself in the manner of time and space.
answered Jan 28 '09 at 11:14
Big O describes the fundamental scaling nature of an algorithm.
There is a lot of information that Big O does not tell you about a given algorithm. It cuts to the bone and gives
only information about the scaling nature of an algorithm, specifically how the resource use (think time or
memory) of an algorithm scales in response to the "input size".
Consider the difference between a steam engine and a rocket. They are not merely different varieties of the
same thing (as, say, a Prius engine vs. a Lamborghini engine) but they are dramatically different kinds of
propulsion systems, at their core. A steam engine may be faster than a toy rocket, but no steam piston
engine will be able to achieve the speeds of an orbital launch vehicle. This is because these systems have
different scaling characteristics with regards to the relation of fuel required ("resource usage") to reach a
given speed ("input size").
Why is this so important? Because software deals with problems that may differ in size by factors up to a
trillion. Consider that for a moment. The ratio between the speed necessary to travel to the Moon and human
walking speed is less than 10,000:1, and that is absolutely tiny compared to the range in input sizes software
may face. And because software may face an astronomical range in input sizes there is the potential for the
Big O complexity of an algorithm, it's fundamental scaling nature, to trump any implementation details.
Consider the canonical sorting example. Bubble-sort is O(n^2) while merge-sort is O(n log n). Let's say you
have two sorting applications, application A which uses bubble-sort and application B which uses
merge-sort, and let's say that for input sizes of around 30 elements application A is 1,000x faster than
application B at sorting. If you never have to sort much more than 30 elements then it's obvious that you
should prefer application A, as it is much faster at these input sizes. However, if you find that you may have
to sort ten million items then what you'd expect is that application B actually ends up being thousands of
times faster than application A in this case, entirely due to the way each algorithm scales.
answered Jan 28 '09 at 13:12
algorithm - Plain English explanation of Big O - Stack Overflow http://stackoverflow.com/questions/487258/plain-english-explanation-of-...
10 of 15 10/16/2013 2:52 PM
Wedge
10.8k 4 30 51
Brownie
3,186 3 13 29
John C Earls
190 1 4
Big O is a measure of how much time/space an algorithm uses relative to the size of its input.
If an algorithm is O(n) then the time/space will increase at the same rate as its input.
If an algorithm is O(n^2) then the time/space increase at the rate of its input squared.
and so on.
answered Jan 28 '09 at 11:19
1 It's not about space. It's about complexity which means time. S.Lott Jan 28 '09 at 11:35
11 I have always believed it can be about time OR space. but not about both at the same time. Rocco Jan 28
'09 at 12:58
7 Complexity most definitely can be about space. Have a look at this: en.wikipedia.org/wiki/PSPACE pelotom
Aug 8 '10 at 15:58
This answer is the most "plain" one here. Previous ones actually assume readers know enough to understand
them but writers are not aware of it. They think theirs are simple and plain, which are absolutely not. Writing a
lot text with pretty format and making fancy artificial examples that are hard to non-CS people is not plain and
simple, it is just attractive to stackoverflowers who are mostly CS people to up vote. Explaining CS term in
plain English needs nothing about code and math at all. +1 for this answer though it is still not good enough.
W.Sun May 29 at 12:36
Big O notation is a way of describing the upper bound of an algorithm in terms of space or running time. The
n is the number of elements in the the problem (i.e size of an array, number of nodes in a tree, etc.) We are
interested in describing the running time as n gets big.
When we say some algorithm is O(f(n)) we are saying that the running time (or space required) by that
algorithm is always lower than some constant times f(n).
To say that binary search has a running time of O(logn) is to say that there exists some constant c which you
can multiply log(n) by that will always be larger than the running time of binary search. In this case you will
always have some constant factor of log(n) comparisons.
In other words where g(n) is the running time of your algorithm, we say that g(n) = O(f(n)) when g(n) <= c*f(n)
when n > k, where c and k are some constants.
answered Jul 17 '10 at 2:29
We can use BigO notation to measure the worst case and average case as well. en.wikipedia.org/wiki
/Big_O_notation cdiggins Sep 5 '11 at 16:36
Ok, my 2cents.
Big-O, is rate of increase of resource consumed by program, w.r.t. problem-instance-size
Resource : Could be total-CPU time, could be maximum RAM space. By default refers to CPU time.
Say the problem is "Find the sum",
int Sum(int*arr,int size){
int sum=0;
while(size-->0)
sum+=arr[size];
return sum;
}
problem-instance= {5,10,15} ==> problem-instance-size = 3, iterations-in-loop= 3
algorithm - Plain English explanation of Big O - Stack Overflow http://stackoverflow.com/questions/487258/plain-english-explanation-of-...
11 of 15 10/16/2013 2:52 PM
Shashi Bhushan
2,401 3 16 49
Ajeet
1,240 8 21
James Oravec
1,344 5 24
problem-instance= {5,10,15,20,25} ==> problem-instance-size = 5 iterations-in-loop = 5
For input of size "n" the program is growing at speed of "n" iterations in array. Hence Big-O is N expressed
as O(n)
Say the problem is "Find the Combination",
void Combination(int*arr,int size)
{ int outer=size,inner=size;
while(outer -->0) {
inner=size;
while(inner -->0)
cout<<arr[outer]<<"-"<<arr[inner]<<endl;
}
}
problem-instance= {5,10,15} ==> problem-instance-size = 3, total-iterations = 3*3 = 9
problem-instance= {5,10,15,20,25} ==> problem-instance-size = 5, total-iterations= 5*5 =25
For input of size "n" the program is growing at speed of "n*n" iterations in array. Hence Big-O is N^2
expressed as O(n^2)
edited Jan 3 at 10:11 answered Aug 23 '11 at 4:06
1 while (size-->0) I hope this wouldn't ask again. mr5 Jun 18 at 14:41
What is a plain English explanation of Big O? With as little formal definition as possible and simple
mathematics.
A Plain English Explanation of the Need for Big-O Notation:
When we program, we are trying to solve a problem. What we code is called an algorithm. Big O notation
allows us to compare the worse case performance of our algorithms in a standardized way. Hardware specs
vary over time and improvements in hardware can reduce the time it takes an algorithms to run. But
replacing the hardware does not mean our algorithm is any better or improved over time, as our algorithm is
still the same. So in order to allow us to compare different algorithms, to determine if one is better or not, we
use Big O notation.
A Plain English Explanation of What Big O Notation is:
Not all algorithms run in the same amount of time, and can vary based on the number of items in the input,
which we'll call n. Based on this, we consider the worse case analysis, or an upper-bound of the run-time as
n get larger and larger. We must be aware of what n is, because many of the Big O notations reference it.
edited Oct 7 at 14:02 answered Feb 22 at 1:00
It is very difficult to measure the speed of software programs, and when we try, the answers can be very
complex and filled with exceptions and special cases. This is a big problem, because all those exceptions
and special cases are distracting and unhelpful when we want to compare two different programs with one
another to find out which is "fastest".
As a result of all this unhelpful complexity, people try to describe the speed of software programs using the
smallest and least complex (mathematical) expressions possible. These expressions are very very crude
approximations: Although, with a bit of luck, they will capture the "essence" of whether a piece of software is
fast or slow.
Because they are approximations, we use the letter "O" (Big Oh) in the expression, as a convention to signal
to the reader that we are making a gross oversimplification. (And to make sure that nobody mistakenly thinks
that the expression is in any way accurate).
If you read the "Oh" as meaning "on the order of" or "approximately" you will not go too far wrong. (I think the
choice of the Big-Oh might have been an attempt at humour).
The only thing that these "Big-Oh" expressions try to do is to describe how much the software slows down as
we increase the amount of data that the software has to process. If we double the amount of data that needs
algorithm - Plain English explanation of Big O - Stack Overflow http://stackoverflow.com/questions/487258/plain-english-explanation-of-...
12 of 15 10/16/2013 2:52 PM
William Payne
345 2 9
Glitch
2,252 2 10 30
Priidu Neemre
343 4 8
to be processed, does the software need twice as long to finish it's work? Ten times as long? In practice,
there are a very limited number of big-Oh expressions that you will encounter and need to worry about:
The good:
O(1) Constant: The program takes the same time to run no matter how big the input is.
O(log n) Logarithmic: The program run-time increases only slowly, even with big increases in the size
of the input.
The bad:
O(n) Linear: The program run-time increases proportionally to the size of the input.
O(n^k) Polynomial: - Processing time grows faster and faster - as a polynomial function - as the size
of the input increases.
... and the ugly:
O(k^n) Exponential The program run-time increases very quickly with even moderate increases in the
size of the problem - it is only practical to process small data sets with exponential algorithms.
O(n!) Factorial The program run-time will be longer than you can afford to wait for anything but the
very smallest and most trivial-seeming datasets.
answered May 29 at 13:51
I've also heard the term Linearithmic - O(n log n) which would be considered good. Jason Down May 29 at
18:45
There are others - but it stops being "plain english" very quickly. William Payne Jun 20 at 15:06
Not sure I'm further contributing to the subject but still thought I'd share: I once found this blog post to have
some quite helpful (though very basic) explanations & examples on Big O:
Via examples, this helped get the bare basics into my tortoiseshell-like skull, so I think it's a pretty descent
10-minute read to get you headed in the right direction.
edited Jan 15 at 20:23 answered Sep 29 '12 at 20:54
@William ...and people tend to die of old age, species go extinct, planets turn barren etc. Priidu Neemre May
30 at 5:52
Big O
f(x) = O(g(x)) when x goes to a (for example, a = +) means that there is a function k such that:
f(x) = k(x)g(x) 1.
k is bounded in some neighborhood of a (if a = +, this means that there are numbers N and M such
that for every x > N, |k(x)| < M).
2.
In other words, in plain English: f(x) = O(g(x)), x a, means that in a neighborhood of a, f decomposes into
the product of g and some bounded function.
Small o
By the way, here is for comparison the definition of small o.
f(x) = o(g(x)) when x goes to a means that there is a function k such that:
f(x) = k(x)g(x) 1.
k(x) goes to 0 when x goes to a. 2.
Examples
sin x = O(x) when x 0.
sin x = O(1) when x +,
x^2 + x = O(x) when x 0,
algorithm - Plain English explanation of Big O - Stack Overflow http://stackoverflow.com/questions/487258/plain-english-explanation-of-...
13 of 15 10/16/2013 2:52 PM
Alexey
369 1 2 11
Khaled A Khunaifer
1,345 4 14
x^2 + x = O(x^2) when x +,
ln(x) = o(x) = O(x) when x +.
Attention! The notation with the equal sign "=" uses a "fake equality": it is true that o(g(x)) = O(g(x)), but
false that O(g(x)) = o(g(x)). Similarly, it is ok to write "ln(x) = o(x) when x +", but the formula "o(x) = ln(x)"
would make no sense.
edited Mar 17 at 9:23 answered Mar 15 at 21:18
Algorithm example (Java):
public boolean simple_search (ArrayList<Integer> list, int key)
{
for (Integer i : list)
{
if (i == key)
{
return true;
}
}
return false;
}
Algorithm description:
This algorithm search a list, item by item, looking for a key,
Iterating on each item in the list, if it's the key then return True,
If the loop has finished without finding the key, return False.
Big-O notation represent the upper-bound on the Complexity (Time, Space, ..)
To find The Big-O on Time Complexity:
Calculate how much time (regarding input size) the worst case takes:
Worst-Case: the key doesn't exist in the list.
Time(Worst-Case) = 4n+1
Time: O(4n+1) = O(n) | in Big-O, constants are neglected
O(n) ~ Linear
There's also Big-Omega, which represent complexity of the Best-Case:
Best-Case: the key is the first item.
Time(Best-Case) = 4
Time: (4) = O(1) ~ Instant\Constant
answered Mar 23 at 15:19
"What is a plain English explanation of Big O? With as little formal definition as possible and simple
mathematics."
Such a beautifully simple and short question seems at least to deserve an equally short answer, like a
student might receive during tutoring.
Big O notation simply tells how much time* an algorithm can run within, in terms of only the amount of
input data**.
( *in a wonderful, unit-free sense of time!)
algorithm - Plain English explanation of Big O - Stack Overflow http://stackoverflow.com/questions/487258/plain-english-explanation-of-...
14 of 15 10/16/2013 2:52 PM
Joseph Myers
2,128 2 13
Arjun
21 3
(**which is what matters, because people will always want more, whether they live today or tomorrow)
Well, what's so wonderful about Big O notation if that's what it does?
Practically speaking, Big O analysis is so useful and important because Big O puts the focus squarely
on the algorithm's own complexity and completely ignores anything that is merely a proportionality
constantlike a JavaScript engine, the speed of a CPU, your Internet connection, and all those things
which become quickly become as laughably outdated as a Model T. Big O focuses on performance only
in the way that matters equally as much to people living in the present or in the future.
Big O notation also shines a spotlight directly on the most important principle of computer
programming/engineering, the fact which inspires all good programmers to keep thinking and dreaming:
the only way to achieve results beyond the slow forward march of technology is to invent a better
algorithm.
edited Aug 24 at 6:50 answered Aug 15 at 1:57
1 Being asked to explain something mathematical without mathematics is always a personal challenge to me, as
a bona fide Ph.D. mathematician and teacher who believes that such a thing is actually possible. And being a
programmer as well, I hope that no one minds that I found answering this particular question, without
mathematics, to be a challenge that was completely irresistible. Joseph Myers Aug 15 at 2:09
Big O notation seeks to describe the relative complexity of an algorithm by reducing the growth rate to the
key factors when the key factor tends towards infinity. For this reason, you will often hear the phrase
asymptotic complexity. In doing so, all other factors are ignored. It is a relative representation of complexity.
answered Oct 8 at 7:31
protected by Community May 27 '11 at 22:05
This question is protected to prevent "thanks!", "me too!", or spam answers by new users. To answer it, you must
have earned at least 10 reputation on this site.
Not the answer you're looking for? Browse other questions tagged algorithm
complexity-theory computer-science big-o time-complexity or ask your own question.
algorithm - Plain English explanation of Big O - Stack Overflow http://stackoverflow.com/questions/487258/plain-english-explanation-of-...
15 of 15 10/16/2013 2:52 PM