What is Big O notation? Do you use it?

One important thing most people forget

You cannot use Big-O to compare the speed of two algorithms. Big-O only says how much
slower an algorithm will get (approximately), if you double the number of items processed or
how much faster it will get, if you cut the number in half.
However, if you have to entirely different algorithms and one (A) is O(n^2) and the other one
(B) is O(log n), it is not said that A is slower than B. Actually with 100 items, A might be ten
times faster than B. It only says that with 200 items, A will grow slower by the factor n^2 and
B will grow slower by the factor log n. So if you benchmark both and you know how much
time A takes to process 100 item and how much time B needs for the same 100 items, and
A is faster than B, you can calculate at how many items B will overtake A in speed (as the
speed of B decreases much slower than the one of A, it will overtake A sooner or later, this
is for sure).

'Big-O' notation is used to compare the growth rates of two functions of a variable (say n)
as n gets very large. If function f grows much more quickly than function g we say that g =
O(f) to imply that for large enough n, f will always be larger than g up to a scaling factor.
It turns out that this is a very useful idea in computer science and particularly in the analysis
of algorithms, because we are often precisely concerned with the growth rates of functions
which represent, for example, the time taken by two different algorithms. Very coarsely, we
can determine that an algorithm with run-time t1(n) is more efficient than an algorithm with
run-time t2(n) if t1 = O(t2) for large enough n which is typically the 'size' of the problem - like
the length of the array or number of nodes in the graph or whatever.

This stipulation, that n gets large enough, allows us to pull a lot of useful tricks. Perhaps the
most often used one is that you can simplify functions down to their fastest growing terms.
For example n^2 + n = O(n^2) because as n gets large enough, the n^2 term gets so much
larger than n that the n term is practically insignificant. So we can drop it from consideration.
However, it does mean that big-O notation is less useful for small n, because the slower
growing terms that we've forgotten about are still significant enough to affect the run-time.

What we now have is a tool for comparing the costs of two different algorithms, and a
shorthand for saying that one is quicker or slower than the other. Big-O notation can be
abused which is a shame as it is imprecise enough already! There are equivalent terms for
saying that a function grows less quickly than another, and that two functions grow at the
same rate.

Oh, and do I use it? Yes, all the time - when I'm figuring out how efficient my code is it gives
a great 'back-of-the-envelope- approximation to the cost.

Every programmer should be aware of what Big O notation is, how it applies for actions with
common data structures and algorithms (and thus pick the correct DS and algorithm for the
problem they are solving), and how to calculate it for their own algorithms.
1) It's an order of measurement of the efficiency of an algorithm when working on a data
structure.

2) Actions like 'add' / 'sort' / 'remove' can take different amounts of time with different data
structures (and algorithms), for example 'add' and 'find' are O(1) for a hashmap, but O(log
n) for a binary tree. Sort is O(nlog n) for QuickSort, but O(n^2) for BubbleSort, when dealing
with a plain array.

3) Calculations can be done by looking at the loop depth of your algorithm generally. No
loops, O(1), loops iterating over all the set (even if they break out at some point) O(n). If the
loop halves the search space on each iteration? O(log n). Take the highest O() for a
sequence of loops, and multiply the O() when you nest loops

It may also be worth considering that the complexity of many algorithms is based on more
than one variable, particularly in multi-dimensional problems. For example, I recently had to
write an algorithm for the following. Given a set of n points, and m polygons, extract all the
points that lie in any of the polygons. The complexity is based around two known variables,
n and m, and the unknown of how many points are in each polygon. The big O notation
here is quite a bit more involved than O(f(n)) or even O(f(n) + g(m)). Big O is good when
you are dealing with large numbers of homogenous items, but don't expect this to always be
the case.

It is also worth noting that the actual number of iterations over the data is often dependent
on the data. Quicksort is usually quick, but give it presorted data and it slows down. My
points and polygons alogorithm ended up quite fast, close to O(n + (m log(m)), based on
prior knowledge of how the data was likely to be organised and the relative sizes of n and
m. It would fall down badly on randomly organised data of different relative sizes.

A final thing to consider is that there is often a direct trade off between the speed of an
algorithm and the amount of space it uses. Pigeon hole sorting is a pretty good example of
this. Going back to my points and polygons, lets say that all my polygons were simple and
quick to draw, and I could draw them filled on screen, say in blue, in a fixed amount of time
each. So if I draw my m polygons on a black screen it would take O(m) time. To check if any
of my n points was in a polygon, I simply check whether the pixel at that point is green or
black. So the check is O(n), and the total analysis is O(m + n). Downside of course is that I
need near infinite storage if I'm dealing with real world coordinates to millimeter
accuracy.... ...ho hum.
t may also be worth considering amortized time, rather than just worst case. This means, for
example, that if you run the algorithm n times, it will be O(1) on average, but it might be
worse sometimes.
A good example is a dynamic table, which is basically an array that expands as you add
elements to it. A naïve implementation would increase the array's size by 1 for each
element added, meaning that all the elements need to be copied every time a new one is
added. This would result in a O(n2) algorithm if you were concatenating a series of arrays
using this method. An alternative is to double the capacity of the array every time you need
more storage. Even though appending is an O(n) operation sometimes, you will only need
to copy O(n) elements for every n elements added, so the operation is O(1) on average.
This is how things like StringBuilder or std::vector are implemented.
What is Big O notation?
Big O notation is a method of expressing the relationship between many steps an algorithm
will require related to the size of the input data. This is referred to as the algorithmic
complexity. For example sorting a list of size N using Bubble Sort takes O(N^2) steps.

Do I use Big O notation?

Concrete Examples?
I have used the theory of complexity analysis to create algorithms for efficient stack data
structures which require no memory reallocation, and which support average time of O(N)
for indexing. I have used Big O notation to explain the algorithm to other people. I have also
used complexity analysis to understand when linear time sorting O(N) is possible

The "Intuitition" behind Big-O

Now, if from some point on (some x) one function always has a higher value then the other,
then let's call this function "faster" than the other.

So, for example, if for every x > 100 you see that f(x) > g(x), then f(x) is "faster" than g(x).

In this case we would say g(x) = O(f(x)). f(x) poses a sort of "speed limit" of sorts for g(x),
since eventually it passes it and leaves it behind for good.

This isn't exactly the definition of big-O notation, which also states that f(x) only has to be
larger than C*g(x) for some constant C (which is just another way of saying that you can't
help g(x) win the competition by multiplying it by a constant factor - f(x) will always win in the
end). The formal definition also uses absolute values. But I hope I managed to make it
intuitive.
 have seen this term "O(1) access time" used to mean "quickly" but I don't
understand what it means. The other term that I see with it in the same context is
"O(n) access time". Could someone please explain in a simple way what these terms
mean?
n essence, It means that it takes the same amount of time to look up a value in your
collection whether you have a small number of items in your collection or very very many
(within the constraints of your hardware)

O(n) would mean that the time it takes to look up an item is proportional to the number of
items in the collection.

Typical examples of these are arrays, which can be accessed directly, regardless of their
size, and linked lists, which must be traversed in order from the beginning to access a given
item.

The other operation usually discussed is insert. A collection can be O(1) for access but O(n)
for insert. In fact an array has exactly this behavior, because to insert an item in the middle,
You would have to move each item to the right by copying it into the following slot.

t's called the Big O notation, and describes the search time for various algorithms.
O(1) means that the search time is constant. For most situation it means that you dont
acctually need to search the collection, you cand find what you are searching for right away

O(1) does not necessarily mean "quickly". It means that the time it takes is constant,
and not based on the size of the input to the function. Constant could be fast or slow. O(n)
means that the time the function takes will change in direct proportion to the size of the
input to the function, denoted by n. Again, it could be fast or slow, but it will get slower as
the size of n increases.

Big O notation" is a way to express the speed of algorithms. n is the amount of data the
algorithm is working with. O(1) means that, no matter how much data, it will execute in
constant time. O(n) means that it is proportional to the amount of data.

asically, O(1) means its computation time is constant, while O(n) means it will
depend lineally on the size of input - i.e. looping through an array has O(n) - just looping -,
because it depends on the number of items, while calculating the maximum between to
ordinary numbers has O(1).

Big O notation as applied to an algorithm refers to how the run time of the algorithm
depends on the amount of input data. For example, a sorting algorithm will take longer to
sort a large data set than a small data set. If for the sorting algorithm example you graph the
run time (vertical-axis) vs the number of values to sort (horizontal-axis), for numbers of
values from zero to a large number, the nature of the line or curve that results will depend
on the sorting algorithm used. Big O notation is a shorthand method for describing the line
or curve.

In big O notation, the expression in the brackets is the function that is graphed. If a variable
(say n) is included in the expression, this variable refers to the size of the input data set.
You say O(1) is the best. This is true because the graph f(n) = 1 does not vary with n. An
O(1) algorithm takes the same amount of time to complete regardless of the size of the
input data set. By contrast, the run time of an algorithm of O(n^n) increases with the square
of the size of the input data set.

That is the basic idea, for a detailed explanation, consult the wikipedia page titled 'Big O
Notation'.

Big O refers to the worst case run-time order. It is used to show how well an algorithm
scales based on the size of the data set (n->number of items).

Since we are only concerned with the order, constant multipliers are ignored, and any terms
which increase less quickly than the dominant term are also removed. Some examples:

A single operation or set of operations is O(1), since it takes some constant time (does not
vary based on data set size).

A loop is O(n). Each element in the data set is looped over.

A nested loop is O(n^2). A nested nested loop is O(n^3), and onward.

Things like binary tree searching are log(n), which is more difficult to show, but at every
level in the tree, the possible number of solutions is halved, so the number of levels is log(n)
(provided the tree is balanced).

Something like finding the sum of a set of numbers that is closest to a given value is O(n!),
since the sum of each subset needs to be calculated. This is very bad.

Specifically O(n) means that if there's 2x as many items in the list, it'll takes No more
than twice as long, if there's 50 times as many it'll take No more than 50 times as long.
See the wikipedia article dreeves pointed out for more details
Edit (in bold above): It was pointed out that Big-O does represent the upper bound, so if
there's twice as many elements in the list, insertion will take at most twice as long, and if
there's 50 times as many elements, it would take at most 50 times as long.
If it was additionally Ω(n) (Big Omega of n) then it would take at least twice as long for a list
that is twice as big. If your implementation is both O(n) and Ω(n), meaning that it'll take both
at least and at most twice as long for a list twice as big, then it can be said to be Θ(n) (Big
Theta of n), meaning that it'll take exactly twice as long if there are twice as many elements.
According to Wikipedia (and personal experience, being guilty of it myself) Big-O is often
used where Big-Theta is what is meant. It would be technically correct to call your function
O(n^n^n^n) because all Big-O says is that your function is no slower than that, but no one
would actually say that other than to prove a point because it's not very useful and
misleading information, despite it being technically 
(n) is Big O Notation and refers to the complexity of a given algorithm. n refers to the size of
the input, in your case it's the number of items in your list.
O(n) means that your algorithm will take on the order of n operations to insert an item. e.g.
looping through the list once (or a constant number of times such as twice or only looping
through half).
O(1) means it takes a constant time, that it is not dependent on how many items are in the
list.
O(n^2) means that for every insert, it takes n*n operations. i.e. 1 operation for 1 item, 4
operations for 2 items, 9 operations for 3 items. As you can see, O(n^2) algorithms become
inefficient for handling large number of items.
For lists O(n) is not bad for insertion, but not the quickest. Also note that O(n/2) is
considered as being the same as O(n) because they both grow at the same rate with n.