You Are Dangerously Bad at Cryptography: Najaf Ali
You Are Dangerously Bad at Cryptography: Najaf Ali
Curator
Lim Cheng Soon
Contributors
Ram Rachum
Najaf Ali
Craig Gidney
Sacha Chua
Kyle MacDonald
Jeff Escalante
Jason Cohen
David Cain
Shaanan Cohney
Proofreaders
Emily Griffin
Sigmarie Soto
Advertising
Published by
Netizens Media
46, Taylor Road,
11600 Penang,
Malaysia.
Contact
[email protected]
Illustrator
Stefan Hartmann
Ebook Conversion
Ashish Kumar Jha
Printer
MagCloud
Hacker Monthly is published by Netizens Media and not affiliated with Y Combinator in any way.
Contents
FEATURES
04 Indescribable Numbers
By Ram
Rachum
PROGRAMMING
14
By Craig Gidney
20
By Sacha Chua
22
STARTUPS
26
By Jason Cohen
SPECIAL
30
By David Cain
36
The Attack
By Shaanan Cohney
FEATURES
Indescribable Numbers
The theorem that made me fall in love with math
By Ram Rachum
Koan:
A monk asked Zen master
Nansen, Is there any teaching no
master has ever taught before?
Nansen replied, Yes, there is.
What is it? asked the monk.
Nansen answered, It is not
mind, it is not Buddha, it is not
things.
Mumons comments about the
koan:
Being asked a question, old
Nansen gave away his treasure
words. He must have been greatly
upset.
Mumons poem about the koan:
Nansen was too kind and lost his
treasure,
Clearly words have no power.
Even though the mountain
becomes the sea,
Words cannot open anothers
mind.
4 FEATURES
6 FEATURES
I found his love for math contagious, even when it was hard for
me to keep up with the technical
aspects, which was often. My fellow
Electrical Engineering students,
more enchanted by the promise
of great money in EE rather than
a love for science, were less cooperative with him. It was sad to see
someone so passionate trying to
inspire those who simply did not
want to be inspired. (No disrespect
intended to these students; I love
money too.)
After the class ended, and all the
students were putting their notebooks back in their backpacks and
leaving, I went to the professors
desk. He was packing as well. I told
him I had a mathematical thought
that I didnt really know what to
do with. He was intrigued. He was
too busy at that time to continue
the conversation, but he told me to
email him, and via email we set a
time for an appointment.
I arrived at his office. It was small
and cramped and incredibly messy.
There were boxes of papers everywhere. Fortunately, there was a
whiteboard.
8 FEATURES
3*Sin(57)^74*Arctan(0.42^3.5)
will also be paired with the number
it describes. Were in effect pairing up each member of the set of
descriptions to a member in the set
of real numbers.
But, remember what we said
before, that if you try to pair up an
infinite set of size Aleph zero with
an infinite set of size Aleph one,
you will never be able to cover the
entire Aleph one set. There will
always be unfortunate members
in that bigger set that would be
left without a counterpart in the
smaller set.
Those unfortunate members are
indescribable numbers.
Why? Because consider what
they are: They are real numbers,
for which we have just proven it
is impossible to find a description
that will match them. We have
proven that no description will ever
describe them.
We have proven that indescribable numbers exist. Q.E.D.,
and please someone hand me a
cigarette.
I explained all this to the professor. He asked a bunch of questions,
and I managed to answer all of
them without having an Oh shit,
you just discovered a critical flaw in
my proof, my entire proof is wrong
now fuck me moment. At the
end, when he was quite convinced
that my proof was correct, he said
something along the lines of thats
pretty cool. He said he was unfamiliar with this area of mathematics, but that he thought my proof
was correct and really interesting.
I felt so proud. I managed to find
a proof that impressed not only me,
but a genuine crazy math professor!
Im smart!!!
k
c
a
H
k
c
u
D
k
c
u
D
Create instant answer plugins for DuckDuckGo
duckduckhack.com
10 FEATURES
Any developer who didnt know about this beforehand would have easily been caught out. The developers at Flickr, Vimeo and Remember the Milk rolled this
out to production.
The point isnt that you should know about every
esoteric detail of the internals of cryptographic functions. The point is there are a million ways to mess up
cryptography, so dont touch it.
Not convinced? OK, lets try fixing this example and
see if we can make it secure...
11
require 'openssl'
## Grab the user credentials out of the DB
user
= User.find(params[:user_id])
secret = user.secret
## Get the MAC out of the request params
challenge_mac = params.delete(:hmac)
## Calculate the HMAC
## We'll do the same thing on the client when we
## generate the challenge
message
= params.each.map { |key, value|
"#{key}:#{value}" }.join('&')
calculated_hmac = OpenSSL::HMAC.
hexdigest(OpenSSL::Digest.new('md5'), secret,
message)
## Compare the challenge and calculated MAC
if challenge_hmac == calculated_hmac
# The user authenticates successfully, do what
# they ask
else
# The user is not authenticated, fail
end
12 FEATURES
13
PROGRAMMING
Breaking a
Toy Hash Function
By Craig Gidney
14 PROGRAMMING
This year, I finally succeeded. I reversed the password, and all three usernames.
Note that I am not a cryptographer. The way I broke
this function is probably naive. I assume that, to a
real cryptographer, this function is a toy to be crushed
in an hour (Ha! Just use X!).
Nevertheless, I broke the hash function and Im
going to explain how.
Leaking Entropy
The first thing to notice about the above function
that suggests it should be easy to break, is that it leaks
entropy. It is using non-reversible operations, which
decrease the number of states the system might be in.
To make it easier to talk about that, heres a spread
out version of the internal loop, with the multiplication
by -6 factored and the division by 3 split into rounding
followed by inverse-multiplying.
a *=
a *=
a +=
a +=
a -=
b -=
b *=
(mod
b +=
b +=
b -=
2;
-3;
b;
0x74FA;
e;
b % 3; // round to multiple of 3, towards 0
-1431655765; // multiplicative inverse of 3
2^32)
a;
0x81BE;
e;
When working in modular arithmetic, some multiplications are reversible (do not leak entropy), but
others arent.
Multiplying a 32-bit integer by 3 does not decrease
the amount of entropy because it is reversible. Every
input state corresponds to exactly one output state.
You can even efficiently run the operation backwards by multiplying by the modular multiplicative
inverse of 3. The multiplicative inverse of 3 is 3 1=
-1431655765 (mod 232) because multiplying them
together gives a result equivalent to one: 3 3-1 = 3
-1431655765 = -42846795 = -232 + 1 = 1 (mod 232).
Multiplying by 2 is NOT reversible. It does decrease
the amount of entropy. This happens because (x + 231)
2 = x 2 + 232 = x 2 (mod 232), meaning both inputs
of either x or x + 231 are collided into the single output
of 2 * x. In the worst case this limits the possible
number of output states to be half the number of input
states, destroying 1 bit of entropy. Many inputs map to
one output, so the operation is not reversible and leaks
entropy.
The other non-reversible operation is rounding to
the nearest multiple of 3 towards 0. In the worst case
this destroys about 1.5 bits of entropy, reducing the
number of possible states by about a third.
15
These leaks occur every single round, and its possible for their cumulative effects to be very bad. Its
a bit like those mixing tank problems you solve
when learning differential equations, except the input
mixture keeps changing color. If the tank is leaking
then the contributions of the early colors to the average color decrease exponentially, instead of linearly, as
more colors are added.
These leaks make me suspect that earlier values are
in danger of diluting away. Every round destroys a
couple bits and replaces them with mixtures of the
remaining entropy. Later values dont get destroyed and
mixed much, but early ones do. Maybe, to find a preimage, I only have to care about the last few characters
instead of all the characters. Maybe, to find a collision,
I can significantly increase my chances by adding the
same long suffix to any two starting strings.
It turns out that these leaks werent devastating, but
they really shouldnt have existed in the first place.
Fixing the leak caused by multiplying by -6 is as easy as
changing 6 to 7. Fixing the leak caused by rounding to
a multiple of 3 is also easy: just remove the rounding.
Wait, no, that last idea is terrible.
Almost Linear
All of the operations in the hash function, except
rounding to a multiple of three, are linear. They distribute over addition.
If we removed the rounding operation, the contributions of every input could be separated and reduced
to a single multiplicative constant that depended only
on the position relative to the end of the string. Each
input value would be multiplied by the constant corresponding to its position, youd sum up the products,
and thatd be the result of hashing. Suddenly, finding
an input that hashes to a given value would be like
solving the subset sum problem, and there'd be all this
structure we might be able to take advantage of to save
huge amounts of time.
Fun fact: if you fixed the entropy leak due to the
rounding (by removing it), but didnt fix the leak due
to the multiplication by -6, youd have made things
far, far worse. The constants corresponding to positions
would keep gaining factors of two. Ultimately, only the
last four characters would get non-zero corresponding
constants and collisions would be somewhat easier to
find.
16 PROGRAMMING
Collision
Weve finally reached the weakness I ultimately used to
beat the hash function: the size of its output.
The output size is 64 bits, which allows a bit more
than 1019 possibilities. I can search through every string
up to ten characters (with 93 possibilities per character), which is 9310 possibilities. Thats about five times
1019.
Right. At this point it doesnt matter how long the
real password is. By pure brute luck, Im going to
stumble onto strings that hash to the same thing.
My work is done. I just need to let the computer
churn.
17
Code
This is the code I used to break the hash function:
/// Returns a given start state and a sequence of values of the given length that reach the given
/// end state. If not such sequence exists, returns null.
public static Tuple<HashState, int[]> Break(HashState end,
int assumedLength,
IEnumerable<HashState> startStates) {
// generate bloom filter going backwards from end
var numExpandBackward = (assumedLength - 1).Min((assumedLength * 2) / 3).Max(0).Min(4);
var filter = HashStateBloomFilter.GenReverseCache(end, numExpandBackward, pFalsePositive: 0.0001);
// explore forward from starts to filter, discard states that don't match
var possiblePartialSolutions =
from start in startStates
from midStateAndData in start.ExploreTraceVolatile(assumedLength - numExpandBackward)
where filter.MayContain(midStateAndData.Item1)
select new { start, data = midStateAndData.Item2.ToArray(), end = midStateAndData.Item1 };
// base case: not enough length to bother meeting in the middle.
// Partials are actually complete solutions.
if (numExpandBackward == 0) {
return possiblePartialSolutions
.Select(e => Tuple.Create(e.start, e.data))
.FirstOrDefault();
}
// we don't want to wait for all possible partial solutions before checking. That would take
// tons of memory. we also don't want to check after every single possible partial solution,
// because that's expensive. so we partition possible solutions and check whenever there's
// enough to make it worth the time.
var partitions = possiblePartialSolutions.PartitionVolatile(10000);
// complete any partial solutions
var solutions =
from partition in partitions
let partialSolutionMap = partition.ToDictionary(e => e.end, e => e)
// recursively solve the gap
let secondHalf = Break(end, numExpandBackward, partialSolutionMap.Keys, true)
where secondHalf != null
// Anything reaching here is a solution. Combine it with the first half and return it.
let partialSolution = partialSolutionMap[secondHalf.Item1]
let start = partialSolution.start
let data = partialSolution.data.Concat(secondHalf.Item2).ToArray()
select Tuple.Create(start, data);
// actually run the queries
return solutions.FirstOrDefault();
}
18 PROGRAMMING
Solutions
After about two days of computing, and one dead
laptop, the code returned a password that matched the
password hash. The password is <+nt1AkgbMht (or
rather, <+nt1AkgbMht is a string that hashes to the
same thing as the true password). If youre wondering
why the password has 12 characters, when I said I was
searching 10, recall that the first two characters of the
password were given away in the JASS code. I searched
10 additional characters.
(Its tempting to pretend I didnt know those two
characters, because 9312 1024, so I could say I literally
searched a trillion trillion possibilities.)
After another three days, I had both remaining usernames. These are clearly collisions, instead of the actual
names, but here they are nonetheless: hRlGz%W3&R
and b>4FXVXf8 match the first and third hashed
usernames respectively (the second was Procyon).
Summary
Things weve learned about writing hash functions:
Dont use the hashs entire state as its result. Running backwards from the result should be hard. (See
also: length extension attack.)
My Reward
With the solutions in hand I can finally download
Phase Killer, play it in single player with a profile called
Procyon, say <+nt1AkgbMht and see... a VALID
message.
Worth it.
19
20 PROGRAMMING
21
Initial Setup
Assuming you are running an Ubuntu box, once you
have set up your login and have apt updated, just run
apt-get install nginx and everything should install
cleanly. Visit your servers IP address in a web browser
and youll see the welcome to nginx message. Great
success.
22 PROGRAMMING
Finding nginx
When nginx is installed (through apt), it provides a
solid basic structure for how to set up your config files.
All nginx config files are located in /etc/nginx, so cd
there and poke around. The place youll want to add
new configurations is the sites-enabled folder. If you
check this folder out, youll find that theres a single
text file called default in there, and opening that up
youll see an nginx configuration and the code that
causes the welcome to nginx page to display. Now
lets make our own config file with the bare basics to
display a page. Touch a new file inside sites-enabled
called test, open it up in your text editor of choice,
and lets get to it.
Note: Youll also find a /etc/nginx/sites-available
directory. If you find yourself managing many different
sites that are coming up-and-down, this folder can help
keep things organized. Add your nginx configuration
files here instead and then symlink them to sitesenabled. This command might look something like
this...
ln -s /etc/nginx/sites-available/dotcom /etc/
nginx/sites-enabled/dotcom
server_name
server {
}
server {
listen 80;
server_name example.com;
}
server {
listen 80;
}
Note that this is the default and is not strictly necessary to enter, but its good to do it anyway in this case
to show whats going on. Bam, first directive written.
Feels great. Lets keep rolling.
23
root
Now that weve got the basic variables set, lets actually listen for hits to a specific route.
location / {
try_files $uri $uri/ /index.html;
}
location
}
}
24 PROGRAMMING
Note that you can twist this configuration any way you
want. For example, on carrot.is, we have it configured so
that when you hit a filename without the .html extension,
try_files looks for $uri.html and matches that as well.
So you could go to http://carrot.is/about as well as http://
carrot.is/about.html, and they would both return the
same document. The amount of wizardry you can do with
the server config is limited only to your crazy ambitions.
Ship It
Okay, so what have we actually done here? What weve
done is that weve added a server declaration to nginxs
configuration. When nginx runs, it slurps up all of the
configuration files you have put into /etc/sites-enabled
and uses those to know what to display to your viewers.
But wait! If you stopped here, you might not be able to
see your new server this is because nginx doesnt quite
know about your new changes yet. To get nginx to know
about your new configuration you need to reload nginx so
that it can pull in your new configuration. The easiest way
to do this is to run
service nginx reload
25
STARTUPS
Photo: flickr.com/photos/bongonian/8534960933/
26 STARTUPS
Why, exactly?
Theres a tacit assumption that if
only we just stopped spending to
grow, wed be profitable. Thus, this
really is a profitable company, and
the only reason its not is because of
growth, which means market domination, which is a Good Thing.
The fallacy is: That time never
comes. No company stops trying
to grow! The mythical time when
growth rates are small so the company reaps the rewards of having a
huge stable of profitable customers
never arrives. When do you show
me the money?
Its worse. Growth becomes
harder and harder for SaaS companies because of cancellations.
Even with a great retention rate
(e.g., 75%/year), you have to
replace 25% of your revenue with
new which means unprofitable
customers just to break even in
top-line revenue! More losses, more
unprofitability.
Even with very broad numbers,
you can see how this model doesnt
work. Heres typical numbers for an
enterprise SaaS company at scale:
27
28 STARTUPS
SPECIAL
Procrastination Is
Not Laziness
By David Cain
it.
30 SPECIAL
go along; its like this for everyone at first. I have since worked
it through, mostly, but not before
it set off a pretty bad stress cycle
that brought some ugly stuff to the
surface.
Honestly, it probably would have
been a much easier adjustment for
most people to make than it has
been for me, but my initial uncertainty combined in very ugly ways
with my lifelong phobias of asking
for help, admitting ignorance, and
talking to people I dont know on
the phone. Paralysis set in. Stress,
which has been a mostly-dormant
force in my life for the last five
years or so, became prominent
again.
Once you lose track of the
specific items that are causing you
stress, you tend to regard it all as
one big ugly entity that you want to
avoid. My unaddressed duties and
grey areas at work became mixed
with my unaddressed duties and
grey areas outside of work, to create
a stifling mutant stressor that only
leaves me alone while Im sleeping. All the work Ive done towards
learning to effect the quality of the
day can be easily short-circuited by
31
is a neurotic self-defense
Procrastination
behavior that develops to protect a persons
sense of self-worth.
32 SPECIAL
33
All I want
As I mentioned, on Monday I will
begin Experiment 11, which is
direct attack on my procrastination
problem. Ill give you the details
then about how Im going to go
about it.
All my experiments must have
a clear aim. Dealing with my procrastination problem is too vague
a goal here. I have to define what
specific change I want to make.
What I want to get out of it is
very simple. I want to be able to do
something many (most?) people do
every day, and would never consider it a problem:
I want to write down what Im
going to do the next day, and actually do it.
I am really good at the first half
of that. Planning is something I do
very well. I have planned the next
day (or week) thousands of times.
Ive taped it to my door or bathroom mirror. Ive set alarms, made
promises, left trails of instructional
sticky notes all through my apartment. But I am not sure if Ive ever
executed one of these plans all the
way through. Honestly, in my 30
years I cannot think of one time I
ever did. I will do anything but the
5 to 10 items I thought would be
smart ones to do.
Its hard to pinpoint exactly why
Ill do anything but what I planned,
but its not that theyre necessarily
difficult tasks. Sometimes theyre so
easy that I dont feel any urge to do
them right away, and therefore can
justifiably do something even easier,
like check my email, watch online
documentaries, or try a new recipe.
34 SPECIAL
The Attack
By Shaanan Cohney
36 SPECIAL
Phase 3: Exploitation
To exploit our position of power
we had many options, some of
which would be pushing the assignment over the edge. With this level
of trust it would be feasible to
gain access to information protecting online accounts, a very scary
thought. However, we decided
to go down a different route and
instead convinced them of the need
Also, are there any current projects in Java you are working on
for which a codebase is available
for our engineers to review? Even
a work in progress is fine. Were
really interested in seeing material
and your personal projects from
this course given the nature of the
internship.
Finally we struck gold! A few
hours later the following appeared
in my inbox:
Please find attached herewith 2
Java source code files. (server.java
and client.java)
These are for a basic chat system
application. Further, I and my
group would be adding some
encryption techniques in it (Ill
send you those once we start
working on it and progress to some
level).
And later:
Hi Joseph,
Please find attached herewith 2
Java code files for a chat system
with AES encryption.
Thanks.
Regards.
37
38 SPECIAL
Without affiliate.io...
With affiliate.io...
Affiliate #042
- Lisa, Marketing expert
Affiliate #011
- Tim, power user & ambassador
Affiliate #094
- Diana, owns 7 blogs
Affiliate #027
- Tom, industry expert
AFFILIATE.IO
The fast and easy way to accept affiliates into your online business
YOUR
YOUR
YOUR
APP
APP
APP
PUSHER
PUSHER
PUSHER
YOURYOUR
DATAYOUR
DATA DATA
YOUR
DATA
YOUR
DATA
YOUR
DATA
YOUR
YOUR
DATA
YOUR
DATA
YOUR
YOUR
DATA
YOUR
DATA DATA
DATA
"BUILD
"BUILD
"BUILD
REALTIME
REALTIME
REALTIME
APPS
APPS
APPS
ININ
5IN
MINUTES"
5 MINUTES"
5 MINUTES"
www.pusher.com
www.pusher.com
www.pusher.com