Scribd
Upload
321 views

Lessons Learned: Scaling A Social Network

The document discusses lessons learned from scaling a social network, myYearbook.com. Key points include: - myYearbook grew rapidly from 100M requests/month in 2007 to 2.5B requests/month in 2010, requiring extensive infrastructure changes. - A variety of techniques were used to reduce database transactions per second and scale out PostgreSQL, including memcached, pgbouncer, pgproxy, and sharding. - The architecture evolved over time from a single database server to a multi-tiered structure with application servers, load balancers, external and internal pgbouncer tiers, and PostgreSQL sharding. - Other lessons included focusing on APIs, automating processes,

Uploaded by

Oleksiy Kovyrin
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PDF, TXT or read online on Scribd
321 views

Lessons Learned: Scaling A Social Network

The document discusses lessons learned from scaling a social network, myYearbook.com. Key points include: - myYearbook grew rapidly from 100M requests/month in 2007 to 2.5B requests/month in 2010, requiring extensive infrastructure changes. - A variety of techniques were used to reduce database transactions per second and scale out PostgreSQL, including memcached, pgbouncer, pgproxy, and sharding. - The architecture evolved over time from a single database server to a multi-tiered structure with application servers, load balancers, external and internal pgbouncer tiers, and PostgreSQL sharding. - Other lessons included focusing on APIs, automating processes,

Uploaded by

Oleksiy Kovyrin
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 52

Lessons Learned

Scaling a
Social Network
[email protected]
senior data architect
O’Reilly MySQL Conference & Expo 2011
2011–04–13
Who?
日本
UTF-8
PostgreSQL
community
Codd
Why?
myYearbook.com
casual social network
founded in 2006
Google Analytics
by gender by age
52% female 37% 13–17
48% male 27% 18–24
13% 25–34
12% 35–44
11% 45+
myYearbook Teens and Twitter Survey (August 2009, ~%)

myYearbook Facebook
70% meet people 80% keep in touch with friends

50% play games 35% meet people

40% keep in touch with friends 30% share photos

35% flirt/date 25% play games


comScore Teens Category (April 2009)
rank site visits (K) uniques (K) minutes (M) page views (M)

1 myYearbook 55,808 4,604 851 1,630

2 MEEZ 8,629 1,407 226 469

3 Zwinky 11,558 3,691 153 108


Hearst Teen
4 Network 6,940 2,314 53 117

5 Quizilla 7,924 2,058 75 75


comScore page views comScore time spent
July 2009 July 2009
rank site views (M) rank site minutes (M)

20 GaiaOnline 1,105 20 Amazon 806

21 Chase 1,056 21 CNN 744

22 ESPN 984 22 GaiaOnline 709

23 myYearbook 953 23 Bing 707

24 Wikipedia 903 24 MSNBC 691

25 Onemanga 840 25 myYearbook 678

26 Mapquest 833 26 Iwin 670

27 Foxsports 815 27 NickJr 665


2007
100M HTTP requests/month
1 PostgreSQL server
1 Apache/PHP server
static content servers
1 outage per night
2008
2.5G page views
30 PostgreSQL servers
120 Apache/PHP servers
8 ActiveMQ message brokers
99.84% uptime
2010
2.5G HTTP requests/month
Database servers
connection poolers, routers
Web servers
Apache/PHP, Tornado/Python, lighthttpd
Message brokers
ActiveMQ, RabbitMQ
Monitoring, R&D, other servers
rapid
growth
resources
1
API
bottleneck
time
Relative data access latencies
1 CPU Register
1–2 L1 cache
6–10 L2 cache
25–50 main memory (RAM)
10,000,000 hard disk
100,000,000 LAN
1,000,000,000 WAN
fsync
memory
more activity?
views/month 2007 100M ➙ 2009 1.5G
more TPS
more servers
more connections
more configuration
more pain!
reduce TPS!
➙ memcached
1.2 TB
get 140K/s, set 15K/s
pgfouine
Skype
PL/Proxy
pgBouncer
less configuration
fewer connections
app-server 1 app-server 2 app-server 3

postgres 1

PL/Proxy

postgres 2
fewer connections!
app-server 1 app-server 2 app-server 3

postgres 1

postgres 2 postgres 3 postgres 4


app-server 1 app-server 2 app-server 3

pgbouncer 1

postgres 1

postgres 2 postgres 3 postgres 4


app-server 1 app-server 2 app-server 3

pgbouncer 1

postgres 1

internal pgbouncer

postgres 2 postgres 3 postgres 4


reduce TPS/server
less configuration
app-server 1 app-server 2 app-server 3

pgbouncer 1

pgproxy

internal pgbouncer

postgres 1 postgres 2 postgres 3 postgres 4


app-server 1 app-server 2 app-server 2 app-server n

app-server 1 app-server 2 app-server 3 app-server n

app-pgbouncer 1 app-pgbouncer 2 app-pgbouncer 3 app-pgbouncer n

load balancer

external pgbouncer tier

pgbouncer 1 pgbouncer 2 pgbouncer 3

pgproxy

internal pgbouncer

postgres 1 postgres 2 postgres 3 postgres 4 postgres n


app-server 1 app-server 2 app-server 2 app-server n

app-server 1 app-server 2 app-server 3 app-server n

app-pgbouncer 1 app-pgbouncer 2 app-pgbouncer 3 app-pgbouncer n

load balancer

external pgbouncer tier

pgbouncer 1 pgbouncer 2 pgbouncer 3

pgproxy

load balancer

internal pgbouncer tier

pgbouncer 5 pgbouncer 4

postgres 1 postgres 2 postgres 3 postgres n


app-server 1 app-server 2 app-server 3 app-server n

app-server 1 app-server 2 app-server 3 app-server n

app-pgbouncer 1 app-pgbouncer 2 app-pgbouncer 3 app-pgbouncer n

load balancer

external pgbouncer tier

pgproxy 2

pgbouncer 3 pgbouncer 1 pgbouncer 2 pgbouncer

pgproxy pgproxy

load balancer

internal pgbouncer tier

pgbouncer 5 pgbouncer 4

postgres 1 postgres 2 postgres 3 postgres n


app-server 1 app-server 2 app-server 3 app-server n

app-server 1 app-server 2 app-server 3 app-server n

app-pgbouncer 1 app-pgbouncer 2 app-pgbouncer 3 app-pgbouncer n

load balancer

external pgbouncer tier

pgproxy 1 pgproxy 2 pgproxy 3

pgbouncer pgbouncer pgbouncer

pgproxy pgproxy pgproxy

load balancer

internal pgbouncer tier

pgbouncer 5 pgbouncer 4

postgres 1 postgres 2 postgres 3 postgres n


1
SOA
REST
DB API
death
failure is
a choice
network
measure
overview
learn
logical
versus
physical
thought
word
deed
Fusion-io
direct IO buffered IO
Postgres buffer

application buffer CPU

CPU kernel buffer

Fusion-io card
CPU

Fusion-io card
improve
automate
test
Lessons Learned
Scaling a
Social Network
[email protected]
Senior Data Architect
O’Reilly MySQL Conference & Expo 2011
2011–04–13
partition
sharding
inter and intra server

roll off old data

You might also like