MySQL for Big Data

An excerpt from article on mysql for big data published in Dow Jones Venture Wire by Scott Denne.

There is one possible solution to the problem that doesn't include companies having to buy new software tools or even an all-new database: With the right expertise, MySQL can be engineered to handle almost any data-intensive application. The only problem is that there's a shortage of people who have the expertise to make it work.

"There's a big time gap until we, as an industry, think we have data under control," said Frank Mashraqi, chief technology officer at MyLawsuit.com and former database chief at Fotolog Inc., a photo blogging site. "The roadmap to getting that expertise is very difficult and time doesn't allow for it."

MySQL at Facebook

Mark your calendars for Nov 2 as Mark Callaghan and Facebook's MySQL team will be talking about how MySQL is used at Facebook.

Community One East - What will Sun announce?

I will be attending Community One tomorrow and on Thursday at Marriott Marquis Hotel, New York, NY. I am especially looking forward to the announcements tomorrow which sound very interesting :)

The first day is a free event featuring:

* Cloud Platforms – Development and deployment in the cloud.
* Social and Collaborative Platforms – Social networks and Web 2.0 trends.
* RIAs and Scripting – Rich Internet Applications, scripting and tools.
* Web Platforms – Dynamic languages, databases, and Web servers.
* Server-side Platforms – SOA, tools, application servers, and databases.
* Mobile Development – Mobile platforms, devices, tools and application development.
* Operating Systems and Infrastructure – Operating systems and virtualization.
* Free and Open – Open-source projects, business models, and trends.

The second day of the event is focused on Deep Dives with two half-day sessions on MySQL and two full-day sessions on Java and Web development. I will be attending the session, "Using Java EE and SOA to Architect and Design Robust Enterprise Applications."

Following the conference, I will be a panelist at a Cloud Computing Seminar at Microsoft office in NY. It's going to be a long but exciting day!

It will be great to catch up with old and new friends at the event.

FriendFeed uses MySQL to store "Schema-less" data

Came across an interesting post by Bret (co-founder of FriendFeed) about how FriendFeed uses MySQL to store "schema-less" data. According to the post, they weren't having issues with scaling existing features but rather they were experiencing pain when trying to add features.

Now the way they are using MySQL is interesting and bizarre at the same time. At a very high level, it seems their approach is to use a RDBMS as if it is a column-oriented database. Of course, it makes me wonder why not just use a column-oriented database? I need to read the post again in the morning (too tired right now so just gave it a quick glance).

I am very interested in hearing thoughts from my peers at Planet MySQL regarding this approach. They seem to have gone great lengths to go this route. What issues and benefits you see of this approach and whether you ever see yourself taking this route? I, for one, am not entirely convinced of this approach and whether it can really scale down the road. Also, if it was someone other than Friend feed going down that route, I might have actually lost my tempered and yelled :)

Side note: Friendfeed is growing fast, and it would have been cool if Bret was speaking at one of the three upcoming MySQL events in April.

Notes from Structure 08, Velocity and Graphing Social Patterns East

I attended several events in June of this year including Graphing Social Patterns East, Velocity and Structure 08. At each of these events, I tried to take some notes and posted them to my personal blog. I received a few pings from readers of this blog to point them to a list of these posts. It took some time but here is the list of my notes. In some cases, I have linked directly to the presentation files.

High-performance Ajax Applications: Julien Lecomte (Yahoo!) talked about how to effectively use AJAX in your applications without compromising performance.
Slideshare: High performance Ajax Applications

Stress, Load and Performance Testing in Quality Assurance: Excellent tips on stress and performance testing by Goranka Bjedov of Google.

Actionable Logging for Smoother Operation and Faster Recovery: Mandi Walls from AOL talked about logging in general including actionable logging, why it's important, logging goals, log file management, things to avoid in logs and more. (Presentation slides)

Clouds are No Substitute for Competence: Presented by Javier Soltero of Hyperic

Energy Efficient Operations: Some Challenges and Opportunities: Luiz Barroso from Google presented this very interesting and informative session about making operations energy efficient.

Innovation That Drives Opportunity for the Web Infrastructure: John Folwer (Sun Microsystems) was the speaker at this talk about Web 2.0 architectures. (Presentation slides)

Importance of Operations and Performance: Artur Bergman of Wikia talked about lessons learned while running 7000 wikis.

Jiffy: Real World Performance Measurement: In this session Scott Ruthfield talks about Jiffy, an open source tool for performance measurement and instrumentation. (Presentation slides)

KITE: Keynote Internet Testing Environment Launch: KITE was one of the interesting products launched at Velocity. KITE allows you to test from desktop to the Internet cloud. At the time of launch KITE was free. Don't know the current pricing model. (Presentation slides)

Harnessing Explosive Growth: Infrastructure Strategies and Tactics: Panelists including Sandy Jen, Akash Garg, Jeremiah Robinson, Jonathan Heiliger and James Barrese discussed strategies and tactics for handling explosive growth.

The Race to the Next Database: Overclocking and Analytics Augment Your Data Layer: At Structure 08, panelists on this session included Mayank Bawa (Aster Data Systems), Doug Judd (Zvents), Luke Lonergan (Greenplum), Damian Black (SQLstream), Dave Schrader (Teradata) and Scott Wiener (Cloud9Analytics). Each panelist provided insight into the ground breaking work their company is doing in solving data processing and handling BI challenges faced by consumers today.

Working the Clouds: NextGen Infrastructure for New Entrepreneurs: This panel on cloud computing featured panelists including Geva Perry (GigaSpaces), Jason Hoffman (Joyent), Tony Lucas (XCalibre), Lew Moorman (Rackspace), Christophe Bisciglia (Google / AppEngine) and Joe Weinman (AT&T). Christophe got grilled heavily by other panelists but he handled it pretty darn well.

Werner Vogels: Keynote at Structure 08: Dr. Vogels keynote was one of the highlights of Structure 08. He presented case study of Animoto and talked about the 70/30 switch among other things.

The Platform Revolution: A Look into disruptive technologies: Jonathan Yarmis AMR Research (VP of Disruptive Technologies) talked about technology trends, social networking, mobility, mobile , cloud computing, stream computing, business models, user 2.0 and the new enterprise reality.

Green Data Centers: Bill Coleman (Cassatt Corporation) presented this session. Bill is known for being responsible for the B in BEA. (Presentation slides)

Creating Bebo Applications: Bebo is now part of OpenSocial and this presentation presented at Graphing Social Patterns talks about how to create applications for Bebo.

Open Social and Google App Engine: Patrick Chanezon (API Evangelist) and Paul McDonald (Product Manager for Google App Engine) presented a technical overview of OpenSocial and Google App Engine at Graphing Social Patterns East.

OpenSocial: Open for Business: In this session, panelists were Patrick Chanezon (Google), Paul Lindner (hi5), Max Newbould (MySpace) and Sachin Rekhi (imeem). (Also see)

Viral Marketing and Advertising Strategies for social networks: One of the best sessions at the Graphing Social Patterns conference presented by Kevin Barenblat and Jeff Ragovin. (Also see)

Mobile Social Networks: A Comparison: Benjamin Joffe's excellent eye opening session for anyone interested in using mobile platform for creating social networking solutions.

Top 5 Things that fail and win on social networks: Dave McClure, chair of Graphing Social Patterns, presented this concise but every effective presentation on what fails and what wins on social networks.

Geek Metrics: Using App Analytics to Drive Distribution, Engagement, & Monetization: Dave McClure (500 Hats) moderated this panel which included Hiten Shah (CrazyEgg / KISSmetrics), Ian Swanson (Sometrics, Inc.), Albert Lai (Kontagent) and Roy Pereira (Refresh)

Social + Mobile = Sociable (Social Networks for SMS, IM & Mobile Devices: Panelists in this session included Benjamin Joffe, Ben Keighran, Gregory Cypes, Craig Dalton and Chris Butler.

Widget Strategies & Social Platforms: Hooman Radfar, CEO of Clearspring Technologies discussed the new role of widgets and how to go about creating them.

Facebook Business and Marketing Solutions: Kent Schoen talked about how to use Facebook for business and marketing.

Developing and Promoting Social Network Applications: Rules of thumb: What does FACEBOOK means when it comes to creating and promoting applications for social networks?

Social Networks for Business and Marketing Managers: Ro Choy of Rock You! gave an overview of social networks for business managers:

Scaling MySQL - powered Web Sites by Sharding and Replication: Slides from Peter Zaitsev's session at Velocity. (Presentation slides)

Capacity Management: John Allspaw's signature presentation on capacity management. John also has a book coming out on this topic. (Presentation slides)

Structure 08 on demand: Watch the Structure 08 conference on demand at Mogulus.

LinkedIn Communication Architecture: Slides about LinkedIn's platform built in Java. (Presentation slides)

SOX Compliance: A presentation by Skye Rogers. I missed this presentation but then caught up with Skye at dinner. I wish Skye would have received more time to discuss SOX Compliance. (Presentation slides)

There were several sessions I didn't get to go to which is a sad thing. You may want to check the conference websites directly (linked at top of this post) to see if there are presentation slides available. Also, if you took notes at these sessions, please feel free to drop the links as comments to this post.

Web Developer / Graphic Designer Job Openings

Currently, there are several great opportunities with exciting companies available in the New York area. If you're a rock star Java/PHP/Ruby developer or a pixel-obsessed designer, contact me at your earliest convenience.

Web Developer:

Give Real is a well-funded startup in the midst of an exciting period of growth and success. Our technology uses a patent pending platform that combines the ubiquity of credit card transactions and the power of social networks to create a new gifting experience.

Our primary platform is Rails, but there are programming challenges that range from SOAP APIs to Facebook application development. We are searching for full-time developers with expertise and broad experience in:

* Ruby on Rails (we also use rSpec, Starling, Memcache)
* MySQL
* xHTML & CSS, and comfort with Javascript
* Team development with tools like Git & Trac

In addition, we are also interested in candidates who have:

* Expert Javascript skills
* Java & SOAP experience
* Experience scaling with Rails, or any other web platform
* Comprehensive Linux knowledge
* UI and graphic design backgrounds

We are willing to pay top-notch developers very competitively (plus the possibility of options) to join our team and help write code that will be used by hundreds of thousands of users within a few months. We are ideally located in downtown Manhattan less than a minute walk from the BDFV and NRQW lines

Also, if you know someone who may be a good fit for us (developer or graphic designer), we are offering a $1000 referral reward for anyone we hire.

Please contact us at [email protected]

Graphic Design:

Give Real is a well-funded startup in the midst of an exciting period of growth and success. Our technology uses a patent pending platform that combines the ubiquity of credit card transactions and the power of social networks to create a new gifting experience.

We're searching for full-time designers with experience in:

* Design for advertisements
* Design for consumer focused websites & applications
* xHTML & CSS coding
* HTML & design for emails
* Working on top of an MVC or template system (we use Rails)

In addition, we are also interested in candidates who have:

* Team development with tools like Git & Trac
* Comfort with Javascript programming
* Rails programming experience

We are willing to pay top-notch developers very competitively (plus the possibility of options) to join our team and help design the look and feel of a service that will be used by hundreds of thousands of users with a few months.

Also, if you know someone who may be a good fit for us (RoR developer or graphic designer), we are offering a $1000 referral reward for anyone we hire.

Please contact us at [email protected]

Memcached for MySQL: Advanced Use Cases (Recoding and Slides)

My second webinar (hey it's an officially recognized word now!) on memcached, Memcached for MySQL: Advanced Use Cases, is now available on-demand from MySQL website.

Also see:

• Slides for Memcached for MySQL: Advanced Use Cases
  

Memcached for MySQL Webinar: Advanced Use Cases

Today at 1PM EST I am presenting the second part of memcached for MySQL webinar. I was told that the registration numbers look as good as the previous one. This one will be a bit more technical than the previous webinar. Sorry for the late notice but hope you can join!

Michael Arrington Asks Twitter a Few Tough Questions

Michael Arrington of TechCrunch asks Twitter a few questions. I have only included a sample list below but you should read his blog post for all the questions:

• Is it true that you only have a single master MySQL server running replication to two slaves, and the architecture doesn’t auto-switch to a hot backup when the master goes down?
• Do you really have a grand total of three physical database machines that are POWERING ALL OF TWITTER?
• Is it true that the only way you can keep Twitter alive is to have somebody sit there and watch it constantly, and then manually switch databases over and re-build when one of the slaves fail?
  

A 'yes' answer to any of these questions by Twitter would be disturbing to say the least. However, it won't be surprising as companies expect databases to just somehow magically work without creating and supporting a proper architecture. High availability doesn't comes cheap and reputation for companies is everything.

I find it amusing that Twitter isn't even looking for a DBA. May be that's considered a job for the SA over there :)

Memcached Webinar - 560+ registrants

A big thank you to all those who attended the memcached webinar today on which I was a panelist. I was told that there were more than 560 registrants.

The feedback I received directly and indirectly shows that there is a lot of interest about memcached. In the future, I hope to work again with MySQL/Sun on more memcached related webinars.

If you attended the webinar and have some suggestions, comments or questions, please contact me at fmashraqi at yahoo dot com or post a comment on this blog.

Special thanks to Jimmy Guerrero, Monty Taylor, Rich Taylor, Edwin DeSouza and Alex Roedling for their hard work in arranging the webinar. Also thanks to Brian Aker, Matt Ingenthron and Trond Norbye for their assistance at various phases.

In case you missed the webinar:

• webinar recording: Designing and Implementing Scalable Applications with Memcached and MySQL ( you'll need Webex player to view the recording )
• memcached webinar slides

Memcached Webinar - 420 Registrants and Counting!

Regarding my earlier post on memcached webinar, I was informed today that more than 420 registrants have signed up. Space is limited and filling up fast so if you are interested in memcached and haven't registered yet, click on the following link to register now!

Designing and Implementing Scalable Applications with Memcached and MySQL (June 29)

Presenting a Webinar on Memcached Use Cases

---

Quick link: register for Designing and Implementing Scalable Applications with Memcached and MySQL webinar (June 29)

---

Ever since its introduction, memcached has been changing the way cost-efficient caching is perceived. Some passionately love it, others cynically hate it.

Today, many large scale web 2.0 properties (including my employer) save millions of dollars by depending on memcached to bring their application response time under control and to offload pressure from databases.

There are several success stories about using memcached to speed up database driven websites. Facebook, for instance, runs the largest memcached installation and the numbers only keep increasing. In May 2007, Facebook was reportedly running 200 dedicated servers with 3TB of memory in their memcached cluster. At the "Scaling MySQL Up or Out" Keynote, Facebook revealed they are now using 805 dedicated memcached servers. That's more than a 400% increase in less than a year!

Twitter, digg, Wikipedia, SourceForge, and even Slashdot depend on memcached to keep their users happy.

For my employer, memcached has been a crucial component of the infrastructure that has been instrumental in handling explosive growth in a cost-efficient manner. In addition, memcached has helped us offload billions of queries from our database.

To highlight several real-life use cases of memcached (see below), I will be presenting a memcached webinar on Thursday, June 29 at 1 PM EST (10 AM PST). Monty Taylor (Senior Consultant, Sun Microsystems) and Jimmy Guerrero (Sr Product Marketing Manager, Sun Microsystems - Database Group) will also be speaking at the event. Space is limited and filling up fast (200+ registrants already) so I highly recommend registering now.

In this webinar, I will be covering several use cases for memcached including (but not limited to):

• deterministic cache
• non-deterministic cache
• proactive cache
• "state" cache
• filesystem cache replacement

Hope to "see" you at the webinar.

Note: This memcached webinar is not to be confused with the memcached webinar being presented by Ivan Zoratti on June 28.

Interview by Sun TV at MySQL Conference

At the MySQL Conference and Expo, right after my participation in scaling up or scaling out keynote panel, I talked to Sun's Multimedia team about Sun and MySQL in our environment.

Recently, I found the interview on Sun's Multimedia page. The video of my discussion is embedded below:

Solaris 10 User Group Part X

Tomorrow I will be attending the Solaris 10 User Group Part X at the offices of Sun Microsystems, 101 Park Ave., New York, NY. This is an all day event and there is even a MySQL talk by Philip Antoniades. Other presenters include Ambreesh Khanna, Isaac Rozenfeld, Neal Weiss, Sunay Tripathi, Amjad Khan, Damien Farnham and Dave Teszler.

Unfortunately, the event registration is now closed, but if you're attending I look forward to meeting you.

Sun loses 23% market capital

Sun missed its earnings and sales estimates and as a result it lost approximately 23% of its market capital. Even more disturbing news is the announcement that Sun will be cutting 1500 to 2500 jobs. Eric Day raised his concerns as to whether this job cut will affect MySQL hiring to which Marten replied and pointed to several open positions within MySQL.

Sun has an array of very interesting and useful technologies under its hood. The amount of care Sun takes for its customers is truly impressive and I hope MySQL will follow in Sun's foot steps. Yesterday, I met with a Sun engagement architect and the amount of interest he showed in the technical challenges my team faces was unmatched. I am already working on a blog post to highlight some of the technologies my team discussed with Sun's representative.

With Sun's stock now down, I think it is an excellent time to buy some JAVA stocks which closed at 12.64. I may actually put a small order myself.

MySQL / Linux swap problem doesn't exist on Solaris 10

Right now there is a discussion on Planet MySQL regarding MySQL / Linux swap problem. Peter Zaitsev originally brought the problem of MySQL swapping to light. Recently, Dathan Pattishall also wrote about it in his post Linux 64-bit, MySQL, Swap and Memory. Don McAskill followed up with his post, MySQL and the Linux Swap problem, and an interesting way to get around the issue: "make swap partitions out of RAM disks." Don also points to another article by Kevin regarding using O_DIRECT to fix the swap issue.

To get to the point, some time ago, I experienced a similar issue on few of my old servers running Solaris V210, UFS with plenty of memory available. My initial thinking was that I am experiencing similar issue so during my presentation at MySQL Conference, Optimizing MySQL and InnoDB on Solaris 10, I pointed that this *may* exist in Solaris 10. Luckily a Sun representative (I believe it was Matt Ingenthron) corrected me towards the end of my session and pointed that UFS and Solaris 10 kernel have features built to avoid just that. That confirmation from a Sun representative was authoritative. We have already decommissioned the affected servers from production so it may be some time before I can find the precise reason why we experienced the swapping issue. Note that I haven't seen this issue on any of our other V210, V440 and T5120s in production.

Optimizing MySQL and InnoDB on Solaris 10 for World's Largest Photo Blogging Community - Video

The video of one of my three sessions, "Optimizing MySQL and InnoDB on Solaris 10 for World's Largest Photo Blogging Community", presented at MySQL Conference & Expo 2008 has been uploaded by Sheeri. I am very thankful to her for doing all the hard work and making it available.

There are a few slides that were edited out of video because of reasons beyond my control. However, you should still be able to enjoy most of the video.

There is one point related to this video that I would like to make: Based on my particular experience I was leading to believe that Solaris 10 Kernel had the same issue as Linux Kernel related to swappiness and swapping where the kernel will start putting more importance on maintaining file system cache than the mysqld process. However, towards the end of the session, it was pointed out by a Sun engineer (thanks!) that there must be something else going on as UFS on Solaris 10 shouldn't depict this behavior and a process shouldn't swap in favor of maintaining file system cache. I am having this issue on 3 of my servers and I am currently working with Sun engineers to get to the bottom of the issue.

Don McAskill - People I met at MySQL Conference

"The two metrics that are most important to me are first customer satisfaction and second growth." - Don McAskill

Today, I noticed Don is featured on Sun's customer success stories page:


Don McAskill is the CEO and Chief Geek of Smugmug, a photo and now hi-def video (using H.264) sharing site with a successful business model behind it.

I initially met Don last year at the MySQL Conference when my then boss told me that he is interested in meeting him. That was my introduction to Smugmug. I was impressed by SmugMug's presentation of photos and the care they took to make your photos and galleries look awesome.

This year, as a member of Smugmug, me and my wife got to interact with Don on a personal level.

We had several suggestions related to how our Smugmug experience could be improved and Don listened very carefully. One of the things I was most interested in seeing implemented was blocking Smugmug subdomains from being indexed if a customer is hosting them on their own subdomain.

I was truly impressed by how much Don thinks and cares about his members. It isn't a surprise that he runs a very successful site. From my conversations with Don, It seems there are many interesting projects Don and his team are working on and I can't wait to see them implemented. Almost all of the projects we heard about were focused towards customers. No wonder Smugmug has a high customer retention rate.

Technology wise, I am a fan of decisions Don has made to run Smugmug. He uses MySQL, S3, EC2 for processing and video conversion, Solaris 10 and Sun hardware.

Despite being the CEO, Don is the MySQL guy at Smugmug. His latest blog post, Death of MySQL read replication high exaggerated, was a good natured jab at discussion Brian Aker started with Arjen Lentz and me jumping in.

In the following video, Don Grantham interviews Don McAskill (yup, two Dons together) about Smugmug's relationship with Sun and the challenges of running a successful Web 2.0 business with more than 350,000 paying customers and more than 300,000,000 photos. As you can see in the video, customer satisfaction is more important than growth to Smugmug.


Since I joined Smugmug, several of my friends including Ronald Bradford have also joined. You can view my galleries by clicking on the image below and Ronald's photos from the MySQL conference by clicking on the image underneath:





If you use Smugmug as well, drop your Smugmug URL as a comment (of course, only if you want to share).

To stay up to date with exciting stuff happening at Smugmug checkout Don's blog.

Scaling Up Or Out - Keynote at MySQL Conference 2008

At this year's MySQL Conference I was invited to be a keynote panelist at Scaling MySQL Up Or Out keynote. Other keynote panelists included Jeff Rothschild (VP of technology at Facebook and a consulting partner with Accel Partners), Paul Tuckfield (DBA at YouTube), John Allspaw (manager of operations engineering at Flickr) and Domas Mituzas (DBA at Wikipedia). There were also representatives from MySQL (Monty Taylor) and Sun (Matt Ingenthron).

I really enjoyed being a keynote panelist with my peers. We were seated according to our Alexa ranking with the highest ranking YouTube on the right side. Even though I was representing the thirteenth largest site, our traffic compared to Facebook and YouTube was humbling.

All of the keynote panelists met early in the morning to get equipped with microphones and to go over the format.

See the video (below) to hear some funny "can't say" answers by Paul Tuckfield. I wish Google won't keep him so secretive about numbers such as how many database servers etc. Does that really give out YouTube's secrets?

Following are some photos, videos and links to notes from the keynote.


From left to right:
Monty Taylor (MySQL),Matt Ingenthron (Sun), John Allspaw (Flickr), Farhan "Frank" Mashraqi (Fotolog), Domas Mituzas (MySQL/Wikipedia), Jeff Rothschild (Facebook) and Paul Tuckfield (YouTube)


Jam packed ballroom during the keynote.

Above Photos copyright: James Duncan Davidson.


Kaj Arnö leads the scaling MySQL keynote panel discussion.


Me getting animated.


Domas Mituzas, Jeff Rothschild and Paul Tuckfield at the keynote.


Matt answers a question as everyone listens

More photos from the keynote session are available at http://photos.mashraqi.com.


Video of keynote session:
-

- Sheeri/Technocation: Download, Play

- A short video by Zack Urlocker


Notes from scaling up or out keynote:
- Biographies of keynote panelists
- Keith Murphy: Scaling MySQL - - Up or Out? Panel @ UC
- Ronald Bradford: Scaling Wisdom
- Venu Anuganti: Notes from Scaling MySQL Up or Out

MySQL on Solaris 10 -- Buffer Overflow and Security Bypass Vulnerabilities

So found some recently discovered buffer overflow and security bypass vulnerabilities when running MySQL on Solaris 10. According to FrSIRT, these vulnerabilities "could be exploited by attackers or malicious users to bypass security restrictions, gain knowledge of sensitive information, cause a denial of service, or execute arbitrary code." A final resolution for these vulnerabilities is pending completion according to their website.

Unfortunately, I do not have a FrSIRT account currently (need to get one ASAP) so I couldn't dig this vulnerability further. However, I am dying to learn more about this.