5th International Electrical Engineering Congress, Pattaya, Thailand, 8-10 March 2017

A Comparative Study of Relational Database and

Key-Value Database for Big Data Applications

Wittawat Puangsaijai Sutheera Puntheeranurak

Faculty of Engineering
Faculty of Engineering King Mongkut's Institute of Technology Ladkrabang
King Mongkut's Institute of Technology Ladkrabang Bangkok, Thailand
Bangkok, Thailand [email protected]
[email protected]

Abstract— Nowadays, Demands of web scale are in increasing can be processed with large amounts of data quickly and with
and growing rapidly. Mobile applications, web technologies, flexibility situations. Although the Relational database is
social media always generates unstructured data that had lead to famous, it may not be compatible with the growing
the advent of various NoSQL databases. Therefore, Big data requirements that require more performance system. To
applications are necessary to have an efficient technology to overcome this problem, NoSQL, which is a non-relational
collect these data. However, a relational database is the database, is introduced. Therefore, there are many discussions
traditional database that always uses in many applications and in the database world to replace the relational database system
still has more valuable to play a significant role in the current with the NoSQL database system. Currently, NoSQL database
information system. The main characteristics of NoSQL
has been more popular in the current information system.
databases are schema-free, no relationship, no need to join as a
relational database. The business organization expects that
However, there is a question for the database designer to
NoSQL database has better performance than a relational choose NoSQL database or relational database to store
database. In This paper, we aim to compare the performance of information in their systems. They have information that
Redis, which is a key-value database, one kind of NoSQL NoSQL database is useful for unstructured data to store a large
database, and MariaDB, which is a popular relational database. quantity of data. In fact, when we need to collect structured
We designed a set of experiments with a large amount of data data and to store a large amount of data, NoSQL database may
and compared the efficiency of the insert, update, delete and be good enough when not compare to the relational database
select transactions from various aspects on the same dataset. We because the relational database can collect data in the form of
measure the processing time of each transaction to evaluate the a table and can store data that is structured as well.
comparison. The results have shown that Redis has better
runtime performance for insert, delete, update transaction under Also, NoSQL databases provide high flexibility. They do
a specific condition or complex queries. MariaDB still is good for not have a fixed database schema. They can store information
some conditions especially when we have a small data. Our study in a variety of formats. We choose Redis as representatives of
can help to choose a database that will be suitable for the real NoSQL databases because it is a robust open source database
world applications because relational databases and NoSQL with a growing community developing and supporting it.
databases have different strengths and weakness.
In this paper, we aim to compare the performance between
Keywords—NoSQL database; Redis; Key-value database, Big
Redis as NoSQL database which is the key-value type and
data applications, MariaDB. MariaDB as a relational database. We investigate which
situation that one is better and measure the performance from
processing time. We experiment on insert operation, delete
I. INTRODUCTION operation, update operation and select operation in the same
In the past, many applications widely used relational dataset.
databases to collect information. The relational database
The paper is organized as follows. In section 2, we discuss
resembled the basic structure in the form of a table which
related work.We show our experimental setup in section 3 and
contained a row and column. It could be ordered through the
describe the results of our experiments in section 4. We
SQL query language to insert, delete, update, and query data.
describe the conclusion and future work in section 5.
By the way, a technology that can scale, perform and deliver
continuous availability needs for supporting online
applications. Thus, relational databases have struggled to keep II. RELATED WORKS
up with the wave of modernization. In recent year, there are many researchers offered the
The most viable database option for online web and mobile comparison between a non-relational database and relational
applications have to support the amount of data that increases database because they would like to find the suitable database
tremendously as well as the type of data with more variety for their work and there are many NoSQL databases that have
such as unstructured data, structured data, and semi-structured advantages and disadvantages. Many papers compare the
data. Moreover, it is necessary to have a database system that efficiency between MongoDB and a relational database. Such
978-1-5090-4666-9/17/$31.00 ©2017 IEEE
5th International Electrical Engineering Congress, Pattaya, Thailand, 8-10 March 2017

as [1],[2] compared the effectiveness between MongoDB and representing the employee, dept_manager, department, salary,
SQL server. In [1] experimented in 3 operations that consisted title, and dept_emp. The data of the employee table contains
of the insert, update, select operation. The result of running an 300,024 records. The dept_manager table contains 24 records.
experiment showed that MongoDB had better runtime The data of department table includes 9 records. The salary
performance for insert, update and simple query. SQL server table contains 2,844,047 records. The title table and the
performed better when updating and querying by non-key dept_emp table contains 443,308 records and 331,603 records,
attributes, as well as for aggregate queries. In [2] respectively.
experimented in 5 operations that consisted of the insert,
update, delete, select and general queries. They tested on the B. Experimental Design
same dataset for a common e-commerce schema. The results We use PHP language to develop the experiments that run
showed that MongoDB performed better than SQL server for under four scales of data as 100, 1,000, 10,000, 100,000
most operations except aggregate function queries. Moreover, records. We divided into four operations.
some research, In [3] compared MongoDB with MySQL. In 1) Insert Operation: To add data, we used Salary table in
[3] divided the experiment into four operations that consisted this operation. We split into two styles of inserting. First, we
of the insert, update, delete and select operation. They add one by one command to the databases, called single write.
experimented in forum application. The results of the Second, we combine all of the insert commands to execute at
experiment showed MongoDB had better performance than once, called multiple inserts. In Redis database, we used the
MySQL. In [4] compared performance of querying. They pipelining feature as multiple inserts. The pipelining feature is
divided querying into two categories that consisted of the the way that we can send multiple commands to execute at
simple query and complex query. Their results showed once without waiting for the server replies. Therefore,
MongoDB had better performance than Oracle in the simple Pipelining feature is very necessary for Redis to evaluate the
query, but Oracle had better performance than MongoDB in a performance of insertion operation.
complex query. In [5] experimented to compare the
2) Delete Operation: We separate into two experiments.
performance of graph database (Neo4J) and relational
First, we delete data with defined scales from salary table at
database (MySQL). Their results showed that Neo4J was
one time. Second, we delete data by considering the specific
faster than MySQL in querying the results. Many researchers
use MongoDB to be representative of NoSQL database conditions.
because it established longer than other databases. However, it 3) Update Operation: We separate into two experiments
can show only that MongoDB, one kind of document as update salary table with four scales of data and update data
databases, is outperformed than a relational database in many with the specific conditions.
case studies. There are still have many NoSQL databases that 4) Select Operation: First, we query data from salary table
are useful. that have the results as four mentioned scales with no
In this paper, we focus on the comparison between Redis conditions. Second, we separate queries into two categories
as NoSQL database and MariaDB as a relational database in which consist of the simple query and complex query. The
the same environment.We do our experiments in four simple query involves selecting data from only one table. The
operations that consist of an insert operation, an update complex query involves selecting data from multiple tables
operation, delete operation and select operation. The several and selecting data in nested queries. The following list
queries performed on Redis database which equivalent queries summarizes querying in the second experiment.
in SQL language have been carried out on MariaDB. We refer
to the method of [6] for uses as guidelines in the coding of our
experiment.

III. EXPERIMENTAL SETUP

For our experimental setup, we installed both Redis and
MariaDB on the same machine that is running an Intel Core
i5-2450M 2.5 GHz with 8 GB of DDR3 memory. This
experiment was developed on Windows operating system and
was coded in PHP framework and using Xampp Server to
simulate a web server. For Redis, we used a library which
named predis for connecting between PHP framework and
Redis database and used the latest version 3.2.100. We used
MariaDB version 10.1.13 in our experiments.

A. Data Model Schema

We download data from sample database with test suite
1.0.6 [7]. The data model schema used for the experiments as
shown in Fig. 1. The data model consisted of 6 tables/objects
Fig. 1. Data model schema of our experiment
5th International Electrical Engineering Congress, Pattaya, Thailand, 8-10 March 2017

• Q1: Read personal data of the employees from the We ran ten times for each test and calculated the average
employee table with gender = “M” (simple query) performance time. In our experiment, we used the PHP
• Q2: Read data from dept_emp table with emp_no = function microtime which is used to measure the duration of
“10021” (simple query) processing.
• Q3: Read data from dept_emp table and dept_name
IV. EXPERIMENTAL RESULTS
table. To find out which department of each
employee. We use join operation on MariaDB query. A. Insert Operation
(complex query)
• Q4: Read data from department table and When inserting the data into both databases with four
dept_manager table. To find out which department of scales of 100, 1,000, 10,000, 100,000 records, respectively.
each employee who is a manager. We use join The results show that Redis is more outperform than MariaDB
operation on MariaDB query. (complex query) for both styles of insertion as shown in Fig. 2.
• Q5: Read data from employee table, dept_manager B. Delete Operation
table and department table. To find out personal data In delete operation, we separate into two experiments.
and departments' name of each employee who is a 1) The first test of delete operation
manager. We use join operation on MariaDB query. We execute the first test of a delete operation in both
(complex query)
databases with four scales of data. Thus, the results in Fig.
• Q6: Read data from employee table and Three show that Redis is faster than MariaDB in this case.
dept_manager table. To find out the personal data of
employees who is a manager in dept_no = “d001”.
We use a subquery in MariaDB query. (complex
query)
• Q7: Read data from employee table and salary table.
To find out Personal data of employees whose salary
is less than or equal to 50,000 baht. We use a
subquery in MariaDB query. (complex query)
Fig. 5. The results of the first experiment of the update operation.

Fig. 6. The results of the second experiment of the update operation.

Fig. 2. The results of the insert operation.

Fig. 7. The results of the first experiment of select operation.

Fig. 3. The results of the first experiment of the delete operation.

Fig. 8. The results of the second experiment of select operation.

Fig. 4. The results of the second experiment of the delete operation.
5th International Electrical Engineering Congress, Pattaya, Thailand, 8-10 March 2017

2) The second test of delete operation more than Redis. However, only Q3 in complex query’s
The second test of a delete operation, we remove the data experiment shows that MariaDB is slightly faster than Redis.
by defining the specific conditions. The following list
summarizes the conditions that we used in this analysis. V. CONCLUSION
delete_condition_1: Delete data from salary table with The replacing a relational database with NoSQL database
emp_no = “10004” need to study and find the suitable solution for a particular
delete_condition_2: Delete data from department table application, especially big data applications. Therefore, it is
with dept_no = “d009” necessary to consider feature and every operation before
In Fig. 4, the results show that Redis has better developers make a decision which database should be useful
performance than MariaDB in this experiment. Therefore, we in big data applications. It depends on their requirements.
can conclude that for all styles delete operation; Redis is more From the experimental results of the comparison between
efficiency than MariaDB. Redis, NoSQL database as a key-value database, and
MariaDB, relational database, can show that NoSQL database
C. Update Operation
does not get more performance than the relational database in
In update operation, we separate into two experiments. every operation. Redis has better runtime performance of
1) The first experiment of update operation insert operation, delete operation, update operation especially
We applied update operation in Redis and MariaDB. In updating by using specific conditions, and select operation in
Fig. Five can show that MariaDB is faster than Redis for querying single result of the simple query as well as querying
update operation without any specific conditions even though data of complex query. Because Redis has many advantages
data is very huge. such as flexible, no fixed schema, and easy to use.
2) The second experiment of update operation Furthermore, the data model of MariaDB is more complex.
We modified the data by considering the specific Thus, it decreased querying performance. However, Redis also
conditions and used the following conditions list in our second has disadvantages in regular querying and updating that
experiment. applied in an extensive data. Therefore, we should use Redis
update_condition_1: Update data from department table in the form of a temporary database to enhance the
with dept_no = “d001” performance of database systems.
update_condition_2: Update data from title table with In future works, we plan to enhance by involving several
emp_no = 10277 and from_date = 1985-06-16 NoSQL database systems to improve the performance of an
For the results in Fig. 6, it can show that Redis is application by evaluating Redis and MariaDB as a distributed
outperformed more than MariaDB when we updated with database.
specific conditions. References
D. Select Operation [1] Z. Parker, S. Poe and S.V. Vrbsky, “Comparing NoSQL MongoDB to an
In select operation, we separate into two experiments. SQL DB,” In Proceedings of the 51st ACM Southeast Conference, p. 5.
ACM, 2013.
1) The first experiment of select operation
[2] S.H. Aboutorabi, M. Rezapour, M. Moradi and N. Ghadiri,
We use simple query without any specific conditions on “Performance evaluation of SQL and MongoDB databases for big e-
Redis and MariaDB. In Fig. 7, it can show that MariaDB commerce data,” In Computer Science and Software Engineering
provides lower execution times than Redis. Therefore, we can (CSSE), 2015 International Symposium on, pp. 1-7. IEEE. 2015.
conclude that MariaDB has more performance than Redis [3] C. GyĘrödi, R. GyĘrödi, G. Pecherle and A Olah, “A comparative study:
MongoDB vs. MySQL,” In Engineering of Modern Electric Systems
when we use selection operation even on the larger data. (EMES), 2015 13th International Conference on, pp. 1-6. IEEE, 2015.
2) The second experiment of select operation [4] A. Faraj, B. Rashid and T. Shareef, “Comparative study of relational and
We divide into two categories which consist of the simple non-relations database performances using Oracle and MongoDB
query and complex query. systems,” In International Journal of Computer Engineering and
The performance of the simple query, Q1, indicates that Technology (IJCET) Volume 5, Issue 11, November (2014), pp. 11-22.
MariaDB has better than Redis when we refer to the multiple [5] G. Jaiswal and A.P. Agrawal, “Comparative Analysis of Relational and
Graph Databases,” In IOSR Journal of Engineering (IOSRJEN) e-ISSN:
results from the single table. However, Redis is better than 2250-3021, p-ISSN: 2278-8719 Vol. 3, Issue 8 (August. 2013), ||V2|| PP
MariaDB when we query a single result from the table by 25-27
using Q2. In the case of complex query, join operation, or [6] Josh Sherman, “MySQL and Redis Command Equivalents,” Available
subquery operation is used to query data in MariaDB. When in https://joshtronic.com/2013/07/29/mysql-and-redis-command-
equivalents/.
we run Q4, Q5, Q6, and Q7 on Redis and MariaDB, the results
[7] Canonical Group Ltd., “Sample database with test suite 1.0.6
is shown in Fig. 8. It can demonstrate that the performance of "employees-db-1.0.6,” Available in https://launchpad.net/test-
MariaDB will decrease when we execute the complex query as db/employees-db-1/1.0.6
query data from more than one table and query data by using a
subquery operation. MariaDB spent the time to process query