Advances in Engineering Software 148 (2020) 102878

Contents lists available at ScienceDirect

Advances in Engineering Software

journal homepage: www.elsevier.com/locate/advengsoft

An API gateway design strategy optimized for persistence and coupling T

a,b a,b a,b a,b,c,⁎
Xianyu Zuo , Yuehan Su , Qianqian Wang , Yi Xie
a
Henan Key Laboratory of Big Data Analysis and Processing, Henan University, Kaifeng, 475004, PR China
b
Institute of Data and Knowledge Engineering, School of Computer and Information Engineering, Henan University, Kaifeng, 475004, PR China
c
Key Laboratory of Geospatial Technology for Middle and Lower Yellow River Regions (Henan University), Ministry of Education, Kaifeng, 475004, PR China

ARTICLE INFO ABSTRACT

Keywords: Microservices play a more and more important role in software development nowadays. Almost every pro-
API gateway gramming language has its own microservices development framework. The characteristics of microservices
Microservices make microservices have cross-platform compatibility issues and inconsistent call standards issues in the process
RPC of development and call microservices. The birth of API Gateway alleviates these problems to some extent. For
Coupling degree
small and medium-sized enterprises using today's popular API Gateways, it is difficult for them to get a balance
Persistence
between cost, performance and maintainability. This paper proposes a scheme to optimize the API Gateway.
Firstly, the framework of API Gateway is optimized. Next, the coupling degree of API Gateway is optimized by
reducing the coupling degree of core services and extended functions. In this way, the optimized Gateway can
adapt to the plug-in mode, improve the user experience and reduce development costs. Then, the persistent
design of the configuration information of the API Gateway is carried out, and the read-write optimization is
carried out so that the optimized API Gateway not only has advantages in the configuration persistence, but also
has further improved the I/O performance. Based on the optimized design, this paper implements a cross
platform compatible API Gateway. Then it compares the performance of two popular API Gateway schemes
through performance testing. The test results show that the optimized design of API Gateway achieves a new
balance among cost, performance and maintainability. At the end of this paper, the work of this paper is
summarized, and the next work is projected.

1. Introduction or location services. They may be frequently called by other services, so

more energy is needed to maintain common micro services. If the
As the scale of software application systems gradually increases, number of common microservices increases, the maintenance cost of
multiple independent business logic contained in the software structure managing them one by one will be difficult to estimate.
are intertwined with each other and management is difficult. At the With the further expansion of the scale of software application
same time, because all businesses are using the same database or da- systems, software developers may expand from one department to
tabase platform, some businesses that require large-scale reading and multiple departments, from an enterprise to multi-enterprise joint de-
writing would affect the database reading and writing performance of velopment, and the number of common microservices is also increasing.
other businesses, making the entire software application system very For multiple departments’ development, the classic example is the
vulnerable [4]. cloud service department of Alibaba, which provides virtual machine
The emergence of microservices [1-3] effectively addresses these planning and start-stop service interfaces for other business depart-
issues. To change business logic in software application systems to ments. This is an independent microservice, so the virtual machine
microservices logic (This paper refers to this change as microserviced) start-stop and planning information needs to be written an independent
is to abstract the complicated business logic in the entire software ap- database belonging to this microservice. Start-stop and planning in-
plication system into independent business logic as much as possible. At formation may need to be recorded by calling a log microservice — this
the same time, an independent business logic owns its own database. microservice is most likely developed by another department.
After the microserviced software application system, some common For multi-enterprise joint development, the classic example is that
business logic will inevitably appear (This paper refers to such micro- Baidu's map service is packaged as an independent software develop-
services as common microservices), such as log services, login services, ment kit (SDK), and then deployed to the public network for other

⁎
Corresponding author at: Henan Key Laboratory of Big Data Analysis and Processing, Henan University, Kaifeng, 475004, PR China.
E-mail addresses: [email protected] (Y. Su), [email protected] (Y. Xie).

https://doi.org/10.1016/j.advengsoft.2020.102878
Received 17 April 2020; Received in revised form 1 June 2020; Accepted 1 July 2020
0965-9978/ © 2020 Published by Elsevier Ltd.
X. Zuo, et al. Advances in Engineering Software 148 (2020) 102878

enterprises and units that need to use the map service. makes Nginx a heavyweight ESB to a certain extent. The problem of
A large number of microservices have generated a large number of excessive coupling will lead to increasing development difficulty and
RPC (Remote Procedure Calls) [5] interfaces, and also a variety of higher development costs, and distributed deployment and cluster de-
calling standards. If a developer uses both Alibaba and Baidu's open ployment after development are more difficult. This is the coupling
microservices, then he/she must understand two different calling rules problem facing the API Gateway.
and the business logic he/she has to be adapted separately. In the case Therefore, for companies facing these problems, they need an API
of joint development by multiple departments or enterprise, different Gateway that is easy to expand, easy to maintain, and pre-examined
departments or enterprises are responsible for implementing a large with performance considerations and optimizations, which is the opti-
number of different microservices, the interface interaction costs will mized design of the API Gateway proposed in this paper. This paper first
become increasingly high, and the large number of microservices will optimizes the framework of the API Gateway and adds a database
increase management and maintenance costs. persistence mechanism. This paper decouples plug-ins with filtering
Facing these problems, API Gateway [6-9] is needed to solve this rules from API Gateway core services as much as possible. This method
problem. API Gateway is a fully managed service that can assist hosted alleviates the problem of high degree of coupling of API Gateway
service developers in developing or services that need to be deployed. business logic to a certain extent. Then based on the optimized fra-
After the developer's RPC service is published on the network, the RPC mework and database persistence mechanism, the database that the
calling address is registered on the API Gateway. The caller sends a plug-in depends on is separated according to business. In this way, the
request to the API Gateway uniformly, the API Gateway forwards the performance impact between the plug-in and the plug-in, and between
caller's request to the registered RPC service to implement the RPC the plug-in and the core business of the API Gateway is reduced as
service call, without having to worry about the diversification of the much as possible, thereby plug-ins in plug-in modules and the core
RPC service's call path. Calling standards are also unified — otherwise business of the API Gateway become microservices, and greatly redu-
the API Gateway will not allow to register this microservice. cing the coupling between the various services of the API Gateway. The
The API Gateway has been developed for many years. Large en- addition of the database persistence mechanism and the design to
terprises such as Amazon, Alibaba, Baidu, and Tencent have developed transform various services into microservices have reduced the devel-
their own API Gateways, and used stable versions as commercial pro- opment and maintenance costs of the API Gateway. In this way, the
ducts for other enterprises to purchase. Organizations like Netflix and above problems can be solved. In addition, the performance optimiza-
the Spring team have also developed their own open source API tion design of the database persistence mechanism is also made in this
Gateways for developers. But for some companies, the cost of using a paper, which lets the company using the API Gateway design strategy of
commercial API Gateway is too high. So they often choose to use an this paper consume less energy in performance optimization.
open source API Gateway. However, using open source API Gateways
often requires additional coding and deployment by users. The coding 2. Maximum optimization strategy of API Gateway
of the API Gateway's filter business is relatively complicated and re-
quires maintenance by experienced developers. The company needs to In the process of selecting an API Gateway, an enterprise can choose
consider the performance indicators of the API Gateway while using it, a commercial API Gateway. Such an API Gateway has been enhanced by
and add some additional modules to optimize the performance of the the high-performance cluster of a large enterprise with hundreds of
API Gateway. millions of users and optimized by a professional team. The deployment
Take an example of the issue of development and maintenance and maintenance problems are solved by a professional customer ser-
costs. The API Gateway that combines SpringCloud and Zuul is a low- vice team, but the cost is high, and even need to pay for the number of
cost solution. The Zuul Gateway is very suitable for microservices ar- calls. For a growing SME (small to medium-sized enterprise, a company
chitecture. The deployment method is like deploying a SpringBoot with no more than 500 employees), this option carries the risk of un-
project. It is easy to be configured and has many ready-made filters. It predictable costs.
also supports custom filters. However, API Gateways such as Zuul, often Enterprises can also choose the popular open source API Gateway
require companies to continue part of the development in order to run technology and use their own development team to develop and build it
smoothly. The construction process is almost the same as setting up a themselves. There is no additional cost for using such an API Gateway,
project combining Spring and Maven, with a large workload and but if there is not enough cost to set up a dedicated team to optimize or
complicated steps. In addition, their routing data is basically stored in build a high-performance cluster, the performance of the built API
the configuration file. It is almost impossible to directly manage the Gateway will often be much lower than the commercial API Gateway.
background data through the background page. If you want to make the For example, the API Gateway combined with Springcloud and Zuul has
configuration and routing information of the API Gateway more con- more complicated configuration and higher deployment and main-
venient for persistent and anti-persistent management, companies often tenance costs. Another example is the API Gateway built on Nginx. The
need to design and implement persistent strategies themselves. For configuration file may be coupled with the logic of service invocation.
``reverse discovery'' API Gateways such as combining SpringCloud and There may be a chain reaction when modifying the filtering rules. This
Zuul (microservices send heartbeats to the Gateway). Once the address also increases the cost of deployment and maintenance in disguise.
of the Gateway changes, not only the consumer needs to modify the In order to better reflect the relationship between these issues, this
configuration, but the microservice needs to modify the configuration paper proposes a new API Gateway design model called the cost-per-
one by one. For a company using an open source API Gateway, the formance-maintainability model of the API Gateway. There are three
above issues can lead to high deployment and maintenance costs. basic attributes in the model.
In order to avoid excessive development costs, some companies will
choose to use Nginx to deploy their own API Gateway. Nginx is a high- • Low-cost index, indicating the degree to which enterprise needs to
performance proxy server that supports external services as an HTTP pay extra in the process of using the API Gateway;
proxy service. The load balancing strategy is simple and the structure is • Maintainable index, indicating the convenience when setting up the
modular. It is different from Zuul. Nginx is easier to deploy or configure API Gateway and the maintenance effort consumed in the process of
than Zuul. The configuration process is all in the configuration file, and using the API Gateway;
the filter components like Zuul are further encapsulated, which can be • The performance index, indicating the length of response time in the
called directly through the configuration file or script. However, as an process of using the API Gateway when facing a large number of
intermediate layer, Nginx couples the logic of service invocation in the microservice consumer requests.
configuration file, which weakens the integrity of microservices and

2
X. Zuo, et al. Advances in Engineering Software 148 (2020) 102878

Fig. 1. Cost-performance-maintainability model diagram.

This paper matches the two choices of the company's API Gateway increases the development cost, and causes a problem of high devel-
with the model. As shown in Fig. 1, the closer the corner of the inner opment cost. If some of the information in the configuration file can be
triangle is to the corner of the outer triangle, the higher the corre- shared with the filter, in the process of developing the filter, there is no
sponding degree to this angle of the outer triangle, and the better the need to worry about message synchronization and special development
effect of the degree. work for reading the information in the configuration file. In addition,
As shown in Fig. 1(a), when an enterprise chooses a commercial API it is more difficult to modify the configuration by the configuration file
Gateway, the deployment and maintenance issues are solved by a method adopted by the API Gateway combining Springcloud and Zuul,
professional customer service team, which means that enterprises using which also causes the problem of high development costs.
commercial API Gateways rarely need to deal with deployment and For the filter of the API Gateway combining Springcloud and Zuul, it
maintenance issues, so the maintainability degree is high. Since the does not have the complete characteristics of microservices, and has a
commercial API Gateway has been optimized by the high-performance high degree of coupling with the core business of the Gateway. After
cluster of a large enterprise with hundreds of millions of users and modifying the filter, the API Gateway system needs to be restarted,
optimized by a professional team [10,11], the performance degree is which causes inconvenience to the cluster deployment and thus the
also very high. However, since companies using commercial API problem of high maintenance cost.
Gateways need to pay high fees, the low cost degree is very low. If the filter can be combined with the database design that has been
As shown in Fig. 1(b), when an enterprise chooses an open source added, the filter can be rewritten as an independent microservice,
API Gateway technology to build an API Gateway by itself, the de- which will reduce the coupling between the filter and the core business
ployment and maintenance need to be performed by the enterprise it- of the API Gateway. If the optimization is performed in the above
self, so the maintainability degree is low. Because the enterprise may manner, the business of modifying a filter only needs to restart the
not set up a high-performance cluster and a professional API Gateway modified microservice instead of restarting the entire system which can
development team, the performance optimization during the con- reduce maintenance costs.
struction of the API Gateway is limited and the performance degree is This article optimizes the framework of the API Gateway based on
low. But because the API Gateway is not required to pay additional fees, the above ideas. The optimized API Gateway (OAPIG) framework is
the low cost degree is high. shown in Fig. 3.
If we can optimize the deployment and maintenance issues, reduce
the coupling degree of each module of the API Gateway, and then im-
3.2. Optimized API gateway framework includes that
prove the performance bottlenecks, we can form a model as shown in
Fig. 1(c), which is the global idea of the API Gateway optimization
design proposed in this paper. Next, the API Gateway will be optimized • API-Gateway Core. API-Gateway Core is the core business of API
Gateway. The core business of the optimized API Gateway is highly
and designed based on the ideas of this model.
streamlined.
• API-Gateway Database. API-Gateway Database is a persistent busi-
3. Optimization of API gateway ness module of API Gateway. This module replaces the configuration
file and related reading services.
3.1. Optimized API gateway framework • Plug-in Module. Plug-in is an extended business module of the API
Gateway. This module replaces the filter part.
Facing the problems of high development cost and high main- • API-Gateway Manager. API-Gateway Manager is a preset visual
tenance cost, the first optimization measure is to optimize the frame- management module plug-in for API Gateway.
work of the API Gateway itself.
The framework of a classic API Gateway consists of a core business About API-Gateway Core, the separated Core module only focuses
module, a filter module, and a configuration module. The core business on the forwarding of consumer requests and the return of the return
module is responsible for the forwarding service of the API Gateway, value of the producer (microservice). The business logic is highly in-
and the filter module is responsible for filtering illegal requests, such as dependent, achieving the characteristics of microservice business logic,
overload request distribution (load balancing filter), overload intercept which greatly reduces the coupling between the core business of the API
(circuit breakers), unauthorized access (authentication control filter), Gateway and other parts. The low coupling design provides great
etc. Some API Gateways need to implement these filters by themselves, convenience for plug-in expansion and cluster deployment. The front
and some API Gateways have some similar filters built in Fig. 2 is the and rear of the core module's business logic are respectively Pre
API Gateway that combines Springcloud and Zuul frame. Component module and Post Component module, which are used to
Zuul supports some filters natively, but also supports custom filters. cooperate with Plug-in to implement filtering business and more plug-in
It can be seen from Fig. 2 that the core business of the API Gateway that services.
combines Springcloud and Zuul mainly deals with configuration files About API-Gateway Database, this module replaces the configura-
and accepts requests filtered by the filter. The filter and the core busi- tion file and related reading services of traditional API Gateway like the
ness of the Gateway do not share configuration data. This increases the API Gateway combining Springcloud and Zuul, then realizes the data-
coding difficulty for developers in the process of filter development, base persistence of configuration information. The routing table and

3
X. Zuo, et al. Advances in Engineering Software 148 (2020) 102878

Fig. 2. API Gateway framework combining Springcloud and Zuul.

most configuration information of the optimized API Gateway are database reading-writing interface, and enterprises can also manually
persisted by the database. At the same time, the persistent business open or close a microservice call interface.
logic caches some commonly used data based on memory, which makes By adding a database persistence mechanism, the above framework
the data reliability and reading and writing performance better than separates the plug-in with filtering rules from the core business of the
those of the current open source API Gateway configuration file mode. API Gateway, reduces the coupling between the two, and lays the
About Plug-in, This module replaces the filter part of traditional API foundation for further coupling optimization described in Section 2.3.
Gateway like the API Gateway combining Springcloud and Zuul, and
further microservices the filter and combines it with the database 3.3. Optimization of coupling degree of API gateway business
business to form an independent plug-in, which reduces the main-
tenance cost when the plug-in business is modified. When using the API Based on the framework optimization of the first step, this paper
Gateway, enterprises can choose the pre-extension (plug-in before mi- further optimizes the coupling degree between the API Gateway core
croservices call) and post-extension (plug-in after microservices call) of business and plug-ins, in order to effectively reduce development costs
the plug-in by calling the open database reading-writing interface, so and maintenance costs.
that the new plug-in is constantly added to Plug-in. For example, the The main steps of coupling degree optimization are as follows:
start-stop status field, load number field, and cluster number field of a
microservice exist in the database. Through these fields, plug-ins for 1) . In order to reduce the coupling between the API Gateway core
judging microservice status or for load balancing can be implemented. business and plug-ins, the ``routing table'' and related sub tables in
If enterprises choose to further expand the database fields to a sub-table the database of the API gateway are separated.
in the original database table, a plug-in can be used as an independent 2) . In order to reduce the coupling between plug-ins, the database of
microservice. API Gateway can call this microservice before or after the each plug-in is independent, which also gives the characteristics of
logic of API-Gateway Core. Such business expansion plug-in modules microservice to each plug-in.
are completely microserviced, with extremely low coupling and en- 3) . A kind of plug-in design is added, which is responsible for syn-
hanced usability[12]. This paper will describe this method in details in chronizing core database and plug-in database to enhance the ro-
Section 2.3. bustness of Plug-in Module and prevent plug-in chain failure caused
About API-Gateway Manager, it will show the status of the micro- by a plug-in downtime.
service registered by the current API Gateway by calling the open

Fig. 3. Optimized API Gateway framework.

4
X. Zuo, et al. Advances in Engineering Software 148 (2020) 102878

All requests of microservice consumers rely on the core business of • Plug-ins that do not need to read the data of the database used by
the API Gateway. If only a certain plug-in or filtering rule is modified, the core business of the API Gateway.
all programs including the core business of the API Gateway need to be • Plug-ins that need to read the database used by API Gateway core
restarted, which will greatly increase maintenance costs. It will also business.
cause losses to consumers of microservices, so this paper needs to op-
timize the coupling degree of the core business of the API Gateway to For plug-ins that do not need to read the database data used by the
ensure that the core business of the API Gateway is not affected by other core services of the API Gateway, such as load balancing plug-ins, it
services like plug-in. only needs to obtain the load status of each microservice. If no new
After the database persistence mechanism is introduced, the in- microservice is registered with the API Gateway, it does not need to
formation in the configuration file is converted into information in the care about the data of the “routing table” database (Core Database in
database table, and this information is divided into different tables Fig. 4) used by the core services of the API Gateway. This means that
according to the attributes. The core table is called the “routing table”. the database tables that this category of plug-in depends on can also be
This table is similar to the router's routing table and stores the corre- separated into an independent database like the API Gateway core
spondence between forwarding paths and objects. For the core services business, such as a database that stores the load status of each micro-
of the API Gateway, the core service only cares about the Gateway service. This plug-in is like a microservice, so the coupling between this
routing table. When the microservice producer registers the necessary category of plug-in and the API Gateway core business and plug-in can
information such as the microservice invocation path and parameters be reduced.
with the API Gateway, the corresponding routing table information is For plug-ins that need to read the database used by the API Gateway
persisted in the database, and the core service only reads the in- core business, because the database used by the API Gateway core
formation of this table. For a ``router'' like API Gateway, only the business is independent, all plug-ins that want to obtain information
“routing table” can complete its basic job [13]. At this time, the core about newly registered microservices must read the “routing table”
business of the API Gateway only depends on the information of a table. database (Core Database in Fig. 4). But it is impossible for all the plug-
This paper can also separate this table into a proprietary database. This ins to read data from the database used by the core business of the API
database only serves the core services of the API Gateway to further Gateway, because this is contrary to the low coupling design. The
reduce the performance impact of other services when reading and purpose of this category of plug-in is to avoid too many unrelated plug-
writing data. ins from reading the core database. Such plug-ins will synchronize some
If the above design is adopted, then the core services of the API newly added necessary data to the independent database of other plug-
Gateway at this time already have the characteristics of microservices, ins. Although this category of plug-in will partially depend on the
which means that it is independent into a microservice that is specifi- business of the first category of plug-in, this category of plug-in does not
cally responsible for request forwarding. The coupling degree between have filtering rules, so even if this category of plug-in fails, it will only
the core services of the API Gateway and other services is extremely affect the unsynchronized plug-ins information and have no effect to
low, because the above design gives the characteristics of microservices existing plug-ins, which will not cause a plug-in failure to cause sub-
to the API Gateway core business. At this time, the coupling degree of sequent dependency plug-in chain failures. For example, during the
the core services of the API Gateway is optimized. operation of API gateway, the database of plug-in A(plug-in that do not
After separating the database associated with the API Gateway core need to read the database data used by the core services of the API
business from the original database, the framework is shown in Fig. 4. Gateway) needs to receive the basic data read by plug-in B(plug-in that
Just optimizing the coupling degree of the core services of the API need to read the database used by the API Gateway core business) from
Gateway cannot completely solve the coupling degree problem, and it is the core database. Suddenly, plug-in B is down due to some failures. At
also very important to reduce the business coupling between plug-ins. If this time, the update of basic data required by plug-in A stops at the
the business coupling between plug-ins is too high, then a plug-in time when plug-in B is down. Then, the information of a new micro-
failure will cause a series of filtering rules to fail. Encountering such service is registered in the core database. In the above scene, only plug-
problems during the deployment process will increase maintenance in A is affected. Plug-in A is affected by its inability to operate on newly
costs. Encountering such problems in an actual production environment registered microservices after plug-in B goes down, but has no impact
will have a serious impact on the producer microservices registered in on other plug-ins, and will not cause the plug-in chain failure. In ad-
the API Gateway. dition, for the core business of the API Gateway, this type of plug-in
The plug-ins in the OAPIG can be divided into two categories: only reads the “routing table” database (Core Database in Fig. 4) and

Fig. 4. Optimized API Gateway framework (Separated core business database).

5
X. Zuo, et al. Advances in Engineering Software 148 (2020) 102878

Fig. 5. Optimized API Gateway framework (After further coupling optimization).

does not perform other operations, so that the performance impact of from a consumer, the API Gateway core service will necessarily query
the plug-in on the database of the core business of the API Gateway is Core Database based on the information sent by the consumer to find
minimized. the basic information (call path, parameters, etc.) of the microservice
The above design makes the API Gateway core business separated that the consumer needs to consume. If the core business directly reads
from the plug-in as much as possible through database independence, Core Database by calling the database interface, the efficiency is lower
thereby reducing the coupling and performance impact of the core than that of reading from the memory, which is similar to the principle
business and plug-in business. The data table of the first category of of the in-memory database[14].
plug-in is then separated from the original database to form a separate In order to avoid the impact of database reading and writing per-
database, and the first category of plug-in is decomposed into micro- formance bottlenecks, this paper adds an active memory buffer me-
services, which further reduces the impact between plug-ins’ business. chanism between the database and the business module to improve the
The second type of plug-in is responsible for data synchronization, and database reading and writing performance. This article uses a hash
only reads the core business database to reduce the performance im- table as the data structure for the buffered data in the active memory
pact. Finally, the coupling degree optimization of the API Gateway is buffer mechanism. The core of the hash table is a hash function. The
completed. The optimized API Gateway framework is shown in Fig. 5. hash function can convert any input to a compression map with a
After the above-mentioned optimization of the coupling degree, specified length of output, and different inputs may have the same hash
databases originally shared by all plug-ins are separated one by one, so value. Therefore, it is not possible to directly determine the corre-
that one plug-in does not affect the database performance of other plug- spondence between input and output without knowing the hash func-
ins when performing database reading-writing operations. By adding tion. Because of this feature, the hash table has properties such as se-
plug-ins that specifically synchronizes the API Gateway core business curity, integrity, and non-repudiation [15]. In this way, even if the
database and plug-in database data, the problem of data synchroniza- database's encryption protection is lost, the database data can be safely
tion after the database independence is solved, and the database cou- buffered into memory.
pling between the plug-in with filtering rules and the API Gateway core Taking JAVA as an example, a Hash-Map (a key-value hash table
business is minimized, so that it reduces performance impact between provided by JAVA) can be stored directly in the JAVA virtual machine
plug-ins and core business. (JVM), and the data of the JVM is stored in local memory (For the
It can be found that a framework with loose coupling between core purposes of this paper, it is stored on the server where the API Gateway
business and plug-ins, loose coupling between plug-ins and plug-ins, is deployed), the read speed is faster than that of reading a remote
little interference between core business databases and plug-in data- memory database (such as Redis). In this paper, when implementing the
bases, and little interference between plug-in databases has been active memory buffer mechanism, JAVA-based Hash-Map implements
formed. The framework shown in Fig. 5 further reduces the perfor- the storage of database buffered data through the Key-Value data
mance impact and business interleaving between the modules than the structure. The advantages of the Key-Value data structure are high
framework shown in Fig. 3. With a comparison between the framework performance, high scalability, and a simple and flexible data model
shown in Fig. 2 and the framework shown in Fig. 5, the degree of [16].
coupling between the modules in the frame shown in Fig. 5 is greatly The strategy designed in this paper is: If the basic information of the
reduced. microservice that needs to be found is stored in memory, related
business will read directly from memory; otherwise the business con-
3.4. Performance Optimization of API Gateway Persistence Business tinue to read from the database to, but after the reading is finished, the
data read from the database by related businesses need to be buffered
After the introduction of the database and persistence business, the into memory and backed up for the next reading directly read from the
basic information such as the microservice address and calling method memory buffer.
in the Core Database in Fig. 5 is the most frequently read data by the As mentioned above, the Hash-Map based Key-Value data structure
API Gateway core business. If this aspect is not further optimized, the is combined with the local cache of the JVM, and through the active
reading and writing performance of OAPIG will become a new perfor- caching strategy, the active memory buffer mechanism which have
mance bottleneck, and the reading and writing performance of the high-performance, high-reliability, and simple data structure is im-
database will limit the normal distribution of forwarded requests by the plemented.
API Gateway core business. Next, compare the call delay before adding the active memory
For example, after the API Gateway core business obtains a request buffer and the call delay after adding the active memory buffer through

6
X. Zuo, et al. Advances in Engineering Software 148 (2020) 102878

Table 1 Table 2
Experimental environment. Performance improvement ratio of database mode and database mode with
memory buffer.
Software/hardware Model/version
Number of threads Total call duration Total call duration R
CPU Intel Core-i7 7700 (One thread one call) (Has Memory Buffer) (None Memory Buffer)
RAM DDR4-2400MHz 24GB
Disk Samsung 860EVO 1TB 100 374.53 265.25 1.41
DataBase Sqlite Version 3.0 500 639.39 357.14 1.79
JDK Version 11 1000 840.34 405.02 2.07
Springboot Version 2.2.0 5000 1,013.58 490.20 2.07
Springcloud Version 2.2.0 10000 1,179.66 486.45 2.43
Zuul(for springcloud) Version 2.2.0 50000 1,618.18 528.37 3.06
Eureka(for springcloud) Version 2.2.0 100000 1,690.53 556.96 3.04
Nginx Version 1.16.1
Network card Killer E2500 Gigabit Ethernet

the experimental results to obtain the response rate improvement ratio.

Use formula (2) to calculate the response rate improvement ratio.
experiments. The programming language used in the experiment is
JAVA, the RPC call engine library is HPROSE, the database is SQLITE, TPS1 TPS0
R=1+
and the memory buffer is the public member variables of the JAVA TPS0 (2)
interface to simulate the implementation. The experimental environ-
ment used in this section is shown in Table 1. Where R is the response rate improvement ratio, TPS0 is the re-
Seven groups of experiments were performed separately. The 1st sponse rate of OAPIG without active memory buffering, and TPS1 is the
group included 100 threads of continuous calls, the 2nd group included response rate of OAPIG with active memory buffering.
500 threads of continuous calls, the 3rd group included 1000 threads of Since the data recorded in Fig. 6 is already an average value cal-
continuous calls, the 4th group included 5000 threads of continuous culated based on the results of multiple experiments, the average value
calls, and the 5th group included 10000 threads of continuous calls, the is no longer taken in the calculation of formula (2). The calculation and
6th group included 50,000 threads of continuous calls, and the 7th analysis results are shown in Table 2.
group included 100,000 threads of continuous calls. Experiments are From the experimental results and calculations, it can be seen that
performed on the API Gateway without the active memory buffer me- compared to the API Gateway without the active memory buffer me-
chanism (database mode) and the API Gateway with the active memory chanism (database mode), the API Gateway with the active memory
buffer mechanism (database mode with memory buffer), and call once buffer mechanism (database mode with memory buffer) has improved
per thread. Start the API Gateway without buffering before calling. the response rate by at least 1.41 times and up to 3.06 times.
Each group of calls is performed 20 times and the average total call
duration is calculated. 3.5. Summary of optimization works
Transaction Per Second (TPS) is an important indicator for mea-
suring the concurrent performance of software systems (the higher the After the optimization work in this section, compared with the API
better)[20]. Gateway that combines Springcloud and Zuul and the Nginx-based API
Gateway, the maintainability and performance of OAPIG are improved.
n
TPS =
(1) The comparison of elements in maintainability is shown in Table 3.
t
Some of the same elements of the three did not appear in Table 3.
Where n is the number of concurrent transactions, t represents the The element of persistent configuration based on Database indicates
complete time from the start of the request to the response of all con- whether API gateway has database persistence design. The API Gateway
current transactions in a single experiment. TPS can be considered as that combines Springcloud and Zuul and the Nginx-based API Gateway
the response rate of the API Gateway. do not consider database persistence design, but OAPIG adds database
The experimental statistical results are shown in Fig. 6. persistence design, making the process of configuration and main-
Further calculation and analysis are conducted in combination with tenance more convenient.

Fig. 6. Comparison of response rates between database mode and database mode with memory buffer.

7
X. Zuo, et al. Advances in Engineering Software 148 (2020) 102878

Table 3
The comparison of elements in maintainability.
Elements Springcloud and Zuul Based on Nginx OAPIG

Persistent configuration based on Database - - S

Design of low coupling plug-in S - S
Design of Highly available plug-in - - S
The active memory buffer mechanism - - S
Compatible with Spring technology stack S D S

S: The element feature is supported. D: Additional development work required.

Table 4 structure as the body of the message. JSON is a lightweight data ex-
OAPIG client-to-microservice server data structure. change format. It is based on a subset of ECMAScript (the JavaScript
KEY VALUE specification made by the European Computer Association) and uses a
text format that is completely independent of the programming lan-
Request Source The address of the request source (string) guage to store and represent data. The concise and clear hierarchy
Token Authentication flag (string) to which the request source
makes JSON an ideal data exchange language. It is easy for humans to
was assigned
Request Parameters All input parameters required by the microservice
read and write, it is also easy for machines to parse and generate, and
Service Name Service name registered with the OAPIG effectively improves network transmission efficiency[18].
In the core business part of the API Gateway, HPROSE is introduced
as the RPC call engine library. HPROSE will automatically encapsulate
The element of design of low coupling plug-in indicates whether the JSON datagrams into HTTP datagrams. In this way, microservice con-
API gateway considers microservice plug-ins. Nginx couples the logic of sumers only need to provide a JSON containing the microservice name,
service invocation in the configuration file, which weakens the integrity parameter sequence, and description information. The key-value
of microservices and makes Nginx a heavyweight ESB to a certain ex- structure of the JSON object is easy to understand, simple to assemble,
tent. So the Nginx-based API Gateway does not have this element. and only needs to call an interface when using HPROSE.
The element of design of Highly available plug-in indicates whether Because the optimized API Gateway already has the characteristics
the API gateway has high availability design for failure of plug-ins. As of microservices, this paper relies on the Springboot framework[19] to
described in Section 3.2, OAPIG has this mechanism. implement it and deploy it based on the Springboot deployment
The element of the active memory buffer mechanism indicates method.
whether the API gateway has special memory optimization for the The implementation process is as following.
persistent business. As described in Section 3.3, OAPIG has this me- Step 1: the definition of the data structure. Here JSON is used for
chanism. data communication. During the communication process of the opti-
The element of compatible with Spring technology stack indicates mized API Gateway, the datagram from the client to the microservice
whether the API gateway supports the framework under the spring contains the authentication code (token), the request source, the re-
technology stack. As described in Section 3.3, OAPIG has this me- quested service name and input parameters sequence. As shown in
chanism. Table 4.
Nginx requires additional development of compatible programs to During the communication process of the optimized API Gateway,
support the Spring technology stack. Zuul supports Spring technology the datagram from the microservice to the client contains the request
stack, and the design of OAPIG can also be compatible with spring source, the microservice return code (for quickly identifying the status
technology stack, this paper will introduce the implementation of of this call), the microservice return message (used to describe the
OAPIG based on Spring technology stack in Section 4.1. status of this call in more detail) and the return value sequence. As
shown in Table 5.
4. Testing procedure The information of the service name and the input parameters re-
quired by the microservice are registered on the API Gateway in ad-
4.1. Implementation of the optimized API gateway vance by the developer of the microservice or the enterprise using the
API Gateway. After the caller views the information, the caller can
The first step for a microservice consumer to access the API Gateway assemble the input sequence of the microservice according to the in-
is to send an access request. This request generally contains the name of formation format displayed by the API-Gateway Manager.
the requested microservice, the input parameter sequence, and other Step 2: implementation of API Gateway services. HPROSE is used as
descriptive information. The API Gateway that combines Springcloud OAPIG's RPC call engine library. The programming language used is
and Zuul and the Nginx-based API Gateways all accept HTTP data- JAVA. The database is SQLITE. The database connection pool is JDBC
grams. The datagram messages need to be assembled by microservice provided by JAVA. The focus here is on the API Gateway core service
consumers themselves. Such messages are not easy to understand or development process. The first is to complete the encoding of the for-
parse. warding part according to the HPROSE document standard; then add
Therefore, this article redefines the data structure of the message, the parameter analysis encoding before forwarding the business to
and takes the JavaScript Object Notation (JSON) [17] key-value data determine the name of the requested microservice, and go to the

Table 5
OAPIG microservice-to-client server data structure.
KEY VALUE

Code Return code attached to the result which contains a simple state of the call
Message Detailed call status descriptions and remarks for returned results
Return Values Result information calculated based on parameters sent from the client
Request Source Client's address

8
X. Zuo, et al. Advances in Engineering Software 148 (2020) 102878

Fig. 7. Comparison of response rates between OAPIG, SCZAPIG and NAPIG.

database and memory buffer to find the address. If an enterprise using performance optimization of the persistent service, this paper imple-
OAPIG needs to add a plug-in or filter, it should call the plug-in's mi- ments and deploys an API Gateway according to the optimization idea.
croservice interface here; finally, after forwarding the business, add In order to prove that the coupling degree optimization of the API
logic that wraps the microservice return value and returns the return Gateway will not affect the performance of the API Gateway and prove
value to the request source. the effectiveness of the performance improvement of the persistence
The pseudocode of OAPIG core business is: service combined with other business of modules in the framework after
performance optimization, this paper conducted a performance test.
The experimental environment used in this section is shown in
1 public JSONObject apiGateway(String serverName,Object[] args) {
Table 1.
2 //pre-components
3 Pre_plugin1(serverName, args); The test comparison object is the API Gateway combined with
4 Pre_plugin2(serverName, args); Springcloud and Zuul (hereinafter abbreviated as SCZAPIG), and the
5 //You can insert more plugins(pre-components) at here API Gateway (hereinafter abbreviated as NAPIG) built on Nginx
6 //get a micro-service's real IP-address (hereinafter abbreviated as NAPIG); The microservices registered on the
7 if(AddressCache.has(serverName)) {
8 //try to get address from RAM
API Gateway for testing only return the input parameters and have no
9 serverAddress=AddressCache.getAddress (serverName); computing services to avoid the impact of the computing business of the
10 }else { microservices on the test. The source of access to microservices, the API
11 //get address from Database Gateway, and the program deployment environment of the micro-
12 serverAddress=serverFind.getAddressfromDB(serverName);
services are same. The three API Gateways did not add any filtering
13 Addresscache.put(serverName, serverAddress);
14 } rules with computing business to avoid the impact of the filtering rules
15 //send request message to a micro-service and get return-value on the performance.
16 JSONObject content=SendbyHprose(serverAddress, serverName, Seven groups of experiments were performed separately. The 1st
args); group included 2000 threads full calling, the 2nd group included 4000
17 //post-components
18 Post_plugin3(serverName, args, content);
threads full calling, the 3rd group included 8000 threads full calling,
19 Post_plugin4(serverName, args, content); the 4th group included 16000 threads full calling, the 5th group in-
20 //You can insert more plugins(post-components) at here cluded 32000 threads full calling, the 6th group included 64,000
21 return content; threads full calling, and the 7th group included 128,000 threads full
22 }
calling, the microservice registered on the Gateway are called once per
thread. After obtaining the test results, calculation was made by for-
Step 3: deployment. The API Gateway is deployed as an independent mula (1).
microservice in the SpringBoot framework. The producer microservices
are all independent microservices and deployed in the SpringBoot fra-
5. Results and discussion
mework. The client simulates HTTP calls through JAVA and acts as a
separate Java Archive package, which is started during testing.
5.1. The experimental results are shown in Fig. 7

4.2. Test With the experimental results shown in Fig. 7 for further calculation
and analysis, the response rate improvement ratio is obtained by using
After optimizing the coupling degree of the API Gateway and the formula (2). Where is the response rate improvement ratio, r_0

9
X. Zuo, et al. Advances in Engineering Software 148 (2020) 102878

Table 6 possibility for the computer to process more efficiently. In addition, in

Calculation and analysis results of the response rate ratio of OAPIG and the process of implementation, OAPIG uses HPROSE as the RPC call
SCZAPIG. engine, and HPROSE optimizes the multi-threaded request transmis-
Number of threads Response rate Response rate R sion, which improves the communication efficiency between the client
(One thread one call) (Springcloud and Zuul) (Optimized API and OAPIG.
Gateway)

2000 556.63 1739.11 3.12

4000 572.96 1904.73 3.32 6. Conclusions
8000 583.51 1913.00 3.27
16000 586.91 1905.20 3.25 This paper proposes a new optimization design model for the cou-
32000 589.23 1900.67 3.23
pling degree and persistence service of the API Gateway. Based on the
64000 583.57 1885.57 3.23
128000 582.00 1856.50 3.19
optimization design in this paper, a lightweight and pluggable API
Average 3.23 Gateway is implemented, which uses Springboot as the development
framework and HPROSE as the RPC engine. It has the characteristics of
microservices and has good reading and writing performance.
represents the response rate of the API Gateway combined with Based on the obtained results the following conclusions can be
Springcloud or Zuul or the API Gateway built on Nginx, and r_1 re- drawn
presents the response rate of the API Gateway based on the optimized
API Gateway policy.
Where R is the response rate improvement ratio, TPS0 is the re-
• The API Gateway optimized in this paper can effectively improve
the response rate of the API Gateway and can respond to large-scale
sponse rate of SCZAPIG or NAPIG, and TPS1 is the response rate of data requests.
OAPIG.
Since the data recorded in Fig. 7 is already the average value cal-
• The API Gateway has added the persistence mechanism. And an
active memory buffer design which is added in the optimized API
culated based on the results of multiple experiments, the average value Gateway by this paper significantly improves the performance of the
is no longer taken in the calculation of formula (2). Table 6 shows the API Gateway.
calculation and analysis results of the response rate of SCZAPIG and the
response rate of OAPIG. Table 7 shows the calculation and analysis
• The coupling degree optimization design of the optimized API
Gateway in this paper can promote performance.
results of the response rate of NAPIG and the response rate of OAPIG.
It can be seen from the experimental results that the optimized API
• The core service logic processing process of the implemented API
Gateway is carefully designed to separate the two parts of the fil-
Gateway has added a database persistence mechanism and actively tering strategy and the routing strategy. The persistent database
buffered the database I / O, from which, under high concurrency and replaces the original configuration file, and makes it possible to
continuous access, the efficiency of reading data is higher than the ef- introduce more dynamic configuration information, enabling sub-
ficiency of reading data from the configuration file by the API Gateway sequent expansion to be very convenient and easy to understand,
combined with Springcloud and Zuul. When the data that the API which also makes the strategy of the optimized API Gateway more
Gateway needs to read is hit in the memory buffer, the efficiency will be flexible, easy to manage and deploy.
higher. Compared with the API Gateway combined by Springcloud and
Zuul, the optimized API Gateway has a minimum response rate of 3.12
• The optimized API Gateway strategy is a new API Gateway design
strategy that achieves a good balance between cost, performance
times and a maximum increase of 3.32 times. The average increase is and maintainability, which meets the model shown in Fig. 1(c).
3.23 times.
Compared with the API Gateway built based on Nginx, the opti- Next, the design of the plug-in extension will be further optimized.
mized API Gateway has a minimum response rate improvement of Although the currently optimized API Gateway reduces the workload of
17.84 times, a maximum improvement of 28.71 times, and an average users on plug-in expansion and supports hot plugging of existing plug-
improvement of 21.57 times. ins, when adding new plug-ins, a small amount of modification of the
As mentioned earlier, Nginx's filtering rules affect the coupling de- Pre Component or Post Component of core business shown in Figs. 3-5
gree of business functions, so the optimized API Gateway has not only is still needed. Further optimized design will enable full hot-plugging of
the performance advantages of database persistence combined with the plug-in. In addition, the analysis of the relationship between cou-
active memory buffering, but also the coupling degree. In the state of pling and performance mentioned at the end of Section 5 will be further
high concurrency and continuous calling, this coupling degree ad- studied.
vantage is reflected in performance. Due to the low coupling degree
between the core business of OAPIG and plug-ins of OAPIG, each in-
dependent part has even become a separate thread in the JVM, rather Funding
than multiple businesses coupled in one thread, which provides the
This work is supported by grants from National Basic Research
Table 7 Program of China [grant number 2017YFD0301105], National Natural
Calculation and analysis results of the response rate ratio of OAPIG and NAPIG. Science Foundation of China [grant numbers U1704122, U1604145]
Number of Threads Response rate Response rate R and Key R&D, Promotion Projects of Henan Province [grant numbers
(One thread one call) (Based on Nginx) (Optimized API 182102210242, 182102110065, 192102210096] and Kaifeng science
Gateway) and technology development plan [grant number 2002001].
2000 97.53 1739.11 17.84
4000 101.50 1904.73 18.77
8000 95.61 1913.00 20.01 CRediT authorship contribution statement
16000 93.64 1905.20 20.35
32000 87.42 1900.67 21.75 Xianyu Zuo: Supervision, Writing - review & editing, Formal ana-
64000 79.91 1885.57 23.60
lysis. Yuehan Su: Writing - review & editing, Methodology, Software,
128000 67.00 1856.50 28.71
Average 21.57 Formal analysis. Qianqian Wang: Writing - review & editing. Yi Xie:
Methodology.

10
X. Zuo, et al. Advances in Engineering Software 148 (2020) 102878

Declaration of Competing Interest Microserv Archit Aug 2019;28:149–74. https://doi.org/10.1201/

9780429329920-5.
[9] Encyclopedia of Biometrics. eds Boston, MA: Springer; 2009. https://doi.org/10.
The authors declare that they have no known competing financial 1007/978-0-387-73003-5_858.
interests or personal relationships that could have appeared to influ- [10] Suram S, MacCarty NA, Bryden KM. Engineering design analysis utilizing a cloud
platform. Adv Eng Softw January 2018;115:374–85. https://doi.org/10.1016/j.
ence the work reported in this paper. advengsoft.2017.10.004.
[11] Amazon. Overview of Amazon web services, https://d0.awsstatic.com/
Supplementary materials whitepapers/aws-overview.pdf (Retrieved July, 2019).
[12] Sievi-Korte O, Beecham S, Richardson I. Challenges and recommended practices for
software architecting in global software development. Inf Softw Technol 2018.
Supplementary material associated with this article can be found, in https://doi.org/10.1016/j.infsof.2018.10.008.
the online version, at doi:10.1016/j.advengsoft.2020.102878. [13] S Murthy, JJ Garcia-Luna-Aceves. An efficient routing protocol for wireless net-
works . Mob Netw Appl . 1996 (2), pages. 183–197. 10.1007/BF01193336.
[14] Yun Ji-Tae, Yoon Su-Kyung, Kim Jeong-Geun, Kim Shin-Dug. Effective data pre-
References diction method for in-memory database applications. J Supercomput 2020;76(1).
https://doi.org/10.1007/s11227-019-03050-x.
[1] Schwartz A. Microservices. Informatik-Spektrum 2017(6). https://doi.org/10.1007/ [15] Yang Y, chen F, Zhang X, Yu J, Zhang P. Research on the hash function structures
s00287-017-1078-6. and its application. Wirel Pers Commun 2017;94(4):2969–85. https://doi.org/10.
[2] Thones J. Microservices. IEEE Softw January 2015;32(1). https://doi.org/10.1109/ 1007/s11277-016-3760-4.
MS.2015.11. 116-116. [16] Wen-Long MA, Yu-Qing ZHU, De-Jun JIANG, Jin XIONG, Li-Xin ZHANG, Xiao
[3] Avritzer A, Ferme V, Janes A, Russo B, vanHoorn A, Schulz H, et al. Scalability MENG, et al. A survey on local key-value store of NoSQL system. Chin J Comput
assessment of microservice architecture deployment configurations: a domain-based 2018;41(8):1722–51. https://doi.org/10.11897/SP.J.1016.2018.01722.
approach leveraging operational profiles and load tests. J Syst Softw July [17] Bourhis P, Reutter JL, Vrgoč D. JSON: Data model and query languages. Inf Syst
2020;165:110564https://doi.org/10.1016/j.jss.2020.110564. 2020;89:101478https://doi.org/10.1016/j.is.2019.101478.
[4] Wei W, Lihua Y. RPCI:RPC for internet applications. Comput Eng Appl [18] Gracia J, Bayo E. An effective and user-friendly web application for the colla-
2013;49(21):106–10. https://doi.org/10.3778/j.issn.1002-8331.1202-0082. borative analysis of steel joints. Adv Eng Softw May 2018;119:60–7. https://doi.
[5] Birrell AD, Nelson BJ. Implementing remote procedure calls. ACM Trans Comput org/10.1016/j.advengsoft.2018.02.007.
Syst February 1984;2(1):39–59. https://doi.org/10.1145/2080.357392. [19] Xiong Y. Analysis and research of application development technology based on
[6] Montesi F, Weber J . Circuit breakers, discovery, and API gateways in microservices. spring boot framework. Comput Knowl Technol 2019;15(36):76–7. https://doi.org/
2016. 10.14004/j.cnki.ckt.2019.4290.
[7] Xu R, Jin W, Kim D. Microservice security agent based on API gateway in edge [20] Peng ZHOU, Hai-ying ZHOU, De-cheng ZUO, Tao LI. Spirent-based web application
computing. Sensors (Basel) 2019 Nov 10;19(22):4905. https://doi.org/10.3390/ performance evaluation. Comput Eng 2012;38(24):57–61. https://doi.org/10.
s19224905. 3969/j.issn.1000-3428.2012.24.014.
[8] Surianarayanan C, Ganapathy G, Raj P. Service discovery and API gateways. Essent

11