Addressing Performance Challenges in

Serverless Computing
Erwin van Eyk Alexandru Iosup
Platform9 Systems Inc. Computer Systems
Sunnyvale, California Vrije Universiteit
TU Delft, The Netherlands Amsterdam, The Netherlands
[email protected] [email protected]

ABSTRACT technical challenges. Towards an industry-wide benchmark and in

As a key part of the emerging serverless computing paradigm, collaboration with the SPEC CLOUD RG[2], we provide an analy-
Function-as-a-Service (FaaS) platforms allow users to run arbitrary sis of the challenges and perspectives in this field (section 2). To
functions without being concerned about operational issues. How- address the technical challenge of serverless function composition,
ever, there are several (performance-related) challenges surround- we present our work on a prototype of a serverless workflow man-
ing the state-of-the-art FaaS platforms that can hinder widespread agement system (section 3).
adoption of FaaS, including size-able overheads, unreliable perfor-
mance, and the absence of benchmarks. In this work provide an 2 PERSPECTIVES AND CHALLENGES
overview of our ongoing work towards addressing these challenges. In our prior work [7] we analyzed and proposed four perspectives
Thus far, we have been building a workflow engine, Fission Work- that aim to capture the direction of the serverless field, and in
flows, to provide efficient function composition within the FaaS particular of FaaS architectures. First, the stateless, short-lived na-
model. Furthermore, together with the SPEC CLOUD RG, we have ture of these cloud functions makes FaaS an ideal candidate for
been working on solving more high-level challenges; defining a the dynamic models of hybrid clouds and fog computing. Second,
common terminology and reference architecture, as well as devel- as serverless leads to the near-absence of CAPEX, companies will
oping a industry-wide benchmark of these FaaS platforms. Besides focus on managing the OPEX costs of running applications, rais-
presenting our work so far, we also propose a roadmap towards ing interesting performance engineering challenges, along with a
solving these challenges to invite researchers to join this effort. reevaluation of the existing Dev and Ops roles. Third, we forecast
that for individual cloud functions absolute performance will start
KEYWORDS to matter less, whereas the application-specific performance/cost
Function-as-a-Service, Serverless Computing, Performance evalua- ratio will start to matter more. Finally, FaaS architectures could be-
tion, Workflows come the basis for a programming model that is native to the cloud,
that is, it considers services as naturally as any other programming
1 INTRODUCTION concepts, to be developed, composed, and shared with ease.
Towards these perspectives, the current challenges in this do-
Software-development paradigms are increasingly transitioning
main can be divided into 3 categories[7]:
from monolithic applications to compositions of smaller services
that are more granular and distributed (e.g., through Software Ori- (1) Software engineering challenges have been identified as
ented Architectures and, recently, microservice architectures) [1, 3]. some of the most pressing issues with the serverless para-
Correspondingly, cloud vendors have started offering serverless com- digm, such as developer experience[5]. To overcome these
puting services, hosting on granularly-billed resources the emerg- challenges posed by the FaaS model, advances need to be
ing application services. The notion of Function-as-a-Service (FaaS) made in testing, tooling, functionality, and training and edu-
can be seen as a combination of both developments, allowing users cation.
to provide cloud platforms with arbitrary functions leaving the (2) System (operational) challenges arise due to the highly
operational logic to be managed by the cloud provider. dynamic nature of cloud functions, calling for advances in
This model offers new benefits but also raises new challenges. [7] security, cost predictability, and cloud function lifecycle man-
On a conceptual level, the field lacks consistent terminology, along agement.
with a clear reference architecture, inhibiting objective compari- (3) Performance engineering challenges in this emerging
son of different FaaS platforms. On a more technical level, there paradigm, arising due to the increased delegation of concerns
are various challenges, in domains such as software engineering, to the cloud platform, which include size-able overheads, un-
system operations, and performance engineering, that potentially reliable performance, and new forms of the cost-performance
hinder further adoption of the paradigm. trade-off.
Towards addressing these challenges we provide an overview Leading the effort of the SPEC RG CLOUD group in serverless
of our ongoing work, in which we address both the conceptual as computing, we specifically target the performance engineering
challenges, which we expand upon in [6]: Scheduling policies that
ICT.OPEN 2018, March 2018, Amersfoort, The Netherlands
2018. ACM ISBN 978-x-xxxx-xxxx-x/YY/MM. . . $15.00
specifically target the constraints and trade-offs of FaaS, such as
https://doi.org/10.1145/nnnnnnn.nnnnnnn caching function source code efficiently.[4] There are various of
ICT.OPEN 2018, March 2018, Amersfoort, The Netherlands van Eyk et al.

Within the Fission Workflows prototype, the workflow scheduler

contains scheduling policies that optimize the serverless workflow
execution. One of the implemented scheduling policies aims to
reduce the execution time by booting functions ahead of the ac-
tual invocation, reducing the provisioning delay of new functions.
Additionally, using the data collected in the workflow engine the
scheduler is able to profile the individual functions and thereby
improve its runtime predictions.
We aim to evaluate the performance of this workflow engine on
four aspects, namely fault tolerance, throughput, performance, and
resource consumption. Additionally, experiments will be conducted
to compare the performance of Fission Workflows with alterna-
Figure 1: FaaS Reference Architecture (Work-in-Progress) tive approaches to function composition, including coordinator
ordered from business logic (BL) to operational logic (OL). functions, hosted cloud coordinators, such as AWS Step Functions,
and traditional workflow management systems, such as Luigi or
possibilities to reduce, the currently significant, overhead of FaaS Apache Airflow. Finally, we want to evaluate the effectiveness of
platforms, including reusing deployed functions efficiently. Per- different scheduler policies; their impact on the workflow execution
formance isolation could be improved to obtain a more resource- compared to their effect on the overhead of the WMS.
efficient placement of functions, avoiding a bottleneck of particular
resource dimension on a machine. With the level of introspection 4 CONCLUSION & ROADMAP
that FaaS platforms have in scheduled functions, platforms could The emerging serverless and FaaS models hold good promise for
be improve to better profile the characteristics of functions and future cloud applications, but also raise new (performance-related)
provide better performance predictions. The control of FaaS plat- challenges that can hamper its adoption. In this work, we pre-
forms that have on function life-cycles, also allows for further sented the approaches how we aim to contribute to solving these
exploration of the cost-performance trade-off, where we can make challenges on different levels. Specifically for function composi-
distinctions between high-priority functions and background func- tion and the reduction of FaaS performance overhead, we propose
tions. Finally, the field of serverless computing currently lacks a a workflow-based approach to the orchestration of dependency
systematic, objective benchmark of FaaS platforms, along with a graphs of functions with the Fission Workflows prototype. On a
consistent terminology for the common components. higher level, we address community-wide problems, such as ter-
As part of our ongoing work we are focusing on this last chal- minology and benchmarking with a joint industry and academic
lenge of being able to objectively compare the performance of effort with the SPEC CLOUD RG.
common components in different FaaS platforms. Our analysis of In our envisioned roadmap we continue with this wide approach
multiple FaaS platforms so far are shown in the form of a reference to the various challenges in serverless computing. We see the Fis-
architecture as shown in figure 1. It shows how this stack of abstract sion Workflows prototype as the foundation of further research into
layers that comprise a FaaS platform abstract away the operational function composition, with challenges such as data transfer, hybrid
concerns, like resource management, and autoscaling, the higher cloud deployments, and advanced scheduling policies. Furthermore,
up you get in the stack. with the collaborative SPEC CLOUD RG effort we will continue
to encourage researchers to join the effort of developing an com-
3 FISSION WORKFLOWS prehensive reference architecture and industry-wide benchmark of
Separate from the SPEC RG CLOUD effort, we are specifically look- FaaS platforms.
ing into how to introduce composition in the FaaS model. Function
composition in FaaS should allow the developer to avoid operational REFERENCES
concerns related to function composition, such as communication [1] Nick Gottlieb. 2016. State of the Serverless Community Survey Results. https:
logic and state management. Additionally, as these functions, simi- //serverless.com/blog/state-of-serverless-community/. (Nov 2016).
[2] Cloud Group. 2017. SPEC Research Group’s Cloud chapter. https://research.spec.
lar to current functional programming languages, can get composed org/working-groups/rg-cloud.html. (2017).
into increasingly complex functions, it is vital that the overhead of [3] Eric Jonas, Shivaram Venkataraman, Ion Stoica, and Benjamin Recht. 2017. Occupy
the Cloud: Distributed Computing for the 99%. In SoCC. 1:1. (In print, available as
composition should be minimized. CoRR article http://arxiv.org/abs/1702.04024 ).
For this purpose we developed Fission Workflows1 , which is [4] Edward Oakes, Leon Yang, Kevin Houck, Tyler Harter, Andrea C Arpaci-Dusseau,
a workflow management system (WMS) specifically targeted at and Remzi H Arpaci-Dusseau. 2017. Pipsqueak: Lean Lambdas with Large Li-
braries. In Distributed Computing Systems Workshops (ICDCSW), 2017 IEEE 37th
serverless workflows. Similar to existing WMSs, it provides a re- International Conference on. IEEE, 395–400.
liable, traceable executions of function dependency graphs, trans- [5] Mike Roberts. 2016. Serverless Architectures. https://martinfowler.com/articles/
ferring data between functions where needed. In contrast to tradi- serverless.html. (2016).
[6] E. van Eyk, A. Iosup, C. Abad, Eismann S., and Grohmann J. 2018. A SPEC RG
tional WMSs, it focuses on minimizing the overhead of the WMS Cloud Group’s Vision on the Performance Challenges of FaaS Cloud Architectures
and provides a workflow definition language focused on promoting (submitted). In ICPE.
[7] E. van Eyk, A. Iosup, S. Seif, and M. Thömmes. 2017. The SPEC cloud
composition of functions and workflows. group’s research vision on FaaS and serverless architectures. In WoSC held with
ACM/IFIP/Usenix Middleware.
1 https://github.com/fission/fission-workflows