Original
Original
+MICROSERVICES
The ultimate guide to creating an
enterprise API platform
3 Intro
TABLE
11 Benefits of moving to an API-centric enterprise
OF CONTENTS
24 The emergence of microservices
47 Conclusion
Unlike the many choices that a technical leader must make regarding programming
INTRO
languages, libraries, and infrastructure, APIs have a direct impact on the speed of software
delivery within a business. Therefore, leaders must not leave an organization’s API strategy up
to their developers as they build new APIs. Instead, it requires a thoughtful and well-planned
approach across the entire organization.
This e-book is designed for technical leaders tasked with establishing a new API program
for their organization or maturing an existing program. It will provide insights and decision
factors based on established practices in organizations with successful API programs.
A web API doesn’t have to be RESTful. It doesn’t have to use SOAP. It doesn’t have to use JSON or XML
or OAuth or be built in a specific programming language or framework. It doesn’t have to have pretty
URLs. Web APIs may exhibit some, all, or none of these traits. The only requirement for a web API is that
it allows one program or software component to interact with another in a repeatable way over HTTP.
Request
Response
Diagram of a simple web-based application that retrieves data from a database via an API.
With the emergence of messaging platforms, bots, and voice interfaces, things are dramatically changing.
We are starting to see the focus shift from building applications to delivering capabilities via APIs that are
then integrated into these platforms. Rather than users going to an application to get things done, we are
now experiencing the shift to applications going to the user through these third-party platforms.
APIs as capabilities
NO YES
PARTNER APIs
Partner APIs, such as those offered by Sabre and Capital One, are focused on an organization’s set of
development partners. By limiting an API to partners, companies can tightly control who uses an API
and how it is used outside of their organization.
PLATFORM APIs
Platform APIs extend the reach of SaaS providers like Microsoft Dynamics, Fitbit, and API2Cart by
bringing together multiple parties within a marketplace. By reaching beyond the typical provider-
consumer relationship found in SaaS, platforms often provide increased value through innovation and
a greater overall network.
Organizations can use the same techniques that have led Twilio, Stripe, and other companies to deliver API
products to market. Ones that want to solve problems that matter to business developers will craft APIs
APIs as products that address the desired outcomes of users. They seek to apply product ownership to their APIs, soliciting
feedback from stakeholders and continually improving the API design and documentation to make it easier
for API adoption.
Those that do not treat their APIs as products will produce APIs that are limited to serializing data over
HTTP or providing system-to-system integration. While there is a time and place for this style of API,
organizations won’t experience the economy of scale that API products thinking offers: the ability to reuse
them, increase velocity of delivery, and consistency of customer experience.
Transforming Increasing
partner integrations by improving efficiency revenue directly or indirectly by reducing
and freeing up resources. customer churn through deeper integration.
NETFLIX
When Netflix originally started mailing DVDs in 1997, its competition was Blockbuster and the cable
providers. Over the next few years, Netflix quietly consumed the DVD rental market while preparing to
launch its streaming service in February 2007. Today, it is difficult to find a media device that doesn’t have
Netflix streaming support. This is in no small part the result of a strategic API initiative. Netflix ensures that
its API is easily integrated onto any devices that connects to a screen. The next time you use Netflix, think
about each interaction you have up to the point of streaming a show or movie. Each interaction is powered
by an API capability offered by its device integration API.
TWO
APIs encourage reuse and increase velocity
APIs are the ultimate multiplier for organizations because they can lead to reuse and increased velocity
by avoiding the need to rewrite the same thing over and over. What one team creates once can be used
by many developers to produce solutions used by hundreds or even thousands of end users.
Reuse of APIs often starts with internal developers. Rather than sharing libraries in a single
programming language, APIs enable code reuse across a variety of languages. Some organizations have
even gone to the extreme of calculating the number of lines that didn’t need to be written as a result of
adopting an API.
While not all APIs produced by your organization will have hundreds of consumers, it only takes one
team to re-implement a capability—but with slightly different behavior—to negatively impact your
customers and reduce organizational velocity.
These areas offer a clear API that provides the capabilities and data for a specific business concern.
Solutions are then built on these APIs to solve the needs of the internal business, partners, app
developers, and third-party solution providers.
We need to remember that APIs exist to help complete a job-to-be-done for customers. Either the job
requires the use of the API to start and finish the work, it requires the API for some portion of the work, or
the job is to bridge systems and organizations through machine-to-machine communication. Therefore,
APIs must be designed to deliver outcomes with the customer experience in mind. Remember, one poorly
designed API may have a negative impact on many developers and even more customers.
Through well-designed APIs that represent the desired future state, systems that were previously unable to
be sunset are now hidden behind these APIs. Organizations taking an API-centric approach are more likely
to migrate to newer systems and add new capabilities while sunsetting older systems that are currently
relied on every day. Use your emerging API initiative to paint a picture of where the organization needs to
go. Then map out the processes and effort required to get there.
THREE
What is an API platform?
Platforms create an ecosystem for orgs, customers, and partners. They connect everything that the
company does, both internally and externally. Until recently, most platforms were built by vendors
connecting a multisided marketplace such as supply chain management, customer relationship
management (CRM), and enterprise resource planning (ERP). Now API platforms are emerging within
organizations for the purposes of building internal, partner, and public solutions to meet the demands
of the marketplace.
Let’s take a look at each of these elements and how they turn business and technical capabilities
into a robust API platform.
It is sometimes estimated that there are 10 times more private APIs than publicly available APIs. It is likely
considerably more, since many organizations prefer to start with private APIs before opening them up to
partners and public developers.
The enterprise API portfolio addresses the needs of the business, partners, customers, and public app developers.
Events allow solutions to be built on top of the platform without the platform knowing they exist. These
Platform event notifications emerging solutions are notified when state changes occur. It also removes the need for systems to
constantly poll an API to see if any data has changed, using resources
create extensibility more efficiently.
Those familiar with event-driven architectures (EDAs) know the power of events for integrating services and
scaling systems. Exposing some internal events or emitting coarse-grained business events as part of an API
platform reduces system-to-system coupling and creates opportunities for innovation in an agile way.
Record 10
Record 1
Record 2
Record 3
Record 4
Record 5
Record 6
Record 7
Record 8
Record 9
high-velocity data and even allows for replaying message history. Consumers
TOPIC A
can revisit past messages in order of publication as new needs emerge or
corrective action is required.
Message streams are a recent addition to the API platform. They are used to
enforce data governance and data lineage requirements of regulated businesses.
In addition, they are becoming the preferred way for data analysis and machine
learning solutions that need to act on data as it becomes available, rather than
the more traditional batch-based approach through technologies such
Consumer A Consumer B
as Hadoop.
Data streams allow consumers to process new and historical messages for analytics.
Enterprise data management powers APIs through message and data streaming.
A common enabler of MASA is the integration platform as a service, or iPaaS. An iPaaS helps
integrate APIs and events together into integration solutions that connect systems, resulting in new
capabilities. These new capabilities may be deployed as services or new APIs that may be consumed
Mesh app and service by other applications.
architecture
With the emerging growth of MASA and iPaaS, there is now a possibility to enable business developers
to construct applications with limited code requirements. This will allow software developers to focus
on the more challenging tasks of building digital capabilities as APIs, while business users, comfortable
with things such as scripting Excel using Visual Basic for Applications (VBA), are able to build complete
applications without waiting for available resources from the IT department.
There is even an opportunity for marketing departments to build ephemeral applications that live for a
limited period of time, such as for supporting a conference, and then are thrown out once they are no
longer needed. Until now, this hasn’t been possible because the cost of development has been too high
to build ephemeral applications. We are still in the early days, but this is becoming more of a possibility.
FOUR
The return of modular software
Over the last decade, we have seen new development frameworks emerge to make it easier than ever
to build and deploy web-based applications. The downside of these frameworks is that they rarely
encourage modular design. The result is quick development at the start of a new project, but over time
complexity increases beyond the ability of the framework to maintain the efficiencies it first enabled.
At the same time, containerization tools such as Docker and orchestration layers such as Kubernetes
have emerged. These tools encourage developers to easily break unwieldy applications into smaller
components and deploy them in a predictable way.
The intersection of these circumstances has led to a renewed interest in modular software design.
Unlike past modular efforts through component-based development, we are seeing these modules
built as services and externalized on the network. This has resulted in the rise of microservices.
A microservice
Deploys Manages
architecture exhibits
independently via CI/CD automation. its own data sources and does not share
the following traits: them with other services.
Enables
replaceability and experimentation by
limiting the scope of each service.
...
API Microservices An organization embarking on a
... microservice journey may not be able to
Web + Mobile app Orchestr. API realize the benefits immediately. Therefore,
...
organizations moving in this direction
API Microservices
... should plan for the time required to mature
Alexa voice skill Orchestr. API Message their microservice approach as they prepare
... broker themselves for the journey ahead.
API Microservices
...
Slack Chatbot Orchestr. API
...
API Microservices
...
A common microservice architecture using orchestration APIs that call one or more microservices behind the scenes.
This tension between speed and safety is common. Organizations must find the balance between delivering
software as fast as possible and the risk of introducing bugs that impact the safety and stability of customer
and partner interactions.
A microservice architecture is primarily about reducing coordination costs of aligning efforts across multiple
teams/developers working on a complex system. This is accomplished by limiting the scope of a single
service while ensuring it is independently deployable. When an enhancement or fix is required, it is made
within the microservice quickly and deployed with confidence. Data sources are assigned to a single service
and cannot be shared across services, otherwise the coordination cost of the microservices is higher when
changes are applied to a shared data store.
Microservices require a fully automated system. A developer should be able to design a microservice,
register the service with an automated deployment system, and deploy code as needed without any
manual processes or approvals. Organizations that have not fully automated their deployment pipeline and
removed all or most of their manual processes will encounter considerable friction. Existing microservices
will need to be used to deploy new capabilities, resulting in a few siloed services.
Microservices must be independently deployable. Some organizations opt to use their existing In general, organizations should think
deployment processes, resulting in a coordinated deployment of all microservices at once. The outcome is smaller. They should seek to decompose
a distributed monolith that prevents organizations from realizing the full potential of speed and safety. solutions into smaller units of independently
deployed code. However, a complete shift to
Microservices should be owned, monitored, and managed by a single team. A team may own a few a microservice architecture may not be the
microservices but should not be responsible for a large number of services. Sometimes, organizational right answer. Instead, perhaps decomposing
structure and culture will be at odds with this style of service ownership, making it difficult to successfully internal systems into modular monoliths with
deploy a complete microservice architecture that supports the desired speed and safety. Keep this in mind clearly bounded contexts that expose web
before shifting to microservices. APIs and business events may be sufficient
for the needs of the organization.
Data sources will be distributed, because each service must own its data. This requires heavy
investment in distributed data management through data streaming or ETL processes to bring together
data from multiple services for the purposes of reporting. Organizations with large databases must use
caution when migrating to a microservice architecture.
The journey toward microservices requires a deep understanding of distributed systems. Those not
as familiar with the concepts of distributed tracing, observability, eventual consistency, and distributed
sagas will encounter a more difficult time with microservices.
FIVE
API management and security
API Management (APIM) layers accelerate the deployment, monitoring, security, versioning, and sharing of
APIs. They are often deployed as a reverse proxy, intercepting all incoming API request traffic and applying
policies to determine if requests are authorized to be routed to the API. The APIM layer may be fully hosted
in the cloud—as with Azure API Management or the Tyk open source API gateway—or on-premises, or
companies can use a hybrid approach.
Additionally, APIM enforces rate limits to prevent unlimited API usage, helps ensure security through
API tokens and/or OAuth 2, and protects against a number of attack vectors. While some offer identity
management as a built-in feature, most organizations will configure APIM to work with a third-party
identity provider to offload user management and authentication.
Keep in mind that security is a process, not a product, and a continual one at that. Even with the ever-
changing security landscape, we can still lay down an evolving set of best practices and test against them.
Don’t forget to develop healthy practices for monitoring your API surface area for malicious attacks.
API documentation was previously captured in a static document using HTML or PDF, but new options
are now available. Tools such as Swagger, RAML, and Blueprint are just a few of the formats available to
capture the definition of your API in a machine-readable format. These formats support the generation
of reference documentation for your API.
Many organizations are also benefiting from the integration of these definitions into their software
development life cycle (SDLC), helping to drive automation and configuration of their APIM and
other infrastructure.
2.
Authentication and authorization that covers how the API is secured, how to obtain an
API access token, and details about token expiration. Some open source and commercial
APIM vendors support publishing your
3. Example workflows, commonly in HTTP request/response format, and code examples to documentation artifacts to a developer
help developers understand how to solve common problems with the API. portal, making it easy for developers to
have important documentation on hand.
4. Rate limiting and related SLA details to understand how an application may be limited in
its use of the API, including pricing (where applicable).
5. API reference documentation, typically generated from your API definitions, for
developers who need to understand the details of each API endpoint.
Azure Active
Directory
API Management
Cosmos DB
Authentication
Putting it together:
Azure SQL
Database
Creates client
apps Cosume API
documentation
Developer portal
API Consumer
SIX
Developing your API strategy
APIs extend beyond a set of technologies with developer interest. They power customer experience,
business relationships, and internal innovation. A clear API strategy transforms your data and processes
into digital capabilities with a programmable interface.
Unfortunately, some organizations realize too late that teams are already designing and deploying APIs.
These APIs are built to solve a specific problem, are often designed in isolation, and lack the necessary
consistency for reuse.
By clearly defining the objectives of your organization’s API program first, you can keep teams focused
on delivering business value that meet these goals. After establishing your API program objectives,
clearly document them and share them throughout the organization. Use key performance indicators
(KPIs) to measure the success of your program, track them over time, and adjust your execution
accordingly to meet your objectives.
An API design-first approach starts by identifying the capabilities to be delivered and then moves
toward an API design to meet those capabilities. All this occurs before a line of code is written. Although
this may seem like a waterfall approach, it isn’t. Instead, it is the application of the Agile Manifesto
principle of combining business and technology into small iterations that deliver business value.
DISCOVER
1 2 3
2.
Model and design the API. Transform the
desired capabilities into a resource-based
Model API Design approach using the power of HTTP.
Identify capabilities
capabilities API
Document the initial API design. The
3.
documentation should provide enough
understanding to demonstrate common
5 4
PROTOYPE
use cases.
API feedback from
4.
Mock API design Seek feedback from stakeholders. Use
stakeholders feedback to validate that the API design
meets stakeholder needs.
DELIVER Decompose to Test / QA Finalize docs, often as needed to refine the API design.
Microservices API tutorials, examples
Produce a mock version of the API. As the
6.
design begins to take shape, use the captured
7 design definition, typically using the OpenAPI
Release API to format, for your stakeholders to review.
stage, product
Release the implemented API as a preview
7.
release. During this preview release, additional
insights and incorrect design assumptions
will emerge. Make necessary changes before
CONSUME 8a 8b officially releasing the API into production.
8.
Promote the release to production. Once
the preview release meets the needs of
stakeholders, release the API to production. At
this point, breaking changes are not allowed.
An API design-first process includes stakeholder feedback.
PARTITION
the portfolio into domain areas. Not all APIs are related (e.g., customer accounts, orders, and inventory Properly executed API portfolio
for an e-commerce platform). Separate your APIs and microservices into domain areas and assign management ensures that teams across
product ownership of each area. the organization can contribute to the
overall portfolio. It will also make it easier
for consuming teams to look for an API to
DEFINE address the needs of their apps. Be sure to
nominate a product manager or small team
a clear process to add new APIs. Setting up a clear process for adding APIs and microservices into the
to own the API portfolio, perhaps as part of
portfolio will prevent confusion about where new APIs belong and make it easier for API consumers to
your API governance strategy.
find the APIs they need.
MANAGE
URL paths like domain names. The URL paths under your API host name (e.g., api.mycompany.com are just
like domain names—they are real estate for developers. Manage your URL paths through the use of well-
known prefixes that support your domain-partitioning scheme. This will prevent scattered API endpoints,
duplicate or confusing resource names, and conflicting paths.
1. Discover
Identify new API capabilities as
5. Share
Inform your stakeholders and other
needs emerge from stakeholders. developers of the new capabilities.
2. Design
Refine those needs with the
6. Enhance
Incorporate consumer feedback and Integrate existing software processes
stakeholders, identify capability use proper versioning techniques to throughout the API life cycle as necessary, but
requirements, and design the enhance the API without disrupting be prepared to adjust processes to prevent
API accordingly. existing consumers. negative effects for existing API consumers.
3. Deliver
Implement, test, and verify
7. Retire
Inform consumers of any plans to stop
that the new capabilities meet supporting an existing API, migrate
stakeholder expectations. consumers to a new API (if available),
and retire the API.
4. Release
Deploy, monitor, and manage
the API at runtime.
Modern architectures often require the combination of distributed logging, distributed tracing, and
API monitoring support (often offered as a feature within APIM) to properly troubleshoot and assess
runtime errors. Some organizations are finding that the instrumentation available via a service mesh
is a nice complement to these tools when adopting microservices.
Empower Craft
solution teams to discover and consume flexible processes and practices that
existing APIs, increasing the likelihood of encourage innovation.
reusing an API rather than duplicating effort.
API governance can be centralized and managed by a single team for the life of the API
program. For larger organizations, governance may be centralized at the start but evolve
to a more federated approach over time to scale the process.
© 2019 Microsoft Corporation. All rights reserved. This document is provided “as is.” Information and views expressed in this document, including
URL and other internet website references, may change without notice. You bear the risk of using it. This document does not provide you with any
legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes.