DevOps for the Modern Enterprise: Winning Practices to Transform Legacy IT Organizations

Praise

Mirco knows his stuff. He has a way of stating simple choices and framing options that encourage action. His unique perspective on how to transform your IT organization, especially when you have heavily invested in outsourcing, are both thoughtful and practical. After you read his book, make sure to subscribe to his blog. You want to make sure to have Mirco by your side as you go through this transformation.

—Mustafa Kapadia, Digital Transformation Leader, IBM

This is a much-needed book on building teamwork to drive technology infrastructure efforts. Mirco introduces a simple set of processes and well-reasoned principles to align the organization, transform technology infrastructure, and deliver value for customers.

—Eric Passmore, Partner Director of Commerce at Microsoft

A pragmatic view on IT and DevOps that doesn’t just focus on the what but the how, based on firsthand experience.

—Ajay Nair, DevOps Architect, Accenture

This is a truly practical companion book to the other leading publications in the DevOps space. Mirco shares real-world lessons from many years of collective experience and provides tried and tested exercises for you to use to help drive insights and improvements in your organization.

—Emily Arnautovic, Software Architect, Accenture

DevOps is a hot topic. Pretty much every major organization wants it, and pretty much every major organization is struggling to come to grips with what it is and exactly how to get it. Mirco is a rare entity in the space. He’s an architect who has grown up in large, complex organizations working with even larger, more complex systems integrators and delivery partners heavily reliant on ERP, CRM, and other COTS applications that seem to challenge much of what DevOps is about. Both pragmatic and practical, he not only gives great tips on how to find your way down the DevOps path in these environments but will help you avoid many a mistake as he shares his own.

—Mark Richards, SAFe Fellow at Coactivation

25 NW 23rd Pl, Suite 6314

Portland, OR 97210

Copyright © 2018 by Mirco Hering

All rights reserved, for information about permission to reproduce selections from this book,

write to Permissions, IT Revolution Press, LLC,

25 NW 23rd Pl, Suite 6314, Portland, OR 97210

First Edition

Printed in the United States of America

24 23 22 21 19 181 2 3 4 5 6 7 8 9 10

Cover and book design by Devon Smith

Author photograph by Julian Dolman

Library of Congress Catalog-in-Publication Data

Names: Hering, Mirco, author.

Title: DevOps for the modern enterprise : winning practices to transform

legacy IT organizations / Mirco Hering.

Description: First edition. | Portland, OR : IT Revolution Press, 2017. |

Includes bibliographical references and index.

Identifiers: LCCN 2017053420 (print) | LCCN 2017056532 (ebook) | ISBN

9781942788201 (ePub) | ISBN 9781942788225 (Kindle) | ISBN 9781942788195 (trade pbk.)

Subjects: LCSH: Computer system conversion. | Operating systems (Computers) |

Computer software--Development. | Management information systems. |

Information technology—Management.

Classification: LCC QA76.9.C68 (ebook) | LCC QA76.9.C68 H47 2017 (print) |

DDC 004.068—dc23

LC record available at https://lccn.loc.gov/2017053420

ISBN TP: 978-1942788195

ISBN ePub: 978-1942788201

ISBN Kindle: 978-1942788225

ISBN PDF: 978-1942788218

For information about special discounts for bulk purchases or for information

on booking authors for an event, please visit our website at ITRevolution.com.

Contents

Figures

Tables

FOREWORD

by Dr. Bhaskar Ghosh

Throughout my many years working in the IT industry, I have encountered numerous disruptions from technology advances, new business models, and even global economic cycles. Among all those waves of change, DevOps in particular stands out. Paradoxically, it stands out not because of the principles it embodies but because of the downstream change it has ushered in.

I like to say that you can do DevOps without Agile, but you cannot do Agile without DevOps. This is just one of many examples of transformations that have been catalyzed by the advent of DevOps. By empowering software developers to do more and, consequently, own more, DevOps is unleashing creativity, which is leading to a feedback loop of continuous improvement in systems delivery.

While I fondly recall the time I spent years ago managing infrastructure operations for large enterprises, the challenges of that job often resulted from the distinction that was drawn between development and operations responsibilities. Of course, this was in an era of largely monolithic systems using Waterfall techniques for large-scale, periodic delivery of software releases. In those past production environments, such separation of duties was a practical and efficient operating model for the needed pace of system changes.

In the digital era, however, speed is paramount. The distinction of responsibilities is not conducive to the more incremental change delivery approaches that are required to meet the demands of business today.

In this book, Mirco shares much more than just the mechanics of DevOps; he also shares his passion for improving software engineering. Through clever analogies and prescriptive advice, Mirco dispenses practical recommendations for how to embrace DevOps in an enterprise. Whether you are just getting started with DevOps or you are a seasoned professional seeking counsel on how to apply its principles at scale, as you go through DevOps for the Modern Enterprise, you will find yourself infected by the enthusiasm Mirco has for DevOps and the benefits it can bring.

Dr. Bhaskar Ghosh

Group Chief Executive—Accenture Technology Services

Bangalore, India

March 2018

Preface

Learning is not compulsory; it’s voluntary.

Improvement is not compulsory; it’s voluntary.

But to survive, we must learn.

—W. Edwards Deming

One of the most rewarding things in my career has been the search to find the most efficient way to deliver meaningful projects and to get as many people as possible to do the same. When we are not working efficiently, we spend time on unnecessary, repetitive, and boring tasks, which is not fun at all. I don’t have lofty goals of changing the world by doing my work, but I think everyone deserves to enjoy his or her work. And when workers enjoy what they do, good outcomes are inevitable.

Since joining Accenture as a consultant over ten years ago, I have worked with dozens of teams to increase their delivery capability through increased productivity or increased speed. But even before my time as a consultant, I was driven to figure out ways to bring efficiency to IT. I came into the workforce in the late 1990s when offshoring IT work was still in its early days. I spent my first few years as a developer in research labs for IBM, working on telematics and developer tools (e.g., developing languages for custom CPUs and providing the associated compilers and IDE extensions). When I started working, packaged software was on the rise, but most work was done in custom development and onshore. The only way to improve productivity was to increase the level of automation, and in all my early projects, we had creative solutions built around shell scripts, Perl scripts, and other custom tooling to make the life of developers and operators easier. I thoroughly enjoyed building these automation solutions and seeing how projects became easier and more enjoyable for everyone involved.

Then something curious happened. I spent the next five years on two large projects and focused on building the kind of developer tooling that I knew helped projects deliver successfully. When I finished those projects and started to look around across organizations, I realized that somehow I didn’t see as much automation as I would have expected. After all, I spent all of my professional life working on automating tasks for delivery teams. When I spoke to others in the industry, it became clear that packaged software and offshore delivery capabilities provided shortcuts to productivity gains and cost reductions that many organizations leveraged rather than investing in good development practices and tooling.* I spent the next few years in what was perceived as a niche market to help organizations implement delivery tooling, but truth be told, it just wasn’t sexy for organizations to invest in this.

Larger and larger offshore percentages and reductions of the average cost per developer workday (otherwise known as ADR, average daily rate) were targets that could more easily be sold to the organization as success than the somewhat more difficult and less easily measurable activity of developing a good delivery platform that makes everyone in IT more productive. After all, how do you measure productivity in IT in the first place? I am with Randy Shoup and Adrian Cockcroft, who both admitted while presenting at conferences that in their whole career, they have been looking for a good measure of productivity but have not been able to find something useful. I elaborated on this in a blog post to describe that productivity is very difficult to measure in IT; instead, measure cycle time, waste, and delivered functionality.¹ It is important to measure some meaningful metrics, as you are otherwise not able to see whether you are improving; it just turns out that productivity in the usual sense is elusive in IT and that we need to look for other measures that help us judge how efficient we are.

I spent the next few years working to understand where the problems in IT are coming from and how to solve them. I was lucky, as my research fell into a time when technology and methodology were evolving to bring out the foundations to a new way of delivering IT: Agile, DevOps, and cloud, among others, made it a lot easier to implement the kind of solutions I had built my entire career. The niche that I had worked in became more and more fashionable; today it is difficult to find an organization that is not talking about Agile and DevOps.

Yet when we take a good, hard look at ourselves in the mirror and see where the IT industry is currently, we realize that IT delivery is still not where it should be. We all tend to agree on continuous delivery being a good practice to use and to build modern application architectures, but when we look for organizations that have mastered it, they are few and far between. Many organizations are working in ways that have evolved over many years by taking lessons from traditional manufacturing. After all, those practices are well codified in many an MBA curriculum and have hundreds of years of experience behind them. But those practices and ideas are not appropriate anymore.

I have not mastered it all myself and am still learning every day and with every engagement, but I want to make my experiences available to as many people as possible. As you can see from Figure 0.1, I am a developer at heart who looked for technical solutions first instead of considering the people involved. I had to learn the hard way that just having the right methods and tools will not magically transform an organization. The cultural change that is the hardest is also the most impactful. It took a good amount of failures and near misses to learn what I have to share in this book and to understand that you need to bring all the ingredients together with the right culture to really transform an organization.

Figure 0.1: How Mirco’s understanding of organizational change evolved

Over the last few years, I have developed a workshop that I run with CIOs and other IT leadership from our clients to explore their challenges and help them identify possible ways forward. The fascinating thing when you are in a room with intelligent and visionary leaders is that you learn more each time. I have been running this workshop for a while, and I am extremely thankful for the experiences and ideas that the CIOs have shared with me, which continue to improve the workshop. (This book contains the accumulated knowledge from those sessions.)

DevOps for the Modern Enterprise is meant to address the different challenges that organizations face as they transform into modern IT organizations, and yes, most organizations today are IT organizations, whether they are car manufacturers or banks, due to the dependence on IT for their core business. We all know that technology is evolving faster and faster. In the meantime, we have legacy applications from many years ago. Even new applications that we are building today will be legacy in a few years’ time. I actually subscribe to the idea that legacy is any code written before today. Software and technology are transforming the business landscape by enabling new ways for people to connect, share, and collaborate. Furthermore, technology has freed people from constraints and geography. As a result, the world is becoming more complex and even faster, and these new consumption patterns are disrupting well-established business practices. Many organizations are facing the fundamental challenge of modernizing their IT infrastructure. Our old mental models and methods are clearly not working anymore. We need new ways of dealing with the need for new solutions. The path toward modernized solutions and improved technology remains mysterious and tricky. Very few legacy organizations have mastered the transformation.

Over the years, I’ve worked with some of the largest technology organizations in every industry vertical. And whatever excuses you may have, whether they’re about technology, complexity, or culture, I claim that I’ve seen worse. And yet, these organizations have been able to radically transform and improve their outcomes. I want to share some of their learning and achievements with you. IT should not be a place where we spend the majority of time solving the same old problems again and again.

Everything in this book is supported by Agile, DevOps, and Lean principles. In fact, I start pretty much all my client discussions by asking What do these principles mean to you? because they are all so ambiguous. I tend to use the overview picture in Figure 0.2 to make sure everyone understands exactly how I use these principles.

Figure 0.2: Relationship between Agile and DevOps: How the principles Lean, Agile, and DevOps relate to each other

The mission I’ve set for myself as an advisor to my clients is to make myself redundant. Once my clients are leveraging the principles of Lean, DevOps, and Agile successfully and are working in an efficient way, I can go off and spend more time on implementing exciting solutions. I cannot think of a better goal in life than to make myself redundant and go on to focus on one or two projects instead.

But as long as IT continues to be a place where too many things go wrong and where people struggle every day, I will help make IT a better place to be. I will aim to help the transition with this book, as my team cannot be everywhere. There is more than enough work for all of us. Like all honest answers to complex problems, this is just a starting point. You should feel free to experiment with the recipe, add your own ingredients, and shake it up. Each journey of transformation is contextual, and there is never only one recipe for success.

I hope that with this book I can make it easier for all of you to navigate the challenges ahead. I am sharing my experience and practical exercises that I run with my clients and that you can use in your organization (whether small or large, old or new) to progress in your journey. I am looking forward to seeing you along the journey: at a conference talking about our successes and failures, at a meet-up over drinks, or perhaps on one of my consulting engagements as we solve some challenges together.

I am a developer at heart, and I want to develop cool new applications. Let’s transform our industry so that we can all spend more time on the creative side of IT.

Mirco Hering

* For more information, see my DevOps.com article Why We Are Still Fighting with the Same Problems in DevOps as 15 Years Ago.

INTRODUCTION

How We Got Here

The greatest danger in times of turbulence is not the turbulence;

it is to act with yesterday’s logic.

—Peter F. Drucker, Managing in Turbulent Times

Before we get into the meat of the book, I want to take a moment to explain why it is so important to change the way we approach IT delivery. We obviously didn’t choose to be in the situation we are in today, where most organizations are struggling to deliver IT in a way that truly supports the business. IT is either too slow or too expensive, or does not deliver the quality that business stakeholders expect, but it’s not because we have consciously made bad decisions. When you look around, we are also pretty much in agreement as to what good looks like—the examples of Netflix or Google are used in literally hundreds of presentations about IT. Pretty much every organization is talking about Agile and flexible delivery, automation across the software delivery life cycle (SDLC), and leveraging modern architecture patterns like cloud native application , twelve-factor applications , and microservices .

Yet if we look around, we still struggle to find organizations that have mastered this new way of working in IT. Many of the organizations with good examples are relatively young organizations that have grown up as cloud or internet natives (companies that were founded with the internet as the target platform). We even have a term for them: DevOps unicorns. We call them unicorns because they are rare and seemingly unattainable for the average organization with a legacy IT architecture.

One could conclude that the challenge must be with this legacy architecture that organizations try to transform. While this is to some degree correct, I think it has even more to do with the mind-set. Many technology leaders are using ideas that were adopted from a traditional manufacturing context even though IT is inherently different*—they leverage the mental model of manufacturing for an inherently creative process. In traditional manufacturing, we follow a predictable process to produce the same outcome (a product) again and again. In contrast,