Why nobody grows up wanting to be a DevOps engineer
When I look at younger generations that didn’t grow up largely offline like I did, I feel slightly sorry for them. I’m in my mid-thirties, so I know what it was like to grow up tapping into dial-up internet as quickly as possible (to avoid blocking the phone line for too long) to access a couple of Wikipedia pages to do my homework and not feel like anything was missing. By personally living through the ascendence of the personal computer we all have in our pockets I feel, not immune, but better equipped to use it to my advantage and not fall victim to its false promises of limitless bliss and fulfillment simply for being more “connected”.
On the other hand, there is another revolution that I was not directly a part of, the revolution that happened in the IT departments worldwide across the 00s in the aftermath of the now-legendary Flickr talk at the O’Reilly Velocity conference in 2009. An event that put DevOps on the map and showed a future that could pivot from the siloed, slow-moving ITIL-based organization to something better.
Since I recently switched careers from teaching to tech less than 5 years ago. I don’t know what it’s like to work in any other professional environment that doesn’t live and breathe the practices imparted in that seminal Flickr talk. Agile methodologies and the still difficult-to-pin-down definition of “DevOps“ is a status quo I’ve never had to question.
But I question it now. If I argue that up-and-coming generations are missing fundamental perspective and have a lot to gain from knowing what it was like to live without smartphones. I have to apply the same logic to myself and make a concerted effort to understand what DevOps really is? What did it emerge as a response to? What were its initial promises and have those promises been delivered? How and why do people end up being DevOps engineers? And what will it mean to be a DevOps engineer in the future?
How DevOps Started and it’s original Promise
I sometimes wonder what it must have been like back in the days when devs had to put in infrastructure provisioning request forms and wait days or weeks to be served. I’ve heard the stories of what stereotypical Ops people of that culture were like, finger-pointing grumps whose favorite words were “NO“ and “Where’s my pager?“ .
John Allspaw and Paul Hammond, as well as many attendees of their famous velocity talk, “10+ Deploys Per Day: Dev and Ops Cooperation at Flickr," didn’t have to wonder, the common friction between devs and ops teams must have been all too vivid to them.
After watching the talk a handful of times over the years, a few things stand out to me. The first, is that I was unaware that talks back then contained so much swearing or maybe it’s just Mr. Allspaw. Another was the key message the speakers put forth, that both Dev and Ops teams effectively shared the same objective. To enable the business.
They go on to talk about the tools and cultural traits organizations might want to adopt to achieve multiple deployments to production a day. They spoke of automated infrastructure, feature flagging, shared alerting and monitoring, all coalescing around a renewed collaborative culture that valued trust and a healthier attitude towards system failures and blame avoidance above all else.
The shape of what DevOps would come to mean in the following years was further crystalized in large part by the hugely influential DevOps Handbook and Site Reliebility Engineering books.
The opening chapter of the former describes in few words what a DevOps way of doing things aspired to unlock
“[Multiple team working together to] enable the fast flow of planned work into production (e.g., performing tens, hundreds, or even thousands of code deploys per day), while achieving world-class stability, availability, and security.”
Have we delivered on the promise?
The feeling of walking on the shoulders of giants comes to mind when I think of the ideas presented in that Flickr talk. Those ideas that must have been so revolutionary to many are the only professional reality I've ever known. So regardless if improvements could be made to current DevOps methodologies, best practices and tooling, I for one am grateful for all the progress that has been achieved so far. Anyone I ask who was configuring servers or hacking before 2009 corroborates that things are better now than they were back then.
Having said that, are most companies shipping like crazy, while achieving world-class stability, availability, and security? Largely no. The DevOps world even with a massive toolbox full of modern tooling available, still runs off of the exact same framework that was proposed in that talk.
In the words of Adam Jacob
“The problem isn’t that we haven’t optimized each individual part of the system enough. We’ve built more efficient tooling at every step. But the way the whole system is put together? The experience of using it? That’s basically identical to how it was in 2009, and it’s the reason we’re stuck.“.
Siloes still exist, handoffs are error-prone and collaboration on many occasions is quite forced and rigid. Anybody who has worked as a DevOps engineer for any length of time will have a long list of things they think their organization gets wrong and will often have equally low amount of faith in their organization’s capacity to do anything about it. Adam, a veteran DevOps practitioner, has even called for a second wave DevOps which goes further than trivially improving tools and invites us to think outside of the box, challenge the established rules, and see what’s on the other side.
Speaking of DevOps practitioners, who are these people? How and why does one become one?
How do people end up in DevOps in the first place?
In just 15 years, the technology industry has evolved significantly. Job titles like DevOps engineers, SREs, and Platform engineers are now common on job boards and are hot items for tech recruiters. However, outside the IT world, these terms are still largely unknown. Despite its rapid growth, DevOps isn’t yet a profession people aspire to join; instead, it’s something many simply "fall into".
I stumbled into DevOps after a conversation with my cousin, who suggested it following my decision against a strict network engineering path after earning a CCNA certificate. Curious about who ends up in DevOps and if future engineers might choose it as their first career option, I asked the /devops subreddit and was surprised by the variety of opinions.
I found some fellow “I just fell into it“ people:
Others are moderately bullish on newer generations:
Others not so optimistic:
Even though the definition of DevOps is still highly disputed, it seems unlikely to ever position itself as a profession students hear about at job fairs or see permanently added to university curriculums. This might be because we tend to imagine ourselves excelling in specific areas, believing that specialization increases our chances of success.
When I was growing up, I fantasized about being the lead guitarist in a famous band, with god-level shredding abilities. I wasn't dreaming about being reasonably good at playing all the instruments.
In DevOps, there are no guitar solos. To excel, you need to be familiar with a long list of tools, languages, frameworks, hyperscalers, and processes. The best DevOps engineers are generalists at heart. This modular, Lego-like nature of the work and experience might make DevOps less popular outside of IT departments.
The case for generalists, tinkerers, and the concept of glue
It’s impossible to attempt to form a non-subjective profile of what traits a DevOps practitioner should have but in my short experience and what I’ve learned from conversations with more experience individuals than I, a few traits emerge.
Generalists
Have you ever read a “How to land a job in DevOps“ guide? Remember the knowledge requirements section? Linux, Docker, CI/CD, Git, Hypersclaer of choice, Networking etc. The list goes on usually in the desired experience section of DevOps job requirements.
If you’ve interviewed for roles such as developer or product design roles you will more than likely have to show a portfolio at some stage of the process. This is rarely the case in DevOps interviews. I can’t think of one person who has assembled and updated a DevOps portfolio on a regular basis? The modular and distributed systems-building nature of DevOps roles just doesn’t lend itself well well-curated displays in a portfolio.
As someone who was bored out of his brain teaching high school-level English for seven years, I naturally gravitated towards DevOps, a field requiring me to learn many tools and concepts and piece them together collaboratively. Not specializing deeply in any one concept but refining the skill of quickly learning new things is the callus I developed. Generalists who thrive in such environments fit right in.
Tinkerers
People who do well in DevOps might think of themselves as tinkerers.
I love this idea, and it fits many DevOps engineers I've met. Being interested in learning how things work simply for the knowledge of it is a common trait among the DevOps mentors I’ve had. Sure, spending the weekend installing a beefier switch in your home lab or rendering new mini figurines on your 3D printer doesn't directly show DevOps skills, but this background knowledge can indicate potential success in DevOps more than certificates can.
Glue
Much of the work in complex systems can go under the radar since it’s hard to plan or predict. Since DevOps involves weaving tools together to build a platform or delivery system, a lot of glue is needed.
Processes must be put in place and automated to keep up with technical debt and maintenance work that accompanies every tool in a tech stack. Individuals who can naturally and often thanklessly act as the glue, connecting disparate parts of the system through automation, communication, or improving repetitive processes, are instrumental to organizational success. This skill isn't something you list on your CV, but combined with the curiosity of a tinkerer and the openness of a generalist, it's a potent combination.
The current state: Platforms vs DevOps
A false dichotomy often arises in these discussions usually for marketing reasons: that DevOps is dead and platforms are the future. Platform engineers aim to give developers autonomy over traditionally Ops-related components (k8s, IaC components, IAM) without direct interaction, allowing developers to self-serve and be independent.
A well-designed, context-specific platform can increase developer velocity. According to the 2023 Puppet State of DevOps report, over two-thirds (68%) of respondents saw improved development velocity after adopting platform engineering. However, velocity shouldn't be the only metric. As georgouslyhumble points out on Reddit, sometimes the goal is to maintain existing velocity while meeting new organizational requirements. For instance, a logging sidecar can standardize log collection without changing developer velocity, enhancing the platform to meet increased company requests.
Ops work remains complex and dynamic, and skilled Ops personnel are essential above a certain threshold of complexity. Platforms enable developers but notice that they don't necessarily reduce silos or integrate Dev and Ops teams more closely.
Teams are rightly cautious when adding new tools to their stack because new tools often introduce introduce maintenance and upkeep overhead that quickly stacks up. Tools like Glasskube, which reduce operational overhead, are essential. These are the tools we need more of to create robust and efficient DevOps platforms for the future.
Future predictions
A certain type of platform will win out
The systems, platforms or methods of working that will win out will not have to “teach“ its users how to work and collaborate together. A future system that delivers the incredible possibilities of endless software shipping without compromising security and stability will only be possible if it makes team collaboration and cooperation the easiest, most intuitive, and most natural way of working while abstracting the work that neither devs nor ops are excited to do.
To create it though we are going to have to think outside of the box.
A second wave of DevOps might be one the way
Thankfully there are many restless and nonconforming people who contribute to the constant improvement of methodologies, processes, and tooling in the DevOps space. Some might say that a formal movement of rethinking what DevOps could be is already emerging and that’s pretty exciting.
The more generalists and tinkerers, the better
The best-equipped individuals to keep connecting the puzzle pieces, close feedback loops, and rethink lousy ideas are those who are not afraid to trade deep specialization for professional genaralization. Those who dare to venture out and learn on the fly by tinkering, testing, and asking questions along the way are the ones who will keep the figurative ship afloat as it moves faster and faster towards engineering excellence.
How to find enough of these people is another question.
Conclusion
It looks like I'm no closer to knowing why people don't grow up wanting to be DevOps engineers, perhaps it's a blend of still being a relatively new field coupled with the fact that generalists aren't usually know as the coolest kids on the block.
Looking ahead to the next wave of talent, whether they consciously choose or stumble into DevOps roles, one thing is certain: understanding the field's history is key. It's the only way future engineers can develop the ability to identity the gulf between the current state and the aspirational future of what could be. By neglecting this gap, the status quo will prevail and we will be destined to stagnant mediocrity.
While it's undeniable that the tech landscape has vastly improved since the pre-DevOps era, it's equally evident that DevOps is still finding its footing 15 years in.
Seasoned professionals need to keep a keen eye to identify and encourage the young tinkerers, generalists, and "glue people" around them to not worry about chasing certain titles but rather help redefine and **evolve DevOps into something that delivers on the original aspirations of 2009.
If you like this sort of content and would like to see more of it, please consider supporting us by giving us a Star on GitHub 🙏