The last decade of programming has seen a number of revolutionary transformations. One has arisen from a cluster of practices around devops, which aligns development and operations teams into a shared work process, and continuous integration and continuous delivery (CI/CD), in which devops teams deliver constant incremental updates to a codebase. Another transformation has come from the related move from monolithic codebases to cloud-based microservices running in containers managed by orchestration platforms like Kubernetes.
Container-based applications running on clustered systems or in the cloud can be complex and difficult to provision and manage, even with a platform like Kubernetes orchestrating things. GitOps is an emerging set of practices that aims to simplify this management task by applying techniques from the worlds of devops and CI/CD.
The key to GitOps is the idea of infrastructure as code, which takes the same approach to provisioning infrastructure as devops uses to provision applications. So, not only the application but also the underlying host machines and networks are described in files that can be treated as any other code within a version control system, with automated processes then working to converge the real-world application with the one described in those files.
In GitOps parlance, the code in the version control system is the single source of truth about what the application should look like in production.
GitOps defined
Weaveworks is the company that has done the most to popularize the concept of GitOps. We’ll go into the details of Weaveworks’s role in a bit, but first, let’s take a look at the company’s definition of GitOps, which is twofold:
- An operating model for Kubernetes and other cloud native technologies, providing a set of best practices that unify deployment, management, and monitoring for containerized clusters and applications.
- A path towards a developer experience for managing applications; where end-to-end CI/CD pipelines and Git workflows are applied to both operations and development.
In other words, GitOps is a specific set of practices designed to manage Kubernetes and similar platforms, which also lends itself to possible wider application as more and more development shops adopt devops practices and migrate code to the cloud. But to understand the secret sauce of GitOps and the problems it solves, we need to talk about the components that go into it.
Git definition
The Git in GitOps refers to the wildly popular distributed version control system developed by Linus Torvalds in 2005. Git is a tool that allows teams of developers to work together on an application codebase, storing various branches of code that they tinker with before merging them into production code. A key concept within Git is the pull request, in which a developer formally asks for some code they’ve been working on to be integrated into another branch within the codebase.
A Git pull request provides an opportunity for team members to collaborate and discuss before reaching consensus on whether the new code should be added to the application. Git also stores older versions of code, which makes it easy to fall back to the last good version if something goes wrong, and lets you quickly see what’s changed between revisions. Git may be best known as the underpinning of GitHub, a cloud-hosted version control system, but Git itself is open source software that can be deployed anywhere, from internal corporate servers to your PC.
Note that while we usually think of Git as a computer programming tool, it’s actually agnostic as to what content you’re using it for. Git will happily treat any set of text files as your “codebase,” and it can, for instance, be used by writers looking to keep track of edits to a collaborative work. This is important because much of the codebase at the core of GitOps consists of declarative configuration files rather than executable code.
One last thing to say before we move on: Despite “Git” being right there in the name, GitOps doesn’t actually require the use of Git. Shops that are already invested in other version control software, such as Subversion, can implement GitOps as well. But Git is widely used with within the devops world to implement CI/CD, so most GitOps projects will end up using Git.
What is the CI/CD process?
A complete look at CI/CD is beyond the scope of this article—see the InfoWorld explainer on the subject—but we need to say a few words about CI/CD because it’s at the core of how GitOps works. The continuous integration half of CI/CD is enabled by version control repositories like Git: Developers can make constant small improvements to their codebase, rather than rolling out huge, monolithic new versions every few months or years. The continuous deployment piece is made possible by automated systems called pipelines that build, test, and deploy the new code to production.
Again, we keep talking about code here, and that usually summons up visions of executable code written in a programming language such as C or Java or JavaScript. But in GitOps, the “code” we’re managing is largely made up of configuration files. This isn’t just a minor detail — it’s at the heart of what GitOps does. These config files are, as we’ve said, the “single source of truth” describing what our system should look like. They are declarative rather than instructive. That means that instead of saying “start up ten servers,” the configuration file will simply say, “this system includes ten servers.”
The CI half of the GitOps equation allows developers to quickly roll out tweaks and improvements to these configuration files; the CD half happens when automated software agents do their best to ensure that the live version of the application mirrors the descriptions in the configuration files — that it converges to the declarative model, in the language of GitOps.
GitOps and Kubernetes
As we’ve mentioned, the concepts of GitOps were originally developed around managing Kubernetes applications. With what we now know about GitOps, let’s revisit Weaveworks’ GitOps discussion and see how they describe how you’d make updates to a Kubernetes managed on GitOps principles. Here’s a summary:
- A developer makes a Git pull request for a new feature.
- The code is reviewed and approved, then merged into the main codebase.
- The merge triggers the CI/CD pipeline, which automatically tests and rebuilds the new code and deploys it to a registry.
- A software agent notices the update, pulls the new code from the registry, and updates the config file (written in YAML) in the config repository.
- A software agent in the Kubernetes cluster detects that the cluster is out of date, based on the config file, pulls the changes, and deploys the new feature.
Weaveworks and GitOps
Clearly steps 4 and 5 here are doing much of the heavy lifting. The software agents that are magically syncing the “source of truth” in the Git repository with the real-world Kubernetes application are the magic that makes GitOps possible. As we’ve said, in GitOps terms the process of making live systems more like the ideal systems described in config files is called convergence. (When the live system and the ideal system aren’t in sync, that’s divergence.) Ideally, convergence would be achieved by automated processes, but there are limits to what automation can do and sometimes human intervention is necessary.
We’ve described the process here in generic terms, but in fact, if you actually go look at Weaveworks’ page, the “software agents” we mentioned are part of the company’s Weave Cloud platform. The term “GitOps” was coined by Weaveworks CEO Alexis Richardson, and it serves in part to make the Weaveworks platform appealing to developers already steeped in the devops and CI/CD worlds.
But Weaveworks has never claimed a monopoly on GitOps, which is more a philosophy and set of best practices than a specific product. As the blog for CloudBees, a company that provides CI/CD solutions, notes, GitOps represents an open, vendor-neutral model that was developed in reaction to managed proprietary Kubernetes solutions being rolled out by big cloud vendors like Amazon, Google, and Microsoft. CloudBees offers its own GitOps solutions, as do a number of players in this space.
GitOps and devops
Atlassian, a company that makes a number of tools for agile developers, has an in-depth blog post on the history and purpose of GitOps that is worth your time. In their view, GitOps represents a logical extension of the ideas that came together as devops. Specifically, GitOps is an elaboration of the concept of infrastructure as code, itself an idea that came out of the devops milieu. GitOps, as Atlassian sees it, bridged the crucial gap between existing devops techniques, which had evolved to solve problems of system administration, and the specific needs of distributed, cloud-hosting applications. The automated convergence offered by various cloud vendors is what makes GitOps special.
And while GitOps remains focused on Kubernetes today, we hope we’ve made clear how it applies to the much wider world of distributed, cloud-based apps. A blog post by open source security vendor WhiteSource outlines the advantages of GitOps:
- Observability: GitOps systems offer monitoring, logging, tracking, and visualization into complex applications so developers can see what’s breaking and where.
- Version control and change management: Obviously this is a key benefit of using a version control system like Git. Flawed updates can be easily rolled back.
- Easy adoption: GitOps builds on the devops skills many developers already have.
- Productivity: GitOps provides the boosts to productivity that devops and CI/CD have brought to other realms.
- Auditing: Thanks to Git, every action can be traced to a specific commit, making it easy to track down the cause of errors.
Even if you don’t use Kubernetes, chances are good that GitOps will be part of your workflow sooner or later.