Prow, Jenkins X Pipeline Operator, and Tekton: Going Serverless With Jenkins X

The serverless flavor of Jenkins X or, as some call it, Jenkins X Next Generation, is an attempt to redefine how we do continuous delivery and GitOps inside Kubernetes clusters. It does that by combining quite a few tools into a single easy-to-use bundle. As a result, most people will not have a need to understand intricacies of how the pieces work independently, nor how they are all integrated. Instead, many will merely push a change to Git and let the system do the rest. But, there are always those who would like to know what’s happening behind the hood. To satisfy those craving for insight, we’ll explore the processes and the components involved in the serverless Jenkins X platform. Understanding the flow of an event initiated by a Git webhook will give us insight into how the solution works and help us later on when we go deeper into each of the new components.

The description that follows might be too much for some people. I will not think less of you if you choose to skip it and enjoy using serverless Jenkins X without knowing its inner workings. After all, the main objective of Jenkins X is to abstract the details and let people practice continuous delivery without spending months trying to learn the intricacies of complex systems like Kubernetes and Jenkins X.

Just as with static Jenkins, everything starts with a push to a Git repository which, in turn, sends a webhook request to the cluster. Where things differ is that there is no Jenkins to accept those requests. Instead, we have Prow. It does quite a few things but, in the context of webhooks, its job is to receive requests and decide what to do next. Those requests are not limited only to push events, but also include slash commands (e.g., /approve) we can specify through pull request comments.

Prow consists of a few distinct components (e.g., Deck, Hook, Crier, Tide, and a few more). However, we won’t go into the roles of each of them just yet. For now, the vital thing to note is that Prow is the entry point to the cluster. It receives Git requests generated either by Git actions (e.g., push) or through slash commands in comments.

Prow might do quite a few things upon receiving a request. If it comes from a command from a Git comment, it might re-run tests, merge a pull request, assign a person, or one of the many other Git related actions. If a webhook informs it that a new push was made, it will send a request to the Jenkins X Pipeline Operator which will make sure that a build corresponding to a defined pipeline is run. Finally, Prow also reports the status of a build back to Git.

Those features are not the only types of actions Prow might perform but, for now, you probably got the general Gist. Prow is in charge of communication between Git and the processes inside our cluster.

When a Prow Hook receives a request from a Git webhook, it forwards it to Jenkins X Pipeline Operator. The role of the operator is to fetch jenkins-x.yml file from the repository that initiated the process and to transform it into Tekton Tasks and Pipelines. They, in turn, define the complete pipeline that should be executed as a result of pushing a change to Git.

The reason for the existence of the Pipeline Operator is to simplify definitions of our continuous delivery processes. Tekton does the heavy lifting, but it is a very low-level solution. It is not meant to be used directly. Writing Tekton definitions can be quite painful and complicated. Pipeline Operator simplifies that through easy to learn and use YAML format for defining pipelines.

Tekton creates a PipelineRun for each build initiated by each push to one of the associated branches (e.g., master branch, PRs, etc.). It performs all the steps we need to validate a push. It runs tests, stores binaries in registries (e.g., Docker Registry, Nexus, and ChartMuseum), and it deploys a release to a temporary (PR) or a permanent (staging or production) environment.

The complete flow can be seen in the following diagram.

As I already mentioned, not everyone needs to understand the flow of events nor to have a deep understanding of all the components involved in the process. For most users, the only important thing to understand is that pushing changes to Git will result in the execution of the builds defined in jenkins-x.yml pipeline. That’s the beauty of Jenkins X. It simplifies our lives by making complicated processes simple.
The DevOps 2.6 Toolkit: Jenkins X

The article you just read is an extract from The DevOps 2.6 Toolkit: Jenkins X.
The book is still in progress, and I do not yet have a clearly defined scope. I write about tech I’m working with and that interests me the most. Right now, that’s Jenkins X.
You can get the book from LeanPub. If you do, you’ll get updates whenever a new chapter is finished. At the same time, you can get more actively involved and send me your comments, suggestions for the next topics, bug reports, and so on. I’d love to hear from you.

photo: panumas nikhomkhai