Contributed by Nicholas Lotz, Harness
Platform engineering teams steadily adopted sets of common tools as the field matures. Still there remains a tension between using broad open source offerings and proprietary solutions. Amid this push and pull, the ecosystem has come to recognize a “modern continuous delivery stack” that includes the following:
- Kubernetes and container orchestration
- Semantic versioning
- RESTful API’s
- Oauth for access delegation
- … and so on
We expect each of our vendors to interoperate with most of the above technologies. They’ve become standards we expect in our software toolchains.
In this blog, I discuss the value of adding CD events to our toolkit of adopted standards. I’ll then show a practical use case for the CDEvents specification in a modern CD pipeline.
Why a common spec for CD events?
Platform teams increasingly use development methodologies like declarative GitOps to manage deployments. Event-driven architectures are also gaining importance in cloud native software. However most vendors today are proprietary in the way they record CD events like:
- Artifact creation
- Environment and service creation
- New pull request, or a reverted commit
- Automated test outputs
Yet teams practicing continuous deployment heavily rely on event handling. For example, an “artifact is created/updated” event is a typical trigger between CI and CD.
Your continuous integration builds and tests your application, and then publishes the resulting artifact. If you’re practicing pure CD, the arrival of the new artifact might automatically trigger an updated deployment. Or, at least, the publish event might trigger a successful build notification.
It’s a problem though when organizations use disparate systems for build, test, and deploy. There isn’t inherent interoperability between the events offered by a package repository vendor and a CD vendor.
The CD Foundation is effecting change in this area by developing a standard for CD events. The CDEvents specification seeks to solve two problems that plague platform engineers.
- Interoperability. CD tools should be able to send events to each other without layers of translation middleware. A single event payload can be passed to multiple sinks.
- Observability. Teams should be able to unify monitoring and reporting tools to gain visibility into the entire end-to-end CI/CD lifecycle.
To this end, the CDEvents spec introduces a common vocabulary and grammar for CD events. It also defines events in each of six categories:
| Event Category | Description | Example Event |
| Core Events | Events fundamental to pipeline orchestration | pipelineRun started |
| Source Code Control Events | Events related to source code management | change merged |
| Continuous Integration Events | Events related to CI activities like build and test | build finished |
| Testing Events | Events related to manual and automated testing | testSuiteRun started |
| Continuous Deployment Events | Events related to deploying services to target environments | service deployed |
| Continuous Operations Events | Events related to operating deployed services in their environments | incident reported |
For example, starting a pipeline run might trigger an event with the following JSON payload:
{
"context": {
"version": "0.3.0",
"id": "271069a8-fc18-44f1-b38f-9d70a1695819",
"source": "/event/source/123",
"type": "dev.cdevents.pipelinerun.started.0.1.1",
"timestamp": "2023-03-20T14:27:05.315384Z"
},
"subject": {
"id": "mySubject123",
"source": "/event/source/123",
"type": "pipelineRun",
"content": {
"pipelineName": "myPipeline",
"url": "https://www.example.com/mySubject123"
}
}
}In the CDEvents spec, each event record includes unique context metadata along with a subject documenting the event entity itself.
Let’s look at an example of mapping a CD tool’s events to the CDEvents spec.
Example: CDEvents in a CD Pipeline
Consider a pipeline that deploys a simple Kubernetes service called Guestbook. Don’t worry about the details of the application and deployment mechanics in this example. Along with the deployment stage, the pipeline also sends up to two notification webhooks to Slack:
- When a user triggers the pipeline, prompting for an admin’s approval, and
- Details on the pipeline run’s success or failure
In each relevant pipeline stage – Approval Request, CD Failure notification, CD Success notification – we send an event per the CDEvents spec to a handling function. Where the function is hosted doesn’t particularly matter; a cloud lambda would be a good candidate. The handling function parses the event JSON, and then sends a human-readable message via Slack webhook.
How do we then map to the CDEvents spec? While many CD platforms do not yet natively support the CDEvents spec, we can use variables to represent most CDEvent context and subject attributes in our event payload.
For example, the first stage in our pipeline, Approval Request, takes place when a user first tries triggering the pipeline. This is the core pipelineRun started CDEvent. Take a look at the attributes in the CD Foundation’s example. Then in a pipeline configuration, most of the attribute values can be built-in variables native to your CD platform. The example below is a pipeline config and variables in Harness CD.
- step:
type: Http
name: Approval Request
identifier: Approval_Request
spec:
url: <URL of handling function>
method: POST
headers:
- key: Content-type
value: application/json
requestBody:
{
"context": {
"version": "0.3.0",
"id": "<+pipeline.executionId>",
"source": "<+pipeline.repo>”
"type": "dev.cdevents.pipelinerun.started.0.1.1",
"timestamp": "<+pipeline.startTs>"
},
"subject": {
"id": "<+pipeline.identifier>/<+pipeline.executionId>",
"source": "<+pipeline.repo>",
"type": "pipelineRun",
"content": {
"pipelineName": "<+pipeline.name>",
"url": "<+pipeline.executionUrl>" }}}Our handling function parses the JSON payload and uses a couple of the event attributes in a bot-sent approval request to a Slack channel.
Similarly, the end of our pipeline runs one of two conditional events: a success Slack notification if the pipeline succeeds, and a failure Slack notification if the pipeline fails. This maps well to the pipelineRun finished CDEvent. The event payload in our pipeline config can also include reference variables for its attributes.
requestBody:
{
"context": {
"version": "0.3.0",
"id": "<+pipeline.executionId>",
"source": "<+pipeline.repo>”
"type": "dev.cdevents.pipelinerun.finished.0.1.1",
"timestamp": "<+pipeline.stage.slack_notify_cd_success.startTS>"
},
"subject": {
"id": "<+pipeline.identifier>/<+pipeline.executionId>",
"source": "<+pipeline.repo>",
"type": "pipelineRun",
"content": {
"pipelineName": "<+pipeline.name>",
"url": "<+pipeline.executionUrl>",
"outcome": "<+pipeline.stages.deploy_guestbook.status>",
"errors": "<+pipeline.stages.deploy_guestbook.errorSummary>"}}}The pipeline step passes the payload to be parsed by our handling function. The function then notifies of the pipeline run’s success or failure in Slack.
Looking ahead: Toward greater CD Events adoption
Just the two events from the example show the utility of adopting a CD event standard. While this article only showcases a single integration (Slack), we can use the same handlers to pass our events to an arbitrary number of sinks, such as monitoring or logging platforms.
The CDEvents specification ensures our CD workflow passes consistent, useful information for any handlers or event sinks. We encourage you to continue exploring CDEvents as well as other CDF-hosted projects.
We also encourage you to read the CDEvents Whitepaper (PDF).
