Skip to main content
search
Category

Project

Mar 11
Love0

From Spinnaker – Monitoring Spinnaker: SLA Metrics

By Blog, Project

Originally posted on the Spinnaker Community blog, by Rob Zienert, Sr Software Engineer @ Netflix

Long, long ago, in an internet that I barely remember, I wrote about monitoring Orca. I haven’t managed to take the time to write another post about a specific service — it’s a lot of work! Instead of going deep this time around, I want to paint with broader strokes: What are the key metrics we can track that help quickly answer the question, “Is Spinnaker healthy?”

Spinnaker is comprised of about a dozen open source services that may vary widely based on configuration, and as such, there’s no singular metric to rule them all. This makes the question, “Is Spinnaker healthy?” a particularly bothersome question since not all services are equally important. If Igor — the service that is responsible for monitoring CI/SCM systems — is unable to communicate with Jenkins, Spinnaker will be in a degraded state, but its core behavior is still healthy. Should Orca’s queue processing drop to zero, however, it’s time to have an elevated heart rate and quick remedy.

Service Metrics

The Service Level Indicators for our individual services can vary depending on configuration. For example, Clouddriver has cloud provider-specific metrics that should be tracked in addition to its core metrics. For the sake of this post’s length, I won’t be going into any cloud-specific metrics.

Universal Metrics

All Spinnaker services are RPC-based, and as such, the reliability of requests inbound and outbound are supremely important: If the services can’t talk to each other reliably, someone will be having a poor experience.

For each service, a controller.invocations metric is emitted, which is a PercentileTimer including the following tags:

  • status: The HTTP status code family, 2xx, 3xx, 4xx...
  • statusCode: The actual HTTP status code value, 204, 302, 429...
  • success: If the request is considered successful. There’s nuance here in the 4xx range, but 2xx and3xx are definitely all successful, whereas 5xx definitely are not
  • controller: The Spring Controller class that served this request
  • method: The Spring Controller method name, NOT the HTTP method

Similarly, each service also emits metrics for each RPC client that is configured via okhttp.requests. That is, Orca will have a variety of metrics for its Echo client, as well as its Clouddriver client. This metric has the following tags:

  • status: The HTTP status code family, 2xx, 3xx, 4xx...
  • statusCode: The actual HTTP status code value, 204, 302, 429...
  • success: If the request is considered successful
  • authenticated: Whether or not the request was authenticated or anonymous (if Fiat is disabled, this is always false)
  • requestHost: The DNS name of the client. Depending on your topology, some services may have more than one client to a particular service (like Igor to Jenkins, or Orca to Clouddriver shards).
Example of our 24/7 request fanout from Gate. One interesting tidbit: The sudden increase in traffic at 9am is the increased traffic to Clouddriver (bottom) from Chaos Monkey starting its daily light mayhem!

Having SLOs — and consequentially, alerts — around failure rate (determined via the succcess tag) and latency for both inbound and outbound RPC requests is, in my mind, mandatory across all Spinnaker services.

As a real world example, the alert Netflix uses for Orca to all of its client services is:

nf.cluster,orca-main.*,:re,
name,okhttp.requests,:eq,:and,
status,(,Unknown,5xx,),:in,:and,
statistic,count,:eq,:and,
:sum,
(,nf.cluster,),:by,
0.2,:gt,3,
:rolling-count,3,:ge

So, for people who can’t read Atlas expressions, if we have more than 0.2 failing/unknown RPS to a specific service over 3 minutes, we’ll get an alert.

Service-specific Metrics

Most of our services have an additional metric to judge operational health, but in/out RPC monitoring will go far if you’re just starting out.

  • Echo
    echo.triggers.count tracks the number of CRON-triggered pipeline executions fired. This value should be pretty steady, so any significant deviation is an indicator of something going awry (or the addition/retirement of a customer integration).
    echo.pubsub.messagesProcessed is important if you have any PubSub triggers. Your mileage may vary, but Netflix can alert if any subscriptions drop to zero for more than a few minutes.
  • Orca
    task.invocations.duration tracks how long individual queue tasks take to execute. While it is a Timer, for an SLA Metric, its count is what’s important. This metric’s value can vary widely, but if it drops to zero, it means Orca isn’t processing any new work, so Spinnaker is dead in the water from a core behavior perspective.
  • Clouddriver: Each cloud provider is going to emit its own metrics that can help determine health, but two universal ones I recommend tracking are related to its cache.
    cache.drift tracks cache freshness. You should group this by agent and region to be granular on exactly what cache collection is falling behind. How much lag is acceptable for your org is up to you, but don’t make it zero.
    executionCount tracks the number of caching agent executions and combined with status , we can track how many specific caching agents are failing at any given time.
Here, one collection for a specific AWS service in our largest region was getting stale. In this case, while AWS availability was fine for Clouddriver, Edda was having trouble refreshing.
It’s OK that there are failures in agents: As stable as we like to think our cloud providers are, it’s still another software system and software will fail. Unless you see sustained failure, there’s not much to worry about here. This is often an indicator of a downstream cloud provider issue.
  • Igor
    pollingMonitor.failed tracks the failure rate of CI/SCM monitor poll cycles. Any value above 0 is a bad place to be, but is often a result of downstream service availability issues such as Jenkins going offline for maintenance.
    pollingMonitor.itemsOverThreshold tracks a polling monitor circuit breaker. Any value over 0 is a bad time, because it means the breaker is open for a particular monitor and it requires manual intervention.

Product SLAs at Netflix

We also track specific metrics as they pertain to some of our close internal customers. Some customers care most about latency reading our cloud cache, others have strict requirements in latency and reliability of ad-hoc pipeline executions.

In addition to tracking our own internal metrics for each customer, we also subscribe to our customers’ alerts against Spinnaker. If internal metrics don’t alert us of a problem before our customers are aware something is wrong, we at least don’t want to wait for our customers to tell us.

Continued Observability Improvements

Since Spinnaker is such a large, varied system, blog posts such as these are fine, but really are meant to get the wheels turning on what could be possible. It also highlights a problem with Spinnaker today: A lack of easily discoverable operational insights and knobs. No one should have to rely on a core contributor to distill information like this into a blog post!

There’s already been a start to improving automated service configuration property documentation, but something similar needs to be started for metrics and matching admin APIs as well. A contribution that documents metrics, their tags, purpose and related alerts would be of huge impact to the project and something I’d be happy to mentor on and/or jumpstart.

Of course, if you want to get involved in improving Spinnaker’s operational characteristics, there’s a Special Interest Group for that. We’d love to see you there!

Mar 09
Love0

From Spinnaker – Future of SRE: Robert Keng Builds a DeploymentBot #withSpinnaker

By Blog, Project

Originally posted on the Spinnaker Community blog, by Rosalind Benoit

Coming soon from Chime to OSS, a software delivery chatbot which uses Slack to deploy apps via Spinnaker

Last month I had the pleasure of chatting with Robert Keng, a Lead SRE at Chime, about a Slack-integrated ChatBot he recently built to facilitate lightweight, direct deployments for developers. Chime’s continuous delivery is based on Spinnaker, driven with signal-based GitOps. Via pipelines, merged release branches are auto-deployed from a continuous integration (CI) solution, through QA to production with no human interaction interaction.

However, it hasn’t always been this way; Chime has roots in a legacy build environment, largely for Ruby-on-Rails development. It’s based on configuration management tools such as Salt, and thus not containerized, but pointed at long-lived infrastructure. So, containerization formed an important milestone in Chime’s continuous delivery adoption. Luckily, according to Robert, its high-trust, growth minded culture and workflows have supported the evolution.

Chime’s culture also provides flexibility that highlights Spinnaker’s power to accelerate digital transformation. Robert explains that, in some instances, it makes sense for developers to deploy straight to a test environment, bypassing CI. When adding a small feature to a mobile app, for example, I might want to bypass CI wait time to deploy and experiment with behavior (raise your hand if you‘ve built an app and never done that…didn’t think so!)

Meeting Chime devs where they’re @

“We’re cutting the straight-to-prod patch fix deployments down to zero,” Robert clarifies, and he’s done it by creating a flexible system with Spinnaker that models Chime’s culture of trust. At any time, if the devs he enables would rather execute commands in Slack to deploy branches to environments of their choosing, they can. Robert has created a tool that allows them that agency, while empowering them to address complex use cases, for example, adding logic into the Slack commands to deploy dynamic environments into different Kubernetes clusters. In production, “If we need to scale customers on the Z-axis, and build multiple app versions with different backends to target different service providers” as deployment targets, with Spinnaker, Chime can. Robert points out:

“Spinnaker offers a lot of agility in that respect. It would be hard to accommodate gitOps and chatOps in the same place without it.”

In a prime example of the opportunities to solve that Spinnaker provides as a platform, Robert has created a golden path which allows Chime’s teams to iterate in a safe environment. To create it, Robert analyzed workflows as they are and designed an alternative workflow that mapped what he observed in Spinnaker. This, combined with the auto-deploy strategy, tells the story, written in pipelines, of how Chime engineers deliver software. This way, as an SRE, he can rely on automated guardrails for safety regardless of the deployment path. As Kelsey Hightower says, it “serializes the culture in tools” in a way that’s seamless, painless, and purposefully abstracted.

Because at the end of the day, it’s not about the tools. It’s about your story, which in Chime’s case, is all about changing the way people feel about banking. What products and services do you delight your customers with? What’s your story? You can tell it #withSpinnaker

One DeploymentBot, Headed for OSS Spinnaker

The tool, in a multi-service design, has a component which handles the request/response communication with Slack, a frontend that leverages Okta user groups to control who can access Spinnaker, and a Python backend which processes the request data in batches. This architecture evolved from using webhooks to, at Armory’s suggestion, using client certs for faster authentication, and from a monolith version to microservices, because of constraints encountered in the bot’s development. The top constraint: the Slack Events API’s requirement that a response from requests arising from message actions be received within 3 seconds.

This constraint presented challenges in actions like querying Vault for certificates to authenticate against Spinnaker, and even in token exchange with Slack. Breaking the chatbot into pieces allowed Robert to create a responsive, extensible service to deliver a full-featured experience for Chime devs. “It’s turned into a monster,” he grins. “I have tons of feature requests for additional functionality already” (because his devs love using it).

Next steps for Robert’s Bot include developing it against the entire Spinnaker API to leverage all features available, and adding more dynamic capability. He wants to enable devs to use the bot to deal with existing pipelines and executions, and adjust parameters and other configuration via a scripted payload directly from Slack.

Another important next step? Open-sourcing the DeploymentBot! Robert’s very busy with projects right now (read more below), but I’ll hook him up with support from Armory engineers, if needed, to help get this invention to the masses.

The Future of Site Reliability, Platforms, and DevOps Engineering

As he describes his plans for the Bot, we start talking about the myth of NoOps. I have my own words about the opportunities and fallacies of Dev + Ops, but here, Robert’s voice speaks for itself:

“My team isn’t DevOps, it’s SRE (Site Reliability Engineering). DevOps is just part of what we do. As tech stacks mature, we’re seeing less dependency on direct hardware interaction, but that doesn’t mean the management complexity goes away; it actually gets worse. Here’s an easy example: We have this awesome thing called Kubernetes. Given config maps and secrets, where is the source of truth? Ask anyone in the community, and they’ll say, ‘Umm…build it yourself!’ I know Hashicorp released a sidecar method to inject values, but none of that is complete. This is why there’s a lot of custom work in the community, and companies are building their own mutating webhook controllers, for example, which is what we’re doing. You can’t buy this stuff, because it doesn’t exist.

We have our own way of injecting Vault secrets which 100% bypasses Kubernetes stuff, because we can’t version it, and we can’t manage it from any source or truth, as it’s scattered across 1000 namespaces. It’s impossible to manage in one place. So in our environment, we put everything in Vault, whether it’s configuration, or secrets. That gives us a common interface to code against. In V1, we’re using init containers, which is exactly what Hashicorp’s sidecar does. In V2, depending on the environment, we’ll grab values from different Vault clusters, since storing production and non-production values in the same place is just, suicide. You’ll get a huge ban hammer from your security team, and no-one wants that.

So we’re building, and we’re operating it at the same time. And are developers ever going to touch these [tools]? No! There are a lot of these instances in Kubernetes where things just don’t exist, so what do you do?Same thing for, EC2, and ECS even. Then, moving into Knative, and Lambas, and serverless computing and functions, it’s even worse. It’s a free-for-all. We’re designing our own framework.

The next thing we’re looking at is building plugins that will plug in our code, and use Spinnaker to deploy it [on that infra]. I heard Armory is working on something similar for deploying Lambdas, and I’m desperately waiting, because it’s going to make my life easier. Functions in general are kind of useless. The ecosystem around them is more important; you’ve got to think about API gateways, API management, queues, load balancers, etc. How do I wrap that into a sane framework where we can consistently build, integrate, test, and deploy? I don’t want to use 10 different ways to do the same thing. I’d rather just have everything work in Spinnaker.”

Then when we start talking about making that happen. I tell Robert about the Community Gardening Days I’m planning for Spinnaker this Spring (keep your eyes peeled! Announcement forthcoming on Spinnaker.io and social), and he gets psyched about Chime’s involvement. Music to my ears!

Look out for more articles from me on the Spinnaker developer and contributor experience. I’ll shine a light on the way Open Source Heroes like Robert are getting into the ecosystem as they enable the delivery of software products and services. Hang on, the latest industrial revolution (where software truly changes the freaking world for the better!) is just taking off.

Please share this on Twitter, LinkedIn, and HackerNews and give Robert some glory : )

Mar 06
Love0

From Spinnaker – Open your first issue to get started as a contributor

By Blog, Project

Originally posted on the Spinnaker Community blog, by Rosalind Benoit

Why are most periodicals so sad? Because we have too many issues!

A problem the Spinnaker community currently ISN’T having ^^

I’d like to start sharing one IC’s experience (mine : ) dropping anchor into the Spinnaker ecosystem. I found the community last year before joining the tribe at Armory, while doing research for another continuous delivery product built on Tekton. First step: join Spinnaker slack, and behold the community live, with active SIG channels, newcomers, and operators constantly pinging to discuss what they encounter in the platform.

What’s the next step? Personally, I began creeping on Spinnaker.io, joined the Docs SIG which maintains the site, and began to engage by commenting on and submitting web-dev PRs in the site’s repository to get the ball rolling. Next up: get my full Spinnaker dev environment set up, and document that process for y’all!

If you’re an end-user, your path may look different: you may have used Spinnaker to deploy an application, and encountered a usability issue or found something not quite right from your perspective. In this case, reaching out on Slack in the #general or relevant SIG channel, or filing an issue in the spinnaker/spinnaker repository describing your observation is the next step. As a final note to end-users, having spoken to Andy Glover and a few TOC members about this at some length, I can say on good authority:Why end-users and new contributors should submit issues to the project

Why end-users and new contributors should submit issues to the project

Push past your fear in filing an issue! At this stage in its growth, the “too many issues” problem doesn’t exist. We’ll skip across that bridge when we come to it. Now, we need your feedback to make the most mature and production-ready continuous delivery platform the BEST platform on the planet. Don’t be shy!

Operators will also follow their own path to begin contributing. Perhaps you’ve found a great growth opportunity for the codebase as you’ve hacked through workflows. Maybe you’ve developed a rockin’ integration to solve for interoperability at your organization, and you know it may benefit others in the community. Or, that small tweak made to your organization’s Spinnaker instance has improved usability, but hasn’t been addressed in the community.

YES, your time is precious, but I urge you, don’t wait! Sharing your contributions will help the ecosystem, and it will also help you. It puts you on the map. It adds gravitas to your resume. It exposes you to peer recognition, and provides networking with some teeth, as your community footprint will speak for itself. Better yet, if you start a fix or conversation, others in the community can advise, or build on what you started, helping you solve faster.

If you’d like to add a feature to Spinnaker, that should start as a discussion, so file an issue in the spinnaker/spinnaker repository describing the purpose and proposed implementation, or start the discussion at a SIG meeting. Got something smaller-scale, like a bug fix? The Submitting A Patch page on spinnaker.io provides guidance. Integrating another service, or building an extension? Check out the Plugin Users Guide, as utilizing the new plugin framework allows you to maintain plugin code in a separate repository and avoid the requirement of loading extensions at Spinnaker runtime.

If that’s a bit overwhelming for now, don’t despair. Getting started is the first step. Noticed something confusing in the docs, or have a suggestion for spinnaker.io? Have ideas on what kinds of Contributor Experience materials would help you move forward? Please ping us in #SIG-Documentation or file an issue in spinnaker/spinnaker.github.io!

That’s all for today, but stay tuned for more N8B Diaries as I work to guide Spinnaker operators in contributing their inventions, and set up my own environment. High-five your imposter syndrome and become a *real* Spinnaker contributor with me!

Mar 04
Love0

From Screwdriver – Improvements and Fixes

By Blog, Project

Originally posted on the Screwdriver blog, by the Screwdriver Team from Verizon Media

UI

  • Enhancement: Upgrade to node.js v12.
  • Enhancement: Users can now link to custom test & coverage URL via metadata.
  • Enhancement: Reduce number of API calls to fetch active build logs.
  • Enhancement: Display proper title for Commands and Templates pages.
  • Bug fix: Hide “My Pipelines” from Add to collection dialogue.
  • Enhancement: Display usage stats for a template.
image

API

Store

Compatibility List

In order to have these improvements, you will need these minimum versions:

  • UI – v1.0.491
  • API – v0.5.851
  • Store – v3.10.5

Contributors

Thanks to the following contributors for making this feature possible:

Questions and Suggestions

We’d love to hear from you. If you have any questions, please feel free to reach out here. You can also visit us on Github and Slack.

Jan 28
Love0

Tekton in 2020 and beyond!

By Blog, Project

This blog post has been written by the owners of the different projects, and in particular, huge thanks to Christie Wilson, Andrea Frittoli, Adam Roberts and Vincent Demeester!

At the end of last year Dan wrote the blog post: A Year of Tekton. It was a great retrospective on what happened since the bootstrap of the project; a highly recommended read! Now that we’re getting into the swing of 2020, let’s reflect again back on 2019 and look forward to what we can expect for Tekton this year!

Tekton in 2019

We can safely say 2019 (more or less the project’s first year!) was a great year for Tekton. Just like a toddler we tried things, sometimes failed and learned a lot; we are growing fast!

The year 2019 saw 9 releases of Tekton Pipelines, from the first one (0.1.0) to the latest (0.9.2). We shared the work of creating the releases as much as possible, though many contributors are behind the work in each!!

If you are curious about the naming of the release starting from 0.3.x, we decide to spice things up a bit and name our release with a composition of a cat breed and a robot name (in reference to our amazing logo, a robot cat).

Chip, our 0.5.0 mascot!

Aside from the initial project (tektoncd/pipeline), we bootstrapped a bunch of new projects:

  • tektoncd/cli: This project aims to provide an easy to use command line interface to interact with the tekton components. As Tekton objects are Kubernetes components you can always interact with them via the Kubernetes CLI — kubectl, but the kubectl experience can be very ‘raw’ and not very focused. The `tkn` CLI has the ambition to provide an easy to use user experience without having to know anything about kubectl (or Kubernetes for that matter).
  • tektoncd/dashboard: Alongside the CLI project, the Tekton Dashboard provides a user interface for the Tekton components, in a browser. It allows users to manage and view Tekton PipelineRuns and TaskRuns and the resources involved in their creation, execution, and completion.
  • tektoncd/catalog: Tekton pipeline is designed to provide highly shareable objects (Task, Pipeline, Condition, …), so creating a repo to store a catalog of shared Tasks and Pipelines came naturally!
  • tektoncd/experimental: With growing interest in Tekton came a growing number of “feature requests”. In order to be careful about how we expand the scope of Tekton pipeline while still allowing contributors to experiment, we created this repository to allow experiments to happen more easily. Experiments can graduate with enough traction. The biggest project so far is the webhooks extension which combines using the Dashboard project and Triggers to allow users to create webhooks for Git that trigger PipelineRuns.
  • tektoncd/operator: This project aims to provide an operator to manage installation, updation and uninstallation of tektoncd projects (pipeline, …). It has yet to be published in the community OperatorHub.
  • tektoncd/triggers: And speaking of the experimental repo, we have Triggers which started its life there! This project provides lightweight event triggering for Pipelines. 

Looking forward into 2020 🔮

We’ve come a long way but we’ve got more to do! Though we can’t predict what will happen for sure, here is a preview of what we’d like to make happen in 2020!

Beta API, GA

As you’ve seen, we’ve made a lot of changes! Going forward we want to make sure folks who are using and building on top of Tekton can have more stability guarantees. With that in mind, we are pushing for Tekton Pipelines to have a beta release early in 2020. If you are interested in following along with your progress, please join the beta working group! Or keep an eye on our Twitter for the big announcement.

Once we’ve announced beta, users should be able to expect increased stability as we’ll be taking our lead from kubernetes and mirroring their deprecation policy, for example any breaking changes will need to be rolled out across 9 months or 3 releases (whichever is longer).

And once we get to beta, why stop there? We’d love to be able to offer users GA stability as soon as we possibly can. After we get to beta, we’ll be looking to progress the types that we didn’t promote to beta (e.g. Conditions), add any important features we don’t yet have (we’re looking at you on failure handling and  “pause and resume” aka “the feature that enables manual approval”!), and then we should be ready to announce GA!

Task Interfaces and PipelineResources

Speaking of types that won’t be going beta right away: PipelineResources! PipelineResources are a type in Tekton that is meant to encapsulate and type data as it moves through your Pipelines, e.g. an image you are building and deploying, or a git commit you’re checking out and building from.

This concept was introduced early on in Tekton and bares a close resemblance to Concourse resources. However as we started trying to add more features to them, we started discovering some interesting edges to the way we had implemented them that caused us to step back and give them a re-think. Plus, some folks in our community asked the classic question “why PipelineResources” and we found our answer wasn’t as clear as we’d like!

As we started down the path of re-designing, and re-re-designing again, we started to get some clarity on what exactly it was we were trying to create: the interface between Tasks in a Pipeline! And thanks to a revolutionary request to improve our support for volumes, we finally feel we are on the right path! The next steps along this path are to add a few key features, namely the concept of workspaces (i.e. files a Task operates on) and allow Tasks to output values (aka “results”).

Once we have these in place we’ll revisit our designs and our re-designs.

tekton.dev

Hand in hand with our beta plans, we’re revamping our website! Soon at tekton.dev you’ll be able to find introductory material, tutorials, and versioned docs.

The Tekton Catalog

Besides making it easy for folks to implement cloud native CI/CD, one of the most important goals of Tekton is for folks to be able to share and reuse the components that make up your Pipeline. For example, say you want to update Slack with the results of a Task – wouldn’t it be great if there were one battle tested way to do that, with a clean interface?

That’s what the Tekton catalog is all about! To date we’ve received more than 20 Tasks from the community to do everything from running Argo CD to testing your configuration with conftest.

But there’s so much more we want to do! We want to offer versioning and test guarantees that can make it painless for folks to depend on Tasks in the Catalog – and for companies to create Catalogs of their own.

Plus, the Catalog is a great place for us to build better interoperability even between the Tekton projects, for example with the Task that runs tkn (the Tekton CLI).

Shout outs 😻

A community is nothing without its users, contributors, adopters and friends, so we want to explicitly shout outs to our community for their tremendous effort and support in 2019 and hopefully even more in 2020.

Friends

We’ve gained friends and more are always welcome! Our current list of “known friends” includes:

  • Alibaba
  • Ant Financial
  • Cloudbees
  • Elastic
  • Google
  • IBM
  • Puppet
  • Red Hat
  • TriggerMesh

We welcome friend requests! Please submit a PR to https://github.com/tektoncd/friends, this repository acts as a place that allows members of the ecosystem (known as “Tekton Friends”) to self-report in a way that is beneficial to everyone. We’d love to have you as a friend if your company is using Tekton and/or contributing to it 😀

Projects

Adoption of Tekton has grown and became a part of both free and commercial offerings by various companies, demonstrating that Tekton’s valuable and ready for anything

In mid-2019, Puppet launched a new cloud-native CD service called Project Nebula that’s built on Tekton Pipelines. It provides a friendly YAML workflow syntax and niceties like secrets management and a spiffy GUI on top of Tekton instance running in GCP. To coincide with the public beta of Nebula, Scott Seaward keynoted at the Puppetize PDX user conference to talk about how Tekton works under the hood. Since then, the Nebula team has contributed several PRs to the Pipelines repo and are looking forward to working on step interoperability, triggers, and other awesome upstream features in 2020.

Other notable examples include:

Talks, Tweets and Moar 📣

It has been such a privilege to see more and more people get excited about Tekton and share it with the world! Here are some (but not all!!) of the great talks and tweets we saw about Tekton in 2020, not to mention our Tekton contributor summit!

Tara Hernandez recommends standards over specifications at DevOps World

Contributors

Huge shoutouts to all of the tektoncd projects contributors ❤️❤️❤️:

If you are even more curious on the contributions happening in the tektoncd project, you can visit the tekton.devstats.cd.foundation site (e.g. a page showing the overall contributors on all tektoncd projects).

Tekton birds of a feather at Devops Days Bangalore, India

Join us!

If you are interested in contributing to Tekton, we’d love to have you join us! Every tektoncd project has a CONTRIBUTING.md that can point you in the right direction, and our community contains helpful links and guidelines. Feel free to open issues, join slack, or pop into one of our working groups! Hope to see you soon 😀

Jan 27
Love0

Fair Winds & Following Seas: Another Year of Safe, Reliable CD #withSpinnaker

By Blog, Project

It’s 2020, so what’s our New Year’s Resolution?  #1, Make Spinnaker the Perfect Continuous Delivery Platform. 

🤦 Voltaire said “perfect is the enemy of good,” and we’ve seen some resolution-minded ads lately reviving that adage (I’m thinking of Michael Phelps reminding me that Progress IS perfection : )  Striving for perfection in software development can lead to obsolete products. So, we hack. We listen to our users and iterate. When we do that as a community, we can progress towards something truly brilliant. Spinnaker’s progress was perfection in 2019, and by all accounts will exceed that trajectory in 2020. 

Enterprise Adoption Crescendo (of Production workloads)

Spinnaker saw promising early adoption from large companies like Target and Adobe, and this year has been no exception. While literally everyone books stays on the site, oblivious to digital transformation, Airbnb is using Spinnaker to migrate from monolith to service-oriented architecture, and from brittle deployments to continuous delivery. SAP joyfully leverages Spinnaker on its mission to run the world better, and Pinterest uses it to boost productivity as it pioneers visual discovery. Transunion stays ahead of the fintech curve, providing total credit protection through applications it now deploys with Spinnaker, a more full-featured fit for ephemeral infrastructure than its previous Ansible solution.  Companies like Comcast, going all-in on Kubernetes as a software-defined datacenter, have added Spinnaker to manage deployment pipelines. Meanwhile, Salesforce has adopted Spinnaker to bake images based on both Helm charts for Kubernetes and Packer templates for VMs, to support its complex software delivery requirements.

In 2019, we proudly welcomed engineers from new enterprises, including JPMorgan Chase and Home Depot, into the Spinnaker community. Now more than 175 companies have contributed to the project, with over 200 new ICs just last year, and many more companies have become key stakeholders, using and extending Spinnaker. These demonstrate Spinnaker as the mature CD solution, proven to handle production workloads flexibly and scalably. 

Organic Growth Through Governance

As adoption continues to rise, and our community grows, it becomes crucial to create a growth-adapted project space. A transparent structure for building and maintaining the project invites new companies and users to take an active role in shaping Spinnaker’s future. To that end, 2019 saw the governance process and entities created in 2018 codified in GitHub, along with a process for modifying it via PRs and votes from members of the TOC and Steering Committee. Spinnaker governance also blossomed into an active space encompassing 8 community-initiated SIGs which organize contributors around feature growth and maintenance in areas of interest. SIGs welcome anyone to join, and we saw growing attendance from end-user companies in H2 2019. As the TOC experiments with public Open Office Hours, Spinnaker Slack is always open, and welcomes nearly 8500 participants to troubleshoot, chat with a SIG team, or reach out to a community member any time. Coupled with the donation of the project to the CDF, these growth factors signal the founding of Spinnaker as a neutral, democratized project space. Our goal? Fuel rapid innovation as we work to empower humanity to deliver their innovations, faster.

#BetterTogether: Driving SDLC Ecosystem Innovation

What came out of this investment in 2019? Where to begin…! An OSS ecosystem thrives with modular components that allow operators to optimize for business goals and maintain compliance. As our user base grows, the problem set expands, use cases vary, and we innovate across a richer toolchain. This allows us to create a smarter, more automated experience. Case(s) in point: 

  • New data sources added for SignalFX and New Relic, to inform Automated Canary Analysis decisions that let app owners sleep instead of being paged. 
  • A new Gremlin integration allowing chaos experimentation in Spinnaker pipelines will expand in 2020 to provide results useful for automated decision support. 
  • Integrations with artifact repositories Nexus and JFrog’s Artifactory have added new native triggers for Spinnaker pipelines. 
  • New end-to-end secrets management dynamically decrypts Spinnaker secrets as needed for validation and deployment from a backing store of your choosing, such as Vault or S3.

Since interoperability is crucial to Spinnaker, implementing a reliable plugin system was a key 2019 milestone. As our community leverages Spinnaker to solve problems, we must remove friction from the dev’s experience in contributing those extensions to the project. A plugin framework provides libraries and application context to devs, and defines clear extension points to start from when integrating something new. In 2019, we adopted PF4J as our backend plugin-loader framework. In 2020, we’ll implement plugin loading in the Spinnaker UI, and foster community around building and sharing plugins, to enrich our ecosystem.

Cloud Providers – Raining Champions  : P

Spinnaker depends on cloud provider investment in extending the project for deployment to the ever-growing variety of ephemeral infra solutions. In 2019, engineers at Google developed a blueprint for a production-ready Spinnaker instance on GKE, integrated with Google Cloud services such as Cloud Build. Amazon Engineers have extended cloud providers for AWS services, ensuring that we can deploy with Spinnaker to any attribute available in Fargate or ECS (Elastic Container Service). As of this year, that includes any task definition attribute. AWS also added full support for deployments to serverless applications using AWS Lambda, including the ability to use Lambda functions as ALB targets.

Migrating to the cloud alleviates headaches, while bringing new operational challenges. Spinnaker evolves to capture and solve for these new challenges as we encounter them.  The extendable Swabbie service, created in 2019, tackles the tedium (and potential nightmare at scale!) of reaping unused resources programmatically, to help optimize cloud spend. With Swabbie, an operator can set rules for cleanup candidates via YAML, and clean resources according to a configurable schedule. Deployments to highly automated cloud environments prompted enablement of dynamic account creation, discovery, and configuration for Cloud Foundry and Kubernetes accounts in the cloud.

Upleveling Functionality = Perfect Progress 

The Kubernetes V2 Provider for Spinnaker also came into its own last year, offering the ability to deploy, delete, scale, and roll back K8S manifests as artifacts managed as code. The Kubernetes SIG iterated fast to improve the V2 user experience by surfacing more kubectl commands in the Spinnaker UI, and improving management of rollout strategies. They also enhanced traffic management to enable more deployment patterns with the provider, such as blue/green (AKA red/black) and dark canary. In 2020, simplifying the Kubernetes developer experience is an important roadmap element, and the community will tackle it by visualizing more K8S resources in the Spinnaker UI, and improving terminology, error, and workflow management.

Under the hood, 2019 saw lots of effort to provide operators the option to back Spinnaker with a MySQL database instead of Redis. Stateful data in Spinnaker enables event routing and orchestration for pipelines, integrated CI and SCM events, and Swabbie cleanup notifications. The choice of whether to use a relational DB or in-memory store to manage that data gives operators the freedom to optimize performance for their workloads and infrastructure. This makes all that effort, which required updating several microservices, including Echo, the eventing service, and Orca, the orchestration engine, well worth it. Likewise, updates to the Authorization model have allowed even more granular permissions to be durably API-driven across the platform.

A Bright Future Won’t Blind Us (to Your Story)

One high-level 2020 goal aims to better incorporate user stories and enterprise use cases into Spinnaker’s trajectory. The steering committee has committed to building a roadmap that tells high-level stories about using Spinnaker to solve problems. Tool chain interoperability, notably with Kubernetes, cloud providers, and monitoring systems figure large in the H1 2020 Roadmap. Managed Delivery, an exciting Spinnaker CD initiative incubating at Netflix, uniquely responds to a common narrative around software delivery. It uses declarative automation to alleviate the operational knowledge and maintenance burden that comes with ownership of modern, continuously delivered applications.

Users can help us tell the best Spinnaker stories by submitting comments and issues describing usage and business context. Please visit Spinnaker.io (which the Docs SIG will overhaul in 2020) and check out our Success Stories page. Join us on Spinnaker Slack or in the Spinnaker org and tell your tale! 

Jan 16
Love0

Happy Second Birthday Jenkins X!

By Blog, Project

Originally posted in the Jenkins X blog by Rob Davies

Second year of Jenkins X

The Jenkins X project started the beginning of 2019 by celebrating its first birthday on the 14th January, a big event for any open source project, and we have just celebrated our 2nd – hooray!

image by Ashley McNamara, creative commons license

Two Years of Jenkins X!

Jenkins X has evolved from a vision of how CI/CD could be reimagined in a cloud native world, to a fast-moving, innovative, rapidly maturing open source project.

Jenkins X was created to help developers ship code fast on Kubernetes. From the start, Jenkins X has focused on improving the developer experience. Using one command line tool, developers can create a Kubernetes cluster, deploy a pipeline, create an app, create a GitHub repository, push the app to the GitHub repository, open a Pull Request, build a container, run that container in Kubernetes, merge to production. To do this, Jenkins X automates the installation and configuration of a whole bunch of best in breed open source tools, and automates the generation of all the pipelines. Jenkins X also automates the promotion of an application through testing, staging, and production environments, enabling lots of feedback on proposed changes. For example, Jenkins X preview environments allow for fast and early feedback as developers can review actual functionality in an automatically provisioned environment. We’ve found that preview environments, created automatically inside the pipelines created by Jenkins X, are very popular with developers, as they can view changes before they are merged to master.

Jenkins X is opinionated, yet easily extensible. Built to enable DevOps best practices, Jenkins X is designed to the deployment of large numbers of distributed microservices in a repeatable and manageable fashion, across multiple teams. Jenkins X facilitates proven best practices like trunk based development and GitOps. To get you up and running quickly, Jenkins X comes with lots of example quickstarts.

Highlights of 2019

February 2019: The rise of Tekton!

In the second half of 2018, Jenkins X embarked on a journey to provide a Serverless Jenkins and run a pipeline engine only when required. That pipeline engine was based on the knative build-pipeline project which evolved into Tekton with much help and love from both the Jenkins and Jenkins X communities. The Jenkins X project completed its initial integration with Tekton in February 2019. Tekton is a powerful and flexible kubernetes-native open source framework for creating CI/CD pipelines, managing artifacts and progressive deployments.

March 2019: Jenkins X joined The Continuous Delivery Foundation!

Jenkins X joined the Continuous Delivery Foundation (CDF) as a founding project alongside Jenkins, Spinnaker, and Tekton. Joining a vendor-neutral foundation, focused on Continuous Delivery, made a lot of sense to the Jenkins X community. There had already been a high level of collaboration with the Jenkins and Tekton communities, and there have been some very interesting and fruitful (in terms of ideas) discussions about how to work better with the Spinnaker communities also.

June 2019: Project Lighthouse

When Jenkins X embarked on its serverless jenkins journey, it chose to use Prow, an event handler for GitHub events and ChatOps. Prow is used by the Kubernetes project for building all of its repos and includes a powerful webhook event handler. Prow is well proven, but heavily tied to GitHub, and not easily extendable to other SCM providers. At the end of June 2019, work commenced on a lightweight, extensible alternative to Prow, called Lighthouse. Lighthouse supports the same plugins as Prow (so you can still ask via ChatOps for cats and dogs) and the same config – making migration between Prow and Lighthouse extremely easy.

June 2019: Jenkins X Boot!

We were very busy in June – a frantic burst of activity before summer vacations! One common problem Jenkins X users were facing was the installation of Jenkins X on different Kubernetes clusters. Installing services, ensuring DNS and secrets are correct, and done in the right order is completely different from vendor to vendor, and sometimes cluster to cluster. We realised that to simplify the install, we really needed a pipeline, and whilst this may sound a little like the plot to a film, running a Jenkins X pipeline to install jx really is the best option. The jx boot command interprets the boot pipeline using your local jx binary. The jx boot command can also be used for updating your cluster.

As part of the move to the CDF the Jenkins X project took the opportunity to redesign its logo. An automaton represents the ability of Jenkins X to provide automated CI/CD on Kubernetes and more!

Second half 2019: Big focus on Stability and Reliability

The Jenkins X project has been fast paced with lots of different components and moving parts. This fast pace unfortunately led to some instability and a growth of serious issues that risked undermining all the great work there had been on Jenkins X. There has been a concerted effort by the community to increase stability and reduce outstanding issues – the graph below shows the trend over the last year, with a notable downward trend in the number of issues being created in the last 6 months.

CloudBees also aided this effort by introducing the CloudBees Jenkins X Distribution with increased testing around stabilized configurations and deployments and regular releases every month.

October 2019: Jenkins X Steering Committee inaugural meeting

The Jenkins X Bootstrap Steering Committee is tasked with organising the transition to an elected steering committee, as well as determining what responsibilities the steering committee will have in guiding the Jenkins X project.

December 2019: First Jenkins X Outreachy mentee!

Neha Gupta is adding support for Kustomize in Jenkins X, to enable Kubernetes native configuration management, while participating in Outreachy from December 2019 to March 2020. We welcome Neha’s work on Jenkins X and look forward to building on our culture of continuous mentoring!

Jenkins X usage in Products

The easiest way to try out Jenkins X is undoubtedly through CloudBees CI/CD powered by Jenkins X which provides Jenkins X through the convenience and ease of use of SaaS. No cluster setup, no Jenkins X install, that is all done for you! Currently, CloudBees CI/CD powered by Jenkins X is available for preview. Sign up here to try out the new Jenkins X Saas!

What’s next?

The Jenkins X project is going to be encouraging the community to get involved with more innovation. There are a lot of great ideas to extend the continuous story with integrated progressive delivery (A/B testing, Canary and Blue/Green deployments) and Continuous Verification, alongside more platforms support. We are expecting lots of awesome new features in the CloudBees UI for Jenkins X too.

Expect lots more exciting new announcements from Jenkins X in 2020!

Jan 14
Love0

From Jenkins: Happy New Year! 2019/2020 edition

By Blog, Project

Originally published in the Jenkins blog by Oleg Nenashev

Jenkins project congratulates all users and contributors with the New Year! Let’s take a look at some changes this year.

NewYear

Highlights

If you are interested to know more about Jenkins features introduced in 2019, stay tuned for a separate blog post about it (coming soon!).

Project updates

Highlights above do not cover all advancements we had in the project. Below you can find slides from the Jenkins contributor summit in Lisbon. There we had project updates by officers, SIG and sub-project leaders. See the slide deck to know about: Jenkins Core, Pipeline, Configuration-as-Code, Security, UX Overhaul, Jenkins Infrastructure, platform support and documentation.

Some stats and numbers

If this section seems to be too long for you, here is some infographic prepared by Tracy Miranda. As you may see, Jenkins is pretty big 🙂

Jenkins 2019 in numbers

Community. Over the past year we had 5433 contributors in GitHub repositories (committers, reviewers, issue submitters, etc.). We had 1892 unique committers who created 7122 pull requests and 45484 commits, bots excluded. Contributors represent 273 companies and 111 countries, 8% of contributors are recognized as independent. The most active repositories were Jenkins Core and jenkins.io. The most active month was October 2019 when we reached the record high number of contributions: 915 unique contributors, 124 of them were first-timers, thanks to Hacktoberfest!.

Jenkins core. In 2019 Jenkins core had 54 weekly and 13 LTS releases with several hundreds of notable fixes/enhancements. There was a login screen extensibility rework, many update manager and administrative monitors improvements. We also introduced support for user timezones, not speaking of emojis support 🥳. There was also a lot of housekeeping work: better APIs, codebase refresh, cleaning up static analysis warnings and removing deprecated features like Remoting CLI. The core’s components also got major updates. Only Jenkins Remoting got 11 releases with stability improvements and new features like support of inbound connections to headless Jenkins masters. There are also major incoming features like JEP-222: WebSocket Services support, UI look&feel updates, JENKINS-12548: Readonly system configuration support, Docker images for new platforms like Arm. To facilitate further changes we created a new Core pull request reviewers team and added 9 contributors there.

Plugins. There were 2654 plugin releases, and 157 NEW plugins have been hosted in the Update Center. Jenkins ecosystem got a lot of new integrations with Development and DevOps tools. Also, warm welcome back to the Scriptler Plugin which was depublished in 2017 due to security issues. If you are afraid about such plugin numbers and dependency management, there is a new Plugin Installation Manager CLI Tool which should help Jenkins users to manage plugins more efficiently.

Security. It was a hot year for the Jenkins Security Team. There were 5 security advisories for the core and 20 – for plugins. In total we disclosed 288 vulnerabilities across the project, including some backlog cleaning for unmaintained plugins. Script Security Plugin was the hottest plugin with 10 critical fixes addressing various sandbox bypass vulnerabilities. Plain text storage and unprotected credentials were the most popular vulnerability type 120 disclosures in 2019. It was made possible by hundreds of reports submitted by contributors after code surveys, special thanks to Viktor Gazdag who reported the most of the issues and became the Jenkins 2019 Security MVP (check out his story here).

Infrastructure. Got Jenkins? If so, you rely on Jenkins update centers, website and issue tracker. All these and many other services are maintained by the Jenkins Infrastructure Team. This year the team handled more than 400 requests in the bugtracker, and many other informal requests. In total, more than 30 people contributed to Jenkins infrastructure this year (website content is excluded). We also deployed 4 new services, migrated 7 services from Azure Container Service to Azure Kubernetes Service and updated many other services. More changes will happen in the next months, and we are looking for new INFRA team members!

Documentation. Only last quarter we had 178 contributors to Jenkins documentation. It includes jenkins.io and other documentation hosted on GitHub, Wiki is not included. There is also ongoing migration plugin documentation from Jenkins Wiki to GitHub (announcement). Since the beginning of the project in Sep 2019, more than 150 plugin were migrated, and they got significant documentation revamp during the migration. You can see the current status https://jenkins-wiki-exporter.jenkins.io/progress. We also work on introducing changelog automation in the project. 123 plugins have already adopted the new changelog tools, powered by Release Drafter. Also, we had more than 60 technical blog posts published on jenkins.io.

Configuration as Code was one of the most popular areas this year. Jenkins Configuration as Code Plugin had more than 30 releases with new features and bug fixes. More than 50 plugins have been also updated in order to offer better configuration-as-code support. As a result, the JCasC Plugin got massive adoption this year (from 2000 to almost 8000 installations), and now it becomes a de-facto standard for managing Jenkins as code. This year we also ran our very first CommunityBridge project devoted to JCasC Schema validation and developer tools.

Events and outreach programs. In 2019 we participated in multiple conferences, including FOSDEM, DevOps World | Jenkins World, SCALE. More than 40 Jenkins Area Meetups were organized across the world, and there were many other meetups devoted to Jenkins. We also kept expanding our outreach programs. In total we had 12 students who participated in Google Summer of Code, Outreachy and newly introduced Community Bridge. We also had the biggest ever Hacktoberfest with 664 pull requests and 102 participants. These outreach programs help us to deliver new features in Jenkins. For example, this year we added Multi-branch Pipeline support for Gitlab and a new Plugin Installation Manager Tool during GSoC, and Outreachy resulted in a new Audit Log Plugin.

Where did we get those stats? GitHub stats came from the CDF DevStats service. These stats include all repositories in the jenkinsci organization and most popular repositories in jenkins-infra, Jenkins X and other organizations/repositories within the project are not included. Other stats came from project reports, component changelogs, Jenkins usage statistics service, plugin releases history.

What’s next?

Year 2020 will be pretty busy for the Jenkins project. There are many long-overdue changes in the project, which need to happen if we want the project to succeed. As it was written Board elections blogpost, there are many areas to consider: UX revamp, cloud native Jenkins, pluggable storage, etc. In the coming months there will be a lot of discussions in mailing lists and special interest groups, and we invite all teams to work on their roadmaps and to communicate them in the community.

Next month we will participate in FOSDEM, and there will be a Jenkins stand there. On January 31st we will also host a traditional contributor summit in Brussels, where we will talk about next steps for the project, in terms of technical roadmaps and the project governance. If you are interested in Jenkins, stop by at our community booths and join us at the summit! See this thread for more information.

We also plan to continue all outreach programs. At the moment we are looking for Google Summer of Code 2020 mentors and project ideas (announcement), and we will be also interested to consider non-coding projects as a part of other programs like CommunityBridge. We also work on improving contribution guidelines for newcomers and expert contributors. If you are interested, please contact the Advocacy and Outreach SIG.

And even more

This blog post does not provide a full overview of what changed in the project. The Jenkins project consists of more than 2000 plugins and components which are developed by thousands of contributors. Thanks to them, a lot of changes happen in the project every day. We are cordially grateful to everybody who participates in the project, regardless of contribution size. Everything matters: new features, bug fixes, documentation, blog posts, well reported issues, Stackoverflow responses, etc. THANKS A LOT FOR ALL YOUR CONTRIBUTIONS!

So, keep updating Jenkins and exploring new features. And stay tuned, there is much more to come next year!

About the Author – Oleg Nenashev

Jenkins core maintainer and board member. Oleg started using Hudson for Hardware/Embedded projects in 2008 and became an active Jenkins contributor in 2012. Nowadays he leads several Jenkins SIGs, outreach programs (Google Summer of Code, Hacktoberfest) and Jenkins meetups in Switzerland and Russia. Oleg works for CloudBees and focuses on key projects in the community. GitHub Twitter Blog

Dec 12
Love0

What’s New in Tekton 0.9

By Blog, Project
By Dan Lorenc
dlorenc@google.com
twitter.com/lorenc_dan


Another 6 weeks, another Tekton release. It’s mostly common knowledge that Tekton’s logo is a robot cat, but it’s lesser known that the releases are named after robots and cats! Each Tekton Pipelines release is given a codename of a type of cat followed by a famous robot.

On Monday, December 2nd, Andrea Frittoli of IBM cut the v0.9.0 release, dubbed “Bengal Bender”. This release contained commits from 21 different individuals. I wanted to take the time to highlight some of the new features and API changes, as well as to point out what much of the “under the covers” work is laying the groundwork for.

So let’s jump in!

New Features and Bug Fixes

“Bengal Bender” includes a solid set of features, bug fixes and performance improvements. Apologies if I missed anything here, this list is simply what I find most exciting.

Script mode!

https://github.com/tektoncd/pipeline/pull/1432

If you were at Kubecon San Diego you might have come away with the impression that Go is the language of the cloud. And while that is true to some extent, good old bash and yaml also play a huge part, especially when it comes to “glue” systems like delivery pipelines.

If you’ve spent much time working with containers, you’ve probably seen a yaml file with something like this in it: 

- name: hello
  image: ubuntu
  command: ['bash']
  args:
  - -c
  - |
      set -ex
      echo "hello"

This is a lot of complicated boilerplate just to run a simple bash script inside a container. And if you’re not deeply familiar with how bash, shells, entrypoints and shebangs all work at the system level, this is a bit opaque. It’s also prone to subtle and confusing bugs. Even if you are familiar with these things, you’ve probably wasted time debugging issues when the shell in your container is set to something you’re not used to, or the entrypoint is overridden, or “bash -c” doesn’t do what you expect.

This makes delivery pipelines harder to write, understand and maintain.

Enter script mode! Inspired by some ideas from Ahmet Alp Balkan, Jason Hall put together a proposal to make it much easier to define tasks that just need to run a simple bash script. Here’s what it looks like:

- name: hello
  image: ubuntu
  script: |
    #!/bin/bash
    echo "hello"

You’ll notice that there are far fewer lines of boilerplate. No more need to specify args, an entrypoint or remember the tricky “-c” part. Just specify the interpreter you want to use and the commands to run. This has already let us simplify dozens of test cases and examples!

Performance

https://github.com/tektoncd/pipeline/pull/1545

Tekton has long suffered from poor performance around starting PipelineRuns. Christie Wilson and I did a debugging/coding session last spring to try to improve this, and identified PVC mounting as a major contributor. Unfortunately, our attempted fix didn’t work and needed to be rolled back.

A more general longer term fix is being handled by Scott Seaward and the re-resources effort, but I decided to take another stab at a short term fix. Fingers crossed that it works, but initial testing shows improvements of anywhere between 5 and 20 seconds per PipelineRun!

API Changes

There have been a few changes to the API as we start to firm things up for a beta release. We’re hoping to get the majority of these breaking changes in over the next few releases so users can start to build production systems on top of stable versions.

The breaking changes in v0.9.0 include:

Standardization of output paths for image digests

Tekton currently provides a mechanism to store the digests of container images built by Tasks. This mechanism predated the PipelineResource subsystem, and required Task authors to write these digests to a specific location at /builder/image-outputs. This change moves that to the standard path for output resources, at /workspace/output/<resource-name>.

Simplification of the cluster resource

Cluster PipelineResources make deploying and working with Kubernetes clusters from within Tasks simple. They provide mechanisms for users to declare where a cluster endpoint is and how to authenticate with it. Then, during Task execution, they automatically configure a .kubeconfig file so Kubernetes tooling can find that cluster. This release contained a few changes to make these cluster resources easier to work with.

Previously, users had to specify a name parameter twice: once in the resource name and once as a parameter to the resource. The second parameter has been removed.

Ground Work

Following the age-old programming advice above, most of the work contained in each Tekton release is in service of features that won’t be exposed until a later release. Digging into the commit log shows a detailed picture of what the community is working toward.

Here’s my editorialized version of what’s coming soon 🙂

Revamped PipelineResources

A lot of work went into cleaning up the existing PipelineResource subsystem, including the interface exposed to PipelineResource types and the types themselves. Getting these both solid will form the basis for the larger Re-resources effort, which is currently underway. This project will make the resource types extensible, allowing anyone to add and use their own types. It will also hopefully leave us with some embeddable components, so that other systems can make use of Tekton PipelineResources and the coming catalog.

API Versioning

One of the most important steps in shipping a stable API is figuring out how to make changes in a backwards compatible way. No API is perfect, so the ability to upgrade one is paramount. Vincent Demeester and his colleagues at Red Hat have been hard at work designing and implementing an API versioning system that will allow users to upgrade Tekton API versions without breaking existing workloads. This will be key to the coming beta release.

Wrapping Up

The Tekton project has been amazing to watch grow. This post only details the changes in the Tekton Pipelines release, but there has also been some awesome work done in the Triggers, CLI and Dashboard projects. Triggers now support out-of-the-box Github and Gitlab validation. The CLI has improved support for creating PipelineResources and starting Tasks interactively. Visualization is coming soon to the Dashboard! I’d like to thank everyone that has made Tekton what it is today.

The Tekton community has been hard at work shipping the right APIs and components to build cloud-native software delivery systems. If you’re using Tekton, or interested in learning more about Tekton, we’d love to hear from you. Please consider joining the community, becoming a Tekton Friend or contributing directly. 

Nov 15
Love0

A Year of Tekton

By Blog, Project

Author: Dan Lorenc (dlorenc@google.com, twitter.com/lorenc_dan, github.com/dlorenc)

Last year at Kubecon Seattle, Tekton was just an idea in the heads of a few people, and a bit of code inside knative/build-pipeline. Fast forward to today, and we have a thriving community, independent governance inside a great foundation, and we’re quickly approaching our beta release! This year has flown by, so I wanted to highlight some of the original goals of the Tekton project, and some of the milestones we’ve hit toward them.

The Start

When Christie, Jason and I started sketching out the original Tekton APIs on a whiteboard in San Francisco, we had a straightforward goal in mind: make the hard parts of building CI/CD systems easy. There are already dozens of CI/CD systems designed to run on kubernetes, and for the most part they all have to build the same basic infrastructure before they can start solving customer problems. Kubernetes makes scheduling, orchestration and infrastructure management easier than anyone could have imagined, but it still leaves users with a few pieces to assemble before they can use it as an application delivery platform.

These were some of the original whiteboard sketches on what later became the Tekton Pipeline CRD.

Things like basic DAG orchestration, artifact management, and even reliable log storage are outside the bounds of the core Kubernetes APIs. Our plan was to use the new Custom Resource Definition feature to try to define a few more “nouns”, on top of the existing Kubernetes primitives, that were better suited for Continuous Delivery workloads. In doing so, we would make it easier for people and trans to create delivery systems designed for their exact use cases, while making sure the underlying primitives allowed for some degree of compatibility.

Vague ideas are great, but it’s much more productive to collaborate with others when you have something concrete to share and get feedback on. So in August of 2018, we released the sketch and principles we used to design it on knative-dev and to a few other interested parties. The feedback was amazing. The core pipeline team on Jenkins at Cloudbees jumped in with some hard earned lessons from their experience working on the most widely used orchestration system in history. The Concourse team at Pivotal helped redesign our extensibility system based on what they learned from the successful Concourse resource model.

Then we got to work building it all out! Our goal must have really resonated with others, because we could really feel the power of open source from day one. Even when there were only a few of us working together at Google, we were part of a much larger effort. We really wouldn’t be here today without the help of our contributors, maintainers and governing committee members.

A New Home

Around the time we prepared for our first release, it started to become obvious that Tekton (then called knative/build-pipeline) needed its own home. The knative brand and community helped immensely, but Tekton was meant to provide CI/CD for everyone – not just serverless or even kubernetes users. So in February of 2019, we decided to split out the project, name it “Tekton” and donate it to the newly-forming Continuous Delivery Foundation just in time for the 0.1.0 release. 

Kim Lewandowski announcing Tekton at the Open Source Leadership Summit

Open source, governance, and communities are hard. The move out of knative and into a new foundation was a big change for the project, but has proven worth the effort! Tekton still works great with other knative components, but has had the chance to grow its own community and evolve to a spot that its users need. Thanks to the community, Tekton has expanded into multiple projects like the Dashboard and CLI, and in Tekton Pipelines we have been so lucky to gain the expertise of folks like Vincent Demeester and Andrea Frittoli.

The rest of this year has felt like a blur, but I wanted to call out some major milestones we hit:

  • March – The first Tekton Pipelines release (v0.2.0) using Tekton itself!
  • June – First release of the Tekton CLI!
  • July – The first Tekton Pipelines release (v0.5.0) by a community member not at Google!
  • July – Tekton Friends repo created! 
  • August – First release of the Tekton Dashboard!
  • September – First Triggers release!
  • October – Tekton passed 100 contributors in October!

What’s Coming Next?

We’re rapidly approaching the first Tekton beta release! As part of this effort, we evaluated our API surface and identified quite a few areas that need hardening. This includes finishing up the table-stakes requirements for a CI/CD platform – things like triggers, metadata storage and building up our catalog. The Triggers v0.1.0 release has made Tekton usable in so many new ways, and we’re just getting started there still! 

Scott Seaward has has just started work on refactoring PipelineResources into an extensible system that will form the basis for the Tekton catalog, and Jason Hall is working on a metadata storage system that will help power some new ideas around software supply chain management.

If you’d like to get involved in the Tekton project, you can find us on Github, Slack or our email list . We’ll also be at Kubecon next week! Come attend one of the many sessions or the CD Summit. Here’s to Tekton in 2020!

Close Menu