Skip to main content
search
Tag

Spinnaker

Mar 16
Love0

From Spinnaker – April’s Spinnaker Gardening #CommunityHack is Going Virtual!

By Blog, Project
Spinnaker Gardening Days Community Hack

Originally posted on the Armory blog, by Rosalind Benoit

Guess what?! Our Hackathon is going fully online! “Spinnaker Gardening Days #CommunityHack” happens in one month, and we’re gearing up for an international open-source work-from-home extravaganza! Via Zoom, Slack, and Github, we’ll empower you to move the needle on continuous delivery projects. Teams will hack, newcomers will train, and champions will share Spinnaker secrets. Click here to register and get your free tickets for the hackathon, training track, lunchtime learnings, or all three.

 Join other Spinnaker users and companies to learn and let your skills shine at this collaborative event. We’ll address open-source feature requests, extend the ecosystem, and have lots of fun. Thanks to our generous sponsor Salesforce, all logged-in participants will score prizes, premium swag, and lunch on us! Hack through the workday, or check out our noontime lightning talks. Visit the Spinnaker Gardening repository for the schedule and details.

Salesforce logo

The Armory Tribe celebrates the support of Salesforce and, in particular, Edgar Magana, a Spinnaker champion and Cloud Operations Architect. We recently sat down to discuss the Ops SIG, modeling and standardizing Spinnaker, and his ideas for hackathon projects. Read the full article here.

A relative newcomer to the Spinnaker community, but a veteran in matters of cloud computing, networking, and OSS projects like OpenStack, Edgar recently founded the Operations SIG (Special Interest Group). Just as he recognized that “the community needed a place to discuss how to operate Spinnaker better,” he also urges us to jump-start the Spinnaker community. He’s recommended improvements to the contributor experience, and persuaded Salesforce to sponsor this first-ever Spinnaker hackathon.

Of course, we touched on his most pressing open-source Spinnaker initiatives in our chat. Next up? Gather a team! 

“We really want to come to the hackathon with goals, and to put extra motivation for folks to address them as a community,” Edgar explains their sponsorship.

From Salesforce and the Ops SIG perspective, Edgar has two features stories to focus on at the hackathon:

  • “Run any OSS source code scanning software against Spinnaker microservices, and you’ll find a number of vulnerabilities in the libraries that Spinnaker leverages. We’d like to minimize and solve those as much as possible.” 
    • I’m pumped about this one because a) in many instances, this is a low-barrier-to-entry task that newer contributors can make a huge dent in, and b) every ops freak knows that fixing OSS dependencies is probably the most important security measure we touch. 
  • “Cloud driver scalability is another key initiative in progress. The dynamic account system works, but performance can be improved drastically for those using a large system with 800-1000 Kubernetes accounts. There was a bugfix in 1.17, but it still takes lots of time for clouddriver to cache new accounts, and this means a long startup time.”
    • Edgar would like to see new accounts dynamically appended to the cache instead of triggering another cache of all accounts, and has been collaborating with Armory engineers on a solution. Another excellent project goal for Community Gardening!

Here on Armory’s Community team, we second Edgar’s suggestion to make Spinnaker more “beginner-friendly” and welcoming to new contributors. Our top goals for the first half of 2020 revolve around improving the contributor experience, from promoting issue triage in SIGs, to creating and organizing documentation around Spinnaker development environment, release cycle, and contribution guidelines so that newcomers know where to find answers and how to get started. Expect to see a contributor experience project from us at the hackathon!

In the meantime, the Plugin Framework for Spinnaker that Armory and Netflix are building is maturing fast. This work will make Spinnaker more welcoming to contributors in another way: it provides clear extension points in the codebase, along with an easy way to load extensions to a running Spinnaker instance. With the Spinnaker Gardening Days, we want encourage you to build extensions. Moreover, we know that many teams using Spinnaker in production have already built custom tooling around it; we’re encouraging those teams to leverage the plugin framework to quickly share their work with the OSS community (sounds like a stellar hackathon project!). We’re better together, and with a widely adopted project like Spinnaker, you can feel sure that paying it forward will reap big dividends for you and your organization. Check out the Plugin Creators Guide and Plugin Users Guide to learn more!

Calling Edgar and all other incredible Spinnaker developers: it’s time to add your fantastic Spinnaker Gardening ideas to the Project Ideas Wiki, create a slack channel for your project, and start prepping for the most exciting online event of 2020! Don’t forget to register here and reserve your ticket : )

spinnaker-hackathon gardening readme

Learn more in the spinnaker-hackathon/gardening README

Mar 11
Love0

From Spinnaker – Monitoring Spinnaker: SLA Metrics

By Blog, Project

Originally posted on the Spinnaker Community blog, by Rob Zienert, Sr Software Engineer @ Netflix

Long, long ago, in an internet that I barely remember, I wrote about monitoring Orca. I haven’t managed to take the time to write another post about a specific service — it’s a lot of work! Instead of going deep this time around, I want to paint with broader strokes: What are the key metrics we can track that help quickly answer the question, “Is Spinnaker healthy?”

Spinnaker is comprised of about a dozen open source services that may vary widely based on configuration, and as such, there’s no singular metric to rule them all. This makes the question, “Is Spinnaker healthy?” a particularly bothersome question since not all services are equally important. If Igor — the service that is responsible for monitoring CI/SCM systems — is unable to communicate with Jenkins, Spinnaker will be in a degraded state, but its core behavior is still healthy. Should Orca’s queue processing drop to zero, however, it’s time to have an elevated heart rate and quick remedy.

Service Metrics

The Service Level Indicators for our individual services can vary depending on configuration. For example, Clouddriver has cloud provider-specific metrics that should be tracked in addition to its core metrics. For the sake of this post’s length, I won’t be going into any cloud-specific metrics.

Universal Metrics

All Spinnaker services are RPC-based, and as such, the reliability of requests inbound and outbound are supremely important: If the services can’t talk to each other reliably, someone will be having a poor experience.

For each service, a controller.invocations metric is emitted, which is a PercentileTimer including the following tags:

  • status: The HTTP status code family, 2xx, 3xx, 4xx...
  • statusCode: The actual HTTP status code value, 204, 302, 429...
  • success: If the request is considered successful. There’s nuance here in the 4xx range, but 2xx and3xx are definitely all successful, whereas 5xx definitely are not
  • controller: The Spring Controller class that served this request
  • method: The Spring Controller method name, NOT the HTTP method

Similarly, each service also emits metrics for each RPC client that is configured via okhttp.requests. That is, Orca will have a variety of metrics for its Echo client, as well as its Clouddriver client. This metric has the following tags:

  • status: The HTTP status code family, 2xx, 3xx, 4xx...
  • statusCode: The actual HTTP status code value, 204, 302, 429...
  • success: If the request is considered successful
  • authenticated: Whether or not the request was authenticated or anonymous (if Fiat is disabled, this is always false)
  • requestHost: The DNS name of the client. Depending on your topology, some services may have more than one client to a particular service (like Igor to Jenkins, or Orca to Clouddriver shards).
Example of our 24/7 request fanout from Gate. One interesting tidbit: The sudden increase in traffic at 9am is the increased traffic to Clouddriver (bottom) from Chaos Monkey starting its daily light mayhem!

Having SLOs — and consequentially, alerts — around failure rate (determined via the succcess tag) and latency for both inbound and outbound RPC requests is, in my mind, mandatory across all Spinnaker services.

As a real world example, the alert Netflix uses for Orca to all of its client services is:

nf.cluster,orca-main.*,:re,
name,okhttp.requests,:eq,:and,
status,(,Unknown,5xx,),:in,:and,
statistic,count,:eq,:and,
:sum,
(,nf.cluster,),:by,
0.2,:gt,3,
:rolling-count,3,:ge

So, for people who can’t read Atlas expressions, if we have more than 0.2 failing/unknown RPS to a specific service over 3 minutes, we’ll get an alert.

Service-specific Metrics

Most of our services have an additional metric to judge operational health, but in/out RPC monitoring will go far if you’re just starting out.

  • Echo
    echo.triggers.count tracks the number of CRON-triggered pipeline executions fired. This value should be pretty steady, so any significant deviation is an indicator of something going awry (or the addition/retirement of a customer integration).
    echo.pubsub.messagesProcessed is important if you have any PubSub triggers. Your mileage may vary, but Netflix can alert if any subscriptions drop to zero for more than a few minutes.
  • Orca
    task.invocations.duration tracks how long individual queue tasks take to execute. While it is a Timer, for an SLA Metric, its count is what’s important. This metric’s value can vary widely, but if it drops to zero, it means Orca isn’t processing any new work, so Spinnaker is dead in the water from a core behavior perspective.
  • Clouddriver: Each cloud provider is going to emit its own metrics that can help determine health, but two universal ones I recommend tracking are related to its cache.
    cache.drift tracks cache freshness. You should group this by agent and region to be granular on exactly what cache collection is falling behind. How much lag is acceptable for your org is up to you, but don’t make it zero.
    executionCount tracks the number of caching agent executions and combined with status , we can track how many specific caching agents are failing at any given time.
Here, one collection for a specific AWS service in our largest region was getting stale. In this case, while AWS availability was fine for Clouddriver, Edda was having trouble refreshing.
It’s OK that there are failures in agents: As stable as we like to think our cloud providers are, it’s still another software system and software will fail. Unless you see sustained failure, there’s not much to worry about here. This is often an indicator of a downstream cloud provider issue.
  • Igor
    pollingMonitor.failed tracks the failure rate of CI/SCM monitor poll cycles. Any value above 0 is a bad place to be, but is often a result of downstream service availability issues such as Jenkins going offline for maintenance.
    pollingMonitor.itemsOverThreshold tracks a polling monitor circuit breaker. Any value over 0 is a bad time, because it means the breaker is open for a particular monitor and it requires manual intervention.

Product SLAs at Netflix

We also track specific metrics as they pertain to some of our close internal customers. Some customers care most about latency reading our cloud cache, others have strict requirements in latency and reliability of ad-hoc pipeline executions.

In addition to tracking our own internal metrics for each customer, we also subscribe to our customers’ alerts against Spinnaker. If internal metrics don’t alert us of a problem before our customers are aware something is wrong, we at least don’t want to wait for our customers to tell us.

Continued Observability Improvements

Since Spinnaker is such a large, varied system, blog posts such as these are fine, but really are meant to get the wheels turning on what could be possible. It also highlights a problem with Spinnaker today: A lack of easily discoverable operational insights and knobs. No one should have to rely on a core contributor to distill information like this into a blog post!

There’s already been a start to improving automated service configuration property documentation, but something similar needs to be started for metrics and matching admin APIs as well. A contribution that documents metrics, their tags, purpose and related alerts would be of huge impact to the project and something I’d be happy to mentor on and/or jumpstart.

Of course, if you want to get involved in improving Spinnaker’s operational characteristics, there’s a Special Interest Group for that. We’d love to see you there!

Mar 09
Love0

From Spinnaker – Future of SRE: Robert Keng Builds a DeploymentBot #withSpinnaker

By Blog, Project

Originally posted on the Spinnaker Community blog, by Rosalind Benoit

Coming soon from Chime to OSS, a software delivery chatbot which uses Slack to deploy apps via Spinnaker

Last month I had the pleasure of chatting with Robert Keng, a Lead SRE at Chime, about a Slack-integrated ChatBot he recently built to facilitate lightweight, direct deployments for developers. Chime’s continuous delivery is based on Spinnaker, driven with signal-based GitOps. Via pipelines, merged release branches are auto-deployed from a continuous integration (CI) solution, through QA to production with no human interaction interaction.

However, it hasn’t always been this way; Chime has roots in a legacy build environment, largely for Ruby-on-Rails development. It’s based on configuration management tools such as Salt, and thus not containerized, but pointed at long-lived infrastructure. So, containerization formed an important milestone in Chime’s continuous delivery adoption. Luckily, according to Robert, its high-trust, growth minded culture and workflows have supported the evolution.

Chime’s culture also provides flexibility that highlights Spinnaker’s power to accelerate digital transformation. Robert explains that, in some instances, it makes sense for developers to deploy straight to a test environment, bypassing CI. When adding a small feature to a mobile app, for example, I might want to bypass CI wait time to deploy and experiment with behavior (raise your hand if you‘ve built an app and never done that…didn’t think so!)

Meeting Chime devs where they’re @

“We’re cutting the straight-to-prod patch fix deployments down to zero,” Robert clarifies, and he’s done it by creating a flexible system with Spinnaker that models Chime’s culture of trust. At any time, if the devs he enables would rather execute commands in Slack to deploy branches to environments of their choosing, they can. Robert has created a tool that allows them that agency, while empowering them to address complex use cases, for example, adding logic into the Slack commands to deploy dynamic environments into different Kubernetes clusters. In production, “If we need to scale customers on the Z-axis, and build multiple app versions with different backends to target different service providers” as deployment targets, with Spinnaker, Chime can. Robert points out:

“Spinnaker offers a lot of agility in that respect. It would be hard to accommodate gitOps and chatOps in the same place without it.”

In a prime example of the opportunities to solve that Spinnaker provides as a platform, Robert has created a golden path which allows Chime’s teams to iterate in a safe environment. To create it, Robert analyzed workflows as they are and designed an alternative workflow that mapped what he observed in Spinnaker. This, combined with the auto-deploy strategy, tells the story, written in pipelines, of how Chime engineers deliver software. This way, as an SRE, he can rely on automated guardrails for safety regardless of the deployment path. As Kelsey Hightower says, it “serializes the culture in tools” in a way that’s seamless, painless, and purposefully abstracted.

Because at the end of the day, it’s not about the tools. It’s about your story, which in Chime’s case, is all about changing the way people feel about banking. What products and services do you delight your customers with? What’s your story? You can tell it #withSpinnaker

One DeploymentBot, Headed for OSS Spinnaker

The tool, in a multi-service design, has a component which handles the request/response communication with Slack, a frontend that leverages Okta user groups to control who can access Spinnaker, and a Python backend which processes the request data in batches. This architecture evolved from using webhooks to, at Armory’s suggestion, using client certs for faster authentication, and from a monolith version to microservices, because of constraints encountered in the bot’s development. The top constraint: the Slack Events API’s requirement that a response from requests arising from message actions be received within 3 seconds.

This constraint presented challenges in actions like querying Vault for certificates to authenticate against Spinnaker, and even in token exchange with Slack. Breaking the chatbot into pieces allowed Robert to create a responsive, extensible service to deliver a full-featured experience for Chime devs. “It’s turned into a monster,” he grins. “I have tons of feature requests for additional functionality already” (because his devs love using it).

Next steps for Robert’s Bot include developing it against the entire Spinnaker API to leverage all features available, and adding more dynamic capability. He wants to enable devs to use the bot to deal with existing pipelines and executions, and adjust parameters and other configuration via a scripted payload directly from Slack.

Another important next step? Open-sourcing the DeploymentBot! Robert’s very busy with projects right now (read more below), but I’ll hook him up with support from Armory engineers, if needed, to help get this invention to the masses.

The Future of Site Reliability, Platforms, and DevOps Engineering

As he describes his plans for the Bot, we start talking about the myth of NoOps. I have my own words about the opportunities and fallacies of Dev + Ops, but here, Robert’s voice speaks for itself:

“My team isn’t DevOps, it’s SRE (Site Reliability Engineering). DevOps is just part of what we do. As tech stacks mature, we’re seeing less dependency on direct hardware interaction, but that doesn’t mean the management complexity goes away; it actually gets worse. Here’s an easy example: We have this awesome thing called Kubernetes. Given config maps and secrets, where is the source of truth? Ask anyone in the community, and they’ll say, ‘Umm…build it yourself!’ I know Hashicorp released a sidecar method to inject values, but none of that is complete. This is why there’s a lot of custom work in the community, and companies are building their own mutating webhook controllers, for example, which is what we’re doing. You can’t buy this stuff, because it doesn’t exist.

We have our own way of injecting Vault secrets which 100% bypasses Kubernetes stuff, because we can’t version it, and we can’t manage it from any source or truth, as it’s scattered across 1000 namespaces. It’s impossible to manage in one place. So in our environment, we put everything in Vault, whether it’s configuration, or secrets. That gives us a common interface to code against. In V1, we’re using init containers, which is exactly what Hashicorp’s sidecar does. In V2, depending on the environment, we’ll grab values from different Vault clusters, since storing production and non-production values in the same place is just, suicide. You’ll get a huge ban hammer from your security team, and no-one wants that.

So we’re building, and we’re operating it at the same time. And are developers ever going to touch these [tools]? No! There are a lot of these instances in Kubernetes where things just don’t exist, so what do you do?Same thing for, EC2, and ECS even. Then, moving into Knative, and Lambas, and serverless computing and functions, it’s even worse. It’s a free-for-all. We’re designing our own framework.

The next thing we’re looking at is building plugins that will plug in our code, and use Spinnaker to deploy it [on that infra]. I heard Armory is working on something similar for deploying Lambdas, and I’m desperately waiting, because it’s going to make my life easier. Functions in general are kind of useless. The ecosystem around them is more important; you’ve got to think about API gateways, API management, queues, load balancers, etc. How do I wrap that into a sane framework where we can consistently build, integrate, test, and deploy? I don’t want to use 10 different ways to do the same thing. I’d rather just have everything work in Spinnaker.”

Then when we start talking about making that happen. I tell Robert about the Community Gardening Days I’m planning for Spinnaker this Spring (keep your eyes peeled! Announcement forthcoming on Spinnaker.io and social), and he gets psyched about Chime’s involvement. Music to my ears!

Look out for more articles from me on the Spinnaker developer and contributor experience. I’ll shine a light on the way Open Source Heroes like Robert are getting into the ecosystem as they enable the delivery of software products and services. Hang on, the latest industrial revolution (where software truly changes the freaking world for the better!) is just taking off.

Please share this on Twitter, LinkedIn, and HackerNews and give Robert some glory : )

Mar 06
Love0

From Spinnaker – Open your first issue to get started as a contributor

By Blog, Project

Originally posted on the Spinnaker Community blog, by Rosalind Benoit

Why are most periodicals so sad? Because we have too many issues!

A problem the Spinnaker community currently ISN’T having ^^

I’d like to start sharing one IC’s experience (mine : ) dropping anchor into the Spinnaker ecosystem. I found the community last year before joining the tribe at Armory, while doing research for another continuous delivery product built on Tekton. First step: join Spinnaker slack, and behold the community live, with active SIG channels, newcomers, and operators constantly pinging to discuss what they encounter in the platform.

What’s the next step? Personally, I began creeping on Spinnaker.io, joined the Docs SIG which maintains the site, and began to engage by commenting on and submitting web-dev PRs in the site’s repository to get the ball rolling. Next up: get my full Spinnaker dev environment set up, and document that process for y’all!

If you’re an end-user, your path may look different: you may have used Spinnaker to deploy an application, and encountered a usability issue or found something not quite right from your perspective. In this case, reaching out on Slack in the #general or relevant SIG channel, or filing an issue in the spinnaker/spinnaker repository describing your observation is the next step. As a final note to end-users, having spoken to Andy Glover and a few TOC members about this at some length, I can say on good authority:Why end-users and new contributors should submit issues to the project

Why end-users and new contributors should submit issues to the project

Push past your fear in filing an issue! At this stage in its growth, the “too many issues” problem doesn’t exist. We’ll skip across that bridge when we come to it. Now, we need your feedback to make the most mature and production-ready continuous delivery platform the BEST platform on the planet. Don’t be shy!

Operators will also follow their own path to begin contributing. Perhaps you’ve found a great growth opportunity for the codebase as you’ve hacked through workflows. Maybe you’ve developed a rockin’ integration to solve for interoperability at your organization, and you know it may benefit others in the community. Or, that small tweak made to your organization’s Spinnaker instance has improved usability, but hasn’t been addressed in the community.

YES, your time is precious, but I urge you, don’t wait! Sharing your contributions will help the ecosystem, and it will also help you. It puts you on the map. It adds gravitas to your resume. It exposes you to peer recognition, and provides networking with some teeth, as your community footprint will speak for itself. Better yet, if you start a fix or conversation, others in the community can advise, or build on what you started, helping you solve faster.

If you’d like to add a feature to Spinnaker, that should start as a discussion, so file an issue in the spinnaker/spinnaker repository describing the purpose and proposed implementation, or start the discussion at a SIG meeting. Got something smaller-scale, like a bug fix? The Submitting A Patch page on spinnaker.io provides guidance. Integrating another service, or building an extension? Check out the Plugin Users Guide, as utilizing the new plugin framework allows you to maintain plugin code in a separate repository and avoid the requirement of loading extensions at Spinnaker runtime.

If that’s a bit overwhelming for now, don’t despair. Getting started is the first step. Noticed something confusing in the docs, or have a suggestion for spinnaker.io? Have ideas on what kinds of Contributor Experience materials would help you move forward? Please ping us in #SIG-Documentation or file an issue in spinnaker/spinnaker.github.io!

That’s all for today, but stay tuned for more N8B Diaries as I work to guide Spinnaker operators in contributing their inventions, and set up my own environment. High-five your imposter syndrome and become a *real* Spinnaker contributor with me!

Feb 24
Love0

Descartes Labs – Implementation of Spinnaker Pipelines – The End of Waterfall

By Blog, Community

By Tracy Ragan, CEO DeployHub, CD Foundation Board Member

The New Mexico CI/CD CDF Meetup, hosted by DeployHub, enjoyed an amazing presentation by Louis Vernon, Site Reliability Engineer at Descartes Labs.  Louis showed in detail how Descartes Labs improved service levels to customers, dumped a waterfall release approach and simplified their GKE releases using Spinnaker, Istio and StackDriver. 

Louis’ presentation covers how the team at Descartes Lab implemented Spinnaker to push continuous deployments including the integration of Istio to route updates between Dev, Beta, Pre-Release, and Release, all running in the same cluster.

The use of both Istio and Spinnaker at Descartes Labs is a mature example of what can be done to build out a modern Kubernetes Pipeline.

While Descartes Labs still implements different ‘states’ of the pipeline, the release process uses a single cluster with Istio properly routing using virtual service names.

Louis explains how the team at Descartes Labs got to the point where they understood that a shift of this magnitude was essential for creating a stable environment for all users. 

In my humble opinion, moving away from separate Dev, Test and Prod clusters is the direction we will all be moving. 

The full recorded demo is here:

Louis also presented at the Spinnaker Summit 2019. You can download his presentation at:

https://github.com/louisvernon/SpinnakerSummit2019

Jan 27
Love0

Fair Winds & Following Seas: Another Year of Safe, Reliable CD #withSpinnaker

By Blog, Project

It’s 2020, so what’s our New Year’s Resolution?  #1, Make Spinnaker the Perfect Continuous Delivery Platform. 

🤦 Voltaire said “perfect is the enemy of good,” and we’ve seen some resolution-minded ads lately reviving that adage (I’m thinking of Michael Phelps reminding me that Progress IS perfection : )  Striving for perfection in software development can lead to obsolete products. So, we hack. We listen to our users and iterate. When we do that as a community, we can progress towards something truly brilliant. Spinnaker’s progress was perfection in 2019, and by all accounts will exceed that trajectory in 2020. 

Enterprise Adoption Crescendo (of Production workloads)

Spinnaker saw promising early adoption from large companies like Target and Adobe, and this year has been no exception. While literally everyone books stays on the site, oblivious to digital transformation, Airbnb is using Spinnaker to migrate from monolith to service-oriented architecture, and from brittle deployments to continuous delivery. SAP joyfully leverages Spinnaker on its mission to run the world better, and Pinterest uses it to boost productivity as it pioneers visual discovery. Transunion stays ahead of the fintech curve, providing total credit protection through applications it now deploys with Spinnaker, a more full-featured fit for ephemeral infrastructure than its previous Ansible solution.  Companies like Comcast, going all-in on Kubernetes as a software-defined datacenter, have added Spinnaker to manage deployment pipelines. Meanwhile, Salesforce has adopted Spinnaker to bake images based on both Helm charts for Kubernetes and Packer templates for VMs, to support its complex software delivery requirements.

In 2019, we proudly welcomed engineers from new enterprises, including JPMorgan Chase and Home Depot, into the Spinnaker community. Now more than 175 companies have contributed to the project, with over 200 new ICs just last year, and many more companies have become key stakeholders, using and extending Spinnaker. These demonstrate Spinnaker as the mature CD solution, proven to handle production workloads flexibly and scalably. 

Organic Growth Through Governance

As adoption continues to rise, and our community grows, it becomes crucial to create a growth-adapted project space. A transparent structure for building and maintaining the project invites new companies and users to take an active role in shaping Spinnaker’s future. To that end, 2019 saw the governance process and entities created in 2018 codified in GitHub, along with a process for modifying it via PRs and votes from members of the TOC and Steering Committee. Spinnaker governance also blossomed into an active space encompassing 8 community-initiated SIGs which organize contributors around feature growth and maintenance in areas of interest. SIGs welcome anyone to join, and we saw growing attendance from end-user companies in H2 2019. As the TOC experiments with public Open Office Hours, Spinnaker Slack is always open, and welcomes nearly 8500 participants to troubleshoot, chat with a SIG team, or reach out to a community member any time. Coupled with the donation of the project to the CDF, these growth factors signal the founding of Spinnaker as a neutral, democratized project space. Our goal? Fuel rapid innovation as we work to empower humanity to deliver their innovations, faster.

#BetterTogether: Driving SDLC Ecosystem Innovation

What came out of this investment in 2019? Where to begin…! An OSS ecosystem thrives with modular components that allow operators to optimize for business goals and maintain compliance. As our user base grows, the problem set expands, use cases vary, and we innovate across a richer toolchain. This allows us to create a smarter, more automated experience. Case(s) in point: 

  • New data sources added for SignalFX and New Relic, to inform Automated Canary Analysis decisions that let app owners sleep instead of being paged. 
  • A new Gremlin integration allowing chaos experimentation in Spinnaker pipelines will expand in 2020 to provide results useful for automated decision support. 
  • Integrations with artifact repositories Nexus and JFrog’s Artifactory have added new native triggers for Spinnaker pipelines. 
  • New end-to-end secrets management dynamically decrypts Spinnaker secrets as needed for validation and deployment from a backing store of your choosing, such as Vault or S3.

Since interoperability is crucial to Spinnaker, implementing a reliable plugin system was a key 2019 milestone. As our community leverages Spinnaker to solve problems, we must remove friction from the dev’s experience in contributing those extensions to the project. A plugin framework provides libraries and application context to devs, and defines clear extension points to start from when integrating something new. In 2019, we adopted PF4J as our backend plugin-loader framework. In 2020, we’ll implement plugin loading in the Spinnaker UI, and foster community around building and sharing plugins, to enrich our ecosystem.

Cloud Providers – Raining Champions  : P

Spinnaker depends on cloud provider investment in extending the project for deployment to the ever-growing variety of ephemeral infra solutions. In 2019, engineers at Google developed a blueprint for a production-ready Spinnaker instance on GKE, integrated with Google Cloud services such as Cloud Build. Amazon Engineers have extended cloud providers for AWS services, ensuring that we can deploy with Spinnaker to any attribute available in Fargate or ECS (Elastic Container Service). As of this year, that includes any task definition attribute. AWS also added full support for deployments to serverless applications using AWS Lambda, including the ability to use Lambda functions as ALB targets.

Migrating to the cloud alleviates headaches, while bringing new operational challenges. Spinnaker evolves to capture and solve for these new challenges as we encounter them.  The extendable Swabbie service, created in 2019, tackles the tedium (and potential nightmare at scale!) of reaping unused resources programmatically, to help optimize cloud spend. With Swabbie, an operator can set rules for cleanup candidates via YAML, and clean resources according to a configurable schedule. Deployments to highly automated cloud environments prompted enablement of dynamic account creation, discovery, and configuration for Cloud Foundry and Kubernetes accounts in the cloud.

Upleveling Functionality = Perfect Progress 

The Kubernetes V2 Provider for Spinnaker also came into its own last year, offering the ability to deploy, delete, scale, and roll back K8S manifests as artifacts managed as code. The Kubernetes SIG iterated fast to improve the V2 user experience by surfacing more kubectl commands in the Spinnaker UI, and improving management of rollout strategies. They also enhanced traffic management to enable more deployment patterns with the provider, such as blue/green (AKA red/black) and dark canary. In 2020, simplifying the Kubernetes developer experience is an important roadmap element, and the community will tackle it by visualizing more K8S resources in the Spinnaker UI, and improving terminology, error, and workflow management.

Under the hood, 2019 saw lots of effort to provide operators the option to back Spinnaker with a MySQL database instead of Redis. Stateful data in Spinnaker enables event routing and orchestration for pipelines, integrated CI and SCM events, and Swabbie cleanup notifications. The choice of whether to use a relational DB or in-memory store to manage that data gives operators the freedom to optimize performance for their workloads and infrastructure. This makes all that effort, which required updating several microservices, including Echo, the eventing service, and Orca, the orchestration engine, well worth it. Likewise, updates to the Authorization model have allowed even more granular permissions to be durably API-driven across the platform.

A Bright Future Won’t Blind Us (to Your Story)

One high-level 2020 goal aims to better incorporate user stories and enterprise use cases into Spinnaker’s trajectory. The steering committee has committed to building a roadmap that tells high-level stories about using Spinnaker to solve problems. Tool chain interoperability, notably with Kubernetes, cloud providers, and monitoring systems figure large in the H1 2020 Roadmap. Managed Delivery, an exciting Spinnaker CD initiative incubating at Netflix, uniquely responds to a common narrative around software delivery. It uses declarative automation to alleviate the operational knowledge and maintenance burden that comes with ownership of modern, continuously delivered applications.

Users can help us tell the best Spinnaker stories by submitting comments and issues describing usage and business context. Please visit Spinnaker.io (which the Docs SIG will overhaul in 2020) and check out our Success Stories page. Join us on Spinnaker Slack or in the Spinnaker org and tell your tale! 

Aug 14
Love0

Jenkins Celebrates 15 Years of Transforming Software Delivery

By Announcement

Jenkins Community Celebrates 15 years of Continuous Delivery Automation and Innovation at DevOps World | Jenkins World

DEVOPS WORLD | JENKINS WORLD, SAN FRANCISCO – AUGUST 14, 2019 – The Jenkins project, comprised of the community of practitioners using Jenkins®, today celebrated its 15th birthday at DevOps World | Jenkins World with a recap of milestones showcasing the community’s growth and the project’s defining impact on the global software industry.

Jenkins is the world’s leading open source automation server, used by companies large and small to implement continuous integration (CI) and continuous delivery (CD). Originally developed in 2004 and called Hudson, Jenkins’ impact has grown consistently over the years to the point where experts regularly describe Jenkins as the de facto tool for CI/CD.

So far this year, the Jenkins project has been a key driver in the formation of the Continuous Delivery Foundation, has continued to experience strong uptake in the use of Jenkins and will recognize key contributors to the Jenkins project around the globe during Thursday’s keynote session at DevOps World | Jenkins World.

In establishing the Continuous Delivery Foundation, the Jenkins community worked with The Linux Foundation, CloudBees, Google and Netflix to create a new foundation for the diverse CI/CD space. In addition to Jenkins, the Continuous Delivery Foundation was established with several other CI/CD open source projects, including Jenkins X, Spinnaker and Tekton. It serves as a vehicle to develop, nurture and promote open source projects, best practices and industry specifications related to continuous delivery.

The Continuous Delivery Foundation fosters vendor-neutral collaboration between the industry’s top developers, end users and vendors to further CI/CD best practices and industry specifications. Its mission is to grow and sustain projects that are part of the broad and growing continuous delivery ecosystem.

“This has been a great year for Jenkins, the Continuous Delivery Foundation and open source collaboration as a whole,” said Chris Aniszczyk, vice president at the Linux Foundation. “We all share a common mission – to support community-based development of projects that advance the state of software delivery. The Jenkins project has been squarely behind this effort from day one and today the community is stronger than ever.”

Also playing a key role in Jenkins’ transition to the CDF was CloudBees’ Tracy Miranda. Miranda took on the dual roles of CloudBees director of open source community and member of the governing board of the CDF. “CD is becoming a differentiator for organizations in every industry, yet adoption remains challenging. It’s an industry-wide problem that needs an industry-wide solution. From the CloudBees perspective, we see it as critical to have a neutral foundation where all agents of change can collaborate and contribute openly,” said Miranda. “Looking ahead to the next 15 years, we need to solve the complexity of CD adoption. With the CDF, we are well-equipped to do this in open source – building on top of all that we have learned in the Jenkins community over the years.”

During the period from July 1, 2018, through June 30, 2019, the Jenkins project achieved these milestones:

  • 46% growth in active Jenkins installations1 reporting usage data was in the period August 1, 2018 – July 31, 2019
    The community experienced approximately 46 percent growth in active installations, reaching 265,956 installations as of July 31, 2019, compared to 182,236 installations as of August 1, 2018. Active installations are defined as Jenkins instances that report usage information back to the Jenkins project. This number is not representative of the total Jenkins instances in use worldwide; that number is significantly greater.
  • Approximately 15.8 million Jenkins developers
    A recent Datanyze analysis of the CI vendor landscape showed that about 66 percent of continuous integration is being run on Jenkins. With an estimated 24 million developers, globally, according to Evans Data in its 2019 Global Developer Population and Demographics Study, approximately 15.8 million developers are using Jenkins.
  • 254% growth in Jenkins Pipeline jobs
    Finally, the combined number of defined Jenkins jobs increased during this same period from 19,946,119 in July 2018 to 30,281,905, or 52 percent growth. Specifically, Jenkins Pipeline jobs grew 254 percent in the same period. The dramatic growth in Jenkins Pipeline jobs demonstrates that organizations are accelerating their investment in modern software pipeline automation practices with Jenkins.

“Over the past 15 years, the Jenkins project has revolutionized the way software is built and delivered,” said Jenkins creator Kohsuke Kawaguchi, who also serves as chief scientist at CloudBees. “We have touched every industry and made a difference to every software team in the world.”

Additional Resources

1The Jenkins community tracks statistics from active Jenkins installations that transmit usage information back to the project. The numbers do not represent a majority of Jenkins installations, only those who choose to report. Therefore, the numbers are conservative.

About the Continuous Delivery Foundation
Continuous delivery (CD) is a software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released at any time. The Continuous Delivery Foundation (CDF) is a Linux Foundation initiative that serves as the vendor-neutral home for many of the fastest-growing projects, including Jenkins, Jenkins X, Spinnaker, and Tekton. The CDF fosters collaboration between the industry’s top developers, end users and vendors to further continuous delivery best practices. For more information about the CDF, please visit https://cd.foundation.

Mar 12
Love0

Spinnaker Sets Sail to the Continuous Delivery Foundation

By Blog, Staff

By Andy Glover, Director of Delivery Engineering, at Netflix

This is a contributed Blog from our Premier founding member Netflix on the donation of Spinnaker to CDF. Originally Posted on the Netflix Technology Blog.

Since releasing Spinnaker to the open source community in 2015, the platform has flourished with the addition of new cloud providers, triggers, pipeline stages, and much more. A myriad new features, improvements, and innovations have been added by an ever growing, actively engaged community. Each new innovation has been a step towards an even better Continuous Delivery platform that facilitates rapid, reliable, safe delivery of flexible assets to pluggable deployment targets.

Over the last year, Netflix has improved overall management of Spinnaker by enhancing community engagement and transparency. At the Spinnaker Summit in 2018, we announced that we had adopted a formalized project governance plan with Google. Moreover, we also realized that we’ll need to share the responsibility of Spinnaker’s direction as well as yield a level of long-term strategic influence over the project so as to maintain a healthy, engaged community. This means enabling more parties outside of Netflix and Google to have a say in the direction and implementation of Spinnaker.

A strong, healthy, committed community benefits everyone; however, open source projects rarely reach this critical mass. It’s clear Spinnaker has reached this special stage in its evolution; accordingly, we are thrilled to announce two exciting developments.

First, Netflix and Google are jointly donating Spinnaker to the newly created Continuous Delivery Foundation (or CDF), which is part of the Linux Foundation. The CDF is a neutral organization that will grow and sustain an open continuous delivery ecosystem, much like the Cloud Native Computing Foundation (or CNCF) has done for the cloud native computing ecosystem. The initial set of projects to be donated to the CDF are Jenkins, Jenkins X, Spinnaker, and Tekton. Second, Netflix is joining as a founding member of the CDF.  Continuous Delivery powers innovation at Netflix and working with other leading practitioners to promote Continuous Delivery through specifications is an exciting opportunity to join forces and bring the benefits of rapid, reliable, and safe delivery to an even larger community.

Spinnaker’s success is in large part due to the amazing community of companies and people that use it and contribute to it. Donating Spinnaker to the CDF will strengthen this community. This move will encourage contributions and investments from additional companies who are undoubtedly waiting on the sidelines. Opening the doors to new companies increases the innovations we’ll see in Spinnaker, which benefits everyone.

Donating Spinnaker to the CDF doesn’t change Netflix’s commitment to Spinnaker, and what’s more, current users of Spinnaker are unaffected by this change. Spinnaker’s previously defined governance policy remains in place. Overtime, new stakeholders will emerge and play a larger, more formal role in shaping Spinnaker’s future. The prospects of an even healthier and more engaged community focused on Spinnaker and the manifold benefits of Continuous Delivery is tremendously exciting and we’re looking forward to seeing it continue to flourish.  

Mar 12
Love0

Introducing the Continuous Delivery Foundation, the new home for Tekton, Jenkins, Jenkins X and Spinnaker

By Blog, Staff

By Dan Lorenc and Kim Lewandowski, DevOps at Google Cloud

This is a contributed blog from our Premier founding member Google on the donation of Tekton and Spinnaker to CDF. Originally published on the Google Open Source Blog.

We’re excited to announce that Google is a founding member of the newly formed Continuous Delivery Foundation (CDF). Continuous delivery (CD) is a critical part of modern software development and DevOps practices, and we’re excited to collaborate in a vendor-neutral foundation with other industry leaders.

We’re also thrilled to announce the contribution of two projects as part of our membership: Tekton, and in collaboration with Netflix, Spinnaker. These donations will enter alongside Jenkins and Jenkins X, providing an exciting portfolio of projects for the CDF to expand upon.

Continuous Delivery Foundation

Currently, the continuous integration/continuous delivery (CI/CD) tool landscape is highly fragmented. As companies migrate to the cloud and modernize their infrastructure, tooling decisions become increasingly complicated and difficult. DevOps practitioners constantly seek guidance on software delivery best practices and how to secure their software supply chains but gathering this information can be difficult. Enter the CDF.

The CDF is about more than just code. Modern application development brings new challenges around security and compliance. This foundation will work to define the practices and guidelines that, together with tooling, will help application developers everywhere deliver better and more secure software at speed.

At a foundation level, the CDF will help make CI/CD tooling easier. And at a project level, Tekton helps address complexity problems at their core. We will team up with the open source community and industry leaders to design and build the critical pieces common to CI/CD systems.

Tekton

Tekton is a set of shared, open source components for building CI/CD systems. It provides a flexible, extensible workflow that accommodates deployment to Kubernetes, VMs, bare metal, mobile or even emerging use cases.

The project’s goal is to provide industry specifications for pipelines, workflows, source code access and other primitives. It modernizes the continuous delivery control plane by leveraging all of the built-in scaling, reliability, and extensibility advantages of Kubernetes, and moves software deployment logic there. Tekton was initially built as a part of Knative, but given its stand-alone power, and ability to deploy to a variety of targets, we’ve decided to separate its functionality out into a new project.

Today, Tekton includes primitives for pipeline definition, source code access, artifact management, and test execution. The project roadmap includes adding support for results and event triggering in the coming months. We also plan to work with CI/CD vendors to build out an ecosystem of components that will allow you to use Tekton with existing tools like Jenkins X, Knative and others.

Spinnaker

Spinnaker is an open source, multi-cloud continuous delivery platform originally created by Netflix and jointly led by Netflix and Google. It is typically used in organizations at scale, where DevOps teams support multiple development teams, and has been battle-tested in production by hundreds of teams and in millions of deployments.

Spinnaker is a multi-component system that conceptually aligns with Tekton, and that includes many features important to making continuous delivery reliable, including support for advanced deployment strategies, and Kayenta, an open source canary analysis service.

Given Google’s significant contributions to both Tekton and Spinnaker, we’re very pleased to see them become part of the same foundation. Spinnaker’s large user community has a great deal of experience in the continuous delivery domain, and joining the CDF provides a great opportunity to share that expertise with the broader community.

Next Steps

To learn more about the CDF, listen to this week’s Kubernetes Podcast from Google, where the guest is Tracy Miranda, Director of Open Source Community from our partner CloudBees.

If you’d like to participate in the future of Tekton, Spinnaker, or the CDF, please join us in Barcelona, Spain, on May 20th at the Continuous Delivery Summit ahead of KubeCon/CloudNativeCon EU. If you can’t make it, don’t worry, as there will be many opportunities to get involved and become a part of the community.

We look forward to working with the continuous delivery community on shaping the next wave of CI/CD innovations, alignments, and improvements, no matter where your applications are delivered to.

Close Menu