It’s 2020, so what’s our New Year’s Resolution? #1, Make Spinnaker the Perfect Continuous Delivery Platform.
🤦 Voltaire said “perfect is the enemy of good,” and we’ve seen some resolution-minded ads lately reviving that adage (I’m thinking of Michael Phelps reminding me that Progress IS perfection : ) Striving for perfection in software development can lead to obsolete products. So, we hack. We listen to our users and iterate. When we do that as a community, we can progress towards something truly brilliant. Spinnaker’s progress was perfection in 2019, and by all accounts will exceed that trajectory in 2020.
Enterprise Adoption Crescendo (of Production workloads)
Spinnaker saw promising early adoption from large companies like Target and Adobe, and this year has been no exception. While literally everyone books stays on the site, oblivious to digital transformation, Airbnb is using Spinnaker to migrate from monolith to service-oriented architecture, and from brittle deployments to continuous delivery. SAP joyfully leverages Spinnaker on its mission to run the world better, and Pinterest uses it to boost productivity as it pioneers visual discovery. Transunion stays ahead of the fintech curve, providing total credit protection through applications it now deploys with Spinnaker, a more full-featured fit for ephemeral infrastructure than its previous Ansible solution. Companies like Comcast, going all-in on Kubernetes as a software-defined datacenter, have added Spinnaker to manage deployment pipelines. Meanwhile, Salesforce has adopted Spinnaker to bake images based on both Helm charts for Kubernetes and Packer templates for VMs, to support its complex software delivery requirements.
In 2019, we proudly welcomed engineers from new enterprises, including JPMorgan Chase and Home Depot, into the Spinnaker community. Now more than 175 companies have contributed to the project, with over 200 new ICs just last year, and many more companies have become key stakeholders, using and extending Spinnaker. These demonstrate Spinnaker as the mature CD solution, proven to handle production workloads flexibly and scalably.
Organic Growth Through Governance
As adoption continues to rise, and our community grows, it becomes crucial to create a growth-adapted project space. A transparent structure for building and maintaining the project invites new companies and users to take an active role in shaping Spinnaker’s future. To that end, 2019 saw the governance process and entities created in 2018 codified in GitHub, along with a process for modifying it via PRs and votes from members of the TOC and Steering Committee. Spinnaker governance also blossomed into an active space encompassing 8 community-initiated SIGs which organize contributors around feature growth and maintenance in areas of interest. SIGs welcome anyone to join, and we saw growing attendance from end-user companies in H2 2019. As the TOC experiments with public Open Office Hours, Spinnaker Slack is always open, and welcomes nearly 8500 participants to troubleshoot, chat with a SIG team, or reach out to a community member any time. Coupled with the donation of the project to the CDF, these growth factors signal the founding of Spinnaker as a neutral, democratized project space. Our goal? Fuel rapid innovation as we work to empower humanity to deliver their innovations, faster.
#BetterTogether: Driving SDLC Ecosystem Innovation
What came out of this investment in 2019? Where to begin…! An OSS ecosystem thrives with modular components that allow operators to optimize for business goals and maintain compliance. As our user base grows, the problem set expands, use cases vary, and we innovate across a richer toolchain. This allows us to create a smarter, more automated experience. Case(s) in point:
- New data sources added for SignalFX and New Relic, to inform Automated Canary Analysis decisions that let app owners sleep instead of being paged.
- A new Gremlin integration allowing chaos experimentation in Spinnaker pipelines will expand in 2020 to provide results useful for automated decision support.
- Integrations with artifact repositories Nexus and JFrog’s Artifactory have added new native triggers for Spinnaker pipelines.
- New end-to-end secrets management dynamically decrypts Spinnaker secrets as needed for validation and deployment from a backing store of your choosing, such as Vault or S3.
Since interoperability is crucial to Spinnaker, implementing a reliable plugin system was a key 2019 milestone. As our community leverages Spinnaker to solve problems, we must remove friction from the dev’s experience in contributing those extensions to the project. A plugin framework provides libraries and application context to devs, and defines clear extension points to start from when integrating something new. In 2019, we adopted PF4J as our backend plugin-loader framework. In 2020, we’ll implement plugin loading in the Spinnaker UI, and foster community around building and sharing plugins, to enrich our ecosystem.
Cloud Providers – Raining Champions : P
Spinnaker depends on cloud provider investment in extending the project for deployment to the ever-growing variety of ephemeral infra solutions. In 2019, engineers at Google developed a blueprint for a production-ready Spinnaker instance on GKE, integrated with Google Cloud services such as Cloud Build. Amazon Engineers have extended cloud providers for AWS services, ensuring that we can deploy with Spinnaker to any attribute available in Fargate or ECS (Elastic Container Service). As of this year, that includes any task definition attribute. AWS also added full support for deployments to serverless applications using AWS Lambda, including the ability to use Lambda functions as ALB targets.
Migrating to the cloud alleviates headaches, while bringing new operational challenges. Spinnaker evolves to capture and solve for these new challenges as we encounter them. The extendable Swabbie service, created in 2019, tackles the tedium (and potential nightmare at scale!) of reaping unused resources programmatically, to help optimize cloud spend. With Swabbie, an operator can set rules for cleanup candidates via YAML, and clean resources according to a configurable schedule. Deployments to highly automated cloud environments prompted enablement of dynamic account creation, discovery, and configuration for Cloud Foundry and Kubernetes accounts in the cloud.
Upleveling Functionality = Perfect Progress
The Kubernetes V2 Provider for Spinnaker also came into its own last year, offering the ability to deploy, delete, scale, and roll back K8S manifests as artifacts managed as code. The Kubernetes SIG iterated fast to improve the V2 user experience by surfacing more kubectl commands in the Spinnaker UI, and improving management of rollout strategies. They also enhanced traffic management to enable more deployment patterns with the provider, such as blue/green (AKA red/black) and dark canary. In 2020, simplifying the Kubernetes developer experience is an important roadmap element, and the community will tackle it by visualizing more K8S resources in the Spinnaker UI, and improving terminology, error, and workflow management.
Under the hood, 2019 saw lots of effort to provide operators the option to back Spinnaker with a MySQL database instead of Redis. Stateful data in Spinnaker enables event routing and orchestration for pipelines, integrated CI and SCM events, and Swabbie cleanup notifications. The choice of whether to use a relational DB or in-memory store to manage that data gives operators the freedom to optimize performance for their workloads and infrastructure. This makes all that effort, which required updating several microservices, including Echo, the eventing service, and Orca, the orchestration engine, well worth it. Likewise, updates to the Authorization model have allowed even more granular permissions to be durably API-driven across the platform.
A Bright Future Won’t Blind Us (to Your Story)
One high-level 2020 goal aims to better incorporate user stories and enterprise use cases into Spinnaker’s trajectory. The steering committee has committed to building a roadmap that tells high-level stories about using Spinnaker to solve problems. Tool chain interoperability, notably with Kubernetes, cloud providers, and monitoring systems figure large in the H1 2020 Roadmap. Managed Delivery, an exciting Spinnaker CD initiative incubating at Netflix, uniquely responds to a common narrative around software delivery. It uses declarative automation to alleviate the operational knowledge and maintenance burden that comes with ownership of modern, continuously delivered applications.
Users can help us tell the best Spinnaker stories by submitting comments and issues describing usage and business context. Please visit Spinnaker.io (which the Docs SIG will overhaul in 2020) and check out our Success Stories page. Join us on Spinnaker Slack or in the Spinnaker org and tell your tale!
