Skip to main content
search
Tag

Screwdriver

Apr 09
Love0

Screwdriver: Build cache – Disk Strategy

By Blog, Project

Screwdriver now has the ability to cache and restore files and directories from your builds to either s3 or disk-based storage. Rest all features related to the cache feature remains the same, only a new storage option is added. Please DO NOT USE this cache feature to store any SENSITIVE data or information.

The graph below is our Internal Screwdriver instance build-cache comparison between disk-based strategy vs aws s3.

Build cache – get cache – (disk strategy)

image

Build cache – get cache – (s3)

image

Build cache – set cache – (disk strategy)

image

Build cache – set cache – (s3)

image

Why disk-based strategy?

Based on the cache analysis, 1. The majority of time was spent pushing data from build to s3, 2. At times the cache push fails if the cache size is big (ex: >1gb). So, simplified the storage part by using a disk cache strategy and using filer/storage mount as a disk option. Each cluster will have its own filer/storage disk mount.

NOTE: When a cluster becomes unavailable and if the requested cache is not available in the new cluster, the cache will be rebuilt once as part of the build.

Cache Size: 

Max size limit per cache is configurable by Cluster admins.

Retention policy:

Cluster admins are responsible to enforce retention policy.

Cluster Admins:

Screwdriver cluster-admin has the ability to specify the cache storage strategy along with other options like compression, md5 check, cache max limit in MB

Reference: 

  1. https://github.com/screwdriver-cd/screwdriver/blob/master/config/default.yaml#L280
  2. https://github.com/screwdriver-cd/executor-k8s-vm/blob/master/index.js#L336
  3. Issue: https://github.com/screwdriver-cd/screwdriver/issues/1830

Compatibility List:

In order to use this feature, you will need these minimum versions:

Contributors:

Thanks to the following people for making this feature possible:

Screwdriver is an open-source build automation platform designed for Continuous Delivery. It is built (and used) by Yahoo. Don’t hesitate to reach out if you have questions or would like to contribute: http://docs.screwdriver.cd/about/support.

Mar 04
Love0

From Screwdriver – Improvements and Fixes

By Blog, Project

Originally posted on the Screwdriver blog, by the Screwdriver Team from Verizon Media

UI

  • Enhancement: Upgrade to node.js v12.
  • Enhancement: Users can now link to custom test & coverage URL via metadata.
  • Enhancement: Reduce number of API calls to fetch active build logs.
  • Enhancement: Display proper title for Commands and Templates pages.
  • Bug fix: Hide “My Pipelines” from Add to collection dialogue.
  • Enhancement: Display usage stats for a template.
image

API

Store

Compatibility List

In order to have these improvements, you will need these minimum versions:

  • UI – v1.0.491
  • API – v0.5.851
  • Store – v3.10.5

Contributors

Thanks to the following contributors for making this feature possible:

Questions and Suggestions

We’d love to hear from you. If you have any questions, please feel free to reach out here. You can also visit us on Github and Slack.

Feb 20
Love0

DevClass: Continuous Delivery Foundation adds Screwdriver to incubation toolkit

By In The News

The Continuous Delivery Foundation has recruited its first incubation project since its birth just a year ago, in the shape of container focused build service Screwdriver.

Screwdriver was originally spawned at Yahoo as “simplified interfacing” for Jenkins, before it was open sourced in 2016 and “completely rebuilt to handle deployments at scale along with CI/CD goals.”

According to the CDF, the project “ties directly into DevOps teams’ daily habits. It tests pull requests, builds merged commits, and deploys to any environment. It also defines load tests, canary deployments, and multi-environment deployment pipelines with ease.”

READ MORE

Feb 19
Love0

Screwdriver Joins CD Foundation as Its First Incubation Project – Treating Continuous Delivery as a First-Class Citizen in the Build Pipeline

By Announcement, Blog

The Continuous Delivery Foundation (CDF), a vendor-neutral home for many of the fastest-growing projects for continuous delivery, is announcing Screwdriver as its newest incubation project. Screwdriver is a self-contained, pluggable service to help developers build, test, and continuously deliver software using the latest containerization technologies. Screwdriver was originally developed by Yahoo, now Verizon Media, as simplified interfacing for Jenkins. It was open sourced in 2016 and completely rebuilt to handle deployments at scale along with CI/CD goals.

Screwdriver ties directly into DevOps teams’ daily habits. It tests pull requests, builds merged commits, and deploys to any environment. It also defines load tests, canary deployments, and multi-environment deployment pipelines with ease.

Begin contributing to Screwdriver today. Pull requests are always welcome. Start by browsing the Screwdriver contributing guide.

“The CD Foundation welcomes Screwdriver. We believe Screwdriver is off to an excellent start, and we’re excited to be working together. By joining the CD Foundation, Screwdriver will be able to scale more quickly, taking greater strides forward in development and deployment,” said Dan Lopez, CDF program manager. “With so many supported integrations, Screwdriver provides the openness and flexibility that DevOps teams require.”

“The Screwdriver team and platform are heroes at Yahoo and Verizon Media for helping us run our massive software engineering operations at scale. Together we can make your CI/CD team heroes at your company too. We invite you to work with us in this neutral home for open source excellence.” Gil Yehuda, Sr. Director of Open Source, Verizon Media/Yahoo.

“It’s great to see Screwdriver joining the CDF. I know the people behind the project are passionate about the same thing we are, and together we can make a bigger impact faster. Open source has a proven unique ability of achieving that across project boundaries,” said Kohsuke Kawaguchi, Co-CEO at Launchable, Inc.

The CD Foundation provides a wide range of services to projects, and the first step is starting as an Incubation Project. Full details on bringing an open source CI/CD project to the CDF are available here.

“Our team is thrilled to join the CDF. Together with our partners from Yahoo! Japan and all our external contributors we’ll continue to rapidly deliver solutions which support developer workflows and interoperability with various Continuous Delivery solutions.” Jithin Emmanuel, Sr. Engineering Manager, Verizon Media/Yahoo and Product Owner for Screwdriver.

For more information on getting involved with Screwdriver, please visit:

CD Foundation Resources

Close Menu