SIG

By Fatih Degirmenci

Continuous Delivery Foundation (CDF) Technical Oversight Committee (TOC) approved the formation of Special Interest Group (SIG) Interoperability January 14, 2020. SIG Interoperability aims to increase integration and interoperability across different tools and technologies in the open source CI/CD ecosystem. One of the prerequisites to achieve this is to provide a neutral forum, enabling dialog between projects and end-users so they can come together and discuss their use cases, needs, and challenges. This will allow projects and communities to explore additional collaboration opportunities and increase the visibility of ongoing work.

One of the means the SIG adapted to provide a forum for discussion is to invite representatives of project and end-user communities to regular SIG meetings so they can present what they are doing. The presentations are then followed by open discussions which allows community members to ask questions, raise concerns, and more importantly start talking with each other. However, one of the things the community noticed is the lack of shared terminology and vocabulary as the tools and technologies employ different terms to describe what is often the same thing.

This is actually not a surprising finding since there are many ways to greet someone and as humans if we do not understand the word being used we have the ability to observe body language, process tone, and even touch. These many different natural inputs allow us as humans to establish shared vocabulary upon which we have been able to build successful components relevant to our way of living and social norms of interacting.

Unfortunately for machines, this process is not so easy as we humans have to decide if we want to establish norms which we often surface when talking about machine interactions as protocols and best practices or requirements.

Continuous Integration (CI) and Continuous Delivery (CD) practitioners have many tools at their disposal but it is often the case that what we call a pipeline in today’s tool of choice is not called the same thing in the tool we use tomorrow. Again, we can within our sphere of influence and interaction adjust for these nuances but machines talking to one another do not have that same luxury necessarily.

These are the thoughts that made contributors to SIG to work on vocabulary and terminology as the first thing right after the SIG was approved to be formed because we believe that if we can establish a shared vocabulary across the industry in CI/CD domain, we can remove the barriers between humans so we can start tackling with getting machines to talk to each other. The way this work is envisioned to be done is to collect the existing terms used by CI/CD tools and technologies in a document, and create a mapping of the terms across projects, essentially making the Rosetta Stone for CI/CD domain. We think that we can continue on this work and look for possibilities to come up with shared vocabulary in a collaborative manner.

The document SIG is working on is available in SIG Interoperability repository on GitHub and it currently contains terms for 10 CI/CD projects as shown on the table below.

Due to the fact that when organizations establish CI/CD pipelines, they employ not just CI/CD tools but also Software Configuration Management (SCM) systems, Artifact Repository Managers (ARM) and so on. That’s why we included terminology for SCM tools such as Gerrit, GitHub, and GitLab and we expect to have terms used by other tools in adjacent areas collected as well.

It is important to highlight that we consider this work as still ongoing and we encourage and welcome everyone to add terminology used by the project they use and/or are involved in to the document so we have broader coverage of the tools and technologies. If you also notice that there are things that can be improved, feel free to send a pull request to CDF SIG Interoperability repository and improve the existing documentation.

Continuous Delivery Foundation (CDF) Governing Board (GB) agreed to and ratified 9 strategic goals in early October 2019. One of the strategic goals identified by CDF GB is fostering tool interoperability.

Recognition of the importance of interoperability and identifying it as one of the strategic goals is a very important step for CDF to take for users. Users and organizations employ various CI/CD tools and technologies depending on their needs and where they are in their CI/CD transformation. Organizations often employ more than one tool in various stages of their CI/CD pipelines due to different capabilities provided by the tools and this is perhaps one of the biggest benefits users get by using open technologies for their CI/CD needs. For example, CDF member Salesforce has over 20 different CI/CD tools internally thanks to acquisitions and different requirements in teams. 

However, one of the challenges users face is the lack of interoperability across the CI/CD tools and technologies, resulting in various issues while constructing and running pipelines such as passing metadata and artifacts between the tools or achieving traceability from commit to deployment. Often users end up building their “own glue code” to address what is a common problem, further complicating moving from one tool to another and adopting new technologies and methodologies.

These “glue code solutions” are generally specific to users’ needs and tools rather than being loosely coupled and agnostic to tooling and technology. Additionally these solutions are not visible to other users and the communities, making them vulnerable to the risk of outage in their CI/CD pipelines due to potential changes (i.e. non-backward changes to the APIs, changes in data models) that happen to the tools in respective projects.

Therefore, focusing on tool interoperability is critical. 

There has been significant collaboration going on in this area. Linux Foundation Networking (LFN), OpenStack Foundation (OSF), and Cloud Native Computing Foundation (CNCF) projects have done a lot  to raise awareness of CI/CD interoperability challenges. In addition to these communities, Spinnaker, Jenkins, Tekton, and Jenkins X, CDF founding projects, have been collaborating and sharing ideas. However, there are many more users, projects and communities, either looking for answers to similar interoperability challenges, on their way to developing solutions, or simply trying to find like minded people to work with together.

We believe the work should happen in a neutral forum where users come together with maintainers of open source CI/CD projects and have a dialog about the challenges we need to address. 

Which is why the CDF Interoperability SIG was launched, led by Fatih Degirmenci of Ericsson and with support from representatives from Netflix, Google, China Mobile, CloudBees and others. 

We, the CDF Interoperability SIG, aim to provide such a forum and enable a dialog around interoperability in order to:

• clarify what interoperability means for the CI/CD ecosystem
• promote the need to collaborate on interoperability challenges in a neutral forum
• highlight and promote the needs of the users who face challenges constructing complex end-to-end CI/CD flows and pipelines by employing different tools and technologies
• explore synergies between, and enable collaboration across, the CI/CD projects with regards to interoperability
• pursue solutions which are loosely coupled, scalable, flexible, and tool and technology agnostic
• reduce the need for users to implement in-house solutions by promoting native interoperability between tools
• attract and assist projects that work on interoperability

Membership to the Interoperability SIG is open to the public. We invite users and contributors to open source CI/CD projects to join us to share ideas, use cases, challenges, and solutions with each other. 

Here are some of the ways you can take part in the Interoperability SIG and start collaborating:

• Maillist: https://lists.cd.foundation/g/sig-interoperability
• Slack Channel: #sig-interoperability on CDF Slack
• Git repository: https://github.com/cdfoundation/sig-interoperability

CDF SIG Meets every even week on Thursdays at 15:00UTC on Zoom and the meeting agenda and minutes are available here. Our first meeting will be on January 23, 2020.

Finally, we would like to thank everyone who has listened to our ideas, shared their thoughts, taken part in crafting the proposal, and most importantly, encouraged us with their +1s!