Jorge Vicente Cantero
Today we’re happy to release our (initial) plans for the Scala Platform Process. If you’re unfamiliar with previous calls for a Scala Platform, the idea is simple; let’s reduce the Scala standard library to a very small core, and open up the decision about what popular libraries should be shipped in a “batteries included” distribution of Scala. This is the basic idea of the Scala Platform.
The Scala Platform is the logical continuation of the modularization effort
that began in Scala 2.11. Ultimately, the scala-library should be a small core,
that consists of the Scala types we’ve come to think of as part of the language,
such as the collections and most classes defined directly in the scala package.
The modularization of xml, swing and parser combinators has made it easier for
community maintainers to step up, and the SPP will continue opening up the
standard library to community involvement.
However, bringing many open source libraries together, isn’t easy, nor is is it easy to ensure that these essential libraries continue to be developed. This is why we’ve come up with and proposed an initial draft of what we call the Scala Platform Process. The goal of the Scala Platform Process is simple; provide infrastructure (for building, continuous integration, and releasing) as well as help with project governance (processes and guidelines to recruit contributors) to try and make it easier on platform library authors to ensure that maintenance and releases of their library can scale beyond just them alone.
Infrastructure
The infrastructure that the Scala Center provides includes:
- sbt-platform plugin,
- continuous integration, and
- a release manager.
The sbt-platform plugin bundles together many tasks required for Scala Platform libraries such as; configuring automatic nightly builds, automatic releases of stable versions when a git tag is found, MiMa compatibility checks and PGP signatures for artifacts (with Scala Platform keys), integration with our Drone setup for writing your own sbt scripts, and other configuration that is oftentimes tedious to manage.
Continuous integration (CI) is provided by Drone, on an EPFL cluster. The CI integration comes with a default Scala template that requires minor customization, and yet allows users to remotely configure the build, store their tokens, send notifications and act on certain events.
When new changes are forwarded to the corresponding projects’ platform-release
branch, our release manager releases them every night. If anything goes wrong,
it will notify maintainers via email with an error log.
Governance
The Scala Platform process also establishes process and policies to enable outside contributors, companies, and industrial users to contribute back to the modules, and to help along the modules’ maintenance.
The goals of these policies are to create and evolve friendly communities around the modules and to provide a way for libraries to synchronize on releases and updates while still providing stability guarantees to users. For that, the process offers contracts for maintaining and contributing to modules as well as a predictable release process.
The aforementioned policies can be best summed up as a slightly modified version of the C4 contract, Pieter Hintjens’ set of rules that made ZeroMQ successful.
The Collective Code Construction Contract (C4) contract
The C4 process can perhaps be most succinctly motivated by its goals, taken directly from the C4 specification (dead link: rfc.zeromq.org/spec:42/C4/):
C4 is meant to provide a reusable optimal collaboration model for open source software projects. It has these specific goals:
- To maximize the scale and diversity of the community around a project, by reducing the friction for new Contributors and creating a scaled participation model with strong positive feedbacks;
- To relieve dependencies on key individuals by separating different skill sets so that there is a larger pool of competence in any required domain;
- To allow the project to develop faster and more accurately, by increasing the diversity of the decision making process;
- To support the natural life cycle of project versions from experimental through to stable, by allowing safe experimentation, rapid failure, and isolation of stable code;
- To reduce the internal complexity of project repositories, thus making it easier for Contributors to participate and reducing the scope for error;
- To enforce collective ownership of the project, which increases economic incentive to Contributors and reduces the risk of hijack by hostile entities.
As an example, the C4 contract sets rules for developer interaction such as what happens when someone submits a patch, how users should create pull-requests and motivate them, how maintainers can evolve and document APIs, or how they should deal with bad actors in case of a conflict.
Our modifications to the process focus on minor Scala Platform requirements and licensing, where all modules can choose their open-source licenses so long as they are compatible with the modules already in the Platform. For those that don’t have one yet, Mozilla Public License v2 is recommended. The complete description of the rules is available here.
A predictable release process
The predictable release process allows potential contributors to more easily reason about releases. It consists of three release pipelines:
- nightly (for beta testers and early adopters),
- stable (for users), and
- bug fixes (in case that critical bugs are found).
While nightlies are published automatically every night, stable versions of platform libraries are released cyclically every 12 weeks, if a release is ready.
The release process asks modules to use semantic versioning to reflect breaking behaviour. Therefore, major releases happen every 18 months or when new Scala versions are released. Maintainers that want to experiment with major changes can use the @experimental annotation, whose semantics are documented here.
Modules also need to be cross-compiled with the latest two Scala major versions and, in order to ensure Platform’s long-term compatibility, bug fixes are accepted and released for major Platform releases in the last 24 months. Those modules that fall unmaintained are removed in every major release.
The current draft
We think that the benefits of these policies are twofold: they help maintainers and contributors to organize themselves and they make it easier for outside contributors, companies and organizations to engage in the development of these libraries.
With the help of the Scala Platform Committee members, we have drafted these processes in the Scala Platform specification.
These policies are open for discussion and we hope that the Scala community will help us improve them!
The Committee
The Scala Platform Committee is exclusively composed of members of the Scala community. They set the direction of the Platform, review proposals from the community, and decide on the modules that join the Platform.
The members of the Committee are dedicated developers with a long record of contributions to the Scala ecosystem, the development of the language and major open-source projects. They have been selected to represent and serve the broad interest of the Scala community, not their personal or employers’ viewpoints.
- Dale Wijnand (@dwijnand)
- Aleksandar Prokopec (@axel22)
- Lars Hupel (@larsrh)
- Marius Eriksen (@mariusae)
- Bill Venners (@bvenners)
- Konrad Malawski (@ktoso)
- Pathikrit Bhowmick (@pathikrit)
- Alexander Podkhalyuzin (@Alefas)
- Mathias Doenitz (@sirthias)
- Rex Kerr (@Ichoran)
- David Hall (@dlwh)
We need your help!
To make the Scala Platform successful, we need the help from the Community. Here’s how you can get involved!:
- Make suggestions to improve the Scala Platform policies on our Scala Contributors Discourse Forum.
- Create a Scala Platform proposal;
- Discuss and give feedback on current proposals on our Scala Contributors Discourse Forum;
The process has been designed to “give feedback early, give feedback often” on any proposal. We encourage you to get your hands dirty and submit yours!
SPP Meetings
Stay tuned, you can follow our regular SPP meetings, which are broadcast live on YouTube at the Scala Process YouTube Channel!