The QMSTR sprints 4.2 and 4.3 (this report covers both) are part of our current efforts to merge the ongoing QMSTR development and the FASTEN project. A key necessity for this was documentation. Everybody loves to write documentation, especially developers. Right? :-) To make it even more fun, we work on integrating end user and administrator documentation with the project source code and perform automated integration test automatically. We hope that this approach leads to documentation that works and is reliable. Even though this effort is still ongoing, some results are already visible. One of them is the project documentation on the main web site. Some feature development primarily on the node management command line interface is also in progress. It should be finished in time for the QMSTR 0.4 release, planned for April 17.
To prepare the development work on the quarterly QMSTR milestone releases, the team organizes requirements workshops open to the community. We are currently working on version 0.4, which is expected to be released in early April. 0.5 will be developed in the second quarter of 2019. The 0.5 requirements workshop will take place on April 3 and 4, 2019 at Endocode’s office in Berlin, Germany, right before the Compliance Tooling Group meeting, FOSS-NORTH and LLW 2019. There is a limited availability of 20 seats, so please register early by using the contact form on the web site.
The key goal of QMSTR milestone 4 is to stabilize the APIs and streamline the documentation and tutorials to prepare the integration with the development work that takes part in the FASTEN project and in ACT. To prepare for that, sprint 4.1 focused on refactoring the integration libraries for Go, Java and Python that developers use to create QMSTR modules and client side tools. These libraries help users to create diverse sets of modules and client side code to fit their needs, and facilitate the integration with various services like CI/CD environments.
Version 0.3 of Quartermaster was tagged on February 1, 2019. It
includes improvements to allow for example the Linux kernel, glibc and
openssl as projects under analysis. It delivers support for snapshots
of the knowledge graph, which allows rolling back changes to a known
state, as well as support for source code in assembly language. It
improves the support for
objcopy, and for
analyzing source code elements that are generated during the build and
are not part of the original source code package. Quartermaster is
Free and Open Source software and developed under a collaborative open
governance model. As usual, the source code is available on
Github. Read more
for all the details on the new release.
After another quarter of intense software development, we are proud to announce the availability of Quartermaster v0.2. Quartermaster is a toolchain that automates the analysis and documentation of Open Source license compliance. Software vendors - businesses as well as Open Source communities - deploy Quartermaster in their build pipelines to create compliance documentation while software package share being created. With the new version, Quartermaster learns to ingest SPDX formatted source code manifests, adds a client library for developing analyzer or reporter modules in the Python programming language, adds support for running multiple build processes on the same hardware concurrently, and much more. Quartermaster is Free and Open Source software and developed under a collaborative open governance model. Get the source code from Github while it is hot! Read more for all the details on the new release.
Quartermaster Milestone 1 Development Report: Voilà, a modular, extendable FOSS Compliance Toolchain
Version 0.1 is here. After a proof-of-concept, plenty of drafting, feedback and discussions, a prototype, and finally three months of development focused on creating a useful product, we are tagging a first version of Quartermaster. The theme of the first version was to implement the toolchain basics: the compliance knowledge graph, the master container, the elemental workflow with a construction, analysis and reporting phase, and the APIs for modules to interact with the knowledge graph in each of these phases. There are public showcases that demonstrate the functionality implemented so far. After gathering functional and legal requirements, the team will now move on to milestone 2, where we will focus on making use of the building blocks from the first version to implement badly needed functions of generating license compliance documentation - an SPDX manifest analyzer, integration with Fossology, and features to aggregate analysis results from different sources into reports.
Spring is coming, and so is Quartermaster v0.1! One more sprint to get there. Sprint 5 saw the implementation of initial support for Gradle based Java projects, a finished definition of the reporting API, the introduction of “easy mode” (see below), improved author detection and more steps towards an automated HTML reporter. Sprint 5 was the last round of new features before Quartermaster v0.1 will be wrapped up for release. We will use the final sprint in this quarter to tie up loose ends, polish, containerise, document and demonstrate, and to prepare the v0.1 release. The first milestone will be concluded with the v0.2 requirements workshop on April 11.
After the basics of the Quartermaster toolchain are in place, we focused on a refactoring of the master APIs. The key goal is to make implementing modules for the three phases straightforward, composable and as simple as possible. All modules now run in separate processes that communicate with the master over the network. This decouples both the module and the master code, as well as the licensing models of the modules and the master. It is now possible to run multiple analysers configurable from the master. Analyzers may report multiple or overlapping findings of the same type, like copyright holders, without interfering with each other. Based on this, we can now implement modular reporters that generate HTML and SPDX, or notifications, or other kinds of feedback.
In sprint #3 towards Quartermaster v0.1, we made it easer to package
the master into a Docker container using
multi-stage Docker file,
qmstr-container repository into the
qmstr one, set up
more CI to endure quality of the master HEAD and the incoming PRs,
extended license detection with ScanCode and Ninka, and began
implementing the reporting API endpoint. You can now see the build and
test results both for incoming PRs as well as for the master mainline
branch. Finally we prepared Quartermaster to be presented at
Open Source Leadership Summit. We
made good progress, even though the setup is still a bit rough at the
edges. It works, but the APIs are not yet as modular as we want them
to be. We will focus on that in the next sprint.
It feels great when a plan comes together. After the requirements
workshop on January 17, the Quartermaster team had a pretty good
understanding of what the architecture of the toolchain should look
like. Two weeks into development, we now see that the design is solid,
and that the runtime phases and modular analysis and reporting
concepts work well. The details of the architecture will be explained
in an upcoming blog post. Right now, we are making good progress
towards a working system. The
curl demo was extended, we added the
qmstr-cli tool to manage the master, and set up the first internal