A story of how we moved from Microsoft Word to AsciiDoc for the collaboration on files whilst also reducing overhead for formatting to a minimum.

Context

To facilitate preparation for the iSAQB® CPSA-Foundation Level® examination, the iSAQB® maintains a freely available mock examination, containing around 40 questions. In addition, a second document containing the correct solutions to the questions is available. These document-pairs are maintained in several languages, starting with English, German and Spanish.

Stakeholders of the Mock Exam and its translations.
Stakeholders of the Mock Exam and its translations.

Right from its creation in mid-2020, the iSAQB® used Microsoft Word (*.docx) for this purpose in a disctinct GitHub repository, allowing for public issue tracking and change requests.
Questions, answers and different language versions were all kept in separate documents. Each version was then converted to PDF files which are available for download.

As the mock exam has been actively used by more than 100 people since then, dozens of change requests were raised and numerous bugs identified in either questions or anwsers. That resulted in frequent updates to the *.docx files without any proper option to see what had changed in each commit. Language versions got out of sync, and due to the lack of proper diff capabilities, reviews of changes became increasingly difficult.

Start from scratch: Collecting requirements

We (the authors, Ben and Gernot) started improving the maintenance and publishing process of these documents by collecting some requirements:

  1. Questions and corresponding answers need to be absolutely in sync. The (PDF) output for questions and answers shall be contained in two separate documents, but the content (source) shall be a single source. See below for an excerpt of a hypothetical question document (left side) plus answer document (right side).
  2. Multiple language versions (English, German, Spanish and others) need to be kept in sync.
  3. Downloadable documents shall be PDF, formatting should follow iSAQB® conventions, using the same layout and fonts that are used for other official iSAQB® documents.
  4. Changes to both questions and answers shall be documented by a detailed changelog, so that trainers of iSAQB® courses and workshops can track those changes and update their own derived artifacts accordingly. Side note: One of our colleagues maintains the mock examination as an online quiz with automatic score calculation.
  5. Contributions and suggestions for improvements shall be simple and transparent (like in GitHub pull requests)
  6. The tooling shall be platform independent, as contributors cannot be forced to use a specific operating system.

Overall approach

Several other official iSAQB® documents are already maintained in AsciiDoc format, layouted and published by a fully automated workflow. As the existing approach has been working smoothly for more than 2 years for multiple curricula in multiple languages, we decided to take that as a starting point. As an added benefit, these proven and public tooling allows everyone to provide insights on and suggestions for improvements to the mock exam by using GitHub issues, or even create pull requests.

Our workhorse is an automated document build process based upon Gradle and AsciiDoctor. We assume that you (dear readers) know about markup languages like Markdown and AsciiDoc, therefore we will skip an introduction to their underlying concepts (the link section below contains some references).

AsciiDoc for complex documents

AsciiDoc has been created with large and complex documents in mind, which is why it provides some highly useful features: Our most valuable of those is the flexible include: The diagram below shows our modularization approach, which is centered around the separation of concerns principle. Starting from the root document, an asciidoc file, other files are included. These files are either content files or structure files:

Mock exam document structure

As we love the separation-of-concern principle so much, we factored out a few parts into separate files:

Mock Exam Document Structure
Mock Exam Document Structure

Multi-lingual documents

Remember our requirement No. 2: Support multiple languages. Our content files contain text in several languages (currently English and German).

How come our output documents are either English or German?

The answer lies in the combination of the include statement with the powerful AsciiDoctor tag mechanism plus our automated build.

Let’s begin with the tagged-include. See the following diagram for an example:

Include with tag: Only the language specific parts are included
Include with tag: Only the language specific parts are included

Within our content documents, the specific translations are placed within a pair of tags= as shown in the diagram above and the AsciiDoc example below. The important part of the include statement is contained in the square brackets: [tags="EN"]. Only the parts of each AsciiDoc document are included, which are written between the beginning // tag::EN[] and end // end::EN[] of such a named tag. These tag statements are always written in a commented line, so they can never be confused with real document content.

// tag::EN[]
What is the result of "6 * 7"?
// end::EN[]

// tag::DE[]
Was ist das Ergebnis von "6 * 7"?
// end::DE[]
Writing multi-lingual in AsciiDoc

Now we’re getting close—but how to build two different languages with the same structure file? This is where the concept of AsciiDoctor variables comes into play: Our include statements contain the language as a variable, which itself is set by the Gradle build process. The real include statement, taken from the q-structure.adoc file, reads as follows:

include::question01.adoc[{include_configuration}]
Include with variable

The {include_configuration} variable is either set to “EN” and “DE”, depending on what language version shall be generated. Neat, right? This include-tag-build combination allows us to implement the “single-source” principle, keeping a single examination question in multiple languages in a single file.

Creating question and answer sheets

But we’re still not finished: Apart from generating multiple languages in distinct PDF documents, we need to generate a separate document for questions and answers.

For this purpose, we make use of AsciiDoctor variables again, combined with the ifdef:: directive. Take a look at the following snippet, taken from our setup.adoc file (indentation added for better readability):

:withAnswers!:
   :n: [ ]
   :y: [ ]
ifdef::withAnswers[]
   :n: [ ]
   :y: [X]
endif::withAnswers[]
AsciiDoctor variable declaring symbols for yes/no

We declare two variables, :n: (no, representing a wrong answer) and :y (yes, representing a correct answer). The approximately 40 different examination question files are all written as follows (some formating tags excluded for better readability):

// tag::EN[]

=== Question 3 

A-Question:   Select one option      1 Point

What is the result of "6 * 7"

{n}  (a)  67
{n}  (b)  13
{y}  (c)  42

// end::EN[]
Question Files in iSAQB Mock Exam

The third and correct answer is preceeded by the yes variable {y}, as AsciiDoctor variables are declared within :, but used within {}.

Again, our Gradle build sets the variable :withAnswers: to both true and false subsequently, resulting in one document generated with only [ ]. Due to the flexibility of Gradle, it’s easy to change the name of the generated build artifacts to “mock-exam-questions-en.pdf” when the withAnswers variable is false, and to “mock-exam-answers-en.pdf” when it’s true.

Finally, it’s done! We created a build matrix for documents! Really cool, and incredibly more fun than editing a docx binary document.

But wait (again)—what about styling and layout? We need to use the official iSAQB® font (actually, a free Roboto font).

Layout and styling with AsciiDoctorPDF

AsciiDoc can be transformed directly to PDF via AsciiDoctorPDF. There we can configure formatting and styles in YAML: Logos, spacing, margins, table formats, fonts, anything you need. We outsourced this configuration to a separate Git repository—an approach already in use with other iSAQB® documents. We add this repository as Git submodule, which may sound cumbersome at first, but it allows us to change essential layout options centrally (for example, adding a new logo to the headers of all documents, not only the mock-exam discussed here). In such a case, the individual document repositories only need to update the pdf-theme submodule via the Git command git submodule update.

We’re honest with you: Configuring styling and layout in YAML isn’t exactly for the fainthearted. But as the braver one of us (Ben) has already conquered this dungeon in his quest for properly layouting various curricula, we could easily re-use his former effort.

Automatic releases with GitHub Actions

Alright, let’s recap: We have our toolchain (Gradle/AsciiDoctorPDF), content and structure files, the possibility to create language- and content-specific results, and a styling configuration that ensures beautiful and consistent output. All that’s missing is some automation. Since all our stuff is already on GitHub, it only seems natural to use GitHub Actions to automatically create new releases. We defined some workflow scripts which generate the PDF files for us, for a specific version. All releases are available from the iSAQB® GitHub overview page.

In addition to the script that creates real releases, we also added automated builds for pull requests and the main branch of the repository. This guarantees that no broken files or code are checked in with a pull request that would break our release build.

The only fiddly part of the release process is the definition of the release version number. At the moment, it has to be defined in its own file and adjusted manually with each new release. However, we’re already working on extracting that information from the release tags that we use to mark our releases and trigger the automated build.

The Future

As examination candidates usually fail at the same questions, trainers always have to explain these questions over and over again. We currently think of including explanations for the most difficult questions in the answer documents to give candidates the chance to understand why specific options are correct or wrong.

To enable these explanations, we have added another variable to our include concept, called :explanation:. If this variable is set during build, then we will also include content written between the tag::[explanation]. As of March 2021, this has not been used in any of the mock examination questions, but in our opinion it’s only a matter of time until the first volunteers contribute explanations or links, to facilitate examination preparation for future iSAQB candidates.

Conclusion

Although binary document formats like *.docx are ubiquitous and accepted, they impose restrictions not suitable for complex document requirements. A purely textual format like AsciiDoc enables highly flexible configuration and generation options—plus adjustable options for output and styling. Besides, it’s in line with established collaborative development processes and more fun for developers. Did we mention that AsciiDoctor rocks? It allows for great modularization and seperation-of-concern.

Acknowledgements

Thanks to the numerous contributors to the content of the iSAQB® mock examination and the members of the iSAQB® Foundation Level Working Group. Our work of initially creating and maintaining the toolchain that now drives most iSAQB® documents was made possible by countless contributions by other volunteers: Special thanks to Ralf D. Müller, Peter Götz, Alexander Heusingfeld, Alexander Lorz and Roger Rhoades.

Without the awesome underlying open source tools like AsciiDoc, AsciiDoctor, Gradle and AsciiDoctorPDF, the approach described in this post would not have been possible. We are grateful to the authors and maintainers of these libraries and frameworks!

Despite having families and non-IT hobbies, doing open source work is a source of pleasure and satisfaction for both of us. May this post motivate others to invest time and energy in free software.

Links and further information

TAGS