From version 0.6.0,
orderly supports the concept of a “changelog”. This turns out to be somewhat more complicated than expected, so the details are explained here.
From the user perspective, the changelog functionality should be fairly simple. Alongside any
orderly.yml file, one can have a
changelog.txt file, which will contain description of changes. This file will look like:
[public] Started working with new version of the data. This includes everything sent up to 2018-10-10 [internal] Fixed incorrect plotting
The short strings within
] are a label - everything between a label and the next label or the end of file is a value (these can span multiple lines, contain blank lines, etc).
Over time, the changelog is prepended, i.e., new information is added to the top of the changelog. Existing entries must be left unaltered.
Messages can be provided to
orderly run on the command line) and these are required to be in the format
Once a report is run (via
orderly::orderly_run(), or via
orderly run on the command line), the given changelog, along with any message, is compared with the last committed version of this report, and entries that are introduced on this round are identified. We add the report id to the entries, and a randomly generated unique id to each new entry. This is saved in
orderly_run.rds along with other report metadata.
Dear intrepid reader, you can stop reading now unless you are interested in the details of how we have implemented the changelog - this turned out to be a bit fiddly and this section documents some of the issues.
There are a few complications here
- we need to accept changelog entries that are not present in a file (e.g., from the
- we don’t know the id until after running a report so we can’t easily structure a changelog file in the conventional sense. Instead we have to detect “old” and “new” entries
- detecting “old” entries is complicated by the fact that there might be multiple places that reports are run (development machine, staging environment, production environment) but ultimately one source of truth
This section documents the logic involved in making this work.
When preparing to run an orderly report:
- we read in the plaintext changelog if it exists
- parse all messages provided and add as if they were a changelog entry
- we look for the “latest” archived version of this report and read the changelog from that report’s
orderly_run.rdsfile. Alternatively we can use a remote copy of orderly via the interface here (not implemented yet).
- filter the previous changelog to remove any
from_file = FALSEentries and confirm that our new changelog can be prepended to the previous
- add the
idto the “new” entries and prepend this - save the resulting data into the new report’s
For checking against the API, we will use
GET reports/:name/versions/version/latest/changelog/ with the current report name as