Investigate the dependency structure in a set of orderly reports. This function allows the dependency graph to be created for set of reports that have been run and committed (the archive) or of a set of reports that could be run (the src) to be discovered and printed to screen. This is experimental and somewhat subject to change and improvement.

Usage

orderly_graph(
name,
id = "latest",
root = NULL,
locate = TRUE,
direction = "downstream",
propagate = TRUE,
max_depth = Inf,
recursion_limit = 100,
show_all = FALSE,
use = "archive"
)

Arguments

name

the name of the report

id

the id of the report, if omitted, use the id of the latest report

root

The path to an orderly root directory, or NULL (the default) to search for one from the current working directory if locate is TRUE.

locate

Logical, indicating if the configuration should be searched for. If TRUE and config is not given, then orderly looks in the working directory and up through its parents until it finds an orderly_config.yml file.

direction

A string indicating if we want to move up or down the tree permitted values are upstream, downstream

propagate

A boolean indicating if we want to propagate out of date through the tree

max_depth

A numeric, how far back should the tree go, this can be useful to truncate a very large tree. (default = Inf)

recursion_limit

A numeric, limit for depth of tree, if the tree goes beyond this then an error is thrown. (default = 100)

show_all

A boolean, should we show all reports in the tree, not just the latest.

use

Character string indicating what we read to infer the dependency tree. Current valid values are archive (the default), which reads from archive reports and src which reads from the source reports.

Value

An orderly tree object with the root corresponding to the given report.

Details

orderly allows a report to rely on the artefacts of one or more other orderly reports. This allows users to develop a network of interconnected reports where the output from report becomes the source of data for another. There are two natural questions that can develop around this workflow:

1. We have updated a report; what are the reports that depend on this so that we can re-run them?

2. We have a report that we want to re-run to ensure uses the latest information. Which other reports are used (directly or indirectly) by this report?

This function displays this information in an easily readable format. Allowing users to see the dependency tree and which reports are out of date and need to be re-run.

Remark

By default the tree is built using data from the local report database (see orderly_commit, orderly_db). This means that it will not find changes from a report that has not be run and committed. That is, if a user changes a report to use or create different artefacts this will not be picked up by the function until the reports are re-run and committed to the archive.

It is possible to generate a tree from the source reports by using use = "src" - this generates the "theoretical tree", and has no concept of being "up to date" or of ids.

Warning

This interface is considered experimental and may change without notice. Please do not depend on it in scripts as it may break things. Consider this a (hopefully) useful way of exploring the dependencies in your reports interactively - let us know what is missing and we'll try and build it out.

Examples

path <- orderly::orderly_example("demo")

id <- orderly::orderly_run("other", root = path, parameters=list(nmin=0))
#> [ name       ]  other
#> [ id         ]  20221116-140029-b63ea013
#> [ sources    ]  functions.R
#> [ parameter  ]  nmin: 0
#> [ start      ]  2022-11-16 14:00:29
#> [ data       ]  source => extract: 20 x 2
#> [ parameter  ]  nmin: 0
#>
#> > extract$number <- extract$number + rnorm(1)
#>
#> > write.csv(extract, "summary.csv", row.names = TRUE)
#>
#> > png("graph.png")
#>
#> > par(mar = c(15, 4, 0.5, 0.5))
#>
#> > do_plot(extract)
#>
#> > dev.off()
#> agg_png
#>       2
#> [ end        ]  2022-11-16 14:00:29
#> [ elapsed    ]  Ran report in 0.01974392 secs
#> [ artefact   ]  summary.csv: 3fac8347e152c84c96e6676413c718b7
#> [ ...        ]  graph.png: a70cdfd037035a0b34e71b921fc9de42
orderly::orderly_commit(id, root = path)
#> [ commit     ]  other/20221116-140029-b63ea013
#> [ copy       ]
#> [ import     ]  other:20221116-140029-b63ea013
#> [ success    ]  :)
#> [1] "/tmp/Rtmpj7MjPt/file466534ab98/archive/other/20221116-140029-b63ea013"
id <- orderly::orderly_run("use_dependency", root = path)
#> [ name       ]  use_dependency
#> [ id         ]  20221116-140030-04cc29e1
#> [ depends    ]  [email protected]:summary.csv -> incoming.csv
#> [ start      ]  2022-11-16 14:00:30
#>
#> > d <- read.csv("incoming.csv", stringsAsFactors = FALSE)
#>
#> > png("graph.png")
#>
#> > par(mar = c(15, 4, 0.5, 0.5))
#>
#> > barplot(setNames(d$number, d$name), las = 2)
#>
#> > dev.off()
#> agg_png
#>       2
#>
#> > info <- orderly::orderly_run_info()
#>
#> > saveRDS(info, "info.rds")
#> [ end        ]  2022-11-16 14:00:30
#> [ elapsed    ]  Ran report in 0.01402497 secs
#> [ artefact   ]  graph.png: a70cdfd037035a0b34e71b921fc9de42
#> [ ...        ]  info.rds: 571769bba04429ad445144b9711bc894
orderly::orderly_commit(id, root = path)
#> [ commit     ]  use_dependency/20221116-140030-04cc29e1
#> [ copy       ]
#> [ import     ]  use_dependency:20221116-140030-04cc29e1
#> [ success    ]  :)
#> [1] "/tmp/Rtmpj7MjPt/file466534ab98/archive/use_dependency/20221116-140030-04cc29e1"
id <- orderly::orderly_run("use_dependency_2", root = path)
#> [ name       ]  use_dependency_2
#> [ id         ]  20221116-140030-3398b4de
#> [ depends    ]  [email protected]:graph.png -> graph.png
#> [ start      ]  2022-11-16 14:00:30
#>
#> > stopifnot(file.exists("graph.png"))
#>
#> > saveRDS(orderly::orderly_run_info(), "third.rds")
#> [ end        ]  2022-11-16 14:00:30
#> [ elapsed    ]  Ran report in 0.003107071 secs
#> [ artefact   ]  third.rds: 07414b56f633490b5229ccfc72946b9d
orderly::orderly_commit(id, root = path)
#> [ commit     ]  use_dependency_2/20221116-140030-3398b4de
#> [ copy       ]
#> [ import     ]  use_dependency_2:20221116-140030-3398b4de
#> [ success    ]  :)
#> [1] "/tmp/Rtmpj7MjPt/file466534ab98/archive/use_dependency_2/20221116-140030-3398b4de"
orderly::orderly_graph("other", root = path)
#> other [20221116-140029-b63ea013]
#> └──use_dependency [20221116-140030-04cc29e1]
#>     └──use_dependency_2 [20221116-140030-3398b4de]
#>
orderly::orderly_graph("use_dependency_2", root = path,
direction = "upstream")
#> use_dependency_2 [20221116-140030-3398b4de]
#> └──use_dependency [20221116-140030-04cc29e1]
#>     └──other [20221116-140029-b63ea013]
#>