Skip to contents

Download dependent reports from an orderly remote. This can only be used if the orderly_config.yml lists a remote. This allows for a centralised workflow where a central orderly store exists and holds the canonical copies of reports, from which versions can be downloaded into local stores.

Usage

orderly_pull_dependencies(
  name = NULL,
  root = NULL,
  locate = TRUE,
  remote = NULL,
  parameters = NULL,
  recursive = TRUE
)

orderly_pull_archive(
  name,
  id = "latest",
  root = NULL,
  locate = TRUE,
  remote = NULL,
  parameters = NULL,
  recursive = TRUE
)

orderly_push_archive(
  name,
  id = "latest",
  root = NULL,
  locate = TRUE,
  remote = NULL
)

Arguments

name

Name of the report to download dependencies for. Alternatively, the default of NULL is useful if you have already set the working directory to be the source directory.

root

The path to an orderly root directory, or NULL (the default) to search for one from the current working directory if locate is TRUE.

locate

Logical, indicating if the configuration should be searched for. If TRUE and config is not given, then orderly looks in the working directory and up through its parents until it finds an orderly_config.yml file.

remote

Description of the location. Typically this is a character string indicating a remote specified in the remotes block of your orderly_config.yml. It is also possible to pass in a directly created remote object (e.g., using orderly_remote_path(), or one provided by another package). If left NULL, then the default remote for this orderly repository is used - by default that is the first listed remote.

parameters

Parameters to pass through when doing dependency resolution. If you are using a query for id that involves a parameter (e.g., latest(parameter:x == p)) you will need to pass in the parameters here. Similarly, if you are pulling a report that uses query dependencies that reference parameters you need to pass them here (the same parameter set will be passed through to all dependencies).

recursive

Logical, indicating if all dependencies of a report should also be pulled. Setting this to FALSE only the direct reports, along with metadata for the dependencies; this will be potentially much faster, but leaves your archive in a more fragile state.

id

The identifier (for orderly_pull_archive). The default is to use the latest report.

Value

No return value, these functions are called only for their side effects

Details

The orderly_pull_archive function pulls report directly (without it being a dependent report).

After setting your username up you can run orderly_pull_dependencies("reportname") to pull the dependencies of "reportname" down so that "reportname" can be run, or you can run orderly_pull_archive("reportname") to pull a copy of "reportname" that has been run on the remote server.

Pulling an archive report from a remote also pulls its dependencies (recursively), and adds all of these to the local database. This may require migrating old orderly archives (orderly_migrate()). Note that this migration will likely fail for remote orderly versions older than 0.6.8 because the migration needs to read data files on disk that are not included in the downloaded archive in order to collect all the information required for the database. In this case, ask the administrator of the remote orderly archive to migrate their archive, and then re-pull.

Pushing an archive is possible only if the remote supports it. Currently this is supported by orderly_remote_path() remotes, though not by orderlyweb remotes. There is no control over what will accept a push at this point, nor any check that what you've pushed is "good" except that it exists in your archive. As with pulling an archive, pushes are recursive with respect to dependencies. The configuration interface here will likely change a little over time.

See also

orderly_remote_path(), which implements the remote interface for orderly repositories at a local path. See also OrderlyWeb for a system for hosting orderly repositories over an HTTP API. vignette("remote", package = "orderly") describes the remote system in more detail.

Examples

# Suppose we have a "remote" orderly repository at some path.
# This might be read-only for you in practice and available via a
# network filesystem or a dropbox folder synced to your computer.
# We'll populate this with a pair of reports:
path_remote <- orderly::orderly_example("demo")
id <- orderly::orderly_run("other", list(nmin = 0),
                           root = path_remote, echo = FALSE)
#> [ name       ]  other
#> [ id         ]  20220714-150513-b079cd9b
#> [ sources    ]  functions.R
#> [ parameter  ]  nmin: 0
#> [ start      ]  2022-07-14 15:05:13
#> [ data       ]  source => extract: 20 x 2
#> [ parameter  ]  nmin: 0
#> [ end        ]  2022-07-14 15:05:13
#> [ elapsed    ]  Ran report in 0.0172143 secs
#> [ artefact   ]  summary.csv: 3fac8347e152c84c96e6676413c718b7
#> [ ...        ]  graph.png: a70cdfd037035a0b34e71b921fc9de42
orderly::orderly_commit(id, root = path_remote)
#> [ commit     ]  other/20220714-150513-b079cd9b
#> [ copy       ]
#> [ import     ]  other:20220714-150513-b079cd9b
#> [ success    ]  :)
#> [1] "/tmp/RtmpeHSYKu/file4615f35736/archive/other/20220714-150513-b079cd9b"
id <- orderly::orderly_run("use_dependency",
                           root = path_remote, echo = FALSE)
#> [ name       ]  use_dependency
#> [ id         ]  20220714-150513-f366c871
#> [ depends    ]  [email protected]:summary.csv -> incoming.csv
#> [ start      ]  2022-07-14 15:05:13
#> [ end        ]  2022-07-14 15:05:13
#> [ elapsed    ]  Ran report in 0.01205134 secs
#> [ artefact   ]  graph.png: a70cdfd037035a0b34e71b921fc9de42
#> [ ...        ]  info.rds: 433238d0a0dd8a0e8e89c8d23dae8f96
orderly::orderly_commit(id, root = path_remote)
#> [ commit     ]  use_dependency/20220714-150513-f366c871
#> [ copy       ]
#> [ import     ]  use_dependency:20220714-150513-f366c871
#> [ success    ]  :)
#> [1] "/tmp/RtmpeHSYKu/file4615f35736/archive/use_dependency/20220714-150513-f366c871"

# We'll create a an object to interact with this remote using
# orderly_remote_path.
remote <- orderly::orderly_remote_path(path_remote)

# We can use this object directly
remote$list_reports()
#>  [1] "changelog"          "connection"         "default-param"     
#>  [4] "global"             "html"               "interactive"       
#>  [7] "minimal"            "multi-artefact"     "multifile-artefact"
#> [10] "other"              "slow1"              "slow10"            
#> [13] "slow3"              "spaces"             "use_dependency"    
#> [16] "use_dependency_2"   "use_resource"       "use_resource_dir"  
#> [19] "view"              
remote$list_versions("other")
#> [1] "20220714-150513-b079cd9b"

# More typically one will interact with the functions
# orderly_pull_archive and orderly_pull_dependencies.

# Now, suppose that you have your "local" copy of this; it shares
# the same source (ordinarily these would both be under version
# control with git):
path_local <- orderly::orderly_example("demo")

# If we wanted to run the report "use_dependency" we need to have
# a copy of the report "other", on which it depends:
try(orderly::orderly_run("use_dependency", root = path_local))
#> [ name       ]  use_dependency
#> Error in orderly_find_report(id, name, config, draft = use_draft, must_work = TRUE) : 
#>   Did not find archive report other:latest

# We can "pull" dependencies of a report before running
orderly::orderly_pull_dependencies("use_dependency", remote = remote,
                                   root = path_local)
#> [ depends    ]  use_dependency has 1 dependency
#> [ pull       ]  other:20220714-150513-b079cd9b
#> [ import     ]  other:20220714-150513-b079cd9b

# Now we can run the report because we have a local copy of the
# dependency:
orderly::orderly_run("use_dependency", root = path_local)
#> [ name       ]  use_dependency
#> [ id         ]  20220714-150514-5d002132
#> [ depends    ]  [email protected]:summary.csv -> incoming.csv
#> [ start      ]  2022-07-14 15:05:14
#> 
#> > d <- read.csv("incoming.csv", stringsAsFactors = FALSE)
#> 
#> > png("graph.png")
#> 
#> > par(mar = c(15, 4, 0.5, 0.5))
#> 
#> > barplot(setNames(d$number, d$name), las = 2)
#> 
#> > dev.off()
#> agg_png 
#>       2 
#> 
#> > info <- orderly::orderly_run_info()
#> 
#> > saveRDS(info, "info.rds")
#> [ end        ]  2022-07-14 15:05:14
#> [ elapsed    ]  Ran report in 0.01283407 secs
#> [ artefact   ]  graph.png: a70cdfd037035a0b34e71b921fc9de42
#> [ ...        ]  info.rds: f941c8125ee84a396a5623a971468d20
#> [1] "20220714-150514-5d002132"

# We can also directly pull previously run reports:
orderly::orderly_pull_archive("use_dependency", id, remote = remote,
                              root = path_local)
#> [ pull       ]  use_dependency:20220714-150513-f366c871
#> [ depends    ]  other/20220714-150513-b079cd9b
#> [ pull       ]  other:20220714-150513-b079cd9b already exists, skipping
#> [ import     ]  use_dependency:20220714-150513-f366c871
orderly::orderly_list_archive(root = path_local)
#>             name                       id
#> 1          other 20220714-150513-b079cd9b
#> 2 use_dependency 20220714-150513-f366c871