1 - Develop, adapt and maintain computational health economic models

ready4 software framework libraries provide tools for authoring, documenting and maintaining living and transferable health economic models analyses.

1.1 - Template module

ready4 provides a template to facilitate modular implementation of health economic models.

This below section renders a vignette article from the ready4 library. You can use the following links to:

Motivation

A potentially attractive approach to modelling complex health systems is to begin with a relatively simple computational model and to progressively extend its scope and sophistication. Such an approach could be described as “modular” if it is possible to readily combine multiple discrete modelling projects (potentially developed by different modelling teams) that each independently describe distinct aspects of the system being modelled.

Implementation

The ready4 facilitates modular model development by supplying a template module that enables model developers to avail of the encapsulation and inheritance features of Object Oriented Programming (OOP). The ready4 framework uses two of R’s systems for implementing OOP - S3 and S4. An in-depth explanation of R’s different class system is beyond the scope of this article, but is explored in Hadley Wickham’s Advanced R handbook. However, it is useful to know some very high level information about S3 and S4 classes:

  • S4 classes are frequently said to be “formal”, “strict” or “rigorous”. The elements of an S4 class are called slots and the type of data that each slot is allowed to contain is specified in the class definition. An S4 class can be comprised of slots that contain different types of data (e.g. a slot that contains a character vector and another slot that contains tabular data).

  • S3 classes are often described as “simple”, “informal” and “flexible”. S3 objects attach an attribute label to base type objects (e.g. a character vector, a data.frame, a list), which in turn is used to work out what methods should be applied to the class.

Use

ready4 Model Modules

As we use the term, a “model module” is comprised of both a data-structure (an S4 class) and the algorithms (or “methods”) that are associated with that data-structure. Model modules can be created from a template - the ready4 package’s Ready4Module class.

We can create an object (X) from the Ready4Module template using the following command.

X <- Ready4Module()

However, if we inspect X we can see it is of limited use as it contains no data other than an empty element called dissemination_1L_chr.

str(X)
#> Formal class 'Ready4Module' [package "ready4"] with 1 slot
#>   ..@ dissemination_1L_chr: chr NA

The Ready4Module class is therefore not intended to be called directly. Instead, the purpose of Ready4Module is to be the parent class of other model modules. Prototype tools for authoring modules from this template are described here.

ready4 Concept

Module

An instance of Ready4Module (or classes that inherit from Ready4Module) and its associated methods.

ready4 Model Sub-modules

In ready4, S3 classes are principally used to help define the structural properties of slots (elements) of model modules and the methods that can be applied to these slots. S3 classes created for these purposes are called sub-modules.

ready4 Concept

Sub-Module

An instance of an informal (S3) class and its associated methods that describes, validates and applies algorithms to a slot of a Module.

1.2 - Provide end users with a simple and consistent syntax for using model modules

Methods are associated with ready4 modules using a novel syntax.

This below section renders a vignette article from the ready4 library. You can use the following links to:

Motivation

Transparency is one of the underpinning principles of ethical modelling practice. One way to improve the transparency of computational health economic models is to ensure that the programs implementing model analyses can be meaningfully inspected by readers with different levels of technical expertise. Even non-technical readers should be able to follow the high-level logic implemented by model algorithms. If multiple analysis programs are written using a common simplified syntax then reviewers of those programs need to contend with relatively fewer new concepts.

Implementation

ready4 provides a simple syntax that can be consistently applied to attach algorithms (methods) to model modules. It does so by taking advantage of the abstraction and polymorphism features of Object Oriented Programing and R’s use of generic functions. Generic functions don’t implement algorithms themselves - their most salient features are a name and a high level description of the type of task that any method associated with that generic should perform. Whenever a developer creates a method for classes that use R’s S4 and S3 systems (the types used for model modules and sub-modules), they can associate that method with a generic that has a description that is a good match for the algorithm being implemented.

Use

ready4 includes a number of core generic functions which describe the main types of method to be implemented by model modules. These generics correspond exactly to the “core”, “slot” and “extended” commands described in another article.

Notably, the ready4 package does not associate any core or extended methods with the Ready4Module template module. Instead, model developers need to decide which core and extended generics they associate with the modules they derive from the Ready4Module template. This decision is typically made when authoring the methods associated with model modules.

Currently, the only methods defined for Ready4Module are slot-methods. By default, these slot methods are inherited by all modules derived from the Ready4Module template. These methods can be itemised using the get_methods function.

get_methods()
#>  [1] "authorSlot"        "characterizeSlot"  "depictSlot"        "enhanceSlot"       "exhibitSlot"       "ingestSlot"        "investigateSlot"   "manufactureSlot"  
#>  [9] "metamorphoseSlot"  "procureSlot"       "prognosticateSlot" "ratifySlot"        "reckonSlot"        "renewSlot"         "shareSlot"

1.3 - Author and share model modules

Tools from the ready4class, ready4 fun and ready4pack R libraries streamline and standardise the authoring of ready4 modules.

1.3.1 - Authoring model data structures

The ready4class R package supports partially automated and standardised workflows for defining the data structures to be used in computational models.

This below section renders a vignette article from the ready4class library. You can use the following links to:

Motivation

The ready4 framework uses object oriented programming (OOP) to implement modular approaches to computational models of mental health systems. That means that a standardised approach to developing modules (S4 classes) and sub-modules (S3 classes) is required. ready4class provides the tools to implement this workflow.

Workflow

Prototyes, constructor and manifest

The main classes exported as part of ready4class are readyclass_manifest and ready4class_constructor. ready4class_pt_lup is a tibble based ready4 sub-module, which contains metadata on the prototypes of classes that can be used as sub-components of ready4 modules and sub-modules (for example a tibble based class can be used as a slot in an S4 class). When authoring ready4 R packages, you will create a ready4class_pt_lup instance and store it in an online repository that you have write permissions to. As you create new ready4 modules and sub-modules using ready4class tools, your ready4class_pt_lup object will be updated so that these classes can be made available to any future modules or sub-modules that you author. The ready4class_pt_lup sub-module recently used in workflows for authoring ready4 modules is reproduced below.

x <- ready4use::Ready4useRepos(gh_repo_1L_chr = "ready4-dev/ready4",
                               gh_tag_1L_chr = "Documentation_0.0") %>%
  ingest(fls_to_ingest_chr = "prototype_lup",
         metadata_1L_lgl = F) 
x %>%
  exhibit(scroll_box_args_ls = list(width = "100%"))
Class Prototypes Lookup Table
Class Value Namespace Function Default Is Old Class
TTUSynopsis TTUSynopsis() TTU TTUSynopsis FALSE
TTUReports TTUReports() TTU TTUReports FALSE
TTUProject TTUProject() TTU TTUProject FALSE
AusLookup aus::AusLookup() aus AusLookup FALSE
AusOrygen aus::AusOrygen() aus AusOrygen FALSE
AusHeadspace aus::AusHeadspace() aus AusHeadspace FALSE
AusProjections aus::AusProjections() aus AusProjections FALSE
AusTasmania aus::AusTasmania() aus AusTasmania FALSE
AusACT aus::AusACT() aus AusACT FALSE
character NA_character\_ base NA_character\_ FALSE
data.frame data.frame() base data.frame() FALSE
integer NA_integer\_ base NA_integer\_ FALSE
list list(list()) base list list() FALSE
logical NA base NA FALSE
numeric NA_real\_ base NA_real\_ FALSE
POSIXt .POSIXct(NA_character\_) base .POSIXct NA_character\_ FALSE
CostlySource costly::CostlySource() costly CostlySource FALSE
CostlySeed costly::CostlySeed() costly CostlySeed FALSE
CostlyStandards costly::CostlyStandards() costly CostlyStandards FALSE
CostlyCorrespondences costly::CostlyCorrespondences() costly CostlyCorrespondences FALSE
CostlyCountries costly::CostlyCountries() costly CostlyCountries FALSE
CostlyCurrencies costly::CostlyCurrencies() costly CostlyCurrencies FALSE
dfidx dfidx::dfidx(dfidx()) dfidx dfidx dfidx() FALSE
Ready4Module Ready4Module() ready4 Ready4Module FALSE
Ready4Private Ready4Private() ready4 Ready4Private FALSE
Ready4Public Ready4Public() ready4 Ready4Public FALSE
ready4class_constructor ready4class::ready4class_constructor() ready4class ready4class_constructor TRUE
ready4class_pt_lup ready4class::ready4class_pt_lup() ready4class ready4class_pt_lup TRUE
ready4class_manifest ready4class::ready4class_manifest() ready4class ready4class_manifest TRUE
ready4fun_badges ready4fun::ready4fun_badges() ready4fun ready4fun_badges TRUE
ready4fun_abbreviations ready4fun::ready4fun_abbreviations() ready4fun ready4fun_abbreviations TRUE
ready4fun_objects ready4fun::ready4fun_objects() ready4fun ready4fun_objects TRUE
ready4fun_functions ready4fun::ready4fun_functions() ready4fun ready4fun_functions TRUE
ready4fun_executor ready4fun::ready4fun_executor() ready4fun ready4fun_executor TRUE
ready4fun_description ready4fun::ready4fun_description() ready4fun ready4fun_description TRUE
ready4fun_metadata_a ready4fun::ready4fun_metadata_a() ready4fun ready4fun_metadata_a TRUE
ready4fun_metadata_b ready4fun::ready4fun_metadata_b() ready4fun ready4fun_metadata_b TRUE
ready4fun_manifest ready4fun::ready4fun_manifest() ready4fun ready4fun_manifest TRUE
ready4fun_dataset ready4fun::ready4fun_dataset() ready4fun ready4fun_dataset TRUE
ready4pack_manifest ready4pack::ready4pack_manifest() ready4pack ready4pack_manifest TRUE
ready4show_authors ready4show::ready4show_authors() ready4show ready4show_authors TRUE
ready4show_institutes ready4show::ready4show_institutes() ready4show ready4show_institutes TRUE
ready4show_correspondences ready4show::ready4show_correspondences() ready4show ready4show_correspondences TRUE
Ready4showPaths ready4show::Ready4showPaths() ready4show Ready4showPaths FALSE
Ready4showSynopsis ready4show::Ready4showSynopsis() ready4show Ready4showSynopsis FALSE
ready4use_distributions ready4use::ready4use_distributions() ready4use ready4use_distributions TRUE
ready4use_dataverses ready4use::ready4use_dataverses() ready4use ready4use_dataverses TRUE
ready4use_imports ready4use::ready4use_imports() ready4use ready4use_imports TRUE
ready4use_mapes ready4use::ready4use_mapes() ready4use ready4use_mapes TRUE
ready4use_dictionary ready4use::ready4use_dictionary() ready4use ready4use_dictionary TRUE
Ready4useFiles ready4use::Ready4useFiles() ready4use Ready4useFiles FALSE
Ready4useRaw ready4use::Ready4useRaw() ready4use Ready4useRaw FALSE
Ready4useProcessed ready4use::Ready4useProcessed() ready4use Ready4useProcessed FALSE
Ready4useArguments ready4use::Ready4useArguments() ready4use Ready4useArguments FALSE
Ready4useDyad ready4use::Ready4useDyad() ready4use Ready4useDyad FALSE
Ready4useIngest ready4use::Ready4useIngest() ready4use Ready4useIngest FALSE
Ready4useRepos ready4use::Ready4useRepos() ready4use Ready4useRepos FALSE
Ready4usePointer ready4use::Ready4usePointer() ready4use Ready4usePointer FALSE
Ready4useRecord ready4use::Ready4useRecord() ready4use Ready4useRecord FALSE
ScorzProfile scorz::ScorzProfile() scorz ScorzProfile FALSE
ScorzAqol6 scorz::ScorzAqol6() scorz ScorzAqol6 FALSE
ScorzAqol6Adol scorz::ScorzAqol6Adol() scorz ScorzAqol6Adol FALSE
ScorzAqol6Adult scorz::ScorzAqol6Adult() scorz ScorzAqol6Adult FALSE
ScorzEuroQol5 scorz::ScorzEuroQol5() scorz ScorzEuroQol5 FALSE
sf sf::st_sf(sf::st_sfc()) sf st_sf sf::st_sfc() FALSE
specific_models specific::specific_models() specific specific_models TRUE
specific_predictors specific::specific_predictors() specific specific_predictors TRUE
SpecificParameters specific::SpecificParameters() specific SpecificParameters FALSE
SpecificPrivate specific::SpecificPrivate() specific SpecificPrivate FALSE
SpecificShareable specific::SpecificShareable() specific SpecificShareable FALSE
SpecificResults specific::SpecificResults() specific SpecificResults FALSE
SpecificProject specific::SpecificProject() specific SpecificProject FALSE
SpecificInitiator specific::SpecificInitiator() specific SpecificInitiator FALSE
SpecificModels specific::SpecificModels() specific SpecificModels FALSE
SpecificPredictors specific::SpecificPredictors() specific SpecificPredictors FALSE
SpecificFixed specific::SpecificFixed() specific SpecificFixed FALSE
SpecificMixed specific::SpecificMixed() specific SpecificMixed FALSE
SpecificConverter specific::SpecificConverter() specific SpecificConverter FALSE
SpecificSynopsis specific::SpecificSynopsis() specific SpecificSynopsis FALSE
tbl_df tibble::tibble() tibble tibble FALSE
vicinity_parameters vicinity::vicinity_parameters() vicinity vicinity_parameters TRUE
vicinity_values vicinity::vicinity_values() vicinity vicinity_values TRUE
vicinity_abbreviations vicinity::vicinity_abbreviations() vicinity vicinity_abbreviations TRUE
vicinity_processed vicinity::vicinity_processed() vicinity vicinity_processed TRUE
vicinity_raw vicinity::vicinity_raw() vicinity vicinity_raw TRUE
vicinity_resolutions vicinity::vicinity_resolutions() vicinity vicinity_resolutions TRUE
vicinity_points vicinity::vicinity_points() vicinity vicinity_points TRUE
vicinity_templates vicinity::vicinity_templates() vicinity vicinity_templates TRUE
vicinity_identifiers vicinity::vicinity_identifiers() vicinity vicinity_identifiers TRUE
vicinity_mapes vicinity::vicinity_mapes() vicinity vicinity_mapes TRUE
VicinityLookup vicinity::VicinityLookup() vicinity VicinityLookup FALSE
VicinityMacro vicinity::VicinityMacro() vicinity VicinityMacro FALSE
VicinityMesoRegion vicinity::VicinityMesoRegion() vicinity VicinityMesoRegion FALSE
VicinityMesoArea vicinity::VicinityMesoArea() vicinity VicinityMesoArea FALSE
VicinityMicro vicinity::VicinityMicro() vicinity VicinityMicro FALSE
VicinityProfile vicinity::VicinityProfile() vicinity VicinityProfile FALSE
VicinitySpaceTime vicinity::VicinitySpaceTime() vicinity VicinitySpaceTime FALSE
VicinityArguments vicinity::VicinityArguments() vicinity VicinityArguments FALSE
VicinityLocal vicinity::VicinityLocal() vicinity VicinityLocal FALSE
VicinityLocalRaw vicinity::VicinityLocalRaw() vicinity VicinityLocalRaw FALSE
VicinityLocalProcessed vicinity::VicinityLocalProcessed() vicinity VicinityLocalProcessed FALSE
youthvars_aqol6d_adol youthvars::youthvars_aqol6d_adol() youthvars youthvars_aqol6d_adol TRUE
youthvars_chu9d_adolaus youthvars::youthvars_chu9d_adolaus() youthvars youthvars_chu9d_adolaus TRUE
youthvars_phq9 youthvars::youthvars_phq9() youthvars youthvars_phq9 TRUE
youthvars_bads youthvars::youthvars_bads() youthvars youthvars_bads TRUE
youthvars_gad7 youthvars::youthvars_gad7() youthvars youthvars_gad7 TRUE
youthvars_oasis youthvars::youthvars_oasis() youthvars youthvars_oasis TRUE
youthvars_scared youthvars::youthvars_scared() youthvars youthvars_scared TRUE
youthvars_k6 youthvars::youthvars_k6() youthvars youthvars_k6 TRUE
youthvars_k6_aus youthvars::youthvars_k6_aus() youthvars youthvars_k6_aus TRUE
youthvars_k10 youthvars::youthvars_k10() youthvars youthvars_k10 TRUE
youthvars_k10_aus youthvars::youthvars_k10_aus() youthvars youthvars_k10_aus TRUE
youthvars_sofas youthvars::youthvars_sofas() youthvars youthvars_sofas TRUE
YouthvarsDescriptives youthvars::YouthvarsDescriptives() youthvars YouthvarsDescriptives FALSE
YouthvarsProfile youthvars::YouthvarsProfile() youthvars YouthvarsProfile FALSE
YouthvarsSeries youthvars::YouthvarsSeries() youthvars YouthvarsSeries FALSE

ready4class_constructor is another tibble based ready4 sub-module that summarises the desired features of the ready4 modules and sub-modules that you are authoring. An instance of ready4class_constructor is combined with a ready4fun_manifest sub-module to create a ready4class_manifest sub-module. Instances of ready4class_constructor are most efficiently created using the make_pt_ready4class_constructor function.

Typical use

The most important method included in ready4class is the author method for the ready4class_manifest sub-module, that enhances the author method defined for the ready4fun_manifest so that consistently documented R package classes are also generated.

## Not run
author(y)

Examples

ready4class sub-modules and methods are not intended for independent use, but instead should be deployed as part of ready4pack R package authoring workflow.

Future documentation

It should be noted that some ready4class methods require files of a standardised format to be saved in specific sub-directories of the package data-raw directory. Detailed instructions on how to prepare these files are not yet available, but will be outlined in documentation to be released in 2022.

1.3.2 - Authoring model algorithms

The ready4fun R package supports standardised approaches to code authoring that facilitate partial automation of the documenting of model algorithms.

This below section renders a vignette article from the ready4fun library. You can use the following links to:

Motivation

The ready4 framework uses an object-oriented programming (OOP) approach to implement computational health economic models. One motivation for using OOP is the concept of “abstraction” - making things as simple as possible for end-users of ready4 modules by exposing the minimal amount of code required to implement each method.

However, some users of the ready4 modules will want to “look under the hood” and examine the code that implements module algorithms in much more detail. Reasons to do so include to:

  • gain detailed insight into how methods are implemented;
  • test individual sub-components of methods as part of code verification and model validation checks;
  • re-use sub-components of existing methods when authoring new methods.

To help facilitate achieving these objectives, methods associated with ready4 modules can be de-composed into functions that can be used independent of ready4 modules. However, these functions need to be documented and will be easier to comprehend if they adopt a consistent house style (e.g. naming conventions). ready4fun provides workflow tools (classes, methods, functions and datasets) to achieve these goals.

ready4fun function authoring taxonomies, abbreviations and workflow

The ready4fun package uses taxonomy and abbreviation datasets to ensure standardised function code style and documentation.

Function names begin with a meaningful verb

Consistent with a naming convention popular in the R development community, all functions authored with the ready4 framework need to begin with a verb. Furthermore, the choice of verb is meaningful - it communicates something about the type of task a function implements. For example, all functions beginning with the word “fit” will fit a model of a specified type to a dataset. The definitions of all meaningful verbs used in functions authored for a ready4 framework model implementation can be retrieved using get_fn_types(), which by default returns a dataset instance of the ready4fun_functions submodule.

x <- get_fn_types(gh_repo_1L_chr = "ready4-dev/ready4")
class(x)
#> [1] "ready4fun_functions" "tbl_df"              "tbl"                 "data.frame"
exhibit(x,select_int = 1:2, scroll_box_args_ls = list(width = "100%"))
Meaningful verbs
Verb Description
Add Updates an object by adding new values to new or empty fields.
Assert Validates that an object conforms to required condition(s). If the object does not meet all required conditions, program execution will be stopped and an error message provided.
Bind Binds two objects together to create a composite object.
Calculate Performs a numeric calculation.
Close Closes specified connections.
Fit Fits a model of a specified type to a dataset.
Force Checks if a specified local or global environmental condition is met and if not, updates the specified environment to comply with the condition.
Format Modifies the format of an object.
Get Extracts data from an object.
Import Reads a data object in its native format and converts it to an R object.
Impute Imputes data.
Knit Knits an RMD or Rmarkdown file.
Launch Launches an R Shiny app.
Make Creates a new R object.
Plot Plots data.
Predict Applies a model to make predictions.
Print Prints output to console.
Randomise Randomly samples from data.
Read Reads an R script into memory.
Remove Edits an object, removing a specified element or elements.
Rename Renames elements of an object based on a pre-specified schema.
Reorder Reorders an object to conform to a pre-specified schema.
Replace Edits an object, replacing a specified element with another specified element.
Reset Edits an object, overwriting the current version with a default version.
Rowbind Performs custom rowbind operations on table objects.
Scramble Randomly reorders an object.
Transform Edits an object in such a way that core object attributes - e.g. shape, dimensions, elements, type - are altered.
Unload Performs a custom detaching of a package from the search path.
Update Edits an object, while preserving core object attributes.
Validate Validates that an object conforms to required criteria.
Write Writes a file to a specified local directory.

Function inputs and outputs have meaningful suffices

The type of input (arguments) required and output (return) produced by a function can be efficiently communicated by using meaningful suffices. For example all objects ending in “_chr” are character vectors and all objects ending in “_int” are integer vectors. Definitions of all meaningful suffices used in functions authored for a ready4 framework model implementation can be retrieved using get_obj_types(), which by default returns a dataset instance of the ready4fun_objects submodule.

y <- get_obj_types(gh_repo_1L_chr = "ready4-dev/ready4")
class(y)
#> [1] "ready4fun_objects" "tbl_df"            "tbl"               "data.frame"
exhibit(y, select_int = 1:2, scroll_box_args_ls = list(width = "100%"))
Meaningful suffices
Suffix Description
arr array
chr character
dbl double
df data.frame
dtm date
env environment
fct factor
fn function
int integer
lgl logical
ls list
lup lookup table
mat matrix
mdl model
plt plot
prsn person
r3 ready4 submodule
r4 ready4 module
rgx regular expression
s3 S3
s4 S4
sf simple features object
tb tibble

Consistent use of abbreviations

Further information about the purpose of a function and the nature of its inputs and outputs can be encoded by using naming conventions that make consistent use of abbreviations. A master table of the abbreviations used in a ready4 framework model implementation can be retrieved using get_abbrs(), which by default returns a dataset instance of the ready4fun_abbreviations submodule.

z <- get_abbrs(gh_repo_1L_chr = "ready4-dev/ready4")
class(z)
#> [1] "ready4fun_abbreviations" "tbl_df"                  "tbl"                     "data.frame"
exhibit(z %>% head(50), select_int = 1:2, scroll_box_args_ls = list(width = "100%"))
Abbreviations
Abbreviation Description
... additional arguments
1L length one
1L_chr character vector of length one
1L_chr_ls list of character vectors of length one
1L_dbl double vector of length one
1L_dbl_ls list of double vectors of length one
1L_dtm date vector of length one
1L_dtm_ls list of date vectors of length one
1L_fct factor vector of length one
1L_fct_ls list of factor vectors of length one
1L_int integer vector of length one
1L_int_ls list of integer vectors of length one
1L_lgl logical vector of length one
1L_lgl_ls list of logical vectors of length one
1L_rgx regular expression vector of length one
1L_rgx_ls list of regular expression vectors of length one
1Ls length ones
8d 8 Dimension
8ds 8 Dimensions
abbr abbreviation
abbrs abbreviations
abs absolute
abss absolutes
addl additional
addls additionals
adol adolescent
adol6d Assessment of Quality of Life Six Dimension (Adolescent version)
adols adolescents
alg algorithm
algs algorithms
altv alternative
anlys analysis
anlyss analyses
aqol Assessment of Quality of Life
aqol6d Assessment of Quality of Life Six Dimension
aqol6dU Assessment of Quality of Life Six Dimension Health Utility
arg argument
args arguments
arr array
arr_ls list of arrays
arr_r3 ready4 submodule extension of array
arr_r3_ls list of ready4 submodule extension of arrays
artl article
artls articles
att attribute
atts attributes
aus Australia
AusACT Meta data for processing ACT population projections.
AusHeadspace Meta data for constructing Headspace Centre geometries.
AusLookup Lookup tables for Australian geometry and spatial attribute data.

The ready4fun_abbreviations submodule is searchable. It is therefore possible to see if an abbreviation has been defined for an existing word or phrase…

procure(z,"template")
#> # A tibble: 2 × 3
#>   short_name_chr long_name_chr plural_lgl
#>   <chr>          <chr>         <lgl>     
#> 1 tmpl           template      FALSE     
#> 2 tmpls          templates     TRUE

…and to look-up the meaning of an abbreviation…

procure(z,"org",type_1L_chr = "extension")
#> # A tibble: 2 × 3
#>   short_name_chr long_name_chr plural_lgl
#>   <chr>          <chr>         <lgl>     
#> 1 org            organisation  FALSE     
#> 2 orgs           organisations TRUE

…or whether a potential abbreviation has already been defined.

procure(z,"org", type_1L_chr = "extension", what_1L_chr = "string")
#> [1] "org"  "orgs"

Workflow

Manifest

The main class exported as part of ready4fun is the ready4 sub-module ready4fun_manifest which is used to specify metadata (including details of the repository in which the fn_types_lup, seed_obj_lup_tb and abbreviations_lup objects are stored) for the functions being authored and the R package that will contain them.

Typical Usage

A ready4fun_manifest object is most efficiently created with the aid of the make_pkg_desc_ls and make_manifest functions rather than a direct call to the ready4fun_manifest() function.

## Not run
x <- ready4fun::make_pkg_desc_ls(
  pkg_title_1L_chr = "Your Package Title",
  pkg_desc_1L_chr = "Your Package Description.",
  authors_prsn = c(
    utils::person("Author 1 Name",
      role = c("aut", "cre")
    ),
    utils::person("Author 2 Name", role = c("cph"))
  ),
  urls_chr = c(
    "Package website url",
    "Package source code url",
    "Project website"
  )
) %>%
  ready4fun::make_manifest(
    copyright_holders_chr = "Organisation name",
    custom_dmt_ls = ready4fun::make_custom_dmt_ls(user_manual_fns_chr = c("Functions to be included in main user manual are itemised here")),
    dev_pkgs_chr = c("Any development package dependencies go here"),
    path_to_pkg_logo_1L_chr = "Local path to package logo goes here",
    piggyback_to_1L_chr = "GitHub Release Repository to which supporting files will be uploaded",
    ready4_type_1L_chr = "authoring",
    zenodo_badge_1L_chr = "DOI badge details go here"
  )

The main method defined for ready4fun_manifest is author which, assuming the raw undocumented function files are saved in the appropriate directories, will author an R package in which all functions are consistently documented.

## Not run
author(x)

Examples

The ready4fun_manifest sub-module and its methods along with the make_pkg_desc_ls and make_manifestfunctions are designed to be used as part of the ready4pack R package authoring workflow. That vignette includes links to two examples of where the ready4pack workflow has been used to author R package. To illustrate how readyfun tools used as part of that workflow are used to document functions, we are just going to focus on the program used to create the ready4show package.

That program makes use of ready4fun tools that read all undocumented package functions, performs automated checks to ensure that these functions appropriately use the taxonomies and abbreviations mentioned previously (prompting authors to make specific amendments if they do not) and then rewrites these functions to the package R directory, appending tags (with the aid of the sinew package) that will generate meaningful documentation.

For example, one of the functions to be documented is the knit_from_tmpl, which is transformed to a version with tags. The tags added to all functions are then used to generate the package documentation, including the package manual. Two versions of the ready4show package manual are generated - a slimmed down version for end-users and a more detailed inventory of contents intended for developers.

Future documentation

Detailed guidance for how to apply ready4fun workflow tools has yet to be prepared but is planned for 2024.

1.3.3 - Dissemating citable, documented and quality assured model module libraries

ready4 supports tools to streamline the testing, description and distribution of computational model modules.

This below section renders a vignette article from the ready4pack library. You can use the following links to:

ready4pack is a toolkit for bundling collections of modules for computational health economic models authored with the ready4 framework as R packages that are:

  • Citable (with a Zenodo generated DOI and an algorithm generated CITATION file);
  • Community-minded (applying deprecation conventions supported by lifecycle);
  • Documented (applying a function self-documenting algorithm that extends sinew, deploying a GitHub pages hosted and pkgdown generated website and authoring PDF manuals stored in a GitHub Release via piggyback);
  • Internally consistent implementing automated checks to ensure consistency in naming conventions, etc;
  • Licensed (via a usethis generated GPL-3 license);
  • Quality assured (using continuous integration via GitHub actions and R-CMD-Check); and
  • Versioned (applying usethis version increments).

ready4pack extends ready4 framework tools for authoring module algorithms (ready4fun) and data structures (ready4class) and wraps functions from a number of third party R development workflow tools (such as devtools). ready4pack integrates these tools in a common workflow, while adding tools for authoring and documenting datasets to be shipped with model module R packages.

A combination of the ready4_pack_manifest class and author method are used to implement this workflow. This workflow has been used to author all public versions of the ready4 R packages available in the ready4 github repository.

Workflow

Manifest

The main class exported as part of ready4pack is readypack_manifest list based ready4 sub-module, that extends the ready4fun_manifest and ready4class_manifest sub-modules.

Typical usage

readypack_manifest sub-module is most efficiently created with the aid of the make_pt_ready4pack_manifest function and combines instances of the ready4fun_manifest and ready4class_constructor sub-modules.

x <- make_pt_ready4pack_manifest(ready4fun::ready4fun_manifest(),
                                 constructor_r3 = ready4class::ready4class_constructor()) %>%
  ready4pack_manifest()

The main method defined for readypack_manifest is author which extends the author method for ready4class_manifest to author a consistently documented R package.

## Not run
author(x)

Examples

Workflow example one

The program to author and document the ready4show package is relatively simple and authors:

Workflow example two

The program to author and document the youthvars package is a bit more complex as it includes syntax to create package datasets. In addition to the package datasets, the algorithm creates content corresponding to the previous example, specifically:

Future documentation

A more detailed guide to using ready4pack will be created in 2023.

1.4 - Partially automate maintenance of a modelling project's website

ready4 includes tools that can help maintain the documentation website for a modelling project.

This below section renders a vignette article from the ready4 library. You can use the following links to:

Motivation

Manually keeping track of modules libraries, programs, reporting templates and datasets authored by different teams and stored in different locations can be an onerous undertaking. The ready4 library therefore includes tools to allow a modelling project’s maintainers to perform automated searches for model artefacts and to output tabular summaries of these assets in formats suitable for inclusion on a project documentation website.

Implementation

The ready4 library includes tools to allow a modelling project’s maintainers to partially automate searching for and creating summaries of relevant modelling project assets (e.g. tutorials, releases, etc.) that are suitable for inclusion on documentation website pages.

Use

The documentation website maintenance tools in the ready4 library are designed to be used on a docsy documentation website derived from this template repository. An example of a website created from this template is the readyforwhatsnext model project website, for which source code is available here.

The write_to_render_post is the main ready4 function used specifically for website maintenance tasks. Importantly, the non-CRAN library hugodown needs to be installed to use this function.

write_to_render_post is designed for help overcome practical challenges of rendering RMD or Rmarkdown files (pareticularly those sourced from an individual module library’s documentation website) to Markdown output in an overall modelling project website. Examples of its use are in this script that is run when updating the readyforwhatsnext project website. The RMD / Rmarkdown files rendered by this example script call other useful functions from the ready4 package, such as:

  • get_libraries_tb, update_libraries_tb and print_packages for updating details on module libraries (see this example)

  • make_modules_tb and print_modules for updating details on individual modules (see this example)

  • make_datasets_tb and print_data for updating details on module datasets (see this example)

  • get_datasets_tb, make_dss_tb and make_ds_releases_tbl for updating release statuses of module datasets (see this RMD file and its output)

  • make_programs_tbl for updating details on analysis programs or reporting sub-routines that use model modules (see this RMD file and its associated output as well as this RMD file and its output)

  • make_code_releases_tbl for updating release statuses of module libraries and programs or reporting sub-routines that use model modules (see this RMD file and its output)

2 - Use modules of health economic models to implement repeatable research

ready4 software framework libraries provide tools for finding reusable modules of health economic models, supplying them with compatible data and authoring reproducible analysis scripts.

2.1 - Finding modules of health economic models

The ready4 library provides tools to help retrieve details of model module collections and documentation on individual model modules.

2.1.1 - Find themed collections of ready4 modules

Tools from the ready4 R library can help find details about module libraries from a modelling project.

This below section renders a vignette article from the ready4 library. You can use the following links to:

Motivation

To use and combine health economic model modules it is first useful to find modules that have been developed using a common framework that promotes inter-operability.

Implementation

A table summarising currently available module libraries authored with the ready4 framework within a specified GitHub organisation can be retrieved from an online dataset by using the get_libraries_tb and update_libraries_tb functions.

Use

In the below example we will search for modules from the readyforwhatsnext model. The source code for these modules all reside in the ready4-dev GitHub repository. The value supplied to the gh_repo_1L_chr argument specifies the repository in which a dataset of readyforwhatsnext module libraries is stored.

libraries_tb <- get_libraries_tb(gh_repo_1L_chr = "ready4-dev/ready4") %>% update_libraries_tb(include_1L_chr = "modules", url_stub_1L_chr = "https://ready4-dev.github.io/")

In this example, module libraries have been grouped in to the following thematic model “sections”.

libraries_tb$Section %>% unique()
#> [1] "People"   "Places"   "Programs"

We can use the print_packages function to display a HTML summary of the module libraries currently available for each section. The resulting table summarises the types of module library (e.g. those for description, developing models or predicting with models), the name and purpose of those libraries, the locations from which development and archived library code can be downloaded and details of supporting documentation for each library (e.g. website, manuals and examples).

print_packages(libraries_tb %>% dplyr::filter(Section == "People"))
Type Package Purpose Documentation Code Examples
Describe and Validate Youth Mental Health Dataset Variables Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive 12, 13
Score Multi-Attribute Utility Instruments Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive 14, 15
Model Youth Choice Behaviours Citation , Website , Citation Dev , Archive
Implement Transfer to Utility Mapping Algorithms Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive 16
Explore and Characterise Heterogeneity in Quality of Life Data Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive
Specify Models to Solve Inverse Problems Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive 17
Transform Youth Outcomes to Health Utility Predictions Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive 18
print_packages(libraries_tb %>% dplyr::filter(Section == "Places"))
Type Package Purpose Documentation Code Examples
Model Australian Spatial Data Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive
Model Spatial Features of Health Systems Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive
print_packages(libraries_tb %>% dplyr::filter(Section == "Programs"))
Type Package Purpose Documentation Code Examples
Undertake Health Economic Budget Impact Analysis. Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive
Develop, Use and Share Unit Cost Datasets for Health Economic Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive 19, 20

Details of how to search for details on individual modules is described in another article.

2.1.2 - Find examples of how model modules are used

Tools from the ready4 R library can help find details about the use of individual model modules.

This below section renders a vignette article from the ready4 library. You can use the following links to:

Motivation

To use and combine health economic model modules it is first useful to find modules that have been developed using a common framework that promotes inter-operability.

Implementation

A table summarising currently available module libraries authored with the ready4 framework within a specified GitHub organisation can be retrieved from an online dataset by using the get_libraries_tb and update_libraries_tb functions.

Use

In the below example we will search for modules from the readyforwhatsnext model. The source code for these modules all reside in the ready4-dev GitHub repository. The value supplied to the gh_repo_1L_chr argument specifies the repository in which a dataset of readyforwhatsnext module libraries is stored.

libraries_tb <- get_libraries_tb(gh_repo_1L_chr = "ready4-dev/ready4") %>% update_libraries_tb(include_1L_chr = "modules", url_stub_1L_chr = "https://ready4-dev.github.io/")

In this example, module libraries have been grouped in to the following thematic model “sections”.

libraries_tb$Section %>% unique()
#> [1] "People"   "Places"   "Programs"

We can use the print_packages function to display a HTML summary of the module libraries currently available for each section. The resulting table summarises the types of module library (e.g. those for description, developing models or predicting with models), the name and purpose of those libraries, the locations from which development and archived library code can be downloaded and details of supporting documentation for each library (e.g. website, manuals and examples).

print_packages(libraries_tb %>% dplyr::filter(Section == "People"))
Type Package Purpose Documentation Code Examples
Describe and Validate Youth Mental Health Dataset Variables Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive 12, 13
Score Multi-Attribute Utility Instruments Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive 14, 15
Model Youth Choice Behaviours Citation , Website , Citation Dev , Archive
Implement Transfer to Utility Mapping Algorithms Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive 16
Explore and Characterise Heterogeneity in Quality of Life Data Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive
Specify Models to Solve Inverse Problems Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive 17
Transform Youth Outcomes to Health Utility Predictions Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive 18
print_packages(libraries_tb %>% dplyr::filter(Section == "Places"))
Type Package Purpose Documentation Code Examples
Model Australian Spatial Data Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive
Model Spatial Features of Health Systems Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive
print_packages(libraries_tb %>% dplyr::filter(Section == "Programs"))
Type Package Purpose Documentation Code Examples
Undertake Health Economic Budget Impact Analysis. Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive
Develop, Use and Share Unit Cost Datasets for Health Economic Citation , Website , Manual - Short (PDF) , Manual - Full (PDF) Dev , Archive 19, 20

Details of how to search for details on individual modules is described in another article.

2.2 - Ingest, label and share model data

The ready4use R package provides tools for supplying data to model modules.

2.2.1 - Ingest data from an open access repository

A tutorial from the Acumen website about using ready4 to search and retrieve data from the Australian Mental Health Systems Models Dataverse.

This below section renders a R Markdown program from the Acumen website. You can use the following links to:

1. Objectives

On completion of this tutorial you should be able to:

  • Understand basic concepts relating to the Australian Mental Health Systems Models Dataverse Collection; and

  • Have the ability to search for, download and ingest files contained in Dataverse Datasets that are linked to by the Australian Mental Health Systems Models Dataverse Collection using two alternative approaches;

    • Using a web based interface; and
    • Using R commands.

2. Prerequisites

You can complete most of this tutorial without any specialist skills or software other than having a web-browser connected to the Internet. However, if you wish to try running the R code for finding and downloading files described in the last part of the tutorial, then you must have R (and ideally RStudio as well) installed on your machine. Instructions for how to install this software are available at https://rstudio-education.github.io/hopr/starting.html .

3. Concepts

Before searching for or retrieving data from the Australian Mental Health Systems Models Dataverse Collection, the following concepts are useful to understand:

  • The Dataverse Project is “an open source web application to share, preserve, cite, explore, and analyze research data.” It is developed at Harvard’s Institute for Quantitative Social Science (IQSS). More information about the project is available on the Dataverse Project’s website.

  • There are many Dataverse Installations around the world (85 at the time of writing this tutorial). Each Dataverse Installation is an instance of an organisation installing the Dataverse Project’s software on its own servers to create and manage online data repositories. At the time of writing there is one Australian Dataverse Installation listed on the Dataverse Project’s website, which is the Australian Data Archive.

  • The Harvard Dataverse is a Dataverse Installation that is managed by Harvard University, that is open to researchers from all disciplines from anywhere in the world. More details are available from its website.

  • A Dataverse Collection (frequently and confusingly also referred to as simply a “Dataverse”) is a part of a Dataverse Installation that a user can set up to host multiple “Dataverse Datasets” (see next bullet point). Dataverse Collections typically share common attributes (for example, are in the same topic area or produced by the same group(s) of researchers) and can be branded to a limited degree. Dataverse Collections will also contain descriptive metadata about the purpose and ownership of the collection.

  • A Dataverse Dataset is a uniquely identified collection of files (some of which, again confusingly, can be tabular data files of the type that researchers typically refer to as “datasets”) within a Dataverse Collection. Each Dataverse Dataset will have a name, a Digital Object Identifier, a version number, citation information and details of the licensing/terms of use that apply to its contents.

  • A Linked Dataverse Dataset is a Dataverse Dataset that appears in a Dataverse Collection’s list of contents without actually being in that Dataverse Collection (it is hosted in another Dataverse Collection and is potentially owned and controlled by another user).

  • The Australian Mental Health Systems Models Dataverse Collection (which we will refer to as “our Dataverse Collection”) is a Dataverse Collection of Linked Dataverse Datasets within the Harvard Dataverse. We established our Dataverse Collection in the Harvard Dataverse because of the robustness and flexibility that this service provides. A factor in our choice of the Harvard Dataverse was that the aim of our Dataverse Collection is to promote easy access to non-confidential data relevant to modelling Australian mental health policy and service planning topics. The non-confidential nature of the data means that the additional administrative requirements that some other Dataverse Installations place on users were potentially unnecessary for our specific purposes. As a collection of Linked Dataverse Datasets, our Dataverse Collection can be used by modelling groups as both a centralised location to find relevant data and as an additional promotion / distribution channel to share Dataverse Datasets from their own Harvard Dataverse Collections without surrendering any control over the management of their data (they continue to curate their Dataverse Dataset and can modify Dataverse Dataset contents, metadata and terms of use as they see fit).

3. Search and download dataset files

There are multiple options for searching and downloading files contained in our Dataverse Collection. This tutorial will discuss just two - one based on using a web browser and the other based on using R commands. For details on other options, it is recommended to consult the Harvard Dataverse user guide and (for more technical readers) api guide.

3.1. Web browser approach

Searching and retriving data from our Dataverse Collection via a web-browser is very simple, and this methods is suitable for low volume requests (i.e. occasional use) where reproducibility is not important.

To find and download data using your web browser, implement the following steps:

  • Go to our Dataverse Collection at https://dataverse.harvard.edu/dataverse/openmind

  • Search for the Linked Dataverse Dataset most of interest to you by using the tools provided on the landing page.

  • Click on the link to your selected Dataverse Dataset. Note that by doing so you will leave our Dataverse Collection and be taken to the Dataverse Collection controlled by the Dataverse Dataset’s owner.

  • (Optional) - Click on the “Metadata”, “Terms” and “Versions” tabs or (if available) the Related Publication links to discover more about the dataset. When you are done, click on the “Files” tab to review the files contained in the Dataverse Dataset.

  • Select the files that you wish to download using the checkboxes and click on the “Download” button.

  • When prompted, review any terms of use you are presented with and either accept them or cancel the download as you feel appropriate.

More detail on some of the above steps is available in the following section of the Harvard Dataverse user guide: https://guides.dataverse.org/en/latest/user/find-use-data.html#finding-data

3.2 Using R commands

Some limitations of relying purely on a web-browser are that it is a purely manual approach that can become inefficient for large number of data requests and which is not reproducible (thereby limiting transparency about the specific data items / versions used in an analysis). It can therefore be desirable to explore alternatives that are based on programming commands. Programmatic approaches have the advantage of being more readily incorporated into automated and reproducible workflows.

There are a range of software tools in different languages that can be used to programmatically search and retrieve files in Dataverse Collections. More information on these resources on a dedicated page within the Dataverse Project’s documentation.

One of these tools is dataverse - “the R Client for Dataverse Repositories”. The dataverse R package has a range of functions that are very helpful for general tasks relating to the search and retrieval of files contained in Dataverse Datasets. These functions are not the focus of this tutorial, but you can read more about them on the [packages documentation website]((https://iqss.github.io/dataverse-client-r/).

The remainder of this tutorial is focused on the use of another R package called ready4use which created by Orygen to help manage open-source data for use in mental health models. The ready4use R package extends the dataverse R package and one of its applications is to ingest R objects stored in Dataverse Datasets in the “.Rds” file format directly into an R Session’s working memory. More information about ready4use is available on its documentation website.

3.2.1 Install and load required R packages

As ready4use is still a development package, you may need to first install the devtools package to help install it. The following commands entered in your R console will do this.

utils::install.packages("devtools")
devtools::install_github("ready4-dev/ready4use")

We now load the ready4use package and the ready4 framework for youth mental health modelling that it depends on. The ready4 framework will have been automatically installed along with ready4use.

3.2.2 Specify repository details

The next step is to create a Ready4useRepos object, which in this example we will call X, that contains the details of the Dataverse Dataset from which we wish to retrieve R objects. We need to supply three pieces of information to Ready4useRepos. Two of these items of information will be the same for any data item retrieved from our Dataverse Collection and are the Dataverse Collection identifier (which for us is “openmind”) and the server on which the containing Dataverse Installation is hosted (in our case “dataverse.harvard.edu”). The one item of information that will vary based on your requirements is the name / identifier (DOI) of the Dataverse Dataset from which we wish to retrieve data. In this example we are using the DOI for the “Synthetic (fake) youth mental health datasets and data dictionaries” Dataverse Dataset.

X <- Ready4useRepos(dv_nm_1L_chr = "openmind",
                    dv_server_1L_chr = "dataverse.harvard.edu",
                    dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/HJXYKQ")

Having supplied the details of where the data is stored we can now ingest the data we are interested in. We can either ingest all R object in the selected Dataverse Dataset or just objects that we specify. By default R objects are ingested along with their metadata, but we can choose not to ingest the metadata.

3.2.3 Ingest all R objects from a Dataverse Dataset along with its metadata

To ingest all R objects in the dataset, we enter the following command.

Y <- ingest(X)

We can now create separate list objects for the ingested data and its metadata.

data_ls <- procureSlot(Y,"b_Ready4useIngest@objects_ls")
meta_ls <- procureSlot(Y,"a_Ready4usePointer@b_Ready4useRepos@dv_ds_metadata_ls$ds_ls")

We can itemise the data objects we have ingested with the following command.

names(data_ls)
#> [1] "ymh_clinical_dict_r3" "ymh_clinical_tb"      "eq5d_ds_dict"         "eq5d_ds_tb"           "ymh_phq_gad_tb"

We can also see what metadata fields we have ingested.

names(meta_ls)
#>  [1] "id"                  "datasetId"           "datasetPersistentId" "storageIdentifier"   "versionNumber"       "versionMinorNumber"  "versionState"        "lastUpdateTime"      "releaseTime"         "createTime"         
#> [11] "publicationDate"     "citationDate"        "termsOfUse"          "fileAccessRequest"   "metadataBlocks"      "files"

There can be a lot of useful information contained in this metadata list object. For example, we can retrieve descriptive information about the Dataverse Dataset from which we have ingested data.

meta_ls$metadataBlocks$citation$fields$value[[7]]$dsDescriptionValue$value
#> [1] "The datasets in this collection are entirely fake. They were developed principally to demonstrate the workings of a number of utility scoring and mapping algorithms. However, they may be of more general use to others. In some limited cases, some of the included files could be used in exploratory simulation based analyses. However, you should read the metadata descriptors for each file to inform yourself of the validity and limitations of each fake dataset. To open the RDS format files included in this dataset, the R package ready4use needs to be installed (see https://ready4-dev.github.io/ready4use/ ). It is also recommended that you install the youthvars package ( https://ready4-dev.github.io/youthvars/) as this provides useful tools for inspecting and validating each dataset."

The metadata also contains descriptive information on each file in the Dataverse Dataset.

meta_ls$files$description[5]
#> [1] "Synthetic (fake) population of young people with EQ-5D item responses, psychological distress and well-being and demographic variables. NOTE: This dataset was generated purely for the purposes of demonstrating generalised applications of utility scoring and utility mapping algorithms that we developed. It is not in any meaningful way representative of the likely true population distributions of its included variables. For this reason, it cannot be used as an input for any model designed to inform decision making."

3.2.4 Ingest all R objects from a Dataverse Dataset without metadata

If we wished to ingest only the R objects without metadata, we could have simply run the following command.

data_2_ls <- ingest(X,
                    metadata_1L_lgl = F)

We can see that this ingest is identical to that made using the previous method.

identical(data_ls, data_2_ls)
#> [1] TRUE

3.2.5 Ingest selected R objects

If we only want to ingest one specific object, we can supply its name.

ymh_clinical_tb <- ingest(X,
                          fls_to_ingest_chr = c("ymh_clinical_tb"),
                          metadata_1L_lgl = F)

The output from an object specific call to the ingest method is the requested object.

ymh_clinical_tb %>%
  head()
#> # A tibble: 6 × 43
#>   fkClientID    round    d_interview_date d_age d_gender d_sex_birth_s d_sexual_ori_s d_ATSI d_country_bir_s d_english_home d_english_native d_studying_working  d_relation_s s_centre c_p_diag_s c_clinical_staging_s k6_total phq9_total bads_total
#>   <chr>         <fct>    <date>           <int> <chr>    <chr>         <fct>          <chr>  <chr>           <chr>          <chr>            <chr>               <chr>        <chr>    <chr>      <chr>                   <int>      <int>      <int>
#> 1 Participant_1 Baseline 2020-03-22          14 Male     Male          Heterosexual   No     Australia       Yes            Yes              Not studying or wo… In a relati… Southpo… Other      0-1a                        8          7         96
#> 2 Participant_2 Baseline 2020-06-15          19 Female   Female        Heterosexual   Yes    Other           No             No               Studying only       In a relati… Regiona… Anxiety    0-1a                       13         13         63
#> 3 Participant_3 Baseline 2020-08-20          21 Female   Female        Other          NA     NA              NA             NA               Studying only       Not in a re… Canberra Anxiety    1b                         12         17         72
#> 4 Participant_4 Baseline 2020-05-23          12 Female   Female        Heterosexual   Yes    Other           No             No               Not studying or wo… In a relati… Southpo… Depressio… 2-4                        17         17         75
#> 5 Participant_5 Baseline 2020-04-05          19 Male     Male          Heterosexual   Yes    Other           No             No               Not studying or wo… Not in a re… Southpo… Depressio… 0-1a                       12         22         82
#> 6 Participant_6 Baseline 2020-06-09          19 Male     Male          Heterosexual   Yes    Other           No             No               Studying only       In a relati… Regiona… Anxiety    1b                         11          8        105
#> # ℹ 24 more variables: gad7_total <int>, oasis_total <int>, scared_total <int>, c_sofas <int>, aqol6d_q1 <int>, aqol6d_q2 <int>, aqol6d_q3 <int>, aqol6d_q4 <int>, aqol6d_q5 <int>, aqol6d_q6 <int>, aqol6d_q7 <int>, aqol6d_q8 <int>,
#> #   aqol6d_q9 <int>, aqol6d_q10 <int>, aqol6d_q11 <int>, aqol6d_q12 <int>, aqol6d_q13 <int>, aqol6d_q14 <int>, aqol6d_q15 <int>, aqol6d_q16 <int>, aqol6d_q17 <int>, aqol6d_q18 <int>, aqol6d_q19 <int>, aqol6d_q20 <int>

We can also request to ingest multiple specified objects from a Dataverse Dataset.

data_3_ls <- ingest(X,
                    fls_to_ingest_chr = c("ymh_clinical_tb","ymh_clinical_dict_r3"),
                    metadata_1L_lgl = F)

This last request produces a list of ingested objects.

names(data_3_ls)
#> [1] "ymh_clinical_dict_r3" "ymh_clinical_tb"

2.2.2 - Add a data dictionary to a dataset

Pairing a dataset with its dictionary makes it easier to interpret. This tutorial describes how a module from the ready4use R package can help you to pair a dataset and its dictionary.

This below section renders a vignette article from the ready4use library. You can use the following links to:

Note: This vignette is illustrated with fake data. The dataset explored in this example should not be used to inform decision-making.

ready4use includes a number of tools for labeling health economic model data and forms part of the ready4 framework.

Create a dataset-dictionary pair

A data dictionary contains useful metadata about a dataset. To illustrate this point, we can ingest examples of a toy (fake) dataset and its data-dictionary.

objects_ls <- Ready4useRepos(dv_nm_1L_chr = "fakes",
                    dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/HJXYKQ",
                    dv_server_1L_chr = "dataverse.harvard.edu") %>%
  ingest(metadata_1L_lgl = F)

Importantly (and a requirement for subsequent steps), the data dictionary we ingest is a ready4use_dictionary object.

class(objects_ls$eq5d_ds_dict)
#> [1] "ready4use_dictionary" "ready4_dictionary"    "tbl_df"               "tbl"                  "data.frame"

We can now pair the data dictionary with its dataset in a new object X, a Ready4useDyad.

X <- Ready4useDyad(ds_tb = objects_ls$eq5d_ds_tb,
                   dictionary_r3 = objects_ls$eq5d_ds_dict)

Inspect data

We can inspect X by printing selected information about it to console using the exhibit method. If we only wish to see the first or last few records, we can pass “head” or “tail” to the display_1L_chr argument.

 exhibit(X,
         display_1L_chr = "head",
         scroll_box_args_ls = list(width = "100%"))
Dataset
uid Timepoint data_collection_dtm d_age Gender d_sex_birth_s d_sexual_ori_s d_relation_s d_ATSI CALD Region d_studying_working eq5dq_MO eq5dq_SC eq5dq_UA eq5dq_PD eq5dq_AD K10_int Psych_well_int
1 BL 2019-10-22 14 Male Male Heterosexual In a relationship No No Metro Not studying or working 1 1 1 1 2 11 87
2 BL 2019-10-17 19 Female Female Heterosexual In a relationship Yes Yes Regional Studying only 1 2 1 1 1 14 65
2 FUP 2020-02-14 19 Female Female Heterosexual In a relationship Yes Yes Regional Studying only 3 1 1 1 1 10 71
3 BL 2020-02-15 21 Female Female Other Not in a relationship NA NA Metro Studying only 1 1 3 1 1 13 74
3 FUP 2020-06-14 21 Female Female Other Not in a relationship NA NA Metro Studying only 1 1 2 1 1 10 64
4 BL 2019-12-14 12 Female Female Heterosexual In a relationship Yes Yes Metro Not studying or working 1 1 1 3 1 18 40

The dataset may be more meaningful if variables are labelled using the descriptive information from the data dictionary. This can be accomplished using the renew method.

X <- renew(X,
           type_1L_chr = "label")
exhibit(X,
        display_1L_chr = "head",
         scroll_box_args_ls = list(width = "100%"))
Dataset
Unique identifier Data collection round Date of data collection Age Gender (grouped) Sex at birth Sexual orientation Relationship status Aboriginal or Torres Strait Islander Culturally And Linguistically Diverse Region of residence (metropolitan or regional) Education and employment status EQ5D - Mobility domain score EQ5D - Self-Care domain score EQ5D - Usual Activities domain score EQ5D - Pain / Discomfort domain score EQ5D - Anxiety / Depression domain score Kessler Psychological Distress - 10 Item Total Score Overall Wellbeing Measure (Winefield et al. 2012)
1 BL 2019-10-22 14 Male Male Heterosexual In a relationship No No Metro Not studying or working 1 1 1 1 2 11 87
2 BL 2019-10-17 19 Female Female Heterosexual In a relationship Yes Yes Regional Studying only 1 2 1 1 1 14 65
2 FUP 2020-02-14 19 Female Female Heterosexual In a relationship Yes Yes Regional Studying only 3 1 1 1 1 10 71
3 BL 2020-02-15 21 Female Female Other Not in a relationship NA NA Metro Studying only 1 1 3 1 1 13 74
3 FUP 2020-06-14 21 Female Female Other Not in a relationship NA NA Metro Studying only 1 1 2 1 1 10 64
4 BL 2019-12-14 12 Female Female Heterosexual In a relationship Yes Yes Metro Not studying or working 1 1 1 3 1 18 40

To remove dataset labels, use the renew method with “unlabel” passed to the type_1L_chr argument.

X <- renew(X,
           type_1L_chr = "unlabel")

By default, the exhibit method will print the dataset part of the Ready4useDyad instance. To inspect the data dictionary, pass “dict” to the type_1L_chr argument.

exhibit(X,
        display_1L_chr = "head",
        type_1L_chr = "dict",
        scroll_box_args_ls = list(width = "100%"))
Data Dictionary
Variable Category Description Class
CALD demographic Culturally And Linguistically Diverse factor
d_age demographic age integer
d_ATSI demographic Aboriginal or Torres Strait Islander character
d_relation_s demographic relationship status character
d_sex_birth_s demographic sex at birth character
d_sexual_ori_s demographic sexual orientation factor

2.2.3 - Share data via online repositories

The retrieval and dissemination of data from online data repositories is an essential enabler of open source modelling. This tutorial describes how a module from the ready4use R package can help you to manage this process.

This below section renders a vignette article from the ready4use library. You can use the following links to:

Note: This vignette is illustrated with fake data. The dataset explored in this example should not be used to inform decision-making.

ready4use includes a number of tools for sharing health economic model data that forms part of the ready4 framework.

Identify data to be shared

To illustrate how to share data using ready4use classes and methods, we will first need some data to publish. In this example, we are going to share X, a Ready4useDyad (a data structure explained in another vignette) that we can create using data ingested from an online repository.

objects_ls <- ingest(Ready4useRepos(dv_nm_1L_chr = "fakes",
                                    dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/HJXYKQ",
                                    dv_server_1L_chr = "dataverse.harvard.edu",
                                    gh_repo_1L_chr = "ready4-dev/ready4",
                                    gh_tag_1L_chr = "Documentation_0.0"),
                     fls_to_ingest_chr = c("ymh_clinical_tb","ymh_clinical_dict_r3"),
                     metadata_1L_lgl = F)
X <- Ready4useDyad(ds_tb = objects_ls$ymh_clinical_tb,
                   dictionary_r3 = objects_ls$ymh_clinical_dict_r3) %>%
  renew()

Share data

We now specify where we plan to publish X in Y, a Ready4useRepos object (described in another vignette). Note, you must have write permissions to the repositories you specify in this step. The values entered in this example (the https://doi.org/10.7910/DVN/W95KED dataset from the fakes dataverse will not work for you).

Y <- Ready4useRepos(dv_nm_1L_chr = "fakes", # Replace with values for a dataverse & dataset for which
                    dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/W95KED", #  you have write permissions.
                    dv_server_1L_chr = "dataverse.harvard.edu")

We can now upload X to our preferred data repository using the share method. By default, if more than one data repository was specified in Y, then the dataverse repository will be preferred when sharing. We can overwrite this default by passing either “prefer_gh” or “all” values to the type_1L_chr argument. The Ready4useDyad object is now available for download at https://doi.org/10.7910/DVN/W95KED.

Y <- share(Y,
           obj_to_share_xx = X,
           fl_nm_1L_chr = "ymh_clinical_dyad_r4",
           description_1L_chr = "An example of a Ready4useDyad - a dataset (clinical youth mental health, AQoL-6D) and data dictionary pair. Note this example uses fake data.")

2.3 - Using model modules for repeatable analyses

The ready4 framework provides tools to support the authoring of programs and subroutines to run and report analyses with ready4.

2.3.1 - Commands for using model modules

Apply model modules using a simple and consistent syntax.

This below section renders a vignette article from the ready4show library. You can use the following links to:

Motivation

To be used in health economic analyses, model modules need to be called using a programming syntax. Ideally that syntax should be relatively simple, with the name and description of each command reliably communicating the category of operations it performs.

Implementation

ready4 provides a simple syntax that is used by all model modules developed with the ready4 framework.

Use

A table that itemises ready4 commands along with examples of how these commands are used can be ingested from a periodically updated database using get_methods_tb. In the below example we will search for examples of where that syntax has been used by modules from the readyforwhatsnext model. The value supplied to the gh_repo_1L_chr argument specifies the repository in which a dataset of readyforwhatsnext module libraries is stored.

x <- get_methods_tb(gh_repo_1L_chr = "ready4-dev/ready4")

Core commands

A HTML table of ready4’s core commands and examples of the use of each command can be displayed using the print_methods function, using the return_1L_chr = "core" argument.

print_methods(x,
              return_1L_chr = "core",
              scroll_width_1L_chr = "100%") 
Method Purpose Examples
author Author and save files 5 , 6, 7 , 16 , 17
characterize Characterize model module data by generating (tabular) descriptive statistics
depict Depict (plot) features of model module data 13, 14 , 15
enhance Enhance a model module by adding new elements
exhibit Exhibit features of model module data by printing them to the R console 2 , 5 , 6, 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20
ingest Ingest data 1 , 2 , 3 , 6, 13 , 14 , 15 , 16 , 17 , 18
investigate Investigate solutions to an inverse problem 16 , 17
manufacture Manufacture a new object
metamorphose Metamorphose a model module to a model module of a different (non-inheriting) class 17
procure Procure items from a dataset 5, 17
prognosticate Prognosticate (make predictions) by solving a forward problem
ratify Ratify that input or output data meet validity criteria 13, 17 , 19 , 20
reckon Reckon (calculate) a value
renew Renew (update) values 1 , 2 , 13, 14 , 15 , 16 , 17 , 19 , 20
share Share data via an online repository 1 , 13, 14 , 15 , 16

Applying commands to module “slots”

Each of the “core” commands also has a “slot” version, which applies the command to a specified slot (a named element of a module). Two of these “slot” methods can also be used for additional purposes:

  • procureSlot is a “getter” method - its default behaviour is to return the value of a specified slot. If the argument use_procure_mthd_1L_lgl = T is included in the method call, procureSlot will instead apply the procure method to a specified slot.

  • renewSlot is a “setter” method - if any value other than “use_renew_mthd” (the default) is passed to the new_val_xx argument, that value will be assigned to the specified slot.

A HTML table of slot commands and relevant examples can be displayed using the print_methods function, using the return_1L_chr = "slot" argument.

print_methods(x,
              return_1L_chr = "slot",
              scroll_width_1L_chr = "100%")
Method Purpose Examples
authorSlot Apply the author method to a model module slot
characterizeSlot Apply the characterize method to a model module slot
depictSlot Apply the depict method to a model module slot
enhanceSlot Apply the enhance method to a model module slot
exhibitSlot Apply the exhibit method to a model module slot 4, 17 , 19 , 20
ingestSlot Apply the ingest method to a model module slot
investigateSlot Apply the investigate method to a model module slot
manufactureSlot Apply the manufacture method to a model module slot
metamorphoseSlot Apply the metamorphose method to a model module slot
procureSlot Procure (get) data from a slot 4, 14 , 16 , 17
prognosticateSlot Apply the prognosticate method to a model module slot
ratifySlot Apply the ratify method to a model module slot
reckonSlot Apply the reckon method to a model module slot
renewSlot Renew (set) the values of data in a module slot 4, 14 , 17 , 19 , 20
shareSlot Apply the share method to a model module slot

Extended commands

Finally, there are a small number of other commands that are more general extensions of the core commands. Currently, these extended commands are all variants on the author command, with each extension specifying the type of output to be authored by the method. A HTML table of the extended generics bundled with ready4 can be displayed using the print_methods function, using the return_1L_chr = "extended" argument.

print_methods(x,
              exclude_mthds_for_chr = "Ready4Module",
              return_1L_chr = "extended",
              scroll_width_1L_chr = "100%")
Method Purpose Examples
authorClasses Author and document classes
authorData Author and document datasets 4, 17
authorFunctions Author and document functions
authorReport Author and save a report 4

2.3.2 - Authoring scientific manuscripts

Tools from the ready4show R package support authoring of scientific summaries of analyses with ready4.

This below section renders a vignette article from the ready4show library. You can use the following links to:

Motivation

Open science workflows should ideally span an unbroken chain between data-ingest to production of a scientific summary such as a manuscript. Such extensive workflows provide an explicit means of linking all content in a scientific summary with the analysis that it reports.

Implementation

ready4show includes a number of classes and methods that help integrate manuscript authoring into a reproducible workflow. These tools are part of the ready4 framework for transparent, reusable and updatable health economic models.

Load required libraries

We first begin by loading the libraries we will require to implement this workflow.

By default, methods in the ready4show package will request your consent before writing files to your machine. This is the safest option. However, as there are many files that need to be written locally for this program to execute, you can overwrite this default by supplying the value “Y” to methods with a consent_1L_chr argument.

consent_1L_chr <- "" # Default value - asks for consent prior to writing each file.

Create a synopsis of the manuscript to be authored

To start with we create X, an instance of Ready4showSynopsis, a ready4 module (S4 class). We can use X to record metadata about the manuscript to be authored (including details about the study being summarised and the title and format of the intended output).

X <- Ready4showSynopsis(background_1L_chr = "Our study is entirely fictional.",
                        coi_1L_chr = "None declared.",
                        conclusion_1L_chr = "These fake results are not interesting.",
                        digits_int = 3L,
                        ethics_1L_chr = "The study was reviewed and granted approval by Awesome University's Human Research Ethics Committee (1111111.1).",
                        funding_1L_chr = "The study was funded by Generous Benefactor.",
                        interval_chr = "three months",
                        keywords_chr = c("entirely","fake","do", "not","cite"),
                        outp_formats_chr = "PDF",
                        sample_desc_1L_chr = "The study sample is fake data that pretends to be young people aged 12 to 25 years who attended Australian primary care services for mental health related needs between November 2019 to August 2020.",
                        title_1L_chr = "A hypothetical study using fake data")

Add authorship details

Authorship details can be added to slots of X that contain ready4show_authors and ready4show_instututes ready4 sub-modules.

As we can see from the below call to exhibitSlot, X was created with no authorship information.

exhibitSlot(X,
            "authors_r3",
            scroll_box_args_ls = list(width = "100%")) 
First-name Middle-name Last-name Title Qualifications Institutes Sequence Position Corresponding Email Joint-first

We can add details on each author by repeated calls to the renewSlot method.

X <- renewSlot(X,
          "authors_r3",
          first_nm_chr = "Alejandra",
          middle_nm_chr = "Rocio",
          last_nm_chr = "Scienceace",
          title_chr = "Dr",
          qualifications_chr = "MD, PhD",
          institute_chr = "Institute_A, Institute_B",
          sequence_int = 1,
          is_corresponding_lgl = T,
          email_chr = "fake_email@fake_institute.com") %>%
  renewSlot("authors_r3",
            first_nm_chr = "Fionn",
            middle_nm_chr = "Seamus",
            last_nm_chr = "Researchchamp",
            title_chr = "Prof",
            qualifications_chr = "MSc, PhD",
            institute_chr = "Institute_C, Institute_B",
            sequence_int = 2,
            email_chr = "fake_email@unreal_institute.com") 

The updated authorship table can now be inspected.

X %>%
  exhibitSlot("authors_r3",
              scroll_box_args_ls = list(width = "100%")) 
First-name Middle-name Last-name Title Qualifications Institutes Sequence Position Corresponding Email Joint-first
Alejandra Rocio Scienceace Dr MD, PhD Institute_A, Institute_B 1 TRUE \_institute.com NA
Fionn Seamus Researchchamp Prof MSc, PhD Institute_C, Institute_B 2 NA \_institute.com NA

We now need to add additional information for each author institute.

X <- renewSlot(X,
          "institutes_r3",
          short_name_chr = "Institute_A", 
          long_name_chr = "Awesome University, Shanghai") %>%
  renewSlot("institutes_r3",
            short_name_chr = "Institute_B", 
            long_name_chr = "August Institution, London") %>%
  renewSlot("institutes_r3",
            new_val_xx = "use_renew_mthd",
            short_name_chr = "Institute_C", 
            long_name_chr = "Highly Ranked Uni, Montreal")

The updated institutes table can now be inspected.

X %>%
  exhibitSlot("institutes_r3",
              scroll_box_args_ls = list(width = "100%")) 
Reference Name
Institute_A Awesome University, Shanghai
Institute_B August Institution, London
Institute_C Highly Ranked Uni, Montreal

Add correspondences

We can also add a look-up table about any changes we wish to make from the analysis code of how names of variables / parameters are presented in the manuscript text.

X <- renewSlot(X,
               "correspondences_r3",
               old_nms_chr = c("PHQ9", "GAD7"),
               new_nms_chr = c("PHQ-9", "GAD-7"))

These edits can now be inspected with a call to exhibitSlot.

X %>%
  exhibitSlot("correspondences_r3",
              scroll_box_args_ls = list(width = "100%")) # Add Exhibit Method
Old name New name
PHQ9 PHQ-9
GAD7 GAD-7

Specify output directory

We now update X with details of the directory to which we wish to write the manuscript we are authoring and all its supporting files.

X <- renewSlot(X,
               "a_Ready4showPaths@outp_data_dir_1L_chr",
               new_val_xx = tempdir())

Create dataset of literate programming files

Our next step is to copy a dataset of files that can implement a literate program to generate our manuscript. If you have a template you wish to work with, you can specify its local path using the a_Ready4showPaths@mkdn_source_dir_1L_chr slot of the X. Skip this step if you wish to use the default markdown dataset, which leverages popular rmarkdown toolkits such as bookdown and rticles.

## Not run
# procureSlot(X,
#             "a_Ready4showPaths@mkdn_source_dir_1L_chr",
#             new_val_xx  = "PATH TO MARKDOWN DATASET")

We create the dataset copy with the authorData method.

authorData(X, consent_1L_chr = consent_1L_chr)

Having created a local copy of the template literate program files dataset, it is now possible to manually edit the markdown files to author the manuscript. However, in this example we are skipping this step and will continue to use the unedited template in conjunction with the metadata we have specified in X. We combine the two to author a manuscript using the authorReport method.

authorReport(X, consent_1L_chr = consent_1L_chr)

If we wish, we can now ammend X and then rerun the authorReport method to generate Word and HTML versions of the manuscript.

renewSlot(X,
          "outp_formats_chr",
          new_val_xx = "Word") %>%
  authorReport(consent_1L_chr = consent_1L_chr)
renewSlot(X,
          "outp_formats_chr",
          new_val_xx = "HTML") %>%
  authorReport(consent_1L_chr = consent_1L_chr)

The outputed files are as follows: