resultcheck

R-CMD_CI_Tests_Badge CRAN_Status_Badge CRAN_Downloads_Badge

Result Stability Checks for Empirical R Projects

Overview

resultcheck provides lightweight helpers for checking whether empirical results remain unchanged across code revisions, platform differences, and package updates. Call snapshot() on key outputs (models, tables, derived datasets) in your analysis scripts to detect unintended result drift automatically during CI or local testing.

Installation

Latest Stable Version

install.packages("resultcheck")

Latest Development Version (Unstable)

# install.packages("devtools")
devtools::install_github("kv9898/resultcheck")

Workflow

The package supports a two-phase workflow:

  1. Interactive development — run your analysis script and call snapshot() on objects you care about. On first run the snapshot is saved as a human-readable .md file. On subsequent interactive runs, differences are shown and you are prompted to update.

  2. Automated testing — wrap your script in setup_sandbox() / run_in_sandbox() / cleanup_sandbox(). Inside run_in_sandbox(), snapshot() switches to testing mode: it errors immediately if a snapshot is missing or has changed, making the test fail.

Integrated Example

with_example() can generate this layout for documentation/testing under tempdir():

myproject/
├── _resultcheck.yml
├── analysis.R
└── tests/
    ├── _resultcheck_snaps/
    │   └── analysis/
    │       ├── model.md
    │       └── model_mismatch.md
    └── testthat/
        └── test-analysis.R

analysis.R — snapshot key results

model <- lm(mpg ~ wt, data = mtcars)
resultcheck::snapshot(model, "model")

tests/testthat/test-analysis.R — automated test

library(testthat)
library(resultcheck)

test_that("analysis produces stable results", {
  sandbox <- setup_sandbox()
  on.exit(cleanup_sandbox(sandbox), add = TRUE)

  expect_true(run_in_sandbox("analysis.R", sandbox))
})

To try this quickly without creating files in your current project:

resultcheck::with_example({
  sandbox <- setup_sandbox()
  on.exit(cleanup_sandbox(sandbox), add = TRUE)
  stopifnot(isTRUE(run_in_sandbox("analysis.R", sandbox)))
})

Function Reference

snapshot(value, name, script_name = NULL, method = NULL)

Creates or verifies a snapshot of any R object.

You can override the default snapshot directory in _resultcheck.yml:

snapshot:
  max_print: 1000
  dir: "custom/snapshots/path"
  method: "print + str"
  method_defaults_file: "snapshot-method-overrides.R"
  method_by_class:
    lm: "summary"

max_print sets the base R printing limit for each snapshot method (default: 1,000 entries). Set it in _resultcheck.yml rather than relying on the session’s options(max.print = ...). It must be a whole number from 1 to 2,147,483,647. The limit counts entries, not rows or bytes: a 200-row, seven-column data frame contains 1,400 entries and needs a higher limit, such as max_print: 2000. Larger base-printed objects may be truncated, with an omission notice; changes in omitted values may not be detected. Increase the limit when full output is needed, or select a meaningful summary method. Class-specific methods such as tibble printing can have their own limits, and custom methods can explicitly override the print limit. The caller’s R options are restored afterwards.

Snapshot serialization also temporarily sets useFancyQuotes = FALSE, so base R quotation marks (including model-summary significance legends) are consistent between interactive sessions and Quarto. The caller’s setting is restored even if a snapshot method fails. Custom methods may explicitly choose other formatting.

The method argument controls how the object is serialized:

Value Behavior
NULL (default) Captures both print() and str()
print Only print() output is captured
str Only str() output is captured
length Any callable function can be used
stats::coef Namespaced callables are supported
list(print = print, summary = summary) Runs multiple functions in order

When a list of methods is used, section headers are taken directly from the method names (or list names), e.g. ## print, ## summary.

If method is omitted, defaults are resolved as:

  1. class override from snapshot.method_by_class,
  2. global default from snapshot.method,
  3. fallback to list(print = print, str = str).

Config methods are parsed from strings in _resultcheck.yml, so values like "print + str" or "stats::coef" are converted to callable methods.

You can also define class defaults in a separate R file:

# snapshot-method-overrides.R
method_by_class <- list(
  lm = "summary",
  glm = "summary"
)

Snapshots are plain text and intended to be committed to version control.

setup_sandbox(files = NULL, temp_base = NULL)

Creates a temporary directory and copies the listed files and/or directories into it, preserving their path structure relative to the project root. Directories are copied recursively. Snapshot files do not need to be listed.

run_in_sandbox(script_path, sandbox = NULL, ...)

Runs an R script inside the sandbox. The working directory is set to the sandbox, but find_root() and snapshot() automatically resolve back to the original project root so snapshots are found correctly.

Returns TRUE invisibly on success, so you can use expect_true(run_in_sandbox(...)) directly in testthat.

cleanup_sandbox(sandbox = NULL, force = TRUE)

Removes the sandbox directory. Omit the argument to clean up the most recently created sandbox.

find_root(start_path = NULL)

Locates the project root by searching upward for a _resultcheck.yml (or legacy resultcheck.yml), .Rproj, or .git marker. Called automatically by snapshot() and setup_sandbox().

Place an empty _resultcheck.yml at your project root to make detection reliable:

# _resultcheck.yml

with_example(code, mismatch = FALSE)

Creates a temporary example project in tempdir(), sets the working directory there while evaluating code, then cleans up automatically.

License

MIT © Dianyi Yang