---
title: "Getting Started with DataAudit"
author: "Vinodh Kumar Obli Rajendran and Keerthi Aaradhana"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Getting Started with DataAudit}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

# Introduction

Data quality is an important prerequisite for reliable statistical analysis,
visualization, modelling, and interpretation.

`DataAudit` provides a structured framework for identifying common
data-quality problems in R. The package combines general data auditing,
individual validation functions, reusable validation rules, structured audit
reports, and data-quality scoring.

A typical workflow is:

1. inspect the dataset;
2. perform a general audit;
3. conduct focused checks when required;
4. define study-specific validation rules;
5. integrate the rules into the audit;
6. inspect the resulting report; and
7. calculate an overall data-quality score.

# Load DataAudit

```{r load-package}
library(DataAudit)
```

# Example dataset

Consider a small dataset containing several intentional data-quality problems.

```{r example-data}
dat <- data.frame(
  ID = c(
    "A001",
    "A002",
    "A002",
    "BAD",
    "A005"
  ),
  Species = c(
    "Dog",
    "Horse",
    "",
    "Cat",
    "Dog"
  ),
  Age = c(
    5,
    40,
    -2,
    10,
    NA
  ),
  stringsAsFactors = FALSE
)

dat
```

The dataset contains examples of:

- a duplicated identifier;
- a value outside the intended species categories;
- a blank value;
- ages outside an intended range;
- a missing age; and
- an identifier that does not follow the intended pattern.

# General data audit

The primary high-level function is `audit_data()`.

```{r general-audit}
report <- audit_data(
  dat,
  include_nzv = FALSE
)

report
```

The resulting object inherits from the `DataAuditReport` class.

```{r report-class}
class(report)
```

The report contains several components.

```{r report-components}
names(report)
```

These provide information about:

- the overall dataset;
- variable-level issues;
- duplicated rows;
- overall issue counts; and
- custom validation-rule results when rules are supplied.

# Dataset overview

The overview component provides high-level audit statistics.

```{r overview}
report$overview
```

# Variable-level diagnostics

Variable-level diagnostics are stored in the `variables` component.

```{r variable-results}
report$variables
```

This provides information such as missing values, blank values, infinite
values, numbers of unique values, constant variables, near-zero variance
variables, and whether a variable contains a detected issue.

# Duplicate observations

Duplicated observations can be inspected through:

```{r duplicates}
report$duplicates
```

This checks duplicated rows. Identifier uniqueness can be assessed separately
using `unique_check()` or through validation rules.

# Individual checks

DataAudit functions can also be used independently.

## Missing values

```{r missing-check}
missing_check(dat)
```

## Blank values

```{r blank-check}
blank_check(dat)
```

## Range validation

Suppose valid ages are expected to range from 0 to 30.

```{r range-check}
range_check(
  dat,
  variable = "Age",
  min = 0,
  max = 30
)
```

## Category validation

Suppose only `"Dog"` and `"Cat"` are accepted categories.

```{r category-check}
category_check(
  dat,
  variable = "Species",
  allowed = c(
    "Dog",
    "Cat"
  )
)
```

## Identifier uniqueness

```{r unique-check}
unique_check(
  dat,
  variable = "ID"
)
```

## Pattern validation

Suppose identifiers must begin with `A` followed by three digits.

```{r pattern-check}
pattern_check(
  dat,
  variable = "ID",
  pattern = "^A[0-9]{3}$"
)
```

# Reusable validation rules

For repeated or study-specific validation, multiple requirements can be
defined together using `audit_rules()`.

```{r define-rules}
rules <- audit_rules(
  range = list(
    Age = c(0, 30)
  ),
  category = list(
    Species = c(
      "Dog",
      "Cat"
    )
  ),
  unique = "ID",
  required = "Species",
  pattern = list(
    ID = "^A[0-9]{3}$"
  )
)

rules
```

The rules object can be reused with datasets having the same expected
structure.

# Apply validation rules

Rules can be evaluated directly using `apply_audit_rules()`.

```{r apply-rules}
rule_results <- apply_audit_rules(
  dat,
  rules
)

rule_results
```

The returned object contains the rule-specific results and total number of
violations.

```{r rule-summary}
rule_results$summary
rule_results$total_violations
```

A compact summary can also be obtained using:

```{r summarize-rules}
summary(rule_results)
```

# Integrated audit

The validation rules can be incorporated directly into `audit_data()`.

```{r integrated-audit}
report <- audit_data(
  dat,
  include_nzv = FALSE,
  rules = rules
)

report
```

The resulting report now combines automatic data-quality assessment with
user-defined validation rules.

```{r integrated-components}
report$summary
report$rules
```

# Summarizing an audit

The report can be summarized using the standard `summary()` generic.

```{r report-summary}
audit_summary <- summary(report)

audit_summary
```

The summary object provides compact access to dataset-level information,
variables with issues, duplicated rows, and rule results.

```{r summary-components}
audit_summary$dataset
audit_summary$variables_with_issues
audit_summary$duplicate_rows
audit_summary$rules
```

# Data-quality score

An overall data-quality score can be calculated from a `DataAuditReport`.

```{r score}
score <- audit_score(report)

score
```

The score ranges from 0 to 100, with higher values indicating fewer detected
issues relative to dataset size.

The score is accompanied by one of five qualitative categories:

- Excellent;
- Good;
- Moderate;
- Poor; or
- Critical.

The individual components can be inspected directly.

```{r score-components}
score$score
score$quality
score$automatic_issues
score$rule_violations
score$overall_issues
```

The data-quality score is intended as a compact audit summary. It should not
replace domain-specific assessment of whether a dataset is suitable for a
particular scientific or analytical purpose.

# Additional checks

DataAudit provides additional focused functions for different aspects of data
quality, including:

- `constant_check()`;
- `nzv_check()`;
- `infinite_check()`;
- `outlier_check()`;
- `type_check()`;
- `date_check()`;
- `id_check()`;
- `length_check()`;
- `case_check()`;
- `whitespace_check()`;
- `sequence_check()`;
- `group_sequence_check()`;
- `dependency_check()`; and
- `consistency_check()`.

These functions can be combined with the general audit when more detailed
validation is required.

# Recommended workflow

A practical workflow is:

```text
Raw data
   |
   v
audit_data()
   |
   +---- General data-quality assessment
   |
   +---- Individual checks
   |
   v
audit_rules()
   |
   v
apply_audit_rules()
   |
   v
Integrated DataAuditReport
   |
   +---- print()
   |
   +---- summary()
   |
   v
audit_score()
```

This separates general data-quality screening from domain-specific validation
while allowing both to be incorporated into a single structured report.

# Conclusion

`DataAudit` provides a reproducible framework for detecting, organizing, and
summarizing common data-quality problems in R.

The package can be used for rapid preliminary screening with `audit_data()`,
focused validation through individual check functions, reusable validation
with `audit_rules()`, and compact reporting through `summary()` and
`audit_score()`.