--- title: "Searching catalogs" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Searching catalogs} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include = FALSE} knitr::opts_chunk$set(collapse = TRUE, comment = "#>") library(httptest2) start_vignette("search") set_redactor(function(x) { gsub_response(x, "index.php/api/catalog/", "") }) ``` `search_catalog()` is the main way to find studies. This guide goes through its arguments one at a time. The examples use the World Bank catalog unless a section says otherwise, and the output comes from saved responses. ```{r library} library(nadaverse) ``` Every search returns one row per study. To keep the output short, this guide prints only a few columns of the first five rows with a small helper: ```{r brief} brief <- function(x) head(x[, c("idno", "nation", "year_start")], 5) ``` ## Keywords `keyword` searches titles, descriptions and keywords. `ps` is the number of studies returned per page. ```{r keyword} res <- search_catalog("wb", keyword = "poverty", ps = 5) brief(res) ``` ## Years `from` and `to` take four-digit years and select studies by the period their data cover. ```{r years} res <- search_catalog("wb", from = 2015, to = 2020, ps = 5) brief(res) ``` ## Countries `country` takes country names or ISO3 codes, and you can give several at once. `country_codes()` lists the valid values for a catalog. ```{r country} res <- search_catalog("wb", country = c("ETH", "UGA"), ps = 5) brief(res) ``` ## Collections and access types Many catalogs group their studies into collections. `collections()` lists them, and the `repositoryid` column is the value that `collection` expects. ```{r collections} cl <- collections("wb") head(cl[, c("repositoryid", "title")], 5) ``` ```{r collection} res <- search_catalog("wb", collection = "LSMS", ps = 5) brief(res) ``` Studies also differ in how you can get the data. `access_codes()` lists the access types, and `dtype` filters by them. ```{r access} access <- access_codes("wb") head(access[, c("type", "title")], 5) ``` ```{r dtype} res <- search_catalog("wb", dtype = access$type[1], ps = 5) brief(res) ``` ## Sorting and pages `sort_by` accepts `"rank"`, `"title"`, `"nation"` and `"year"`. The word `"country"` also works and is treated as `"nation"`. `sort_order` is `"asc"` or `"desc"`, and `page` moves through the results. ```{r sort} res <- search_catalog( "wb", country = "ETH", sort_by = "year", sort_order = "desc", ps = 5 ) brief(res) ``` ```{r page} res2 <- search_catalog( "wb", country = "ETH", sort_by = "year", sort_order = "desc", ps = 5, page = 2 ) brief(res2) ``` The `created` argument filters by the date a study was added to the catalog. Give one date, or a range, in `YYYY/MM/DD` form: ```{r created, eval = FALSE} search_catalog("wb", created = "2023/01/01-2023/12/31") ``` ## Search information By default you get only the table of studies. With `rows = FALSE` you get the full result list, which also says how many studies matched. ```{r info} info <- search_catalog("wb", keyword = "poverty", ps = 5, rows = FALSE) info$found info$total names(info) ``` `found` is the number of studies that match your search and `total` is the number of studies in the whole catalog. The table of studies is still there as `info$rows`. `inc_iso = TRUE` adds country codes to the results: ```{r iso} res <- search_catalog("wb", country = "ETH", inc_iso = TRUE, ps = 5) names(res) ``` ## When nothing matches A search with no matches gives a warning and returns `NULL`, not an empty table. Check for it before you use the result. ```{r none} none <- suppressWarnings(search_catalog("fao", keyword = "novalidsk")) is.null(none) ``` ## Comparing catalogs The same arguments work on every catalog, so a short loop can compare them. Here `ps = 1` keeps each download small, because only the count is needed. ```{r compare} codes <- c("wb", "fao", "ihsn") found <- vapply( codes, function(code) { info <- suppressWarnings( search_catalog(code, keyword = "household", ps = 1, rows = FALSE) ) if (is.null(info)) 0 else as.numeric(info$found) }, numeric(1) ) found ``` Once you have a study, `vignette("studies", package = "nadaverse")` shows how to read its description, files and variables. ```{r teardown, include = FALSE} end_vignette() ```