--- title: "Analysing and visualising a literature" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Analysing and visualising a literature} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, include = FALSE} knitr::opts_chunk$set(collapse = FALSE, comment = "") # Console colour carries no meaning on a rendered page. pkgdown turns it on for # its own build, and the escape sequences then reach the reader as literal text, # so colour is switched off here for a plain vignette render and a site build # alike. The fixed width keeps tibbles inside the documentation column. options(cli.num_colors = 1, cli.hyperlink = FALSE, crayon.enabled = FALSE, width = 80) # Print data frames and tibbles as formatted tables. local({ kp <- function(x, ...) { if (any(vapply(x, is.list, logical(1)))) return(knitr::normal_print(x)) knitr::knit_print(knitr::kable(x)) } for (cls in c("data.frame", "tbl_df", "tbl")) { registerS3method("knit_print", cls, kp, envir = asNamespace("knitr")) } }) has_ggplot2 <- requireNamespace("ggplot2", quietly = TRUE) # Draw the figures on a transparent background so they sit on whatever colour # the page behind them happens to be. An opaque white matte reads as a white # slab on the site's dark theme, and worse once pkgdown's dark-mode filter # inverts it into a black one. Both halves below are needed: the device option # gives the file an alpha channel, and the theme override clears the white # rectangle that ggplot2's complete themes paint over it regardless. The # override belongs here, in the vignette, because a figure saved for a paper # usually does want a background of its own, so the package's plotting # functions leave it alone. knitr::opts_chunk$set(dev.args = list(bg = "transparent")) if (has_ggplot2) { sf_on_page <- ggplot2::theme( plot.background = ggplot2::element_rect(fill = "transparent", colour = NA), panel.background = ggplot2::element_rect(fill = "transparent", colour = NA) ) knitr::opts_chunk$set(render = function(x, ...) { if (inherits(x, "ggplot")) x <- x + sf_on_page knitr::knit_print(x, ...) }) } ``` ```{r setup} library(scopusflow) ``` Once a set of records is in hand, the package offers a small analysis layer that turns it into the figures a bibliometric study usually needs. The steps that contact the API are shown but not run. The rest run on `example_records`, the corpus of 138 real journal articles on graphene supercapacitors that the package bundles because Scopus records may not be redistributed, and which carries the schema a retrieval returns. `vignette("scopusflow")` describes where it comes from. ## What is in a record set `summary()` is the quickest way to see what a retrieval holds, and it is worth running before any closer analysis. ```{r} records <- example_records summary(records) ``` `scopus_top()` tallies the most frequent sources or authors. ```{r} scopus_top(records, by = "source") scopus_top(records, by = "author", n = 5) ``` The source tally is long-tailed, as a real literature is. These 138 articles are spread across 90 journals, and only *ACS Applied Materials & Interfaces*, with eight, appears more than five times. The author tally is flatter still, with 119 distinct authors, the most prolific of whom contributed three papers. An author string that holds several names is split, so each contributor is counted once per record. The bundled corpus names only the first author of each paper, so it cannot show that happening. Two records with placeholder names can, and the same splitting applies to the semicolon-joined author lists a live harvest returns. ```{r} multi <- scopus_records(list(entry = list( list(`dc:creator` = "Author A.; Author B."), list(`dc:creator` = "Author B.") ))) scopus_top(multi, by = "author") ``` ```{r eval = has_ggplot2, fig.alt = "A horizontal bar chart of the most frequent sources", fig.width = 7, fig.height = 3.5} plot_scopus_top(scopus_top(records, by = "source")) ``` The same plot works on the author tally. ```{r eval = has_ggplot2, fig.alt = "A horizontal bar chart of the most frequent authors", fig.width = 7, fig.height = 3.5} plot_scopus_top(scopus_top(records, by = "author", n = 5)) ``` A record set also has an honest default view. `autoplot()` draws its records per year. Because this corpus is a complete harvest of one query, those bars are the real number of publications per year for that query. The same `autoplot()` generic dispatches on `scopus_trend` and `scopus_top` objects too, delegating to the plots above. ```{r eval = has_ggplot2, fig.alt = "A bar chart of publications per year from 2015 to 2024, fluctuating around fifteen a year", fig.width = 7, fig.height = 3.5} ggplot2::autoplot(records) ``` ## How a literature grows `scopus_trend()` counts how many records match a query in each year, which is the size of a literature over time. It issues one count request per year, so it needs the API. ```{r eval = FALSE} tr <- scopus_trend("graphene supercapacitor", years = 2015:2024, field = "TITLE-ABS-KEY") plot_scopus_trend(tr) ``` The offline equivalent costs no requests at all. A complete harvest already contains its own yearly counts, so tallying the records by year gives what a count per year would have returned, in the shape `scopus_trend()` returns it. ```{r} by_year <- table(records$year) tr <- tibble::tibble( query = "TITLE-ABS-KEY(graphene supercapacitor)", year = as.integer(names(by_year)), n = as.numeric(by_year) ) class(tr) <- c("scopus_trend", class(tr)) tr ``` ```{r eval = has_ggplot2, fig.alt = "A line and area chart of publications per year from 2015 to 2024, peaking in 2019", fig.width = 7.5, fig.height = 4} plot_scopus_trend(tr) ``` The figure draws the same counts as the bar chart above, and the agreement is worth noticing. A trend is something a record set already knows, and something `scopus_trend()` can find out for a query whose records you never download. The caption is the function's own attribution, since a `scopus_trend` object ordinarily comes from the Search API, whereas this one was tallied from the bundled stand-in. The curve itself is flat because the query is a narrow phrase, some fifteen papers a year over a decade. A broader query, 'graphene' on its own, would show the steep growth the field is better known for, on a scale that runs into the offset ceiling discussed below. ## Where a niche sits A study that crosses two or more fields is best introduced by sizing those fields and their overlap. Each parent literature may hold thousands of records while their intersection holds a handful, which is the niche the study occupies. `scopus_intersections()` counts a named set of concepts and any requested intersections of them, at one count request per row, so the whole landscape is as cheap as a few `scopus_count()` calls. Concept values may be bare terms, wrapped in `field` for you, or complete field-tagged expressions, used exactly as given. ```{r eval = FALSE} sets <- scopus_intersections( concepts = c( "semantic priming" = "semantic priming", "mental simulation" = "mental simulation", # A synonym set, given as a complete expression and used exactly as given. "embodied simulation" = 'TITLE-ABS-KEY("mental simulation") OR TITLE-ABS-KEY("embodied simulation")' ), intersections = list(c("semantic priming", "mental simulation")), field = "TITLE-ABS-KEY" ) plot_scopus_intersections( sets, highlight = sets$label[sets$type == "intersection"] ) ``` Here `field` leaves the third value untouched, since it already reads as a complete field-tagged expression, so a concept can be a whole synonym set. This one cannot be derived from a record set, because it counts whole literatures, where a record set holds only the records already in hand, so here the result is rebuilt in its own shape with illustrative counts, purely to show the plot. The lollipop chart uses a log-scale axis, so the small intersection stays legible beside its large parent fields, and the highlighted row draws the eye to the niche itself. ```{r eval = has_ggplot2, fig.alt = "A log-scale lollipop chart showing three concepts and a small intersection, with the intersection highlighted", fig.width = 7.5, fig.height = 3} sets <- tibble::tibble( label = c("semantic priming", "mental simulation", "embodied simulation", "semantic priming × mental simulation"), query = c("TITLE-ABS-KEY(semantic priming)", "TITLE-ABS-KEY(mental simulation)", 'TITLE-ABS-KEY("mental simulation") OR TITLE-ABS-KEY("embodied simulation")', "(TITLE-ABS-KEY(semantic priming)) AND (TITLE-ABS-KEY(mental simulation))"), n = c(6600, 2100, 3400, 15), type = c("concept", "concept", "concept", "intersection"), size = c(1L, 1L, 1L, 2L), members = c("semantic priming", "mental simulation", "embodied simulation", "semantic priming; mental simulation") ) class(sets) <- c("scopus_intersections", class(sets)) plot_scopus_intersections( sets, highlight = sets$label[sets$type == "intersection"] ) ``` ## Reading the fuller record The Search API returns a few fields per record. To read the abstract and the fuller metadata for a record you already know, `scopus_abstract()` calls the Abstract Retrieval API, by DOI or 'Scopus' identifier. A batch is resilient, so an identifier that cannot be found yields a row of `NA`s with a warning rather than stopping the run. ```{r eval = FALSE} ab <- scopus_abstract(head(scopus_extract_dois(records), 2)) ``` The result is a tibble of class `scopus_abstracts`, one row per identifier. To show its shape without a key, here is a stand-in built from the two most-cited records of the corpus, which supplies every column but the abstract itself. The abstract is the one thing a live call adds and the corpus does not carry, so it is marked as a placeholder, with nothing invented to fill it, and the columns are listed by name because the prose is far too wide to typeset. ```{r} top2 <- records[order(-records$citations), ][1:2, ] ab <- tibble::tibble( id = top2$doi, scopus_id = NA_character_, doi = top2$doi, title = top2$title, abstract = "", publication = top2$publication, year = top2$year, citations = top2$citations ) class(ab) <- c("scopus_abstracts", class(ab)) names(ab) ab[, c("title", "publication", "year", "citations")] ``` ## Beyond five thousand records A single Search API query returns at most its first 5000 records under the ordinary offset paging. When you need the whole of a larger result set in one pass, `scopus_fetch(cursor = TRUE)` follows the API's cursor instead, which has no such ceiling. ```{r eval = FALSE} recs <- scopus_fetch("TITLE-ABS-KEY(microplastics)", cursor = TRUE) nrow(recs) ``` That query matched 38,374 records when this article was written (the literature keeps growing), several times the offset ceiling, and the cursor retrieves all of them in one call. The records then arrive in the API's deep-paging order, which is the right trade for a complete harvest and the wrong one when you want the most relevant few hundred. This is the one-call alternative to the year-partitioned plan in the *Search plans and quota-aware retrieval* article. A plan keeps each cell under the ceiling and preserves relevance order, whereas `cursor = TRUE` harvests the whole set in a single pass. Reach for the plan when you want cached, resumable cells and the cursor when you want the complete set at once.