Calling external functions

Since f_n points to a function name, any function can be passed to Normalise() as long as it is available in the namespace. To illustrate, consider an example where we want to categorise into discrete bins. We can use base R’s cut() function for this purpose. We simply need to specify the number of bins. We could directly call cut(), but for clarity we will create a simple wrapper function around it, then pass that function to Normalise().

# wrapper function
f_bin <- function(x, nbins){
  cut(x, breaks = nbins, labels = FALSE)
}

# pass wrapper to normalise, specify 5 bins
coin <- Normalise(coin, dset = "Raw",
                   global_specs = list(f_n = "f_bin",
                                        f_n_para = list(nbins = 5)))
#> Written data set to .$Data$Normalised
#> (overwritten existing data set)

To illustrate the difference with the linear transformations above, we again plot the raw against normalised indicator:

plot_scatter(coin, dsets = c("Raw", "Normalised"), iCodes = "Goods")

Obviously this is not linear.

Generally, the requirements of a function to be passed to Normalise() are that its first argument should be x, a numeric vector, and it should return a numeric vector of the same length as x. It should also be able to handle NAs. Any further arguments can be passed via the f_n_para entry.

Directions

By default, the directions are taken from the coin. These will have been specified as the Direction column of iMeta when constructing a coin with new_coin(). However, you can specify different directions using the directions argument of normalise(): in this case you need to specify a data frame with two columns: iCode (with an entry for each indicator code found in the target data set) and Direction giving the direction as -1 or 1.

To show an example, we take the existing directions from the coin, modify them slightly, and then run the normalisation function again:

# get directions from coin
directions <- coin$Meta$Ind[c("iCode", "Direction")]

head(directions, 10)
#>       iCode Direction
#> 9     Goods         1
#> 10 Services         1
#> 11      FDI         1
#> 12   PRemit         1
#> 13  ForPort         1
#> 31    Renew         1
#> 32 PrimEner        -1
#> 33      CO2        -1
#> 34   MatCon        -1
#> 35   Forest        -1

We’ll change the direction of the “Goods” indicator and re-normalise:

# change Goods to -1
directions$Direction[directions$iCode == "Goods"] <- -1

# re-run (using min max default)
coin <- Normalise(coin, dset = "Raw", directions = directions)
#> Written data set to .$Data$Normalised
#> (overwritten existing data set)

Goalposts, targets and iMeta specification

For some normalisation methods we may use the same basic normalisation function for all indicators but use different parameters for each indicator - for example, using distance to target normalisation or goalpost normalisation. COINr now supports specifying these parameters in the iMeta table. This is intended to give an easier interface to these normalisation methods, as opposed to using the full individual specifications list (see the next section on that).

The basic idea is that you specify the parameters as columns in the iMeta data frame. COINr currently supports this for a selected subset of inbuilt normalisation functions:

• n_minmax()
• n_scaled()
• n_zscore()
• n_dist2targ()
• n_goalposts()

To use this approach, you have to (a) add the relevant columns to iMeta before building the coin, and (b) set f_n_para = "use_iMeta" within the global_specs list when calling Normalise().

Let’s clarify this with an example. We will apply distance to target normalisation using individual targets contained within iMeta. In fact, the example data with COINr already has a “Target” column which can be used for this purpose:

# show first few rows of iCode and Target cols in built-in iMeta
head(ASEM_iMeta[c("iCode", "Target")])
#>     iCode     Target
#> 1     LPI   4.118031
#> 2 Flights 200.332655
#> 3    Ship  20.113377
#> 4    Bord 115.900000
#> 5    Elec 104.670585
#> 6     Gas  90.060420

The documentation for the n_dist2targ() function explains that the required iMeta columns for this function are Target, and dist2targ_cap_max. This latter parameter (which specifies whether to cap values that surpass the target) is included for completeness, even though it is unlikely you would want to vary it from one indicator to another. Since the Target column is already present, we will just add the missing column:

# copy built in data
iMeta <- ASEM_iMeta
# set cap_max
iMeta$dist2targ_cap_max <- TRUE

Now we build the coin and normalise.

# build coin
coin <- new_coin(ASEM_iData, iMeta, quietly = TRUE)

# normalise, referencing iMeta columns
coin <- Normalise(coin, dset = "Raw", global_specs = list(f_n = "n_dist2targ", f_n_para = "use_iMeta"))
#> Written data set to .$Data$Normalised

Note that the global_specs$f_n_para is set to "use_iMeta", telling the function to use the iMeta columns. Let us visualise the result for a selected indicator:

plot_scatter(coin, dsets = c("Raw", "Normalised"), iCodes = "LPI")

This simply shows that (a) the indicator is now scaled between 0 and 1, and any raw values of LPI above the target of 4.1 are capped at a value of 1.

We now take another example using the “goalposts” approach, also known as “distance to frontier”. In this method, each indicator is scaled as a distance between an upper and lower bound. The following equation is for a positive direction indicator.

\[ x_n = \frac{x - G_{low}}{G_{high} - G_{low}} \]

where \(x_n\) is the normalised indicator value and \(G_{low}\) and \(G_{high}\) are respectively the lower and upper goalpost values. The negative direction formula is analogous.

To use this approach we again need to define the necessary columns in iMeta. Referencing again the documentation of the relevant function n_goalposts(), we need to add columns goalpost_lower, goalpost_upper, goalpost_scale and goalpost_trunc2posts, which correspond to arguments to the n_goalposts() function.

For the sake of this example, we just define goalposts as 5% of the range from the maximum and minimum of each indicator. We set the scaling to 1 and truncation to TRUE.

# get iCodes and raw data
iCodes <- iMeta$iCode[iMeta$Type == "Indicator"]

# set general parameters
iMeta$goalpost_scale <- 1
iMeta$goalpost_trunc2posts <- TRUE

# set goalposts for each indicator
for(iCode in iCodes){
  maxx <- max(ASEM_iData[[iCode]], na.rm = TRUE)
  minx <- min(ASEM_iData[[iCode]], na.rm = TRUE)
  rx <- maxx - minx
  # fake goalposts in 5% of range
  iMeta$goalpost_lower[iMeta$iCode == iCode] <- minx + 0.05*rx
  iMeta$goalpost_upper[iMeta$iCode == iCode] <- maxx - 0.05*rx
}

# build coin
coin <- new_coin(ASEM_iData, iMeta, quietly = TRUE)

# normalise using minmax
coin <- Normalise(coin, dset = "Raw", 
                  global_specs = list(f_n = "n_goalposts", f_n_para = "use_iMeta"))
#> Written data set to .$Data$Normalised

Again, let’s plot an indicator before and after normalisation:

plot_scatter(coin, dsets = c("Raw", "Normalised"), iCodes = "Cov4G")

Here we see that goalposts act like distance to target, but also cap at both ends.

For both the distance to target method and the goalposts method, directionality is automatically accounted for. That is, if an indicator has Direction -1 in iMeta, the distance to target function will assume that values higher than the target are worse and will give lower scores. , and values below the target will get a full score. Similarly, the goalposts function will measure the progress from the higher goalpost to the lower one for negative directionality indicators. This means that in defining the goalposts in iMeta, always assign the goalpost with the higher value to “goalpost_upper” - COINr will account for the directionality.

Importantly, all of the functions mentioned in this section can also be used without the special columns in iMeta, either by specifying a single set of parameters to apply to all indicators using global_specs, OR by providing detailed individual specifications via the indiv_specs argument (see the next section).

Individual normalisation

Finally let’s explore how to specify different normalisation methods for different indicators. The indiv_specs argument takes a named list for each indicator, and will override the specifications in global_specs. If indiv_specs is specified, we only need to include sub-lists for indicators that differ from global_specs.

To illustrate, we can use a contrived example where we might want to apply min-max to all indicators except two. For those, we apply a rank transformation and distance to maximum approach. Note, that since the default of global_specs is min-max, we don’t need to specify that at all here.

# individual specifications:
# LPI - borda scores
# Flights - z-scores with mean 10 and sd 2
indiv_specs <- list(
  LPI = list(f_n = "n_borda"),
  Flights = list(f_n = "n_zscore",
                 f_n_para = list(m_sd = c(10, 2)))
)

# normalise
coin <- Normalise(coin, dset = "Raw", indiv_specs = indiv_specs)
#> Written data set to .$Data$Normalised
#> (overwritten existing data set)

# a quick look at the first three indicators
get_dset(coin, "Normalised")[1:4] |>
  head(10)
#>    uCode LPI   Flights      Ship
#> 1    AUS  36  9.889993  66.14497
#> 2    AUT  44  9.588735   0.00000
#> 3    BEL  45  9.711512  97.14314
#> 4    BGD   4  8.529810  45.80661
#> 5    BGR   7  8.741971  37.40495
#> 6    BRN   9  8.433044  35.38920
#> 7    CHE  42 10.563483   0.00000
#> 8    CHN  30 13.235114 100.00000
#> 9    CYP  14  8.721372  55.21211
#> 10   CZE  31  9.001961   0.00000

This example is meant to be illustrative of the functionality of Normalise(), rather than being a sensible normalisation strategy, because the indicators are now on very different ranges.

In practice, if different normalisation strategies are selected, it is a good idea to keep the indicators on similar ranges, otherwise the effects will be very unequal in the aggregation step.

Linear methods

# example data
set.seed(69)
x <- runif(20)

library(ggplot2)
#> Warning: package 'ggplot2' was built under R version 4.3.3
# generic plot function to use in the other examples
nplot <- function(x, y, plot_title = NULL){
  ggplot(data = NULL, aes(x = x, y = y)) +
    geom_point() +
    xlab("x") +
    ylab("x (normalised)") +
    ggtitle(plot_title)
}

# rescale onto [50, 100] interval
xdash <- n_minmax(x, l_u = c(50, 100))
nplot(x, xdash, "Min-max onto [50, 100]")

nplot(x, n_zscore(x, c(10, 2)))

nplot(x, n_scaled(x, c(1,10)), "Linear scaling using c(1,10)")

nplot(x, n_dist2max(x), "Distance to maximum")

nplot(x, n_fracmax(x), "Fraction of max value")

Rank methods

nplot(x, n_rank(x), "Rank")

# normalise
xdash <- n_borda(x)
nplot(x, xdash, "Borda")

nplot(x, n_prank(x), "Percentile ranks")

Targets and references

xdash <- n_dist2ref(x, iref = 3, cap_max = FALSE)
nplot(x, xdash, "Distance to reference (no cap)")

xdash <- n_dist2ref(x, iref = 3, cap_max = TRUE)
nplot(x, xdash, "Distance to reference (no cap)")

xdash <- n_dist2targ(x, targ = 0.75, direction = 1, cap_max = TRUE)
nplot(x, xdash, "Distance to 0.75 target (positive direction: with cap)")

xdash <- n_dist2targ(x, targ = 0.2, direction = -1, cap_max = TRUE)
nplot(x, xdash, "Distance to 0.2 target (negative direction: with cap)")

xdash <- n_goalposts(x, gposts = c(0.2, 0.75, 1), direction = 1)
nplot(x, xdash, "Goalposts with positive indicator")

xdash <- n_goalposts(x, gposts = c(0.2, 0.75, 1), direction = -1)
nplot(x, xdash, "Goalposts with negative indicator")