---
title: "Variable importance plots: an introduction to vip"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Variable importance plots: an introduction to vip}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
bibliography: '`r system.file("references.bib", package = "vip")`'
link-citations: yes
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.width = 6,
  fig.height = 4,
  fig.align = "center"
)
# Optional sections only evaluate when the corresponding package is installed
has_pdp <- requireNamespace("pdp", quietly = TRUE)
has_fastshap <- requireNamespace("fastshap", quietly = TRUE)
has_yardstick <- requireNamespace("yardstick", quietly = TRUE)
```

Machine learning models are often summarized by a single accuracy metric and
then put into production. Understanding *which* features actually drive a
model's predictions—variable importance (VI)—is a fundamental part of
interpretable machine learning. The **vip** package provides a single,
consistent interface for computing VI scores (`vi()`) and plotting them
(`vip()`) across dozens of model types, using both *model-specific* and
*model-agnostic* approaches. This vignette is a condensed, up-to-date tour;
for the full methodology, please see (and cite) our article in *The R Journal*
[@RJ-2020-013].

**vip** supports four methods, selected via the `method` argument to `vi()`:

* `method = "model"` (the default): model-specific VI scores, extracted from
  the fitted model itself (e.g., split-based importance in trees); see
  `?vi_model` for the full list of supported classes.
* `method = "permute"`: permutation importance [@random-breiman-2001;
  @fisher-model-2018], the drop in performance after shuffling each feature.
* `method = "firm"`: variance-based importance computed from feature effect
  plots [@greenwell-simple-2018].
* `method = "shap"`: mean absolute Shapley values [@strumbelj-2014-explaining],
  computed via the [fastshap](https://cran.r-project.org/package=fastshap)
  package.

## Example data

Throughout we simulate data from the Friedman 1 benchmark problem
[@multivariate-friedman-1991]: ten uniform features, of which only `x1`--`x5`
truly affect the response.

```{r friedman}
library(vip)

trn <- gen_friedman(500, seed = 101)  # see ?vip::gen_friedman
head(trn)
```

## Model-specific importance

When a model class provides its own importance measure, `vi()` extracts it
directly. For example, decision trees measure importance through the goodness
of split [@classification-breiman-1984]:

```{r model-specific}
library(rpart)

tree <- rpart(y ~ ., data = trn)
vi(tree)  # a data frame of class "vi"
```

`"vi"` objects have a `plot()` method (drawing with lightweight base R
graphics via the [tinyplot](https://grantmcdermott.com/tinyplot/) package)
that invisibly returns the plotted `"vi"` object; `vip()` is a convenience
wrapper that computes the scores and plots them in one call:

```{r model-specific-plot}
vip(tree)  # equivalent to plot(vi(tree))
```

## Permutation importance

Permutation importance works for *any* model. You supply the training data,
the target, a performance `metric`, and a prediction wrapper (`pred_wrapper`)
that tells **vip** how to generate predictions from your model:

```{r permute, eval = has_yardstick}
pfun <- function(object, newdata) predict(object, newdata = newdata)

set.seed(102)  # for reproducibility
vis <- vi(tree,
  method = "permute", train = trn, target = "y", metric = "rmse",
  pred_wrapper = pfun, nsim = 10  # average over 10 permutations
)
vis
```

With `nsim > 1` the raw per-permutation scores are retained, so the variation
in the scores can be displayed with boxplots or violins; additional arguments
to `plot()` are passed on to `tinyplot::tinyplot()`:

```{r permute-plot, eval = has_yardstick}
plot(vis, type = "boxplot", all_permutations = TRUE, jitter = TRUE,
     fill = "grey90")
```

## Variance-based importance (FIRM)

The FIRM approach measures the relative "flatness" of each feature's effect,
estimated via partial dependence [@friedman-2001-greedy] using the
[pdp](https://cran.r-project.org/package=pdp) package:

```{r firm, eval = has_pdp}
vi(tree, method = "firm", train = trn)
```

## Shapley-based importance

SHAP-based importance aggregates the mean absolute Shapley value of each
feature, computed with
[fastshap](https://cran.r-project.org/package=fastshap):

```{r shap, eval = has_fastshap}
set.seed(103)  # for reproducibility
vi(tree, method = "shap", train = subset(trn, select = -y),
   pred_wrapper = pfun, nsim = 10)
```

## Plotting options

`plot()` supports bar charts (`type = "bar"`, the default), Cleveland dot
plots (`type = "point"`), boxplots, and violins (the latter two require raw
permutation scores; see above). Graphical parameters (e.g., `col`, `pch`,
`cex`) are passed straight through to `tinyplot::tinyplot()`, and
`include_type = TRUE` adds the importance type to the axis label:

```{r plotting-options}
plot(vi(tree), type = "point", horizontal = FALSE, include_type = TRUE,
     col = "forestgreen", pch = 17, cex = 1.5)
```

## References