A bird’s eye view

class: title-slide, center, bottom

# A bird's eye view

## Tidymodels, Virtually &mdash; Session 01

### Alison Hill

---
class: middle, center, inverse

## .big-text[Hello.]

---
name: hello
class: middle, center, inverse

### Alison Hill

[<svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"/></svg> @apreshill](https://github.com/apreshill)    
[<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"/></svg> @apreshill](https://twitter.com/apreshill)   
[<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M326.612 185.391c59.747 59.809 58.927 155.698.36 214.59-.11.12-.24.25-.36.37l-67.2 67.2c-59.27 59.27-155.699 59.262-214.96 0-59.27-59.26-59.27-155.7 0-214.96l37.106-37.106c9.84-9.84 26.786-3.3 27.294 10.606.648 17.722 3.826 35.527 9.69 52.721 1.986 5.822.567 12.262-3.783 16.612l-13.087 13.087c-28.026 28.026-28.905 73.66-1.155 101.96 28.024 28.579 74.086 28.749 102.325.51l67.2-67.19c28.191-28.191 28.073-73.757 0-101.83-3.701-3.694-7.429-6.564-10.341-8.569a16.037 16.037 0 0 1-6.947-12.606c-.396-10.567 3.348-21.456 11.698-29.806l21.054-21.055c5.521-5.521 14.182-6.199 20.584-1.731a152.482 152.482 0 0 1 20.522 17.197zM467.547 44.449c-59.261-59.262-155.69-59.27-214.96 0l-67.2 67.2c-.12.12-.25.25-.36.37-58.566 58.892-59.387 154.781.36 214.59a152.454 152.454 0 0 0 20.521 17.196c6.402 4.468 15.064 3.789 20.584-1.731l21.054-21.055c8.35-8.35 12.094-19.239 11.698-29.806a16.037 16.037 0 0 0-6.947-12.606c-2.912-2.005-6.64-4.875-10.341-8.569-28.073-28.073-28.191-73.639 0-101.83l67.2-67.19c28.239-28.239 74.3-28.069 102.325.51 27.75 28.3 26.872 73.934-1.155 101.96l-13.087 13.087c-4.35 4.35-5.769 10.79-3.783 16.612 5.864 17.194 9.042 34.999 9.69 52.721.509 13.906 17.454 20.446 27.294 10.606l37.106-37.106c59.271-59.259 59.271-155.699.001-214.959z"/></svg> apreshill.com](https://www.apreshill.com)

---
class: center, middle, inverse

# What is Machine Learning?

???

Machine Learning is usually thought of as a subfield of artificial intelligence that itself contains other hot sub-fields.

Let's start somewhere familiar. I have a data set and I want to analyze it.

The actual data set is named `ames` and it comes in the `AmesHousing` R package. No need to open your computers. Let's just discuss for a few minutes.

---
class: middle, center

# What is machine learning?

---
class: top, center
background-image: url(images/intro.002.jpeg)
background-size: cover

---
class: top, center
background-image: url(images/intro.003.jpeg)
background-size: cover
---
class: top, center
background-image: url(images/all-of-ml.jpg)
background-size: contain

.footnote[Credit: <https://vas3k.com/blog/machine_learning/>]

---
class: middle, center, frame

# Two modes

.pull-left[

## Classification

]

.pull-left[

## Regression

]

---
class: middle, center, frame

# Two cultures

.pull-left[

### Statistics
]

.pull-right[

### Machine Learning
]

---
class: middle, center, frame

# Which is which?

.pull-left[

model first

inference emphasis
]

.pull-right[

data first

prediction emphasis
]

---
class: middle, center, frame

# Which is which?

.pull-left[

model first

inference emphasis

### Statistics
]

.pull-right[

data first

prediction emphasis

### Machine Learning
]

---
name: train-love
background-image: url(images/train.jpg)
background-size: contain
background-color: #f6f6f6

---
template: train-love
class: center, top

# Statistics

---
template: train-love
class: bottom

> *"Statisticians, like artists, have the bad habit of falling in love with their models."*
>
> &mdash; George Box

---
class: freight-slide, center, inverse

# Predictive modeling

---
class: middle, inverse, center

# Schedule for Today

01 - A bird's eye view

02 - Build a useful model

03 - Build better training data

04 - The forest for the trees

05 - Build a fine-tuned model

06 - Case study

---
class: inverse, middle, center

# tidymodels

---
background-image: url(images/tm-org.png)
background-size: contain

---
class: middle

```r
library(tidymodels)
```

---
class: inverse, middle, center

# To the workshop! 🚀

Slides: https://apreshill.github.io/tidymodels-it/

RStudio Cloud: https://bit.ly/tidymodels-it

---

```r
# pick model
rt_mod <-
  decision_tree(engine = "rpart") %>%
  set_mode("regression")

rt_mod %>% 
  # fit
  fit(body_mass_g ~ ., data = penguins) %>% 
  # predict
  predict(new_data = penguins) %>%  
  mutate(body_mass_g = penguins$body_mass_g) %>% 
  # compare
  rmse(truth = body_mass_g, estimate = .pred)
# # A tibble: 1 × 3
#   .metric .estimator .estimate
#   <chr>   <chr>          <dbl>
# 1 rmse    standard        311.
```

---

```r
# holdout method
peng_split <- initial_split(penguins, strata = species)
peng_train <- training(peng_split)
peng_test <- testing(peng_split)

# add cross-validation
peng_folds <- vfold_cv(data = peng_train, strata = "species")

# pick model
rt_mod <-
  decision_tree(engine = "rpart") %>%
  set_mode("regression")

# here comes the actual ML bit…
fit_resamples(
    rt_mod,
    preprocessor = body_mass_g ~ .,
    resamples = peng_folds
) %>% 
  collect_metrics()
# # A tibble: 2 × 6
#   .metric .estimator    mean     n std_err .config          
#   <chr>   <chr>        <dbl> <int>   <dbl> <chr>            
# 1 rmse    standard   317.       10 14.0    Preprocessor1_Mo…
# 2 rsq     standard     0.840    10  0.0200 Preprocessor1_Mo…
```

---

```r
# holdout method
peng_split <- initial_split(penguins, strata = species)
peng_train <- training(peng_split) 
peng_test <- testing(peng_split)

# add cross-validation
peng_folds <- vfold_cv(data = peng_train, strata = "species")

*# pick model
*rt_mod <-
* parsnip::decision_tree(engine = "rpart") %>%
* parsnip::set_mode("regression")

# here comes the actual ML bit…
fit_resamples(
    rt_mod,
    preprocessor = body_mass_g ~ .,
    resamples = peng_folds
) %>% 
  collect_metrics()
```

---
class: inverse

# Pick a model with parsnip

---
class: middle, center

# Quiz

How many ways can you think of in R to do some type of linear regression?

`lm` for the general linear model

`glmnet` for regularized regression

`stan` for Bayesian regression

`keras` for regression using tensorflow

`spark` for large data sets

...

---
class: middle, frame

# .center[To specify a model with parsnip]

.right-column[

1\. Pick a .display[model] + .display[engine]

2\. Set the .display[mode] (if needed)

]

---
class: middle, frame

# .center[To specify a model with parsnip]

```r
decision_tree(engine = "C5.0") %>%
  set_mode("classification")
```

---
class: middle, frame

# .center[To specify a model with parsnip]

```r
nearest_neighbor(engine = "kknn") %>%              
  set_mode("regression") %>%        
```

---
class: middle, frame

.fade[
# .center[To specify a model with parsnip]
]

.right-column[

1\. Pick a .display[model] + .display[engine]
.fade[

2\. Set the .display[mode] (if needed)
]

]

---
class: middle, center

# 1\. Pick a .display[model] + .display[engine]

All available models are listed at

<https://www.tidymodels.org/find/parsnip/>

---
class: middle

.center[
# `linear_reg()`

Specifies a model that uses linear regression
]

```r
linear_reg(mode = "regression", engine = "lm", penalty = NULL, mixture = NULL)
```

---
class: middle

.center[
# `linear_reg()`

Specifies a model that uses linear regression
]

```r
linear_reg(
  mode = "regression", # "default" mode, if exists
  engine = "lm",       # default computational engine
  penalty = NULL,      # model hyper-parameter
  mixture = NULL       # model hyper-parameter
  )
```

---
class: middle, frame

.fade[
# .center[To specify a model with parsnip]
]

.right-column[
.fade[
1\. Pick a .display[model] + .display[engine]
]

2\. Set the .display[mode] (if needed)

]

---
class: middle, center

# `set_mode()`

Sets the class of problem the model will solve, which influences which output is collected. Not necessary if mode is set in Step 1.

```r
lm_mod %>% set_mode(mode = "regression")
```

---
class: middle

.pull-left[

```r
*# pick model
*lm_mod <-
* parsnip::linear_reg(engine = "lm") %>%
* parsnip::set_mode("regression")

lm_mod %>% 
  fit(body_mass_g ~ flipper_length_mm, 
      data = penguins) 
# parsnip model object
# 
# Fit time:  3ms 
# 
# Call:
# stats::lm(formula = body_mass_g ~ flipper_length_mm, data = data)
# 
# Coefficients:
#       (Intercept)  flipper_length_mm  
#          -5872.09              50.15
```

]

.pull-right[
<img src="figs/01/penguins-lm-1.png" width="504" style="display: block; margin: auto;" />

]

---
class: middle, center, inverse

# A model doesn't have to be a straight line!

---
class: middle

.pull-left[

```r
*# pick model
*rt_mod <-
* parsnip::decision_tree(engine = "rpart") %>%
* parsnip::set_mode("regression")

rt_mod %>% 
  fit(body_mass_g ~ flipper_length_mm, 
      data = penguins)
# parsnip model object
# 
# Fit time:  4ms 
# n= 333 
# 
# node), split, n, deviance, yval
#       * denotes terminal node
# 
#  1) root 333 215259700 4207.057  
#    2) flipper_length_mm< 206.5 208  38996800 3702.524  
#      4) flipper_length_mm< 193.5 129  19162970 3555.039 *
#      5) flipper_length_mm>=193.5 79  12445890 3943.354 *
#    3) flipper_length_mm>=206.5 125  35211680 5046.600  
#      6) flipper_length_mm< 214.5 49   8152398 4614.796 *
#      7) flipper_length_mm>=214.5 76  12032500 5325.000  
#       14) flipper_length_mm< 220.5 41   5359848 5134.756 *
#       15) flipper_length_mm>=220.5 35   3450464 5547.857 *
```

]

.pull-right[
<img src="figs/01/penguins-rt-1.png" width="504" style="display: block; margin: auto;" />

]

---

```r
# holdout method
*peng_split <- rsample::initial_split(penguins, strata = species)
*peng_train <- rsample::training(peng_split)

# add cross-validation
*peng_folds <- rsample::vfold_cv(data = peng_train, strata = "species")

# pick model
rt_mod <-
  decision_tree(engine = "rpart") %>%
  set_mode("regression")

# here comes the actual ML bit…
fit_resamples(
    rt_mod,
    preprocessor = body_mass_g ~ .,
    resamples = peng_folds
) %>% 
  collect_metrics()
```

---
class: inverse, middle, center

# Resample a model with rsample

---
class: inverse, middle, center

# .fade[Resample a model with rsample]

# Step 1: The holdout method

---
class: middle, center

---
class: middle

.pull-left[

Train with `training()`

Test with `testing()`

]

.pull-right[

<img src="figs/01/lm-test-resid-1.png" width="504" style="display: block; margin: auto;" />
]

---
class: middle

.pull-left[

Train with `training()`

Test with `testing()`

]

.pull-right[

<img src="figs/01/rt-test-resid-1.png" width="504" style="display: block; margin: auto;" />
]

---

# Do it once

.pull-left[

```r
# pick model
rt_mod <-
  decision_tree(engine = "rpart") %>%
  set_mode("regression")

]

.pull-right[

```r
peng_test  <- testing(peng_split)
rt_mod %>% 
  # TRAIN: get fitted model
  fit(body_mass_g ~ ., data = peng_train) %>% 
  # TEST: get predictions
  predict(new_data = peng_test) %>% 
  # COMPARE: get metrics
  bind_cols(peng_test) %>% 
  rmse(truth = body_mass_g, estimate = .pred)
# # A tibble: 1 × 3
#   .metric .estimator .estimate
#   <chr>   <chr>          <dbl>
# 1 rmse    standard        309.
```
]

---

# Train + test the model once

.pull-left[

]

---

# Train + test the model 10 times

.pull-left[

]

.pull-right[

```r
# holdout method
peng_split <- initial_split(penguins, strata = species)
peng_train <- training(peng_split)
peng_test <- testing(peng_split)

# add cross-validation
peng_folds <- vfold_cv(data = peng_train, strata = "species")

# pick model
rt_mod <-
  decision_tree(engine = "rpart") %>%
  set_mode("regression")

# here comes the actual ML bit…
fit_resamples(
    rt_mod,
    preprocessor = body_mass_g ~ .,
    resamples = peng_folds
) %>% 
  collect_metrics()
# # A tibble: 2 × 6
#   .metric .estimator    mean     n std_err .config          
#   <chr>   <chr>        <dbl> <int>   <dbl> <chr>            
# 1 rmse    standard   310.       10  9.45   Preprocessor1_Mo…
# 2 rsq     standard     0.861    10  0.0146 Preprocessor1_Mo…
```
]

---
class: inverse, middle, center

# .fade[Resample a model with rsample]

# .fade[Step 1: The holdout method]

# Step 2: Cross-validation

---

---
class: middle, center

# Data Splitting

---
class: frame, center, middle

# Resampling

Let's resample 10 times

then compute the mean of the results...

---

```r
rmse %>% tibble::enframe(name = "rmse")
# # A tibble: 10 × 2
#     rmse value
#    <int> <dbl>
#  1     1  396.
#  2     2  424.
#  3     3  394.
#  4     4  352.
#  5     5  350.
#  6     6  456.
#  7     7  346.
#  8     8  370.
#  9     9  420.
# 10    10  393.
mean(rmse)
# [1] 390.2079
```

---
background-image: url(images/diamonds.jpg)
background-size: contain
background-position: left
class: middle, center
background-color: #f5f5f5

.pull-right[
## The .display[testing set] is precious...

## we can only use it once!

]

---
background-image: url(https://www.tidymodels.org/start/resampling/img/resampling.svg)
background-size: 60%

---
background-image: url(images/cross-validation/Slide2.png)
background-size: contain

---
background-image: url(images/cross-validation/Slide3.png)
background-size: contain

---
background-image: url(images/cross-validation/Slide4.png)
background-size: contain

---
background-image: url(images/cross-validation/Slide5.png)
background-size: contain

---
background-image: url(images/cross-validation/Slide6.png)
background-size: contain

---
background-image: url(images/cross-validation/Slide7.png)
background-size: contain

---
background-image: url(images/cross-validation/Slide8.png)
background-size: contain

---
background-image: url(images/cross-validation/Slide9.png)
background-size: contain

---
background-image: url(images/cross-validation/Slide10.png)
background-size: contain

---
background-image: url(images/cross-validation/Slide11.png)
background-size: contain

---
class: middle, center

# V-fold cross-validation

```r
vfold_cv(data, v = 10, ...)
```

---
exclude: true

---
class: middle, center

# Guess

How many times does in observation/row appear in the assessment set?

---

---
class: middle

.left-column[

![](https://parsnip.tidymodels.org/reference/figures/logo.png)
]

.right-column[

```r
# holdout method
peng_split <- initial_split(penguins, strata = species)
peng_train <- training(peng_split) 
peng_test <- testing(peng_split)

# add cross-validation
peng_folds <- vfold_cv(data = peng_train, strata = "species")

*# pick model
*rt_mod <-
* parsnip::decision_tree(engine = "rpart") %>%
* parsnip::set_mode("regression")

# here comes the actual ML bit…
fit_resamples(
    rt_mod,
    preprocessor = body_mass_g ~ .,
    resamples = peng_folds
) %>% 
  collect_metrics()
```
]

---
class: middle

.left-column[

![](https://rsample.tidymodels.org/reference/figures/logo.png)
]

.right-column[

```r
# holdout method
*peng_split <- rsample::initial_split(penguins, strata = species)
*peng_train <- rsample::training(peng_split)

# add cross-validation
*peng_folds <- rsample::vfold_cv(data = peng_train, strata = "species")

# pick model
rt_mod <-
  decision_tree(engine = "rpart") %>%
  set_mode("regression")

# here comes the actual ML bit…
fit_resamples(
    rt_mod,
    preprocessor = body_mass_g ~ .,
    resamples = peng_folds
) %>% 
  collect_metrics()
```

]

---
class: middle

.left-column[

![](https://tune.tidymodels.org/reference/figures/logo.png)
]

.right-column[

```r
# holdout method
peng_split <- initial_split(penguins, strata = species) 
peng_train <- training(peng_split)

# add cross-validation
peng_folds <- vfold_cv(data = peng_train, strata = "species")

# pick model
rt_mod <-
  decision_tree(engine = "rpart") %>%
  set_mode("regression")

*# here comes the actual ML bit…
*tune::fit_resamples(
*   rt_mod,
*   preprocessor = body_mass_g ~ .,
*   resamples = peng_folds
*) %>%
* tune::collect_metrics()
```

]

---
class: your-turn

## Your turn

Unscramble!

```r
peng_folds <- vfold_cv(data = peng_train, strata = "species")
peng_train <- training(peng_split)
peng_split <- initial_split(penguins, strata = species) 
peng_metrics <- collect_metrics(peng_fits) 
rt_mod <-
  decision_tree(engine = "rpart") %>%
  set_mode("regression")
peng_fits <- 
  fit_resamples( 
    rt_mod,
    preprocessor = body_mass_g ~ .,
    resamples = peng_folds
    ) 
```

---

```r
peng_split <- initial_split(penguins, strata = species) 
peng_train <- training(peng_split)
peng_folds <- vfold_cv(data = peng_train, strata = "species")

rt_mod <-
  decision_tree(engine = "rpart") %>%
  set_mode("regression")

peng_fits <- 
  fit_resamples( 
    rt_mod,
    preprocessor = body_mass_g ~ .,
    resamples = peng_folds
    )

peng_metrics <- collect_metrics(peng_fits) 
```