怎么推导

Linear Model Subset Selection (13:44)

Forward Stepwise Selection (12:26)

Backward Stepwise Selection (5:26)

Estimating Test Error — Mallow’s Cp, AIC, BIC, Adjusted R-squared (14:06)

Estimating Test Error — Cross-Validation (8:43)

Ridge Regression (12:37)

Lasso (15:21)

Tuning Parameters (5:27)

Dimension Reduction (4:45)

Principal Components and Partial Least Squares (15:48)

1. Linear Model Subset Selection (13:44)
2. Forward Stepwise Selection (12:26)
3. Backward Stepwise Selection (5:26)
4. Estimating Test Error — Mallow’s Cp, AIC, BIC, Adjusted R-squared (14:06)
5. Estimating Test Error — Cross-Validation (8:43)
6. Ridge Regression (12:37)
7. Lasso (15:21)
8. Tuning Parameters (5:27)
9. Dimension Reduction (4:45)
10. Principal Components and Partial Least Squares (15:48)

Pros of Linear…

Keys: Cross-validation & Bootstrap

1. Prediction Error and Validation Set

2. K-Fold Cross-Validation

3. Cross-Validation Do’s and Don’ts

4. Bootstrap

1. Prediction Error and Validation Set

How do we test methods like regression and classification out?

Usually a new sample, but we don’t always have new data

→ 2 resampling methods: Cross-validation & the Bootstrap

Goal: how well your prediction method works → the test set error of a model

• estimate test-set prediction error
• variance & bias of parameter estimates (Bootstrap)

Validation set approach — Estimate the test error

The Validation Process

1. What is the classification problem?

Classification is used to produce discrete results, eg.spam/not spam

Qualitative variables: unordered

More interested in❤️❤️： The probabilities that X belongs to each category in C

i.e. the estimate of the probability that an insurance claim is fraudulent

2. Why need it?

What is linear regression?

Linear regression belongs to supervised learning.

The assumption of linear regression?

Assumption: dependence of Y on X1, X2…Xp is linear

*However, the true regression functions are never linear!

Advantages & disadvantages of linear regression?

Advantage:

1. Linear regression is extremely useful both conceptually and practically although it may seem overly simplistic

Disadvantage:

1. It over-simplifies real-world problems by assuming a linear relationship between the variables.

Questions we might ask

• Is there a relationship between a & b?
• How strong is the relationship if so?
• Which x contributes to dependence Y?
• How accurately can we predict future dependence Y?
• …

Notes

Agenda

2.1 What is statistical learning?

2.1.1 Why estimate f?

2.1.2 How do we estimate f?

2.1.3 The trade-off between prediction accuracy and model interpretability

2.1.4 Supervised v.s. unsupervised learning

2.1.5 Regression vs classification problems

2.1 What is statistical learning?

statistical learning includes 1. input variable 2. output variable

1. input variable: predictors/independent variables/features/variables
2. 2. output variable:response/dependent variable

Why learn statistical learning?

i.e. in a marketing campaign, the goal is to develop an accurate model to predict sales on the basis of the three media budgets

2.1.1 Why estimate f?

2 main reasons we wish to estimate f: 1. …

Experiment Design

Final Project Instructions

Experiment Overview: Free Trial Screener

At the time of this experiment, Udacity courses currently have two options on the course overview page: “start free trial”, and “access course materials”.

If the student clicks “start free trial”, they will be asked to enter their credit card information, and then they will be enrolled in a free trial for the paid version of the course. After 14 days, they will automatically be charged unless they cancel first.

If the student clicks “access course materials”, they will be able to view the videos and take the quizzes for free, but they will not receive coaching support or a…

-How it works

-Real-life analogy

-Feature importance

-Difference between decision trees and random forests

-Important hyperparameters (predictive power, speed)

-How it works

Random forest — a supervised learning algorithm — is an ensemble of the decision trees. Put simply: random forest builds multiple decision trees and merges them together to get an accurate and stable prediction. It searches for the best features among a random subset of features.

Benefits: Can be used for both classification & regression problems.

-Real-life analogy

I decided where to go on vacation, my friends try to create rules to guide me based on my likes…

Agenda

• Sanity Checks
• Single Metric
• Multiple Metrics
• Analysis Gotchas

Running A/B tests is an iterative process

Sanity Checks

Choosing invariant metrics

• Population sizing metrics
• Any other metrics you don’t expect to change — What would be the same in the control and experiment group?

The course order change may change what courses visitors are enrolled in, so the time to complete a course will vary.

# events are good since it is the unit of diversion: event, which is randomly assigned between the experiment and control groups

cookies are being explicitly randomized

user Id is typically larger than cookies since one user…