Search

18 questions · 1 formulas

RMSE

RMSE = √[(e1² + e2² + … + en²) / n]

Trap: Lower hold-out RMSE → better out-of-sample. Higher adj R² → better in-sample.

Model Fit & RMSED–A level★ High priority — official Canvas example

Training model: ln(Y-hat)=0.50+0.80X. Hold-out observations (X, lnY): (1,1.10), (2,2.30), (3,2.80), (4,4.00). Find RMSE.

→ C) 0.212

Canvas Example MCQ — Q88

A) 0.045
B) 0.150
C) 0.212
D) 0.450
E) None of the above.

Predictions: 1.30, 2.10, 2.90, 3.70. Errors actual-predicted: -0.20, +0.20, -0.10, +0.30. Squared errors: .04,.04,.01,.09; mean = .18/4 = .045; RMSE = √.045 = .2121.

Rule: Do not stop at MSE=.045. RMSE is the square root, giving .212.

Model Fit & RMSED–A levelVery high priority

Estimate following model using the data br5 (sheet “br5” in the Excel file “exam_data_AI2152.xlsx”). Use all 900 observations to estimate the regression model: The estimated coefficient of the variable Pool is closest to_________

→ C) 0.0704

Old Exam — Oct 2023, Q9

Given: Historical software-dependent regression. Current adaptation: if a regression output is given, read the POOL coefficient directly and interpret it based on model form.

A) 0.0572
B) 0.0043
C) 0.0704
D) 0.1267
E) None of the above (A, B, C, D) is close to be correct.

Current exam rule: prediction uses the stated fitted equation; lower RMSE/AIC/SC is preferred when the criterion is being minimized; hold-out data must remain outside estimation/selection. Source answer: C) 0.0704.

Rule: The old exam supplied a spreadsheet/software. For the 2026 paper exam, expect the necessary values/table/output to be printed. Use the method, not the historical data file.

Model Fit & RMSED–A levelVery high priority

What is the Schwarz criterion (SC) for the model above (MCQ 9)? Use the formula The SC is closest to_________

→ A) ‒2.650

Old Exam — Oct 2023, Q10

Given: Historical model-selection calculation using Schwarz/BIC. Current concept: lower (more negative) SC/BIC is preferred.

A) ‒2.650
B) ‒2.930
C) ‒4.650
D) 2.930
E) None of the above (A, B, C, D) is close to be correct.

Rule: The old exam supplied a spreadsheet/software. For the 2026 paper exam, expect the necessary values/table/output to be printed. Use the method, not the historical data file.

Model Fit & RMSED–A levelVery high priority

In the original exam, the model was estimated using a supplied dataset.xlsx”). However this time use the first 850 observations to estimate the regression model: You want to compute the hold-out RMSE for the last 50 observations. The hold-out RMSE is closest to_______.

→ D) 0.2798

Old Exam — Oct 2023, Q11

Given: Historical hold-out RMSE calculation. Current concept: estimate on training sample only, predict test sample, RMSE=sqrt(mean squared prediction error).

A) 3.9149
B) 0.0783
C) 0.2852
D) 0.2798
E) None of the above (A, B, C, D) is close to be correct.

Rule: The old exam supplied a spreadsheet/software. For the 2026 paper exam, expect the necessary values/table/output to be printed. Use the method, not the historical data file.

Model Fit & RMSED–A levelVery high priority

Using the above model, predict the prices of a house with 2500 square feet that is not close to the university. Use the natural predictor (not the corrected predictor). The predicted price is closest to

→ B) $212 021

Old Exam — Dec 2022, Q23

Given: Historical software-dependent prediction question. Original model: ln(PRICE)=b0+b1 ln(SQFT)+d1 CLOSE+d2[CLOSE x ln(SQFT)].

A) $207 479
B) $212 021
C) $218 756
D) $225 335
E) None of the above (A, B, C, D) is close to be correct.

Rule: The old exam supplied a spreadsheet/software. For the 2026 paper exam, expect the necessary values/table/output to be printed. Use the method, not the historical data file.

Model Fit & RMSED–A levelVery high priority

Using the above model, predict the prices of a house with 2500 square feet that is close to the university. Use the natural predictor (not the corrected predictor). The predicted price is closest to

→ C) $303 169

Old Exam — Dec 2022, Q24

Given: Same historical prediction model as Q23, but CLOSE=1.

A) $228 665
B) $282 742
C) $303 169
D) $315 158
E) None of the above (A, B, C, D) is close to be correct.

Rule: The old exam supplied a spreadsheet/software. For the 2026 paper exam, expect the necessary values/table/output to be printed. Use the method, not the historical data file.

Model Fit & RMSED–A levelVery high priority

Suppose you want to apply the following formula when you work with predictive model selection: Which one of the following statements is true? Assume that you have data consisting of 900 observations in total (N = 900).

→ B) The observations included in N2 can either be the last x number of observations (e.g. x = 100) or chosen randomly.

Old Exam — Dec 2022, Q25

Given: Hold-out RMSE setup: total N=900 split into estimation sample N1 and test sample N2.

A) The observations included in N2 must be the last x number of observations (e.g. x = 100).
B) The observations included in N2 can either be the last x number of observations (e.g. x = 100) or chosen randomly.
C) N2 is usually much larger than N1.
D) N2 = N1 + N.
E) None of the above (A, B, C, D) is close to be correct.

A hold-out/test set can be a designated final block or a random split. It does not have to be the last observations. The key requirement is that it is not used to estimate/select the model.

Rule: Training data estimates/selects; hold-out data evaluates. Lower RMSE = better prediction.

Model Fit & RMSED–A levelVery high priority

→ A) $207 479

Old Exam — Oct 2022, Q17

Given: Historical software-dependent prediction; keep the log-model prediction concept, not the original spreadsheet mechanics.

A) $207 479
B) $185 650
C) $218 756
D) $204 454
E) None of the above (A, B, C, D) is close to be correct.

Rule: The old exam supplied a spreadsheet/software. For the 2026 paper exam, expect the necessary values/table/output to be printed. Use the method, not the historical data file.

Model Fit & RMSED–A levelVery high priority

Using the above model, predict the prices of a house with 2500 square feet that is close to the university. Use the natural predictor (not the corrected predictor). The predicted price is closest to

→ B) $282 742

Old Exam — Oct 2022, Q18

Given: Same as Q17 but dummy CLOSE=1.

A) $228 665
B) $282 742
C) $235 545
D) $265 350
E) None of the above (A, B, C, D) is close to be correct.

Rule: The old exam supplied a spreadsheet/software. For the 2026 paper exam, expect the necessary values/table/output to be printed. Use the method, not the historical data file.

Model Fit & RMSED–A levelVery high priority

When you performed the regression analysis above, you did not correct for problem with heteroscedasticity. Suppose that you have a problem with heteroscedasticity in the above model. Therefore, you estimate the model using the White heteroscedasticity-consistent estimator. (You do not need to do it to answer this question!) Which of the following statements is true? The White’s heteroscedasticity-consistent estimator prevent us from computing incorrect values_____

→ A) for test statistics or incorrect interval estimates in the presence of heteroskedasticity.

Old Exam — Oct 2022, Q19

Given: Concept question about White/heteroskedasticity-consistent standard errors.

A) for test statistics or incorrect interval estimates in the presence of heteroskedasticity.
B) for test statistics or incorrect interval estimates in the presence of homoscedasticity.
C) for unbiased estimators.
D) robust standard errors.
E) None of the above (A, B, C, D) is close to be correct.

Heteroskedasticity-robust/White SEs are used so t-tests and confidence intervals remain valid under heteroskedasticity. OLS coefficients themselves do not change.

Rule: Training data estimates/selects; hold-out data evaluates. Lower RMSE = better prediction.

Model Fit & RMSED–A levelVery high priority

Which model should you choose based on the SC criteria?

→ D) Model 5.

Old Exam — Dec 2021 re-exam, Q17

Given: Model-selection table: choose lowest SC/BIC.

A) Model 1.
B) Model 3.
C) Model 4.
D) Model 5.
E) None of the above (A, B, C, D) is correct.

SC/BIC: lower is better. Smallest SC is -4.63 for Model 5 -> D.

Rule: Training data estimates/selects; hold-out data evaluates. Lower RMSE = better prediction.

Model Fit & RMSED–A levelVery high priority

Which model should you choose based on the RMSE criteria?

→ C) Model 3.

Old Exam — Dec 2021 re-exam, Q18

Given: Model-selection table: choose lowest RMSE.

A) Model 1.
B) Model 2.
C) Model 3.
D) Model 4.
E) None of the above (A, B, C, D) is correct.

RMSE: lower is better. Smallest RMSE is 0.72 for Model 3 -> C.

Rule: Training data estimates/selects; hold-out data evaluates. Lower RMSE = better prediction.

Model Fit & RMSED–A levelVery high priority

Which model should you choose based on the AIC criteria?

→ D) Model 4.

Old Exam — Oct 2021, Q14

A) Model 1.
B) Model 2.
C) Model 3.
D) Model 4.
E) None of the above (A, B, C, D) is correct.

AIC: lower is better. The smallest value is -4.42 for Model 4 -> D.

Rule: Training data estimates/selects; hold-out data evaluates. Lower RMSE = better prediction.

Model Fit & RMSEE-levelVery high priority

What does RMSE = sqrt[(1/N)Σ(y−ŷ)²] measure, and in what units is it expressed if Y = ln(PRICE)?

→ A) Typical magnitude of prediction errors, expressed in log-price units (not dollars)

Computer Lab 6 — Exercise 1, Task 2

A) Typical magnitude of prediction errors, expressed in log-price units (not dollars)
B) The percentage of correctly classified observations, expressed as a percent
C) Typical prediction error, always expressed in dollars regardless of how Y is defined
D) The correlation between predicted and actual prices

RMSE summarizes typical prediction error magnitude, with squaring penalizing large misses more heavily; its units match the dependent variable, so if Y=ln(PRICE), RMSE is in log-price units, not dollars.

Rule: RMSE units always match the dependent variable's units/scale.

Model Fit & RMSED–A levelVery high priority

A sample of 900 observations is split into 800 training and 100 hold-out observations. What is the correct procedure to compute a valid hold-out RMSE for model selection purposes?

→ A) Estimate coefficients using only the 800 training observations, then predict and compute RMSE on the 100 hold-out observations without refitting

Computer Lab 6 — Exercise 1, Task 3

A) Estimate coefficients using only the 800 training observations, then predict and compute RMSE on the 100 hold-out observations without refitting
B) Estimate coefficients using all 900 observations, then compute RMSE on the 100 hold-out observations
C) Estimate coefficients using the 100 hold-out observations, then test on the 800 training observations
D) Use the hold-out set's actual outcomes to pick the model with the best fit, then report that RMSE

Valid out-of-sample evaluation requires estimating the model strictly on the training data and then generating predictions for the held-out data without re-estimating or peeking, preserving its role as a genuine test set.

Rule: Never use hold-out observations in estimation or model selection; they should be reserved strictly for a final independent assessment.

Model Fit & RMSED–A levelVery high priority

Comparing models m2 (Adj R²=0.7198, RMSE=0.2714), m9 (0.7377, 0.2615), m10 (0.7405, 0.2590), m11 (0.7408, 0.2633), m12 (0.7405, 0.2648): which model has the BEST in-sample fit, and which has the BEST out-of-sample prediction?

→ A) Best fit: m11 (highest adjusted R² = 0.7408); best prediction: m10 (lowest hold-out RMSE = 0.2590)

Computer Lab 6 — Exercise 1, Task 4

A) Best fit: m11 (highest adjusted R² = 0.7408); best prediction: m10 (lowest hold-out RMSE = 0.2590)
B) Best fit: m10; best prediction: m11
C) Best fit and best prediction are both m12
D) Best fit: m2; best prediction: m12

Among these models, m11 has the highest adjusted R² (0.7408), indicating the best in-sample fit, while m10 has the lowest hold-out RMSE (0.2590), indicating the best out-of-sample predictive accuracy; these need not be the same model.

Rule: Choose the model objective (fit vs. prediction) first: best in-sample fit → highest adjusted R²; best prediction → lowest hold-out RMSE.

Model Fit & RMSED–A levelVery high priority

An expanded model m13 was built by trying many size/age transformations and interactions, selected via cross-validation using only the first 800 (training) observations, achieving adj R²=0.7910 and hold-out RMSE=0.2384 — both better than the textbook models. What is the key methodological safeguard that makes this a fair comparison?

→ A) Cross-validation and model selection occurred strictly within the training data; the hold-out set was used only once for final evaluation

Computer Lab 6 — Exercise 1, Task 5

A) Cross-validation and model selection occurred strictly within the training data; the hold-out set was used only once for final evaluation
B) Using more variables automatically guarantees better out-of-sample prediction
C) The hold-out RMSE was used repeatedly to pick the best of many candidate models
D) Adding interactions always improves prediction regardless of how the model is selected

To avoid overfitting to the test data, all model search and tuning (cross-validation) was confined to the 800 training observations; the 100 hold-out observations were used only once, to report final, honest performance.

Rule: Model search/tuning must happen within training data only; the test/hold-out set should be touched only once, at the end.

Model Fit & RMSEE-levelVery high priority

You must choose the model that PREDICTS best on new data. Which measure?

→ B) Lowest hold-out RMSE

Formula Battle — method recognition (built from the Formula Book)

A) Highest R² in training sample
B) Lowest hold-out RMSE
C) Highest adjusted R²
D) Most significant coefficients

Out-of-sample matters. Correct method: Lowest hold-out RMSE.

Rule: Lower hold-out RMSE → better prediction