Very frequent
0/14
Supervised Learning
The algorithms Round 1 asks you to reason about and Round 2 asks you to fit, with the parameters worth touching in 90 minutes.
The algorithms Round 1 asks you to reason about and Round 2 asks you to fit, with the parameters worth touching in 90 minutes.
9 multiple choice at 2 marks and 3 fill-in-the-blank at 5, marked exactly as Round 1 marks them. Each answer is explained as soon as you check it. The clock is shown, not enforced.
CAIO tests supervised learning twice over: Round 1 asks why an algorithm behaves as it does, Round 2 asks you to make one work on a real dataset in ninety minutes.
Linear regression fits by least squares. Logistic regression predicts a probability through the sigmoid:
Round 1 favourites: logistic regression is a classifier despite the name; its decision boundary is linear in the features; and a coefficient is a change in log-odds, not in probability.
Lasso drives coefficients to exactly zero and so performs feature selection; ridge shrinks without eliminating. Increasing increases bias and reduces variance. Those three sentences answer most regularisation questions on this paper.
Splits maximise purity gain, measured by entropy or Gini:
Information gain is the parent impurity minus the weighted average of the children's. Practise this by hand on a five- or six-row table - it is a natural fill-in-the-blank question, and the arithmetic is quick once rehearsed.
An unrestricted tree overfits, which is why max_depth and min_samples_leaf exist and why ensembles work better.
For a 90-minute Round 2, HistGradientBoostingClassifier with early stopping is the highest-value default on tabular data:
from sklearn.ensemble import HistGradientBoostingClassifier
model = HistGradientBoostingClassifier(
max_iter=300, learning_rate=0.08,
early_stopping=True, validation_fraction=0.15, random_state=0,
)
It handles missing values natively and needs no scaling, which removes two preprocessing steps you would otherwise spend time on.
kNN classifies by majority vote among the nearest neighbours. It requires scaling, degrades in high dimensions, and has no training phase - all three are examinable properties.
Naive Bayes assumes conditional independence of features given the class. It is fast, works well on text, and the independence assumption is usually false without preventing it from working - a nuance Round 1 likes.
High training error and high validation error means underfitting: the model is too simple. Low training error with high validation error means overfitting. Learning curves distinguish them, and the standard fixes differ - more capacity or better features for the first, more regularisation or more data for the second.
Logistic regression is a classifier with a linear boundary; coefficients are log-odds. Lasso selects, ridge shrinks; larger means more bias, less variance.
Practise information gain by hand; it is quick marks. HistGradientBoosting* is the best 90-minute default on tabular data.
You increase the L2 penalty on a linear model. What happens?
A stronger penalty shrinks coefficients: the fit to the training sample gets worse (bias up) and less sample-sensitive (variance down).
Select an answer
A node holds 8 positives and 0 negatives. What is its Gini impurity ?
, so . A pure node has Gini 0.