Somewhat frequent
0/14
Kernel Methods & SVM
Margins, support vectors, and the kernel trick - syllabus domain nine, at the level Round 1 examines it.
Margins, support vectors, and the kernel trick - syllabus domain nine, at the level Round 1 examines it.
10 multiple choice at 2 marks and 2 fill-in-the-blank at 5, marked exactly as Round 1 marks them. Each answer is explained as soon as you check it. The clock is shown, not enforced.
Kernel methods are their own domain in the syllabus CAIO shares with IAIO. Round 1 examines the concepts; Round 2 may want you to fit one.
An SVM finds the separating hyperplane with the widest margin. The margin width is , so maximising it means minimising :
Only the points on or inside the margin determine the boundary. Every other training point could be deleted and the solution would be identical.
This is the property most likely to be examined, and it is also what makes SVMs memory-efficient at prediction time and sensitive to points near the boundary.
Real data is not separable, so slack variables permit violations, penalised by :
Note that this is the reverse of the usual regularisation convention, where a larger parameter means more regularisation. Questions exploit that reversal.
The optimisation depends on the data only through inner products, so replacing with operates in a higher-dimensional space without ever computing the mapping.
For RBF, sets how far a single point's influence reaches. Large means a tight, wiggly boundary that overfits; small approaches linear.
from sklearn.svm import SVC
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
model = make_pipeline(StandardScaler(), SVC(C=1.0, kernel="rbf", gamma="scale"))
Scaling is mandatory - the RBF kernel is a distance, so an unscaled feature with a large range swamps everything else. gamma="scale" is a sensible default that adapts to feature variance.
One practical warning: SVMs scale poorly with sample count, roughly quadratically to cubically. On a large Round 2 dataset within ninety minutes, a linear SVM or a boosted tree model is the safer choice.
Margin width is ; only support vectors define the boundary. Large means less regularisation - the opposite of the usual convention.
Always scale before an RBF kernel. SVMs get slow on large datasets; choose accordingly under time pressure.
In a soft-margin SVM, raising does what?
multiplies the slack penalty. Large makes violations expensive, so the margin tightens to avoid them.
Select an answer
. The geometric margin width is which integer?
.