GATE CS & DA Preparation Guide Module 9 – Machine learning and AI (DA sections 6-7)

k-NN, naive Bayes, LDA, SVM, trees and MLPs

The GATE DA classifiers by hand: k-NN votes, naive Bayes with Laplace smoothing, LDA directions, SVM margins, tree splits and MLP backpropagation.

Advanced
20 minutes
Examples run with Python 3.14.8 and Pyodide 314.0.7
By MySmartCoPilot

Free: Six classifiers, six ways to decide · k-NN votes among the nearest points, and k decides · Naive Bayes multiplies a prior by one likelihood per feature · 2 more sections. With a pass: 2 in-depth parts and 3 interview answers, about 13 minutes.

What you will learn

Classify points by hand with k-NN and with naive Bayes using Laplace smoothing
Compute a two-class LDA direction and decision rule, and the margin and support vectors of a linear SVM
Choose decision-tree splits by entropy and information gain or by Gini impurity
Compute forward passes, parameter counts and one backpropagation step for small MLPs

Before you start

Linear, ridge and logistic regression: GATE DA

What you’ll learn · Before you start

What you will learn

Classify points by hand with k-NN and with naive Bayes using Laplace smoothing
Compute a two-class LDA direction and decision rule, and the margin and support vectors of a linear SVM
Choose decision-tree splits by entropy and information gain or by Gini impurity
Compute forward passes, parameter counts and one backpropagation step for small MLPs

Before you start

Linear, ridge and logistic regression: GATE DA

On this page

Will the college library be crowded on a rainy weekend in exam week? Six classifiers of section 6 of the GATE DA syllabus, besides logistic regression, answer such a question in six ways. k-nearest neighbour (k-NN) takes a vote among the k closest training points. Naive Bayes multiplies the prior of a class by one likelihood per feature, treating the features as independent within the class. Linear discriminant analysis (LDA) projects the points on the direction that separates the class means best compared with the spread inside the classes. A linear support vector machine (SVM) picks the separating line with the widest margin. Decision trees ask about one feature at a time. Multi-layer perceptrons (MLPs), feed-forward neural networks, pass the input through layers of weighted sums and nonlinear activations.

GATE asks for a class, a probability, a margin, a gain or a parameter count; each is worked here on small invented data.

Not affiliated

No office of IIT Madras, IISc or any other IIT is connected with MySmartCoPilot or with this lesson.

Six classifiers, six ways to decide

Read each table across: the rule a classifier applies, what training does, its formula and the shape of its boundary.

k-NN and naive Bayes
Criterion	k-NN	Naive Bayes
Rule	Vote of the k nearest points	Largest prior × likelihoods
Training	None: keep the points	Count values per class
Formula	A distance, usually Euclidean	P(c \| x) ∝ P(c) Π P(xᵢ \| c)
Boundary	Any shape	Often linear

LDA and a linear SVM
Criterion	LDA	Linear SVM
Rule	Side of a threshold on wᵀx	Sign of w·x + b
Training	Class means and one covariance	The widest margin
Formula	w ∝ S_W⁻¹(μ₁ − μ₂)	Margin = 2/‖w‖
Boundary	Linear	Linear, or curved by a kernel

k-NN does no work in training, and every prediction rests on distances to the stored points, so a feature on a large scale dominates unless the features are standardised. Naive Bayes rarely has truly independent features, but the assumption is often good enough to rank the classes. LDA assumes every class is Gaussian with one shared covariance, which is what makes its boundary linear; unlike PCA, it uses the labels. Each needs something the others do not: k-NN a distance and a value of k, naive Bayes the counts per class, LDA the class means and their spread.

Checkpoint

Check: How each of the six classifiers decides

One question Why is the boundary of LDA a straight line, or a plane, between two classes?
Choose one answer.

LDA ignores the class labels and keeps the direction of largest variance LDA keeps only the support vectors LDA splits on one feature at a time LDA models every class as a Gaussian with one shared covariance, and with a shared covariance the quadratic terms cancel

Show the answer: One question

Answer: LDA models every class as a Gaussian with one shared covariance, and with a shared covariance the quadratic terms cancel

LDA assumes that every class is Gaussian with one covariance shared by all classes. Comparing two class scores then cancels the quadratic terms, and the boundary is linear. Unlike PCA, LDA uses the labels.

k-NN votes among the nearest points, and k decides

Eight invented points carry the labels pass and fail, and the query is (5, 5). Its five nearest neighbours are at squared distances 1, 2, 4, 5 and 8: a pass, two fails and two passes.

The query and its nearest neighbours for k = 1, 3 and 5

Text description of the diagram

A grid with feature x1 from 0 to 9 across and feature x2 from 0 to 8 up. Points labelled pass are filled circles: (5, 6), (3, 4), (7, 7) and (2, 7). Points labelled fail are hollow squares: (6, 4), (5, 3), (8, 2) and (1, 2). The query, a cross, is at (5, 5).

Three dashed circles are centred on the query. The smallest, radius 1, reaches the pass point (5, 6): with k = 1 the query is a pass. The middle one, radius 2, also holds the fail points (6, 4) and (5, 3): with k = 3 the vote is 2 to 1 for fail. The largest, radius about 2.83, adds the pass points (3, 4) and (7, 7): with k = 5 the vote is 3 to 2 for pass.

The dashed circles in the figure hold the 1, 3 and 5 nearest points. So the query is a pass for k = 1, a fail for k = 3 and a pass again for k = 5. Step through the vote:

Interactive · stage by stage The k-NN vote for the query (5, 5), as k grows Step through the vote for k = 1, 3 and 5. The answer changes twice.

The drawing shows the whole scene; its 4 steps are listed below.

Every step as text

Sort the points. Compute the squared distance from the query (5, 5) to each of the eight training points and sort them. The five nearest are at 1, 2, 4, 5 and 8.
One neighbour. With k = 1 only the nearest point votes, (5, 6) at squared distance 1, which is a pass. The query is a pass.
Three neighbours. With k = 3 the next two join, (6, 4) at 2 and (5, 3) at 4, both fails. The vote is two fails to one pass, so the query is a fail.
Five neighbours. With k = 5, (3, 4) at 5 and (7, 7) at 8 join, both passes. Three passes beat two fails, and the query is a pass again. The answer depends on k, which is why a question states it.

The answer depends on k, which is why a question always states it; in practice k is chosen by cross-validation, and an odd k avoids ties between two classes.

Checkpoint

Check: The k-NN vote and the choice of k

Question 1 of 2 For the query (5, 5) of the lesson, what does k = 2 give?
Choose one answer.

A fail, because fails are the majority of the training points A tie, one pass and one fail, which is why an odd k is used for two classes The same as k = 1, since k = 2 only breaks ties A pass, because the nearest point is a pass

Show the answer: Question 1 of 2

Answer: A tie, one pass and one fail, which is why an odd k is used for two classes

The two nearest points are (5, 6) at squared distance 1, a pass, and (6, 4) at 2, a fail. One vote each is a tie, which a two-class question avoids with an odd k.
Question 2 of 2 A k-NN model uses age in years and monthly income in rupees as its two features. What goes wrong, and what is the fix?
Choose one answer.

k is too small; use k = n Nothing, because k-NN does no training Two features are too few for k-NN Income differences in thousands swamp age differences in tens, so distance is nearly income alone; standardise both features first

Show the answer: Question 2 of 2

Answer: Income differences in thousands swamp age differences in tens, so distance is nearly income alone; standardise both features first

Every prediction measures the distance to every stored point, and the squared income difference is millions of times the squared age difference. Standardising each feature puts them on the same scale.

Naive Bayes multiplies a prior by one likelihood per feature

For naive Bayes, ten invented days at a college library are crowded or quiet. The counts per class are all that the classifier keeps:

Class	Days	Weekend	Exam week	Rain
Crowded	6	1	5	2
Quiet	4	3	0	2

Take a new day that is a weekend in exam week, with rain. Without smoothing, P(exam week | quiet) = 0/4, the whole product for quiet is 0 and the posterior of crowded is 1, however strongly the other features point to quiet. Laplace smoothing: Adding 1 to every count, and the number of possible values to every denominator, so that a value never seen with a class gets a small probability in naive Bayes instead of 0. adds 1 to every count of a feature value and the number of values, 2, to every denominator, while the priors stay 6/10 and 4/10. The scores become 27/640 for crowded and 1/45 for quiet, so P(crowded | weekend, exam week, rain) = 243/371 ≈ 0.655. Step through both computations:

Interactive · stage by stage Naive Bayes for a rainy weekend in exam week Step through the two class scores, first from the raw counts, then with Laplace smoothing.

The drawing shows the whole scene; its 5 steps are listed below.

Every step as text

The priors. Six of the ten days were crowded and four quiet, so the priors are 6/10 and 4/10. Each class score multiplies its prior by one likelihood per feature, here weekend, exam week and rain.
One zero count. No quiet day fell in exam week, so P(exam week | quiet) = 0/4. That single zero makes the whole product for quiet 0, and crowded wins with probability 1, whatever the weekend and the rain say.
Laplace smoothing. Smoothing adds 1 to every count of a feature value and the number of values, 2, to every denominator; the priors stay as they were. P(exam week | quiet) becomes (0 + 1)/(4 + 2) = 1/6, small but no longer 0.
The two scores. Crowded scores 3/5 × 1/4 × 3/4 × 3/8 = 27/640 and quiet scores 2/5 × 2/3 × 1/6 × 1/2 = 1/45. The weekend and the rain now count for quiet.
Normalise. Dividing by the sum of the two scores gives P(crowded | weekend, exam week, rain) = 243/371 ≈ 0.655, far less sure than the 1 of the raw counts.

k-NN for k = 1, 3 and 5, then naive Bayes with and without smoothing Python · nb_knn.py

# k-nearest neighbours for k = 1, 3 and 5, then naive Bayes with and without Laplace smoothing.
# Every dataset is invented. Fractions keep the probabilities exact.
from fractions import Fraction

# ---- k-NN: eight labelled points and one query, squared Euclidean distance (same order as the distance itself)
points = [((5, 6), "pass"), ((6, 4), "fail"), ((5, 3), "fail"), ((3, 4), "pass"),
          ((7, 7), "pass"), ((2, 7), "pass"), ((8, 2), "fail"), ((1, 2), "fail")]
query = (5, 5)
ranked = sorted(points, key=lambda p: (p[0][0] - query[0]) ** 2 + (p[0][1] - query[1]) ** 2)
print("k-NN for the query", query)
for (x, label) in ranked:
    print(f"  {x}  {label:<4}  squared distance {(x[0] - query[0]) ** 2 + (x[1] - query[1]) ** 2}")
for k in (1, 3, 5):
    votes = [label for _, label in ranked[:k]]
    winner = max(("pass", "fail"), key=votes.count)
    print(f"  k = {k}: {votes.count('pass')} pass, {votes.count('fail')} fail -> {winner}")

# ---- naive Bayes: ten invented days at a college library; is it crowded or quiet?
days = [  # (day, exam, weather, class)
    ("weekday", "exam week", "dry", "crowded"), ("weekday", "exam week", "rain", "crowded"),
    ("weekday", "exam week", "dry", "crowded"), ("weekend", "exam week", "dry", "crowded"),
    ("weekday", "no exam", "rain", "crowded"), ("weekday", "exam week", "dry", "crowded"),
    ("weekend", "no exam", "rain", "quiet"), ("weekend", "no exam", "dry", "quiet"),
    ("weekday", "no exam", "rain", "quiet"), ("weekend", "no exam", "dry", "quiet"),
]
new_day = ("weekend", "exam week", "rain")
values_per_feature = 2


def scores(alpha):
    """Prior times one likelihood per feature, with add-alpha smoothing of every count."""
    out = {}
    for c in ("crowded", "quiet"):
        rows = [d for d in days if d[3] == c]
        score = Fraction(len(rows), len(days))
        parts = [f"P({c}) = {score}"]
        for i, v in enumerate(new_day):
            count = sum(1 for d in rows if d[i] == v)
            p = Fraction(count + alpha, len(rows) + alpha * values_per_feature)
            score *= p
            parts.append(f"P({v} | {c}) = {p}")
        out[c] = score
        print("   ", ", ".join(parts), f"-> {score}")
    return out


for alpha in (0, 1):
    print(f"Naive Bayes for {new_day}, alpha = {alpha}")
    s = scores(alpha)
    posterior = s["crowded"] / (s["crowded"] + s["quiet"])
    print(f"  P(crowded | weekend, exam week, rain) = {posterior} = {float(posterior):.4f}")

Output

k-NN for the query (5, 5)
  (5, 6)  pass  squared distance 1
  (6, 4)  fail  squared distance 2
  (5, 3)  fail  squared distance 4
  (3, 4)  pass  squared distance 5
  (7, 7)  pass  squared distance 8
  (2, 7)  pass  squared distance 13
  (8, 2)  fail  squared distance 18
  (1, 2)  fail  squared distance 25
  k = 1: 1 pass, 0 fail -> pass
  k = 3: 1 pass, 2 fail -> fail
  k = 5: 3 pass, 2 fail -> pass
Naive Bayes for ('weekend', 'exam week', 'rain'), alpha = 0
    P(crowded) = 3/5, P(weekend | crowded) = 1/6, P(exam week | crowded) = 5/6, P(rain | crowded) = 1/3 -> 1/36
    P(quiet) = 2/5, P(weekend | quiet) = 3/4, P(exam week | quiet) = 0, P(rain | quiet) = 1/2 -> 0
  P(crowded | weekend, exam week, rain) = 1 = 1.0000
Naive Bayes for ('weekend', 'exam week', 'rain'), alpha = 1
    P(crowded) = 3/5, P(weekend | crowded) = 1/4, P(exam week | crowded) = 3/4, P(rain | crowded) = 3/8 -> 27/640
    P(quiet) = 2/5, P(weekend | quiet) = 2/3, P(exam week | quiet) = 1/6, P(rain | quiet) = 1/2 -> 1/45
  P(crowded | weekend, exam week, rain) = 243/371 = 0.6550

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 nb_knn.py

The run prints the k-NN vote of the previous section too.

Checkpoint

Check: Naive Bayes scores and Laplace smoothing

Question 1 of 2 With Laplace smoothing on the ten library days, what is P(exam week | quiet)?
Choose one answer.

1/5 0 1/4 1/6

Show the answer: Question 1 of 2

Answer: 1/6

Laplace smoothing adds 1 to every count and the number of values of the feature, 2 here, to every denominator: (0 + 1)/(4 + 2) = 1/6.
Question 2 of 2 Without smoothing, use the counts of the ten days to find P(crowded | weekday, no exam, dry), to three decimals.
Type a number.
Your answer

A hint

Weekday, no exam and dry are the complements of the counted values, so crowded has 5, 1 and 4 such days out of 6.

Show the answer: Question 2 of 2

Answer: 0.526 (anything from 0.524 to 0.528 counts)

Crowded scores 6/10 × 5/6 × 1/6 × 4/6 = 1/18 and quiet 4/10 × 1/4 × 4/4 × 2/4 = 1/20. Normalising, (1/18)/(1/18 + 1/20) = 10/19 ≈ 0.526.

SVMs, trees and MLPs each come down to one formula

The last two classifiers, in the same four rows as the tables above:

A decision tree and an MLP
Criterion	Decision tree	MLP
Rule	Answers from the root to a leaf	The largest output of the last layer
Training	Greedy splits by impurity	Gradient descent with the chain rule
Formula	H = −Σ p log₂ p	Σ (inputs + 1) × outputs parameters
Boundary	Boxes with axis-parallel sides	Any shape, with a hidden layer

An SVM’s support vector: A training point on an edge of a linear SVM's margin, where y(w·x + b) = 1. The support vectors alone fix the boundary: deleting any other training point leaves it where it is. are the training points on the edges of the margin, and they alone fix the boundary. A soft margin admits points inside the margin at a price set by the constant C. A kernel replaces the dot product so that the boundary can bend. Decision trees pick each split by the largest information gain: How much a split lowers the entropy of a node's classes: the node's entropy minus the weighted entropy of its children. A decision tree splits on the feature with the largest gain., or the largest fall in Gini impurity: 1 − Σ p² over the class shares of a node: 0 for a pure node and 0.5 for two classes in equal shares. A tree may split where it falls most, instead of by information gain., 1 − Σ p². Pruning stops a tree from memorising its training set. MLPs use activations such as the sigmoid, tanh or ReLU between layers. backpropagation: Computing the gradient of a network's loss with respect to every weight by the chain rule, from the output layer back towards the input, so that gradient descent can update the weights. computes the gradient of every weight with the chain rule. XOR is the classic function that no single linear unit computes. Step through one formula of each kind:

Interactive · stage by stage Three formulas, each on one small case Step through an SVM margin, the impurity of a tree node and the parameters of a network.

The drawing shows the whole scene; its 3 steps are listed below.

Every step as text

An SVM's margin. A linear SVM with w = (6, 8) has ‖w‖ = √(36 + 64) = 10, so its margin is 2/‖w‖ = 0.2. The longer w is, the narrower the margin.
A tree node's impurity. A node of four days, two crowded and two quiet, is as mixed as two classes get. Its entropy is −0.5 log₂ 0.5 − 0.5 log₂ 0.5 = 1 bit and its Gini impurity 1 − 0.5² − 0.5² = 0.5. A pure node scores 0 on both.
A network's parameters. A layer with n inputs and m outputs has n × m weights and m biases, (n + 1) × m in all. A 10-5-1 network has 11 × 5 + 6 × 1 = 61 parameters.

Checkpoint

Check: SVM margins, node impurity and parameter counts

Question 1 of 2 A linear SVM has w = (5, 12). How wide is its margin, to three decimals?
Type a number.
Your answer

Show the answer: Question 1 of 2

Answer: 0.154 (anything from 0.153 to 0.155 counts)

‖w‖ = √(25 + 144) = 13, so the margin is 2/13 ≈ 0.154.
Question 2 of 2 How many parameters, weights and biases, does a fully connected 6-4-1 network have?
Type a number.
Your answer

Show the answer: Question 2 of 2

Answer: 33

(6 + 1) × 4 + (4 + 1) × 1 = 28 + 5 = 33.

Practise: classifiers in GATE’s formats

Practise the four ideas, starting with the one you know least.

LDA, SVM margins and tree splits, worked in numbers Pro: opens with a Pro, Premium or Ultimate pass

Three classifiers of the list come down to one computation each: a direction, a margin and a split. This part computes all three on invented data, checks them by program and shows where the usual shortcuts fail.

In this part:

Fisher’s direction for two classes, computed exactly
The widest margin of a linear SVM, found by search
Information gain and Gini impurity for every split of the ten days
Your turn: choose a split by information gain
- Choose a decision-tree split by information gain

2 diagrams · 3 runnable examples (Python) · 1 exercise with sample tests · about 11 minutes

Opening this part with your pass…

This part opens with any MySmartCoPilot pass.

Pro pass · 1 week₹79$2.99
30 days₹179$5.99
1 year₹1,699$45

One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.

Choose a pass I have a licence key

It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).

MLPs by hand: parameter counts, XOR and one backpropagation step Pro: opens with a Pro, Premium or Ultimate pass

Network questions are counting and calculus done carefully. This part counts parameters, proves why XOR needs a hidden layer and takes one backpropagation step through a small network, to six decimals, with a numerical check.

In this part:

Counting parameters layer by layer, with the bias trap
Why one threshold unit cannot compute XOR, and two hidden units can
One backpropagation step through a 2-2-1 sigmoid network
Activation functions, their slopes and vanishing gradients
Comparing classifiers on a test set, with precision and recall
A ten-minute set, with answers

2 diagrams · 1 runnable example (Python) · about 10 minutes

Opening this part with your pass…Pro part. It opens with the same pass as the part above.

Interview questions

Fresher · Warm-up

What does the "naive" in naive Bayes assume, and what problem does Laplace smoothing fix?

It assumes that the features are independent once the class is known, so P(x₁, …, x_d | c) is the product of the P(xᵢ | c) and the classifier needs one small table per feature instead of a table of every combination. The assumption is rarely true, but naive Bayes only has to rank the classes and often ranks them well. Laplace smoothing fixes zero counts. A feature value never seen with a class gets probability 0, and that single zero wipes out the whole product for the class, whatever the other features say. Adding 1 to every count of a feature value, and the number of values to every denominator, gives it a small probability instead. In the lesson's library example the quiet class goes from a score of 0 to 1/45, and the posterior of crowded falls from 1 to 243/371, about 0.655.
Fresher to mid

A k-NN model predicts loan default from age in years, 20 to 60, and monthly income in rupees, ₹15,000 to ₹2,00,000. Its predictions barely change when age changes. Explain why with a distance calculation, fix it, and say how you would choose k.

Model answer, Pro. Model answers open with any pass; prices are in the notice on this page. First step and follow-ups
Fresher to mid

You train a hard-margin linear SVM, delete one training point and train again. When is the new boundary exactly the same, and when does it move? Show both cases on a small example and say what can happen to the margin.

Model answer, Pro. First step and follow-ups
Fresher to mid

Prove that a single linear threshold unit cannot compute the XOR of two bits. Then give a network with one hidden layer that does, with every weight and bias, and count its parameters.

Model answer, Pro. First step and follow-ups

Key takeaways

k-NN votes among the k nearest points, so the query (5, 5) is a pass, a fail and a pass for k = 1, 3 and 5.
Naive Bayes multiplies a prior by one likelihood per feature, and Laplace smoothing stops one zero count from deciding the class.
LDA and a linear SVM draw straight boundaries, trees draw boxes, and k-NN and MLPs can draw any shape.
An SVM’s margin is 2/‖w‖, a tree splits on the largest information gain, and an MLP layer has (inputs + 1) × outputs parameters.

Terms in this lesson

backpropagation: Computing the gradient of a network's loss with respect to every weight by the chain rule, from the output layer back towards the input, so that gradient descent can update the weights.
Gini impurity: 1 − Σ p² over the class shares of a node: 0 for a pure node and 0.5 for two classes in equal shares. A tree may split where it falls most, instead of by information gain.
information gain: How much a split lowers the entropy of a node's classes: the node's entropy minus the weighted entropy of its children. A decision tree splits on the feature with the largest gain.
Laplace smoothing: Adding 1 to every count, and the number of possible values to every denominator, so that a value never seen with a class gets a small probability in naive Bayes instead of 0.
support vector: A training point on an edge of a linear SVM's margin, where y(w·x + b) = 1. The support vectors alone fix the boundary: deleting any other training point leaves it where it is.

References

GATE 2027 syllabus, Data Science and Artificial Intelligence (section 6, machine learning) (Indian Institute of Technology Madras (GATE 2027))
scikit-learn user guide: nearest neighbours (scikit-learn developers)
scikit-learn user guide: naive Bayes (the independence assumption and smoothing) (scikit-learn developers)
scikit-learn user guide: linear and quadratic discriminant analysis (scikit-learn developers)
scikit-learn user guide: support vector machines (scikit-learn developers)
scikit-learn user guide: decision trees (entropy and Gini impurity) (scikit-learn developers)
scikit-learn user guide: neural network models (supervised) (scikit-learn developers)
Nearest neighbor pattern classification (Cover and Hart) (IEEE Transactions on Information Theory)
The use of multiple measurements in taxonomic problems (Fisher) (Annals of Eugenics)
Support-vector networks (Cortes and Vapnik) (Machine Learning (Springer))
Induction of decision trees (Quinlan) (Machine Learning (Springer))
Learning representations by back-propagating errors (Rumelhart, Hinton and Williams) (Nature)
Perceptrons: an introduction to computational geometry (Minsky and Papert) (The MIT Press)

Report a problem with this lesson

In this module: Machine learning and AI (DA sections 6-7)

Linear, ridge and logistic regression: GATE DA Free core, Pro depth: 9 of 25 minutes free
k-NN, naive Bayes, LDA, SVM, trees and MLPs Free core, Pro depth: 7 of 20 minutes free
k-means, k-medoids, hierarchical clustering and PCA Free core, Pro depth: 7 of 20 minutes free
AI search: uninformed, informed and adversarial Free core, Pro depth: 10 of 25 minutes free
Logic, Bayesian networks and inference: GATE DA Free core, Pro depth: 8 of 20 minutes free

Your country

k-NN, naive Bayes, LDA, SVM, trees and MLPs

What you will learn

Before you start

What you will learn

Before you start

Six classifiers, six ways to decide

k-NN votes among the nearest points, and k decides

Naive Bayes multiplies a prior by one likelihood per feature

SVMs, trees and MLPs each come down to one formula

Practise: classifiers in GATE’s formats

LDA, SVM margins and tree splits, worked in numbers Pro: opens with a Pro, Premium or Ultimate pass

MLPs by hand: parameter counts, XOR and one backpropagation step Pro: opens with a Pro, Premium or Ultimate pass

Interview questions

What does the "naive" in naive Bayes assume, and what problem does Laplace smoothing fix?

A k-NN model predicts loan default from age in years, 20 to 60, and monthly income in rupees, ₹15,000 to ₹2,00,000. Its predictions barely change when age changes. Explain why with a distance calculation, fix it, and say how you would choose k.

You train a hard-margin linear SVM, delete one training point and train again. When is the new boundary exactly the same, and when does it move? Show both cases on a small example and say what can happen to the margin.

Prove that a single linear threshold unit cannot compute the XOR of two bits. Then give a network with one hidden layer that does, with every weight and bias, and count its parameters.

Key takeaways

References

Related tools