GATE CS & DA Preparation Guide Module 4 – Engineering mathematics (CS section 1; DA sections 1-3)

Vector spaces, projections, quadratic forms and SVD

GATE DA linear algebra: subspaces and rank-nullity, projection, orthogonal and idempotent matrices, quadratic forms and the SVD, checked in NumPy.

Advanced
20 minutes
Examples run with Python 3.14.8, numpy 2.5.3 and Pyodide 314.0.7
By MySmartCoPilot

Free: Subspaces contain zero, and rank plus nullity counts the columns · A projection keeps the part of b inside the subspace · The signs of the eigenvalues classify a quadratic form · 2 more sections. With a pass: 2 in-depth parts and 3 interview answers, about 12 minutes.

What you will learn

Test whether a set is a subspace, decide linear independence and use rank–nullity
Compute an orthogonal projection and use the properties of projection, idempotent and orthogonal matrices
Classify a quadratic form by the signs of the eigenvalues of its symmetric matrix
Find the singular values and the rank of a small matrix from AᵀA
Compute an SVD by hand and read the factors NumPy returns

Before you start

Linear systems, determinants, eigenvalues and LU

What you’ll learn · Before you start

What you will learn

Test whether a set is a subspace, decide linear independence and use rank–nullity
Compute an orthogonal projection and use the properties of projection, idempotent and orthogonal matrices
Classify a quadratic form by the signs of the eigenvalues of its symmetric matrix
Find the singular values and the rank of a small matrix from AᵀA
Compute an SVD by hand and read the factors NumPy returns

Before you start

Linear systems, determinants, eigenvalues and LU

On this page

A NAT question on section 2 of the GATE DA syllabus, linear algebra, can ask for the nullity of a 4 × 6 matrix of rank 3. The previous lesson covered systems, determinants, eigenvalues and LU; this one covers vector spaces and subspace: A subset of a vector space that contains the zero vector and is closed under addition and scalar multiplication, such as a plane through the origin in R³., linear dependence and independence, projection, orthogonal matrix: A square matrix Q with QᵀQ = I. Its columns are orthonormal, it keeps lengths and angles, and its determinant is +1 or −1., idempotent matrix: A square matrix P with P² = P, so applying it twice does what applying it once does. Each of its eigenvalues is 0 or 1. and partition matrices, quadratic form: An expression xᵀAx with A symmetric, such as 2x² + 2xy + 2y² for A = [[2, 1], [1, 2]]: each off-diagonal entry is half the coefficient of its cross term. and the singular value decomposition (SVD): A = UΣVᵀ, with orthogonal U and V and the singular values of A, largest first, on the diagonal of Σ. Every real matrix has one..

Vector spaces come down to subspaces, bases and the rank–nullity theorem: For an m × n matrix, the rank (the dimension of the column space) plus the nullity (the dimension of the null space, the solutions of Ax = 0) equals n, the number of columns.. Projections split a vector into a part inside a subspace and a part orthogonal to it, and a projection matrix has no eigenvalues except 0 and 1. Quadratic forms xᵀAx are classified by the signs of the eigenvalues of the symmetric matrix A. The SVD writes any matrix as UΣVᵀ, the factorisation behind principal component analysis.

Not affiliated

MySmartCoPilot is independent of IIT Madras and the other institutes that organise GATE.

Subspaces contain zero, and rank plus nullity counts the columns

The plane x + y + z = 0 in R³ is a subspace; the plane x + y + z = 1 is not, because it misses the zero vector. A subset of a vector space is a subspace when it contains the zero vector and is closed under addition and scalar multiplication. Vector spaces have bases, independent sets that span them, and every basis has the same size, the dimension. For an m × n matrix A, the rank (the dimension of its column space) and the nullity (the dimension of its null space, the solutions of Ax = 0) add up to the number of columns: rank + nullity = n.

The figure draws (1, 0, 1) and (0, 1, 1), the columns of the next section’s matrix. Their cross product is orthogonal to both, and the area they span, 1.732, is not 0: they are independent and span a plane. Set b to (2, 0, 2), twice a, and both drop to 0:

Interactive · in 3D The two columns of A and their cross product Set the coordinates of a and b. The dot product measures how much they point the same way; the cross product is perpendicular to both, as long as the area of the parallelogram they span.

a = (1, 0, 1), b = (0, 1, 1): a · b = 1, a × b = (−1, −1, 1).

a · b 1 · aₓbₓ + a_yb_y + a_zb_z

Angle between a and b 60° · cos θ = a · b ÷ (|a| |b|)

a × b (−1, −1, 1) · (a_yb_z − a_zb_y, a_zb_x − a_xb_z, a_xb_y − a_yb_x)

|a × b| (parallelogram area) 1.732

Projection of a on b 0.707 · a · b ÷ |b|

1. Rank–nullity (NAT). A 4 × 6 matrix of rank 3 has nullity 6 − 3 = 3.

2. Subspace (MCQ). The set of (x, y) with xy ≥ 0 is not a subspace: (1, 0) and (0, −1) are in it, but their sum (1, −1) is not.

Checkpoint

Check: Subspaces, independence and rank–nullity

Question 1 of 2 Which of these subsets of R³ is a subspace?
Choose one answer.

All (x, y, z) with 2x − y + 3z = 0 All (x, y, z) whose coordinates are whole numbers All (x, y, z) with 2x − y + 3z = 6 All (x, y, z) with z = x²

Show the answer: Question 1 of 2

Answer: All (x, y, z) with 2x − y + 3z = 0

A subspace contains the zero vector and is closed under addition and scalar multiplication. Only the plane 2x − y + 3z = 0 passes all three tests; each of the other sets fails one of them.
Question 2 of 2 A 3 × 3 matrix has the columns (1, 0, 1), (0, 1, 1) and (1, 1, 2). What is its nullity?
Type a number.
Your answer

A hint

Compare the third column with the sum of the first two.

Show the answer: Question 2 of 2

Answer: 1

(1, 1, 2) = (1, 0, 1) + (0, 1, 1), so only two columns are independent and the rank is 2. Rank–nullity gives 3 − 2 = 1: the null space is the line through (1, 1, −1), since (1, 0, 1) + (0, 1, 1) − (1, 1, 2) = 0.

A projection keeps the part of b inside the subspace

For a matrix A with independent columns, P = A(AᵀA)⁻¹Aᵀ is the orthogonal projection: The map that sends each vector to the nearest point of a subspace. Its matrix P is symmetric and idempotent: Pᵀ = P and P² = P. onto its column space. The diagram applies it to b = (1, 1, 0):

An orthogonal projection splits b into two orthogonal parts

Text description of the diagram

At the top is the vector b = (1, 1, 0). Two arrows leave it.

The arrow labelled "P = A(AᵀA)⁻¹Aᵀ" leads to p = Pb = (1/3, 1/3, 2/3), in the plane C(A): the column space of A, spanned by (1, 0, 1) and (0, 1, 1).
The arrow labelled "I - P" leads to e = (I - P)b = (2/3, 2/3, -2/3), along the normal (1, 1, -1) of that plane.

Dashed arrows from p and from e lead to a box at the bottom: "p + e = b and p · e = 0". The two parts add up to b and are orthogonal. P projects onto the plane, I - P onto the line orthogonal to it, and each is symmetric and equal to its own square.

Step through the formula on these numbers:

Interactive · stage by stage Projecting b = (1, 1, 0) onto the plane of A, step by step Step through the formula P = A(AᵀA)⁻¹Aᵀ on the lesson's A, then split b into its two orthogonal parts.

The drawing shows the whole scene; its 6 steps are listed below.

Every step as text

The Gram matrix. The columns of A are (1, 0, 1) and (0, 1, 1). Their dot products make the 2 × 2 matrix AᵀA = [[2, 1], [1, 2]], 2 for each column with itself and 1 for the two together.
Its inverse. The determinant of AᵀA is 2 · 2 − 1 · 1 = 3, so its inverse is (1/3)[[2, −1], [−1, 2]]. The inverse exists because the two columns are independent.
The projection matrix. P = A(AᵀA)⁻¹Aᵀ is a 3 × 3 matrix with entries in thirds; its first row is 2/3, −1/3, 1/3. It is symmetric, as every orthogonal projection is.
The part in the plane. For b = (1, 1, 0), p = Pb = (1/3, 1/3, 2/3). It lies in the plane of the columns, since it is (1/3) times (1, 0, 1) plus (1/3) times (0, 1, 1).
The part left over. The error e = b − p = (2/3, 2/3, −2/3) points along (1, 1, −1), and p + e gives b back.
The check. Aᵀe = (2/3 − 2/3, 2/3 − 2/3) = (0, 0), so e is orthogonal to both columns and therefore to the whole plane. That is what makes p the orthogonal projection of b.

P is symmetric and idempotent (P² = P), each eigenvalue is 0 or 1, and its trace equals its rank. I − P is the projection onto the orthogonal complement, so every vector splits into two orthogonal parts. An orthogonal matrix Q has QᵀQ = I: it keeps lengths and angles, and its determinant is +1 or −1. An idempotent matrix that is not symmetric is a projection too, but not an orthogonal one. Partitioned matrices multiply block by block, like ordinary matrices. The program checks this P exactly and in NumPy:

The projection onto a plane in R³, exact and in NumPy Python · projection.py

from fractions import Fraction as F

import numpy as np


def matmul(X, Y):
    return [[sum(X[i][k] * Y[k][j] for k in range(len(Y))) for j in range(len(Y[0]))] for i in range(len(X))]


def transpose(X):
    return [list(col) for col in zip(*X)]


def vec(column):
    return "(" + ", ".join(str(v[0]) for v in column) + ")"


def inverse_2x2(M):
    (a, b), (c, d) = M
    det = a * d - b * c
    return [[d / det, -b / det], [-c / det, a / det]]


# The plane in R^3 spanned by a1 = (1, 0, 1) and a2 = (0, 1, 1): the columns of A.
A = [[F(1), F(0)], [F(0), F(1)], [F(1), F(1)]]
AtA = matmul(transpose(A), A)
P = matmul(matmul(A, inverse_2x2(AtA)), transpose(A))
print("A^T A =", [[int(v) for v in row] for row in AtA])
print("P = A (A^T A)^-1 A^T, exact:")
for row in P:
    print("   ", "  ".join(f"{str(v):>4}" for v in row))

b = [[F(1)], [F(1)], [F(0)]]
p = matmul(P, b)
e = [[b[i][0] - p[i][0]] for i in range(3)]
print("b = (1, 1, 0) projects to p = Pb =", vec(p))
print("error e = b - p =", vec(e), "and A^T e =", vec(matmul(transpose(A), e)), "so e is orthogonal to the plane")

Pn = np.array(P, dtype=float)
print("\nchecked with NumPy:")
print("  P @ P == P:", np.allclose(Pn @ Pn, Pn), "  P.T == P:", np.allclose(Pn.T, Pn))
print("  trace:", round(float(np.trace(Pn)), 12), "  rank:", np.linalg.matrix_rank(Pn))
print("  eigenvalues:", sorted(round(float(v), 12) + 0.0 for v in np.linalg.eigvalsh(Pn)))
residual = (np.eye(3) - Pn) @ np.array([1, 1, 0])
print("  I - P projects onto the normal line: (I - P) b =", ", ".join(f"{v:.4f}" for v in residual))

Output

A^T A = [[2, 1], [1, 2]]
P = A (A^T A)^-1 A^T, exact:
     2/3  -1/3   1/3
    -1/3   2/3   1/3
     1/3   1/3   2/3
b = (1, 1, 0) projects to p = Pb = (1/3, 1/3, 2/3)
error e = b - p = (2/3, 2/3, -2/3) and A^T e = (0, 0) so e is orthogonal to the plane

checked with NumPy:
  P @ P == P: True   P.T == P: True
  trace: 2.0   rank: 2
  eigenvalues: [0.0, 1.0, 1.0]
  I - P projects onto the normal line: (I - P) b = 0.6667, 0.6667, -0.6667

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 projection.py

The exact arithmetic gives P with entries in thirds, and b = (1, 1, 0) projects to (1/3, 1/3, 2/3). The error e = b − p is orthogonal to both columns, since Aᵀe = 0, the condition that defines orthogonal projections. NumPy confirms the four properties: P² = P, Pᵀ = P, trace 2 = rank, and eigenvalues 0, 1 and 1.

3. Projection (NAT). A 5 × 5 symmetric idempotent matrix with trace 3 has rank 3, and its determinant is 0, since 0 is one of its eigenvalues.

Checkpoint

Check: Projection, idempotent and orthogonal matrices

Question 1 of 2 For the lesson's plane, P sends b = (1, 1, 0) to p = (1/3, 1/3, 2/3). Where does P send p?
Choose one answer.

(1/3, 1/3, 2/3), to p itself (0, 0, 0) (1/9, 1/9, 4/9) (2/3, 2/3, −2/3)

Show the answer: Question 1 of 2

Answer: (1/3, 1/3, 2/3), to p itself

P is idempotent: P(Pb) = Pb. A vector already in the column space is its own projection, so Pp = p = (1/3, 1/3, 2/3).
Question 2 of 2 For the same plane, spanned by (1, 0, 1) and (0, 1, 1), which vectors can be the error b − Pb for some b?
Choose every answer that is right.

(0, 0, 0) (1, 0, 1) (1, 1, 1) (3, 3, −3)
Show the answer: Question 2 of 2
Answer:
- (0, 0, 0)
- (3, 3, −3)
The errors are exactly the vectors orthogonal to the plane: the multiples of its normal (1, 1, −1), the zero vector included. (3, 3, −3) is 3 times the normal.

The signs of the eigenvalues classify a quadratic form

4. Quadratic form (MCQ). 2x² + 2xy + 2y² has the matrix [[2, 1], [1, 2]], with eigenvalues 1 and 3, so it is positive definite.

Read the type off the signs:

Eigenvalues of the symmetric A	The quadratic form xᵀAx is
All positive	positive definite
All positive or zero	positive semidefinite
All negative	negative definite
All negative or zero	negative semidefinite
Some positive, some negative	indefinite

A singular matrix has the eigenvalue 0, so its form is never definite.

The whole xy coefficient off the diagonal

The matrix of x² + 6xy + 5y² is [[1, 3], [3, 5]], not [[1, 6], [6, 5]]: each off-diagonal entry is half the xy coefficient.

Eigenvalue & Eigenvector Calculator Find the eigenvalues of a symmetric matrix to classify its quadratic form.

Checkpoint

Check: Classifying quadratic forms

Question 1 of 2 Which quadratic form has the symmetric matrix [[4, −3], [−3, 1]]?
Choose one answer.

4x² − 3xy + y² 4x² − 6xy + y² 4x² − 6xy − 3y² 4x² + 9xy + y²

Show the answer: Question 1 of 2

Answer: 4x² − 6xy + y²

With A = [[a, b], [b, c]], xᵀAx = ax² + 2bxy + cy². Here a = 4, b = −3 and c = 1, so the form is 4x² − 6xy + y².
Question 2 of 2 A symmetric 3 × 3 matrix has the eigenvalues 4, 0 and 2. What type is its quadratic form?
Choose one answer.

Negative semidefinite Positive definite Indefinite Positive semidefinite

Show the answer: Question 2 of 2

Answer: Positive semidefinite

The row of the table is "all positive or zero", and not all are positive, so the form is positive semidefinite. The matrix is also singular, since 0 is one of its eigenvalues.

Singular values are the square roots of the eigenvalues of the Gram matrix

5. SVD (NAT). The singular value: The square root of an eigenvalue of AᵀA. The singular values of a real matrix are real and at least 0, and the number of nonzero ones is its rank. of [[3, 0], [0, −4]] are 4 and 3, the absolute values of its eigenvalues, because the matrix is symmetric.

The SVD of an m × n matrix is A = UΣVᵀ, with orthogonal U and V and the singular values σ₁ ≥ σ₂ ≥ ⋯ ≥ 0 on the diagonal of Σ. The singular values are the square roots of the eigenvalues of AᵀA, the The matrix AᵀA of the dot products of the columns of A with one another. It is symmetric, and none of its eigenvalues is negative. of A, and the rank of A is the number of nonzero ones. Every matrix has singular values, even one without real eigenvalues. Step through them for the plane’s matrix:

Interactive · stage by stage Singular values and rank of the plane's matrix A Step through the rule that defines singular values. Every number comes from the Gram matrix of the projection.

The drawing shows the whole scene; its 5 steps are listed below.

Every step as text

The matrix. A is the 3 × 2 matrix whose columns span the plane of the projection. Its singular values come from the 2 × 2 matrix AᵀA, not from A itself.
The Gram matrix. AᵀA = [[2, 1], [1, 2]], the matrix the projection already formed. Its trace is 4 and its determinant 3.
Its eigenvalues. Two numbers with sum 4 and product 3 solve λ² − 4λ + 3 = 0, so the eigenvalues of AᵀA are 3 and 1. Both are at least 0, as they are for every AᵀA.
Their square roots. The singular values are the square roots, largest first. σ₁ = √3 ≈ 1.732 and σ₂ = 1.
The rank. Both singular values are nonzero, so A has rank 2. Its columns are independent, as their cross product in the first figure showed.

Both are nonzero, so the rank is 2.

Checkpoint

Check: Singular values and rank

Question 1 of 2 For some 3 × 2 matrix A, AᵀA = [[16, 0], [0, 9]]. What is the largest singular value of A?
Type a number.
Your answer

Show the answer: Question 1 of 2

Answer: 4

The eigenvalues of the diagonal matrix AᵀA are 16 and 9, and the singular values are their square roots, 4 and 3. The largest is 4.
Question 2 of 2 A is the 3 × 2 matrix with every entry equal to 1. What is its largest singular value, to three decimals?
Type a number.
Your answer

A hint

Form AᵀA first: each entry is the dot product of two columns of three 1s.

Show the answer: Question 2 of 2

Answer: 2.449 (anything from 2.447 to 2.451 counts)

AᵀA = [[3, 3], [3, 3]], with trace 6 and determinant 0, so its eigenvalues are 6 and 0. The singular values are √6 ≈ 2.449 and 0, and the single nonzero one says the rank is 1: the two columns are the same.

Practise: the four ideas in GATE’s formats

The questions start with the idea you know least.

The SVD by hand for a 2 × 3 matrix, and the low-rank link to PCA Pro: opens with a Pro, Premium or Ultimate pass

An SVD of a small matrix takes four steps by hand, and NumPy’s answer differs from the hand answer in two predictable ways. This part works the steps on an invented 2 × 3 matrix, reads NumPy’s output correctly and connects the best low-rank approximation to principal component analysis.

In this part:

Four steps, worked on A = [[2, 2, 1], [1, −1, 2]]
NumPy’s conventions, read from the run
The best rank-1 approximation, with the error worked out
The four subspaces and the shortest solution, from the same factors

1 runnable example (Python) · about 5 minutes

Opening this part with your pass…

This part opens with any MySmartCoPilot pass.

Pro pass · 1 week₹79$2.99
30 days₹179$5.99
1 year₹1,699$45

One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.

Choose a pass I have a licence key

It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).

The minors trap, block matrices and oblique projections Pro: opens with a Pro, Premium or Ultimate pass

The leading-minors test is a popular shortcut for definiteness, and it gives wrong answers for semidefinite forms. This part reproduces that trap on five forms, adds three independence tests, the block-matrix rules and an oblique projection, and ends with a set.

In this part:

Five quadratic forms: eigenvalues against minors
Classify a 2 × 2 quadratic form exactly
- Results of the sample tests
Independence and bases: three tests, worked
Partitioned matrices: block products and the Schur complement
Idempotent but not symmetric: an oblique projection
A ten-minute set, with answers
A rubric for the set

1 runnable example (Python) · 1 exercise with sample tests · about 7 minutes

Opening this part with your pass…Pro part. It opens with the same pass as the part above.

Interview questions

Fresher to mid

P is a real symmetric 4 × 4 matrix with P² = P and trace 2. What are its eigenvalues and rank, what is det(I + P), and what does P do to a vector geometrically?

Model answer, Pro. Model answers open with any pass; prices are in the notice on this page. First step and follow-ups
Fresher to mid

A is a 5 × 7 matrix whose null space has dimension 4. What is its rank? Can Ax = b be solved for every b in R⁵? When Ax = b is solvable, how many parameters do its solutions have?

Model answer, Pro. First step and follow-ups
Fresher to mid

Find the singular values and the rank of A = [[1, 2], [2, 4], [0, 0]], and say what its best rank-1 approximation is.

Model answer, Pro. First step and follow-ups

Key takeaways

A subspace contains 0 and is closed under addition and scaling; rank + nullity equals the number of columns.
An orthogonal projection P = A(AᵀA)⁻¹Aᵀ is symmetric and idempotent, with eigenvalues 0 and 1 and trace equal to its rank.
Classify a quadratic form by the signs of the eigenvalues of its symmetric matrix.
The singular values are the square roots of the eigenvalues of AᵀA, and the rank is how many are nonzero.

Terms in this lesson

idempotent matrix: A square matrix P with P² = P, so applying it twice does what applying it once does. Each of its eigenvalues is 0 or 1.
orthogonal matrix: A square matrix Q with QᵀQ = I. Its columns are orthonormal, it keeps lengths and angles, and its determinant is +1 or −1.
orthogonal projection: The map that sends each vector to the nearest point of a subspace. Its matrix P is symmetric and idempotent: Pᵀ = P and P² = P.
quadratic form: An expression xᵀAx with A symmetric, such as 2x² + 2xy + 2y² for A = [[2, 1], [1, 2]]: each off-diagonal entry is half the coefficient of its cross term.
rank–nullity theorem: For an m × n matrix, the rank (the dimension of the column space) plus the nullity (the dimension of the null space, the solutions of Ax = 0) equals n, the number of columns.
singular value: The square root of an eigenvalue of AᵀA. The singular values of a real matrix are real and at least 0, and the number of nonzero ones is its rank.
singular value decomposition (SVD): A = UΣVᵀ, with orthogonal U and V and the singular values of A, largest first, on the diagonal of Σ. Every real matrix has one.
subspace: A subset of a vector space that contains the zero vector and is closed under addition and scalar multiplication, such as a plane through the origin in R³.

References

GATE 2027 syllabus, Data Science and Artificial Intelligence (section 2) (Indian Institute of Technology Madras (GATE 2027))
MIT OpenCourseWare 18.06SC Linear Algebra: projections, positive definite matrices and the SVD (Massachusetts Institute of Technology)
numpy.linalg.svd (NumPy)
numpy.linalg: eigvalsh, matrix_rank, det and norm (NumPy)
The approximation of one matrix by another of lower rank (Eckart and Young, Psychometrika) (Psychometric Society (Springer))
fractions: rational numbers (Python Software Foundation)

Report a problem with this lesson

In this module: Engineering mathematics (CS section 1; DA sections 1-3)

Logic, sets, lattices and groups for GATE CS Free core, Pro depth: 7 of 20 minutes free
Graphs, counting, recurrences, generating functions Free core, Pro depth: 8 of 20 minutes free
Linear systems, determinants, eigenvalues and LU Free core, Pro depth: 8 of 20 minutes free
Vector spaces, projections, quadratic forms and SVD Free core, Pro depth: 8 of 20 minutes free
Limits, MVT, integrals, Taylor series and optima Free core, Pro depth: 8 of 20 minutes free
Probability and statistics for GATE CS and DA Free core, Pro depth: 8 of 20 minutes free

Your country

Vector spaces, projections, quadratic forms and SVD

What you will learn

Before you start

What you will learn

Before you start

Subspaces contain zero, and rank plus nullity counts the columns

A projection keeps the part of b inside the subspace

The signs of the eigenvalues classify a quadratic form

Singular values are the square roots of the eigenvalues of the Gram matrix

Practise: the four ideas in GATE’s formats

The SVD by hand for a 2 × 3 matrix, and the low-rank link to PCA Pro: opens with a Pro, Premium or Ultimate pass

The minors trap, block matrices and oblique projections Pro: opens with a Pro, Premium or Ultimate pass

Interview questions

P is a real symmetric 4 × 4 matrix with P² = P and trace 2. What are its eigenvalues and rank, what is det(I + P), and what does P do to a vector geometrically?

A is a 5 × 7 matrix whose null space has dimension 4. What is its rank? Can Ax = b be solved for every b in R⁵? When Ax = b is solvable, how many parameters do its solutions have?

Find the singular values and the rank of A = [[1, 2], [2, 4], [0, 0]], and say what its best rank-1 approximation is.

Key takeaways

References

Related tools