Let $M$ be a $3\times 3$ matrix with real entries such that $M^{T}M=I$ and $\det (M)=1$. If $M\ne I$, find the value of $\det (M-I)$.

Solution

1. Since $M^{T}M=I$, $M$ is an orthogonal matrix, so its eigenvalues have modulus 1.
2. The characteristic polynomial is $p(\lambda )=\det (M-\lambda I)$.
3. Since $\det (M)=1$ and $M$ is $3\times 3$, the product of eigenvalues is $1$.
4. For a $3\times 3$ orthogonal matrix with $\det (M)=1$, $\lambda =1$ is always an eigenvalue.
5. Thus $\det (M-I)=0$. Hence the answer is (B).