Checks if a matrix is orthogonal.