Calculates the determinant of a matrix.