Calculates the null space of a matrix.