totalPenaltySpace function

Obtaining (orthogonal) basis for null space and range of the penalty matrix