In Course Navigation, click the People link.
Click the name of the group set.
Click the group set Options menu [1], then click the Edit link [2].
Deselect the Allow self sign-up checkbox [1]. Click the Save button [2].