I tested it Rudi and you are correct, and it still is visible. But, when I clicked on the image I was asked for a password.
So if someone wants to make an image completely invisible, then yes, they must remove it from all other Collections and leave it only in the Private one.
I don't know why my images stopped showing though, when I accidentally left them in the Private Collection. I always log OUT to get a link to give the buyer, and that is when I could not even see my images, cause they were in my Private collection still. Weird!