When a contact is deleted, it is "soft deleted"--only marked as
deleted. It remains in the deleted item folder. BCM only allows the
database owner to "hard delete" a contact, actually remove it from the
database.
If it is public folder you could set custom permissions by choosing under
Delete Items none. If they have Edit permission they could change all the
information, but the record could not be deleted. You could set a
combination like Edit Own Items, Delete Own Items so only records they
created could be edited or deleted.