I am very uncomfortable having all my files with "previous versions" stored
all over my PC in backups. Can this me limited or turned off?
They are not "all over [your] PC" but rather in the <driveletter>\System
Volume Information folder. No matter, do this:
1. Open Control Panel
2. Open System and Maintenance
3. Open System
4. Click System protection
5. Accept the elevation prompt
6. Uncheck the drives that you want to disable System Restore and Previous
Versions for.
Once you turn this off you lose both Previous Versions and System Restore.
For instance, if you turn it off on the C: drive and then install something
that completely horks your system you need to figure out how to manually
remove it. You cannot use System Restore to go back to a previous working
configuration.