Well, points are for print, and not for pixel-based displays, so I don't use
them at all (except in print stylesheets). And if you do use them, yes, you
will see differences in sizing from one browser/platform to another.
So it is best to use em for all text?
em is just a relative measure. It says "whatever size is currently being
displayed as the default, multiply it by this value and display this text
that size". The reason you would want to use em for letter-spacing is that
when the text size is increased by the viewer, you would want the spacing to
increase proportionately.
What you use to size your text is more a function of your personal
preferences, and your ability to understand what you are getting! Using
relatively sized text *can* be a bit confusing until you get there, since
it's additive across the page, meaning that if you have set your default
size to be .8 em, and you have another container which is explicitly sized
at .5 em, the result will be text sized at .4 em, i.e., .5 x .8.