Yes. You can add calendar entries with a CSV, just as you guessed. Export
your existing calendar so you can see what headers Outlook expects, then
create a file containing the birthdays you want. Import that. It won't add
the dates to the Contact records, though.