It’s not clear if it’s possible to have only some categories of events have alerts (org-icalendar-alarm-time) and some not.

For example it would be awesome if appointments could have a 2 hour alert by default while “scheduled” and “deadline” items do not have an alert but are still visible in the phone’s calendar agenda. (As a workaround it’s probably possible to exclude them from the CalDAV export entirely.)

Here’s someone describing their org-caldav setup.