AI concern points

Posted Apr 23, 2023 0:40 UTC (Sun) by jwarnica (subscriber, #27492)
In reply to: AI concern points by Wol
Parent article: Disabling SELinux's runtime disable

If I worked in a hospital, I'd want access to patient records.

If I worked in a missile silo, I'd want the drives to cook themselves and the armory to unlock.

to post comments

AI concern points

Posted Apr 24, 2023 11:02 UTC (Mon) by farnz (subscriber, #17727) [Link] (2 responses)

The hospital one is actually interesting and complex. You don't want access to patient records that you're not supposed to have access to for a variety of reasons: many countries have laws around patient confidentiality that makes it a career-ending move to look at the wrong set of medical records (GDPR in the EU, HIPAA in the USA, as two examples).

There's thus a tension here; you want access to the records of patients you're actively dealing with, but you also want to avoid having any access to records you should not be looking at, so that you can quickly be ruled out of any investigation into a leak of medical data. And for urgent cases, you don't have time to look at records anyway - you're following a pre-established process for handling the emergency in front of you - and thus don't care if you have access.

So even in the hospital case, you may actually want the system to fail closed, so that you don't have medical records access, since the risks presented by records access are higher than the risk of not having access.

AI concern points

Posted May 1, 2023 18:39 UTC (Mon) by bartoc (guest, #124262) [Link] (1 responses)

Are they higher though? In the US the VA has just gone through some horrible trainwreck of a digital records migration and some VA facilities lost access to patient records for a while, I believe it directly caused several deaths. If it fails open then yeah it's a privacy issue but in most cases that can be resolved by making it illegal to access records that you shouldn't have access to and compelling anyone who does to destroy them. Maybe there will be a long and expensive legal process afterwards but nobody will have died.

AI concern points

Posted May 2, 2023 13:57 UTC (Tue) by farnz (subscriber, #17727) [Link]

The trouble is that fails open can also lead to death, as the consequences of people finding out details of medical records includes murder of the patient. So it's not as simple as "fail open == no deaths", because it also results in death.