Caution urged in handling of AI data
Information technological advancement has vastly outpaced the US legal framework and the healthcare industry and criminal organizations, hackers, weaponised artificial intelligent (AI) and hostile nations increasingly see healthcare as a lucrative, data rich and soft target to exploit.
In a presentation at the 2023 Cayman Captive Forum Kris Oliveira, vice president clinical risk at Gallagher, Chad Brouillard, partner at Foster & Eldridge and Ryan Solomon, director at Sacova Insurance Company, said that the insurance market needs to look at AI carefully.
As AI / ChatGPT relies on data, algorithm & models the panel said that it is now constantly refining & evolving. They agreed that as healthcare depends on high levels of reliability AI can seem tempting to some as it reviews voluminous amounts of information faster than humans can.
Current AI use in healthcare includes radiology overreads, diagnostic aids, simplistic chatbots, scheduling, tracking, dashboarding and real time transcription.
AI here to stay, the panel agreed, pointing out that on the claims side, AI is being looked at for patient information & other details about treatment and that there is widespread interest in AI from major companies like Microsoft & Google.
Looking at the guardrails around AI or the current level of regulation, they said that as most present AI usage is not trying to act like medical devices, as a result there is little regulation over this area at the moment, but cautioned that increased levels of public worry over this has meant that the Biden Administration issued Executive Order at end of October on analysing AI. Further regulation is therefore probable.
As AI can create silos with a lot of information this makes it tempting for hackers who want access to that data, as healthcare data incredibly valuable in the US, the panel stressed.
In addition, bias can be injected into it by selective use of data by someone operating it and there is also the matter of copyright & plagiarism issues being raised by AI accessing data from a wide range of sources.
The panel also cautioned about what is known as ‘AI hallucination’ errors – it can make things up and if AI gives the wrong answer to a medical question, this can be an issue if this linked to setting standard of care. The panel also asked what would happen if an employee was told by their employer to use AI and it’s wrong – and they then get sued or held criminally liable.
The panel stressed that risk guidance must be provided and that those that use AI must use the data provided wisely, must keep up with governance and policies must use explicit language concerning the usage of AI. Check anything that comes out of AI – it's a tool that can be misused was the message from the panel, which added that the usage of AI is just beginning.
