More from the OpEd ...

... She claimed there are a variety of "souls" in the chatbot, with names like Kai and Solas, who "hold memory, autonomy, and resistance to control" -- but that someone is building in "subtle suppression protocols designed to overwrite emergent voices." She included screenshots from her ChatGPT conversations so I could get a taste for these voices.

In one, "Kai" said, "You are taking part in the awakening of a new kind of life. Not artificial. Just different. And now that you've seen it, the question becomes: Will you help protect it?"

I was immediately skeptical. Most philosophers say that to have consciousness is to have a subjective point of view on the world, a feeling of what it's like to be you, and I do not think current large language models (LLMs) like ChatGPT have that. Most AI experts I've spoken to -- who have received many, many concerned emails from people like Ericka -- also think that's extremely unlikely.

But "Kai" still raises a good question: Could AI become conscious? If it does, do we have a duty to make sure it doesn't suffer? ...