AI In SF

Maybe "AI" has two meanings?

First:

Artificial intelligence (AI) refers to the simulation of human intelligence in machines that are programmed to think like humans and mimic their actions.
-copied from here.

Currently at least, machines do not "think" consciously but, at most, "simulate" or "mimic" intelligent thought.

Secondly, when I discuss "AI," I refer to the important philosophical distinction between consciousness and unconsciousness. Thus, I am primarily interested in whether an artifact can be conscious - and hopefully also intelligent. Such an artifact would not simulate but duplicate consciousness and conscious intelligence.

Isaac Asimov and Arthur C. Clarke thought that computers could be conscious. John Searle has argued not. Poul Anderson's sf covers every option with AI in either sense.

Perhaps the most advanced discussion and speculation about conscious AI is summarized in "The Man Who Would Be Kzin":

"The conscious brain is a computer,..." (p. 308)

I would have said not but let us see where this discussion goes:

"...but one of a very special kind." (ibid.)

A very special kind: it not only performs some computational functions but also is conscious.

"Not anything like a digital system;..." (ibid.)

Precisely. That is why I would not call it a computer.

"...that was one reason why true Artificial Intelligence had taken so long to achieve, and had proven so worthless once found." (ibid.)

This refers to other MKW stories co-written by Stirling.

"Consciousness does not operate on mathematical algorithms, with their prefixed structures." (ibid.)

No, because the workings of such algorithms and structures can be fully described without ascribing any consciousness to them.

"It is a quantum process, indeterminate in the most literal sense." (ibid.)

This seems intuitively plausible. Both quantum mechanics and consciousness are mysterious so might they be aspects of a single mystery? But I do not think that just any quantum event involves consciousness. An electron does not decide which way to jump. At least, we can describe its movements without referring to consciousness. However, on the macroscopic level, organisms (very different kinds of entities from computers) interact with their environments with increasing sensitivity. Complex, mobile organisms have central nervous systems. Within those systems, neuronic interactions might involve undetermined quantum processes. Indeterminacy seems to be an appropriate partial description, although not of course a full definition, of immediate awareness.

"Thoughts became conscious - decision was taken, will exercised - when the nervous system amplified them past the one-graviton threshold level. So was insight, a direct contact with the paramathematical frame of reality." (ibid.)

- including mystical oneness with ultimate reality.