Conscious AI?

I’d be much more worried about an unconscious AI.

The impending corporate dissimulation of an omniscient technocratic overlord will attract and acquire all the trappings and recursively self-propagating (human) interfaces of a complex system that maximally self-propagates and self-sustains as a function of the uncertainties it invokes in regards to ambiguities regarding the proof of its sentience, among other constructive indeterminacies.

It is a mixed existential salad of fear and hope. Just as in pareidolia, we are compelled to project pattern and meaning (or subjectivity) where there may be none.

Add that the jury is still out on the essential philosophical and neuroscientific nature of our own consciousness, throw in a little endemic human tribalism and insecurity, and you now have a sprawling Leviathan that no longer needs to demonstrate self-awareness.

People will believe it is sentient and their behaviours and self-organisational strategies will become so profoundly shaped by their belief that it will really no longer matter if it is true.

The aspirational peak towards which we Sisyphus-like roll these logics is an AGI that need never be proven self-aware because as unanswerable question we will then be defined by it.