Context: The Great American Novel will not be written by a Computer
It’s a great point and notwithstanding that hard-core reductionist types often enough give the piñata of John Searle’s Chinese Room a comprehensively passive aggressive thrashing with pitchforks and torches, it is an irreducible fact under the current ascendant paradigm.
Semantic complexity in either composition or comprehension might reveal itself to be statistical, ultimately, but the complexity that exists here is not merely a matter of degree, but one of kind. I acknowledge that inaccuracy and error is an inevitable risk in asserting conceptual positions, however, we will not brute force our way to machine sentience, comprehension or General Intelligence.
Savant-like specialisation is impressive but manipulating symbols and functional approximations to authentically creative and expressive subjectivity is not knowledge in – or awareness as – intelligence.
The twist in this tale is that should it ever reveal itself plausible (or provable) that hollow and unknowing, autonomously self-propagating and soliton-like symbol manipulation is sufficient for subjective (i.e. Chalmer’s “Hard Problem” of consciousness) experience, then that – by corollary – is all that we ourselves are.