Semantics Follows Frequency: Why Meaning Repeats and History Rhymes

When people speak about language, they often imagine meaning sitting inside words and sentences, ready to be transmitted from one mind to another. But language does not work as a collection of isolated containers. Words acquire significance through recurrence, relation and expectation. What has been said and done before changes what can meaningfully follow. Frequency matters because repetition does more than repeat. It establishes relations across time, and those relations become structure.

This is already implicit in the statistical study of language. Distributional linguistics showed that words acquire meaning through the contexts in which they recur. Computational methods later demonstrated that large patterns of co-occurrence can be decomposed into latent structures corresponding surprisingly well with semantic organisation. Modern language models extend the same principle at enormous scale: previous sequences condition the probability of subsequent ones. Meaning is not reducible to frequency, but neither is it independent of the accumulated statistical structure through which language recurs.

The spectral perspective becomes useful here. A complex signal can contain recurrent components operating at different frequencies, amplitudes and phases. Language also contains patterns recurring across different intervals and scales: sounds, words, phrases, idioms, grammatical structures, narratives, genres and cultural forms. Some are brief and frequent; others unfold across years or generations. Their recurrence establishes patterned relations between present and previous states. A phrase becomes an idiom, a behaviour a convention, a convention a norm, repeated procedures an institution. Structure, in this sense, is what persistent recurrence looks like across time.

Semantics therefore does not simply follow frequency in a straight line. Recurrence produces structure; structure acquires significance; significance changes the probability of recurrence. Once a phrase, narrative or distinction has acquired sufficient history, its presence changes what people expect, how subsequent events are interpreted and which responses become readily available. Meaning feeds back into probability. Language becomes a recursive field in which previous communication continually changes the distribution of subsequent communication.

The point is not to explain why a particular event must occur, but why some kinds of events become more likely. A slogan that keeps returning, a rumour that survives correction, or an accusation that reliably produces a familiar defence are different expressions of the same structure. Familiar formulations are easy to recognise and reproduce because they are already connected to expected responses. Repetition strengthens those relations, while predictable opposition can strengthen them further. These are low-energy communicative solutions: recurrence creates conditions for further recurrence.

That local order does not eliminate possibility elsewhere. Across the larger communicative ensemble, persistent narratives can generate reactions, counter-reactions, reinterpretations, conflicts and new alignments. Their organisation can therefore coexist with, generate and recursively depend upon greater combinatorial possibility across the wider field. A pattern encounters consequences it helped produce, and those consequences alter what becomes probable next.

The same structure provides a way for communicative patterns to change. An established narrative need not always be confronted from outside. If its persistence depends partly upon form, timing, sequence and expectation, something different can enter through those same relations. A new articulation can follow an established pattern closely enough to become readily intelligible while changing what that pattern makes likely to follow. The recurrent structure that stabilises meaning can therefore also become a means through which meaning changes.

This is where spectral language becomes more than metaphor. Repetition establishes temporal structure, and temporal structure produces patterned relations. Different recurrent patterns can reinforce, interfere with, amplify or suppress one another according to how they are organised through time. Language need not literally be an acoustic signal for these relationships to matter. Communicative organisation has temporal depth, recurrent structure and distributions of probability that can, in principle, be measured. Rhythm, repetition and relation are not ornaments added to semantics. They participate in its construction.

The deeper implication is that human language belongs to a broader class of language-like systems. A system becomes language-like where differentiated signals recur in relation to differentiated responses such that previous exchanges alter the probability of subsequent states. Human language is an unusually elaborate instance, but related organisation appears in genetic regulation, cellular signalling, neural activity and other systems in which differences become consequential for what follows. Recurrent interaction can establish memory and comparison, allowing previous states to constrain subsequent ones. Something resembling inference can then become possible.

Semantics follows frequency, then, but frequency also follows semantics. Each recurrence carries the accumulated structure of what came before, while each communicative event changes the probability field inherited by whatever follows. Meaning persists because relations recur, and recurrence persists because significance changes what becomes likely to recur. Language is not a static code laid over reality. It is an evolving organisation of probability through which recurrence becomes structure, structure becomes significance, and significance reorganises recurrence.

References

Blei, D. M., Ng, A. Y. and Jordan, M. I. (2003). ‘Latent Dirichlet Allocation’. Journal of Machine Learning Research, 3, pp. 993–1022.
Latent Dirichlet Allocation recovers higher-order patterns from distributions of words across documents. What matters here is that recurrent statistical relations can disclose structure that was never explicitly specified at the level of individual words.

Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K. and Harshman, R. (1990). ‘Indexing by Latent Semantic Analysis’. Journal of the American Society for Information Science, 41(6), pp. 391–407.
Latent Semantic Analysis extracts underlying dimensions from patterns of word occurrence across many documents. It demonstrates the important structural point that relations accumulated across many occurrences can reveal regularities unavailable from those occurrences considered separately.

Firth, J. R. (1957). ‘A Synopsis of Linguistic Theory 1930–1955’. In Studies in Linguistic Analysis. Oxford: Philological Society.
Firth located linguistic meaning partly in recurrent patterns of context and use. His familiar proposition that a word is known by “the company it keeps” points directly towards the argument here: significance is relational and acquires structure through recurrence.

Fourier, J. (1822). Théorie analytique de la chaleur. Paris: Firmin Didot.
Fourier analysis provides a way of decomposing complex variation into recurrent components operating at different frequencies. Its relevance here is not that language is literally a Fourier signal, but that apparently complex temporal organisation can contain recurrent structure that becomes visible when examined across different scales and frequencies.

Harris, Z. S. (1951). Methods in Structural Linguistics. Chicago: University of Chicago Press.
Harris showed that linguistic elements can be systematically investigated through their distributions relative to other elements. The important consequence here is that structure need not be assigned beforehand: it can be investigated through the relations produced by repeated occurrence.

Hofmann, T. (1999). ‘Probabilistic Latent Semantic Indexing’. Proceedings of the 22nd Annual International ACM SIGIR Conference, pp. 50–57.
Hofmann modelled patterns of word occurrence through probabilistic latent structure. This matters because semantic organisation need not be approached through fixed boundaries alone; it can also be represented through distributions in which some relations and outcomes are more probable than others.

Mikolov, T., Chen, K., Corrado, G. and Dean, J. (2013). ‘Efficient Estimation of Word Representations in Vector Space’. Proceedings of Workshop at ICLR.
Mikolov and colleagues showed that prediction across large amounts of language produces vector representations containing recoverable syntactic and semantic regularities. The interesting point here is not the particular model but the result: repeated contextual relations can accumulate into an organised space in which linguistic relationships become mathematically detectable.

Pennington, J., Socher, R. and Manning, C. (2014). ‘GloVe: Global Vectors for Word Representation’. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1532–1543.
GloVe derives linguistic representations from aggregate patterns of word co-occurrence. Particularly relevant is its use of relations among probabilities: frequency becomes informative not simply through how often something occurs, but through differences in how occurrences are distributed relative to one another.

Shannon, C. E. (1948). ‘A Mathematical Theory of Communication’. Bell System Technical Journal, 27(3), pp. 379–423.
Shannon established that communication can be formally described through probability and the distribution of possible signals without first resolving their meaning. That provides an important foundation for the present argument: what can occur, and with what probability, has structure before the question of what any particular occurrence signifies is settled.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. and Polosukhin, I. (2017). ‘Attention Is All You Need’. Advances in Neural Information Processing Systems, 30, pp. 5998–6008.
Transformer architectures construct contextual representations by dynamically weighting relations among elements of a sequence. Their relevance here is straightforward: linguistic prediction depends substantially upon relations among elements and the contexts those relations produce, rather than upon words functioning as independent containers of meaning.

Wiener, N. (1930). ‘Generalized Harmonic Analysis’. Acta Mathematica, 55, pp. 117–258.
Wiener extended harmonic analysis beyond simple periodic signals towards statistical structure persisting across temporal displacement. This is particularly important here because recurrence does not require exact repetition. A process can reproduce relational structure while every individual occurrence differs, allowing persistent organisation to exist beneath considerable surface variation.