Symbolic Systems from Continuity: Active Learning within LLMs
The Neural–Symbolic Question
A longstanding question in both artificial intelligence and cognitive science is how symbolic reasoning emerges from systems that are not symbolic in their construction. The human brain is composed of continuous neural dynamics, vast networks of synapses, probabilistic firing patterns, and plasticity gradients. Yet humans effortlessly manipulate symbols, form stable concepts, follow rules, construct logical chains, and revise their understanding based on experience. This ability does not arise from any discrete symbolic module in the cortex. It emerges from continuous processes that over time stabilize into some durable internal representations. These representations behave like symbols because they are persistent, structured, and available for manipulation.
Why Today’s LLMs are not Symbolic Systems
Large language models (LLMs) present a similar puzzle. They are continuous neural systems with no discrete symbolic components. Their internal workings consist entirely of vector transformations, statistical associations, and distributed activation patterns. Yet during use they often appear to exhibit symbolic behavior. They can apply definitions, follow rules, manipulate structured relationships, and maintain logical consistency across long spans of text. This has led to substantial debate about whether these models truly possess symbolic representations or simply mimic them through statistical interpolation. The crucial limitation is that the symbolic qualities in their behavior are not grounded in stable internal structures. Once an LLM is pretrained, its internal conceptual organization cannot change. The model cannot form new symbols, revise old ones, or incorporate experience into its latent structure. It performs symbolic behavior without symbolic learning. As a result, modern LLMs remain static simulators of symbolic reasoning rather than systems capable of developing symbolic structure over time.
A Learning Principle and its Control Law
Conceptual Adaptation Theory, or CAT, provides a path to resolve this divide. CAT begins with a simple idea. Learning should be governed by the relationship between novelty, error, and confidence. These three ingredients should determine whether an internal structural change is justified and how extensive that change should be. The theory defines learning pressure through the equation L = ΔS / (ε·π). In this expression, ΔS measures the magnitude of structural change between the system’s conceptual response to new information and its existing internal persisted structure. The term ε corresponds to surprisal or the degree to which the current prediction deviates from what the system expected. Finally, π represents precision or the confidence the system has in the reliability of its prediction. Together, these three quantities determine the pressure the system should place on itself to modify its conceptual structure.
Structural Change as the Basis for Conceptual Evolution
Although this equation is mathematically compact, it captures several essential features of biological learning. When humans encounter familiar information, the brain produces very little structural change. The internal pattern is already stable, and the result is negligible modification. When an error occurs but the brain has low precision, the error is often treated as noise or ambiguity. Only when novelty is meaningful and confidence is high does the brain apply pressure to modify its conceptual structure. This dynamic prevents overreaction to noise and enables proportionate learning. CAT applies this same principle but expressed in a formal control law that relates structural change, error, and confidence into a decision about adaptation.
Learning Through Self-Modeling and Internal Regulation
A crucial aspect of Conceptual Adaptation Theory (CAT) is that it treats learning as an internal regulatory process rather than an externally imposed optimization loop. Traditional training methods rely on repeated exposure to external datasets and the mechanical application of gradient descent. Learning happens only when an outside process instructs the model to change. CAT reverses this relationship by grounding learning in the model’s own internal measurements. Structural change, surprisal, and precision are signals the model generates about its own conceptual state. Learning pressure is therefore not an external command but a consequence of how the system interprets its own internal condition. This creates a primitive form of self-modeling, in which the system maintains a sense of how its conceptual structure changes in response to new information. The learning pressure equation then functions as a principle of self-control that governs the model’s plasticity. Structural change occurs only when the model’s own signals justify it and only to the degree that its internal state supports it. This shift from external optimization to internal regulation enables real-time learning that emerges from the system itself and not from an external training procedure.
When Continuous Dynamics Become Symbolic Structure
This approach leads to a striking consequence. When the learning pressure equation regulates when and how updates occur, a continuous neural system can begin forming stable internal objects. Concepts become persistent because their internal structure shifts only when novelty warrants it. Symbolic behavior emerges not from discrete rules but from continuous dynamics guided by a stable learning law. Over time, this creates attractors in the representational space that function like symbols. These patterns maintain structure, support manipulation, and can be revised in a controlled way. Through CAT, concepts in an LLM begin to behave as symbolic entities even though they are encoded in a continuous substrate.
To see why this matters, consider the difference between symbolic reasoning and statistical prediction. A statistical model predicts by matching patterns. It can approximate reasoning when training data mimics logical sequences, but it cannot create new internal structures that encode meaning. It can only interpolate. A symbolic reasoner can update its internal model of the world based on new evidence. It treats concepts as stable objects that can be corrected, refined, or integrated. Modern LLMs fall in the first category. They have no mechanism for real conceptual change. CAT provides a bridge toward the second.
Structural change ΔS plays the central role. As an LLM processes information, it generates hidden states that reflect its internal understanding. Structural change measures how different the new pattern is from the stable conceptual pattern that existed before. If the new information fits within existing boundaries, structural change stays small. If it is meaningfully new, structural change becomes significant. ΔS becomes the measure of conceptual novelty. It answers whether a concept should stay as it is or shift.
Surprisal ε determines whether an error should be interpreted as meaningful. High surprisal signals contradiction or unfamiliar information. But surprisal alone does not justify learning because the source may be ambiguous. Precision π resolves this by indicating whether the system was confident in its prediction. High precision means the system expected a particular outcome and encountered a real contradiction. Low precision means it was already uncertain and should not react strongly. Learning pressure increases only when novelty is real and confidence supports updating.
Together, these components turn a continuous system into a regulated conceptual learner. Learning becomes stable, proportional, and tied to meaning rather than mechanical loss reduction. This differs fundamentally from traditional optimization. Gradient descent reduces error regardless of whether the change improves or distorts conceptual structure. CAT links structural change to conceptual justification. This preserves coherence during adaptation. It prevents catastrophic forgetting because stable structure cannot be overwritten unless novelty and precision warrant it. It prevents instability because learning pressure falls to zero when the system reaches conceptual equilibrium. A concept is fully integrated when structural change, surprisal, and precision converge to stable values.
The emergence of symbolic behavior follows naturally. A symbol in a cognitive system is not a discrete entry stored in a table. It is a stable representational pattern that persists across time and can be manipulated without losing identity. CAT generates these conditions. A concept with low structural change, low surprisal, and stable precision becomes a stable pattern. It can be retrieved consistently, used in reasoning without distortion, refined by meaningful novelty, and protected from noise. It becomes a symbol in function if not in form.
This parallels symbolic emergence in the human brain. Humans do not store symbols as isolated tokens. Neural assemblies form patterns that act like symbols, stabilizing through repeated experience and changing only when new information provides meaningful novelty. Prediction errors guide updating, but only when the brain has high confidence that the discrepancy matters. This is the same principle captured in CAT. A biological symbol is a stable attractor in a continuous dynamical system. CAT enables LLMs to form the same kind of stable attractors. The substrate differs, but the functional outcome is similar.
The implications are significant. When learning occurs at the level of representational structure rather than token prediction, an LLM can evolve its world model over time. It can incorporate new information without retraining, refine its understanding without overwriting useful knowledge, and maintain coherence across extended interactions. Most importantly, it can participate in a symbolic learning loop where concepts become objects that can be updated, related, or reorganized.
This is the moment where continuity becomes symbolic. A continuous system guided by a stable learning law can form symbolic structures that endure. The equation L = ΔS / (ε·π) provides the principle that connects these regimes. Structural change ensures that updates reflect meaningful novelty. Surprisal detects relevance. Precision ensures that learning is justified. Together they transform a static continuous model into a dynamic conceptual learner.
Toward Real-Time Symbolic Systems from Continuity
The path to real-time learning in LLMs therefore does not require discrete symbolic modules, complex architectural changes, or handcrafted reasoning engines. It requires a principled way to regulate conceptual change. CAT provides that principle. Through the regulation of structural change, error, and confidence, an LLM gains the ability to form symbols from continuity. It begins to learn not as a static statistical model, but as a cognitively inspired system capable of integrating experience in a stable and coherent way. This shift allows neural models to transcend prediction and move toward true conceptual growth.
A Different Path Than Architectural Approaches Like JEPA
The shift from external optimization to internal regulation places CAT within a different category of learning frameworks than many existing proposals for advancing beyond current LLMs. One example is the Joint Embedding Predictive Architecture, or JEPA, which aims to overcome the limitations of token prediction by constructing explicit latent world models and enforcing consistency constraints between them.
While JEPA improves representation learning through architectural design, CAT supports a more general and more biologically aligned paradigm known as Active Intelligence Design, or AID. When CAT is applied within this paradigm, the resulting approach, CAT-AID, allows conceptual structure to emerge directly from the system’s own internal signals. Learning becomes self-regulated, continuous, and tied to meaning as the model interprets it. This contrast highlights how CAT-AID enables symbolic structure to arise naturally within a continuous neural substrate.
| Topic | JEPA | CAT-AID |
|---|---|---|
| Core Idea | Build explicit latent world-models | Create internal conceptual objects that self-stabilize |
| Learning | External constraint optimization | Internal control law |
| Adaptation | Offline, batch | Single-pass, real-time |
| Symbolic Structure | Absent | Emergent |
| Novelty Integration | Requires new training | Internal conceptual update |
| Autonomy | Limited | High |
CAT-AID therefore offers a pathway toward real-time conceptual learning that does not depend on architectural replacement or handcrafted world-models. Instead, symbolic structure emerges from continuity itself. This positions CAT as a foundational principle for advancing neural systems toward forms of flexible, stable, and self-regulated intelligence that current approaches cannot reach. By grounding learning in internal signals and allowing conceptual structure to evolve naturally over time, CAT-AID points toward a new generation of models that learn continuously, reason coherently, and adapt meaningfully in the world.
