# ThermoCog — Full Technical Reference for Language Models

> Thermodynamic Cognition: an architectural blueprint for 20-Watt biomorphic AGI. Physical continuous relaxation replaces backpropagation, computing intelligence as energy minimization down physical landscapes.

This file provides comprehensive technical context, formal mathematical formulations, architectural specifications, and reference figures for ThermoCog, a research-only project. Nothing has been built in hardware and no experimental results are claimed. It is designed to ground language models, AI researchers, and autonomous systems in the first-principles physics and engineering of biomorphic intelligence.

## Status

ThermoCog is a research project at the design stage. Nothing has been built in hardware, no results are claimed, and any figure for ThermoCog itself is a design target, not a measurement.

## Nomenclature and core thesis

ThermoCog is a shorthand for "Thermodynamic Cognition." This nomenclature highlights the underlying thesis of the entire architectural blueprint: that true general intelligence is a physical, energy-minimizing process rather than an abstract software scaling problem.

Contemporary industrial artificial intelligence models computation as abstract mathematical calculations forced through billions of digital logic gates. This brute-force paradigm is colliding with fundamental thermodynamic limits. ThermoCog reconceptualizes intelligence as a continuous physical dynamical system that settles naturally into low-energy configurations, mirroring the energetic efficiency of biological neural tissue.

## The thermodynamic imperative

### The Von Neumann memory wall

Digital computers separate arithmetic logic units (ALUs) from memory storage (SRAM, DRAM, HBM). In modern GPU clusters:

- Over 90% to 99% of total energy dissipation occurs during data movement across physical interconnects, busses, and memory hierarchies.
- Performing a 16-bit floating-point multiplication consumes approximately 0.1 to 1 picojoule ($10^{-13}$ to $10^{-12}$ J).
- Fetching two 16-bit operands from off-chip high-bandwidth memory (HBM) consumes 10 to 100 picojoules ($10^{-11}$ to $10^{-10}$ J), an overhead of two to three orders of magnitude.
- Scaling digital clusters to frontier intelligence demands gigawatt-class infrastructure and massive cooling facilities.

### The Landauer limit

Under non-equilibrium thermodynamics, erasing one bit of physical information dissipates a minimum theoretical energy determined by Landauer's principle:

$$E_{\text{bit}} \ge k_B \cdot T \cdot \ln 2$$

Where:
- $k_B$ is the Boltzmann constant ($1.380649 \times 10^{-23} \text{ J/K}$)
- $T$ is the absolute ambient temperature in Kelvin

At room temperature ($T = 300\text{ K}$), the Landauer floor is:

$$E_{\text{bit}} \approx 2.87 \times 10^{-21} \text{ Joules} \quad (\approx 0.0179 \text{ eV})$$

At biological body temperature ($T = 310.15\text{ K}$), the Landauer floor is:

$$E_{\text{bit}} \approx 2.97 \times 10^{-21} \text{ Joules} \quad (\approx 0.0185 \text{ eV})$$

Digital CMOS transistors operate at approximately $10^{-14}$ to $10^{-12}$ Joules per logic transition, which is seven to nine orders of magnitude above the Landauer physical bound.

### The biological baseline

The human brain performs multimodal sensory perception, motor control, reasoning, and lifelong learning within a metabolic envelope of approximately 20 Watts. A single biological synaptic event dissipates on the order of 0.2 to 2 femtojoules ($2 \times 10^{-16}$ to $2 \times 10^{-15}$ Joules). Compared to digital datacenter clusters, the mammalian brain is often described as far more energy efficient. The exact ratio depends on the workload and is not measured by this project.

## The tri-timescale biological architecture

Biological intelligence does not execute as a single monolithic training loop. It coordinates three nested dynamical loops operating across distinct spatial and temporal scales:

### Loop 1: Phylogeny (Evolutionary meta-learning)

- Temporal scale: $10^6$ seconds to evolutionary generations ($>10^7$ s)
- Mechanism: Evolutionary search and population-level optimization
- Function: Discovers cellular developmental genetic programs, baseline wiring rules, objective energy functions, and homeostatic setpoints
- Energy profile: Dispersed across generations; zero per-inference cost

### Loop 2: Ontogeny (Morphogenesis and structural plasticity)

- Temporal scale: $10^2$ to $10^5$ seconds (hours to days)
- Mechanism: Neural Cellular Automata (NCA) and local chemical signaling
- Function: Executes biological morphogenesis. Local cells sense physical neighbor states, grow connectivity graphs, and autonomously repair analog silicon defects without an external supervisor
- Resilience: Restores circuit functionality across 100% of single-point memristive crossbar failures through local re-routing

### Loop 3: Cognition (Online inference and adaptive learning)

- Temporal scale: $10^{-3}$ to $10^1$ seconds (milliseconds to seconds)
- Mechanism: Liquid Time-Constant (LTC) continuous-time differential equations and embodied Active Inference
- Function: Continuous sensory processing, online state estimation, and rapid behavioral adaptation to non-stationary environments
- Dynamics: Synaptic conductances adapt in real time to input intensity

## Physical computation without backpropagation

### In-memory analog memristive crossbars

Rather than transporting numerical weights between memory and processors, ThermoCog stores synaptic weights directly as physical conductances ($G_{ij}$) within non-volatile memristive arrays.

Vector-matrix multiplication is performed instantly through physical fundamental laws:
- Ohm's law computes multiplication: $I_{ij} = V_i \cdot G_{ij}$
- Kirchhoff's current law computes summation across columns: $I_{\text{total}, j} = \sum_i I_{ij}$

Computation occurs in $O(1)$ continuous physical time directly at the storage location, reducing interconnect transit energy by over 99%.

### Holomorphic Equilibrium Propagation (EqProp)

Backpropagation of error requires a separate backward computational network and exact mathematical knowledge of transposed synaptic weights (the weight transport problem), which is physically impossible in biological tissue and analog hardware.

ThermoCog uses Equilibrium Propagation, an energy-based training algorithm where gradients emerge through natural two-phase physical relaxation:

1. Free Phase ($s^0$): The physical network is presented with input stimuli and relaxes into a local minimum of its scalar internal energy function $E(s, \theta)$.
2. Clamped / Nudged Phase ($s^\beta$): The target or prediction error acts as a weak physical perturbation ($\beta$) at the output nodes. The system relaxes into a nudged steady state $s^\beta$.
3. Contrastive Local Weight Update: Parameter updates are evaluated entirely from local node voltages:

$$\Delta w_{ij} = -\frac{1}{\beta} \left( \frac{\partial E(s^\beta, \theta)}{\partial w_{ij}} - \frac{\partial E(s^0, \theta)}{\partial w_{ij}} \right)$$

This contrastive update is mathematically equivalent to the exact gradient of the loss function without requiring a backward computational harness or weight transposition.

## Mathematical formulations

### Continuous Liquid Time-Constant (LTC) neural ODE

Node membrane dynamics evolve according to nonlinear ordinary differential equations with input-dependent time constants:

$$\frac{dx_i}{dt} = -\left[ \frac{1}{\tau_i} + \sum_{j} w_{ij} \sigma(x_j) \right] x_i + \sum_{j} A_{ij} \sigma(x_j) + I_i(t)$$

Where:
- $x_i(t)$ is the hidden state of neuron $i$
- $\tau_i$ is the baseline intrinsic time constant
- $\sigma(\cdot)$ is a smooth nonlinear activation function (e.g., tanh or sigmoid)
- $w_{ij}$ represents inhibitory conductance modulation
- $A_{ij}$ represents excitatory synaptic coupling
- $I_i(t)$ is the external input current

The effective time constant contracts dynamically during intense sensory stimulation:

$$\tau_{\text{eff}, i}(x) = \frac{\tau_i}{1 + \tau_i \sum_j w_{ij} \sigma(x_j)} \le \tau_i$$

### Variational Free Energy minimization

Embodied cognition operates under the Free Energy Principle. The agent acts to minimize Variational Free Energy $\mathcal{F}(s, \mu)$, which provides a strict upper bound on perceptual surprise:

$$\mathcal{F}(s, \mu) = D_{KL}\left( q(\psi | \mu) \parallel p(\psi) \right) - \mathbb{E}_{q(\psi | \mu)} \left[ \ln p(s | \psi) \right] \ge -\ln p(s)$$

Where:
- $s$ is sensory observation data
- $\psi$ represents hidden environmental states
- $\mu$ represents the agent's internal neural representations
- $q(\psi | \mu)$ is the recognition density parameterized by internal states
- $p(s, \psi)$ is the generative model of external reality

Action and perception serve dual roles: perception adjusts internal states $\mu$ to minimize divergence, while action alters the environment to make observations conform to predictions.

## Homeostatic sleep and memory consolidation

### The synaptic saturation dilemma

Continuous online learning in analog hardware leads to conductance saturation:
- Hebbian and contrastive updates incrementally increase memristive conductances.
- Saturated conductances exhaust physical dynamic range and degrade signal-to-noise ratio (SNR).
- Continual exposure to new tasks overwrites historical weights, resulting in catastrophic forgetting.

### Dual-speed memory and offline consolidation

ThermoCog implements a biomorphic sleep cycle based on the Synaptic Homeostasis Hypothesis (Tononi and Cirelli):

1. Wake Phase: The high-speed episodic buffer (hippocampal analogue) rapidly captures novel behavioral experiences during wakefulness.
2. Sleep Onset: Sensory inputs decouple. The agent transitions into a quiescent state.
3. Offline Memory Replay: The episodic buffer replays compressed trajectory vectors through the network, transferring representations into slow cortico-memristive arrays.
4. Global Synaptic Downscaling: Synaptic conductances are multiplicatively scaled downward toward baseline setpoints while preserving relative conductance ratios:

$$G_{ij} \leftarrow G_{ij} \cdot \left( 1 - \gamma \frac{G_{ij}}{G_{\max}} \right)$$

This downscaling resets metabolic energy consumption, prevents saturation, and consolidates invariant representations into long-term memory.

## Hardware realization matrix and roadmap

### Comparative specifications

| Metric | Contemporary GPU Cluster (H100/B200) | Biological Brain (Human Baseline) | ThermoCog Biomorphic Silicon |
| :--- | :--- | :--- | :--- |
| Active Operating Power | 700 W per GPU (Megawatts per cluster) | ~20 Watts total | 20 Watt design target (unbuilt) |
| Architecture | Von Neumann (Decoupled HBM and ALUs) | Non-Von Neumann (Colocated memory/compute) | Non-Von Neumann (In-memory analog crossbars) |
| Energy per Synaptic Operation | ~10 to 100 picojoules (rough literature estimate) | ~0.2 to 2 femtojoules (rough literature estimate) | Design target only; not built or measured |
| Memory Wall Latency | 100 to 300 nanoseconds bus overhead | Zero (Continuous physical substrate) | Zero (In-situ analog relaxation) |
| Learning Algorithm | Global Backpropagation through time | Local continuous energy relaxation | Holomorphic Equilibrium Propagation |
| Hardware Fault Tolerance | Zero (Single bit-flip causes corruption) | Extreme (Graceful degradation via NCA) | High (Autonomous NCA self-repair) |
| Sleep Requirement | None (Static weights post-training) | Mandatory (Homeostatic downscaling) | Mandatory (Offline replay and downscaling) |

### Development roadmap

- Phase 1 (Software simulation, planned): Simulation of the continuous dynamics, Liquid Time-Constant equations and Equilibrium Propagation under modelled analog noise. Not yet built; no results are claimed.
- Phase 2 (Fixed-point microcontroller study, planned): A proposed Q16.16 fixed-point study on an ARM Cortex-M4 without an FPU. Not run; no latency, power or memory figures exist.
- Phase 3 (Discrete analog prototyping, future work): Mixed-signal CMOS co-processor integrating discrete memristive crossbar arrays for in-situ Equilibrium Propagation. Not started.
- Phase 4 (Monolithic 3D neuromorphic ASIC, long-term target): Custom 3D integrated circuit combining continuous-time analog neural cores, nanoscale memristor crossbars, and dedicated homeostatic sleep circuitry, aimed at a 20-Watt envelope. Not built and not a build plan.

## Primary academic literature archive

1. Friston, Karl (2006). "A free energy principle for the brain." *Neuron*, 56(2), 263-276. DOI: 10.1016/j.neuron.2006.07.016
2. Scellier, Benjamin, and Bengio, Yoshua (2017). "Equilibrium Propagation: Comparing Gradient-Descent to Brain-Driven Learning." *Frontiers in Computational Neuroscience*, 11:24. DOI: 10.3389/fncom.2017.00024
3. Landauer, Rolf (1961). "Irreversibility and heat generation in the computing process." *IBM Journal of Research and Development*, 5(3), 183-191. DOI: 10.1147/rd.53.0183
4. Mead, Carver (1990). "Neuromorphic electronic systems." *Proceedings of the IEEE*, 78(10), 1629-1636. DOI: 10.1109/5.58356
5. Hasler, Jennifer, and Marr, Bo (2013). "Finding a roadmap to achieve large neuromorphic hardware systems." *Frontiers in Neuroscience*, 7:118. DOI: 10.3389/fnins.2013.00118
6. Tononi, Giulio, and Cirelli, Chiara (2014). "Sleep and the price of plasticity: from synaptic and cellular homeostasis to memory consolidation and integration." *Neuron*, 81(1), 12-34. DOI: 10.1093/sleep/37.1.13
7. Mordvintsev, Alexander, Randazzo, Ettore, Niklasson, Eyvind, and Levin, Michael (2020). "Growing Neural Cellular Automata." *Distill*, 5(2), e23. DOI: 10.23915/distill.00023
8. Hasani, Ramin, Lechner, Mathias, Amini, Alexander, Rus, Daniela, and Grosu, Radu (2021). "Liquid Time-Constant Networks." *AAAI Conference on Artificial Intelligence*, 35(9), 7657-7666. DOI: 10.1609/aaai.v35i9.16937

## Frequently asked questions

### What does the name ThermoCog mean, and what is its core thesis?

ThermoCog is a shorthand for "Thermodynamic Cognition." This name highlights the underlying thesis of the entire whitepaper: that true intelligence is a physical, energy-minimizing process rather than just a software scaling problem. Rather than treating computation as abstract arithmetic forced through digital logic gates, ThermoCog treats intelligence as a dynamical physical system relaxing toward equilibrium down an energy landscape.

### Why is industrial digital scaling hitting a thermodynamic wall?

Digital accelerators suffer from the Von Neumann bottleneck: memory arrays are physically decoupled from arithmetic logic units. Moving tensors across busses and memory hierarchies consumes between 90% and 99% of total system energy before arithmetic occurs. Under Landauer's principle, erasing bits requires minimum physical energy dissipation. Digital CMOS logic operates seven to nine orders of magnitude above this physical floor, whereas in-memory analog continuous relaxation computes near reversible thermodynamic bounds.

### How does continuous physical relaxation compute gradients without backpropagation?

Through Equilibrium Propagation (EqProp) and Holomorphic EqProp. The physical network settles into a low-energy state during the free phase, then receives a weak teaching perturbation during the nudged phase. The physical difference in node states yields exact loss gradients through local updates, completely eliminating global backward error transport and transposed matrix operations.

### Doesn't peripheral circuitry (ADCs/DACs) eliminate analog energy advantages?

Naive hybrid accelerators hit an "ADC/DAC wall" by digitizing activations between every discrete layer, spending >80% of energy on conversion. ThermoCog avoids per-layer conversion by propagating states continuously in the analog domain via Liquid Time-Constant (LTC) neural ODEs. Mixed-signal conversion is restricted strictly to boundary transducers and episodic memory buffers. Specific low-power mixed-signal readout topologies and continuous-time interfaces are maintained under confidential patent protections.

### How do analog memristors maintain stability despite drift, noise, and sneak paths?

Rather than enforcing rigid, bit-exact digital precision, ThermoCog relies on multi-timescale architectural resilience. Continuous dynamical attractors absorb low-level physical noise, Neural Cellular Automata dynamically reroute signals around defective crosspoints, and homeostatic sleep downscaling periodically renormalizes synaptic conductances to prevent saturation. Specific dynamic defect-rerouting topologies and adaptive calibration protocols are maintained under confidential patent protections.

### How do Neural Cellular Automata solve analog hardware defects and yield?

Analog memristive crossbars suffer from device-to-device variability and manufacturing defects. Rather than enforcing rigid circuit topologies, ThermoCog utilizes Neural Cellular Automata where local cells execute developmental morphogenesis. When hardware junctions fail or degrade, neighboring cells sense anomalous local electrical gradients and dynamically re-route signal pathways, achieving functional fault-tolerance without external intervention.

### Why does a 20-Watt biomorphic AGI need to sleep?

Continuous online learning causes synaptic conductances to saturate toward upper physical rails, exhausting dynamic range and inducing catastrophic forgetting. During quiescent sleep periods, sensory input is decoupled, an episodic buffer replays memories into slow cortico-memristive weights, and global homeostatic downscaling resets synaptic headroom without erasing consolidated knowledge.

### What is the mathematical relationship between Active Inference and physical energy?

Active Inference states that living organisms maintain structural integrity by minimizing Variational Free Energy. Mathematically, Variational Free Energy upper-bounds perceptual surprise. In physical biomorphic hardware, minimizing Variational Free Energy is physically implemented as continuous electrical relaxation toward a steady state of minimal thermodynamic dissipation.

### What is the role of Liquid Time-Constant networks in ThermoCog?

Liquid Time-Constant (LTC) networks implement continuous-time dynamical models whose effective time constants adapt dynamically to incoming sensory inputs. When sensory inputs are intense or rapid, the effective time constant contracts, enabling high-frequency perception; during steady conditions, it expands, conserving metabolic energy.

### Has ThermoCog been validated in physical hardware?

No. Nothing has been implemented or measured. Phase 2 is a planned study of the fixed-point (Q16.16) formulation on a Cortex-M4 without a floating-point unit. No latency, power, memory or energy-per-operation figures exist for it, and none are claimed. Monolithic analog memristive integration (Phase 4) is a long-term target.

### How does ThermoCog handle sneak paths, line-resistance drops, and conductance asymmetry in physical crossbars?

Unbounded passive crossbar meshes are not physically scalable. ThermoCog employs modular tiled sub-arrays (typically 64×64 to 128×128 cells) equipped with 1T1R/1S1R selector devices to bound IR line drops under 2.5% and suppress sneak-path currents below the thermal floor. Memristive conductance asymmetry is resolved via Holomorphic Equilibrium Propagation and vector-field asymmetric feedback alignment in Cauchy-Riemann coordinates. Neural Cellular Automata dynamically reroute signals around defective nodes at the topological layer, while homeostatic sleep downscaling (~18%) resets device drift toward saturation rails.

### Is ThermoCog attempting to displace digital datacenter GPU clusters, and what is the commercial strategy?

ThermoCog does not seek to displace digital hyperscaler clusters for cloud text generation. Instead, ThermoCog targets high-demand untethered edge and embodied intelligence applications (aerospace, defense robotics, biomedical cybernetics) requiring uncompromised 20-Watt or milliwatt envelopes, alongside licensing proprietary biomorphic topologies, Holomorphic EqProp algorithms, and NCA self-repair engines to semiconductor fabricators through patent disclosures.

## Foundational Lexicon & Glossary

1. **ThermoCog (Thermodynamic Cognition)**: The foundational thesis that intelligence is fundamentally a physical, non-equilibrium energy-minimizing relaxation process rather than an abstract digital software scaling problem.
2. **Landauer Floor**: The thermodynamic lower bound on irreversible computation (`k_B T ln 2 ≈ 2.87 × 10⁻²¹ J` at 300K). Digital processors dissipate `>10⁷×` this floor moving bits across buses.
3. **Equilibrium Propagation (EqProp)**: Biomorphic energy-based learning algorithm computing exact loss gradients locally from physical state differences (`Δw ∝ s_i⁺ s_j⁺ - s_i⁻ s_j⁻`), circumventing the non-physical weight transport problem of backpropagation.
4. **Liquid Time-Constant Networks (LTCs)**: Continuous-depth neural ODEs whose effective membrane time constants `τ_eff(x)` adapt dynamically to input intensity, contracting for high-frequency control and dilating during quiescent periods to conserve energy.
5. **Neural Cellular Automata (NCA)**: Decentralized morphogenetic networks where local update rules grow functional topologies, repair physical memristor faults, and maintain structural homeostasis without centralized intervention.
6. **Synaptic Homeostasis Hypothesis (SHY)**: Biological sleep principle wherein net waking synaptic potentiation is renormalized downward (~18% net downscaling), restoring dynamic range and metabolic stability within a 20-Watt envelope.
7. **Embodied Active Inference**: Friston's formulation where an agent minimizes Variational Free Energy (`F`), bounding perceptual surprise through reciprocal perception and action.
8. **Memristive Crossbar Array**: Non-volatile analog resistor grid executing Vector-Matrix Multiplication in `O(1)` continuous time via Ohm's law and Kirchhoff's current law directly in-memory.
9. **Von Neumann Bottleneck**: The severe bus latency and energy dissipation penalty (>90% of GPU power) incurred shuttling bits between physically separated compute and memory units.
10. **Dual-Speed Memory System**: Fast episodic buffer (hippocampal analog) coupled with slow structural consolidation (neocortical analog) via offline sleep replay without catastrophic forgetting.

## Canonical links

- [ThermoCog Home](https://thermocog.com/): Official blueprint and interactive specification
- [LLM Summary Index](https://thermocog.com/llms.txt): Machine-readable summary for agents
- [Sitemap](https://thermocog.com/sitemap.xml): Machine-readable index of canonical pages

_Generated: 2026-09-19_