FORMULAS.md

# Geometric Formula Catalog
## Token Topology & Loss System — AbstractPhil + Claude

*ROSE loss discarded. These are the active formulas.*

---

## 1. Multi-Scale Crystal Loss

Classification through learnable crystal prototypes at multiple projection dimensions. Each class has a crystal centroid at each scale. No softmax — geometric distance IS the classifier.

**Scales:** `[64, 128, 256, 512, 1024]` (each is a projection dimension, not spatial)

### 1.1 Per-Scale Crystal Similarity

```
sim(x, c_k) = (x̂ · ĉ_k) / τ

where:
  x̂ = normalize(proj_k(features))     # [B, scale_dim]
  ĉ_k = normalize(crystals_k)          # [num_classes, scale_dim]
  τ = temperature (default 0.07)
```

### 1.2 Per-Scale Coherence Loss

Pull features toward their correct class crystal:

```
L_coherence = -mean(log(exp(sim(x, c_y)) / Σ_j exp(sim(x, c_j))))

where y = true class label
```

### 1.3 Per-Scale Separation Loss

Push class crystals apart with margin:

```
L_separation = Σ_{i≠j} max(0, margin - ||ĉ_i - ĉ_j||₂)² / (C(C-1))

where C = num_classes, margin = 1.0
```

### 1.4 Per-Scale Discretization Loss (Cantor Targets)

Cluster crystal Cantor values toward `{0.0, 0.5, 1.0}`:

```
L_discretization = mean(min_t(||cantor(c_i) - t||²))

where t ∈ {0.0, 0.5, 1.0}
```

### 1.5 Per-Scale Crystal Geometry Loss

Maintain target distance from features to class prototypes:

```
L_geometry = mean((||x - c_y||₂ - d_target)²)

where d_target = 1.0
```

### 1.6 Total Multi-Scale Crystal Loss

```
L_crystal = (1/S) Σ_{k=1}^{S} w_k · (
    w_coh · L_coherence_k +
    w_sep · L_separation_k +
    w_disc · L_discretization_k +
    w_geom · L_geometry_k
)

Proven weights: w_coh=1.0, w_sep=0.5, w_disc=1.0, w_geom=0.5
```

### 1.7 Crystal Prediction (No Softmax Head)

```
logits = Σ_k w_k · (α · cos_sim_k + β · cantor_coherence_k + γ · crystal_geometry_k)

where prediction = argmax(logits)
```

**Results:** 86% ImageNet (CLIP bigG features), 74.87% CIFAR-100 (393K params), ~92% CIFAR-100 (78KB model)

---

## 2. Geometric Basin Compatibility Loss

Classification through geometric formula satisfaction. Four structural checks produce compatibility scores ∈ [0,1]. No cross-entropy needed.

### 2.1 Triadic Compatibility

```
T(x, c) = exp(-||proj(x) - c||₂² / (2σ²))

where c = class centroid, σ = learned bandwidth
```

### 2.2 Self-Similarity Check

```
S(x) = exp(-Var(cantor_levels(x)))

where cantor_levels extracts per-level Cantor measures
High self-similarity → low variance across levels → high score
```

### 2.3 Cantor Coherence Check

```
C(x, p_y) = exp(-||cantor(x) - p_y||₂²)

where p_y = class Cantor prototype
```

### 2.4 Hierarchical Check

```
H(x) = Σ_{k=1}^{L} 0.5^k · match(level_k(x), expected_k)
```

### 2.5 Combined Compatibility Score

```
compat(x, class_j) = T(x, c_j) · S(x) · C(x, p_j) · H(x)

Product of four factors ∈ [0,1] → output ∈ [0,1]
```

### 2.6 Basin Loss (Three-Term, No Cross-Entropy)

```
L_correct = -mean(log(compat(x, y) + ε))
L_incorrect = -mean(log(1 - compat(x, j≠y) + ε))
L_contrastive = NLL(log_softmax(compat / τ), y)

L_basin = L_correct + 0.5 · L_incorrect + 0.5 · L_contrastive
```

**Results:** 67.69% CIFAR-100 with NO attention, NO cross-entropy, NO transformers (geo-beatrix). Beat ViT-beatrix (66.0%).

---

## 3. K-Simplex Channel Formulas

Tokens represented as k-simplices with Cayley-Menger validated geometry. Shape `[B, T, K+1, F]` where K+1 = vertices.

### 3.1 Template + Deformation

```
v_i = v_i^{template} + α · Δv_i

where:
  v_i^{template} = regular k-simplex vertices (frozen)
  α = deformation scale (0.05 base, per-k scaled)
  Δv_i = learned offset from neural network
```

### 3.2 K-Scaled Deformation

Volume scales as `edge^k`, so higher k needs smaller deformation:

```
α_k = α_base / √(k + 1)

k=1: α × 0.71    k=3: α × 0.50
k=2: α × 0.58    k=4: α × 0.45
```

### 3.3 Per-Token Simplex Coordinates

```
coords = proj(token_embedding)        # [B, T, edim]
vertex_weights = softmax(route(token_embedding))  # [B, T, K+1]
simplex_state = vertex_weights @ vertices          # [B, T, edim]
```

### 3.4 K-Simplex Attention (Proven Superior to K-Simplex Classification)

```
For each token pair (i, j):
  d²_ij = ||simplex_i - simplex_j||²   # pairwise simplex distance
  attn_ij = softmax(-d²_ij / τ)        # geometric attention weights

Output = attn @ V                       # standard value projection
```

**Results:** 89.13% FMNIST, 84.59% CIFAR-10, 69.08% CIFAR-100 as attention. Entropy decreases through layers (sharpening). Fewer tokens = sharper attention (25 patches > 64 patches).

---

## 4. Cayley-Menger Formulas

The structural invariant. If CM fails, geometry is invalid. Non-negotiable.

### 4.1 Cayley-Menger Matrix

```
CM = | 0    1      1      ...  1      |
     | 1    0      d₀₁²   ...  d₀ₖ²  |
     | 1    d₀₁²   0      ...  d₁ₖ²  |
     | ⋮    ⋮      ⋮      ⋱    ⋮      |
     | 1    d₀ₖ²   d₁ₖ²   ...  0      |

Size: (K+2) × (K+2) for a K-simplex
```

### 4.2 Volume Formula (Corrected)

```
Vol² = (-1)^(K+1) / (2^K · (K!)²) · det(CM)

Validity: Vol² > 0 indicates non-degenerate simplex
```

### 4.3 Gram Determinant Alternative (More Stable)

```
X_translated = X[:, 1:, :] - X[:, 0:1, :]    # [B, K, D]
G = X_translated @ X_translated.T              # [B, K, K]
Vol = √(det(G)) / K!
```

### 4.4 Validity Loss

```
L_validity = mean(ReLU(-Vol²))

Penalizes collapsed simplices (Vol² < 0)
```

### 4.5 Volume Consistency Loss

```
L_vol_consistency = Var(Vol²) across batch

Encourages uniform geometric structure
```

### 4.6 Hierarchical Cell Loss (k=4 pentachoron)

```
5 cells (tetrahedra), each with 4 vertices, 6 edges:

L_cell = mean(ReLU(ε - Vol²_cell_i))

for i = 1..5 cells of the pentachoron
```

### 4.7 Vol² Scaling Reference

```
k=1: Vol² ~ 1e+0    (edge length squared)
k=2: Vol² ~ 1e-1    (triangle area squared)
k=3: Vol² ~ 1e-2    (tetrahedron volume squared)
k=4: Vol² ~ 1e-3    (5-cell hypervolume squared)
```

---

## 5. Cantor Lens Formulas

The Devil's Staircase as a hierarchical lens for viewing token relationships.

### 5.1 Devil's Staircase (Beatrix Staircase)

```
C(x) = Σ_{k=1}^{levels} bit_k × 0.5^k

where:
  y_k = x × 3^k                              # scale to level k
  p = softmax(-d²/τ)  over centers [0.5, 1.5, 2.5]
  bit_k = p_right + α × p_middle             # soft ternary assignment
  α = learnable middle-third fill (default 0.5)
  τ = softmax temperature (default 0.25)
```

### 5.2 Branch Path Extraction

```
branch_path(x) = [argmax(p_1), argmax(p_2), ..., argmax(p_L)]

Each level: L (left third), M (middle third), R (right third)
```

### 5.3 Hierarchical Alignment (NOT Distance)

**CRITICAL: Distance is meaningless on Cantor set.**

```
alignment(i, j) = Σ_{k=1}^{L} 0.5^k · 𝟙(path_i[k] == path_j[k])

Level weights: [0.5, 0.25, 0.125, 0.0625, 0.03125]
```

Coarse matches = routing highways (wormholes).
Fine matches = local structure only.

### 5.4 Euclidean Bridge (Lossy but Necessary)

```
distance(i, j) = |C(x_i) - C(x_j)|

Use ONLY when interfacing with Euclidean systems (optimizers, standard losses).
Alignment is the Cantor-native metric.
```

### 5.5 Cantor Routing Bias (for Attention)

```
bias[i,j] = alignment(i, j)    # precomputed [S, S] matrix

attn_scores = (Q @ K.T / √d) + λ · bias

where λ = learnable routing weight
```

### 5.6 Alpha Modulation

```
α → 0.0: Pure ternary (Cantor dust, maximally disconnected)
α → 0.5: Triadic equilibrium (proven stable zone: 0.44-0.50)
α → 1.0: Filled (continuous, no fractal structure)
```

---

## 6. Cantor Topological Ropes

Position encodings that encode structural hierarchy, not just sequence order.

### 6.1 Standard RoPE (Baseline)

```
θ_i = 10000^(-2i/d)
R(m) = [cos(mθ_i), -sin(mθ_i); sin(mθ_i), cos(mθ_i)]

for dimension pair (2i, 2i+1) at position m
```

### 6.2 BeatrixRoPE (Devil's Staircase Warping)

```
pos_beatrix(m) = C(m / seq_len)      # Cantor function of normalized position

R_beatrix(m) = R(pos_beatrix(m) × seq_len)
```

Tokens in same ternary branch get **similar** positions → attend easily.
Creates hierarchical plateaus.

### 6.3 CantorRoPE (Wormhole Shortcuts)

```
pos_cantor(m) = trend × m + deviation × wormhole(m)

where:
  trend = 1.0 (aligns macro slope with standard RoPE)
  deviation = learnable perturbation scale
  wormhole(m) = branch_path_alignment signal
```

Tokens with aligned branch paths can shortcut regardless of sequential distance.

### 6.4 Aligned Triad (Proven Configuration)

```
Standard:  linear baseline         "this comes after that"
Beatrix:   hierarchical plateaus   "these belong together"
Cantor:    wormhole perturbations  "these can shortcut"

All share same macro slope (trend=1.0), different micro structure.
```

### 6.5 Tower Assignment

```
Tower_positive = BeatrixRoPE(...)    # hierarchical reasoning
Tower_negative = CantorRoPE(...)     # wormhole reasoning

Signed pairs create differential forces in oscillator fusion.
```

---

## 7. Beatrix Oscillation Formulas (GeoFractal Router)

Physics-based fusion replacing static weighted sums. Tower outputs are force fields, not opinions to average.

### 7.1 Covariant Dynamics

```
dx/dt = v
dv/dt = -2β(t)·v - ω²·Log_x(x_ref) + κ(t)·u_towers + γ(t)·ξ_guide

where:
  x = position on manifold
  v = velocity in tangent space
  β(t) = damping schedule
  ω = spring frequency
  x_ref = conditioning anchor
  κ(t) = tower coupling strength
  u_towers = force from tower opinions
  γ(t) = guidance strength
  ξ_guide = external guidance (DINO, text, etc.)
```

### 7.2 Manifold Operations

```
Log_x(y) = y - x                    # tangent vector from x toward y
Exp_x(v) = x + v                    # move along tangent vector
PT_{x→y}(v) = v                     # parallel transport (flat approx)
```

### 7.3 Tower Force Generation

```
For N towers with signed pairs:
  force_i = proj_i(tower_output_i)   # [B, manifold_dim]
  u_towers = Σ_i w_i · force_i       # weighted combination

Positive towers push toward structure.
Negative towers push away from collapse.
```

### 7.4 Tesla 3-6-9 Schedule

```
β(t) = β_base + resonance(t)

resonance(t) = 0.1·sin(3πt) + 0.05·sin(6πt) + 0.025·sin(9πt)

Resonant peaks at t = 1/3, 2/3, 1.0
Energy doesn't flow linearly — it oscillates.
```

### 7.5 Schedule Types

| Schedule | Formula |
|----------|---------|
| Constant | `s(t) = start` |
| Linear | `s(t) = start + (end - start) · t` |
| Cosine | `s(t) = end + (start - end) · 0.5(1 + cos(πt))` |
| Sigmoid | `s(t) = start + (end - start) · σ(12(t - 0.5))` |
| Tesla 3-6-9 | `s(t) = linear(t) + resonance(t)` |

### 7.6 Intrinsic Tension τ

```
τ = σ(gain · (Σ_i w_i · invariant_i - equilibrium))

where:
  invariant_i = geometric invariants (Vol², edge stats, etc.)
  w_i = learned per-invariant weights
  gain = steepness of sigmoid response
  equilibrium = learned bias

τ → 0: Pure spring (geometric constraint dominates)
τ → 1: Pure control (tower forces dominate)
```

### 7.7 Stability Criterion

```
Eigenvalues of linearized system:
  λ = -β ± √(β² - (1-τ)ω²)

Overdamped:   β² > (1-τ)ω²    (stable, no oscillation)
Underdamped:  β² < (1-τ)ω²    (oscillatory)
Critical:     β² = (1-τ)ω²    (fastest convergence)
```

### 7.8 Energy Tracking

```
E_kinetic = 0.5 · ||v||²
E_potential = 0.5 · ω² · ||Log_x(x_ref)||²
E_total = E_kinetic + E_potential

Healthy training: E_total decreases over integration steps.
```

---

## 8. K-Simplex Linear (Near-Zero Params)

Replaces `nn.Linear` with geometric routing through simplex structure.

### 8.1 Architecture

```
Input (B, input_dim)
  → chunk into (B, num_simplices, K+1) groups
  → per-scalar entry into vertex (K+1 options)
  → private hidden projection per vertex (depth = K+1)
  → pairwise signal passages between all vertex pairs
  → attenuation gates on pairwise influence
  → exit: weighted sum of vertex states
Output (B, output_dim)
```

### 8.2 Parameter Count

```
Per simplex (K+1 inputs):
  Entry:      (K+1) × (K+1) × hidden
  Vertex:     (K+1) × hidden
  Pairwise:   C(K+1, 2) × 3 × hidden
  Attenuate:  C(K+1, 2) × 2
  Exit:       (K+1) × hidden + (K+1)

For K=4, input_dim=512:
  103 simplices × 300 params = 30,900
  vs nn.Linear: 262,656
  Ratio: 0.118x (11.8% of linear params)
```

### 8.3 Structural Comparison

```
Structure size per simplex: (K+1) × (K+1) × C(K+1,2)

K=2: 3×3×3   = 27
K=4: 5×5×10  = 250
K=6: 7×7×21  = 1029
```

### 8.4 Results

```
Fashion-MNIST:
  KSimplex-k4: 85.94% with 8,511 params
  MLP baseline: 89.00% with 101,770 params
  Ratio: 11.5× more parameter-efficient
  
  Epoch 1: 84.28% test (instant useful signal)
  Epoch 19: 85.94% test (stable convergence)
```

---

## 9. K-Simplex Deformation Limitations

Critical stability boundaries from extensive geometric explorer experiments.

### 9.1 Stability Zones by Configuration

| Configuration | Differentiation Zone | Collapse Threshold |
|---------------|---------------------|-------------------|
| k=1-4, edim=16 | 0.15 - 0.35 | ~0.50 |
| k=1-4, edim=32 | 0.15 - 0.50 | >2.0 |
| k=1-6, edim=16 | 0.35 - 0.45 | ~0.50 |
| k=1-6, edim=32 | 0.25 - 0.60 | >2.0 |

### 9.2 Embedding Dimension Safety Ratio

```
stability_ratio = edim / k_max

ratio ≥ 8×  → Very stable, deform up to 2.0
ratio ≥ 4×  → Comfortable margin
ratio ≥ 2×  → Tight but functional
ratio < 2×  → Dangerous, frequent invalidity
```

### 9.3 Deformation Behavior

```
Low deform (0 - 0.15):
  Clear k-level hierarchy
  Vol² decreases exponentially with k
  Conservative but safe

Medium deform (0.15 - 0.35):  ← OPTIMAL ZONE
  Distinct geometric signatures per k
  Maximum useful differentiation
  Training should target this range

High deform (> 0.5):
  Noise dominates
  k-levels converge (lose meaning)
  Geometric structure destroyed
```

### 9.4 Late-Stage K-Simplex Invalidity

```
As k increases:
  - CM determinant computation becomes numerically unstable
  - More edge configurations become geometrically impossible
  - Deeper layers produce invalid simplex configurations

k=4 in 32D: stable with wide margin
k=5 in 32D: functional but tighter
k=6 in 32D: approaching invalidity ceiling

Recommendation: k=4 (pentachoron) as primary, k≤3 for tight budgets
```

### 9.5 Cross-Entropy Degeneracy Problem

```
Cross-entropy applied directly to simplex features:
  → Vertices converge (minimizing distance to class boundary)
  → Volume → 0 (simplex collapses)
  → α diverges from triadic equilibrium
  → Geometric structure destroyed after sufficient epochs

Solution: Use crystal loss or basin loss, NOT cross-entropy on geometric features.
```

---

## 10. Cross-Contrast Capacity Tests

Validating that geometric structure survives training and provides meaningful classification signal.

### 10.1 Geometric Cross-Contrastive Loss

```
sim_matrix = (x̂ @ x̂.T) / τ                    # [B, B] embedding similarity

cantor_positives = (|C(i) - C(j)| < θ_cantor) AND (|Vol(i) - Vol(j)| < θ_vol)

L_cross = -log(Σ_j∈positives exp(sim_ij) / Σ_j∈all exp(sim_ij))

where positives are defined by geometric proximity, not class labels
```

### 10.2 Capacity Invariants to Monitor

```
1. Vol² > 0 for all simplices (validity)
2. α ∈ [0.44, 0.50] (triadic equilibrium)
3. Edge length variance < threshold (structural uniformity)
4. Cantor prototype separation > margin (class distinctness)
5. Crystal distance to prototype ~ d_target (geometric alignment)
```

### 10.3 Differential Cross-Contrast (Tower Pairs)

```
For positive/negative tower pairs:
  Δ_force = force_positive - force_negative

  L_differential = -log(σ(Δ_force · direction_to_correct_class))
                   + log(σ(Δ_force · direction_to_incorrect_class))

Signed pairs create differential forces, not just different opinions.
```

### 10.4 Cross-Scale Consistency

```
For scales s₁, s₂:
  features_s1 = proj_s1(backbone_features)
  features_s2 = proj_s2(backbone_features)
  
  L_consistency = ||rank_order(sim_s1) - rank_order(sim_s2)||₂

Ensures geometric relationships are preserved across crystal scales.
```

### 10.5 OOD Detection via Geometric Violation

```
In-distribution:  Vol² > 0, α stable, Cantor coherent
Out-of-distribution: Violations of above

OOD_score = (1 - σ(Vol² · 10⁶)) + (|α - 0.5|) + (1 - compat_max)
```

### 10.6 Scaling Limitation (Known)

```
Cross-contrastive loss across full vocabulary:
  O(V²) pairwise comparisons
  
  V=100 (CIFAR-100): 10K pairs → feasible
  V=1000 (ImageNet): 1M pairs → expensive
  V=50000 (tokenizer): 2.5B pairs → infeasible
  
Solution: Hierarchical contrastive within Cantor branches.
Only contrast within same coarse branch (routing highways).
Fine branches → local contrast only.
```

---

## Appendix A: Proven Results Summary

| Model | Task | Accuracy | Params | Key Innovation |
|-------|------|----------|--------|----------------|
| David | ImageNet (CLIP bigG) | 86% | ~120K | Multi-scale crystal |
| David | CIFAR-100 | 74.87% | 393K | Crystal prototypes |
| David | CIFAR-100 | ~92% | 78KB | Extreme compression |
| geo-beatrix | CIFAR-100 | 67.69% | — | NO attention, NO CE |
| KSimplex Attention | FMNIST | 89.13% | — | Geometric attention |
| KSimplex Attention | CIFAR-10 | 84.59% | — | Conv stem + geo attn |
| KSimplex Attention | CIFAR-100 | 69.08% | — | Multi-layer sharpening |
| KSimplex Linear | FMNIST | 85.94% | 8,511 | 11.5× efficiency |
| KSimplex LLM | Shakespeare | PPL 113 | 54M | 100% geo validity |
| Beeper v5 | Ethics | Coherent | Random | Architecture IS intelligence |

## Appendix B: Formula Dependencies

```
                    ┌─────────────┐
                    │ Cayley-Menger│ ← structural invariant
                    └──────┬──────┘
                           │
              ┌────────────┼────────────┐
              ▼            ▼            ▼
        ┌──────────┐ ┌──────────┐ ┌──────────┐
        │ K-Simplex│ │ Crystal  │ │ Basin    │
        │ Channel  │ │ Loss     │ │ Compat   │
        └────┬─────┘ └────┬─────┘ └────┬─────┘
             │            │            │
             ▼            ▼            ▼
        ┌──────────────────────────────────┐
        │          Cantor Lens             │
        │  (Staircase + Alignment + Bias)  │
        └──────────────┬───────────────────┘
                       │
              ┌────────┼────────┐
              ▼        ▼        ▼
        ┌─────────┐ ┌──────┐ ┌──────────┐
        │ Topo    │ │ Osc  │ │ KSimplex │
        │ Ropes   │ │ Fuse │ │ Linear   │
        └─────────┘ └──────┘ └──────────┘
```

## Appendix C: What Kills Geometry (Known Failure Modes)

1. **Cross-entropy on geometric features** → simplex collapse
2. **Distance on Cantor set** → meaningless (use alignment)
3. **Deformation > 0.35 at edim/k < 4** → invalidity
4. **k > 4 without edim ≥ 8k** → numerical instability
5. **Uniform Cantor level weights** → hides 8× routing significance difference
6. **Resizing crystal anchors across scales** → destroys pentachoron geometry (use separate init per scale)
7. **Dropout scaling with √dim** → inconsistent information flow across scales