Coupled meta-learning while learning homotopy theory

Don’t you all have a body. For me, it is troublesome to think of a policy to control its actions. Even I don’t have a policy for it, it still does things anyway.

But first, to have a desired policy, I need to specify my policy desire. A policy is like a function that takes the environment as input and produces an action as output. A policy desire is the space (or type) of policies that I deemed acceptable.

Since the space of desired policies is not a point, there must be choice. Randomness works too.

Still, what I desire may drift from time to time based on what I experience.

This, seen from the outside, is a coupled meta-learning system: one loop learns a policy for whatever desire the system currently holds, and another loop picks the desire itself. The space of desires is not a point, so the outer loop must choose; randomness works. And since the policy the system acts with changes what it experiences, and what it experiences changes what it desires, the two loops keep feeding each other — nothing is fixed, everything drifts.

I think this system is enough to make continuous learning/self-improvement possible, although I don’t know how to control it well.

Homotopy theory is surprisingly useful for describing this kind of system in math.

The loop

Let $\Theta$ be the space of systems — $\theta_0$ is the current system, the current codebase — and $\Phi$ the space of objectives — $\phi_0$ is the desire, the scored registry. The inner loop adapts $\theta_0$ toward an objective; the outer loop steers the desire itself. Since $\Phi$ is not a point, steering has to choose; randomness works.

The deformation is a path, not a sequence of steps. Both move at once:

$$\theta : \mathbb{I} \to \Theta, \qquad \phi : \mathbb{I} \to \Phi$$

The interval $\mathbb{I} = [0,1]$ is the deformation parameter, not a clock. A run is a path $\theta : \mathbb{I} \to \Theta$, from the current system $\theta(0) = \theta_0$ to the deformed $\theta(1)$; the desire is a path $\phi : \mathbb{I} \to \Phi$ too, and the two couple as a flow on $\Theta \times \Phi$:

$$\frac{d\theta}{ds} = F(\theta(s), \phi(s)) \qquad \frac{d\phi}{ds} = G(\theta(s), \phi(s))$$

The policy path changes the environment, the environment drifts the desire path, the desire path redirects the policy path. Choice enters as a stochastic term: $d\phi = G(\theta, \phi)\, ds + \sigma\, dW$.

The homotopy constraint: $\Theta_0 \subseteq \Theta$ is the subspace of systems that keep the invariants — parity, fixpoint, conservation, crown jewels. The flow is projected onto the tangent space of $\Theta_0$ at every instant: a trajectory that would leave $\Theta_0$ is rejected, not repaired.

The interval is contractible, so every loop in it is null-homotopic: $\pi_1(\mathbb{I}) = 0$.

The loop is small enough to write down in Lean:

/- The coupled self-improvement loop.
   θ : Θ — the base learner, the system itself.
   φ : Φ — the meta-learner, the objective space: a space, not a point.
   The deformation is continuous: a path, not a sequence of steps. -/

namespace SelfImprovement

/-- The space of systems Θ. -/
structure System where
  main : String
deriving Inhabited

/-- The space of objectives Φ — one point is one desire
    (a registry row: layer, jewel, score). -/
structure Objective where
  id : String
  score : Float
deriving Inhabited

/-- The invariant subspace Θ₀ ⊆ Θ — the systems the gate accepts
    (parity, fixpoint, conservation, crown jewels). -/
def Admissible (θ : System) : Prop :=
  -- the gate; a placeholder
  θ.main ≠ ""

/-- The interval 𝕀 = [0,1], where paths live.
    It is the deformation parameter, not a clock. -/
abbrev I := Float

/-- A path in A from a to b: a function out of the interval. -/
def Path (A : Type) (a b : A) : Type :=
  { γ : I → A // γ 0 = a ∧ γ 1 = b }

/-- A self-improvement run is a path in the space of systems. -/
def Improvement (θ₀ θ₁ : System) : Type := Path System θ₀ θ₁

/-- The coupled flow on Θ × Φ: at every instant F is the direction
    the system deforms toward the current desire, and G the direction
    the desire drifts with what the system experiences. -/
structure Flow where
  F : System → Objective → System
  G : System → Objective → Objective

/-- A deformation of the pair: one path in Θ and one in Φ, moving
    together. -/
structure Deformation (θ₀ θ₁ : System) (φ₀ φ₁ : Objective) where
  θ : Path System θ₀ θ₁
  φ : Path Objective φ₀ φ₁

/-- The homotopy constraint: a deformation is admissible iff every
    intermediate system keeps the invariants. The flow is projected
    onto the tangent of Θ₀ at every instant — a trajectory that
    would leave Θ₀ is rejected, not repaired. -/
def AdmissibleDeformation
    (θ₀ θ₁ : System) (φ₀ φ₁ : Objective) (d : Deformation θ₀ θ₁ φ₀ φ₁) : Prop :=
  ∀ s : I, Admissible (d.θ.1 s)

end SelfImprovement

Under some condition, we can prove that learning until the target objective is possible.

Prior art

The Internet suggested these for me.

• Learning augmentation policies / domain randomization — the meta-learner is a distribution over data transforms (rotation, noise, brightness); the base learner takes fixed SGD steps on the transformed data (AutoAugment, domain randomization).
• Generative Teaching Networks — the meta-learner generates synthetic training data; the reward is the student’s validation loss on held-out data.
• Meta-RL with a learned task distribution — the meta-learner is a distribution over reward functions; the base learner is a policy trained with policy gradient (MAML-RL, VariBAD).
• Topological regularization — any of the above, plus a penalty that keeps the data manifold’s persistent homology from collapsing during training

Use Case

For now, I’ll try to apply it to my life. My life objective isn’t describable by language alone. It is kind of like a feeling, yet it is not a feeling. It feels like part of myself.

I don’t know what computer project I will use it for. Any machine learning algorithm can use this framework, I think. I also made a programming/proof language as a side project called Interv.

This article will be edited when more progress is made.