Geometry that exposes gradients.
From WorldString to ∂Oθ: structured object models for reconstruction and action optimization.
WorldString: one model, one object
WorldString conditions learnable object embeddings on an ordered set of structural keypoints K. A State Transformer encodes the configuration, an Object Transformer integrates the features, and a Voxel Transformer predicts occupancy at any spatial query x.
The keypoints describe the current configuration of one object, while each query asks whether a location lies inside its surface. Training uses occupancy supervision across multiple configurations of that same object. Changing the keypoints therefore changes the geometry queried from the learned model, giving sparse structural controls over a continuous representation.
∂Oθ: shared structure, changing configuration
A shared neural object separates persistent structure z from current keypoints K and global position c. PCA identifies coordinated motion in observed keypoints; the complementary features encode the object's persistent structure. Shared query features support occupancy and signed distance, with surface normals obtained from the SDF gradient.
n(x) = ∇xd(x) / max(‖∇xd(x)‖, ε)
Frozen geometry, optimized actions
The hand and target object provide differentiable distance and normal queries. Contact, force balance, torque balance and penetration objectives guide the hand configuration. Model weights remain fixed while gradients update joint angles and hand pose. The optimized trajectories provide state–action demonstrations for policy learning.
∇qℒ = (∂KH / ∂q)T ∇KHℒ
