Skills

A Skill is a self-contained, restartable behavior that the Controller can launch in response to a high-level decision. A skill is "open the gripper", "track the red cup", "run a diffusion policy", or "navigate to waypoint X". Each runs at its own rate, reads the observations and events its host wires up for it, and returns actions.

Defined in solarian/core/ai/system1/skill.py.

Anatomy

A Skill is an abstract class — not a Process. How it is scheduled is a separate property (see ExecutionPlace and ExecutionMode). It owns:

• A reference to the shared observation ring buffers (read-only).
• A per-module action queue for each module it commands — the one channel out.
• A handle to the skill status buffer for reporting lifecycle transitions.
• A frequency (Hz) and a params dict (its construction arguments, and what the think process forwards).

from solarian.core.skill import Skill, SkillStatus

class PickSkill(Skill):
    def setup(self):
        self.target_position = self.params.get("target", [0.0] * 6)

    def control_step(self, dt):
        pose = self.observations[self.params["arm"]].latest()
        # ... compute the next target from pose ...
        return Action(
            action_type=ActionType.MODULE_ACTION,
            data={
                self.params["arm"]: {
                    "action_type": ActionType.MOVE_L,
                    "data": self.target_position,
                }
            },
        )

Required and optional methods

What a step reads

Whichever host starts a skill — a Robot or a Controller — points these at its own buffers and gives the skill an event inbox. step() then refreshes them before each control_step, so they mean the same thing whether the skill runs in its own process, on a thread, or inline on the host's loop. A skill never branches on how it happens to be scheduled.

A buffer holds recent history, not just the current sample, which is what a policy conditioning on the last few frames needs:

frames = self.observations["wrist_camera"].last_k(16)   # oldest first, clamped
pose = self.observations["arm"].latest()                # None on an empty buffer

last_k never raises for asking too much: it clamps to what the buffer actually holds. Read a module once into a local rather than twice through the buffer — for a process-backed module the two reads can straddle an incoming message.

A step reads its own attributes and its own config, never its host. That is what lets the same skill run on the robot, on the controller, or in a test that just assigns to skill.observations and calls step().

Returning actions

A step returns intent; the host decides where it goes. Return a single Action, a list (a chunk of future waypoints), or None to do nothing this tick.

A skill builds the envelope itself, keyed by module name, and one action may carry several modules — dispatch splits it per module anyway, so that is one fewer thing for the host to merge:

def control_step(self, dt):
    return Action(
        action_type=ActionType.MODULE_ACTION,
        data={
            self.params["arm"]: {
                "action_type": ActionType.SERVO_L,
                "data": pose_fields,
            },
            self.params["gripper"]: {
                "action_type": ActionType.SERVO_J,
                "data": grip_fields,
            },
        },
    )

Names, not Module objects: construct with PickSkill(params={"arm": "right_arm", ...}). Occupancy is those writer keys (arm, gripper). That is what lets one skill class run on either host: a name crosses a link, a Module cannot.

Whichever host runs it, the returned MODULE_ACTION is split per module and pushed onto the skill's own action_queues — one queue per module it commands. On the robot the module at the other end applies it directly. On the controller the control loop drains those queues, merges what every running skill commanded, and sends one message to the robot.

The queues are shared memory when the skill runs in its own process and a plain in-process ring when it does not, the same way observation buffers and event inboxes are chosen. A skill never sees the difference.

Merging is index-wise per queue, so both properties hold at once: actions for different modules sit at index 0 of their own queues and so leave together in one message, while a chunk of successive waypoints for one module keeps every one of them — element 0 goes out this tick and the rest follow on the ticks after it.

Reporting status

Skills publish their lifecycle state through a shared-memory SkillStatusBuffer so the think process can read recent transitions and feed them into its next prompt:

from solarian.core.skill import SkillStatus

self.put_skill_status(SkillStatus.RUNNING)
# ... do work ...
self.put_skill_status(SkillStatus.COMPLETED)

The framework already publishes RUNNING before the loop and COMPLETED/STOPPED/FAILED after. You only need to call put_skill_status to emit intermediate states (e.g. progress markers).

The SkillStatus enum:

Where the skill runs: ExecutionPlace and ExecutionMode

Where a skill lives and how it is scheduled are two independent axes.

ExecutionPlace says which runtime owns the skill:

ExecutionMode says how it is scheduled where it lives — PROCESS (its own OS process), THREAD (a daemon thread on the host), or INLINE (ticked by the host's own control loop). It is the same axis modules use; see Execution Modes. Inline is the right choice for cheap, latency-sensitive skills like teleop, which want to run on the controller's tick rather than pay for a process.

Both are class attributes on the skill, because they are properties of the behaviour rather than of the call site:

from solarian.core.execution import ExecutionMode, ExecutionPlace

class TeleopSkill(Skill):
    execution_mode = ExecutionMode.INLINE

class UMISkill(Skill):
    execution_place = ExecutionPlace.ROBOT

A caller can still override either per instance — MoveSkill(params={"pose": "home", "arm": "right_arm"}, execution_mode=ExecutionMode.INLINE) — for the case where the same behaviour is worth scheduling differently in a test or on a small host.

A config-driven skill such as UMISkill also reads execution_place from its yaml, so a deployment key can move it without code changes:

# conf/skills/umi_target.yaml
execution_place: ${deployment.umi_target_execution_place}

That is how to compare the two placements for one policy: run the same task with a deployment that sets server and one that sets robot. A skill that reads a calculated field must run where that field is computed or downstream of it. The relay sees robot-computed fields, but the robot never sees relay-computed ones.

How the think process selects skills

ThinkProcess.decide() calls a VLM with the current observation, recent skill statuses, and the user message. The VLM returns JSON like:

{
  "skills_to_start": [
    {"skill_name": "pick", "params": {"target": "red_cup"}}
  ],
  "skills_to_stop": ["explore"],
  "reasoning": "Cup detected; switching from exploration to pick."
}

For each entry in skills_to_start:

The think process puts a START_SKILL action carrying {"skill": name, "params": {...}} on the controller's skill_command_queue. The controller looks the name up in its registry, constructs the class with those params, and hands the instance to start_skill() — which then decides where it runs. A CONTROLLER skill is launched locally; a ROBOT/SERVER one is forwarded as a START_SKILL action with the same shape, and the robot's generic handler constructs it there. The channel decides the direction, not the action type.

For each entry in skills_to_stop, request_stop() is called on the active skill, or a STOP_SKILL action naming it is sent for a remote one.

Stopping yourself

A skill can self-terminate by:

• Calling self.finish() — the host stops it after the current step and the final status becomes COMPLETED.
• Calling self.request_stop() from another thread / signal — the run loop exits at the next iteration with status STOPPED.

Completion is state rather than a return value, so the step that finishes can still return an action. A one-shot move commands the pose and completes in the same call:

class TimedSkill(Skill):
    def setup(self):
        self.t0 = time.monotonic()

    def control_step(self, dt):
        if time.monotonic() - self.t0 > 5.0:
            self.finish()          # done after 5 seconds
            return self.final_pose_action()
        # ... do work ...
        return None

Discovering skills

A controller's registry has two sources, keyed by Skill.registry_name() — the skill's config_name if it has one, otherwise its class name. That one string is the skill's identity everywhere a name has to travel: the menu the VLM picks from, the START_SKILL action the controller sends the robot, and the robot's lookup of what to construct.

1. The skills class attribute. A tuple of skill classes the controller can run. Classes rather than configured instances, so the VLM names a class and supplies its arguments as params — MoveSkill with {"pose": "home", "arm": "right_arm"} rather than a pre-built home entry.
2. Installed plugins. Any package that declares a solarian.skills entry point is discovered automatically (see registry.py). A declared class wins over a discovered one of the same name.

A robot has the same two sources for what a START_SKILL name may resolve to, and the same skills attribute for skills that live in the project rather than in a plugin.

Configuration

Skills resolve their config by deep-merging two layers, with the project layer winning:

1. Bundled defaults — a conf.yaml shipped next to the skill module. A skill living in a project rather than a plugin usually skips this and keeps its defaults as class attributes, since sibling skills in one package cannot each own a file of that name.
2. Project override — conf/skills/<config_name>.yaml in your project, where config_name is a class attribute on the skill (for a plugin, its solarian.skills entry-point name, e.g. move).

At launch, individual params keys override both. This keeps project-specific values — most importantly the policy checkpoint path — in your project's conf/ rather than baked into the installed plugin. A project only sets what it needs and inherits the rest:

# <your-project>/conf/skills/move.yaml — overrides plugin defaults
frequency: 10

UMISkill is the exception: the yaml you pass as params["config"] is the whole config, not a thin override. solarian install solarian-umi-skill copies templates/skills/umi.yaml to conf/skills/umi.yaml; copy that file per policy.

# <your-project>/conf/skills/cup_arrange.yaml
policy_server_url: ${deployment.policy_server_url}
policy_server_token: ${oc.select:deployment.solarian_base_endpoint_token,null}
server:
  service: diffusion-policy
  managed: false
  checkpoint: ${deployment.policy_checkpoint}
  host: 0.0.0.0
  port: 8001

UMISkill(params={"config": "cup_arrange.yaml"})

The server: block is what solarian run-service cup_arrange runs, so the server and the skill read one checkpoint and one port. A conf whose service takes a checkpoint but names none (in the file, and none passed with --checkpoint) errors clearly. See Services for the block, and Solarian Base to serve it on a cloud GPU.

A checkpoint trained with targets (a box, mask or point marking something on the wrist image) reads each from a calculated field of the camera, named in targets. One UMISkill picks or places depending on the yaml and the checkpoint: setup checks the mapping covers exactly the checkpoint's targets and that each field has the matching data class (BoundingBoxData, MaskData or PointData). A skill with targets must set execution_place, from the same deployment key as the fields' trackers.

# <your-project>/conf/skills/place_object.yaml
execution_place: ${deployment.policy_execution_place}
targets:
  point: target_point     # a PointData field fed by solarian-tapnext's point tracker

Nothing about this is specific to policies. Any value an operator wants to tune per robot belongs here rather than on the controller, which is how CartesianTeleopSkill gets its jog rates:

# <your-project>/conf/skills/cartesian_teleop.yaml
pos_step: 0.004      # metres per tick a held key jogs the pose
rot_step: 0.012      # radians
gripper_max_mm: 85.0

class CartesianTeleopSkill(Skill):
    config_name = "cartesian_teleop"
    POS_STEP = 0.004      # the default, when the YAML omits it

    def setup(self):
        cfg = self.load_config()
        self.pos_step = float(cfg.get("pos_step", self.POS_STEP))

Skills as plugins

A skill plugin is an installable solarian-skill-* package — the unit a third party can publish and a user can install. It mirrors the module/actuator plugin pattern and builds on three public classes:

• Skill — the behavior (runs in its own process), as above, from solarian.core.ai.system1.
• PolicyClient — a generic HTTP client used by a skill to call a policy, from solarian.services. It is policy-agnostic: the same client works for diffusion, ACT, VLAs, or classical methods (waypoints, SLAM).
• PolicyServer — a service server with the standard policy API, from solarian.services. Subclass it and implement load, predict, model_info and shape_meta; the base hides FastAPI/uvicorn and gives you serve(), bearer auth and a main() CLI launcher.

A skill that uses a policy names the service serving it with the policy_service class attribute (e.g. "diffusion-policy") and talks to the server through PolicyClient. The server is a separate process whose location is independent of the skill: it can run on the robot, the controller, a dedicated GPU box, or Solarian Base. Point the skill at it via a policy_server_url, or set server.managed: true to have the skill launch a co-located server itself.

A plugin advertises its skill via an entry point in pyproject.toml:

[project.entry-points."solarian.skills"]
my_skill = "my_skill_pkg.skill:MySkill"

A skill only calls a policy over HTTP, so it does not host one itself. The policy (training + serving) lives in a separate policy plugin (see Training), which ships the PolicyServer the skill connects to via policy_server_url.

Example plugin

plugins/solarian-umi-skill is a complete reference skill plugin: it ships UMISkill, auto-discovered through its solarian.skills entry point (umi). Copy conf/skills/umi.yaml per policy and start the skill with that filename, then run its policy server (the diffusion-policy service of the solarian-policy-diffusion policy plugin):

# the checkpoint and port from conf/skills/cup_arrange.yaml
solarian run-service cup_arrange
# or any checkpoint, with the service directly:
solarian run-service diffusion-policy --checkpoint <checkpoint.ckpt> --port 8001

See also

• AI Controller — the orchestrator that spawns and coordinates skills.
• Action types — used by remote (ROBOT/SERVER) skills.
• Modules — the per-module action queues skills push into.
• Observations — the shared buffers skills read from.