Fresh observation → model proposal → execution → new feedback
From intent to action.
Back with evidence.
Building the execution foundation between agents, robot skills and the physical world. So every action can inform what comes next.
Observe. Decide. Move.
ROBOT AGENT · HARNESS · NAV2The next decision follows the world.
Fresh observations guide three actual model proposals under the shared Agent coordinator: visit A, visit B, then assess completion. Robot Agent chooses; Harness guards execution; Nav2 moves the robot.
What this run establishes
A releases before B starts. B's result is accepted while settlement remains pending. Model waits are retained. This single Gazebo run demonstrates the loop; it does not establish general reliability or physical-robot safety.
A waiting model can be withdrawn.
REAL MODEL · SHARED COORDINATIONOriginal B cancel/query → Session close → local model process exit
Decision waits do not own execution.
After the visits, an external observer withdraws the identified business task while its real final proposal is still pending. The original B request is cancelled and queried; Session close proceeds without accepting a final answer or submitting more work.
Scope of this run
B remains pending/revoked; native and remote cleanup remain unknown, including observed native ROS shutdown-context errors. Local processes and the container exit. This is task withdrawal after the visits, not an in-motion stop or a physical safety guarantee. Ordering is established by same-run trace.
One coordinator. Distinct skills.
EXPERIMENTAL · CONTROLLED TASK DECISIONSNavigation under the shared Agent coordinator, through the public tutorial. A releases before B starts.
41.4 s · original speed. B remains pending; separate arrival evaluation succeeded.
Recording & scope ↗The same coordination mechanism, with the ALOHA domain contract: 400 ACT transfer steps, then a separately admitted 50-step hold.
9.02 s · simulation time. Inference waits omitted; separate bounded hold evaluation succeeded.
Recording & scope ↗After B succeeds, its final decision is held. Original B cancel/query and Session close proceed independently; the late answer is retained.
39.7 s · original speed. No new submission. B pending/revoked; native cleanup unknown.
Recording & scope ↗Three separate qualified runs, with their own versions. The normal navigation and withdrawal clips use the new public opt-in tutorial; ALOHA uses the earlier private assembly. Withdrawal order comes from same-run trace, not video alone. No real-model decision or in-motion stop qualification in this group.
More ways to act.
MANIPULATION / HANDOFF / CANCELLATION400 ACT transfer steps, fresh feedback, then a separately admitted 50-step hold. A deterministic caller chooses both skills.
9.02 s in simulation time; inference waits omitted. The same-run evaluator confirms a bounded one-second hold.
Recording & evidence ↗A moves, then is cancelled. Native closure, fresh stillness and a ready successor let Harness settle A and admit B.
Green: A's path. Blue: B's path. B moves about 1.18 m; its final receipt remains pending.
Recording & evidence ↗The robot moves about 0.56 m, then stops. Native work closes and fresh observations confirm stillness; no output is delivered.
Without a ready successor, settlement stays pending and another admission is refused.
Recording & evidence ↗Separate actual simulations, with deterministic callers in these three clips. Navigation startup is trimmed and the view cropped, at original speed. No physical-robot validation. Full recordings, scope & setup.
Recover after a confirmed failure
Actual model · two separate runsA succeeds. Fresh public feedback lets the Agent finish; it never submits B.
Same-run evidence and limits →A's planner fails on a static occupied map. Native closure, quiet and fresh B readiness precede A release; only then does the model propose B.
Same-run evidence and limits →Two separate runs at original speed. The occupied map is a test fixture, not a physical obstacle. Final B settlement remains pending; see the original versions and recording scope.
Change course while moving.
BUSINESS INPUT / CANCEL / RELEASE / DECIDEA starts moving. One bound business instruction cancels exact A; native closure, fresh quiet and successor readiness permit release.
The task stops with no completed site or B request. Native cleanup stays unknown.
Recording & scope ↗Cancel comes before the model wait. After A releases, a fresh proposal and revalidated feedback permit one B request.
Only B counts as visited. Its settlement remains pending; model proposals never grant execution authority.
Run the revision tutorial ↗Separate same-run Humble/Nav2 recordings at original speed. Stop uses a four-second instruction delay; redirect uses zero delay. Both business inputs are controlled; only redirect uses an actual model. Task verdicts stay unassessed. No arbitrary-route, Owner recovery, hardware or hard stop qualification.
One checkpoint. Two possible routes.
BUSINESS INPUT / MODEL PROPOSAL / EXECUTIONA reaches its release boundary. One bound business instruction completes this checkpoint task at A; no B request is sent.
Prepared B resources may still exist. Task verdict stays unassessed and native cleanup unknown.
Recording & scope ↗The business provider chooses B. Three actual model proposals follow the task, while fresh feedback and Harness govern execution.
A releases before B starts. Final B settlement remains pending; the model does not supply execution authority.
Run the checkpoint tutorial ↗Separate same-run Humble/Nav2 simulations at original speed. Both business instructions are controlled; only the continue clip uses an actual model. This is a released-A checkpoint, not in-motion replanning or hardware qualification.
When the plan changes.
BUSINESS STOP / LATE PROPOSAL / OWNER LOSSAfter A releases, B starts moving. One signal reaches the public Agent CLI; it requests cancellation of exactly B.
Authority is revoked and the drive is sealed. Fresh quiet and old-command rejection are observed; native cleanup remains unknown and settlement pending.
Recording & scope ↗Nav2 visits A and B. The final proposal exceeds its deadline, then returns a valid answer while terminating. The Agent rejects it and requests help.
B's accepted native result survives cancellation. Revoked authority, pending settlement and unknown cleanup remain separate facts.
Recording & scope ↗A releases, B moves, then the actual Owner process is killed. The drive expires its finite permission and writes wheel targets to zero.
Late renewal and a new generation are refused. External quiet is observed; native cleanup and Core settlement remain unknown.
Recording & scope ↗Separate same-run recordings at original speed, with controlled executable proposals. The first two show business boundaries under a live owner; the third tests actual Owner loss. None tests a real model failure service or physical hardware.
A decision is only the beginning.
EXECUTE / OBSERVE / CONTINUERun with authority.
Keep a request separate from permission to execute. Carry that permission to the backend's submission boundary.
Keep the facts.
Know which action produced a result. Distinguish cancellation requests, native outcomes and work that still needs to settle.
Make the next move.
Give callers a basis for continuing, replacing or recovering work—and, over time, improving their skills from experience.
Built toward a broader loop.
Agents choose intent. Skills act. The world returns evidence. Robot Harness aims to connect these steps across navigation, manipulation and learned behaviors.
Task strategy and learning belong with callers; native robot stacks retain control and device protection.
Explore the architecture ↗Grounded in what runs today.
Execution authority, guarded results, local workers, dependent tasks and limited Host recovery. Nav2 simulation covers sequential navigation, moving cancellation, replacement and bounded loss isolation. The optional Python Runtime runs a MuJoCo/ACT skill and continuous transfer-to-hold operations; local MCP tools let existing agents consume the session.
Early-stage software and simulation: no physical-robot integration or hard stop guarantee. The separate Robot Agent provides bounded ALOHA and model-guided navigation applications. Broader learning backends and self-improvement remain planned work.
Current scope & validation ↗Small first steps.
Room to grow.
Build and run the software example with a C++17 compiler and CMake. No model or robot required.
Setup, commands & troubleshooting ↗git clone https://github.com/xiao-yang25/robot-harness.git
cd robot-harness
cmake -S . -B build
cmake --build build --parallel 2
./build/robot_harness_normal_executionFollow the work. Shape what comes next.
Try an example, report a failure or bring a concrete integration.
Contributing & licensing status ↗