Merge pull request #226 from haorui-harry/main

compiles valuable GUI trajectories into parameterized, self-verifying CLI macros for agents

Author: yuh-yang

openclaw-skill/agent-harness/OPENCLAW.md

@@ -0,0 +1,185 @@
+# OpenClaw Macro System — Agent Harness SOP
+
+## What Is This?
+
+**OpenClaw Macro System** is a layered CLI that turns valuable GUI workflows into
+parameterized, agent-callable macros. The agent sends one command:
+
+```bash
+cli-anything-openclaw macro run export_png --param output=/tmp/out.png --json
+```
+
+The system handles everything else: parameter validation, precondition checks,
+backend selection, step execution, postcondition verification, and structured
+result output. The agent never touches the GUI directly.
+
+## Architecture
+
+```
+Agent
+  └─▶  cli-anything-openclaw macro run <name> --param k=v --json   (L6: CLI)
+             │
+        MacroRuntime                                                  (L5)
+             │  1. Validate params against MacroDefinition schema
+             │  2. Check preconditions (file_exists, process_running, …)
+             │  3. For each step:
+             │       RoutingEngine → select backend by priority       (L3)
+             │       Backend.execute(step, resolved_params)           (L2)
+             │  4. Check postconditions
+             │  5. Collect declared outputs
+             │  6. Record telemetry in ExecutionSession
+             └─▶  { success, output, error, telemetry }
+```
+
+## Layer Mapping
+
+| Layer | Name | Implementation |
+|-------|------|---------------|
+| L7 | Agent Task Interface | Caller (OpenClaw or any agent) |
+| L6 | Unified CLI Entry | `openclaw_cli.py` — Click CLI |
+| L5 | Macro Execution Runtime | `core/runtime.py` |
+| L4 | Parameterized Macro Model | `core/macro_model.py` + `macro_definitions/*.yaml` |
+| L3 | Backend Routing Engine | `core/routing.py` |
+| L2 | Execution Backends | `backends/` (5 backends) |
+| L1 | Target Application | Any GUI-first or closed-source app |
+
+## Execution Backends
+
+| Backend | Priority | Trigger | Use case |
+|---------|----------|---------|----------|
+| `native_api` | 100 | `backend: native_api` | subprocess / shell commands |
+| `gui_macro` | 80 | `backend: gui_macro` | precompiled coordinate replay (pyautogui) |
+| `file_transform` | 70 | `backend: file_transform` | XML, JSON, text file editing |
+| `semantic_ui` | 50 | `backend: semantic_ui` | accessibility API + keyboard (xdotool) |
+| `recovery` | 10 | `backend: recovery` | retry + fallback orchestration |
+
+The RoutingEngine respects the step's explicit `backend:` field; if that backend
+is unavailable it walks down the priority list.
+
+## Macro Definition Format
+
+Macros live in `cli_anything/openclaw/macro_definitions/` as YAML files:
+
+```yaml
+name: export_png
+version: "1.0"
+description: Export the active diagram to PNG.
+
+parameters:
+  output:
+    type: string
+    required: true
+    example: /tmp/diagram.png
+
+preconditions:
+  - process_running: draw.io
+  - file_exists: /path/to/input.drawio
+
+steps:
+  - id: export
+    backend: native_api
+    action: run_command
+    params:
+      command: [draw.io, --export, --output, "${output}", input.drawio]
+    timeout_ms: 30000
+    on_failure: fail     # or: skip | continue
+
+postconditions:
+  - file_exists: ${output}
+  - file_size_gt:
+      - ${output}
+      - 100
+
+outputs:
+  - name: exported_file
+    path: ${output}
+
+agent_hints:
+  danger_level: safe
+  side_effects: [creates_file]
+  reversible: true
+```
+
+### Supported Condition Types
+
+| Type | Args | Checks |
+|------|------|--------|
+| `file_exists` | path | `os.path.exists(path)` |
+| `file_size_gt` | [path, min_bytes] | `os.stat(path).st_size > min_bytes` |
+| `process_running` | name | `pgrep -x name` or psutil |
+| `env_var` | name | `name in os.environ` |
+| `always` | true/false | constant pass/fail |
+
+## Package Layout
+
+```
+openclaw-skill/
+└── agent-harness/
+    ├── setup.py                           entry_point: cli-anything-openclaw
+    └── cli_anything/openclaw/
+        ├── openclaw_cli.py                Main Click CLI
+        ├── macro_definitions/             YAML macro registry
+        │   ├── manifest.yaml
+        │   └── examples/
+        │       ├── export_file.yaml
+        │       ├── transform_json.yaml
+        │       └── undo_last.yaml
+        ├── core/
+        │   ├── macro_model.py             MacroDefinition + YAML loader
+        │   ├── registry.py               MacroRegistry
+        │   ├── routing.py                RoutingEngine
+        │   ├── runtime.py                MacroRuntime (full lifecycle)
+        │   └── session.py               ExecutionSession + telemetry
+        ├── backends/
+        │   ├── base.py                   Backend ABC + StepResult
+        │   ├── native_api.py             subprocess backend
+        │   ├── file_transform.py         XML/JSON/text backend
+        │   ├── semantic_ui.py            accessibility backend
+        │   ├── gui_macro.py              compiled replay backend
+        │   └── recovery.py              retry/fallback backend
+        ├── skills/SKILL.md               Agent-readable skill definition
+        ├── utils/repl_skin.py            Unified REPL skin (cli-anything standard)
+        └── tests/
+            ├── test_core.py              Unit tests (49 tests, no external deps)
+            └── test_full_e2e.py          E2E + CLI subprocess tests (15 tests)
+```
+
+## Installation
+
+```bash
+cd openclaw-skill/agent-harness
+pip install -e .
+```
+
+**Runtime dependencies:** Python 3.10+, PyYAML, click, prompt-toolkit.
+
+**Optional (for specific backends):**
+- `xdotool` — semantic_ui backend on Linux
+- `pyautogui` — gui_macro backend
+- `psutil` — richer process_running checks
+
+## Running Tests
+
+```bash
+cd openclaw-skill/agent-harness
+python3 -m pytest cli_anything/openclaw/tests/ -v -s
+# 64 passed
+```
+
+## Key Design Decisions
+
+**Why YAML macros, not Python?** YAML macros are readable by agents without
+running code, inspectable via `macro info`, and editable without touching the
+harness source.
+
+**Why 5 backends?** Real GUI applications expose many different control
+surfaces. The routing engine picks the most reliable one available — the agent
+doesn't need to know which one ran.
+
+**Why preconditions and postconditions?** Agents operate in environments where
+state is uncertain. Failing loudly before execution (preconditions) and
+verifying after (postconditions) catches problems the agent can act on.
+
+**Why `on_failure: skip | continue`?** Some macro steps are best-effort (e.g.,
+confirming a dialog that may or may not appear). Skipping lets the macro
+continue to the real work.

openclaw-skill/agent-harness/cli_anything/openclaw/README.md

@@ -0,0 +1,75 @@
+# OpenClaw Macro System
+
+**OpenClaw Macro System** is a layered CLI that converts GUI workflows into
+parameterized, agent-callable macros. Agents call `macro run <name>` through
+a stable CLI; the system routes execution to the right backend (native plugin,
+file transform, semantic UI, or compiled GUI replay) — invisible to the agent.
+
+## Installation
+
+```bash
+pip install -e .
+```
+
+**Dependencies:** Python 3.10+, PyYAML, click, prompt-toolkit
+
+## Usage
+
+```bash
+# List available macros
+cli-anything-openclaw macro list --json
+
+# Inspect a macro
+cli-anything-openclaw macro info export_file --json
+
+# Execute a macro
+cli-anything-openclaw macro run transform_json \
+    --param file=/tmp/config.json \
+    --param key=theme --param value=dark --json
+
+# Dry run
+cli-anything-openclaw --dry-run macro run export_file \
+    --param output=/tmp/out.txt --json
+
+# Interactive REPL
+cli-anything-openclaw
+```
+
+## Run Tests
+
+```bash
+cd openclaw-skill/agent-harness
+pip install -e ".[dev]"
+python -m pytest cli_anything/openclaw/tests/ -v -s
+```
+
+## Architecture
+
+```
+cli-anything-openclaw (CLI)
+  └─▶ macro run <name> --param key=value
+         │
+    MacroRuntime
+         │  validate params
+         │  check preconditions
+         │  for each step:
+         │    RoutingEngine → select backend
+         │    Backend.execute(step, params)
+         │  check postconditions
+         └─▶ ExecutionResult { success, output, telemetry }
+```
+
+**Backends:**
+- `native_api` — subprocess / shell commands
+- `file_transform` — XML, JSON, text file editing
+- `semantic_ui` — accessibility controls + keyboard shortcuts
+- `gui_macro` — precompiled coordinate-based replay
+- `recovery` — retry + fallback orchestration
+
+## Adding a Macro
+
+1. Create `cli_anything/openclaw/macro_definitions/my_macro.yaml`
+2. Add it to `macro_definitions/manifest.yaml`
+3. Verify: `cli-anything-openclaw macro validate my_macro --json`
+
+See `skills/SKILL.md` (installed with the package) for full macro YAML schema.

openclaw-skill/agent-harness/cli_anything/openclaw/__init__.py

@@ -0,0 +1 @@
+# cli_anything/openclaw package

openclaw-skill/agent-harness/cli_anything/openclaw/__main__.py

@@ -0,0 +1,5 @@
+"""Enable: python3 -m cli_anything.openclaw"""
+from cli_anything.openclaw.openclaw_cli import cli
+
+if __name__ == "__main__":
+    cli()

openclaw-skill/agent-harness/cli_anything/openclaw/backends/__init__.py

@@ -0,0 +1,89 @@
+"""Backend base classes and result types.
+
+All execution backends inherit from Backend and return StepResult.
+"""
+
+from __future__ import annotations
+
+import time
+from abc import ABC, abstractmethod
+from dataclasses import dataclass, field
+from typing import Any, Optional
+
+
+@dataclass
+class StepResult:
+    """Result of a single macro step execution."""
+    success: bool
+    output: dict = field(default_factory=dict)
+    error: str = ""
+    duration_ms: float = 0.0
+    backend_used: str = ""
+
+    def to_dict(self) -> dict:
+        return {
+            "success": self.success,
+            "output": self.output,
+            "error": self.error,
+            "duration_ms": self.duration_ms,
+            "backend_used": self.backend_used,
+        }
+
+
+class BackendContext:
+    """Runtime context passed to each backend during step execution."""
+
+    def __init__(
+        self,
+        params: dict,
+        previous_results: Optional[list[StepResult]] = None,
+        dry_run: bool = False,
+        timeout_ms: int = 30_000,
+    ):
+        self.params = params
+        self.previous_results: list[StepResult] = previous_results or []
+        self.dry_run = dry_run
+        self.timeout_ms = timeout_ms
+        self._start = time.time()
+
+    def elapsed_ms(self) -> float:
+        return (time.time() - self._start) * 1000
+
+
+class Backend(ABC):
+    """Abstract base class for all execution backends.
+
+    Concrete backends implement execute() and return a StepResult.
+    """
+
+    name: str = "base"
+    priority: int = 0
+
+    @abstractmethod
+    def execute(
+        self,
+        step: "MacroStep",  # type: ignore[name-defined]
+        params: dict,
+        context: BackendContext,
+    ) -> StepResult:
+        """Execute a macro step.
+
+        Args:
+            step: The MacroStep definition being executed.
+            params: Fully resolved (substituted) parameters.
+            context: Runtime context with previous results and flags.
+
+        Returns:
+            StepResult describing success/failure and captured output.
+        """
+
+    def is_available(self) -> bool:
+        """Return True if this backend can be used in the current environment."""
+        return True
+
+    def describe(self) -> dict:
+        return {
+            "name": self.name,
+            "priority": self.priority,
+            "available": self.is_available(),
+        }