evotraders/services/README.md

# 大时代 Service Surfaces

This repository is in a split-first state: local development now assumes
separate app surfaces and a dedicated WebSocket gateway instead of a single
combined backend entrypoint.

For the canonical architecture summary, start with
[docs/current-architecture.md](../docs/current-architecture.md). This file is
service-focused and includes migration details.
The matching visual diagram lives at
[docs/current-architecture.excalidraw](../docs/current-architecture.excalidraw),
and the next-step execution plan lives at
[docs/development-roadmap.md](../docs/development-roadmap.md).

## Service Map

| Surface | Default port | Role |
| --- | --- | --- |
| `backend.apps.agent_service` | `8000` | Control plane for workspaces, agents, skills, guard/approvals |
| `backend.apps.trading_service` | `8001` | Read-only trading data APIs such as prices, financials, insider trades |
| `backend.apps.news_service` | `8002` | Read-only explain/news APIs such as story, similar days, range explain |
| `backend.apps.runtime_service` | `8003` | Runtime lifecycle APIs under `/api/runtime/*` |
| `backend.apps.openclaw_service` | `8004` | Read-only OpenClaw REST facade |
| Gateway (`backend.main`) | `8765` | WebSocket feed, runtime event stream, pipeline execution |
| OpenClaw Gateway | `18789` | External OpenClaw WebSocket endpoint consumed by 大时代 gateway |

## Runtime Modes

### Standalone Mode (Direct Gateway Startup)

For simple deployments or backward compatibility:

```bash
python -m backend.main --mode live --host 0.0.0.0 --port 8765
```

In this mode, Gateway runs as the primary process with all components
(Pipeline, Market Service, Scheduler) loaded in-process.

### Microservice Mode (Recommended)

For development and production with service isolation:

```bash
./start-dev.sh
```

This starts all services with `runtime_service` managing the Gateway lifecycle.

## What Runs By Default In Dev

The supported local dev path is:

```bash
./start-dev.sh
```

That script starts:

- `agent_service` on `8000`
- `trading_service` on `8001`
- `news_service` on `8002`
- `runtime_service` on `8003`
- 大时代 gateway on `8765` (as subprocess of runtime_service)

It does **not** start `openclaw_service` on `8004`.

Instead, the gateway expects an OpenClaw WebSocket server to already be
available at `ws://localhost:18789` unless you override the OpenClaw gateway
configuration outside the script.

## Manual Startup

Run split service surfaces explicitly:

```bash
python -m uvicorn backend.apps.agent_service:app --host 0.0.0.0 --port 8000 --reload
python -m uvicorn backend.apps.trading_service:app --host 0.0.0.0 --port 8001 --reload
python -m uvicorn backend.apps.news_service:app --host 0.0.0.0 --port 8002 --reload
python -m uvicorn backend.apps.runtime_service:app --host 0.0.0.0 --port 8003 --reload
```

The Gateway is started by `runtime_service` via the `/api/runtime/start` API,
not manually. To start a runtime:

```bash
curl -X POST http://localhost:8003/api/runtime/start \
  -H "Content-Type: application/json" \
  -d '{
    "launch_mode": "fresh",
    "tickers": ["AAPL", "MSFT"],
    "schedule_mode": "daily",
    "trigger_time": "09:30",
    "initial_cash": 100000
  }'
```

Optional OpenClaw REST surface:

```bash
python -m uvicorn backend.apps.openclaw_service:app --host 0.0.0.0 --port 8004 --reload
```

## Runtime Responsibilities

The runtime path is intentionally split:

### Control Plane (runtime_service :8003)

- **Gateway lifecycle**: Start, stop, restart Gateway processes
- **Configuration**: Bootstrap values, runtime parameters
- **Health monitoring**: Gateway process status, port management
- **Run history**: List historical runs, restore from snapshots

### Data Plane (Gateway :8765)

- **WebSocket transport**: Live event streaming to frontend
- **Pipeline execution**: Analysis -> Communication -> Decision -> Execution
- **Market data**: Real-time price feeds and backtest simulation
- **Scheduler**: Trading cycle orchestration

### Supporting Services

- `agent_service` handles control-plane reads and writes for agents, workspaces, files, and approvals
- `trading_service` provides read-only trading data
- `news_service` provides news enrichment and explanation APIs

The practical request path looks like:

```
frontend -> runtime_service/control APIs -> gateway/runtime manager -> market service + pipeline + storage
```

## runtime_service as Gateway Process Manager

The `runtime_service` is the **Gateway Process Manager** in the microservice
architecture. Its responsibilities:

1. **Process Management**
   - Spawns Gateway as subprocess via `_start_gateway_process()`
   - Monitors process health via `gateway_process.poll()`
   - Handles graceful shutdown (SIGTERM) and force kill

2. **Port Management**
   - Finds available ports (`_find_available_port()`)
   - Tracks current Gateway port in RuntimeState

3. **Lifecycle APIs**
   - `POST /api/runtime/start` - Create run, spawn Gateway
   - `POST /api/runtime/stop` - Stop Gateway process
   - `POST /api/runtime/restart` - Stop then start new runtime
   - `GET /api/runtime/gateway/status` - Check Gateway health

4. **State Management**
   - Maintains RuntimeState singleton (thread-safe)
   - Tracks runtime_manager reference for in-memory state
   - Falls back to persisted snapshots when Gateway is stopped

### Gateway Subprocess Architecture

```
runtime_service (:8003)
    |
    |-- spawns --> Gateway subprocess (:8765)
                       |
                       |-- TradingPipeline
                       |-- MarketService
                       |-- Scheduler
                       |-- WebSocket server
```

The Gateway subprocess runs `backend.gateway_server` module (not `backend.main`)
with run-specific configuration passed via CLI arguments.

## Environment Variables

### Backend routing preferences

These variables let the gateway or tools prefer split services over in-process fallbacks:

| Variable | Used by | Meaning |
| --- | --- | --- |
| `TRADING_SERVICE_URL` | gateway, data tools | Prefer `trading_service` for trading reads |
| `NEWS_SERVICE_URL` | gateway, data tools | Prefer `news_service` for explain/news reads |
| `RUNTIME_SERVICE_URL` | dev scripts / future follow-up | Reserved for runtime-service-aware flows |
| `OPENCLAW_SERVICE_URL` | dev scripts / future follow-up | Points at the OpenClaw gateway origin in current dev setup |

Current `start-dev.sh` defaults:

```bash
TRADING_SERVICE_URL=http://localhost:8001
NEWS_SERVICE_URL=http://localhost:8002
RUNTIME_SERVICE_URL=http://localhost:8003
OPENCLAW_SERVICE_URL=http://localhost:18789
```

Note that `OPENCLAW_SERVICE_URL` currently points at the OpenClaw gateway origin used by the live WebSocket bridge, not the optional REST app on `:8004`.

### Frontend service targets

The frontend can directly call split services with:

```bash
VITE_CONTROL_API_BASE_URL=http://localhost:8000/api
VITE_RUNTIME_API_BASE_URL=http://localhost:8003/api/runtime
VITE_NEWS_SERVICE_URL=http://localhost:8002
VITE_TRADING_SERVICE_URL=http://localhost:8001
VITE_WS_URL=ws://localhost:8765
```

## Current Frontend Direct-Call Coverage

Direct browser calls currently cover:

- runtime panel loading and runtime discovery
- story
- similar days
- range explain
- news for date
- news categories
- selected trading reads such as stock history and insider trades

Other flows still depend on the gateway WebSocket and control plane APIs.

## OpenClaw Integration Notes

There are two separate OpenClaw integration surfaces in this repo:

- OpenClaw WebSocket gateway on `:18789`
  - used directly by `backend/services/gateway.py`
  - this is what `start-dev.sh` assumes exists
- `backend.apps.openclaw_service` on `:8004`
  - optional REST facade over OpenClaw CLI-backed reads
  - useful for typed client access and service-level testing

Do not treat those as interchangeable in docs or deployment config.

## Internal Module Direction

The codebase is now organized around these boundaries:

```text
frontend
  ├─ runtime/control/news/trading API clients
  └─ WebSocket runtime feed

backend.apps.agent_service
  └─ control-plane routes

backend.apps.runtime_service
  └─ runtime lifecycle routes, Gateway process management

backend.apps.trading_service
  └─ read-only trading contract

backend.apps.news_service
  └─ read-only explain/news contract

backend.apps.openclaw_service
  └─ optional OpenClaw REST facade

backend.gateway_server
  └─ Gateway subprocess entry point (run-scoped)

backend.main
  └─ standalone Gateway entry point (compatibility)
```

## Migration Boundaries

Some agent-migration helpers still exist in the tree, but they are not part of
the supported runtime path yet:

No workspace-loading helper remains on `TradingPipeline`. Runtime agent loading
is expected to stay on the run-scoped creation path.

Also note the remaining naming split:

- `workspaces/` = design-time CRUD registry
- `runs/<run_id>/` = runtime state and run-scoped agent assets

## Future Architecture Direction

### Current State

- Pipeline logic lives in Gateway process
- Gateway is spawned as subprocess by runtime_service
- Standalone mode (`backend.main`) preserved for compatibility

### Target State

- Pipeline stages become independent services
- Gateway becomes thin event router
- runtime_service becomes full orchestrator
- Standalone mode deprecated and removed

See [docs/development-roadmap.md](../docs/development-roadmap.md) for detailed
phase planning.