Backtesting System
==================

OpenClaw includes a comprehensive backtesting engine for testing strategies against historical data.

Overview
--------

The backtesting system simulates trading using historical data to evaluate strategy performance before risking real capital.

Key Features
~~~~~~~~~~~~

* Historical simulation with accurate price data
* Multiple strategy support
* Performance analytics and metrics
* Risk-adjusted returns calculation
* Trade-by-trade analysis

Quick Start
-----------

Basic Backtest
~~~~~~~~~~~~~~

.. code-block:: python

    from openclaw.backtest.engine import BacktestEngine
    from datetime import datetime, timedelta

    # Create engine
    engine = BacktestEngine()

    # Configure backtest
    engine.configure(
        symbols=["AAPL"],
        start_date=datetime(2023, 1, 1),
        end_date=datetime(2023, 12, 31),
        initial_capital=10000.0
    )

    # Run backtest
    results = engine.run()

    # Print summary
    print(f"Total Return: {results.total_return:.2%}")
    print(f"Sharpe Ratio: {results.sharpe_ratio:.2f}")

Advanced Configuration
~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

    from openclaw.backtest.engine import BacktestEngine
    from openclaw.strategy.trend_following import TrendFollowingStrategy

    # Create engine with custom strategy
    engine = BacktestEngine()

    strategy = TrendFollowingStrategy(
        sma_period=50,
        position_size=0.1
    )

    engine.configure(
        symbols=["AAPL", "MSFT", "GOOGL"],
        strategy=strategy,
        start_date="2023-01-01",
        end_date="2023-12-31",
        initial_capital=100000.0,
        commission=0.001,  # 0.1% per trade
        slippage=0.0005    # 0.05% slippage
    )

    results = engine.run()

Backtest Engine
---------------

Configuration
~~~~~~~~~~~~~

.. code-block:: python

    from openclaw.backtest.engine import BacktestEngine

    engine = BacktestEngine()

    # Set parameters
    engine.configure(
        # Required
        symbols=["AAPL", "MSFT"],
        start_date="2023-01-01",
        end_date="2023-12-31",

        # Optional
        initial_capital=10000.0,
        strategy=None,  # Use default strategy
        commission=0.001,
        slippage=0.0005,
        enable_caching=True
    )

Running Backtests
~~~~~~~~~~~~~~~~~

.. code-block:: python

    # Run single backtest
    results = engine.run()

    # Run with progress callback
    def on_progress(progress: float):
        print(f"Progress: {progress:.0%}")

    results = engine.run(progress_callback=on_progress)

    # Run multiple backtests (parameter sweep)
    param_grid = {
        "sma_period": [20, 50, 200],
        "position_size": [0.05, 0.1, 0.2]
    }

    results = engine.run_sweep(param_grid)

Performance Metrics
-------------------

Basic Metrics
~~~~~~~~~~~~~

* **Total Return**: Overall percentage return
* **Annualized Return**: Return adjusted to yearly basis
* **Volatility**: Standard deviation of returns
* **Sharpe Ratio**: Risk-adjusted return metric
* **Max Drawdown**: Largest peak-to-trough decline
* **Win Rate**: Percentage of winning trades

Advanced Metrics
~~~~~~~~~~~~~~~~

.. code-block:: python

    from openclaw.backtest.analyzer import BacktestAnalyzer

    analyzer = BacktestAnalyzer(results)

    # Get all metrics
    metrics = analyzer.calculate_metrics()

    print(f"Sortino Ratio: {metrics.sortino_ratio:.2f}")
    print(f"Calmar Ratio: {metrics.calmar_ratio:.2f}")
    print(f"Omega Ratio: {metrics.omega_ratio:.2f}")
    print(f"Profit Factor: {metrics.profit_factor:.2f}")
    print(f"Expectancy: ${metrics.expectancy:.2f}")

Trade Analysis
~~~~~~~~~~~~~~

.. code-block:: python

    # Get individual trades
    trades = results.trades

    for trade in trades[:5]:  # First 5 trades
        print(f"Date: {trade.date}")
        print(f"Symbol: {trade.symbol}")
        print(f"Side: {trade.side}")
        print(f"Entry: ${trade.entry_price:.2f}")
        print(f"Exit: ${trade.exit_price:.2f}")
        print(f"PnL: ${trade.pnl:.2f}")

    # Trade statistics
    stats = analyzer.get_trade_statistics()
    print(f"Avg winning trade: ${stats.avg_winner:.2f}")
    print(f"Avg losing trade: ${stats.avg_loser:.2f}")
    print(f"Largest winner: ${stats.max_winner:.2f}")
    print(f"Largest loser: ${stats.max_loser:.2f}")

Visualization
-------------

Equity Curve
~~~~~~~~~~~~

.. code-block:: python

    from openclaw.backtest.analyzer import BacktestAnalyzer

    analyzer = BacktestAnalyzer(results)

    # Plot equity curve
    analyzer.plot_equity_curve(
        filename="equity_curve.png",
        show_drawdowns=True
    )

Drawdown Analysis
~~~~~~~~~~~~~~~~~

.. code-block:: python

    # Plot drawdown chart
    analyzer.plot_drawdown(
        filename="drawdown.png"
    )

    # Get drawdown statistics
    dd_stats = analyzer.get_drawdown_statistics()
    print(f"Max drawdown: {dd_stats.max_drawdown:.2%}")
    print(f"Avg drawdown: {dd_stats.avg_drawdown:.2%}")
    print(f"Max duration: {dd_stats.max_duration} days")

Monthly Returns
~~~~~~~~~~~~~~~

.. code-block:: python

    # Plot monthly returns heatmap
    analyzer.plot_monthly_returns(
        filename="monthly_returns.png"
    )

Trade Distribution
~~~~~~~~~~~~~~~~~~

.. code-block:: python

    # Plot trade distribution
    analyzer.plot_trade_distribution(
        filename="trade_dist.png"
    )

Multi-Symbol Backtests
----------------------

Portfolio Backtest
~~~~~~~~~~~~~~~~~~

.. code-block:: python

    from openclaw.backtest.engine import BacktestEngine

    engine = BacktestEngine()

    engine.configure(
        symbols=["AAPL", "MSFT", "GOOGL", "AMZN", "META"],
        weights="equal",  # Equal weighting
        start_date="2023-01-01",
        end_date="2023-12-31",
        initial_capital=100000.0
    )

    results = engine.run()

    # Per-symbol results
    for symbol in results.symbol_results:
        result = results.symbol_results[symbol]
        print(f"{symbol}: {result.total_return:.2%}")

Custom Weights
~~~~~~~~~~~~~~

.. code-block:: python

    # Custom portfolio weights
    engine.configure(
        symbols=["AAPL", "MSFT", "GOOGL"],
        weights={
            "AAPL": 0.5,
            "MSFT": 0.3,
            "GOOGL": 0.2
        }
    )

Strategy Development
--------------------

Creating Custom Strategies
~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

    from openclaw.strategy.base import Strategy, Signal
    from typing import List, Dict
    import pandas as pd

    class MyCustomStrategy(Strategy):
        """Custom trading strategy."""

        def __init__(self, param1: float = 1.0, param2: int = 10):
            super().__init__()
            self.param1 = param1
            self.param2 = param2

        def generate_signals(
            self,
            data: pd.DataFrame
        ) -> List[Signal]:
            """Generate trading signals."""
            signals = []

            # Your strategy logic here
            for i in range(len(data)):
                if self.should_buy(data, i):
                    signals.append(Signal(
                        date=data.index[i],
                        action="buy",
                        confidence=0.8
                    ))
                elif self.should_sell(data, i):
                    signals.append(Signal(
                        date=data.index[i],
                        action="sell",
                        confidence=0.8
                    ))

            return signals

        def should_buy(self, data: pd.DataFrame, index: int) -> bool:
            """Buy condition."""
            # Implement buy logic
            return False

        def should_sell(self, data: pd.DataFrame, index: int) -> bool:
            """Sell condition."""
            # Implement sell logic
            return False

Strategy Optimization
~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

    from openclaw.backtest.optimizer import StrategyOptimizer

    optimizer = StrategyOptimizer()

    # Define parameter grid
    param_grid = {
        "sma_fast": [10, 20, 30],
        "sma_slow": [50, 100, 200],
        "position_size": [0.05, 0.1, 0.15]
    }

    # Run optimization
    best_params = optimizer.optimize(
        strategy_class=TrendFollowingStrategy,
        param_grid=param_grid,
        metric="sharpe_ratio",  # Optimize for Sharpe
        data=data
    )

    print(f"Best parameters: {best_params}")

Walk-Forward Analysis
~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

    from openclaw.backtest.walk_forward import WalkForwardTester

    wf_tester = WalkForwardTester()

    results = wf_tester.run(
        strategy=strategy,
        data=data,
        train_size=252,    # 1 year training
        test_size=63,      # 3 months testing
        step_size=63       # Move forward 3 months at a time
    )

    print(f"Average in-sample Sharpe: {results.avg_in_sample_sharpe:.2f}")
    print(f"Average out-of-sample Sharpe: {results.avg_out_sample_sharpe:.2f}")

Data Handling
-------------

Data Sources
~~~~~~~~~~~~

.. code-block:: python

    from openclaw.data.sources import YahooFinanceData

    # Use Yahoo Finance data
    data_source = YahooFinanceData()

    # Fetch historical data
    data = data_source.get_data(
        symbols=["AAPL"],
        start_date="2023-01-01",
        end_date="2023-12-31",
        interval="1d"
    )

Custom Data
~~~~~~~~~~~

.. code-block:: python

    # Use custom data
    import pandas as pd

    custom_data = pd.read_csv("my_data.csv", index_col=0, parse_dates=True)

    engine = BacktestEngine()
    engine.set_data(custom_data)
    engine.configure(
        symbols=["CUSTOM"],
        start_date="2023-01-01",
        end_date="2023-12-31"
    )

Best Practices
--------------

1. **Out-of-sample testing**: Reserve data for final validation
2. **Transaction costs**: Always include realistic commissions and slippage
3. **Multiple regimes**: Test across different market conditions
4. **Robustness checks**: Sensitivity analysis on parameters
5. **Risk metrics**: Focus on risk-adjusted returns, not just total return
6. **Realistic assumptions**: Account for market impact and liquidity

Common Pitfalls
---------------

* **Overfitting**: Too many parameters optimized on limited data
* **Look-ahead bias**: Using future information in strategy logic
* **Survivorship bias**: Testing only on currently active companies
* **Data mining**: Testing too many strategies on same data
* **Ignoring costs**: Not accounting for fees and slippage