Add new daemons and debug scripts for Sigenergy and Oracle functionalities

- Implement `sigen_daemon.py` to poll Sigenergy plant metrics and store snapshots.
- Create `web_daemon.py` for serving a web interface with various endpoints.
- Add debug scripts:
  - `debug_duplicates.py` to find duplicate target times in forecast data.
  - `debug_energy_forecast.py` to print baseline energy forecast curves.
  - `debug_oracle_evaluations.py` to run the oracle evaluator.
  - `debug_sigen.py` to inspect stored Sigenergy plant snapshots.
  - `debug_weather.py` to trace resolved truth data.
  - `modbus_test.py` for exploring Sigenergy plants or inverters over Modbus TCP.
- Introduce `oracle_evaluator.py` for evaluating stored oracle predictions against actuals.
- Add TCN training scripts in `tcn` directory for training usage sequence models.
This commit is contained in:
rpotter6298
2026-04-28 08:14:00 +02:00
parent ff0c65a794
commit c8e3016fd6
55 changed files with 6385 additions and 633 deletions
+34
View File
@@ -200,6 +200,40 @@ Notes:
- future prediction modules can join this to `weather_forecast_points`
- make this a hypertable on `resolved_at`
### `sigen_plant_snapshots`
High-resolution Sigenergy plant state from Modbus TCP.
Core fields:
- `observed_at`
- `received_at`
- `source`
- `solar_power_w`
- `battery_soc_pct`
- `battery_soh_pct`
- `battery_power_w`
- `grid_power_w`
- `grid_import_w`
- `grid_export_w`
- `load_power_w`
- `plant_active_power_w`
- `accumulated_pv_energy_kwh`
- `daily_consumed_energy_kwh`
- `accumulated_consumed_energy_kwh`
- status fields for EMS, running state, and grid sensor state
- `raw_values`
Notes:
- raw polling target is `SIGEN_POLL_SECONDS=5`
- make this a hypertable on `observed_at`
- keep raw JSON during integration so unsupported or surprising registers can be debugged
- rollup views should preserve averages, min/max spikes, and sample counts so short-duration usage signatures are not erased completely
Initial rollups:
- `sigen_plant_snapshots_1m`
- `sigen_plant_snapshots_15m`
- `sigen_plant_snapshots_1h`
### `system_events`
Operational events from collectors, storage, Gibil, and publishers.
+9 -3
View File
@@ -5,7 +5,7 @@
Start the web UI daemon:
```bash
python3 -m gibil.scripts.web_daemon
python3 -m gibil.scripts.daemons.web_daemon
```
The daemon listens on:
@@ -30,8 +30,10 @@ Install service units:
```bash
sudo cp deploy/systemd/astrape-web.service /etc/systemd/system/
sudo cp deploy/systemd/astrape-db.service /etc/systemd/system/
sudo cp deploy/systemd/astrape-sigen.service /etc/systemd/system/
sudo cp deploy/systemd/astrape-oracle.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now astrape-web.service astrape-db.service
sudo systemctl enable --now astrape-web.service astrape-db.service astrape-sigen.service astrape-oracle.service
```
Check status:
@@ -39,8 +41,12 @@ Check status:
```bash
systemctl status astrape-web.service
systemctl status astrape-db.service
systemctl status astrape-sigen.service
systemctl status astrape-oracle.service
journalctl -u astrape-web.service -f
journalctl -u astrape-db.service -f
journalctl -u astrape-sigen.service -f
journalctl -u astrape-oracle.service -f
```
Both services run as the IPA-managed `gibil` user from `/mnt/astrape`.
@@ -90,7 +96,7 @@ This does not write artificial data to TimescaleDB. It only changes the web UI w
Start the database ingest daemon:
```bash
python3 -m gibil.scripts.db_daemon
python3 -m gibil.scripts.daemons.db_daemon
```
Current behavior: