context/refactor_todo.md

Refactor Plan & Todo – SolarWine Baseline

This file tracks the refactor steps for Gemini utilities, phenology, time features, IMS coverage, and performance/caching improvements.

---

High Priority

• Unify Gemini client & JSON handling

  • genai_utils: verify get_google_api_key, get_genai_client, extract_json_object (already created).
  • day_ahead_advisor:
    • Use extract_json_object via local _extract_json wrapper.
    • Confirm api_key and client now call genai_utils.
  • llm_data_engineer:
    • Replace api_key property with get_google_api_key(self._api_key).
    • Replace client property with get_genai_client(self._api_key).
    • Swap local _extract_json to use extract_json_object.
  • routing_agent:
    • Replace api_key property with get_google_api_key(self._api_key).
    • Replace client property with get_genai_client(self._api_key).
  • vineyard_chatbot:
    • Replace api_key property with get_google_api_key(self._api_key).
    • Replace client property with get_genai_client(self._api_key).
    • Make has_api_key call get_google_api_key in a try/except instead of duplicating logic.

• Centralise phenology logic

  • phenology.py: confirm estimate_stage_for_date and stage_id_and_description_for_date match the biology plan and skills.
  • app.py (Vineyard Advisor vine snapshot):
    • Replace inline month→stage mapping with stage_id_and_description_for_date(today).
  • Any future modules that reason about "current stage" should import from phenology instead of re-implementing month logic.

• IMS cache coverage & observability

  • IMSClient.fetch_and_cache:
    • Add lightweight logging or counters for skipped chunks (date ranges that failed).
  • app.py – Forecasting tab:
    • Add a small IMS coverage panel that shows:
      • First timestamp in cache
      • Last timestamp in cache
      • Number of days of coverage
      • Age (minutes/hours) of the latest record
  • app.py – Vineyard Advisor tab:
    • Reuse the cached vine snapshot to also display IMS coverage (not just last point) where useful.

---

Medium Priority

• Shared time-feature utilities

  • time_features.py: confirm add_cyclical_time_features API (timestamp column or index).
  • preprocessor.create_time_features:
    • Replace manual sin/cos code with add_cyclical_time_features(df, timestamp_col="timestamp_utc").
    • Keep raw month and day_of_year features as now.
  • chronos_forecaster.load_data:
    • Replace inline hour_sin/cos and doy_sin/cos computations with add_cyclical_time_features(resampled, index_is_timestamp=True).
  • llm_data_engineer.engineer_features:
    • Replace its own cyclical feature block with a call to add_cyclical_time_features on the resolved timestamp.

• Cache heavy shading simulations

  • tracker_optimizer.simulate_tilt_angles:
    • Add a simple in-module cache (e.g. dict keyed by (len(df), tuple(sorted(angles))) or (date_min, date_max, angles) for the sample dataset).
    • Ensure caching does not break if df changes (avoid caching across different data sources).
  • vineyard_chatbot._tool_compare_angles:
    • Confirm it benefits from the new simulate_tilt_angles cache (repeated queries reuse results).
  • Optionally expose precomputed angle summaries in the Shading tab for instant advisor-like responses.

• Use phenology helper in future logic

  • When adding or refactoring:
    • Day-ahead planner
    • Energy budget planner
    • Operational modes / real-time control
    • Always derive phenological stage via phenology.estimate_stage_for_date or stage_id_and_description_for_date instead of hand-written month rules.

---

Low Priority (later)

• Vectorise FarquharModel.compute_all if Stage 1 datasets grow significantly or are needed in near real-time.
• Design and implement thingsboard_client according to the thingsboard-client skill, then gradually wire real-time vine telemetry (soil moisture, irrigation, local temps) into Vineyard Advisor and future control logic.
• Add focused tests for:
  • FarquharModel (known inputs, NaN handling).
  • Preprocessor (merge, split, edge cases).
  • SolarGeometry/ShadowModel (tracker tilt, mask shape).
  • ChronosForecaster.load_data (grid coverage, growing-season filter, time features).