Skip to content

PATH: Run tests #996

Description

@raghumanimehta

Run the run_test_plan.py (see here: https://ubcsailbot.atlassian.net/wiki/spaces/prjt22/pages/2996568069/Test+Plans) . This info is also in the repository.

Run all the complicated test plans (especialy the vancouver_to_gulf_of_alaska_complex.yaml) for 2 hours each and get the logs and analyze them (read them yourself and pass it onto GPT/AI for further analysis). Also run the path analysis under sailbot_workspace/notebooks/local_pathfinding/explore_bag_data.ipynb for each of the rosbag collected from the runs. Note that these are always collected in the sailbot_workspace/notebooks/local_pathfinding/session_storage/ with date and time in the file format.

What i want is a detailed report (please DO NOT GENERATE THIS WITH AI WITHOUT REVIEWING, trust me I can tell) about what the logs and the db say about the paths eg. the analysis in the ipynb about the wind and the data covered in that plus what the logs were outputting.

Report should include:

  • Which complicated test plans were run, especially vancouver_to_gulf_of_alaska_complex.yaml
  • Exact command used, timeout, mode, log level, and output folder
  • For each run: status, duration, reason from result.json, final waypoint reached, last distance to target, missing nodes if any, launch.log, and rosbag path
  • Short scenario summary for each YAML: wind, start GPS, global path, AIS traffic, land/exclusion zones, and why the plan is complicated
  • Human-reviewed log notes: warnings/errors, node crashes, missing nodes, path regeneration messages, collision-zone messages, local path expiry/invalidation, and any stuck/oscillating behavior
  • Rosbag / DB analysis from explore_bag_data.ipynb: GPS track, local/global path data, AIS ship count, filtered wind, desired heading, boat heading, true wind, sailing angle, replans/path switches
  • Path quality notes: progress toward route, land avoidance, AIS avoidance, smooth vs jumpy paths, desired-heading changes, whether the planner got trapped, and whether behavior matched the test goal
  • Wind notes: apparent/true wind, upwind/downwind/crosswind behavior, expected tacking, detected tacks/gybes, and whether replans lined up with wind changes
  • Metrics: distance travelled, straight-line distance, distance ratio, runtime, average boat speed, average wind speed, actual vs ideal time, path cycles, heading changes per cycle, tack/gybe count
  • For vancouver_to_gulf_of_alaska_complex.yaml: dense AIS traffic behavior, head-on/crossing/overtaking cases, land/shoal avoidance, Vancouver departure corridor progress, and whether timeout/failure was due to real constraints or planner issues

Artifacts to attach/reference:

  • summary.json
  • each result.json
  • each launch.log
  • rosbag .db3 folders
  • notebook plots/screenshots
  • exported metric tables if available

Storage note:

  • run_test_plans writes run summaries/logs under:
    notebooks/local_pathfinding/session_recordings/test_plans_results/
  • The rosbag data used by explore_bag_data.ipynb is collected separately under:
    notebooks/local_pathfinding/session_storage/
  • For the report, match each launch.log / result.json run with the corresponding timestamped rosbag from session_storage/.

Metadata

Metadata

Assignees

Labels

pathPathfinding team

Fields

No fields configured for Feature.

Projects

Status
Backlog

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions