Integration Test fixes

[yuvraajnarula]

Pull Request

Description

These changes fix multiple errors observed in unit tests and improve PV data handling:

Issues fixed:

• Missing pv.netcdf file in the repo.
• KeyError: 'kwp'
• Row-by-row iteration on large Parquet files was slow.
• ValueError: You are trying to merge on datetime64[ns] and datetime64[ns, UTC] columns
• KeyError: 'generation_wh'
• Metadata CSV inconsistencies.

Improvements in get_pv_metadata and get_pv_truth:

1. Column fixes:

   • Renamed kWp → capacity for consistent naming.
   • Corrected kwp case issues.

2. get_pv_metadata:

   • Clean join with pv_id.
   • Standardized metadata column names (latitude, longitude, capacity).

3. get_pv_truth:

   • Uses Parquet files instead of NetCDF.
   • Selects by ss_id and datetime_GMT.
   • Converts generation values to kWh.
   • Timestamp columns are timezone-aware (UTC) to allow safe merges.
   • Corrects PV generation column name (generation_Wh) to prevent KeyError.
   • Ensures vectorized expansion across horizons for prediction.

These changes improve test reliability and performance when working with PV datasets.

Fixes #292

How Has This Been Tested?

• Local unit tests for get_pv_metadata and get_pv_truth pass successfully.

Checklist

• My code follows [OCF's coding style guidelines](https://github.com/openclimatefix/.github/blob/main/coding_style.md)
• I have performed a self-review of my own code
• I have made corresponding changes to the documentation if needed
• I have added tests that prove my fix is effective or that my feature works
• I have checked my code and corrected any misspellings

[yuvraajnarula]

@peterdudfield, there are two Parquet folders: 30_minutely and 5_minutely. Do you want me to create two separate test cases for them, or should we handle it via a single test with an argument to choose the folder?

[peterdudfield]

one single test, with an arguement sounded good

[yuvraajnarula]

@peterdudfield
python -m pytest tests/integration/eval/test_pv.py --foldername 5_minutely -v
30_minutely as a default parameter

[peterdudfield]

hey @yuvraajnarula did you manage to get the test passing?

[yuvraajnarula]

Here are my logs. Do u need a --foldername all -v
to test both of them at the same time?

@peterdudfield

[yuvraajnarula]

@peterdudfield any updates?

[peterdudfield]

And can this test be removed?
Or what does this do?

[peterdudfield]

It shouldnt be in conftest.py

[yuvraajnarula]

so should this contain the pytest_addoption and pytest_generate_tests hooks that enable the --foldername command-line option, not an actual test function?

[yuvraajnarula]

I have commented down the previous implementation and worked around pytest_addoption and pytest_generate_tests. So sorry for my confusion. Let me know if this is more cleaner and suitable for the PR.