Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate sample with ml configuration #324

Open
peterdudfield opened this issue Feb 18, 2025 · 6 comments
Open

Validate sample with ml configuration #324

peterdudfield opened this issue Feb 18, 2025 · 6 comments
Assignees

Comments

@peterdudfield
Copy link
Contributor

Detailed Description

In order to avoid seeing a matrix multiplication errors, it would be good to validate the sample going into a model.

Context

  • we dont want training to slow down
  • we dont want R&D to slow down

Possible Implementation

  • add a function that can be called by the app, or other scripts.
  • This function will take in a sample, and the ml configruation like this
  • Check for nwp and satellite
  • Check the time dimension is the right size for the model
  • Check the channel dimension is the right size
  • Check the width and height is the right size
  • For site data, check there is enough data for the model. This might be linked with the time or sun features
@peterdudfield peterdudfield added enhancement New feature or request and removed enhancement New feature or request labels Feb 18, 2025
@itsbharatj
Copy link

Hey @peterdudfield, I will start working on it right away.

I understood the issue; as this is my first time contributing to OCF, can you tell where this function would be implemented in the codebase?

Thank You

@peterdudfield
Copy link
Contributor Author

It perhaps could be in its own file, under pvnet/validation.py

@MAYANK12SHARMA
Copy link

Hi @peterdudfield, Could you please assign it to me. I want to work on it.

@peterdudfield
Copy link
Contributor Author

@itsbharatj are you still working on this?

@itsbharatj
Copy link

@peterdudfield, yes I am working on it. Will issue a PR soon.

Would be great if you can assign the issue to me.

Thanks,
Bharat Jain

@itsbharatj
Copy link

Made a PR: #331
@peterdudfield can you please review it.

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants