add cloudy flag to ghi and nighttime to power variables#382
add cloudy flag to ghi and nighttime to power variables#382alorenzo175 merged 10 commits intoSolarArbiter:masterfrom
Conversation
alorenzo175
added
enhancement
|
I've modified the validation.validator.detect_clearsky_ghi to raise warnings instead of errors and default to flagging a time as cloudy if the pvlib detect clearsky can't run. This produces a validation like I think this is probably the right approach, but wanted to get some feedback @wholmgren @cwhanse |
wholmgren
left a comment
wholmgren
left a comment
There was a problem hiding this comment.
The results look good to me. When does the pvlib function fail with our data?
| cloudy_flag = validator.detect_clearsky_ghi(values, clearsky['ghi'], | ||
| _return_mask=True) |
There was a problem hiding this comment.
I thought you'd want a try/except here instead of a warning in the function.
There was a problem hiding this comment.
I went back and forth on this. That is one option, but the rest of the functions in validator.py will not raise an error unless an invalid parameter for the algorith is chosen vs invalid data.
Also, it's much easier to return a series of 0 in the function and have those converted to the flag vs having to pull in logic from quality_mapping.py to figure out what the flag should be in the except
It will fail with too little data or uneven timestamps or long interval lengths |
Or missing timestamps. pvlib #507 |
The label "cloudy" could be misleading, as it's either not clear, or indeterminate. Maybe reverse that part of the plot and color times labeled as Clear? I do think that most of the validation shouldn't apply to night times. |
An added benefit of inverting to a Clear or Cloud Free flag is that the default behavior of excluding positive quality flags from reports for these periods makes more sense. I.e. exclude the periods of clearsky from a GHI report vs excluding the cloudy periods. |
This is only ok to switch the flag name in qualitymapping.py without making a new version because the flag has not been implemented yet in production data.
|
Ok, I've switched to Cloud Free and only flagging times that the detect clearsky algorithm flags as clear.
We can change from CLOUD FREE, but I thought it was more clear when we already have CLEARSKY EXCEEDED |
|
Hm I would have thought that consistency with the existing CLEARSKY EXCEEDED would be a reason to call this flag CLEAR or CLEARSKY. |
|
fair, CLEAR is too ambiguous, but CLEARSKY is ok |
wholmgren
left a comment
wholmgren
left a comment
There was a problem hiding this comment.
minor comments. didn't look carefully at the tests
| `validator.check_ghi_limits_QCRad`, | ||
| `validator.check_ghi_clearsky` | ||
| `validator.check_ghi_clearsky`, | ||
| `validator.detect_clearsky_ghi` |
There was a problem hiding this comment.
not a big deal, but I believe in numpydoc format the tuple is implied by multiple return values, so proper format would be
Returns
-------
timestamp_flag
bitmask from `validator.check_timestamp_spacing`
night_flag
bitmask from `validator.check_irradiance_day_night`
etc
| vol. 9, no. 4, pp. 998-1005, July 2019. doi: 10.1109/JPHOTOV.2019.2914444 | ||
| """ | ||
| if len(ghi) < 2: | ||
| return pd.Series(0, index=ghi.index) |
3 participants
docs/source/api.rstfor API changes.docs/source/whatsnewfor all changes. Includes link to the GitHub Issue with:issue:`num`or this Pull Request with:pull:`num`. Includes contributor name and/or GitHub username (link with:ghuser:`user`).