Update/calculate hourly airqualitydata using bigqdata #4363
Conversation
…ing_bigqdata Updates from airqo staging
|
Caution Review failedThe pull request is closed. 📝 WalkthroughWalkthroughThis pull request updates the Changes
Sequence Diagram(s)sequenceDiagram
participant Caller
participant DataUtils
participant AirQoApi
Caller->>DataUtils: _fetch_devices_from_api(device_network, device_category)
DataUtils->>DataUtils: Convert device_network using .str
DataUtils->>DataUtils: Convert device_category using .str
DataUtils->>AirQoApi: get_devices_by_network(device_network_str, device_category_str)
AirQoApi-->>DataUtils: Return devices or error
DataUtils-->>Caller: Return API response
Possibly related PRs
Suggested reviewers
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
src/workflows/airqo_etl_utils/datautils.py (1)
684-684: Refactor network-specific calculations.The TODO comment suggests finding a better place for network-specific calculations. Consider extracting these calculations into a dedicated NetworkSpecificProcessor class.
Would you like me to propose a design for handling network-specific calculations?
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/workflows/airqo_etl_utils/datautils.py(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Analyze (python)
- GitHub Check: Analyze (javascript)
🔇 Additional comments (4)
src/workflows/airqo_etl_utils/datautils.py (4)
145-146: LGTM! Clean and consistent enum value handling.The changes ensure consistent string representation of enum values by explicitly accessing the
.strattribute. This is a good practice as it makes the code more robust against potential changes in enum string representations.
436-436: Consider moving parameter mappings to configuration.The TODO comment suggests cleaning up the parameter mappings. Consider moving these mappings to a configuration file or constants module for better maintainability.
Would you like me to help create a configuration structure for these parameter mappings?
663-663: Implement structured data quality checks.The TODO comment indicates a need for a more robust implementation of raw data quality checks. Consider implementing a dedicated data quality validation framework.
Would you like me to propose a structured approach for implementing data quality checks?
153-157: Well-implemented error handling.The error handling throughout the file is robust, with:
- Specific exception types
- Detailed error messages
- Proper logging with context
- Appropriate fallback behavior
Description
Just some clean up
Summary by CodeRabbit
RuntimeErrorwhen device data fetching fails, improving awareness of issues during data extraction.