-
-
Notifications
You must be signed in to change notification settings - Fork 19.5k
Closed
Labels
DeprecateFunctionality to remove in pandasFunctionality to remove in pandasNeeds DiscussionRequires discussion from core team before further actionRequires discussion from core team before further action
Description
I think a case could be made for deprecating pd.options.mode.use_inf_as_na. Few thoughts:
- Doesn't seem necessary or particularly useful: it's easy to get the same behavior by checking for NA or infinity.
- Source of bugs: e.g., BUG: pandas.options.mode.use_inf_as_na disables NA checking for StringArray #33655, BUG: pandas 1.0 dropna error with categorical data if pd.options.mode.use_inf_as_na = True #33594. If an array has a cached mask attribute there's no guarantee that it's correct:
import numpy as np import pandas as pd arr = pd.Categorical([1.0, 2.0, np.inf]) pd.options.mode.use_inf_as_na = True arr.isna().any() # False
- General maintenance cost: several null-checking functions either have to be duplicated or equipped with boolean arguments to switch their behavior (e.g., _isna and _isna_ndarraylike from /core/dtypes/missing.py, or isnaobj and isnaobj_old from /_libs/missing.pyx). These could all be cleaned up if NA had one meaning.
Thoughts?
Metadata
Metadata
Assignees
Labels
DeprecateFunctionality to remove in pandasFunctionality to remove in pandasNeeds DiscussionRequires discussion from core team before further actionRequires discussion from core team before further action