Feature Request: PandasFeatureUnion

[mattayes]

Hi folks:

An issue I have with scikit-learn's FeatureUnion is that you can't make it return a DataFrame (unlike with regular transformers). It would be nice to see a variant of FeatureUnion that worked nicely with Pandas workflows.

Here's a prototype of what I'm thinking:

class PandasFeatureUnion(FeatureUnion):
    """FeatureUnion which returns a DataFrame."""
    
    def _to_dataframe(self, X):
        columns = [name for (name, _) in self.transformer_list]
        return pd.DataFrame(X, columns=columns)
    
    def transform(self, X):
        result = super().transform(X)
        return self._to_dataframe(result)
    
    def fit_transform(self, X):
        result = super().fit_transform(X)
        return self._to_dataframe(result)

You could imagine using it in a manner similar to this example. I see this as a complement to the existing DataFrameMapper.

My example above doesn't handle indexes yet and I'd love some advice on how to implement it (ideally without having to rewrite most of FeatureUnion. Here are some concerns I have now:

• Should all the indexes have to match up?
• If not, how should joins be handled?
• Should PandasFeatureUnion accept an ignore_index=True argument?

[dukebody]

Hi @mattayes . The example you link does mainly the same thing as a DataFrameMapper. However you're right that DataFrameMapper doesn't return a pandas dataframe either. Is this what you would want?

If so, I believe you can join forces with @jph00 working at #70

[mattayes]

Hey @dukebody, having that as a part of sklearn-pandas is great and I'm glad it's been merged in! However, the thing I'm really looking for is a way to create a feature using DataFrameMapper that uses more than one column. For example, combining year, month, and day columns into a created_at column. Right now DataFrameMapper only handles applying transformers to a single column, correct?

[dukebody]

Sorry for the long delay.

You can apply transformers to a single column using a DataFrameMapper, see https://github.com/paulgb/sklearn-pandas#transform-multiple-columns. Is this what you want?

[mattayes]

@dukebody yes, that is what I want! Sorry for making you point it out in the docs, I guess I missed it 😔. I see that #70 is merged. Is there anything else I could help on?

[dukebody]

Check any of the issues listed in https://github.com/paulgb/sklearn-pandas/issues and see if you can help with any of them. Thanks!