|
| 1 | +# Conversion between libraries |
| 2 | + |
| 3 | +Some library maintainers must apply complex dataframe operations, using methods and functions that may not (yet) be implemented in Narwhals. In such cases, Narwhals can still be highly beneficial, by allowing easy dataframe conversion. |
| 4 | + |
| 5 | +## Dataframe X in, pandas out |
| 6 | + |
| 7 | +Imagine that you maintain a library with a function that operates on pandas dataframes to produce automated reports. You want to allow users to supply a dataframe in any format to that function (pandas, Polars, DuckDB, cuDF, Modin, etc.) without adding all those dependencies to your own project and without special-casing each input library's variation of `to_pandas` / `toPandas` / `to_pandas_df` / `df` ... |
| 8 | + |
| 9 | +One solution is to use Narwhals as a thin Dataframe ingestion layer, to convert user-supplied dataframe to the format that your library uses internally. Since Narwhals is zero-dependency, this is a much more lightweight solution than including all the dataframe libraries as dependencies, |
| 10 | +and easier to write than special casing each input library's `to_pandas` method (if it even exists!). |
| 11 | + |
| 12 | +To illustrate, we create dataframes in various formats: |
| 13 | + |
| 14 | +```python exec="1" source="above" session="conversion" |
| 15 | +import narwhals as nw |
| 16 | +from narwhals.typing import IntoDataFrame |
| 17 | + |
| 18 | +import duckdb |
| 19 | +import polars as pl |
| 20 | +import pandas as pd |
| 21 | + |
| 22 | +df_polars = pl.DataFrame( |
| 23 | + { |
| 24 | + "A": [1, 2, 3, 4, 5], |
| 25 | + "fruits": ["banana", "banana", "apple", "apple", "banana"], |
| 26 | + "B": [5, 4, 3, 2, 1], |
| 27 | + "cars": ["beetle", "audi", "beetle", "beetle", "beetle"], |
| 28 | + } |
| 29 | +) |
| 30 | +df_pandas = df_polars.to_pandas() |
| 31 | +df_duckdb = duckdb.sql("SELECT * FROM df_polars") |
| 32 | +``` |
| 33 | + |
| 34 | +Now, we define a function that can ingest any dataframe type supported by Narwhals, and convert it to a pandas DataFrame for internal use: |
| 35 | + |
| 36 | +```python exec="1" source="above" session="conversion" result="python" |
| 37 | +def df_to_pandas(df: IntoDataFrame) -> pd.DataFrame: |
| 38 | + return nw.from_native(df).to_pandas() |
| 39 | + |
| 40 | + |
| 41 | +print(df_to_pandas(df_polars)) |
| 42 | +``` |
| 43 | + |
| 44 | +## Dataframe X in, Polars out |
| 45 | + |
| 46 | +### Via PyCapsule Interface |
| 47 | + |
| 48 | +Similarly, if your library uses Polars internally, you can convert any user-supplied dataframe to Polars format using Narwhals. |
| 49 | + |
| 50 | +```python exec="1" source="above" session="conversion" result="python" |
| 51 | +def df_to_polars(df: IntoDataFrame) -> pl.DataFrame: |
| 52 | + return nw.from_arrow(nw.from_native(df), native_namespace=pl).to_native() |
| 53 | + |
| 54 | + |
| 55 | +print(df_to_polars(df_duckdb)) # You can only execute this line of code once. |
| 56 | +``` |
| 57 | + |
| 58 | +It works to pass Polars to `native_namespace` here because Polars supports the [PyCapsule Interface](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html) for import. |
| 59 | + |
| 60 | +Note that the PyCapsule Interface makes no guarantee that you can call it repeatedly, so the approach above only works if you |
| 61 | +only expect to perform the conversion a single time on each input object. |
| 62 | + |
| 63 | +### Via PyArrow |
| 64 | + |
| 65 | +If you need to ingest the same dataframe multiple times, then you may want to go via PyArrow instead. |
| 66 | +This may be less efficient than the PyCapsule approach above (and always requires PyArrow!), but is more forgiving: |
| 67 | + |
| 68 | +```python exec="1" source="above" session="conversion" result="python" |
| 69 | +def df_to_polars(df: IntoDataFrame) -> pl.DataFrame: |
| 70 | + return pl.DataFrame(nw.from_native(df).to_arrow()) |
| 71 | + |
| 72 | + |
| 73 | +df_duckdb = duckdb.sql("SELECT * FROM df_polars") |
| 74 | +print(df_to_polars(df_duckdb)) # We can execute this... |
| 75 | +print(df_to_polars(df_duckdb)) # ...as many times as we like! |
| 76 | +``` |
0 commit comments