Skip to content

Fix TableEngine construction for alembic-rendered zero-arg engines and SummingMergeTree columns - #947

Open
agu2347 wants to merge 4 commits into
ClickHouse:mainfrom
agu2347:fix-tableengine-noarg-and-summing-columns
Open

Fix TableEngine construction for alembic-rendered zero-arg engines and SummingMergeTree columns#947
agu2347 wants to merge 4 commits into
ClickHouse:mainfrom
agu2347:fix-tableengine-noarg-and-summing-columns

Conversation

@agu2347

@agu2347 agu2347 commented Aug 11, 2026

Copy link
Copy Markdown

Closes #946.

Two independent bugs in clickhouse_connect/cc_sqlalchemy/ddl/tableengine.py:

1. Zero-arg engines can't be constructed from their own repr().

Memory, Log, StripeLog, TinyLog, Null, and Set don't define their own __init__, so they fall through to TableEngine.__init__(self, kwargs), which had no default for kwargs. repr(Memory({})) already produced "Memory()", but Alembic's autogeneration writes exactly that no-arg call into the migration file, and running that migration executes Memory() -- which raised TypeError: __init__() missing 1 required positional argument: 'kwargs'.

Fixed by giving kwargs a None default, normalized to {} inside the constructor.

2. SummingMergeTree has no way to accept a column list.

ClickHouse's SummingMergeTree([columns]) engine takes an optional list of columns to sum on merge (e.g. SummingMergeTree((delta, n_tx)) ORDER BY id), but the Python class was a bare class SummingMergeTree(MergeTree): pass -- no columns parameter existed anywhere in the constructor chain.

Fixed by giving SummingMergeTree its own constructor, following the same pattern already used by ReplacingMergeTree/CollapsingMergeTree/etc: extend TableEngine directly (not MergeTree, whose narrower signature has no room for columns), add columns to arg_names as an optional positional arg, and extend TableEngine.__init__'s positional-arg rendering to accept a tuple/list value and render it as a parenthesized column list -- mirroring how eng_params already renders tuples for ORDER BY/PARTITION BY.

Testing

Added 6 tests to tests/unit_tests/test_sqlalchemy/test_alembic.py:

  • test_no_arg_table_engines_accept_zero_args -- all six zero-arg engines construct and round-trip.
  • test_memory_alembic_repr_round_trips -- the exact scenario from the issue: repr(Memory({})) produces code that evals back cleanly.
  • test_summing_merge_tree_accepts_columns -- SummingMergeTree(columns=(...), order_by=...) compiles to valid DDL.
  • test_summing_merge_tree_columns_optional -- columns remains optional.
  • test_summing_merge_tree_requires_order_by_or_primary_key -- existing MergeTree-family constraint preserved.
  • test_summing_merge_tree_repr_round_trips -- repr -> eval round trip with columns.

All 455 unit tests in tests/unit_tests/test_sqlalchemy/ pass (449 existing + 6 new), confirmed with no regressions. Reverting the fix makes 5 of the 6 new tests fail with the exact symptoms described in the issue (the 6th doesn't exercise the changed code path, so it correctly still passes). ruff check is clean on both changed files.

…d SummingMergeTree columns

Two bugs in clickhouse_connect/cc_sqlalchemy/ddl/tableengine.py, both from ClickHouse#946:

1. TableEngine.__init__(self, kwargs) had no default for `kwargs`. Engines
   with no required/optional constructor args (Memory, Log, StripeLog,
   TinyLog, Null, Set) don't define their own __init__, so they fall
   straight through to TableEngine.__init__. repr(Memory({})) already
   produced "Memory()", but calling that rendered code back -- exactly what
   Alembic does when it writes a migration from repr() -- raised
   "TypeError: __init__() missing 1 required positional argument: 'kwargs'".
   Fixed by giving kwargs a None default that's normalized to {} inside the
   constructor.

2. SummingMergeTree had no way to receive the column list that ClickHouse's
   own SummingMergeTree([columns]) engine accepts (e.g.
   SummingMergeTree((delta, n_tx)) ORDER BY id). It was a bare
   `class SummingMergeTree(MergeTree): pass`, so there was no `columns`
   parameter anywhere in its signature. Fixed by giving it its own
   constructor -- following the same pattern already used by
   ReplacingMergeTree/CollapsingMergeTree/etc: extend TableEngine directly
   (not MergeTree, whose narrower signature has no room for `columns`),
   add `columns` to arg_names as an optional positional arg, and extend
   TableEngine.__init__'s positional-arg rendering to accept a tuple/list
   value and render it as a parenthesized column list, matching how
   eng_params already renders tuples for ORDER BY/PARTITION BY.

Testing: added 6 tests to tests/unit_tests/test_sqlalchemy/test_alembic.py
covering both bugs, including the exact repr(Memory({})) -> eval() round
trip from the issue and a SummingMergeTree repr -> eval round trip. All 455
unit tests pass (449 existing + 6 new). Reverting the fix makes 5 of the 6
new tests fail with the exact symptoms from the issue. ruff check is clean.
@CLAassistant

CLAassistant commented Aug 11, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ joe-clickhouse
❌ agu2347
You have signed the CLA already but the status is still pending? Let us recheck it.

@joe-clickhouse joe-clickhouse left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @agu2347 looks good! I added a few finishing touches that do the following:

  • Preserves the existing SummingMergeTree positional API and MergeTree inheritance
  • adds matching columns= support for ReplicatedSummingMergeTree
  • validates column-list inputs
  • address missing type declarations
  • adds the CHANGELOG entry

To get this merged, please sign the CLA. thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TableEngine renders broken alembic migration for some engine types

3 participants