Skip to content

[refactor](fe) Add fe-foundation module with zero-dependency JDK-only utilities#61175

Merged
CalvinKirs merged 4 commits intoapache:masterfrom
CalvinKirs:master-commom-spi-properties
Mar 11, 2026
Merged

[refactor](fe) Add fe-foundation module with zero-dependency JDK-only utilities#61175
CalvinKirs merged 4 commits intoapache:masterfrom
CalvinKirs:master-commom-spi-properties

Conversation

@CalvinKirs
Copy link
Copy Markdown
Member

fe-common has accumulated heavy transitive dependencies (Guava, Hadoop, Trino SPI, ANTLR, Alibaba SDK, etc.), making it unsuitable for lightweight consumers such as SPI plugins and fe-extension modules. This creates a new fe-foundation module that sits at the very bottom of the FE dependency hierarchy (fe-foundation < fe-extension-spi < fe-common < fe-core) and carries zero third-party compile dependencies.

Scope of this first iteration:

  • Create the fe-foundation Maven module with zero-dependency pom.xml
  • Move 10 pure-JDK utility classes from fe-common and fe-core into new org.apache.doris.foundation.{format,property,type,util} packages: FormatOptions, ResultOr, ConnectorProperty, ConnectorPropertiesUtils, ParamRules, StoragePropertiesException, BitUtil, ByteBufferUtil, PathUtils, SerializationUtils
  • Replace trivial Guava/commons-lang3 usage in ConnectorPropertiesUtils with JDK equivalents (Sets.newHashSet -> new HashSet, StringUtils -> private isNotBlank method)
  • Update all import statements across ~80 consuming files
  • Add fe-foundation as dependency in fe-common and fe-core pom.xml

Classes intentionally left in fe-common for future phases:

  • Pair/Triple: use Gson @SerializedName, persisted in editlog metadata
  • Writable and IO interfaces: 194+ importers deeply embedded in the persistence/catalog layer

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@CalvinKirs CalvinKirs requested a review from morningman as a code owner March 10, 2026 07:59
@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

1 similar comment
@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

… utilities

fe-common has accumulated heavy transitive dependencies (Guava, Hadoop,
Trino SPI, ANTLR, Alibaba SDK, etc.), making it unsuitable for lightweight
consumers such as SPI plugins and fe-extension modules. This creates a new
fe-foundation module that sits at the very bottom of the FE dependency
hierarchy (fe-foundation < fe-extension-spi < fe-common < fe-core) and
carries zero third-party compile dependencies.

Scope of this first iteration:
- Create the fe-foundation Maven module with zero-dependency pom.xml
- Move 10 pure-JDK utility classes from fe-common and fe-core into new
  org.apache.doris.foundation.{format,property,type,util} packages:
  FormatOptions, ResultOr, ConnectorProperty, ConnectorPropertiesUtils,
  ParamRules, StoragePropertiesException, BitUtil, ByteBufferUtil,
  PathUtils, SerializationUtils
- Replace trivial Guava/commons-lang3 usage in ConnectorPropertiesUtils
  with JDK equivalents (Sets.newHashSet -> new HashSet, StringUtils ->
  private isNotBlank method)
- Update all import statements across ~80 consuming files
- Add fe-foundation as dependency in fe-common and fe-core pom.xml

Classes intentionally left in fe-common for future phases:
- Pair/Triple: use Gson @SerializedName, persisted in editlog metadata
- Writable and IO interfaces: 194+ importers deeply embedded in the
  persistence/catalog layer
@CalvinKirs CalvinKirs force-pushed the master-commom-spi-properties branch from a151f03 to 5974b5b Compare March 10, 2026 10:08
@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 27790 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ee80f6adb99cd0a1abc74413d35d2fc1b85752b1, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17635	4483	4297	4297
q2	q3	10644	825	518	518
q4	4687	361	251	251
q5	7556	1211	1029	1029
q6	174	177	151	151
q7	789	893	668	668
q8	9288	1439	1327	1327
q9	4765	4793	4796	4793
q10	6243	1898	1664	1664
q11	459	257	236	236
q12	708	580	473	473
q13	18034	2957	2186	2186
q14	235	241	221	221
q15	924	801	797	797
q16	744	725	672	672
q17	717	860	430	430
q18	6004	5384	5239	5239
q19	1123	1008	611	611
q20	511	496	391	391
q21	4365	2089	1547	1547
q22	431	331	289	289
Total cold run time: 96036 ms
Total hot run time: 27790 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4700	4592	4550	4550
q2	q3	3891	4338	3823	3823
q4	866	1276	785	785
q5	4112	4409	4362	4362
q6	184	176	141	141
q7	1786	1682	1559	1559
q8	2536	2738	2735	2735
q9	7458	7323	7388	7323
q10	3810	4124	3648	3648
q11	526	436	430	430
q12	556	644	440	440
q13	2652	3130	2324	2324
q14	286	303	274	274
q15	865	828	814	814
q16	733	750	697	697
q17	1152	1491	1388	1388
q18	7151	6980	6580	6580
q19	884	878	881	878
q20	2156	2204	2173	2173
q21	4015	3534	3374	3374
q22	457	412	363	363
Total cold run time: 50776 ms
Total hot run time: 48661 ms

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 153482 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ee80f6adb99cd0a1abc74413d35d2fc1b85752b1, data reload: false

query5	4327	637	515	515
query6	335	219	209	209
query7	4213	464	270	270
query8	352	252	243	243
query9	8691	2726	2700	2700
query10	535	383	335	335
query11	7324	5897	5666	5666
query12	189	126	122	122
query13	1269	460	360	360
query14	5725	3891	3572	3572
query14_1	2831	2822	2799	2799
query15	214	192	178	178
query16	994	484	446	446
query17	882	721	624	624
query18	2455	449	354	354
query19	215	215	190	190
query20	133	135	134	134
query21	228	144	125	125
query22	5237	5222	5047	5047
query23	15898	15690	15343	15343
query23_1	15494	16388	15981	15981
query24	7340	1699	1309	1309
query24_1	1311	1285	1308	1285
query25	610	571	488	488
query26	1285	284	161	161
query27	3432	501	331	331
query28	4756	1950	1845	1845
query29	875	607	472	472
query30	316	244	215	215
query31	1371	1279	1228	1228
query32	81	69	72	69
query33	507	318	272	272
query34	914	938	553	553
query35	623	671	588	588
query36	1111	1161	993	993
query37	138	91	79	79
query38	2905	2866	2892	2866
query39	899	865	837	837
query39_1	855	819	820	819
query40	229	153	134	134
query41	61	59	58	58
query42	304	298	303	298
query43	247	248	222	222
query44	
query45	191	232	183	183
query46	882	984	603	603
query47	2117	2177	2056	2056
query48	305	323	230	230
query49	630	455	380	380
query50	707	274	210	210
query51	4129	4143	4066	4066
query52	290	290	281	281
query53	287	346	278	278
query54	285	262	264	262
query55	95	86	78	78
query56	310	323	296	296
query57	1375	1346	1285	1285
query58	289	278	277	277
query59	1332	1441	1317	1317
query60	328	332	321	321
query61	148	146	145	145
query62	611	588	535	535
query63	306	276	273	273
query64	5057	1251	976	976
query65	
query66	1481	462	349	349
query67	16519	16460	16394	16394
query68	
query69	386	303	288	288
query70	1011	979	921	921
query71	333	303	295	295
query72	2772	2654	2503	2503
query73	543	552	317	317
query74	9979	9961	9804	9804
query75	2852	2744	2472	2472
query76	2287	1029	684	684
query77	366	373	305	305
query78	11417	11428	10743	10743
query79	2019	763	594	594
query80	1491	623	532	532
query81	575	282	240	240
query82	1014	147	116	116
query83	378	263	245	245
query84	300	121	103	103
query85	889	477	429	429
query86	439	302	300	300
query87	3192	3159	3061	3061
query88	3507	2645	2625	2625
query89	423	365	342	342
query90	1969	171	168	168
query91	163	156	134	134
query92	76	80	73	73
query93	1132	872	499	499
query94	660	307	301	301
query95	588	404	328	328
query96	646	510	231	231
query97	2452	2531	2441	2441
query98	230	222	216	216
query99	1009	992	917	917
Total cold run time: 235026 ms
Total hot run time: 153482 ms

@github-actions
Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Mar 10, 2026
@github-actions
Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@CalvinKirs CalvinKirs merged commit 1a250a6 into apache:master Mar 11, 2026
29 of 31 checks passed
morrySnow pushed a commit that referenced this pull request Mar 12, 2026
…actor (#61262)

## Problem
- commit #61175 moved `ConnectorProperty` from
`org.apache.doris.datasource.property` to
`org.apache.doris.foundation.property`
- `PaimonJdbcMetaStoreProperties` still imported the old package
- FE UT on latest master then failed with `cannot find symbol: class
ConnectorProperty`

## Fix
- switch `PaimonJdbcMetaStoreProperties` to import
`org.apache.doris.foundation.property.ConnectorProperty`

## Validation
- `./run-fe-ut.sh --run
org.apache.doris.datasource.property.metastore.PaimonJdbcMetaStorePropertiesTest`
CalvinKirs added a commit to CalvinKirs/incubator-doris that referenced this pull request Apr 20, 2026
### What problem does this PR solve?

Issue Number: None

Related PR: apache#61175

Problem Summary: Fix the import ordering in InternalCatalog after the fe-foundation backport so fe-core checkstyle passes on branch-4.1.

### Release note

None

### Check List (For Author)

- Test: FE checkstyle
    - mvn -pl fe-core -am -DskipTests checkstyle:check -Dcheckstyle.consoleOutput=true
- Behavior changed: No
- Does this need documentation: No
CalvinKirs added a commit to CalvinKirs/incubator-doris that referenced this pull request Apr 20, 2026
### What problem does this PR solve?

Issue Number: None

Related PR: apache#61175

Problem Summary: Update the branch-4.1-only Paimon JDBC metastore properties class to import ConnectorProperty from fe-foundation and drop an accidentally added VarBinaryLiteralAnalysisTest that does not exist on branch-4.1, restoring FE build success after the fe-foundation backport.

### Release note

None

### Check List (For Author)

- Test: FE build
    - env DORIS_PARALLELISM=5 bash build.sh --fe -j5
- Behavior changed: No
- Does this need documentation: No
yiguolei pushed a commit to CalvinKirs/incubator-doris that referenced this pull request Apr 24, 2026
### What problem does this PR solve?

Issue Number: None

Related PR: apache#61175

Problem Summary: Fix the import ordering in InternalCatalog after the fe-foundation backport so fe-core checkstyle passes on branch-4.1.

### Release note

None

### Check List (For Author)

- Test: FE checkstyle
    - mvn -pl fe-core -am -DskipTests checkstyle:check -Dcheckstyle.consoleOutput=true
- Behavior changed: No
- Does this need documentation: No
yiguolei pushed a commit to CalvinKirs/incubator-doris that referenced this pull request Apr 24, 2026
### What problem does this PR solve?

Issue Number: None

Related PR: apache#61175

Problem Summary: Update the branch-4.1-only Paimon JDBC metastore properties class to import ConnectorProperty from fe-foundation and drop an accidentally added VarBinaryLiteralAnalysisTest that does not exist on branch-4.1, restoring FE build success after the fe-foundation backport.

### Release note

None

### Check List (For Author)

- Test: FE build
    - env DORIS_PARALLELISM=5 bash build.sh --fe -j5
- Behavior changed: No
- Does this need documentation: No
yiguolei pushed a commit that referenced this pull request May 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/4.1.1-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants