Skip to content

[fix](function) Prevent cross-row rounding in months_between - #67935

Merged
Mryange merged 1 commit into
apache:masterfrom
Mryange:fix-months-between-column-view
Sep 15, 2026
Merged

Mryange merged 1 commit into
apache:masterfrom
Mryange:fix-months-between-column-view

Conversation

@Mryange

@Mryange Mryange commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

When both date arguments are constants and round_off varies by row, a rounded result was reused by later rows, causing round_off = false rows to lose precision. Use ColumnView for consistent constant, regular, and nullable column access, and calculate an independent raw value for every row before optional rounding.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Mryange

Mryange commented Sep 14, 2026

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 100.00% (17/17) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 63.30% (29652/46846)
Line Coverage 48.23% (310018/642752)
Region Coverage 43.75% (249542/570393)
Branch Coverage 45.36% (116245/256260)

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 16679 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1e57ced8498e4b898bd0cf34d24b47fa21748ef8, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17575	3023	3011	3011
q2	2065	253	231	231
q3	10265	853	502	502
q4	4667	244	203	203
q5	7683	564	404	404
q6	135	118	93	93
q7	530	517	388	388
q8	9225	832	882	832
q9	3438	2398	2393	2393
q10	6510	891	724	724
q11	397	195	182	182
q12	618	256	194	194
q13	18144	1518	1137	1137
q14	155	144	140	140
q15	q16	429	417	368	368
q17	1301	851	759	759
q18	3225	2217	2218	2217
q19	1247	862	746	746
q20	360	277	195	195
q21	5592	1732	1817	1732
q22	323	266	228	228
Total cold run time: 93884 ms
Total hot run time: 16679 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3377	3318	3304	3304
q2	503	385	379	379
q3	2220	2326	2110	2110
q4	1194	1152	889	889
q5	2169	2100	2095	2095
q6	169	122	85	85
q7	1033	957	853	853
q8	1582	1403	1385	1385
q9	3115	3104	3084	3084
q10	1900	1818	1620	1620
q11	360	267	248	248
q12	455	423	338	338
q13	1478	1541	1149	1149
q14	166	174	163	163
q15	q16	397	406	356	356
q17	3610	3331	3236	3236
q18	4764	4389	4707	4389
q19	877	783	900	783
q20	1019	958	808	808
q21	3811	3116	3270	3116
q22	393	341	320	320
Total cold run time: 34592 ms
Total hot run time: 30710 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 81773 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1e57ced8498e4b898bd0cf34d24b47fa21748ef8, data reload: false

query5	4244	401	344	344
query6	415	138	119	119
query7	4924	406	239	239
query8	295	131	117	117
query9	8693	2889	2880	2880
query10	396	227	188	188
query11	5378	1042	909	909
query12	122	72	70	70
query13	1198	457	321	321
query14	6020	2183	2083	2083
query14_1	1986	1967	1977	1967
query15	175	118	111	111
query16	901	349	349	349
query17	789	448	372	372
query18	2335	330	237	237
query19	162	143	114	114
query20	71	69	71	69
query21	201	107	87	87
query22	5482	5420	5348	5348
query23	6778	6293	6002	6002
query23_1	6000	6217	6094	6094
query24	7287	1077	760	760
query24_1	791	759	770	759
query25	436	292	248	248
query26	1233	235	129	129
query27	2779	457	251	251
query28	4665	1491	1479	1479
query29	896	420	323	323
query30	250	158	129	129
query31	813	393	333	333
query32	130	72	66	66
query33	449	207	182	182
query34	998	828	469	469
query35	392	397	356	356
query36	562	574	505	505
query37	119	81	68	68
query38	1019	844	810	810
query39	523	482	460	460
query39_1	456	455	481	455
query40	203	86	73	73
query41	52	50	51	50
query42	72	72	71	71
query43	240	235	210	210
query44	975	515	547	515
query45	107	104	98	98
query46	789	839	521	521
query47	764	756	732	732
query48	316	297	228	228
query49	527	243	181	181
query50	742	260	197	197
query51	8207	8132	8163	8132
query52	66	72	62	62
query53	189	194	149	149
query54	196	166	145	145
query55	71	61	60	60
query56	183	187	166	166
query57	687	690	640	640
query58	191	165	149	149
query59	1219	1236	1082	1082
query60	213	171	188	171
query61	113	111	101	101
query62	341	204	177	177
query63	181	142	135	135
query64	2732	656	626	626
query65	1600	1595	1661	1595
query66	1835	301	236	236
query67	9961	9771	9714	9714
query68	2769	1167	742	742
query69	353	232	202	202
query70	654	629	612	612
query71	249	175	174	174
query72	2282	1848	1489	1489
query73	646	601	349	349
query74	1574	1215	1141	1141
query75	1164	1074	955	955
query76	2280	715	498	498
query77	259	253	210	210
query78	3870	3638	3193	3193
query79	2826	837	588	588
query80	1648	312	264	264
query81	510	154	132	132
query82	605	141	99	99
query83	277	208	189	189
query84	292	107	88	88
query85	808	337	280	280
query86	463	169	163	163
query87	1023	970	888	888
query88	2920	2096	2091	2091
query89	279	195	169	169
query90	2008	135	129	129
query91	125	115	94	94
query92	91	69	61	61
query93	1955	1070	675	675
query94	600	242	225	225
query95	496	252	227	227
query96	787	552	263	263
query97	1066	1055	996	996
query98	176	139	131	131
query99	414	337	314	314
Total cold run time: 178070 ms
Total hot run time: 81773 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.94 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1e57ced8498e4b898bd0cf34d24b47fa21748ef8, data reload: false

query1	0.00	0.00	0.01
query2	0.08	0.04	0.04
query3	0.25	0.12	0.11
query4	1.60	0.09	0.10
query5	0.17	0.15	0.16
query6	1.27	0.68	0.70
query7	0.03	0.00	0.01
query8	0.04	0.03	0.03
query9	0.28	0.21	0.22
query10	0.36	0.35	0.35
query11	0.17	0.11	0.13
query12	0.15	0.13	0.12
query13	0.32	0.32	0.34
query14	0.47	0.46	0.46
query15	0.36	0.37	0.36
query16	0.22	0.22	0.24
query17	0.70	0.72	0.67
query18	0.18	0.16	0.16
query19	1.23	1.15	1.19
query20	0.02	0.01	0.00
query21	15.44	0.16	0.12
query22	5.07	0.05	0.04
query23	16.18	0.26	0.09
query24	3.05	0.32	0.25
query25	0.11	0.04	0.03
query26	0.74	0.17	0.13
query27	0.04	0.03	0.03
query28	3.66	0.56	0.28
query29	12.44	3.27	2.57
query30	0.25	0.12	0.12
query31	2.75	0.38	0.18
query32	3.50	0.34	0.24
query33	1.39	1.52	1.45
query34	15.36	2.29	1.87
query35	1.79	1.77	1.85
query36	0.46	0.30	0.29
query37	0.06	0.04	0.04
query38	0.04	0.03	0.03
query39	0.04	0.02	0.03
query40	0.12	0.08	0.08
query41	0.08	0.03	0.03
query42	0.03	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 90.53 s
Total hot run time: 14.94 s

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (17/17) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.28% (34613/45379)
Line Coverage 61.20% (388924/635471)
Region Coverage 57.33% (325732/568188)
Branch Coverage 58.25% (148852/255520)

@Mryange
Mryange force-pushed the fix-months-between-column-view branch from 1e57ced to 0d667ca Compare September 14, 2026 16:33
@Mryange

Mryange commented Sep 14, 2026

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (17/17) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.12% (34252/44997)
Line Coverage 61.05% (384697/630106)
Region Coverage 57.33% (323088/563600)
Branch Coverage 58.13% (147171/253192)

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 16714 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0d667cad314c7aa49852cc8a80643a914aa39169, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17569	3035	3040	3035
q2	2109	243	221	221
q3	10253	864	514	514
q4	4678	247	204	204
q5	7677	569	375	375
q6	134	113	92	92
q7	510	529	384	384
q8	9228	881	899	881
q9	3458	2397	2351	2351
q10	6528	870	723	723
q11	385	196	177	177
q12	615	262	195	195
q13	18136	1524	1155	1155
q14	157	149	137	137
q15	q16	435	389	371	371
q17	1407	906	761	761
q18	3080	2274	2228	2228
q19	1256	895	766	766
q20	377	280	202	202
q21	5628	1715	1877	1715
q22	325	267	227	227
Total cold run time: 93945 ms
Total hot run time: 16714 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3408	3351	3327	3327
q2	498	389	382	382
q3	2241	2414	2167	2167
q4	1173	1153	875	875
q5	2165	2145	2100	2100
q6	168	120	85	85
q7	1019	894	892	892
q8	1596	1389	1378	1378
q9	3129	3080	3076	3076
q10	1828	1832	1611	1611
q11	358	271	245	245
q12	448	430	342	342
q13	1470	1534	1174	1174
q14	180	172	159	159
q15	q16	389	402	363	363
q17	3540	3269	3232	3232
q18	4756	4409	4745	4409
q19	862	806	958	806
q20	1017	962	828	828
q21	3814	3092	3202	3092
q22	405	345	309	309
Total cold run time: 34464 ms
Total hot run time: 30852 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 81701 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0d667cad314c7aa49852cc8a80643a914aa39169, data reload: false

query5	4237	413	336	336
query6	379	137	132	132
query7	4954	410	226	226
query8	292	130	123	123
query9	8683	2876	2888	2876
query10	417	242	188	188
query11	5383	1039	918	918
query12	120	71	69	69
query13	1192	464	330	330
query14	6178	2204	2081	2081
query14_1	1964	1960	1947	1947
query15	186	119	115	115
query16	919	366	352	352
query17	798	448	358	358
query18	2337	296	219	219
query19	165	133	103	103
query20	69	69	79	69
query21	196	98	90	90
query22	5337	5373	5389	5373
query23	6762	6228	5922	5922
query23_1	6097	6165	5921	5921
query24	7250	1093	761	761
query24_1	803	776	744	744
query25	395	285	240	240
query26	1240	216	127	127
query27	2806	421	249	249
query28	4703	1488	1505	1488
query29	910	415	335	335
query30	247	154	132	132
query31	811	408	332	332
query32	131	68	79	68
query33	454	210	167	167
query34	980	843	485	485
query35	392	406	338	338
query36	578	581	517	517
query37	119	77	69	69
query38	994	841	841	841
query39	491	478	487	478
query39_1	470	482	491	482
query40	202	87	77	77
query41	53	53	50	50
query42	81	74	73	73
query43	240	240	211	211
query44	989	533	537	533
query45	112	105	98	98
query46	783	818	517	517
query47	773	764	712	712
query48	318	323	237	237
query49	529	246	190	190
query50	748	269	193	193
query51	8186	8099	8271	8099
query52	81	72	64	64
query53	188	200	159	159
query54	208	164	172	164
query55	84	64	60	60
query56	185	196	192	192
query57	705	645	733	645
query58	197	170	177	170
query59	1234	1221	1100	1100
query60	255	193	177	177
query61	150	108	111	108
query62	364	222	184	184
query63	176	136	141	136
query64	2824	719	581	581
query65	1627	1596	1656	1596
query66	1823	273	217	217
query67	9703	9723	9625	9625
query68	2995	1146	762	762
query69	331	244	195	195
query70	659	616	616	616
query71	243	179	162	162
query72	2221	1680	1529	1529
query73	629	592	336	336
query74	1995	1228	1132	1132
query75	1187	1097	957	957
query76	2359	709	501	501
query77	249	258	214	214
query78	4031	3628	3178	3178
query79	2882	770	597	597
query80	1561	324	275	275
query81	508	151	136	136
query82	623	134	102	102
query83	282	207	187	187
query84	299	112	87	87
query85	785	332	282	282
query86	471	175	171	171
query87	1023	964	896	896
query88	3454	2135	2114	2114
query89	285	195	175	175
query90	2201	126	126	126
query91	132	119	98	98
query92	93	70	71	70
query93	3268	1059	689	689
query94	672	241	232	232
query95	510	243	219	219
query96	846	554	284	284
query97	1064	1101	1016	1016
query98	181	151	136	136
query99	423	350	309	309
Total cold run time: 181143 ms
Total hot run time: 81701 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.69 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0d667cad314c7aa49852cc8a80643a914aa39169, data reload: false

query1	0.01	0.00	0.00
query2	0.08	0.04	0.03
query3	0.24	0.11	0.11
query4	1.64	0.10	0.09
query5	0.17	0.16	0.15
query6	1.28	0.68	0.69
query7	0.03	0.01	0.00
query8	0.04	0.03	0.03
query9	0.28	0.22	0.21
query10	0.34	0.33	0.34
query11	0.16	0.12	0.11
query12	0.16	0.12	0.12
query13	0.32	0.31	0.30
query14	0.44	0.46	0.45
query15	0.35	0.38	0.34
query16	0.22	0.23	0.21
query17	0.71	0.67	0.70
query18	0.17	0.16	0.17
query19	1.17	1.18	1.21
query20	0.02	0.01	0.01
query21	15.48	0.16	0.13
query22	5.06	0.04	0.05
query23	16.16	0.25	0.10
query24	3.01	0.34	0.26
query25	0.10	0.05	0.03
query26	0.79	0.17	0.12
query27	0.03	0.04	0.03
query28	3.66	0.54	0.28
query29	12.44	3.15	2.57
query30	0.25	0.12	0.12
query31	2.77	0.36	0.17
query32	3.54	0.34	0.21
query33	1.48	1.64	1.49
query34	15.34	2.18	1.75
query35	1.74	1.75	1.71
query36	0.45	0.30	0.28
query37	0.06	0.04	0.04
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.11	0.08	0.07
query41	0.08	0.03	0.02
query42	0.03	0.02	0.03
query43	0.03	0.03	0.02
Total cold run time: 90.51 s
Total hot run time: 14.69 s

@Mryange

Mryange commented Sep 15, 2026

Copy link
Copy Markdown
Contributor Author

run cloud_p0

@Mryange

Mryange commented Sep 15, 2026

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated static review of exact head 0d667cad314c7aa49852cc8a80643a914aa39169.

Request changes for two issues: the broad rewrite removes the source-level both-date-constant fast path, and it changes binary64 results for asymmetric one-constant calls that could never enter the reported bug path. Both can be avoided by retaining the existing specialized/vector paths and making the both-constant raw result immutable before applying per-row rounding.

Critical checkpoint conclusions:

  • Goal/correctness: the row-local value does stop rounded state from leaking into a later unrounded row. Default null handling, ColumnView indexing/lifetimes, zero rows, and DATEV2/TIMESTAMP_NS dispatch are otherwise sound.
  • Test proof/results: the added DATEV2 case reaches the target state transition (k0=3 true then k0=6 false in the same tiny bucket-6 block) and its new rounded/raw values are correct. The unrelated changes to existing unrounded outputs expose the numerical issue called out inline.
  • Scope/minimality: not yet satisfactory; a narrow immutable-precompute fix avoids both accepted regressions.
  • Concurrency/lifecycle/memory/error handling: no shared state, locking, lifecycle, ownership, allocation-accounting, ignored-Status, or exception-boundary issue was found.
  • Configuration/compatibility/parallel paths: no config, persistence, storage/protocol, rolling-upgrade, transaction, data-write, or FE-BE variable change applies. Exact floating-result compatibility is affected as noted inline; DATEV2 and TIMESTAMP_NS otherwise follow the same safe loop.
  • Observability: no new logging or metric is warranted for this scalar path.
  • Performance: invariant raw-date work now executes per row instead of once per block, including two out-of-line TIMESTAMP_NS conversions.
  • User focus: no additional focus was provided.

Review status: complete and converged after three rounds; all candidates were independently validated, deduplicated, accepted, or dismissed, with no unresolved suspicious point. No local build or test was run because the review contract prohibits builds. The pure-text BE header-hygiene gate passed; exact-head CI reports Compile, BE UT, and P0 successful, while the cloud P0 rerun is still pending.

Comment thread be/src/exprs/function/function_date_or_datetime_computation.h
-1.90322581 0.90322581 \N \N
-12.96774194 \N -12.96774193548387 \N
-13.87096774 0.87096774 -13.87096774193548 \N
-12.96774194 \N -12.967741935483872 \N

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Avoid changing unaffected unrounded results

These oracle changes come from calls with only one constant date (the queries at Groovy lines 79-83), so they can never enter the buggy date_consts[0] && date_consts[1] path. The rewrite nevertheless changes the returned binary64 values: for example, the old -12.96774193548387 is the nearest double to exact -402/31, while the new -12.967741935483872 is the adjacent one-ULP value; 340/31 changes the same way. This is unrelated numerical compatibility drift, and some cases become less accurate. Please keep the asymmetric/vector calculation path unchanged and narrowly fix the both-date-const path by copying an immutable precomputed raw value before per-row rounding, then restore the unaffected expected results.

@Mryange
Mryange dismissed github-actions[bot]’s stale review September 15, 2026 04:01

两个都是常量的,那很罕见了

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (17/17) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.12% (34252/44997)
Line Coverage 61.05% (384697/630106)
Region Coverage 57.33% (323088/563600)
Branch Coverage 58.13% (147171/253192)

@Mryange
Mryange merged commit ab20858 into apache:master Sep 15, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants