Repository navigation
Translate date arithmetic written with INTERVAL operators - #510
samuelmbabhazi wants to merge 1 commit into
Conversation
MySQL accepts `expr + INTERVAL n unit`, `expr - INTERVAL n unit` and `INTERVAL n unit + expr`, and the parser already does too, but the translator passed the expression through unchanged, so SQLite rejected it with a syntax error. The three forms now share the DATETIME() translation used by DATE_ADD() and DATE_SUB(), which gains two fixes on the way: a negative value no longer produces a modifier that SQLite rejects, and a computed value is parenthesized so that `||` does not bind to it first. Fixes WordPress#454 Fixes WordPress#90
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan provides up to 2 included reviews per hour; 1 remains after this review. 📝 WalkthroughWalkthroughThe translator now supports MySQL ChangesInterval arithmetic support
Priority: ➖ Normal Estimated code review effort: 3 (Moderate) | ~25 minutes Change: Bug fix · Severity of issue fixed: Medium Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning Some tools did not complete. Review the errors below. 🔧 PHPMD (2.15.0)packages/mysql-on-sqlite/tests/WP_MySQL_On_SQLite_Tests.phpPHPMD could not process this file (exit code 255): PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in phar:///usr/bin/phpmd/vendor/pdepend/pdepend/src/main/php/PDepend/Util/Cache/Driver/FileCacheDriver.php on line 209 Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
A real-world case for this PR: WooCommerce's checkout stock hold uses the operator form.
SELECT %d, %d, %d, NOW(), ( NOW() + INTERVAL %d MINUTE ) FROM DUALOn With this PR, the statement translates. Happy to test a rebased version against the WooCommerce checkout if that helps. |
|
Thanks @Nandan108, good case. Both WooCommerce statements, The branch is current with |
JanJakes
left a comment
There was a problem hiding this comment.
Thanks for working on this. I drafted a longer review a while ago, but I think a single summary will be more useful.
I don’t think this can be merged in the current form. Some queries that previously failed with a syntax error now run and return incorrect results. In UPDATE and DELETE, that can mean changing or deleting data.
The main problem is that the translation now relies on the parse tree grouping, but our grammar doesn’t consistently represent MySQL’s operator precedence. Previously, the translator preserved the token order and left precedence to SQLite. Wrapping parts of that tree in DATETIME() changes how the expressions are evaluated.
To solve this, we either need to make the grammar represent MySQL precedence correctly, or we could try to regroup the expressions during translation. The new parser implemented in #429 solves all the precedence issues, so a full fix can also mean waiting for the parser switch. We could consider a focused fix here; I'm exploring some options in #539.
Below is a summary of the problems with the current approach.
Main issues
-
The prefix form absorbs logical operators.
In MySQL, the operand after
+stops beforeAND,OR, andXOR. Here, they can end up insideDATETIME():DELETE FROM t WHERE INTERVAL 30 DAY + d < '2025-01-01' AND n = 99; -- Translated as: DELETE FROM t WHERE DATETIME(d < '2025-01-01' AND n = 99, (30) || ' DAY');
SQLite turns the boolean result into an ancient date string, which it treats as true. This can delete every row, while MySQL deletes none.
-
Numeric date operands have different meanings in MySQL and SQLite.
MySQL does include comparisons in the prefix operand, so:
WHERE INTERVAL 1 DAY + d > '2029-01-01'
means:
WHERE INTERVAL 1 DAY + (d > '2029-01-01')
MySQL treats the resulting
0or1as an invalid date and returnsNULL. SQLite interprets it as a Julian day and produces a date string that counts as true. The same problem occurs withIS NULL.DATE_ADD()already has this limitation, but the new prefix form exposes it directly inWHEREclauses. Numeric dates need MySQL’sYYYYMMDD[hhmmss]interpretation, with invalid values returningNULL. -
Unsupported units silently produce
NULL.This affects
QUARTER,MICROSECOND, composite units such asHOUR_MINUTE, andSQL_TSI_*aliases. For example:UPDATE t SET d = d + INTERVAL 1 QUARTER;
sets
dtoNULLinstead of adding three months. Before this PR, the query failed without changing the data.Please resolve units by token type, translate the supported ones, and reject the rest explicitly.
MICROSECONDalso needs special handling becauseDATETIME()drops fractional seconds. -
Mixed ordinary and interval arithmetic is grouped incorrectly.
With
dholding theDATETIMEvalue2024-01-01 10:00:00:d + INTERVAL 1 DAY + 0becomesd + DATETIME(0, ...), returning-2689instead of20240102100000.1 + d + INTERVAL 1 DAYbecomes1 + DATETIME(d, ...), returning2025instead of2024-01-02 10:00:01.
Other findings
- Interval values need MySQL-compatible conversion. For example, numeric
INTERVAL 1.5 DAYadds two days in MySQL but 36 hours here. MySQL also acceptsINTERVAL '1 DAY' DAYas one day; this translation returnsNULL. - The tests need to cover these failure cases, including logical operators, mixed arithmetic,
UPDATE/DELETE, and unsupported or aliased units. - The
translate_bit_expr()docblock says date arithmetic is the only unsupported form, butDIV,MOD, and^also fail in SQLite. - The
INTERVALforms ofADDDATE()andSUBDATE()could share the new helper; they still fail with a syntax error.
|
Thanks for the detailed review, Jan. Agreed: with the current grammar the grouping can't be trusted, and the prefix form and unit handling turned syntax errors into wrong data. #539 is the right way forward; I'll test the WooCommerce reservation case from Nandan against it and report there. |
|
@samuelmbabhazi I'm now exploring an even simpler alternative that would cover the main cases in suffix forms like |
|
@JanJakes Suffix-only with an explicit rejection of the prefix form sounds right to me, and the cases we have in hand all fit it: #454 is |
|
@samuelmbabhazi I repushed #539 with a more focused solution. I'll close this in favor of #539. |
## Summary This PR **adds date arithmetic with `INTERVAL`**, building on #510 by @samuelmbabhazi: ```sql SELECT d + INTERVAL 1 DAY; SELECT d - INTERVAL 5 HOUR; SELECT d + INTERVAL 1 DAY - INTERVAL 2 HOUR; ``` These queries previously failed with a syntax error. They now use SQLite’s `DATE()` and `DATETIME()` functions, sharing the translation with `DATE_ADD()` and `DATE_SUB()`. ## Why the extra fixes Our current parser groups some expressions differently from MySQL. Translating them directly can produce wrong results—or update or delete the wrong rows. This PR adds safeguards to the implementation in #510: - **Reject expressions the parser cannot group correctly**, including `INTERVAL 1 DAY + d`, `1 + d + INTERVAL 1 DAY`, and `d + INTERVAL 1 DAY + x`. Errors suggest adding parentheses. - **Support `QUARTER` and `SQL_TSI_*` units**, and handle negative and computed interval values. - **Reject unsupported units**, including `MICROSECOND` and composite units such as `HOUR_MINUTE`, instead of silently returning `NULL`. - **Return dates for date-only inputs**, as MySQL does. Previously, `'2024-01-01' + INTERVAL 1 DAY` returned `2024-01-02 00:00:00`, so comparisons with `DATE` values could match the wrong rows. This also fixes `DATE_ADD()` and `DATE_SUB()`, the first case of #531. The supported forms cover the linked issues. The expression restrictions can be revisited after the planned switch to a parser based on MySQL’s own grammar. ## Remaining limitations The existing date translation still differs from MySQL in these cases: - **Numeric dates:** Values such as `20240101` are not interpreted as MySQL dates. - **Interval values:** MySQL converts the value to a whole number. Here, `INTERVAL 1.5 DAY` adds 36 hours instead of 2 days, and a string such as `' 1'` returns `NULL` with `+` but works with `-`. - **Time inputs:** `TIME` values return datetime strings, and times past midnight wrap around instead of going over 24 hours. - **Month ends:** Month-end arithmetic can overflow into the following month. - **`ADDDATE()` and `SUBDATE()`:** These functions remain unsupported. Month ends and `ADDDATE()`/`SUBDATE()` are tracked in #531. Expected results in the new tests were checked against MySQL 8.4. Fixes #454 Fixes #90 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Improved support for MySQL date and time interval calculations. - Added support for interval units from seconds through years, including weeks and quarters. - Supports chained additions and subtractions, computed intervals, and parenthesized expressions. - Date and datetime results are now handled appropriately across supported calculations. - **Bug Fixes** - Improved validation and handling of unsupported or ambiguous interval expressions. - Preserved correct NULL behavior and date calculations in updates and default values. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Samuel Mbabhazi <samuelmbabhazi@gmail.com>
Fixes #454
Fixes #90
MySQL accepts date arithmetic written with the
INTERVALoperators, in three forms:expr + INTERVAL n unit,expr - INTERVAL n unitandINTERVAL n unit + expr. The parser already accepts them, but the translator passed the expression through unchanged, so SQLite failed withnear "5": syntax erroron the example from #454, and on theNOW() - INTERVAL 1 DAYfrom #90.The three forms now go through the same
DATETIME()translation asDATE_ADD()andDATE_SUB(), extracted intotranslate_interval_arithmetic():bitExprbranch, which folds each interval into the expression on its left, so a chain likeexpr + INTERVAL 5 HOUR + INTERVAL 1 DAYbecomes nestedDATETIME()calls applied in MySQL's order;translate_simple_expr_body(), next to the existingBINARY,CASTandCONVERTbranches.Sharing the helper also fixes two cases where
DATE_ADD()andDATE_SUB()diverged from MySQL:INTERVAL -5 HOUR, produced the modifier'+-5 HOUR', which SQLite rejects, so the result wasNULL;INTERVAL 2 * 3 HOUR, produced'-' || 2 * 3 || ' HOUR', and since||binds tighter than*in SQLite the result wasNULLas well.The value is now parenthesized and subtraction negates it instead of prefixing
-. As a result, the modifier emitted for a column default such asDATE_ADD(NOW(), INTERVAL 1 YEAR)changes from'+' || 1 || ' YEAR'to(1) || ' YEAR'; the pinned assertion intestCreateTableWithExpressionDefaultis updated accordingly. Both forms are valid SQLite, so existing databases are unaffected.Testing
Every expected value in the new tests was taken from MySQL 8.4 by running the same statements, including the
NULLcases, theWEEKconversion, a lowercase unit, a string value, and the column andWHEREcases intestIntervalArithmeticOnColumns.Without the source change, the three new tests fail: the two infix tests with SQLite's syntax error, the
DATE_ADD()one on the negative value.MySQL rejects
INTERVAL 5 HOUR - exprwith a syntax error, and so does the parser here, so that form is left alone.Not changed, and shared with the existing
DATE_ADD()translation: a date-only input yields aDATETIMEstring where MySQL returns aDATE, month arithmetic does not clamp to the end of the month ('2024-01-31' + INTERVAL 1 MONTHgives2024-03-02, MySQL gives2024-02-29), and theQUARTERand composite units such asHOUR_MINUTEare not translated. Those are worth their own change.Summary by CodeRabbit
New Features
INTERVALdate arithmetic in SQLite-compatible queries.WEEKintervals.Bug Fixes
DATE_ADD()andDATE_SUB()expressions, including negative intervals and null values.