-
Notifications
You must be signed in to change notification settings - Fork 1.1k
[CLAUDE] fix(databricks)!: disambiguate 2-arg date_add from 3-arg dateadd #7588
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
RichardHughes-amp
wants to merge
10
commits into
tobymao:main
Choose a base branch
from
RichardHughes-amp:storm-2954-databricks-dateadd-disambiguation
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
3498f3a
fix(databricks)!: disambiguate 2-arg date_add from 3-arg dateadd
RichardHughes-amp 04dddcb
style: use PEP 585 list instead of typing.List in databricks parser
RichardHughes-amp 0a9d39b
fix: remove Databricks DATE_ADD/DATEADD overrides; inherit from Spark…
RichardHughes-amp 8a7555c
test: tighten test_add_date comments and formatting
RichardHughes-amp 3a31422
style: apply ruff-format to test_optimizer.py
RichardHughes-amp 72f2afb
test: tighten date_add annotation tests; alias long type names
RichardHughes-amp fbf7999
test: restore original assertions; expose T-SQL and generator regress…
RichardHughes-amp 0e42e5f
fix(tsql, databricks): add TimestampAdd -> DATEADD generator entries
RichardHughes-amp 1a61431
fix(databricks): restore _build_date_add dispatcher in DatabricksParser
RichardHughes-amp 4611244
fix(databricks): introduce DateAdd2 for 2-arg date_add disambiguation
RichardHughes-amp File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this got a bit too creative 😆 We don't want to split this into multiple AST nodes, the arity is expressed through the optionality (boolean value) of
arg_types.If we go back to the previous version I think we only need to fix the transpilation of Spark/DBX -> T-SQL. This can be done by adding generation support for
exp.TimestampAddin T-SQL e.g:This is mostly what I meant by fixing the regression.
From a quick search, the APIs are the following:
date_add(start, days)(2-arg)dateadd(start, days)(2-arg)date_add(unit, value, expr)(3-arg)dateadd(unit, value, expr)(3-arg)timestampadd(unit, value, expr)And this is what happens on main today afaict, Spark looks more valid than DBX so the latter should consolidate to the former imo:
read="spark"read="databricks"date_add(e, 24)(2-arg)TsOrDsAdd(unit=DAY)DateAdd(unit=DAY)dateadd(e, 24)(2-arg)TsOrDsAdd(unit=DAY)DateAdd(unit=DAY)date_add(MONTH, 1, e)(3-arg)TimestampAdd(unit=MONTH)DateAdd(unit=MONTH)dateadd(MONTH, 1, e)(3-arg)TimestampAdd(unit=MONTH)DateAdd(unit=MONTH)timestampadd(MONTH, 1, e)TimestampAdd(unit=MONTH)TimestampAdd(unit=MONTH)