Send generated functions to the line that declared them - #110
JesseHerrick wants to merge 12 commits into
Conversation
|
I tried the Ash half of this. Findings, in case they change the plan:
defmodule Gen do
defmacro gen(name, override?) do
quote bind_quoted: [name: name, override?: override?], location: :keep do
if override?, do: @file({"/user/domain.ex", 42})
def unquote(name)(x) do
r = String.to_integer(x) # line 7 of gen.ex
r + 1
end
end
end
end
# stamped frame: {User, :stamped, 1, [file: ~c"/user/domain.ex", line: 7]}What works with this PR as it is: keep Ash issue: ash-project/ash#2970, PR: ash-project/ash#2971. One small thing I hit while testing: this branch indexes the enclosing checkout's |
A function a macro generated has no source definition, so go-to-definition
fell back to the top of its module. The compiled module's debug info records
where each def came from: its :line, and a `file: {path, line}` entry left by
`@file` or `quote location: :keep`. When that path is the module's own source,
the line is where the code asked for the function; when it names the
generator's own file, :line is used instead.
beam.ReadDefinitionLines reads those locations from the Dbgi chunk and steps
over clause bodies without allocating. generatedDefinitionResultsFor uses a
recorded line only when the debug info was compiled from the file being
opened, the BEAM is not older than it, and the line falls after the module's
own line. Every other case keeps the module result.
Bare-call definition and call-hierarchy preparation call it directly.
Qualified definition, the references declaration, and `dexter lookup` reach
it through LookupName, which takes a recorded line even for a strict lookup,
since that line is the function's own definition.
No framework is recognized by name. Ash code interfaces (#108) resolve to
their `define` line once Ash includes ash-project/ash#2971, which expands
each generated def at that line; with released Ash they resolve to line 1 as
before.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
1af27b8 to
8df255d
Compare
Drop the stale-BEAM guard. Dexter cannot compile the project, so a BEAM older than its source is the usual state while editing, and its line is closer than the module line. From #102: - Keep the Docs chunk anno as beam.Function.Line, and use it when a module has no debug info. The CInf chunk's :source (beam.ReadSourcePath) says which file that line is in. - Send a generated module with no source row, such as a Spark entity module, to the file it was compiled from, rebased onto the project's lib or deps when it was built elsewhere. Move the definition-line code into generated_definition.go. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When the source, or an unsaved buffer, has changed since the BEAM was written, move each recorded line to the nearest call whose first argument is the function's name as an atom (`define :list_rooms`). That is the generic shape of a macro call that declares a name. Keep the recorded line when it still declares the function, when no line does, or when two matches are equally near. A current BEAM is never corrected. Also say that the Docs anno is often, but not always, the Dbgi line: Ash's code interfaces give the def its define line and the docs line 1. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Follow drift before the "after the module line" check, so lines added above defmodule no longer drop the recorded line. - Search to both ends of a file that got shorter than the recorded line, and drop a line past the end with nothing to move to. - Treat only open buffers as unsaved edits; a file cached by another request is not one. - Skip heredoc lines and module attributes when looking for a declaration. - Prefer this checkout over the recorded path when a _build was copied from another checkout or worktree, and try the recorded path's tails under the root first, which covers umbrella apps and moved projects. - Decode CInf charlists as Unicode codepoints, so non-ASCII paths work. - Bound the Dbgi definitions preallocation against a corrupt count. - Do not overwrite a newer BEAM's cache entry with an older one. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…file - Follow a stale line only to a declaration in the body of the module that owns it, found with the tokenizer, so a sibling module in the same file that declares the same name is never the answer. A generated module with no source row has no owner in the generator's file, so its line is not corrected there. - When a module has rows in more than one file, as two umbrella apps can, choose the row that shares the most trailing path components with the recorded source. If two rows match equally, keep the module line. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A generator can give a def any line, as `quote line: 99` does, so even a current BEAM can record a line the file does not have. Check each line against the current text, open buffer or disk, and fall back to the module line when it is past the end. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- A function whose only recorded line is the module line, such as one a
@before_compile hook made, goes to the one call in its module that
declares it by name. This also places released Ash code interfaces.
- A function with a clause per DSL call goes to every clause.
- A stale line with no name to search for is placed by the source
definitions around it, from every module in the file; when an edit is
between them, only a one-to-one match to `def unquote` lines counts.
- A BEAM's mtime has whole seconds, so compare staleness by the second;
a save in the same second as the compile no longer looks stale.
- While a buffer has unsaved edits, the module line comes from the buffer.
- Go-to-definition on a module that exists only as a BEAM goes to the file
and line the compiler recorded for it.
- A bare call to a generated function of an imported module resolves.
- Read {line, column} annotations.
- Add integration tests that compile a DSL fixture of each shape with mix,
and run them in the integration job.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Anchors that moved together are the most exact signal, so they now come before the search for a call that spells the function's name. That search is skipped for a function whose clauses several calls made: those calls do not spell its name, so a call that does, such as `plug :match`, would pull every route clause onto one line. Also test the cases from review: route clauses next to `plug :match` in a stale file, a multi-clause `location: :keep` generator, and lines removed above the module in an unsaved buffer. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A stale BEAM's line is now used as it is. Only edits to the declaring file move it, and those files (DSL modules, schemas, routers, dependencies) change less often than their callers; the next compile makes it exact. Correcting it meant guessing at the text the BEAM was compiled from, and each signal either gave up often or could send the editor to a line that looked meaningful and was not. What stays: a line past the end of the file is never returned, the module line check compares lines from the same compile, and the search for the one declaring call, which reads the current text, still places a def made at the module line. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
There are 3 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 477f738. Configure here.
The module line came from the index when the BEAM had none for the module, as for a module a macro nested in its parent. The index has the current text, so after lines were added above defmodule it could fall after a recorded line from the compile and drop it. Take the parent's line from the parent's BEAM, and use the index only when no BEAM has one. Do not search for a declaring call for a function whose clauses were made at several lines: those calls do not spell its name, so a call that does, such as `plug :match`, is not where they were declared. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
blankHeredocs blanked the whole line that opens a heredoc, so a call such as `define :foo, description: """` was not found by the declaring-call search. The opening line now keeps its text before the delimiter. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Closes #108. Supersedes #102, which is folded in here. Go-to-definition for DSL-generated functions such as Ash code interfaces. It does not special-case any framework. The Ash side is ash-project/ash#2971, which is merged and will ship in the next Ash release after v3.33.11. With released Ash, a code interface goes to its
defineline too, from the generic declaring-call search described below.Summary
A function that a macro generates has no source definition, so go-to-definition fell back to the top of its module. The compiled module's
Dbgichunk records the location of each def::lineis the line in the module's own file that was compiled when the def was made. Usually this is the macro call or the line where a before-compile hook ran, but a generator can expand the def at any line.file: {path, line}entry comes from@fileor fromquote location: :keep. Whenpathis the module's own source, this line is where the code asked for the function. Whenpathis another file, it is the generator's own implementation, and:lineis used instead.Released Ash records
line: 1, file: {"deps/ash/lib/ash/code_interface.ex", N}for every generated def. After ash#2971, Ash expands each generated def at itsdefineline, so the debug info isline: <define line>, file: {"deps/ash/lib/ash/code_interface.ex", N}. The foreign-file rule above picks up:line. Ash did not use@file, because the function body's lines still come fromcode_interface.ex, so stacktraces would show the user's file with Ash's line numbers. Their approach keeps stacktraces unchanged.Changes
beam.ReadDefinitionLineswalks{:debug_info_v1, :elixir_erl, {:elixir_v1, map, specs}}and reads onlyfile,relative_file, anddefinitions. It steps over clause bodies without allocating. Clause ASTs nest much deeper than Docs terms, so this reader has a higher ETF depth guard (maxDebugInfoDepth, 2048). Other readers keep the default of 64. Modules without Elixir debug info (debug_info: false, Erlang modules, no chunk) return an error and keep the fallback.generatedDefinitionResultsForuses a recorded line only when all of these conditions are true:In all other cases, the result stays as before. A BEAM older than the source still gives its line. Dexter cannot compile the project, so a stale BEAM is the usual state while you edit, and the line from the last compile is closer than the module line. Only edits to the declaring file move it, and the next compile makes it exact. The line is not corrected for those edits: every way to do that meant guessing at the compiled text, and could send the editor to a wrong line that looked right.
beam.Function.Linekeeps the Docs chunk anno, andbeam.ReadSourcePathreads:sourcefrom theCInfchunk. The Docs anno is often the same line as the Dbgi:line, but not always: Ash's code interfaces give the docs line 1. It is used when a module has no debug info, andCInfsays which file the line is in.lib/<app>,deps/<app>/lib/<app>, ordeps/<app>. The app name comes from the recorded path, because Spark builds Ash's entity modules in Ash's ebin from Spark's source. If no file is found, the lexical parent stays the answer.dexter lookupget it throughLookupName(shared since Unify CLI and editor semantic navigation #107). A recorded line is the function's own definition, soLookupNamereturns it for a strict (ExactModule) lookup too. Without a recorded line, strict and fallback lookups do not change.Also in this PR
@before_compilehook, or released Ash), the module's body is searched for the call whose first argument is the function's name as an atom. Only a single match counts.route :get,route :post) goes to every clause, from the line Dbgi records for each clause.quote line: 99) is dropped.Module.create) goes to the file and line the compiler recorded.TestDefinition_GeneratedFunctionsFromCompiler*compile a DSL fixture of each shape withmix(with and without debug info, and with a stale source) and run in the integration job.Performance
A cold read on real Ash modules takes 0.8–2 ms (inflate plus walk;
Repro.Chat's Dbgi is 19 KB compressed and 350 KB inflated). It happens once per BEAM stamp and only on a definition miss for a generated function. Indexing does not change. No parser or schema changes, so noIndexVersionbump is necessary.Validation
go test ./...andmake lintpass.internal/beamtests: own-file location, foreign location, plain:line, macros and private defs, stripped or Erlang debug info, truncation at every seventh byte, a clause body 600 ETF levels deep, and a test that compiles real modules withelixirc(skipped if Elixir is not installed) for@file,location: :keep, and plain quotes. It also checks that the Docs anno and theCInfsource agree with the debug info.@fileshape and the Ash shape (:linewith a generatorfile).LookupNamereturns the recorded line with and withoutExactModule, and a strict lookup without a recorded line keeps the module. A stale BEAM goes to the line it recorded, and never to a line past the end of the file. A foreign location with no useful:line, and debug info from a different source file, keep the module line. The Docs anno gives the line when there is no Dbgi, and does not when there is noCInfsource. A sourceless generated module goes to its rebaseddepsfile from Dbgi or Docs, and to its lexical parent when that file is missing.main(9fa5088, includes ash#2971), withlspprobefor definition anddexter lookup --strictfor the CLI. Both give the same lines:Chat.get_room_by_slug!chat.ex:1chat.ex:7Chat.create_roomchat.ex:1chat.ex:8Repro.Chat.Room.rename(resourcecode_interface)room.ex:1room.ex:19End to end on a project with no dependencies and a DSL in the project, for macros that are not Ash (
maingives line 1 ofuser.exfor all of them excepthello):quote(field :email)user.ex:4location: :keepuser.ex:6@file {file, line}user.ex:8__using__injectiondsl.ex:5(unchanged, thedefin the quote)defmodulefrom a macrouser.ex:10Module.createin the generator's file (no source row)dsl.ex:43The same results hold for a copy of the project that was not recompiled, for an umbrella copied the same way (
apps/<app>/lib), and for a project in a directory named withéand日本.🤖 Generated with Claude Code
Note
Medium Risk
Touches core LSP navigation and BEAM parsing with many edge cases (stale BEAM, moved trees, ambiguous umbrella paths); risk is mitigated by extensive tests but wrong jumps remain possible on unusual layouts.
Overview
Go-to-definition for macro-generated functions (Ash
define, DSL macros, etc.) no longer stops at the top of the module. Dexter reads BEAMDbgifor per-def lines (:line,@file/location: :keep), with Docs anno andCInf:sourcewhen debug info is missing, and maps recorded paths onto the current checkout when artifacts were built elsewhere.Navigation is centralized in
generatedDefinitionResultsFor, used by LSP definition, call hierarchy,dexter lookup(including strict), and bare/imported generated calls. When the BEAM only has the module line, a declaring-call search finds a uniquemacro :namesite; multi-clause DSL routes return every clause line. Sourceless modules (Module.create, Spark entities) jump to rebased framework files underdeps/.New
internal/beamreaders (ReadDefinitionLines,ReadSourcePath) and a large test suite includemix-compiled integration (TestDefinition_GeneratedFunctionsFromCompiler) wired in CI.Reviewed by Cursor Bugbot for commit d1fa86f. Bugbot is set up for automated code reviews on this repo. Configure here.