Skip to content

parse python panics in LSPClient.Locate (slice bounds out of range) when following imports into .venv #186

@Valen-C12

Description

@Valen-C12

parse python panics in LSPClient.Locate (slice bounds out of range [13:6]) when following imports into .venv

Repo: cloudwego/abcoder
Labels: bug, language/python, crash

Summary

When parsing a Python project whose source imports modules from a .venv/ containing Python 3.12 syntax that the bundled jedi 0.19.1 / parso 0.8.4 cannot fully parse, abcoder hard-crashes:

panic: runtime error: slice bounds out of range [13:6]

goroutine 1 [running]:
github.com/cloudwego/abcoder/lang/lsp.(*LSPClient).Locate(...)
    /tmp/abcoder-build/lang/lsp/lsp_methods.go:316 +0x430
github.com/cloudwego/abcoder/lang/collect.(*Collector).getSymbolByLocation(...)
    /tmp/abcoder-build/lang/collect/collect.go:2114 +0xe58
github.com/cloudwego/abcoder/lang/collect.(*Collector).getSymbolByTokenWithLimit(...)
github.com/cloudwego/abcoder/lang/collect.(*Collector).getDepsWithLimit(...)
github.com/cloudwego/abcoder/lang/collect.(*Collector).processSymbol(...)
    /tmp/abcoder-build/lang/collect/collect.go:2245 +0x328
github.com/cloudwego/abcoder/lang/collect.(*Collector).Collect(...)
    /tmp/abcoder-build/lang/collect/collect.go:206 +0x3c0

The panic is preceded by hundreds of warnings like:

[ERROR] collect.go:2306: get type symbol for token {file:///.../.venv/lib/python3.12/site-packages/redis/_parsers/socket.py:115:24-115:32 function [] readline} failed: definition of token ... not found

…and pylsp stderr noise from the pylsp_semantic_tokens hook:

WARNING - pylsp.config.config - Failed to load hook pylsp_semantic_tokens:
  'Newline' object has no attribute 'children'
  AttributeError in jedi/inference/syntax_tree.py:460 (infer_or_test)

jedi/parso returns a malformed range (start 13 > end 6), and lang/lsp/lsp_methods.go:316 slices a string with that range without bounds-checking → panic.

Reproduction

# Project: any Python 3.12 project with a populated .venv that contains
# files using PEP 701 f-strings or other 3.12-only syntax (e.g. attrs, redis).
abcoder parse python . -o /tmp/ast.json
# …jedi follows imports into .venv/...redis/_parsers/socket.py
# panic: runtime error: slice bounds out of range [13:6]

Expected

  • LSPClient.Locate should validate the Range it receives (Start <= End, both within document bounds) and return an error instead of panicking.
  • Higher up, Collector.getSymbolByLocation should swallow such errors as a "skip this token" rather than propagating a panic.
  • Optionally: parse python should default to excluding common virtualenv directories (.venv, venv, env) since they're never the parse target — they're only ever followed transitively, which is what triggers the bad ranges.

Workaround

Pass --exclude .venv (and any other vendored Python trees):

abcoder parse python . -o /tmp/ast.json --exclude .venv

Root cause notes (for triage)

The malformed range originates from jedi's syntax-tree inference on certain 3.12 constructs that parso 0.8.4 represents incorrectly (e.g. Newline nodes appearing where parso normally yields simple_stmt children). Bumping parso to ≥0.8.5 in the bundled pylsp fork's deps may eliminate the source — but the Go side should still bounds-check defensively.

Environment

  • abcoder v0.0.0-20260427034146-9d8ecc5873be+dirty
  • macOS 25.3.0 (arm64)
  • jedi 0.19.1, parso 0.8.4 (in miniconda Python 3.10)
  • Project: Python 3.12 with .venv containing attrs, redis, pydantic, etc.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions