bpfsnake - #107
Open
varun-r-mallya wants to merge 38 commits into
Open
bpfsnake#107varun-r-mallya wants to merge 38 commits into
varun-r-mallya wants to merge 38 commits into
Conversation
Ports that import from vmlinux need the same treatment as tests/passing_tests/vmlinux/ -- skipped, not failed, when no vmlinux.py has been generated for the running kernel. Without this they fail outright wherever one is absent, CI included. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ports of programs from tools/testing/selftests/bpf/progs/ in the Linux tree,
each naming its upstream original in a header comment.
tracing/tracepoint_sched_switch.py test_tracepoint.c
tracing/get_cgroup_id.py get_cgroup_id_kern.c
tracing/autoattach.py test_autoattach.c
Deliberately a small spike rather than bulk coverage. The three sample different
global-variable shapes, since substituting for globals is what porting the rest
of the corpus will mostly consist of: none at all (the control case), a scalar
read plus a scalar write, and two flags written from two programs on two
different attach points.
They also widen the range of program types under test.
tracepoint/sched/sched_switch and raw_tp/sys_enter were previously unexercised;
@section writes its string straight into the ELF with no allowlist, so that
pass-through had only ever been tested against a handful of types.
Only the BPF half of each selftest is ported -- upstream pairs every program
with a userspace driver in prog_tests/ that loads and asserts, whereas this
framework compiles and verifies but never runs. tests/README.md now says so
explicitly, along with the WORKAROUND(globals) convention marking each map
substituted for a global.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ported faithfully from test_perf_skip.c, which reads the sampled instruction pointer out of a perf_event context: ctx.regs.ip. That is two levels of struct field access and PythonBPF supports one, so this lands as a strict expected failure describing the gap. _allocate_for_attribute in allocation_pass.py declines to allocate unless the attribute's base is a plain Name, so the nested form is skipped. One level on the same context works today -- ctx.sample_period compiles and llc's fine. Worth noting for whoever picks this up: the failure surfaces as 'SyntaxError: Undefined variable actual', naming the assignment target rather than the nested access responsible. The allocation pass declines quietly and the expression pass then trips over the missing symbol. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The four ports were the experiment; this is the result. Covers whether LLM-assisted porting is worth continuing (yes, in feature-sized increments), what a passing port actually proves (a compiler assertion, not a behavioural one), and the four distinct global-variable shapes the spike turned up. Includes the finding that libbpf implements globals as single-element BPF_MAP_TYPE_ARRAY maps, so a one-element ArrayMap -- not a HashMap -- is the structurally faithful stand-in, and that the ELF half of global support already works today via the machinery behind @bpfglobal. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ate-replication-g23gij # Conflicts: # tests/test_config.toml
…scalars The selftest spike substituted a one-entry HashMap for every upstream global because PythonBPF had no global variable support. Integer-scalar @bpfglobal support has since landed, so the three affected ports now declare the upstream globals directly: get_cgroup_id reads expected_pid and writes cg_id, autoattach shares prog1_called/prog2_called between two programs, and perf_skip reads ip. perf_skip stays a strict xfail on the nested ctx.regs.ip access. Update the tests README and porting notes accordingly; the WORKAROUND tag no longer exists in the tree. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DLMW3X7opqAPH6bjmTVkr7
A sub-register context field is loaded zero-extended to i64, and the assignment path accepted it only into a 64-bit slot or one whose type was exactly the field's declared type. The second case never worked in practice: `data_end = skb.data_end` into a c_uint32 global died in llvmlite with "cannot store i64 to i32*" because the value was still i64. Route the store through convert(), sizing from the physical value and signing from the field, like every other integer store since the signedness work: a no-op into an i64 slot, a trunc into anything narrower. Found by porting cgroup_skb_direct_packet_access.c, which is that exact statement. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DLMW3X7opqAPH6bjmTVkr7
Every program on the selftest audit's Tier 1 and Tier 2 lists that the compiler can express today: xdp_dummy, priv_prog, test_xdp_link, xdp_tx, tc_dummy, test_tc_bpf, cgroup_mprog, cgroup_skb_direct_packet_access, test_signed_loader, test_signed_loader_data, test_netfilter_link_attach, kprobe_multi_empty, uprobe_multi_bench, uprobe_multi_usdt, test_link_pinning and test_xdp_devmap_helpers. Fifteen pass at every level, bringing tc, tcx, cgroup/getsockopt, cgroup_skb, socket, netfilter, kprobe.multi, uprobe.multi, usdt and tp_btf program types under test for the first time. test_xdp_devmap_helpers is a negative fixture upstream: reading egress_ifindex is only legal with expected_attach_type = BPF_XDP_DEVMAP and the driver asserts the plain load fails. It is a strict verifier-level xfail here for the same reason, until expected_attach_type can be expressed. The porting notes record the batch and why each remaining program on the two lists is still out of reach. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DLMW3X7opqAPH6bjmTVkr7
The porting assessment was produced by a one-off script that scored every program under tools/testing/selftests/bpf/progs against the compiler's envelope; the notes recommended keeping it so the numbers can be regenerated after each feature lands. This is that script, rebuilt: it strips comments, skips include-shims and header-only fixtures, and reports per program the hard blockers (language gaps) and soft flags (porting costs), with a histogram and a near-miss listing. The helper, map and construct lists at the top are the envelope and are maintained by hand. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DLMW3X7opqAPH6bjmTVkr7
veristat_foo (three socket programs) is clean. test_perf_link, test_enable_stats and test_cgroup_link count with __sync_fetch_and_add, which becomes a plain `x += 1` tagged WORKAROUND(atomics) for a mechanical sweep once atomics exist. connect4_dropper compares against bpf_htons(port), written out as shifts on the low 16 bits. All five pass at IR, llc and verifier level, and bring perf_event, raw_tracepoint/sys_enter, cgroup_skb/egress and cgroup/connect4 under test. The porting notes gain the third batch, the atomics tag, the current blocker histogram from the checked-in audit tool, and how to re-run it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DLMW3X7opqAPH6bjmTVkr7
The annotation types the slot, so `x: c_int32 = 0` is an i32 where the literal alone would give an i64; the store goes through the same handle_variable_assignment plain assignment uses. A bare `x: T` makes the slot and binds nothing. The already-bound and global-shadowing bookkeeping that Assign allocation did inline moves into _bind_name, which both statements now call.
handle_if processed its branches without passing ret_type down, so a return inside an if fell back to process_stmt's i64 default: a -> c_int32 function got `ret i64 7`, which llc rejects. The branches now go through process_block, which threads ret_type and stops at a statement that ends the block instead of emitting after its terminator.
`for i in range(...)` steps a hidden induction counter, allocated in the entry block next to the loop variable, and copies it into the variable each iteration, so rebinding the variable in the body leaves the trip count alone as in Python (and the loop stays visibly bounded for the verifier). The counter is 64-bit, signed unless the bounds make C's arithmetic unsigned. Bounds are evaluated once, before the loop; the step must be a nonzero integer literal, because its sign picks the loop test. `while` re-evaluates its test at the top of each iteration. `break` and `continue` branch through a loop-target stack on the compilation context; outside a loop they are a SyntaxError, as in Python. A loop's else-branch runs on normal exit only. Anything but range() as the iterable is rejected with NotImplementedError. The allocation pass now descends into loop bodies and counts helper temps in loop headers. Checked against clang -O2 on tests/c-form/loops.bpf.c: the constant-bound cases fold to the same ret, and a helper-bounded loop gets the same rotated loop.
- passing_tests/loops/: the seven range/while/break/continue/nested tests move over from failing_tests/, joined by rebinding the loop variable, a negative step, loop else-branches, and a helper-bounded loop over a @bpfglobal that survives -O2 as a real loop for the verifier. - failing_tests/loops/: break outside a loop, and a non-literal range step. - for_map_items stays xfail: map iteration needs its own design. - c-form/loops.bpf.c: the clang reference the lowering was checked against.
The remove-tabs pre-commit hook rejects tab indentation outside docs/ and Makefiles, which failed the Format job.
except Exception turned every import-time defect in the generated module into a skip of every vmlinux test, so CI could pass with no vmlinux coverage. ImportError (no module) skips; anything else propagates. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- The subprog_call pattern also matched SEC("?..."), so every
non-autoloaded program counted as a BPF-to-BPF call blocker, in
contradiction with the soft rule for the same syntax.
- The kfunc pattern matched any bpf_xdp_* or bpf_skb_* call, which are
ordinary helpers (bpf_skb_store_bytes is in SUPPORTED_HELPERS). Only
the kfunc families keep those prefixes; an unsupported helper is still
a hard blocker, classified as one.
- strip_comments removed // inside string literals, truncating section
names like SEC("uprobe//proc/self/exe:func") and classifying the file
as a shim. Strings are matched first and kept.
- Every bpf_map_* helper not in SUPPORTED_HELPERS was dropped before the
unsupported-helper test, so bpf_map_push_elem and friends never showed
up as blockers. The membership test classifies them now.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
_allocate_for_call recognised only c_int32, c_int64, c_uint32, c_uint64 and c_void_p, so queue = c_uint16(0) fell through to an i64 slot and a later store of a u32 field into it never narrowed. Any integer ctypes constructor now declares a slot of its width, which is what the ctx_field_narrow_store test claimed to pin; the IR assertion pins it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The Field branch of handle_variable_assignment rebuilt the field's type by name and duplicated the integer convert() path, and ctypes_to_ir raises NotImplementedError for a pointer or array ctype before the logged error is reached. An integer Field is now normalised to its declared IntTy (field_int_type) before the integer path, in both variable and struct-field assignment, so ctx fields go through the same convert() call as everything else; the Field branch keeps only the non-integer error. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
process_array_map was a stub raising NotImplementedError while ArrayMap was exported from pythonbpf.maps as if usable. The lowering is the hash map's with a different type constant: the same lookup/update/delete helpers, and the kernel fixes the key at a 4-byte index. The lookup allocation path accepts ARRAY alongside HASH, the audit tool lists ARRAY as supported, and the array_map_lookup_update case is a passing test now instead of a strict xfail. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every xfail is strict with raises=Exception, so a verifier-level negative fixture counted any rejection as the expected one, including one caused by a codegen regression, or by bpftool failing. A verifier entry may now carry match = "..."; the harness attaches it as a verifier_match marker and test_kernel_verifier fails, through pytest.fail (not swallowed by raises=Exception), when the program is rejected for any other reason. xdp_devmap_helpers pins its documented "invalid bpf_context access". Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
helpers.py defined all five and return_utils mapped them, but only XDP_DROP and XDP_PASS were exported, so the xdp_tx port had to import XDP_TX from vmlinux and live under vmlinux/, skipped on any host without a generated module. It imports from pythonbpf.helper and lives under xdp/ now, returning XDP_TX bare like its siblings and the C original, which is the shape the return fast path resolves without vmlinux. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This reverts commit 55fb6d6.
get_typed_operand fell through its Name branch for a name that is not a local, a global or a vmlinux constant, and raised the generic "Unsupported operand type" meant for unknown node kinds. Since return goes through it, forgetting `from vmlinux import XDP_PASS` produced that message; it says "Undefined variable XDP_PASS" now, like every other read. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
# Conflicts: # tests/test_config.toml
A non-char-array destination is sized from its pointee, and a 64-bit integer source (a pt_regs field) is the address to read from, as with (void *)ctx->si in C. Also export probe_read_kernel from pythonbpf.helper; it had an emitter but no importable name.
add_debug_info caches nodes by operands, so a second program with a same-named context of the same vmlinux type got the first one's variable, already scoped to the first function. Scoping it again made a metadata cycle that llvmlite recursed on forever.
With a map lookup on both sides, both operands are pointers of one type, so the depth-normalising path emitted a pointer compare. Load both sides through get_typed_operand instead, which also picks the predicate from the map's declared value sign.
A pygame window takes the keyboard from the terminal, so the pty_write probe would never see a key. Python writes the key code into the state map instead, one queued turn per tick, ignoring reversals. The board grows to 20x30 with food anywhere inside the walls, and the snake to 32.
Both front ends share one Game that queues turns, drives the kernel's tick and reads the maps back. The terminal one reads keys in cbreak mode (an arrow's last byte is already the kernel's key code) and draws on the alternate screen, so it needs no pygame; pygame is imported only for the window.
…pt does The terminal mode prints the original's 12x50 board of #, $ and @ from the top-left corner and exits when the snake dies. Both modes go back to the original's board size and BEGIN state, since the walls live in the kernel's tick.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.