Add Debugging Interface

[G-Yong]

Add a debugging interface to the existing architecture.
Using the added debugging interface, we implemented debugging for QuickJS in VSCode. The project address is:[QuickJS-Debugger]

[saghul]

At a quick glance, this looks better than the other approaches I've seen, kudos!

Now, since this will have a performance impact, I'd say we want it gated with a compile time time macro.

@bnoordhuis thoughts?

[G-Yong]

Thanks for the feedback @saghul! I've added a compile-time macro QJS_ENABLE_DEBUGGER to gate all the debug-related code. Here's a summary of the changes:

Compile-time gating (QJS_ENABLE_DEBUGGER)

• All debug fields in JSContext, all debug API implementations (JS_SetOPChangedHandler, JS_GetStackDepth , JS_GetLocalVariablesAtLevel, JS_FreeLocalVariables ) , and the per-opcode callback in JS_CallInternal are now wrapped in #ifdef QJS_ENABLE_DEBUGGER.
• The declarations in quickjs.h are similarly guarded.
• When QJS_ENABLE_DEBUGGER is defined, DIRECT_DISPATCH is automatically set to 0 (alongside the existing EMSCRIPTEN and _MSC_VER conditions), so the switch-based dispatch is used and the opcode callback fires correctly.
• A new xoption(QJS_ENABLE_DEBUGGER ...) has been added to CMakeLists.txt , defaulting to OFF. When not enabled, there is zero overhead — no extra struct fields, no callback checks in the hot path, and DIRECT_DISPATCH remains unaffected.

Other cleanups

• Renamed JSLocalVar to JSDebugLocalVar to avoid confusion with the internal JSVarDef.
• Fixed a potential memory leak where funcname was only freed inside if (filename).
• Removed leftover debug fprintf comments and unnecessary braces in the hot path.

Let me know if there's anything else you'd like adjusted.

[saghul]

@bnoordhuis WDYT?

[bnoordhuis]

Behind a compile-time flag would be good. The diff mostly looks good to me but there are a couple of changes I'd like to see:

• the API functions should always be available but return an error when compiled without debugger support

• can you rename the new emit_source_loc calls to something like emit_source_loc_debug and turn that into a no-op when there's no debugger support?

• I don't love the name JSOPChangedHandler. Suggestions for a better one? Something like JSBytecodeTraceFunc perhaps?

I'm assuming this good enough as a building block to assemble a functional debugger from? I can see how it lets you single-step through code, inspect stack frames, set breakpoints, etc., so I'm guessing... yes?

[G-Yong]

I didn't quite understand why emit_source_loc_debug was needed before. Now I get it a little bit— the newly added emit_source_loc calls are debug-specific (for stepping through if/for/return/etc.), unlike the original 5 upstream calls which are always needed for error reporting. Having emit_source_loc_debug as a no-op without debugger support keeps the bytecode identical to upstream and avoids the performance overhead.

[bnoordhuis]

Having emit_source_loc_debug as a no-op without debugger support keeps the bytecode identical to upstream and avoids the performance overhead.

Yes, exactly. :-)

The reason I suggested it is because it lets the debugger be a runtime option instead of a compile-time option.

[G-Yong]

Thanks @bnoordhuis, I now understand the intent! I've refactored the implementation to make the debugger a runtime option instead of a compile-time option:

Key changes:

• Removed all #ifdef QJS_ENABLE_DEBUGGER guards and the CMake option entirely
• OP_debug is now always defined in the opcode table
• emit_source_loc_debug() checks ctx->debug_trace at runtime — when no trace handler is set, it's a no-op, so the emitted bytecode is identical to upstream with zero performance overhead
• When a user calls JS_SetDebugTraceHandler(ctx, cb), subsequent JS compilation will emit OP_debug opcodes and the debugger becomes active
• All interpreter/optimizer code that handles OP_debug is always compiled in (no #ifdef guards needed — if OP_debug isn't emitted, those code paths are never reached)

This means a single QuickJS binary can serve both as a normal engine and a debug-capable engine, controlled entirely at runtime by whether JS_SetDebugTraceHandler has been called before JS code is compiled.

[G-Yong]

@dblnz A note about the DIRECT_DISPATCH issue you ran into in #119:

The earlier approach (from my fork's QJS_ENABLE_DEBUGGER branch) placed the debug callback at the top of the for(;;) loop, before SWITCH(pc):

restart:
    for(;;) {
#ifdef QJS_ENABLE_DEBUGGER
        if (b && ctx->bytecode_trace != NULL) {
            // callback here...
        }
#endif
        SWITCH(pc) {
        CASE(OP_push_i32): ...

This had a fundamental problem with DIRECT_DISPATCH: when enabled (Linux/GCC), BREAK is defined as SWITCH(pc) which uses computed goto (goto *dispatch_table[*pc++]) to jump directly to the next opcode's label — completely bypassing the loop top where the callback lived. Only the very first instruction would trigger the callback. That's why the old approach had to force DIRECT_DISPATCH=0:

#if defined(EMSCRIPTEN) || defined(_MSC_VER) || defined(QJS_ENABLE_DEBUGGER)
#define DIRECT_DISPATCH  0

The current OP_debug approach from @bnoordhuis is fundamentally better — the debug logic lives inside CASE(OP_debug):, which is a proper entry in the dispatch table. Whether using switch/case or computed goto, OP_debug is dispatched just like any other opcode. No need to disable DIRECT_DISPATCH, so there is zero performance impact on the dispatch mechanism when debugging is not active.

This is why this PR's approach is superior to what I originally proposed in #119.

[dblnz]

@dblnz A note about the DIRECT_DISPATCH issue you ran into in #119:

The earlier approach (from my fork's QJS_ENABLE_DEBUGGER branch) placed the debug callback at the top of the for(;;) loop, before SWITCH(pc):

restart:
    for(;;) {
#ifdef QJS_ENABLE_DEBUGGER
        if (b && ctx->bytecode_trace != NULL) {
            // callback here...
        }
#endif
        SWITCH(pc) {
        CASE(OP_push_i32): ...

This had a fundamental problem with DIRECT_DISPATCH: when enabled (Linux/GCC), BREAK is defined as SWITCH(pc) which uses computed goto (goto *dispatch_table[*pc++]) to jump directly to the next opcode's label — completely bypassing the loop top where the callback lived. Only the very first instruction would trigger the callback. That's why the old approach had to force DIRECT_DISPATCH=0:

#if defined(EMSCRIPTEN) || defined(_MSC_VER) || defined(QJS_ENABLE_DEBUGGER)
#define DIRECT_DISPATCH  0

The current OP_debug approach from @bnoordhuis is fundamentally better — the debug logic lives inside CASE(OP_debug):, which is a proper entry in the dispatch table. Whether using switch/case or computed goto, OP_debug is dispatched just like any other opcode. No need to disable DIRECT_DISPATCH, so there is zero performance impact on the dispatch mechanism when debugging is not active.

This is why this PR's approach is superior to what I originally proposed in #119.

This sounds great, thanks!

[saghul]

@G-Yong looks like there are some conflicts with master now, could you please resolve them? 🙏

[saghul]

Asked Claude for a review:

 Adversarial Review: Debug Trace API Branch

  Audited the 60-commit branch that adds JS_SetDebugTraceHandler / OP_debug / JS_GetLocalVariablesAtLevel plus a one-line scanner
  tweak in next_token. All bugs below are reproduced with compiled C programs against the actual build.

  ---
  BUG: goto exception without a thrown exception leaks JS_UNINITIALIZED to callers
  File: quickjs.c:17697
  Category: Error Handling / Data Integrity
  Severity: CRITICAL

  When the callback returns non-zero without calling JS_Throw*, the OP_debug handler does goto exception, but rt->current_exception is
   still JS_UNINITIALIZED (or stale). JS_GetException then returns this internal sentinel to user code. Stringifying yields the
  literal debug text [uninitialized].

  Trigger:
  static int cb(...) { return -1; }  /* no JS_Throw */
  ...
  JSValue r = JS_Eval(ctx, "42", 2, "<t>", JS_EVAL_TYPE_GLOBAL);
  /* JS_IsException(r) = 1 */
  JSValue e = JS_GetException(ctx);
  /* JS_VALUE_GET_TAG(e) == JS_TAG_UNINITIALIZED (4); ToString(e) = "[uninitialized]" */
  Confirmed on the built binary.

  Fix: before goto exception, ensure an exception is set. Either throw a default InternalError("aborted by debugger") when the
  callback returned non-zero but JS_HasException(ctx) == false, or document and enforce that the callback must throw before returning
  non-zero.

  ---
  BUG: Callback can silently corrupt runtime exception state (throw + return 0)
  File: quickjs.c:17694-17699
  Category: Error Handling / State
  Severity: CRITICAL

  If the callback calls JS_Throw* but returns 0, the OP_debug handler does NOT goto exception — it falls through to BREAK and the next
   opcode runs. rt->current_exception is now set and nothing unwinds. Subsequent opcodes execute with a pending exception;
  JS_HasException(ctx) stays true across the next eval.

  Trigger:
  static int cb(JSContext *ctx, ...) {
      JS_ThrowInternalError(ctx, "oops");
      return 0;
  }
  JSValue r = JS_Eval(ctx, "let a=1; let b=2;", 17, ...);
  /* JS_IsException(r) == 0  -- eval "succeeded" */
  /* JS_HasException(ctx) == 1 -- BUT exception still pending */
  /* Next eval also ends with HasException == 1 */
  Confirmed.

  Fix: after the callback returns, check JS_HasException(ctx). If true, treat it as a non-zero return regardless of the numeric return
   value: goto exception. Callback’s return code should not be the only signal.

  ---
  BUG: Duplicate callback emission for every let and const declaration
  File: quickjs.c:28450, quickjs.c:28457
  Category: Logic Errors (fall-through)
  Severity: HIGH

  case TOK_LET:
  case TOK_CONST:
  haslet:
      emit_source_loc_debug(s);     // fires
      ...
      /* fall thru */
  case TOK_VAR:
      emit_source_loc_debug(s);     // fires AGAIN when entered via LET/CONST

  Entering via TOK_LET or TOK_CONST falls through to TOK_VAR and emits a second OP_source_loc+OP_debug pair for the same statement.
  The callback fires twice with identical line/col.

  Trigger: let x = 42; → callback fires twice (count=2); var x = 42; correctly fires once. Verified on built binary.

  Fix: move the emit_source_loc_debug(s) call from the haslet: label to after the fall-through target so it runs exactly once:
  case TOK_LET:
  case TOK_CONST:
  haslet:
      if (!(decl_mask & DECL_MASK_OTHER)) { ... }
      /* fall thru */
  case TOK_VAR:
      emit_source_loc_debug(s);

  ---
  BUG: Callback fires multiple times per statement for any expression with binary ops / calls / new / throw
  File: quickjs.c:23287-23288 (and callers at 26507, 26624, 27331, 28440, 29095)
  Category: Logic Errors
  Severity: HIGH

  emit_source_loc unconditionally appends OP_debug when a handler is set at parse time — but emit_source_loc is called inside
  expression parsers (js_parse_expr_binary, js_parse_postfix_expr, etc.), not only at statement boundaries. So a single statement
  emits multiple OP_debugs.

  The header comment claims "OP_debug opcodes are always emitted at statement boundaries" — this is false.

  Trigger: 1+2; → callback fires 2 times (statement start + binary op). a+b*c; would fire 3+ times. Inside a for loop, each test
  expression re-fires the callback per iteration. Verified: for (var i=0; i<3; i++) { var x=i; } produces 8 callback events.

  Fix: append OP_debug only inside emit_source_loc_debug, not inside the raw emit_source_loc. Revert the if
  (unlikely(s->ctx->debug_trace)) dbuf_putc(bc, OP_debug); inside emit_source_loc and keep the OP_debug emission confined to
  emit_source_loc_debug.

  ---
  BUG: Handler attached after parse-time is silently a no-op
  File: quickjs.c:23291-23294 (emit_source_loc_debug), quickjs.c:23287-23288
  Category: Logic Errors
  Severity: HIGH

  OP_debug emission is gated at PARSE time by s->ctx->debug_trace. Code parsed before the handler is installed has no OP_debug opcodes
   and cannot be instrumented retroactively. The public doc comment says "OP_debug opcodes are always emitted at statement boundaries"
   — false.

  Trigger:
  JS_Eval(ctx, "function f() { var a=1; var b=2; return a+b; }", ...);  /* no handler */
  JS_SetDebugTraceHandler(ctx, cb);
  JS_Eval(ctx, "f()", ...);   /* handler is set, but count=1: only the f() site, */
                              /* nothing inside f because f was parsed handler-less */
  Verified. Any real "attach debugger" flow breaks.

  Fix: two reasonable options. (a) Change the gate to runtime: emit OP_debug unconditionally at statement boundaries (cost: a no-op
  opcode per statement in non-debug runs). (b) Explicitly document that JS_SetDebugTraceHandler only affects code PARSED after the
  call, and consider making it a per-runtime or JS_EVAL_FLAG_* option so users understand the constraint.

  ---
  BUG: JS_GetLocalVariablesAtLevel leaks JS_UNINITIALIZED values to C callers
  File: quickjs.c:2629-2644
  Category: Data Integrity / Security
  Severity: HIGH

  let/const variables before their declaration sit in the temporal dead zone as JS_UNINITIALIZED (tag 4) in sf->var_buf[i]. The code
  js_dups this directly and hands it back to the caller. JS_UNINITIALIZED is an internal sentinel that should never escape the VM;
  operations on it produce undefined-behavior-ish output (e.g., ToString returns the debug string [uninitialized]) and may trip
  assertions in other QuickJS APIs.

  Trigger:
  function f() { let a; let b = 10; return a; }   // OP_debug fires before 'a' is initialized
  // Callback sees: var a tag=4 (JS_TAG_UNINITIALIZED)
  Verified on built binary.

  Fix: in the loop, substitute JS_UNINITIALIZED slots with JS_UNDEFINED (or a dedicated <uninitialized> sentinel the API documents),
  or skip those vars. Also skip compiler-generated names starting with < (e.g., <ret>, <this_active_func>, etc.) — the test above
  shows <ret> being exposed, which is an internal eval-result slot that has no business appearing in a debugger listing.

  ---
  BUG: JS_GetLocalVariablesAtLevel leaks exceptions from JS_AtomToCString failure
  File: quickjs.c:2629-2644
  Category: Error Handling
  Severity: MEDIUM

  JS_AtomToCString allocates; on OOM it sets rt->current_exception and returns NULL. The loop does NOT check the return value, nor
  does it clear the exception before returning. Caller gets back a partially-populated array with NULL names AND a pending runtime
  exception that will surface on the next API call.

  Trigger: any low-memory condition while the callback is running.

  Fix: on JS_AtomToCString failure inside the loop, free what was allocated so far, clear the exception (JS_FreeValue(ctx,
  JS_GetException(ctx))), return NULL with *pcount = 0.

  ---
  BUG: find_jump_target skips OP_debug, so jump optimizations bypass debug points
  File: quickjs.c:34237-34242 (find_jump_target), quickjs.c:34196-34207 (code_has_label), quickjs.c:33739-33745 (get_label_pos)
  Category: Logic Errors (optimizer)
  Severity: MEDIUM

  find_jump_target iterates past OP_source_loc, OP_label, AND OP_debug when resolving a goto's destination. This is the right call for
   OP_label/OP_source_loc (they are no-ops / metadata), but OP_debug is an executable opcode with side effects (invoking the user
  callback). Skipping it at optimization time means an OP_goto L whose label points to label: OP_debug; OP_real; gets rewritten to
  jump directly to OP_real, silently dropping the debug trace event.

  Trigger: any backward goto in a loop targeting a statement that starts with OP_debug — e.g., the continue edge of a for/while loop,
  when the loop body's first opcode is an OP_debug. Hard to construct a clean visible symptom in small cases because the initial
  OP_source_loc fires at a related position, but the guaranteed per-iteration fire at a labeled statement becomes a no-op.

  Fix: in find_jump_target/code_has_label, do NOT skip OP_debug. Treat it like any other real opcode — stop the skip loop when
  encountered. Leave the OP_source_loc skip alone.

  ---
  BUG: emit_source_loc_debug(s) after emit_break / before emit_label(label_finally) / after OP_throw is dead code
  File: quickjs.c:28736 (break/continue), quickjs.c:28954 (try/catch before label_finally)
  Category: Logic Errors
  Severity: LOW

  Line 28736: emit_break already emitted an unconditional OP_goto; the following emit_source_loc_debug(s) emits into unreachable
  bytecode.
  Line 28954: the emission comes after emit_op(s, OP_throw) (from the preceding catch/finally-only paths) and before emit_label(s,
  label_finally) — unreachable.

  Dead opcodes pollute pc2line tables and grow bytecode size for no effect.

  Fix: remove both calls. The existing trace at the start of the try/catch/finally and inside the finally block already covers the
  user-observable boundaries.

  ---
  BUG: emit_source_loc_debug before final OP_ret uses a stale source location
  File: quickjs.c:28990-28991
  Category: Logic Errors
  Severity: LOW

  emit_source_loc_debug(s);
  emit_op(s, OP_ret);

  At this point the parser has already advanced past the finally block's closing }; s->token.line_num/col_num point to whatever
  follows the try statement. The debug event therefore reports an unrelated later location as the "finally exit" trace point.

  Fix: drop this call, or capture line/col before parsing the finally block and restore before emit.

  ---
  BUG: Tab-indented line in a space-indented file
  File: quickjs.c:22574

  At this point the parser has already advanced past the finally block's closing }; s->token.line_num/col_num point to whatever
  follows the try statement. The debug event therefore reports an unrelated later location as the "finally exit" trace point.

  Fix: drop this call, or capture line/col before parsing the finally block and restore before emit.

  ---
  BUG: Tab-indented line in a space-indented file
  File: quickjs.c:22574
  Category: Style (but indicative of careless integration)
  Severity: LOW

                                s->col_num = max_int(1, s->mark - s->eol);
  Uses hard tabs in the middle of a function that indents with spaces. Mixed indentation will misalign in most editors. Also, this is
  a spot-fix for only one of several js_parse_error sites inside next_token where s->col_num is stale (see 22794 for the proper
  end-of-next_token update) — other error paths in the same function still report stale column numbers.

  Fix: re-indent with spaces. Consider calling a helper js_set_current_col(s) at all js_parse_error sites inside next_token, not just
  this one.

  ---
  BUG: Silent truncation of filename/funcname to 63 bytes in the callback
  File: quickjs.c:17690-17693
  Category: Data Integrity
  Severity: LOW

  ATOM_GET_STR_BUF_SIZE == 64. A filename longer than 63 bytes (common for absolute paths) or an unusually long function name gets
  silently truncated without indication. Debugger clients that match on filename will mis-match.

  Fix: either use JS_AtomToCString (heap-allocated, full string) and free after the callback, or document the 63-byte truncation in
  the API header. Also: document that the const char * pointers are only valid during the callback invocation — the current doc
  comment doesn’t say this, and a reasonable user would assume they can stash the pointer.

  ---
  BUG: Callback return-value codepath contradicts the header comment
  File: quickjs.h:545-548
  Category: Documentation / Contract
  Severity: LOW (but interacts with the CRITICAL bugs above)

  Header says: Return 0 to continue, non-zero to raise an exception. In reality non-zero does NOT raise any exception — it just jumps
  to the exception handler and leaves rt->current_exception untouched (which is the CRITICAL bug above). The caller must JS_Throw*
  themselves if they want a real exception.

  Fix: either make the implementation honor the doc (raise a default InternalError on non-zero-without-throw), or rewrite the comment
  to say "caller is responsible for calling JS_Throw* before returning non-zero; the engine will not synthesize an exception."

  ---

At a quick glance, some of these are legit.

[G-Yong]

Thanks for the thorough review @saghul (and Claude)! I've pushed 1011587 on top of the branch addressing the issues raised. Summary:

OP_debug callback (JS_CallInternal)

• Use JS_AtomToCString instead of a 64-byte stack buffer so filename / funcname are passed at full length (fixes the silent truncation bug).
• If the callback returns non-zero without raising an exception, the engine now throws a default InternalError("aborted by debugger"), so JS_GetException() never returns the JS_UNINITIALIZED sentinel.
• If the callback calls JS_Throw* but returns 0, we still goto exception (checked via JS_HasException), so a pending exception can never silently leak into the next opcode / next eval.

JS_GetLocalVariablesAtLevel

• Skip compiler-generated names starting with < (e.g. , <this_active_func>).
• Substitute JS_UNINITIALIZED slots (let/const TDZ) with JS_UNDEFINED so the internal sentinel never escapes to C callers.
• On JS_AtomToCString OOM, free what was already allocated, clear the pending exception, set *pcount = 0 and return NULL (no more leaked exception state).

Duplicate / stray OP_debug emission

• Removed the unconditional dbuf_putc(bc, OP_debug) from emit_source_loc — it was firing inside expression parsers and producing multiple OP_debug per statement. OP_debug is now only emitted from emit_source_loc_debug, which is called strictly at statement boundaries.
• Moved the emit_source_loc_debug for TOK_LET / TOK_CONST out of the haslet: label so the fall-through into TOK_VAR no longer fires the trace twice for let/const.
• Added the missing emit_source_loc_debug at the top of throw and the default expression-statement path.
• Dropped dead emit_source_loc_debug calls after emit_break, before label_finally, and before the final OP_ret of try/catch/finally (those sites were unreachable / used stale source positions).

Optimizer

• find_jump_target and code_has_label no longer skip OP_debug. It is an executable opcode with a side effect (the user callback) and must not be elided when the optimizer follows a jump.

Misc

• Re-indented the stray hard-tab line in next_token with spaces.

quickjs.h

• Rewrote the doc comments on JSDebugTraceFunc / JS_SetDebugTraceHandler to match the implementation contract: either a non-zero return or raising via JS_Throw* aborts execution; the engine synthesizes a default exception when needed; filename/funcname are only valid for the duration of the callback; and OP_debug is only emitted for code parsed after the handler has been installed (so the handler must be set before evaluating application code).

I have not yet tackled the broader "handler attached after parse-time is silently a no-op" point as a behavior change — that one is now explicitly documented in the header. Happy to also implement option (a) (always emit OP_debug at statement boundaries, gate the dispatch at runtime) if you'd prefer that over the documented contract.

[G-Yong]

Before changing the design I ran the web-tooling-benchmark v0.5.3 suite to compare both options. Setup: Windows / MSVC 2019 Release / x64, upstream c707cf5, three back-to-back runs each, geomean of runs/s.

Option (a) — always emit OP_source_loc + OP_debug, gate at runtime in OP_debug case

• Upstream: 1.39 / 1.38 / 1.32 → 1.363
• Fork (always-emit): 1.28 / 1.22 / 1.23 → 1.245
• Geomean regression: −8.7 %
• Worst hits: prepack −16.2 %, jshint −14.5 %, postcss −13.1 %, babylon −11.7 %, terser −11.4 %, esprima −10.6 %. Every project regressed.

Current PR design — parse-time gate (emit_source_loc_debug checks ctx->debug_trace once, at parse time)

• Upstream: 1.34 / 1.38 / 1.36 → 1.360
• Fork (gated): 1.32 / 1.34 / 1.36 → 1.340
• Geomean Δ: −1.5 % — i.e. within run-to-run noise; per-benchmark deltas are roughly half positive, half negative.

[saghul]

Can you rebase one last time please?

[G-Yong]

I’ve used the ‘Sync fork’ option on GitHub. It seems to be just a merge operation. You could ‘merge --squash’ my pull request, as there are too many changes to make a rebase feasible. 😂

[saghul]

👍 can you take another look to see if more changes are necessary after explicit context management landed?

[G-Yong]

👍 can you take another look to see if more changes are necessary after explicit context management landed?

For the time being, we have made all the changes we could think of and had them reviewed by the AI. To the best of our ability and experience, the revisions are now largely complete.

[saghul]

_{Reviewed with tuicr.}

[saghul]

[NOTE] this change looks unrelated to debugging, is it fixing a problem with column numbers that exists today?

[G-Yong]

You’re right; this doesn’t fix a debug-related issue, but rather an exception error. If you don’t include this line, when running the following code:

function myFunc(){
  var a = 0;
  var b = 1 + 2 + 3 + 4 + 5 + 6 + 12abc;
}
myFunc()

the exception message thrown is: "SyntaxError: invalid number literal", indicating that the exception occurred at line:col=3:9
but in reality it is 3:35.

[saghul]

could you please split it into a separate commit?

[G-Yong]

This bug was discovered and fixed by my colleague @uxxca1;
I’ll ask him to create another pull request later.

[saghul]

👍

There are still changes necessary.