Development and Debugging Tools

Relevant source files

• Doc/library/bdb.rst
• Doc/library/dis.rst
• Doc/library/pdb.rst
• Doc/library/symtable.rst
• Include/internal/pycore_c_array.h
• Include/internal/pycore_compile.h
• Include/internal/pycore_flowgraph.h
• Include/internal/pycore_instruction_sequence.h
• Include/internal/pycore_magic_number.h
• Include/internal/pycore_symtable.h
• Lib/bdb.py
• Lib/dis.py
• Lib/pdb.py
• Lib/symtable.py
• Lib/test/pyclbr_input.py
• Lib/test/test_bdb.py
• Lib/test/test_compile.py
• Lib/test/test_compiler_codegen.py
• Lib/test/test_dis.py
• Lib/test/test_metaclass.py
• Lib/test/test_pdb.py
• Lib/test/test_peepholer.py
• Lib/test/test_pyclbr.py
• Lib/test/test_symtable.py
• Lib/test/test_type_annotations.py
• Misc/NEWS.d/next/Library/2025-12-16-15-32-41.gh-issue-142834.g7mHw_.rst
• Misc/NEWS.d/next/Library/2025-12-20-16-35-42.gh-issue-80744.X4pZ2N.rst
• Misc/NEWS.d/next/Tests/2025-12-17-02-02-57.gh-issue-142836.mR-fvK.rst
• Modules/clinic/symtablemodule.c.h
• Modules/symtablemodule.c
• Programs/test_frozenmain.h
• Python/assemble.c
• Python/ast.c
• Python/codegen.c
• Python/compile.c
• Python/flowgraph.c
• Python/instruction_sequence.c
• Python/symtable.c

Purpose and Scope

This document provides an overview of CPython's development and debugging tools, which enable developers to inspect, analyze, and debug Python code execution. These tools include interactive debuggers, bytecode disassemblers, and compilation analysis utilities that expose CPython's internal workings.

For detailed information about specific tools:

• The Python debugger (pdb and bdb framework) is covered in Python Debugger
• Bytecode disassembly and analysis tools are covered in Bytecode Disassembly and Analysis

Overview of Development Tools

CPython provides three primary categories of development tools:

Tool Category	Primary Modules	Purpose
Interactive Debugging	pdb, bdb	Step through code execution, inspect variables, set breakpoints
Bytecode Analysis	dis, opcode	Examine compiled bytecode, understand optimization behavior
Compilation Inspection	symtable, ast, compile	Analyze symbol tables, AST structures, and compilation process

These tools integrate with CPython's runtime and compilation pipeline to provide visibility into program execution and code generation.

Sources: Lib/pdb.py1-112 Lib/bdb.py1-11 Lib/dis.py1-27

Debugging Infrastructure Architecture

Debugger Architecture: Event Flow from User to Runtime

The debugging infrastructure uses a layered architecture where pdb.Pdb provides the user interface, bdb.Bdb implements the core debugging logic, and either sys.monitoring (default) or sys.settrace (legacy) provides runtime integration.

Sources: Lib/pdb.py351-389 Lib/bdb.py22-75 Lib/bdb.py362-430

Debugger Backends

CPython's debugger supports two backends for intercepting execution:

Monitoring Backend (Default)

• Uses sys.monitoring API (PEP 669) Lib/bdb.py22-75
• Tool ID: sys.monitoring.DEBUGGER_ID
• Registers callbacks for events: PY_START, LINE, PY_RETURN, RAISE, etc.
• More efficient than legacy tracing
• Event types defined in bdb._MonitoringTracer.EVENT_CALLBACK_MAP Lib/bdb.py23-35

Settrace Backend (Legacy)

• Uses sys.settrace() API Lib/bdb.py236-324
• Profile-style tracing with 'call', 'line', 'return', 'exception' events
• Maintained for compatibility
• Activated via backend='settrace' parameter Lib/pdb.py364

Sources: Lib/bdb.py22-75 Lib/bdb.py236-324 Lib/pdb.py338-348

Bytecode Analysis Tools Architecture

Bytecode Analysis: From Code Objects to Human-Readable Output

The dis module provides comprehensive bytecode analysis by iterating over code object bytecode, decoding instructions, and formatting output with opcode metadata.

Sources: Lib/dis.py85-149 Lib/dis.py223-430 Lib/dis.py450-549

Instruction Representation

The Instruction class encapsulates a single bytecode instruction with rich metadata:

Field	Type	Description
opname	str	Human-readable opcode name (e.g., 'LOAD_FAST')
opcode	int	Numeric opcode value
arg	int or None	Instruction argument
argval	Any	Resolved argument value (e.g., variable name, constant)
argrepr	str	Human-readable argument representation
offset	int	Byte offset in code object
start_offset	int	Starting offset including CACHE entries
cache_offset	int	Offset of first cache entry
end_offset	int	Offset after instruction and caches
starts_line	int or None	Source line number if new line starts
line_number	int or None	Current source line number
is_jump_target	bool	Whether this instruction is a jump target
jump_target	int or None	Target offset for jump instructions
positions	dis.Positions	Source location (lineno, end_lineno, col_offset, end_col_offset)

Sources: Lib/dis.py82-150 Lib/dis.py450-549

Compilation Analysis Tools

Compilation Pipeline with Analysis Points

Developers can inspect the compilation process at multiple stages using built-in tools. Each stage exposes different aspects of how Python code is transformed into executable bytecode.

Sources: Python/compile.c1-155 Python/symtable.c1-10 Lib/symtable.py1-8

Symbol Table Analysis

The symtable module provides programmatic access to the symbol table constructed during compilation:

Symbol Table API: Querying Variable Scopes and Properties

The symbol table API allows inspection of variable scopes, nested function relationships, and name binding patterns. This is useful for static analysis tools and IDE features.

Sources: Lib/symtable.py1-300 Python/symtable.c92-171

Integration with Runtime Execution

The debugging and analysis tools integrate deeply with CPython's runtime execution model:

Debugging Integration Points

1. Frame Object Access: Debuggers access frame objects via sys._getframe() to inspect local variables, global namespace, and code object Lib/pdb.py433

2. Execution Control:

   • bdb.set_step(): Execute one instruction/line Lib/bdb.py329-334
   • bdb.set_next(): Execute until next line in current frame Lib/bdb.py336-344
   • bdb.set_continue(): Resume normal execution Lib/bdb.py346-349
   • bdb.set_return(): Execute until current function returns Lib/bdb.py351-354

3. Breakpoint Management: Breakpoints stored in bdb.Breakpoint.bpbynumber list and bdb.Breakpoint.bplist dict Lib/bdb.py602-800

4. Event Callbacks: User-overridable methods for execution events:

   • user_call(frame, argument_list) Lib/pdb.py553-560
   • user_line(frame) Lib/pdb.py562-577
   • user_return(frame, return_value) Lib/pdb.py595-602
   • user_exception(frame, exc_info) Lib/pdb.py604-621

Sources: Lib/bdb.py329-361 Lib/pdb.py553-621

Analysis Integration Points

1. Code Object Inspection: The types.CodeType object exposes compiled bytecode and metadata:

   • co_code: Raw bytecode bytes
   • co_consts: Tuple of constants used
   • co_names: Tuple of global/builtin names
   • co_varnames: Tuple of local variable names
   • co_freevars: Tuple of free variables (closures)
   • co_cellvars: Tuple of cell variables (captured by nested functions)

2. Opcode Metadata: The opcode module provides mappings and metadata:

   • opname: List mapping opcode numbers to names Lib/dis.py60
   • opmap: Dict mapping names to opcode numbers Lib/dis.py61
   • _cache_format: Inline cache entry layouts Lib/dis.py11
   • _specializations: Adaptive specialization families Lib/dis.py18

3. Position Information: Python 3.11+ includes precise source location information:

   • co_positions(): Iterator over position tuples
   • Instruction.positions: Per-instruction source spans Lib/dis.py100-104

Sources: Lib/dis.py8-21 Lib/dis.py450-549

Tool Invocation Methods

Interactive Debugger

Method	Usage	Entry Point
Import & call	import pdb; pdb.set_trace()	Lib/pdb.py430-438
Built-in function	breakpoint()	Calls sys.breakpointhook() → pdb.set_trace()
Command line (script)	python -m pdb script.py	Lib/pdb.py2457-2515
Command line (module)	python -m pdb -m module	Lib/pdb.py2457-2515
Command line (attach)	python -m pdb -p <pid>	Lib/pdb.py2457-2515
Post-mortem	pdb.pm() after exception	Lib/pdb.py2354-2369
Programmatic	pdb.run(statement)	Lib/pdb.py2311-2322
Async debugging	await pdb.set_trace_async()	Lib/pdb.py440-489

Bytecode Disassembler

Method	Usage	Entry Point
Command line	python -m dis script.py	Lib/dis.py815-856
Function call	dis.dis(function)	Lib/dis.py85-149
Bytecode object	bc = dis.Bytecode(code); bc.dis()	Lib/dis.py152-265
Instruction iterator	for instr in dis.get_instructions(code)	Lib/dis.py450-549
Traceback analysis	dis.distb(traceback)	Lib/dis.py268-285
String compilation	dis.disco(source_string)	Lib/dis.py287-302

Symbol Table Analyzer

Method	Usage	Entry Point
From source string	st = symtable.symtable(code, '<string>', 'exec')	Lib/symtable.py1-20
From file	with open(file) as f: st = symtable.symtable(f.read(), file, 'exec')	Lib/symtable.py1-20

Sources: Lib/pdb.py2311-2515 Lib/dis.py85-856 Lib/symtable.py1-20

Command-Line Interface Examples

pdb CLI Options

python -m pdb [options] script.py [args]

Options:
  -c, --command <cmd>   Execute pdb commands from .pdbrc file
  -m <module>           Run module as script (like python -m)
  -p, --pid <pid>       Attach to running process (3.14+)

Lib/pdb.py2457-2515

dis CLI Options

python -m dis [options] [infile]

Options:
  -C, --show-caches     Show inline cache entries
  -O, --show-offsets    Show instruction offsets
  -P, --show-positions  Show source positions
  -S, --specialized     Show specialized bytecode

Lib/dis.py815-856

Performance Considerations

Debugging Overhead

• Monitoring Backend: Lower overhead than sys.settrace(), events dispatched directly by VM
• Settrace Backend: Higher overhead, every line/call/return triggers Python callback
• Trace Opcodes: When trace_opcodes=True, generates event per instruction Lib/pdb.py568-574

Disassembly Optimization

• Lazy Evaluation: Bytecode object doesn't decode instructions until iterated Lib/dis.py152-265
• Caching: Instruction positions cached in dis._get_code_array() Lib/dis.py580-610
• Adaptive Bytecode: adaptive=True shows specialized instructions without re-compiling Lib/dis.py85-149

Sources: Lib/bdb.py22-75 Lib/pdb.py568-574 Lib/dis.py152-265