native compilation¶
by compile builds your modules into native CPython extension modules, which import and answer
like the python they replace
given hot.by:
compile it and import it:
$ by compile hot.by
/app/hot.by -> /app/build/hot.cpython-313-darwin.so
compiled 1 module(s)
$ PYTHONPATH=build python -c "import hot; print(hot.__file__, hot.total(10))"
/app/build/hot.cpython-313-darwin.so 285
nothing about the source changes, and a function the compiler cannot lower still runs, so compiling is a question of speed rather than of what the program does. how much speed depends on the code: the benchmark suite measures it
what it writes¶
by compile writes everything by build writes, and the extensions
as well. build/, in the project root, holds the whole project as importable python, with an
extension beside each module that was compiled. in a project that also has a main.by,
by compile hot.by leaves:
build/
├── .by-manifest
├── _by_build.json
├── _by_sourcemap.py
├── by.h
├── hot.by
├── hot.c
├── hot.cpython-313-darwin.so
├── hot.py
├── main.by
├── main.py
└── pyproject.toml
the tree mirrors the module tree, so pkg/sub/dup.by lands at
build/pkg/sub/dup.cpython-313-darwin.so, and build/ can go on sys.path as it stands. python's
finder prefers an extension to source, which is what makes hot native and main interpreted
here. -o DIR writes somewhere else
build/ is a mirror of the last run: what that run did not write is taken back, so an extension
never outlives the module it was built from
choosing what to compile¶
by compile # every .by and .py file in the project
by compile hot.by # hot is native, every other module is interpreted
by compile a.by b.by # a and b are native
the arguments are files, not module names, and a hand-written .py compiles as a .by does. naming
a file decides what is compiled, not what is written: the whole project is checked and transpiled
either way
each run describes itself and nothing before it, so by compile a.by followed by
by compile b.by leaves b native and a interpreted. name everything you want in one run
running it¶
by run --compiled compiles what the program imports and runs it, without you managing a tree. with
this main.by:
the entry module itself stays interpreted, since running something as __main__ needs a code
object and an extension module has none. --compiled cannot be combined with --coverage: a native
module runs no python lines to measure
code it cannot compile¶
a function the compiler cannot lower is left to its interpreted definition rather than failing the
build. the module's transpiled python is embedded in the extension and runs when it is imported, so
that function still exists and module-level code still runs. --verbose says which ones, and why.
add this to hot.by:
$ by compile hot.by --verbose
/app/hot.by -> /app/build/hot.cpython-313-darwin.so
info[declined]: `run_code` was left to its interpreted definition
--> hot.by:7:1
|
7 | / def run_code(source: str) -> object:
8 | | return eval(source)
| |_______________________^ `eval` with no namespace of its own runs in the calling frame's, and a compiled function pushes none
Found 1 diagnostic
compiled 1 module(s)
1 function(s) left to the interpreted definition
a decline is invisible unless you look for it, so a module that means to be fully compiled can say
so and be held to it. --require-native fails on any function left interpreted:
$ by compile hot.by --require-native
by failed
Cause: could not compile /app/hot.by
Cause: `require-native` is on and 1 function(s) were left interpreted:
run_code: `eval` with no namespace of its own runs in the calling frame's, and a compiled function pushes none
and --no-any fails on a gradual type in a compiled signature, the commonest reason code ends up
slower than it looks:
$ by compile loose.by --no-any
by failed
Cause: could not compile /app/loose.by
Cause: `no-any` is on and 2 place(s) in this module have a gradual type:
loose: `x` is gradual
loose: `return` is gradual
either failure exits 2 and writes no extension for that module
a check error is a different matter. as with by build, everything that could be written is
written and the exit status says what was reported, so a project with an error still produces its
extensions and exits 1
how a compiled module differs¶
compiled and interpreted code agree, and the differences that remain are the semantic deltas. the ones you are most likely to meet:
- a
data classconstructor checks its arguments, since a compiled field is unboxed:Point("a", 1)raisesTypeError - assigning a method onto a class from outside it raises
TypeError, since an emitted class is sealed ison a fixed-length tuple, a smallintor an internedstris unspecifiedf.__code__of a compiled function belongs to the function that forwards to it, andf.__wrapped__is the interpreted definition, which is whereinspectreads the signature
a traceback through a compiled function names its .by file and line. a debugger does not stop
inside one: see debugging
configuration¶
flags are in the cli reference. two project options trade a
behaviour of python for speed, under compile:
bind-functions-early, off by default: a compiled call to a function of the same module goes straight to it, where it would otherwise ask whether the name still holds that function. with it on,mock.patch("hot.helper")no longer reacheshot's own compiled callsfollow-recursion-limit, on by default: a compiled recursion counts its calls against python's recursion limit. with it off,sys.setrecursionlimitdoes not reach compiled code, and a recursion runs until the stack is close to running out, where it raisesRecursionErrorrather than crashing
python versions¶
a native build needs python 3.11 or later. 3.11 through 3.14 are supported, as are the free-threaded builds of 3.13 and 3.14
$ PYTHON=python3.10 by compile hot.by
by failed
Cause: a native build needs python 3.11 or later, and `python3.10` is python 3.10
the interpreter is the one by run would use, chosen the same way:
the project's environment first, and PYTHON only where the project has none. its development
headers have to be available, as does the C compiler it reports it was built with, which is the one
by compile uses
an extension is built for one interpreter. another interpreter does not see it, and imports the
.py beside it instead, which is correct and slower