by CLI reference¶
basedpython ships with two executables: by and buff. by is the basedpython driver — an extension of ty and includes the
type-checker, the transpiler, and a few project-level commands
buff is the basedpython version of ruff
in addition to the cli provided by ty, by includes:
| command | what it does |
|---|---|
run |
transpile and run a module with python -m <module> |
init |
start a project, or set up git's merges of one |
repl |
an interactive session in basedpython |
build |
transpile every .by/.byi file and write to build/ |
compile |
compile .by files to native CPython extension modules |
generate-api-file |
write a public-api lockfile (see api-lock) |
merge-driver |
the merge driver git runs for python files and api.lock |
transpile |
transpile a single file to stdout (reads stdin if no file) |
by run¶
by run MODULE [ARGS...] # transpile + run with `python -m MODULE`
by run # run the configured entry point
by run MODULE --min-version 3.12 # target a specific runtime python version
equivalent to by build && python -m MODULE, but only transpiles the
modules required to import MODULE
the module can be left out when the project configures an entry point:
by run then runs app.cli. a module named on the command line always wins,
so by run other still runs other. the first positional argument is always
the module — to reach the entry point's own arguments, name it: by run app.cli --name asdf
everything after MODULE is forwarded to the program as sys.argv[1:],
including options — by run main --name asdf passes --name asdf on. the one
exception is a leading -h / --help, which prints by run's own help; write
by run main -- --help to reach the program's. when the program's entry point
is a main function, those arguments are parsed
into its parameters
the project is type-checked first, and a program with check errors is not run — the checker's verdict and the runtime must not diverge. warnings don't block; a rule can be downgraded in configuration where its error is unwanted
the interpreter is the project's own: the environment by check resolves
imports against — the environment.python the project configures, else an
activated virtual environment, a conda environment, or a .venv beside the
project's pyproject.toml. --python overrides it for one run, and PYTHON
stands in only where the project has no environment of its own, above the bare
python3 discovery falls back to. a third-party package lives in that
environment's site-packages and nowhere else, so running anywhere else fails
on an import the checker had no complaint about. so the check before the run is
made against the python the run uses: one --python or PYTHON names, a bare name
found on PATH by the sys.prefix it reports, has its packages checked as well as
run. all of that resolves against
the project root rather than the working directory, so by run in a
subdirectory is still this project
--launcher PATH starts the program through another program — a debugger that
has to be the process the program runs in, say, see debugging —
without choosing the
interpreter for it. discovery runs exactly as above, and the program starts as
PATH <interpreter> <runner> MODULE [ARGS...], so the launcher is handed the
interpreter by run chose and runs once, for the program: the version probe is
still made against the interpreter itself
what a debugger finds in that process is the program under <runner>, the
_by_runner.py by run writes into the tree it staged. the runner's frames are
the outermost of the main thread, beneath the program's module, and they are
by's alone: every frame whose file is <runner> is the runner's. the runner
catches nothing the program raises, so an exception the program does not catch
leaves the interpreter as it leaves python main.py — the program's own
exception, with the program's frames on top of its traceback — and python
reports it and exits 1. a debugger that stops where such an exception leaves the
outermost frame stops in the program by treating the runner's frames as it
treats runpy's under python -m: as the launcher's, not the program's
the tree says which file the runner is, so a launcher need not be told: the
_by_sourcemap.py beside the runner names it as LAUNCHER, an absolute path
spelled as the map's other paths are (see sourcemaps).
a by build tree, which no program is started from, says LAUNCHER = None
a signal that ends the program ends by run with 128 and the signal's number,
as a shell reports it
on unix, the signals that ask a process to end — SIGTERM, SIGINT, SIGHUP
and SIGQUIT — are passed on to the program when they are sent to by run (and
to the interpreter of a by repl session the same way), and
by run exits as the program then does: a program that handles one gets to tidy
up, and its status is by run's. so docker stop, a supervisor or an editor's
Stop that sends by run a SIGTERM stops the program, by run as the first
process of a container included. SIGTERM is always passed on. the other three
are what a terminal sends to every process of its foreground process group,
which the program is in with by run, so while by run is in that group they
are left to reach the program from the terminal rather than sent to it twice:
ctrl-c interrupts the program once. a signal sent to the whole process group, or
to every process of the tree, reaches the program twice, so to stop a run,
signal by run alone. the one exception is the SIGHUP of a hangup, which goes
to the first process of the terminal's session: when that is by run, nothing
else has been told, so it is passed on. a signal by run was started ignoring,
as nohup starts it ignoring SIGHUP, stays ignored, by the program too. a
signal that arrives before the program has started ends by run with 128 and
its number. on windows, ctrl-c and ctrl-break reach the program through the
console it shares with by run, which waits for it and exits as it does, and
end by run with 130 before the program has started
--in-build runs the program in the tree by run staged rather than in the
directory it was typed in. the default is right for an application: a relative
path on the command line, and anything the program reads or writes beside the
project, mean what they meant when they were written. but the project is python
only in the staged tree, which is reachable through sys.path alone — and
sys.path serves an import and nothing else. so a program whose arguments are
the project's own files could not name one:
the only tests/test_calc in the working directory is a .by. with
--in-build the whole project is there — every transpiled module at its place
in the module tree, and beside them every file the build carries over unchanged,
pyproject.toml included — so the path resolves, the tool finds its own
configuration where it always was, and it reports against a tree laid out like
the project: tests/test_calc.py::TestGroup::test_add, whatever the source
layout
it is a flag rather than something inferred because only the caller knows which of the two directories they mean, and the program's name does not say
--coverage PATH measures which lines the program runs, with coverage.py, and
writes them to PATH as an LCOV report — the format an editor's gutter,
genhtml and CI coverage services all read. coverage.py measures the python the
program actually ran, so the report is mapped back before it is written: a
transpiled module is reported against its .by file, a .by line counting as
covered when any statement transpiled from it ran, and a .py the build carries
over is reported against the file in the project rather than the staged copy
that ran. that holds for a module a test runner compiled itself, rewriting its
assertions, as much as for one python imported
coverage.py has to be installed in the interpreter the program runs on: without
it the program is not started, and the error names that interpreter. the project's own
coverage.py configuration ([tool.coverage], .coveragerc) applies, except that
no .coverage data file is written: the report is the result. its source,
include and omit name the project's files as they are in the project, with
--in-build too, since every module runs as the file it was written as. the one
exception is a module an import hook other than pytest's assertion rewriter
compiled itself, which runs as the staged copy it compiled: a source or an
omit naming it does not reach it. a .by
module under source that never ran is reported as not run, as coverage.py
reports a .py that never ran. a program that
leaves without returning to by run — on os._exit, an os.exec* or a signal — leaves no report, and
--compiled cannot be combined with it, since a native module runs no python
lines to measure
--profile PATH profiles the program with cProfile and writes the snapshot to
PATH, in the pstats format cProfile writes — the .pstat file python's
pstats module and PyCharm's profiler viewer open. a profiler names each
function by the file and first line of its code, so the snapshot is mapped back
before it is written, as a coverage report is: a function of a transpiled module
is named by its .by file and the .by line it starts at, and one of a .py
the build carries over by the file in the project rather than the staged copy
that ran. a viewer's way from a function to its source then leads to the file it
was written in. that holds for a module a test runner compiled itself too, and
two functions that turn out to be one — the same name at the same .by line —
are added up as one, as pstats adds two profiles. a function the transpiler
wrote on its own has no .by line, and is named by its module's .by at line 0
the runtime's helpers — what the emitted python calls for !!, a lazy import or
a soundness check — are named <by runtime>, at the line of _by_runtime.py
they start on, as by build writes that file, and by their own names:
<by runtime>:103(_force_unwrap). one helper is then one function whether the
transpiler wrote it into a module or it ran from a copy of the runtime — beside
a package's modules, or at the root of the tree by run stages, which every
module in no package imports from — and two helpers that share a name, such as two __init__
methods, stay two. like python's own <frozen runpy>, the name is no file: a
viewer's way to the source opens nothing, where line 0 of the module the helper
was written into would open that module at its first line, as PyCharm's viewer
opens a file it finds at the line it is given
the profile starts at the program's first line and ends after its last atexit
handler, so it holds the program and not by run's search for it, nor the
writing of the snapshot or the traceback of an exception the program did not
catch. by's own python in the staged tree is not in it either: what by run
does to load a module — finding it, and compiling it under its .by lines — is
time the program spends, so it is folded into python's import machinery, which
called it, as python's own finder and loader are under python main.py. so is
pytest's plugin into pytest, under a test run. a function of by's called from
two places is shared between them by how often each called it. cProfile measures every thread on python 3.12 and later, and only the thread the
program starts on before that, as python -m cProfile does. a program that leaves
without returning to by run — on os._exit, an os.exec* or a signal — leaves no snapshot, and --coverage
cannot be combined with it, since a profile taken under coverage.py's tracer
measures the tracer
the emitted code targets that interpreter's version by default. an explicit
--min-version wins, but must not exceed the interpreter — by run refuses
rather than emit code the interpreter cannot parse. an interpreter older than
the version the project declares is refused too: the source may use syntax that
python has no lowering for, and the failure would land as a SyntaxError inside
generated code. passing --min-version for the interpreter's own version builds
for it instead
the program runs in the directory by run was invoked from, the way python -m
does. the transpiled tree lives in a temporary directory, which python puts at
the head of sys.path — so a relative path on the command line, and anything
the program reads or writes beside the project, resolve where they were written
to
hidden directories (.claude, .git, .venv, …) and build outputs are never
treated as project source: they are neither checked nor transpiled, by run
and build alike
by repl¶
by repl # an interactive session, on the project's interpreter
by repl --python python3.14 # on another interpreter
by repl --module app.main # starting in a module of the project, after running it
an interactive session in which you type basedpython, each input checked and transpiled as the
continuation of the inputs before it and run in one python process. it starts where by run
would, with the project staged as by run stages it, so the project's modules import as they
do in a program. --session-file PATH keeps PATH holding the inputs that ran, for an editor's
console to complete against, and --events PORT tells the console, on a connection to PORT,
whether standard input is read for an input or by the program. --module MODULE starts the
session in a module of the project: it runs first, and each input is its continuation. see
the repl
by build¶
by build # transpile every .by/.byi the project claims
by build --min-version 3.12 # target a specific runtime python version
by build walks the project's own file set — the one by check walks — so
src.exclude and the ignore files it honours apply here too, and the two halves
of the toolchain never disagree about which files are in the project. hidden
directories (.claude, .git, .venv, …) and build outputs are skipped
without --min-version the emit target is the project's configured python
version (environment.python-version, else the requires-python lower bound),
so the checker and the emitter agree about which python this project targets
a file that fails to parse fails the build, but only for itself: every other module is still written. a code generator and a test runner are exactly what you reach for when one file is mid-edit
artifacts and exit status answer different questions. everything that could be
emitted is emitted, sourcemap and package markers included; the exit status says
whether anything was reported. so a build that prints an error exits 1 while
still leaving a usable build/, and by build && pytest build/tests runs the tests
only against a tree the checker had nothing to say about. when a reported
diagnostic is fatal the exit status is 101 instead, as it is for by check. the
fatal one is panic, reported when by itself panicked while checking or
transpiling a file, so a script can tell a crash in by from a problem in the
code. by run, by compile and by transpile <dir> answer the same way
writes the transpiled python to the project root's build/ — wherever in the
project it is run, as by compile does too; --out DIR names another directory,
relative to where you typed it — mirroring the module tree. a
src-layout project's src/package_name/main.by is the module
package_name.main, so it lands at build/package_name/main.py — build/ is a
directory you can put on sys.path as it stands, and run.main names a module
the same way an import does. the build leaves a .by-manifest at the top of what
it writes, which keeps the output out of the project: by check does not check
it, and by generate-api-file does not describe it
alongside the python it writes build/_by_sourcemap.py, mapping each generated
line back to the .by line it came from, with a digest of both files so a tool
reading it can tell whether it still describes what is on disk — see
sourcemaps
by compile¶
for how to use it, see native compilation
by compile # every .by and .py file under the project root → build/
by compile hot.by # one file, named by its path rather than its module
by compile -o native hot.by # a different output directory
by compile --verbose # report every function left interpreted, and why
by compile --emit-c-only # write the generated C without compiling it
by compile --no-any # refuse a gradual type in a compiled function
by compile --require-native # refuse to leave *any* function interpreted
by compile --licence-recheck # re-ask, at runtime, every lookup a call skipped
by compile --no-verify-install # leave out the import-time install check
the output directory mirrors the module tree, the way by build's does: the
package member pkg/sub/dup.py lands at build/pkg/sub/dup.cpython-313-darwin.so,
and the package pkg/sub/__init__.py at
build/pkg/sub/__init__.cpython-313-darwin.so — so build/ can go on sys.path as
it stands and every module imports under the dotted name it was compiled as
by compile writes everything by build writes, and the extensions as well.
a compiled module is not a program on its own: it reads its templates and
fixtures relative to itself, and imports the modules beside it. so the tree holds
the whole project as importable python — every .by transpiled, every
hand-written .py, py.typed, the data files, the sourcemap — with a native
extension beside each module that was compiled. python's finder prefers an
extension to source, so those modules load natively and the rest are interpreted,
which is what makes naming files a speed decision rather than a correctness one:
naming files decides what is compiled, not what is written: the whole project
is checked and transpiled either way, so by compile hot.by costs what a
by build costs plus the one module's native compile. that is the price of the
tree being a program rather than a heap.
build/ is a mirror rather than a pile. what the previous run wrote and this one
did not is taken back, which matters most for extensions: python prefers one to
source, so an extension left behind by a deleted or renamed module would go on
shadowing the .py written in its place. the artefacts therefore describe the
last invocation — by compile a.by then by compile b.by leaves b native and
a interpreted, and --emit-c-only, which stops before the C compiler, leaves a
tree with no extensions at all. in every case each module still imports, so this
costs speed rather than correctness; by compile with no arguments compiles the
whole project.
the directory is shared with by build on purpose — the same tree, built two
ways — and the two take each other's output back accordingly. it is also a name
setuptools and python -m build use, so a project using both writes them into one
directory; nothing is destroyed, because the manifest only ever takes back what
by itself wrote.
a directory holding a .by-manifest is a build output, whoever wrote it, and is
never read as source or carried into another tree — a project that builds into
two directories would otherwise put a copy of each inside the other. that is by
the marker rather than by the directory's name, because --out can say anything.
a project that deliberately ships a directory containing one has to rename the
marker or keep the directory outside its source roots.
upgrading from a version that wrote
out/: nothing migrates it, and nothing reads it any more. delete it — a staleout/holds importable python that no build refreshes.
--no-any buys no speed on its own — it is a predictability contract. a
gradual type still compiles, through python's object protocol, so it costs speed
with nothing to say so, and a module that means to be fully typed can be held to
it:
$ by compile loose.by --no-any
by failed
Cause: could not compile /app/loose.by
Cause: `no-any` is on and 2 place(s) in this module have a gradual type:
loose: `x` is gradual
loose: `return` is gradual
--require-native asks a different question. --no-any asks is this module
fully typed; --require-native asks does this module compile entirely, and so
fails on a function the compiler cannot lower, however precise its types:
$ by compile hot.by --require-native
by failed
Cause: could not compile /app/hot.by
Cause: `require-native` is on and 1 function(s) were left interpreted:
run_code: `eval` with no namespace of its own runs in the calling frame's, and a compiled function pushes none
either failure exits 2 and writes no extension for that module
--licence-recheck is for chasing a wrong answer rather than for shipping. the
compiler licenses a call to go straight to a compiled body wherever nothing can
have put something else under the name — a class python can neither subclass nor
rebind a method on, a receiver whose type still matches the one the licence was
taken against. that decision is a claim nothing checks, and one that is wrong is
a wrong answer with nothing to report it: an override that stops being seen, a
rebinding nothing notices. under this flag each of those calls does the lookup it
was licensed to skip, compares where it lands with the body it is about to run,
and stops the process when the two disagree:
$ by compile app.py --licence-recheck -o out && python -c 'import app; app.go()'
by: licence re-check failed on Shape.area: the receiver is not this class (app.Square)
by: the compiled call was licensed to skip this lookup, and the lookup no longer agrees
it costs the lookup each licence exists to avoid, so it is slower than an ordinary build and slower than no compilation at all in the places licences do the most work. a program that passes under it answers exactly what it answers without it — a re-check that agrees changes nothing
a function the compiler cannot lower natively is not an error: the module's transpiled python is embedded in the extension and executed at import, so declined functions still exist and module-level code still runs. the natively compiled functions are installed over the top
$ by compile hot.by --verbose
/app/hot.by -> /app/build/hot.cpython-313-darwin.so
info[declined]: `run_code` was left to its interpreted definition
--> hot.by:1:1
|
1 | / def run_code(source: str) -> object:
2 | | return eval(source)
| |_______________________^ `eval` with no namespace of its own runs in the calling frame's, and a compiled function pushes none
Found 1 diagnostic
compiled 1 module(s)
1 function(s) left to the interpreted definition
a class left interpreted is harder to see, because it answers exactly as the
compiled one would. so the emitted module checks its own classes once at import
and raises ImportError naming any that did not install as the compiler meant
them to, and BY_INSTALL_CENSUS names a file it writes the whole verdict into:
$ BY_INSTALL_CENSUS=census.tsv python -c 'import hot'
$ cat census.tsv
hot Held interpreted
hot Plain installed
comparing that against --annotate's class headings is what says whether a
report of n compiled classes is n classes that ran.
--no-verify-install leaves the check out.
the C toolchain and the cpython headers come from the same interpreter by run
uses, chosen the same way, and it must have development headers available. it
has to be the same one: an extension built against one abi is unimportable by
another, and by run --compiled imports what this wrote
by generate-api-file¶
by generate-api-file # writes api.lock in the project root
by generate-api-file --stdout # writes lockfile to stdout
by generate-api-file -o public.lock # custom output path
by generate-api-file --python-version 3.10 # target a specific python version
by generate-api-file --project lib # the lockfile of the project in lib, lib/api.lock
see api-lock for the lockfile format and workflow, and for what the language server says about a stale lockfile
by init¶
by init my-app # a new application in my-app/
by init my-library --lib # a new library
by init --git-merge # set up git's merges of the project here
by init my-app --no-git-merge # a new project, and leave git as it is
by init my-app --no-sync # a new project, without its environment
writes a new project: see packaging. it
will not write over a pyproject.toml that is already there
a new project gets its environment from uv sync, run in it once it is written: a
.venv on the python the project targets, with the project installed. a .venv
already in the directory is left as it is. --no-sync leaves the environment out;
without uv on PATH, by init writes nothing and fails saying so, and a sync
that fails leaves the project written, for uv sync to finish
in a git work tree it also sets up git's merges of the project, with
by merge-driver as the merge driver for its python and
basedpython files and its api.lock. on a terminal it asks;
--git-merge sets them up and --no-git-merge does not, and without a terminal
or either flag they are left as they are. --git-merge works in a project that is
already there too, and writes nothing else. what it sets up:
- in the project's
.gitattributes,*.by,*.byi,*.py,*.pyiandapi.lockwithmerge=by, after whatever lines are there and in their line ending, for each pattern git does not already merge with the driver. commit it: every clone then merges with it. a.gitattributesthat is there but cannot be read is not written, and nothing is set up - in the repository's own git configuration,
merge.by.nameandmerge.by.driver:'/path/to/by' merge-driver %O %A %B %L %P %S %X %Y, followed by a fallback. the configuration is not committed, so each clone sets it up again. a linked work tree shares its repository's
the driver names the by that set it up by the absolute path it was run as: an
editor's git runs a merge driver without the project's environment on PATH, so a
bare by would not be found there. the path is the one it was run by, not where
its links lead, so a virtual environment's bin/by, uv's ~/.local/bin/by or a
package manager's link keeps working through an upgrade. git takes a driver
that fails as a conflict holding ours alone, the other side's changes nowhere in
the file, so where by merge-driver does not merge the file, because its by is
gone or is too old to have the subcommand, the command runs git's own line merge
instead, exactly as git merge would without the driver, and says so on stderr.
by init --git-merge again sets up a driver whose by is gone, or whose command
an older by wrote, with the by that runs it, and says which command it
replaced; a current driver whose by is there is left alone, so running it twice
changes nothing
it needs git 2.44 or later, the first git that gives a merge driver the labels of
the versions it merges and whose git merge-file takes the diff algorithm
git merge uses. with an older git nothing is set up, since neither the driver
nor its fallback could merge a file. it fails, having said what it wrote, where
git still does not merge every path with the driver afterwards
an editor asks the language server the same, by/gitMergeIntegration
(see merges and diffs)
by merge-driver¶
the merge driver git runs, as by merge-driver %O %A %B %L %P %S %X %Y, for each
file both sides of a merge changed that .gitattributes gives to it. git runs a
merge driver for every such file, conflicting or not, from the command line or
from an editor that runs git, and for a git rebase or git cherry-pick as for
a git merge
for a .by, .byi, .py or .pyi file it
- merges lines as
git mergedoes:git merge-filewith the histogram diffgit mergeuses, the repository'smerge.conflictStyleand git's conflict labels and marker size - resolves each conflict whose sides change different things, as an editor's
resolve simple conflicts asks
by/resolveMergeConflicts(see merges and diffs): imports both sides added, definitions each side added under its own name, and the like - reconciles the imports with the three versions, as
by/reconcileMergeImportsdoes. a line merge can be clean and still leave a file reading a name it no longer imports: ours removesimport jsonwith the last function reading it, theirs adds another reading it. the import is added back, and an import a side removed that nothing reads is dropped. with conflicts left, it does that outside them, with what either side of each reads, so that whatever you choose in a conflict has its imports. where the versions import one name from different places, the imports of it become a conflict,bysays why on stderr, and the file is left for you
it exits 0 only when nothing is left to resolve. a conflict it does not resolve is
left as git writes it, in the repository's conflict style, whatever it resolved
beside it, so with nothing resolved the file is exactly what git merge alone
would have written. the options git merge -X takes are not passed to a
merge driver
api.lock is generated from the source (see api-lock).
git runs a merge driver while other files may still conflict, or not yet be
written, so there is no source to generate it from: a lockfile both sides changed
differently is never merged clean. it is left as git's line merge wrote it,
conflicted even where those lines merged cleanly, and by says which
by generate-api-file to run once the rest of the merge is resolved. a lockfile
only one side changed is not given to a merge driver at all, and may still be
stale after the merge: the language server reports it
by transpile¶
by transpile FILE # read FILE, write transpiled python to stdout
by transpile # read from stdin, write to stdout
by transpile FILE --reverse # convert python source into basedpython idioms
by transpile FILE --min-version 3.12 # target a specific runtime python version
echo 'x: int = 1' | by transpile
by transpile also accepts a directory, transpiling it in place (every .by →
.py, or with --reverse every .py → .by)
stops at the first transpile error and prints a diagnostic
a python source that declares its own encoding with a
PEP 263 comment is decoded as it is read.
what is written back out is utf-8, so the declaration is rewritten to say utf-8
— left alone it would name an encoding the file no longer has. utf-8 and the
latin-1 family are decodable; a file declaring anything else is skipped and named
rather than guessed at