Skip to content

by CLI reference

basedpython ships with two executables: by and buff. by is the basedpython driver — an extension of ty and includes the type-checker, the transpiler, and a few project-level commands

buff is the basedpython version of ruff

by <command> [args...]

in addition to the cli provided by ty, by includes:

command what it does
run transpile and run a module with python -m <module>
init start a project, or set up git's merges of one
repl an interactive session in basedpython
build transpile every .by/.byi file and write to build/
compile compile .by files to native CPython extension modules
generate-api-file write a public-api lockfile (see api-lock)
merge-driver the merge driver git runs for python files and api.lock
transpile transpile a single file to stdout (reads stdin if no file)

by run

by run MODULE [ARGS...]             # transpile + run with `python -m MODULE`
by run                              # run the configured entry point
by run MODULE --min-version 3.12    # target a specific runtime python version

equivalent to by build && python -m MODULE, but only transpiles the modules required to import MODULE

the module can be left out when the project configures an entry point:

[tool.basedpython.run]
main = "app.cli"

by run then runs app.cli. a module named on the command line always wins, so by run other still runs other. the first positional argument is always the module — to reach the entry point's own arguments, name it: by run app.cli --name asdf

everything after MODULE is forwarded to the program as sys.argv[1:], including options — by run main --name asdf passes --name asdf on. the one exception is a leading -h / --help, which prints by run's own help; write by run main -- --help to reach the program's. when the program's entry point is a main function, those arguments are parsed into its parameters

the project is type-checked first, and a program with check errors is not run — the checker's verdict and the runtime must not diverge. warnings don't block; a rule can be downgraded in configuration where its error is unwanted

the interpreter is the project's own: the environment by check resolves imports against — the environment.python the project configures, else an activated virtual environment, a conda environment, or a .venv beside the project's pyproject.toml. --python overrides it for one run, and PYTHON stands in only where the project has no environment of its own, above the bare python3 discovery falls back to. a third-party package lives in that environment's site-packages and nowhere else, so running anywhere else fails on an import the checker had no complaint about. so the check before the run is made against the python the run uses: one --python or PYTHON names, a bare name found on PATH by the sys.prefix it reports, has its packages checked as well as run. all of that resolves against the project root rather than the working directory, so by run in a subdirectory is still this project

--launcher PATH starts the program through another program — a debugger that has to be the process the program runs in, say, see debugging — without choosing the interpreter for it. discovery runs exactly as above, and the program starts as PATH <interpreter> <runner> MODULE [ARGS...], so the launcher is handed the interpreter by run chose and runs once, for the program: the version probe is still made against the interpreter itself

what a debugger finds in that process is the program under <runner>, the _by_runner.py by run writes into the tree it staged. the runner's frames are the outermost of the main thread, beneath the program's module, and they are by's alone: every frame whose file is <runner> is the runner's. the runner catches nothing the program raises, so an exception the program does not catch leaves the interpreter as it leaves python main.py — the program's own exception, with the program's frames on top of its traceback — and python reports it and exits 1. a debugger that stops where such an exception leaves the outermost frame stops in the program by treating the runner's frames as it treats runpy's under python -m: as the launcher's, not the program's

the tree says which file the runner is, so a launcher need not be told: the _by_sourcemap.py beside the runner names it as LAUNCHER, an absolute path spelled as the map's other paths are (see sourcemaps). a by build tree, which no program is started from, says LAUNCHER = None

a signal that ends the program ends by run with 128 and the signal's number, as a shell reports it

on unix, the signals that ask a process to end — SIGTERM, SIGINT, SIGHUP and SIGQUIT — are passed on to the program when they are sent to by run (and to the interpreter of a by repl session the same way), and by run exits as the program then does: a program that handles one gets to tidy up, and its status is by run's. so docker stop, a supervisor or an editor's Stop that sends by run a SIGTERM stops the program, by run as the first process of a container included. SIGTERM is always passed on. the other three are what a terminal sends to every process of its foreground process group, which the program is in with by run, so while by run is in that group they are left to reach the program from the terminal rather than sent to it twice: ctrl-c interrupts the program once. a signal sent to the whole process group, or to every process of the tree, reaches the program twice, so to stop a run, signal by run alone. the one exception is the SIGHUP of a hangup, which goes to the first process of the terminal's session: when that is by run, nothing else has been told, so it is passed on. a signal by run was started ignoring, as nohup starts it ignoring SIGHUP, stays ignored, by the program too. a signal that arrives before the program has started ends by run with 128 and its number. on windows, ctrl-c and ctrl-break reach the program through the console it shares with by run, which waits for it and exits as it does, and end by run with 130 before the program has started

--in-build runs the program in the tree by run staged rather than in the directory it was typed in. the default is right for an application: a relative path on the command line, and anything the program reads or writes beside the project, mean what they meant when they were written. but the project is python only in the staged tree, which is reachable through sys.path alone — and sys.path serves an import and nothing else. so a program whose arguments are the project's own files could not name one:

$ by run pytest -v tests/test_calc.py
ERROR: file or directory not found: tests/test_calc.py

the only tests/test_calc in the working directory is a .by. with --in-build the whole project is there — every transpiled module at its place in the module tree, and beside them every file the build carries over unchanged, pyproject.toml included — so the path resolves, the tool finds its own configuration where it always was, and it reports against a tree laid out like the project: tests/test_calc.py::TestGroup::test_add, whatever the source layout

it is a flag rather than something inferred because only the caller knows which of the two directories they mean, and the program's name does not say

--coverage PATH measures which lines the program runs, with coverage.py, and writes them to PATH as an LCOV report — the format an editor's gutter, genhtml and CI coverage services all read. coverage.py measures the python the program actually ran, so the report is mapped back before it is written: a transpiled module is reported against its .by file, a .by line counting as covered when any statement transpiled from it ran, and a .py the build carries over is reported against the file in the project rather than the staged copy that ran. that holds for a module a test runner compiled itself, rewriting its assertions, as much as for one python imported

$ by run --coverage coverage.lcov --in-build pytest

coverage.py has to be installed in the interpreter the program runs on: without it the program is not started, and the error names that interpreter. the project's own coverage.py configuration ([tool.coverage], .coveragerc) applies, except that no .coverage data file is written: the report is the result. its source, include and omit name the project's files as they are in the project, with --in-build too, since every module runs as the file it was written as. the one exception is a module an import hook other than pytest's assertion rewriter compiled itself, which runs as the staged copy it compiled: a source or an omit naming it does not reach it. a .by module under source that never ran is reported as not run, as coverage.py reports a .py that never ran. a program that leaves without returning to by run — on os._exit, an os.exec* or a signal — leaves no report, and --compiled cannot be combined with it, since a native module runs no python lines to measure

--profile PATH profiles the program with cProfile and writes the snapshot to PATH, in the pstats format cProfile writes — the .pstat file python's pstats module and PyCharm's profiler viewer open. a profiler names each function by the file and first line of its code, so the snapshot is mapped back before it is written, as a coverage report is: a function of a transpiled module is named by its .by file and the .by line it starts at, and one of a .py the build carries over by the file in the project rather than the staged copy that ran. a viewer's way from a function to its source then leads to the file it was written in. that holds for a module a test runner compiled itself too, and two functions that turn out to be one — the same name at the same .by line — are added up as one, as pstats adds two profiles. a function the transpiler wrote on its own has no .by line, and is named by its module's .by at line 0

the runtime's helpers — what the emitted python calls for !!, a lazy import or a soundness check — are named <by runtime>, at the line of _by_runtime.py they start on, as by build writes that file, and by their own names: <by runtime>:103(_force_unwrap). one helper is then one function whether the transpiler wrote it into a module or it ran from a copy of the runtime — beside a package's modules, or at the root of the tree by run stages, which every module in no package imports from — and two helpers that share a name, such as two __init__ methods, stay two. like python's own <frozen runpy>, the name is no file: a viewer's way to the source opens nothing, where line 0 of the module the helper was written into would open that module at its first line, as PyCharm's viewer opens a file it finds at the line it is given

$ by run --profile run.pstat main
$ python -m pstats run.pstat

the profile starts at the program's first line and ends after its last atexit handler, so it holds the program and not by run's search for it, nor the writing of the snapshot or the traceback of an exception the program did not catch. by's own python in the staged tree is not in it either: what by run does to load a module — finding it, and compiling it under its .by lines — is time the program spends, so it is folded into python's import machinery, which called it, as python's own finder and loader are under python main.py. so is pytest's plugin into pytest, under a test run. a function of by's called from two places is shared between them by how often each called it. cProfile measures every thread on python 3.12 and later, and only the thread the program starts on before that, as python -m cProfile does. a program that leaves without returning to by run — on os._exit, an os.exec* or a signal — leaves no snapshot, and --coverage cannot be combined with it, since a profile taken under coverage.py's tracer measures the tracer

the emitted code targets that interpreter's version by default. an explicit --min-version wins, but must not exceed the interpreter — by run refuses rather than emit code the interpreter cannot parse. an interpreter older than the version the project declares is refused too: the source may use syntax that python has no lowering for, and the failure would land as a SyntaxError inside generated code. passing --min-version for the interpreter's own version builds for it instead

the program runs in the directory by run was invoked from, the way python -m does. the transpiled tree lives in a temporary directory, which python puts at the head of sys.path — so a relative path on the command line, and anything the program reads or writes beside the project, resolve where they were written to

hidden directories (.claude, .git, .venv, …) and build outputs are never treated as project source: they are neither checked nor transpiled, by run and build alike

by repl

by repl                             # an interactive session, on the project's interpreter
by repl --python python3.14         # on another interpreter
by repl --module app.main           # starting in a module of the project, after running it

an interactive session in which you type basedpython, each input checked and transpiled as the continuation of the inputs before it and run in one python process. it starts where by run would, with the project staged as by run stages it, so the project's modules import as they do in a program. --session-file PATH keeps PATH holding the inputs that ran, for an editor's console to complete against, and --events PORT tells the console, on a connection to PORT, whether standard input is read for an input or by the program. --module MODULE starts the session in a module of the project: it runs first, and each input is its continuation. see the repl

by build

by build                            # transpile every .by/.byi the project claims
by build --min-version 3.12         # target a specific runtime python version

by build walks the project's own file set — the one by check walks — so src.exclude and the ignore files it honours apply here too, and the two halves of the toolchain never disagree about which files are in the project. hidden directories (.claude, .git, .venv, …) and build outputs are skipped

without --min-version the emit target is the project's configured python version (environment.python-version, else the requires-python lower bound), so the checker and the emitter agree about which python this project targets

a file that fails to parse fails the build, but only for itself: every other module is still written. a code generator and a test runner are exactly what you reach for when one file is mid-edit

artifacts and exit status answer different questions. everything that could be emitted is emitted, sourcemap and package markers included; the exit status says whether anything was reported. so a build that prints an error exits 1 while still leaving a usable build/, and by build && pytest build/tests runs the tests only against a tree the checker had nothing to say about. when a reported diagnostic is fatal the exit status is 101 instead, as it is for by check. the fatal one is panic, reported when by itself panicked while checking or transpiling a file, so a script can tell a crash in by from a problem in the code. by run, by compile and by transpile <dir> answer the same way

writes the transpiled python to the project root's build/ — wherever in the project it is run, as by compile does too; --out DIR names another directory, relative to where you typed it — mirroring the module tree. a src-layout project's src/package_name/main.by is the module package_name.main, so it lands at build/package_name/main.py — build/ is a directory you can put on sys.path as it stands, and run.main names a module the same way an import does. the build leaves a .by-manifest at the top of what it writes, which keeps the output out of the project: by check does not check it, and by generate-api-file does not describe it

alongside the python it writes build/_by_sourcemap.py, mapping each generated line back to the .by line it came from, with a digest of both files so a tool reading it can tell whether it still describes what is on disk — see sourcemaps

by compile

for how to use it, see native compilation

by compile                      # every .by and .py file under the project root → build/
by compile hot.by               # one file, named by its path rather than its module
by compile -o native hot.by     # a different output directory
by compile --verbose            # report every function left interpreted, and why
by compile --emit-c-only        # write the generated C without compiling it
by compile --no-any             # refuse a gradual type in a compiled function
by compile --require-native     # refuse to leave *any* function interpreted
by compile --licence-recheck    # re-ask, at runtime, every lookup a call skipped
by compile --no-verify-install  # leave out the import-time install check

the output directory mirrors the module tree, the way by build's does: the package member pkg/sub/dup.py lands at build/pkg/sub/dup.cpython-313-darwin.so, and the package pkg/sub/__init__.py at build/pkg/sub/__init__.cpython-313-darwin.so — so build/ can go on sys.path as it stands and every module imports under the dotted name it was compiled as

by compile writes everything by build writes, and the extensions as well. a compiled module is not a program on its own: it reads its templates and fixtures relative to itself, and imports the modules beside it. so the tree holds the whole project as importable python — every .by transpiled, every hand-written .py, py.typed, the data files, the sourcemap — with a native extension beside each module that was compiled. python's finder prefers an extension to source, so those modules load natively and the rest are interpreted, which is what makes naming files a speed decision rather than a correctness one:

by compile hot.by        # hot is native, every other module is interpreted

naming files decides what is compiled, not what is written: the whole project is checked and transpiled either way, so by compile hot.by costs what a by build costs plus the one module's native compile. that is the price of the tree being a program rather than a heap.

build/ is a mirror rather than a pile. what the previous run wrote and this one did not is taken back, which matters most for extensions: python prefers one to source, so an extension left behind by a deleted or renamed module would go on shadowing the .py written in its place. the artefacts therefore describe the last invocation — by compile a.by then by compile b.by leaves b native and a interpreted, and --emit-c-only, which stops before the C compiler, leaves a tree with no extensions at all. in every case each module still imports, so this costs speed rather than correctness; by compile with no arguments compiles the whole project.

the directory is shared with by build on purpose — the same tree, built two ways — and the two take each other's output back accordingly. it is also a name setuptools and python -m build use, so a project using both writes them into one directory; nothing is destroyed, because the manifest only ever takes back what by itself wrote.

a directory holding a .by-manifest is a build output, whoever wrote it, and is never read as source or carried into another tree — a project that builds into two directories would otherwise put a copy of each inside the other. that is by the marker rather than by the directory's name, because --out can say anything. a project that deliberately ships a directory containing one has to rename the marker or keep the directory outside its source roots.

upgrading from a version that wrote out/: nothing migrates it, and nothing reads it any more. delete it — a stale out/ holds importable python that no build refreshes.

--no-any buys no speed on its own — it is a predictability contract. a gradual type still compiles, through python's object protocol, so it costs speed with nothing to say so, and a module that means to be fully typed can be held to it:

def loose(x: dynamic) -> dynamic:
    return x
$ by compile loose.by --no-any
by failed
  Cause: could not compile /app/loose.by
  Cause: `no-any` is on and 2 place(s) in this module have a gradual type:
  loose: `x` is gradual
  loose: `return` is gradual

--require-native asks a different question. --no-any asks is this module fully typed; --require-native asks does this module compile entirely, and so fails on a function the compiler cannot lower, however precise its types:

def run_code(source: str) -> object:
    return eval(source)
$ by compile hot.by --require-native
by failed
  Cause: could not compile /app/hot.by
  Cause: `require-native` is on and 1 function(s) were left interpreted:
  run_code: `eval` with no namespace of its own runs in the calling frame's, and a compiled function pushes none

either failure exits 2 and writes no extension for that module

--licence-recheck is for chasing a wrong answer rather than for shipping. the compiler licenses a call to go straight to a compiled body wherever nothing can have put something else under the name — a class python can neither subclass nor rebind a method on, a receiver whose type still matches the one the licence was taken against. that decision is a claim nothing checks, and one that is wrong is a wrong answer with nothing to report it: an override that stops being seen, a rebinding nothing notices. under this flag each of those calls does the lookup it was licensed to skip, compares where it lands with the body it is about to run, and stops the process when the two disagree:

$ by compile app.py --licence-recheck -o out && python -c 'import app; app.go()'
by: licence re-check failed on Shape.area: the receiver is not this class (app.Square)
by: the compiled call was licensed to skip this lookup, and the lookup no longer agrees

it costs the lookup each licence exists to avoid, so it is slower than an ordinary build and slower than no compilation at all in the places licences do the most work. a program that passes under it answers exactly what it answers without it — a re-check that agrees changes nothing

a function the compiler cannot lower natively is not an error: the module's transpiled python is embedded in the extension and executed at import, so declined functions still exist and module-level code still runs. the natively compiled functions are installed over the top

$ by compile hot.by --verbose
/app/hot.by -> /app/build/hot.cpython-313-darwin.so
info[declined]: `run_code` was left to its interpreted definition
 --> hot.by:1:1
  |
1 | / def run_code(source: str) -> object:
2 | |     return eval(source)
  | |_______________________^ `eval` with no namespace of its own runs in the calling frame's, and a compiled function pushes none

Found 1 diagnostic

compiled 1 module(s)
1 function(s) left to the interpreted definition

a class left interpreted is harder to see, because it answers exactly as the compiled one would. so the emitted module checks its own classes once at import and raises ImportError naming any that did not install as the compiler meant them to, and BY_INSTALL_CENSUS names a file it writes the whole verdict into:

$ BY_INSTALL_CENSUS=census.tsv python -c 'import hot'
$ cat census.tsv
hot Held interpreted
hot Plain installed

comparing that against --annotate's class headings is what says whether a report of n compiled classes is n classes that ran. --no-verify-install leaves the check out.

the C toolchain and the cpython headers come from the same interpreter by run uses, chosen the same way, and it must have development headers available. it has to be the same one: an extension built against one abi is unimportable by another, and by run --compiled imports what this wrote

by generate-api-file

by generate-api-file                       # writes api.lock in the project root
by generate-api-file --stdout              # writes lockfile to stdout
by generate-api-file -o public.lock        # custom output path
by generate-api-file --python-version 3.10 # target a specific python version
by generate-api-file --project lib         # the lockfile of the project in lib, lib/api.lock

see api-lock for the lockfile format and workflow, and for what the language server says about a stale lockfile

by init

by init my-app                      # a new application in my-app/
by init my-library --lib            # a new library
by init --git-merge                 # set up git's merges of the project here
by init my-app --no-git-merge       # a new project, and leave git as it is
by init my-app --no-sync            # a new project, without its environment

writes a new project: see packaging. it will not write over a pyproject.toml that is already there

a new project gets its environment from uv sync, run in it once it is written: a .venv on the python the project targets, with the project installed. a .venv already in the directory is left as it is. --no-sync leaves the environment out; without uv on PATH, by init writes nothing and fails saying so, and a sync that fails leaves the project written, for uv sync to finish

in a git work tree it also sets up git's merges of the project, with by merge-driver as the merge driver for its python and basedpython files and its api.lock. on a terminal it asks; --git-merge sets them up and --no-git-merge does not, and without a terminal or either flag they are left as they are. --git-merge works in a project that is already there too, and writes nothing else. what it sets up:

  • in the project's .gitattributes, *.by, *.byi, *.py, *.pyi and api.lock with merge=by, after whatever lines are there and in their line ending, for each pattern git does not already merge with the driver. commit it: every clone then merges with it. a .gitattributes that is there but cannot be read is not written, and nothing is set up
  • in the repository's own git configuration, merge.by.name and merge.by.driver: '/path/to/by' merge-driver %O %A %B %L %P %S %X %Y, followed by a fallback. the configuration is not committed, so each clone sets it up again. a linked work tree shares its repository's

the driver names the by that set it up by the absolute path it was run as: an editor's git runs a merge driver without the project's environment on PATH, so a bare by would not be found there. the path is the one it was run by, not where its links lead, so a virtual environment's bin/by, uv's ~/.local/bin/by or a package manager's link keeps working through an upgrade. git takes a driver that fails as a conflict holding ours alone, the other side's changes nowhere in the file, so where by merge-driver does not merge the file, because its by is gone or is too old to have the subcommand, the command runs git's own line merge instead, exactly as git merge would without the driver, and says so on stderr. by init --git-merge again sets up a driver whose by is gone, or whose command an older by wrote, with the by that runs it, and says which command it replaced; a current driver whose by is there is left alone, so running it twice changes nothing

it needs git 2.44 or later, the first git that gives a merge driver the labels of the versions it merges and whose git merge-file takes the diff algorithm git merge uses. with an older git nothing is set up, since neither the driver nor its fallback could merge a file. it fails, having said what it wrote, where git still does not merge every path with the driver afterwards

an editor asks the language server the same, by/gitMergeIntegration (see merges and diffs)

by merge-driver

by merge-driver BASE OURS THEIRS MARKER_SIZE PATH [BASE_LABEL OURS_LABEL THEIRS_LABEL]

the merge driver git runs, as by merge-driver %O %A %B %L %P %S %X %Y, for each file both sides of a merge changed that .gitattributes gives to it. git runs a merge driver for every such file, conflicting or not, from the command line or from an editor that runs git, and for a git rebase or git cherry-pick as for a git merge

for a .by, .byi, .py or .pyi file it

  1. merges lines as git merge does: git merge-file with the histogram diff git merge uses, the repository's merge.conflictStyle and git's conflict labels and marker size
  2. resolves each conflict whose sides change different things, as an editor's resolve simple conflicts asks by/resolveMergeConflicts (see merges and diffs): imports both sides added, definitions each side added under its own name, and the like
  3. reconciles the imports with the three versions, as by/reconcileMergeImports does. a line merge can be clean and still leave a file reading a name it no longer imports: ours removes import json with the last function reading it, theirs adds another reading it. the import is added back, and an import a side removed that nothing reads is dropped. with conflicts left, it does that outside them, with what either side of each reads, so that whatever you choose in a conflict has its imports. where the versions import one name from different places, the imports of it become a conflict, by says why on stderr, and the file is left for you

it exits 0 only when nothing is left to resolve. a conflict it does not resolve is left as git writes it, in the repository's conflict style, whatever it resolved beside it, so with nothing resolved the file is exactly what git merge alone would have written. the options git merge -X takes are not passed to a merge driver

api.lock is generated from the source (see api-lock). git runs a merge driver while other files may still conflict, or not yet be written, so there is no source to generate it from: a lockfile both sides changed differently is never merged clean. it is left as git's line merge wrote it, conflicted even where those lines merged cleanly, and by says which by generate-api-file to run once the rest of the merge is resolved. a lockfile only one side changed is not given to a merge driver at all, and may still be stale after the merge: the language server reports it

by transpile

by transpile FILE           # read FILE, write transpiled python to stdout
by transpile                # read from stdin, write to stdout
by transpile FILE --reverse         # convert python source into basedpython idioms
by transpile FILE --min-version 3.12 # target a specific runtime python version
echo 'x: int = 1' | by transpile

by transpile also accepts a directory, transpiling it in place (every .by → .py, or with --reverse every .py → .by)

stops at the first transpile error and prints a diagnostic

a python source that declares its own encoding with a PEP 263 comment is decoded as it is read. what is written back out is utf-8, so the declaration is rewritten to say utf-8 — left alone it would name an encoding the file no longer has. utf-8 and the latin-1 family are decodable; a file declaring anything else is skipped and named rather than guessed at