reified type parameters¶
standard python erases type parameters: a def f[T] has no way to recover
T at runtime, so x is T or print(T) inside the body are impossible.
basedpython makes a type parameter available as a real runtime value when the
body actually uses it as one:
the type parameter behaves like an ordinary positional parameter that happens
to be filled by the [...] specialization step rather than the (...) call
step
what the value is¶
the value of T is the type argument exactly as it was written, and a type
argument is not always a class: f[1], f[None] and f[int | str] are all
specializations of a def f[T]. a bound does not change that, because a union of
subclasses satisfies a class bound, so f[A | B] specializes a def f[T: Base].
so T reads as a TypeForm[T], whatever its bound. it can be printed, compared,
tested against (x is T) and handed on (g[T](), list[T]), but not used as a
class:
def f[T: int](x: object):
print(x is T) # ok
T() # error: `TypeForm[T]` is not callable
T.__name__ # error: `TypeForm[T]` has no attribute `__name__`
isinstance(x, T) # error: `isinstance` takes a class
c: type[T] = T # error: a `TypeForm[T]` is not a `type[T]`
isinstance(T, type) asks whether it is a class, and narrows T to a type[T]
where it is. a call is then checked against the bound:
class Base:
def __init__(self, x: int): ...
def make[T: Base]() -> object:
if isinstance(T, type):
T() # error: no argument for `x`
return T(1) # ok
T() # error: `TypeForm[T]` is not callable
return None
the bound is all a call is checked against, as it is for a parameter typed type[Base]. a class
that asks for more than the bound does raises TypeError when it is called:
class Needs:
def __init__(self, x: int): ...
def make[T]() -> object:
if isinstance(T, type):
return T() # ok: `object` takes no arguments
return None
make[Needs]() # TypeError: missing `x`
the read promises no attributes at all, not even object's, since a type argument
need not be an object with the ones a class has. getattr(T, name) with no default
is refused for the same reason (unresolved-attribute). either ask first, or give it
a default:
def named[T](name: str) -> object:
T.__doc__ # error: `TypeForm[T]` has no attribute `__doc__`
getattr(T, name) # error
getattr(T, name, None) # ok
if isinstance(T, type):
return T.__name__ # ok
return None
the elements of a variadic and a keyword pack are type forms too, and narrow the same way
a .py file has no way to write a specialization, so it can't subscript a function
(not-subscriptable). the exception is a .by function that takes type arguments,
which it calls as f[int](1)
a .by file subscripts a generic method as it does a function, obj.m[int](1), whether
the method's body reads the parameter or not. what the receiver gives Self is not one
of the arguments
declaring it¶
a reified function can be pickled, with or without its type arguments, by the name it is defined under
reification is inferred from the body, which means it can also be declared.
reified ahead of a type parameter reifies it whether or not the body ever
reads it as a value:
writing the keyword is the difference between reification being a consequence
of how the body happens to be written and it being part of the declaration —
the specialization step is required either way, so a signature that promises a
runtime T keeps promising it when the body stops printing it. it stacks ahead
of the variance keywords (reified out T) and applies to a
variadic and a keyword pack as well (reified *Ts, reified **Kwargs)
reified is a soft keyword: a type parameter named reified is still just
that, since nothing that can open a parameter follows it
a class reifies too, though it reads the argument from the instance rather than
from a closure — see reified class type parameters.
nothing else has a step to hang a runtime value off, so reified on a type
alias or a type def is an error (invalid-reified-type-param), as it is on a
class's keyword pack, which a subscript has no way to supply
the editor hints the modifier wherever the body reifies a type parameter without declaring it, so what is inferred and what is written read the same
why this is safe to do¶
PEP 695 already compiles the type-parameter list as an implicit enclosing
scope, so inside f the name T is a free variable of the function's code
object — f.__code__.co_freevars == ("T",) — bound to a cell holding the
TypeVar. reification just swaps that cell's contents from the TypeVar to
the concrete type argument before the body runs. the swap is keyed by
free-variable name, so every other cell — a captured outer local, the
implicit __class__ behind zero-arg super() — carries over untouched, and
the rebuilt function keeps its parameter defaults, keyword-only defaults and
qualname. no bytecode is rewritten and no name resolution is special-cased; we
reuse the closure machinery cpython already builds
desugaring¶
a function is reified when one of its type parameters is declared reified or
is referenced in a value position (anywhere other than a type annotation).
that function is wrapped in the generic polyfill and its call
sites route through specialization instead of being stripped:
→
@generic
def f[T](t: object):
return isinstance(T, type) and isinstance(t, T)
f[int](1) # not stripped — routes through generic.__getitem__
the generic wrapper, emitted into the preamble:
class generic:
def __init__(self, fn, args=None, instance=None):
self.fn = fn
self.args = args
self.instance = instance
def __repr__(self):
return f"<generic {self.fn!r}>"
def __getattr__(self, name):
if name == "fn":
raise AttributeError(name)
return getattr(self.fn, name)
def __get__(self, obj, objtype=None):
if obj is None:
return self
return generic(self.fn, self.args, obj)
def __getitem__(self, item):
if self.args is not None:
raise TypeError("type arguments already specified")
if not isinstance(item, tuple):
item = (item,)
params = self.fn.__type_params__
if len(item) > len(params):
raise TypeError(
f"too many type arguments for {self.fn.__name__}: "
f"expected {len(params)}, got {len(item)}"
)
return generic(self.fn, item, self.instance)
def __call__(self, *args, **kwargs):
fn = self.fn
code = fn.__code__
supplied = self.args if self.args is not None else ()
values = {}
for index, param in enumerate(fn.__type_params__):
name = param.__name__
if index < len(supplied):
values[name] = supplied[index]
continue
has_default = getattr(param, "has_default", None)
if has_default is not None and has_default():
values[name] = param.__default__
elif name in code.co_freevars:
raise TypeError(f"{fn.__name__}() missing a type argument for {name!r}")
closure = tuple(
CellType(values[name]) if name in values else cell
for name, cell in zip(code.co_freevars, fn.__closure__ or ())
)
temp_fn = FunctionType(code, fn.__globals__, fn.__name__, fn.__defaults__, closure)
temp_fn.__kwdefaults__ = fn.__kwdefaults__
temp_fn.__qualname__ = fn.__qualname__
if self.instance is not None:
return temp_fn(self.instance, *args, **kwargs)
return temp_fn(*args, **kwargs)
a reified method binds its receiver through the descriptor __get__, so
obj.m[int]() passes self exactly like an ordinary method call. attribute
access falls through to the wrapped function, so introspection (f.__name__,
f.__doc__, f.__type_params__) keeps working, and the wrapper hashes by
identity like the function it replaces
f[int] produces a specialized generic carrying args=(int,); calling it
rebuilds the function with a closure whose type-parameter cells hold the type
arguments, so the body sees T is int. an omitted slot is filled from its
PEP 696 default, read off the function's __type_params__. a type parameter
used only in annotations is left erased exactly as in
explicit generic call sites — the wrapper is the cost of
reification and we only pay it when the body needs it
over- and under-specialization that get past the type checker still fail
cleanly at runtime: too many type arguments raise TypeError at the
subscription, and a reified slot with neither a value nor a default raises
TypeError at the call
decorators¶
the wrapper is inserted innermost — directly above the def, below any
user decorators — so it always receives the raw function object whose closure
it rebuilds. outer decorators then compose with the wrapper exactly as the
type checker models them:
class C:
@staticmethod
def f[T]() -> object:
return T
C.f[int]() # staticmethod's descriptor passes the wrapper through
a decorator that returns a different callable erases the specialization
step — f[int] on its result is an ordinary subscript on whatever the
decorator returned, and ty flags it at the use site if that type has no
__getitem__
the one binding that cannot compose is classmethod: it hides the function
behind an opaque bound method with no __getitem__, so a reified classmethod
could be neither specialized nor called. ty reports reified-classmethod at
the definition and the transpiler refuses to emit it. __init_subclass__ and
__class_getitem__ are implicitly classmethods and get the same treatment
the lowered def keeps its native [T] syntax because reification reuses the
pep 695 closure cells cpython already builds. that syntax is only available on
python 3.12+, so reification requires min_version >= 3.12; a reified function
on an older target is a transpile error rather than code that cannot run. a
defaulted parameter (PEP 696) raises the bar to min_version >= 3.13 — the
default syntax is not valid on 3.12, and the erased polyfill can't stand in
because it discards the native parameter list reification depends on
explicit or inferred specialization¶
a reified generic is structurally a two-step callable (f[...] then (...)),
and the first step is not optional — but it doesn't have to be written. when a
reified type parameter appears in the signature, a bare call reifies it
through inference: the transpiler injects the statically inferred
specialization at the call site, so the two-step call still happens at runtime
the injected argument is a runtime type expression for what the checker
solved, so f(1) hands f the type Literal[1], and print(T) prints
typing.Literal[1]. x is T asks whether x is that value. a literal is not a
class, so it fails isinstance(T, type) and takes the other branch of a guard. to
have T be int, pass something that is only known to be one, such as int(1). a float or
complex literal hands on its class instead, because equal floats can be different
values (0.0 == -0.0) and the runtime couldn't tell them apart. structured
annotations solve through their type arguments (list[T] + [1] → int),
and unions and tuples spell as int | str / tuple[int, str]. a class name
is injected only when the bare name resolves — in the module's globals or
builtins — to that same class, so the emitted expression evaluates to the
intended type object
x is T answers what the same target written in place would. a subscripted class
is compared argument by argument, by the variance the checker read for the class,
whether it was declared, inferred or inherited, through nested generics, unions,
intersections, literals, tuples, packs, type[...], LiteralString, Self and
None, and a use-site out or in written in the type argument is kept with it
a class of the standard library relates as its stub says it does, whatever its
runtime class says. str is a Sequence[str] because class str(Sequence[str])
says so, a tuple[int, str] is a Sequence[int | str], a TypedDict is a
Mapping[str, object], and calling a class is what its constructor says. being
registered with an abc is something only the runtime says, so int is no
numbers.Number, and a protocol has the members it declares and not whatever every
object has:
from collections.abc import Sequence
from typing import Hashable, Sized
class Out[out T]: pass
def chk[T](x: object) -> bool:
return x is T
assert chk[Out[Sequence[str]]](Out[str]())
assert chk[Out[() -> int]](Out[type[int]]())
assert not chk[Out[Hashable]](Out[Sized]())
a type argument that names a class further down the module, or further down a function, is read once the class exists, and relates as that class does:
class Out[out T]: pass
def chk[T](x: object) -> bool:
return x is T
x = Out[Later]()
class Later: pass
assert chk[Out[Later]](x)
what a stub says of a class is written with the helpers a build writes, for every class any of its
modules reads as a type, so the module itself reads as it was written. a module with nowhere to put
that, because its helpers are pasted into it, holds it if it tests something, and a class nothing
in the program reads as a type is treated as invariant unless its own TypeVar declares otherwise,
and as deriving from what its runtime class derives from. what
the runtime can't see, such as the arguments of a value that records none, answers
False, and failing the test says nothing more of the value than it would in place: in the else
of if x is T, x is what it was. a tuple records none, but its elements are there to ask:
(1, "a") is a (int, str)
a function that tests x is T, or hands T to something that does, refuses a type
argument the runtime can't test (erased-type-check), for the same reason
x is UserId is refused: a NewType, a TypedDict, LiteralString, Any, and a
callable with a signature. nothing about a value says it is one of those:
from typing import NewType
UserId = NewType("UserId", int)
def chk[T](x: object) -> bool:
return x is T
def hands[T](x: object) -> bool:
return chk[T](x)
chk[UserId](1) # error
hands[UserId](1) # error
when no injectable solution exists, the bare call is an error
(unspecialized-reified-generic):
def f[T](t: object):
print(T)
f[int](1) # ok
f(1) # error — `T` appears nowhere in the signature
def g[T](t: T):
print(T)
def local():
class Hidden: ...
g(Hidden()) # error — `Hidden` has no runtime spelling at the call site
args = (1,)
g(*args) # error — the injection cannot pass through unpacking
class K: ...
def shadowed(v: K):
K = 5
g(v) # error — the nearer `K` hides the class the call would name
a type parameter with a default (PEP 696) fills its reified slot when
neither the call's arguments nor an explicit [...] supply one — and an
inferred solution beats the default, exactly like a passed argument beats a
parameter default:
def d[T = int](t: T):
print(T)
d("") # transpiles to d[str]("") — the argument wins
d(0) # prints int
def f[T = int]():
print(T)
f() # prints int — default supplies the reified value
f[str]() # prints str
a type argument given by name¶
a type argument can be given by name, as it can for a class. the positional ones fill the parameters no keyword named, and a parameter given nothing takes its default:
def pair[T, U = str]() -> object:
return (T, U)
pair[U=int, T=bytes]() # T is bytes, U is int
pair[T=bytes]() # T is bytes, U is str
a name binds against the signature the call is checked against, so through a base class it reaches an override at the position it named in the base, whatever the override calls that parameter. a keyword-only parameter is the exception: it reaches the override's parameter of the same name
a type parameter the body forwards¶
a body that only ever hands a type parameter on still needs its value when what it hands it to records or reads a specialization. it needs no keyword: the parameter is reified because the body forwards it, however many hops the type argument travels
class Box[T]:
init(var t: T)
def wrap[T](t: T) -> object:
return Box(t) # transpiles to Box[T](t)
def hop[U](u: U) -> object:
return wrap(u) # transpiles to wrap[U](u)
print(hop(1) is Box[1]) # `U` and `T` are both `1` — prints True
a forwarded parameter stays out of the signature's contract. wrap is still an
ordinary callable to the checker: it can be passed to map, and a call with no
spelling for the type argument — an untyped value, a class local to a function —
runs anyway. the cell then keeps the TypeVar, so the box records nothing
useful and is Box[int] answers False. this is the same best-effort rule as
type reification
the exception is a parameter forwarded to something that reads the value: a
reified function, or a reified class. a call that cannot supply it is then
unspecialized-reified-generic, exactly as if it were declared reified
a function that forwards pays what a reified one does, a closure rebuilt per
specialized call, and is left to the interpreter by
by compile. two places have no cell to
forward from, so a construction there records nothing: a classmethod, whose
binding hides the function, and a class's own type parameter, which would have
to be read off the instance, and so make that parameter invariant. below python
3.12, or 3.13 for a type parameter with a default, a function that forwards is a
transpile error, as a reified one is: erased, it would build values that record
nothing, and hop(1) is Box[1] would answer False
subtyping¶
reification changes what counts as a valid subtype. once [...] is a
runtime-significant step, the type-parameter list becomes part of the
callable's interface — it is no longer phantom. assignability between two
reified generics requires the type-parameter lists to be compatible, on top of
the usual value-parameter and return-type rules
these rules are enforced today in the two places they can already be expressed: a reified generic is not assignable to a plain callable (the reified-and-erased distinctness rule), and a method override must keep its base's reified interface — arity, positional matching, contravariant bounds, and reified/erased status are all checked at the definition and reported as
invalid-method-override(see overrides). the writablegeneric[[T], () -> bool]type-form is not yet available as an annotation; examples written with it describe the intended behaviour
arity must match¶
a caller specializes through the target type, so the source must accept
every specialization the target permits. the type-parameter lists must agree
on arity, counted with the same rules value parameters use — defaults, *Ts,
and **P all participate:
type F = generic[[T], () -> bool]
type G = generic[[T, U], () -> bool]
def one[T]() -> bool: ...
def two[T, U]() -> bool: ...
f: F = one # ok
g: F = two # error — F supplies one type argument, two needs two
a default on the target makes the corresponding source slot optional in the same way a defaulted value parameter does
names do not matter, positions do¶
specialization is positional (f[int]), so type-parameter names are
irrelevant to assignability — the same way value parameters match by position,
not by name:
def a[T](x: T) -> T: ...
def b[U](x: U) -> U: ...
a_ref: generic[[T], (T) -> T] = b # ok — T and U are positional
the exception is keyword-specializable type parameters (see generics), which are matched by name just as keyword value parameters are
bounds are contravariant¶
a reified type parameter constrains what concrete types a caller may supply, so its bound behaves like an input — assignability is contravariant in the bound, mirroring value-parameter types. the target's bound must be assignable to the source's bound:
def narrow[T: int](): ...
def wide[T: object](): ...
n: generic[[T: int], () -> None] = wide # ok — wide accepts everything narrow does
w: generic[[T: object], () -> None] = narrow # error — narrow rejects non-int args
reified and erased are distinct¶
a reified generic is not assignable to a plain callable, and vice versa. the plain callable has no slot for the specialization step, so substituting one for the other would skip a structurally required call:
an erased generic (type parameter used only in annotations) keeps its existing PEP 695 assignability and remains usable wherever a plain callable is expected
overrides keep the reified interface¶
a method override is where these rules bite today: a.f[int]() on a: A
dispatches to the override at runtime, so the override must accept every
specialization the base permits. an incompatible list is reported at the
override's definition as invalid-method-override:
class A:
def f[T](self):
print(T)
class B(A):
def f[A2, B2](self): # error — the base supplies one type argument
print(A2, B2)
the same applies to bound narrowing (contravariance), to erasing a reified
method (the base's f[...] would subscript a plain function), and to
reifying an erased one without PEP 696 defaults (a bare call through the
base could not supply the values)
variadics and keyword packs¶
a *Ts reifies like any other type parameter, to the run of type arguments
it absorbs: the parameters before it claim theirs from the front, those after
it from the back, and the wrapper packs the rest into one tuple cell, so a
list containing a variadic has no upper arity bound
def f[T, *Args]() -> None:
print(T, Args)
f[int, str, bool]() # T is int, Args is (str, bool)
f[int]() # T is int, Args is ()
a variadic never makes the specialization step mandatory the way a plain reified parameter does — supplying it nothing is a complete answer, not a missing one — so a bare call stays legal. the run it binds is solved from the arguments, the same way a lone type parameter and a keyword pack are, so writing the step out and leaving it off reach the same answer:
def f[*Ts](*args: *Ts) -> None:
print(Ts)
f(1, "a") # Ts is (Literal[1], Literal["a"])
f[int, str](1, "a") # Ts is (int, str)
each element of the run is what the checker solved for that argument. a float
literal hands on its class under the file's numeric model, as a lone type
parameter's does — 2.0 binds float, not the int | float that a float
argument is merely accepted as
inference can only fill the step with types that have a runtime spelling at the call site. a class local to a function does not, so rather than bind a run naming something the call cannot see, that call is rejected and the step has to be written out
a PEP 696 default is a run too, and fills the slot as one:
a keyword-variadic pack reifies to the mapping of its fields. the pack sits outside the positional slots entirely — the other parameters are given positionally and the pack takes the keyword fields — and, like a variadic, an unfilled one is empty rather than missing:
def f[T, **Kwargs]() -> None:
print(T, Kwargs)
f[int, foo=str]() # T is int, Kwargs is {"foo": str}
def g[**Kwargs]() -> None:
print(Kwargs)
g() # Kwargs is {}
unlike a variadic, a pack is inferred from a call's keyword arguments when
**kwargs: **Kwargs unpacks it into the parameter list:
def f[**Kwargs](**kwargs: **Kwargs) -> None:
print(Kwargs)
f(a=1, b="x") # Kwargs is {"a": int, "b": str}
python's subscript grammar takes no keywords, so a keyword specialization
lowers to the wrapper's __getitem__ call — f[int, foo=str]() becomes
f.__getitem__(int, foo=str)(). an erased pack is stripped like any other
erased specialization, so nothing calls a __getitem__ a plain function does
not have
**Pin a.pyor.byifile is a PEP 612ParamSpec, not a pack, and is never reified: a parameter list has no runtime object to bind to a cell
round-tripping¶
the reverse transpiler recognizes the @generic wrapper by its
ReifiedGeneric provenance in the LoweringMap and re-sugars it back to a
bare def f[T] with reified body usage, restoring the unstripped f[int](...)
call sites. a hand-written @generic decoration without that provenance is
left untouched