Changelog¶
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
1.8.0 (2026-10-04)¶
Features¶
1.7.2 (2026-10-01)¶
Bug Fixes¶
1.8.0 — rows in DEFAULT that no run would reach are reported¶
An application that writes its own key values can put a row into DEFAULT after the table
was adopted: an event dated last year lands there because no partition covers it. Creation
walks forward from the cursor, so nothing ever came back for it. plan said there was
nothing to do, plan --check exited 0, and retention could not drop what was never in a
partition. Found by Outpost's maintainer: its publish API accepts a client-supplied time.
plan now reports such rows as rows_in_default, a WARNING, so plan --check exits 3 and
apply carries it in result.issues; backfill moves them into a partition of their own.
What decides is the earliest row in DEFAULT: one in a window the same plan creates is moved
by its attach, and one dated ahead of the cursor by the run that gets there, so neither is
reported. During an adoption the history waiting for backfill is reported the same way,
until it has moved. ActualTree.default_earliest carries the earliest key, read with the
same index probe partition_data uses.
1.7.2 — a backfill's last attach no longer rejects a live insert¶
The attach that ends a backfilled window is documented to hold writers at the parent, and
when rows had arrived in DEFAULT during the fill it did. When none had, it ran a plain
ATTACH first, which does not: an insert into that window that got past the parent while
the attach was in flight chose DEFAULT from the tree as it was, queued on DEFAULT's lock,
and was then rejected by the constraint the attach had just narrowed, new row for
relation ... violates partition constraint. The current month, the one the application
is writing into, is exactly such a window. Found against Outpost's schema with a writer at
20 inserts a second: two inserts failed in one backfill. A window filled from DEFAULT now
always attaches the documented way, writers held at the parent and whatever arrived after
the last batch taken with it.
1.7.1 (2026-10-01)¶
Bug Fixes¶
1.7.1 — retention no longer deadlocks against a reader¶
A blocking detach, the one retention runs on a table with a DEFAULT partition, locked the
partition and then waited for the parent. A query through the parent locks the parent and
then its partitions, so a reader that took the parent in between waited for the partition
while the detach waited for the parent, and PostgreSQL ended one of them with deadlock
detected after a second. Under steady reads that was a likely outcome, not a rare one:
it was found against Outpost's schema with one reader counting the table in a loop, and
the reader was the one that failed. The detach now takes ACCESS EXCLUSIVE on the parent
alone first, the lock DETACH takes anyway, and then the partition, the order readers and
PostgreSQL's own DETACH use. Readers wait for it as they always did; none fails.
1.7.0 (2026-09-29)¶
Features¶
1.7.0 — rows a stopped backfill hid are found and attached¶
A backfill moves a window's rows out of DEFAULT into a partition it attaches last. If the
process died after the move and before the attach, the rows sat in a table no query
through the parent sees, DEFAULT held nothing that would bring a later backfill back to the
window, and plan and validate both said the table was fine
(#76). 1.6.3 covered an attach
that fails; this covers a process that never gets to fail.
plan now reports such a table as unattached_rows, a WARNING, so plan --check exits 3
and apply carries it in result.issues. partition_data — backfill — attaches it once
DEFAULT is drained, through the same path a window out of DEFAULT takes. A table counts
only when its name is the very name the scheme gives a window under this root:
metrics_remote__2025_01 beside metrics reads as January, and is not a partition of it.
Its rows are read only after that, so a table that merely shares the prefix is never read
and needs no grant. validate is unchanged: it checks the document against the table's
shape, not the table's contents.
get_unattached_tables(table_name) is a method of PartitionMetadataProvider in both
mirrors, answering with the tables named under the root, attached to nothing and carrying
no orphan marker; a provider of your own now has to answer it. ActualTree.unattached and
UnattachedTable carry what it found.
1.6.3 (2026-09-29)¶
Bug Fixes¶
1.6.3 — a backfill that cannot attach gives the rows back¶
partition_data moves a window's rows out of DEFAULT in batches, each committed on its own,
into a partition it attaches last; until the attach, those rows are visible through no
query. When the attach failed, the rows stayed in that detached table, and if the fill had
already taken the last of them, nothing ever looked at the window again: the next
backfill reported DEFAULT drained, validate said ok, and the rows were gone from every
read. 1.6.1 made that easy to reach. Its lock timeout refuses an attach after 3 s instead of
waiting out the 30-second budget, so a reader holding the table for a few seconds was
enough, and the issue then claimed the rows stayed in DEFAULT and nothing had changed.
Now whatever stops the attach, the rows the fill moved go back to DEFAULT, pinned to both tables' identities, and the issue is true: the rows are where readers see them, and the next run takes the window again. If they cannot be returned, the issue names the table they are in instead. A process killed between the fill and the attach can still leave them there, with nobody to move them back; finding and finishing such tables is #76.
1.6.2 (2026-09-29)¶
Bug Fixes¶
1.6.2 — a DSN with sslmode¶
A postgresql:// DSN carrying ?sslmode=require crashed the CLI before it connected:
TypeError: connect() got an unexpected keyword argument 'sslmode'
(#73). The CLI drives such a
DSN with asyncpg, and SQLAlchemy hands the URL's parameters to asyncpg.connect() as
keywords; asyncpg reads sslmode, sslrootcert and the rest of libpq's TLS parameters
only from a DSN it parses itself. They now travel to it as one, so a DSN copied from psql
or a managed PostgreSQL console works as written, sslrootcert included. Everything else
in the DSN stays exactly as it was.
1.6.1 (2026-09-29)¶
Bug Fixes¶
1.6.1 — a lock wait no longer holds every writer¶
An attach, a row move and a blocking detach now run under lock_timeout:
ddl_lock_timeout_ms on the repository and under runtime: in the document, 3000 by
default. Until now only a drop did. A statement waiting for a lock keeps its place in
PostgreSQL's queue, and every writer of the table waits behind it: with a reader holding
the table open, inserts stalled for the whole 30-second statement budget, and apply then
ended with an empty database error: and exit 5
(#75). Now the step gives up
after the timeout having changed nothing, it is recorded in result.issues —
partition_data reports its window as not finished — the rest of the run goes on, and the
next run tries again.
plan --locks said the attach that clears a DEFAULT partition takes SHARE UPDATE
EXCLUSIVE on the parent. It takes EXCLUSIVE, which holds off writes. That is deliberate,
since 1.5.1, so that no writer is routed into DEFAULT between the move and the attach,
and the plan now says so.
A timeout that carries no message of its own is reported as database error: timed out
instead of an empty line.
1.6.0 (2026-09-29)¶
Features¶
1.6.0 — plans on the database's clock¶
A plan is made against the database's clock, now(), unless the caller names an instant.
It used to be made against the clock of the process running maintenance, and nothing
compared the two: a container whose clock ran ahead decided retention for a date the data
had not reached. With the container at June 2027 and the data at September 2026, one
apply --allow-destructive detached and dropped every monthly partition, the current one
included, and exited 0 (#74).
pg_partman never had this problem, because it runs inside the database.
current_time() is a method of PartitionMetadataProvider in both mirrors, and a provider
of your own now has to answer it with the clock of the database it reads. plan(now=...)
still plans against whatever instant it is given.
Not moved yet: the instant a detach is stamped with, which the orphan grace period counts from, is still the process's, so a clock that is off moves the end of the grace period by as much. Moving the stamp changes the repository protocol and is its own change.
1.5.1 (2026-09-07)¶
Bug Fixes¶
1.5.0 (2026-09-04)¶
Features¶
1.4.0 (2026-09-03)¶
Features¶
1.4.0 — the closed check, on the protocol¶
is_partition_closed(partition_name, *, settle_seconds=0, boundaries=None) is a method of
PartitionMetadataProvider in both mirrors. It was always the export pipeline's question —
the archiving guide has a section for it, and PartitionToolkit.from_engine takes a
boundary_codec for that one call — but only the Postgres provider declared it, and the
toolkit hands its provider back under the protocol's type. So under a type checker
kit.metadata.is_partition_closed(...) did not exist, and a caller narrowed with
isinstance first, which is what the toolkit was built to make unnecessary.
A provider of your own now has to answer it. The contract is the Postgres one: True once the partition's upper bound plus the settle buffer has passed on the database's clock; False, never an exception, when the bound cannot be read — a DEFAULT partition, a non-RANGE one, an unbounded upper bound, a bound the codec cannot decode, a detached table, a name that does not resolve.
1.3.0 (2026-09-03)¶
Features¶
- a prefix for the environment, and a stop that waits for nothing (d555cde)
- a prefix for the environment, and a stop that waits for nothing (45160a5)
Bug Fixes¶
- the token check inside the guard, exit 3 by SQLSTATE, and three sentences (8267d89)
Documentation¶
1.3.0 — a prefix for the environment, and a stop that waits for nothing¶
PartitionTableSettings reads PG_PARTSMITH_TABLE_NAME and the rest under that prefix
when it is used as it is. It used to read bare names, so a host's TZ became the
calendar's timezone and an empty one was a validation error. A subclass with a prefix of
its own — the documented shape, EVENTS_TABLE_NAME — is untouched. If you used the base
class directly, add the prefix to your variables.
In the async mirror, and so in the command line, a Python block hook runs on a thread of
its own. SIGTERM used to wait for a block to return, because the block ran on the loop
that handles the signal, and a block that outlasted the grace period had the process
killed under it. The stop now cancels the run, which cleans up and exits 143 at once; a
block still running at that point is abandoned with the process. The sync mirror runs the
block inline, as before: there is no loop to keep free.
The command line tells a database that said no apart from a database that was not there.
A statement the server refused — a missing grant, a constraint — is exit 3, a finding
for a person, where it used to be 5 with the connection failures.
The Redis lock recognises its own token. redis-py retries a SET NX that timed out, and
the first attempt can have landed: the retry answered "not set", and the run gave up on a
lock it held until the TTL expired it. A key that carries the attempt's token is the lock,
acquired. The manager reads the key back for that, so a client of your own handed to
RedisDistributedLockManager needs get alongside set, exists and register_script.
The command line also classifies a database error by its SQLSTATE rather than by the exception class the driver adapter happens to choose: under asyncpg only a syntax or access error and a constraint violation came back as their own classes, and a statement timeout or a dependent object would have stayed with the connection failures.
Smaller: the agents page lists the cli extra; the plan-artifact guard is described as
what it is, a fingerprint of the table's configuration rather than of the whole document;
the lock guides say that redis-py 8 speaks RESP3 and that a server older than 6 wants
?protocol=2; the docs build installs from the lock file like every other job, and refuses
a stale one; and the
Dockerfile names no Python version outside the builder stage.
1.2.0 (2026-09-02)¶
Features¶
Bug Fixes¶
- name the path when --write or --save cannot write it (ec7290b)
- what the review found in the command line and the document (8e5333b)
Documentation¶
- file the notes under 1.2.0, which is where they ship (648f0f9)
- the image as it is, the counts as they are, and a reference page for what 1.2.0 adds (5009656)
1.2.0 — a command line, an image, and hooks from a config file¶
Everything here is reachable without writing Python: pg-partsmith on the command line
or in a container, a document that describes every table a deployment maintains, and
hooks that run a command. Underneath, the library gains what those needed — the
collaborators built once and handed back by name, the document as a model, and a plan
that refuses to be applied under a configuration it was not made from.
The plan as an artifact¶
plan() and apply() were always separable and the plan was always a pydantic model, but
nothing checked that the plan handed to apply() was the one this configuration produced.
In one process the pair cannot disagree; written to a file by one run and read by another
— which is the whole point of the split — it can.
MaintenancePlan.config_fingerprintrecordsTablePartitionConfig.fingerprint, a digest of the configuration the plan was made from.apply()now refuses a plan made for another table, and one made under a configuration that has since been edited, withPlanConfigMismatchErrorand before any DDL runs;allow_config_drift=Trueapplies it as it stands. The identity revalidation each destructive operation already did answers whether this is still the same relation, not whether it is still the same intent — a plan made underretention_count=12names exactly the right partitions to expire for a reason that stopped being true the moment someone wrote120.MaintenancePlan.cursorsis typeddict[str, int]. Only an integer axis records a cursor — a time axis is cursored bygenerated_at— and leaving itAnymeant a plan written to JSON did not always read back as the plan that was written.validate_plan_for_config(config, plan, *, allow_config_drift=False)is exported for anything that pairs a plan with a configuration itself.
PartitionToolkit¶
PartitionToolkit.from_engine(engine, ...) in both mirrors builds the repository, the
metadata provider, the lock manager, the service and the maintainer around one engine, and
returns them by name. Three settings live on two objects and have to agree — marker_prefix
(one writes the orphan marker, the other looks for it), ddl_timezone (one writes naive
boundary literals, the other reads them back, and the two constructors do not even default
to the same value) and boundary_codec — and each is now given once. Parts rather than
just a maintainer, so code calling metadata.is_partition_closed or locks.acquire_lock
directly does not build a second set to keep in step. Construct PartitionToolkit(...)
directly to hold parts of your own.
A configuration written down¶
PartitionsDocument describes every table a deployment maintains, and the wiring it
maintains them through, as one validated model — what a ConfigMap, a mounted YAML or a
JSON blob parses into. document.configs() yields the TablePartitionConfig of each,
config_for(name) one by name, and PartitionToolkit.from_options(engine, document.runtime)
builds the collaborators the file describes. The library still reads no files and opens no
connections: parsing belongs to whoever owns the format.
PartitionTableSpec is one table's entry, and PartitionTableSettings is now that same
model read from the environment rather than a second field list beside it — so a field
added once is available in both. ToolkitOptions is the runtime section, one field per
keyword of PartitionToolkit.from_engine.
Unknown keys are refused wherever they appear, including in defaults, where a typo would
otherwise reach every table at once. A document with no tables, and one relation described
twice, are refused for the same reason: both are silent at 03:00 otherwise.
A command line¶
pip install "pg-partsmith[cli]" installs pg-partsmith, which runs the library's
read-only half over a document and a DSN — no Python in the calling application at all.
Three of the four commands issue no DDL, take no lock and fire no hook:
inspect— the tree that actually exists.plan— what maintenance would do and why, straight fromMaintenancePlan.validate— the document against the catalog. The one that belongs in CI.
Exit codes are the point of a CLI a CronJob runs, so they are distinguishable: 2 for
drift under plan --check (what "maintenance has stopped running" looks like), 3 for
findings that need a person, which outranks drift, 4 configuration, 5 the database, and
6 for a lock another maintainer holds — ordinary operation, and the first false page
every deployment gets if it is reported as a failure.
The arguments are typer declarations — every flag typed, the help generated from the same
declarations that parse it, shell completion via --install-completion. A usage error exits
64 (EX_USAGE), never the 2 argument parsers default to: 2 is what this tool means
"drift" by, and a CronJob alerting on drift must not page over a typo.
A stop signal cancels the run rather than killing the process. SIGTERM — what a pod is
sent at its deadline — used to end it on the spot, with no finally, no line in the log
and, under a command hook, an archiver left running unwatched. Both it and Ctrl+C now take
the path a raised exception takes: the statement is cancelled and rolled back, the lock
released, the engine disposed, a hook's child terminated, and the exit code is 130 or
143 with a line on stderr saying which.
--output json is the models' own dump under a versioned envelope, by_alias=True, so
what comes out of plan and what goes into a document are one vocabulary; logs go to
stderr. The connection comes from --dsn, then $PG_PARTSMITH_DSN, then the document.
--version reads pg_partsmith.__version__: one number for the library and the CLI.
apply carries maintenance out, and withholds every destructive operation unless
--allow-destructive is given — the safe behaviour is the default one rather than a second
mode to remember, and it is what an init container wants. With no --plan it plans and
applies under one lock, which is also what finalizes an interrupted DETACH CONCURRENTLY.
plan --save FILE writes the artifact between the two halves, and apply --plan FILE
reads it back. Both guards from earlier in this release apply to it: a plan made for
another table, or under a configuration that has since been edited, is refused with exit
4 unless --allow-config-drift is passed.
Commands around the lifecycle¶
CommandHooks runs a configured command at each lifecycle phase, handing it the
PartitionEvent as JSON on stdin and treating a non-zero exit exactly as a raised
exception. It needed no new context: the event has been one pydantic model for every phase
since 1.1.0, so the payload is a dump of the object a Python hook already
receives. In a document it is a hooks section, and it is what makes the destructive half
usable by a team that does not write Python — they own the archiver already, they only
need it invoked at the right moment.
The commands run only when the run is told to run them (apply --allow-hooks), and a
document declaring hooks is refused rather than quietly stripped when it is not: silently
skipping a configured before_drop would let an operator read the file, believe their
archiver ran, and be wrong. No sandbox is provided and none is claimed; the container
boundary is the answer if isolation is wanted.
Both mirrors run the command through one implementation, so it does not depend on the event loop being able to spawn processes — a hook that works in production works on a developer's machine too.
A block of Python instead of a command¶
A hook phase in a document can take a block of Python — inline as python, or from a
file as python_file relative to the document — evaluated with event and log in
scope, raising to refuse. PythonHooks is the same from code, in both mirrors.
Every block is compiled when the document is read, and a file-backed one by validate: a
SyntaxError is a validation error with a line number in it, not something a CronJob
finds at 03:00 after some of the run's DDL has committed. Tracebacks name the phase, or
the file, rather than <string>.
It is behind the same --allow-hooks as a command, for the same reason, and no sandbox is
claimed: exec with a filtered __builtins__ is not a security boundary, and the
documentation says so rather than pretending otherwise.
What a plan will lock¶
Every operation now serializes its capabilities — the heaviest lock it takes and on
what, and whether it can run inside a transaction block — and is_destructive beside its
fields. They were on the object before and not in the dump, which meant a plan written to
a file lost the one thing a DBA reads before agreeing to a window. Both are computed on
the way out and ignored on the way in, so the round trip holds. plan --locks prints the
same for a person.
There is deliberately no --sql. The DDL is built at execution time from decisions only
made then — a concurrent detach falling back to the blocking form, rows moved out of a
DEFAULT partition, an orphan marker cleared — and a rendering without them would be
read as a transcript in exactly the cases a review exists to catch.
Monitoring, for free¶
--output metrics renders any command as Prometheus text exposition for a node_exporter
textfile collector. Most systems that partition PostgreSQL have no monitoring of it at all
— they find out at 03:00, from an insert PostgreSQL rejected — and a CronJob already runs
this on a schedule, so the cheapest monitoring available is for that run to leave a file
behind.
plan emits pending operations by kind and findings by severity, inspect the size of the
tree and the age of the oldest orphan, validate whether each table still matches its
configuration, and apply what it did and what it could not. Every run also stamps
pg_partsmith_run_timestamp_seconds, so a textfile nothing refreshed reads as stale rather
than as good news, and a converged table reports zeroes rather than nothing — a missing
series and a zero are different alerts.
The numbers come off the same envelope the JSON does, so a metric cannot disagree with what the same run printed.
Examples that are tested, and a schema for the editor¶
examples/ holds a minimal document, one with everything in it, an id-partitioned queue,
cold tiering, and three hook scripts — a pg_dump before a drop in bash, a webhook after
a create, a COPY out before a detach as a Python file. The test suite validates every
document, compiles every hook and syntax-checks every script, so an example that stops
being right fails a build rather than teaching the wrong vocabulary.
pg-partsmith schema prints the document's JSON Schema; with one comment line at the top
of the file, an editor validates keys and completes them. The committed copy is checked
against the generated one.
Every way to run it¶
A deployment page with a copy-pasteable shape for each harness: plain Docker with host
cron or a systemd timer; Compose as a service you run, as the equivalent of an init
container (service_completed_successfully), and on a schedule with a cron sidecar; Swarm
with swarm-cronjob; Kubernetes as a Pod, a Job, a CronJob and an init container, plus the
plan-on-one-day-apply-on-another pair of Jobs and the node_exporter textfile; GitHub
Actions and GitLab CI; and the note that anything which runs a container and reads an
exit code runs this.
PG_PARTSMITH_DSN_FILE reads the connection string from a file — the Docker and Swarm
secrets convention — after PG_PARTSMITH_DSN and before the document, so a Compose
stack hands a secret over without a shell wrapper and without it showing in
docker inspect.
What is inside the image¶
The runtime is distroless: glibc,
OpenSSL, CA certificates, a timezone database, and nothing else — no shell, no package
manager, no pip. Python 3.14 and a pruned standard library are copied in from the build
stage, the virtualenv is installed from uv.lock with uv sync --locked so the image holds
the versions the tests ran against and a stale lock fails the build, compiled extensions
are stripped, bytecode is precompiled, typer's rich is left out, and exactly the five
shared libraries the kept extension modules link against come along. There is no Debian
userland to carry: the budget is 160 MB, the image sits well under it, and a start is an
interpreter with nothing to byte-compile.
Two flags replace the two things a shell wrapper in the image used to do: --write FILE
writes any command's output to a file atomically (a node_exporter textfile), and
apply --ok-if-locked makes a held lock exit 0 (an init container, which restarts on
anything else). A command hook inside the image is therefore a binary or a Python file
run through /opt/venv/bin/python, not a bash script; the docs say so, and say how to
build an image with bash in it in one line.
Published images carry an SBOM and a SLSA provenance attestation and are signed keylessly
with the release workflow's identity; cosign verify is in the docs. Dependabot watches
the base images, the actions and the lock file.
Pipelines¶
CI caches the uv environment keyed on uv.lock and installs with --frozen, so a stale
lock fails rather than resolving something else, and uv lock --check says so in one
line; a push cancels the run it supersedes; the image build reuses layers across runs.
The matrix grew in every direction that has produced a bug before. Unit tests run on
Python 3.11 through 3.14 on Linux, at both ends of that range on Windows and macOS, on
arm64, and with the clock at UTC+14 and at UTC-12. The integration suite runs on
PostgreSQL 15 through 18, on arm64, on Windows and macOS against a server the runner
installs itself, and once with the server at UTC+14 and the client at UTC-12. One job
installs every direct dependency at the lowest version pyproject.toml admits and runs
both suites on it; it found the declared floors for pydantic and typer below what the code
needs, and they now say what is true. The wheel is built, checked and installed on its own
on all three platforms, and the command is run through to a refused connection. The image
is built and run on both architectures the release publishes. Warnings are errors under
pytest; the workflows go through actionlint and the Dockerfile through hadolint; CodeQL
reads the library and the workflows; a pull request that adds a dependency with a known
vulnerability is refused; and once a week every link in the docs is asked whether it still
answers.
The image is run end to end, on both architectures: as a container with a read-only root
filesystem, every capability dropped and the document mounted, against a PostgreSQL
container on a network of its own — the commands the guides show, with the codes they
promise. A fresh table drifts and apply converges it; --output json is the envelope
alone on stdout; --write lands a textfile in a volume; the DSN arrives from a mounted
secret; a secret only root can read, a rejected password and a document with hooks but no
--allow-hooks each answer with their code and a sentence; SIGTERM during a command
hook is exit 143 inside the grace period with the lock released; an overlapping run
stands aside with 6; plan --save in one container is applied by another and refused
once the document changed; an image derived with one COPY runs a Python file hook from
inside; and the Compose shape that holds an application until its partitions exist runs
as written. After a release, the same suite runs against the image that was actually
pushed, pulled back on both architectures, once the signature, the SBOM, the provenance,
the version inside and the minor tag have been checked. And the Job and CronJob manifests
the running guide shows go through kubeconform on every pull request, while a unit test
reads every YAML block in the guides and refuses a flag the command does not take.
The checks paid for themselves before they were merged. The command line no longer dies
on a console that cannot show an em dash: a Windows pipe or a bare POSIX locale gets the
character escaped rather than a traceback. A password the server rejects is exit 5 like
any other database failure, where it used to be a traceback: the driver raises its own
error before SQLAlchemy has one to wrap. A hook command that was stopped mid-run had its
pipes left open, which a long-lived process would have seen as a warning at garbage
collection; they are closed. And main() runs its coroutine under a runner with a loop
factory of its own, so a program embedding it next to an event loop of its own finds that
loop where it left it.
Running the image found two more. --write into a directory the container's user cannot
write — a root-owned hostPath is the usual case — was a traceback, and plan --save into
one was reported as a database error; both are exit 4 with the path in the message, and
the previous textfile stays where it was. And a Python block hook still running when
SIGTERM arrives holds the stop until it returns, because it runs on the loop that
handles the signal; the guides now say so, and that work which can outlast a grace period
belongs in a command hook.
A read of the whole change before the release found the rest. A run that stopped on a
hook's refusal or a stale plan was a traceback, and is exit 3; a connection string
SQLAlchemy cannot parse was reported as the database's fault, and is exit 4; an unknown
runtime.boundary_codec and an empty command vector passed validate and failed at
apply, and are refused where they are written; a command hook on a table partitioned by
a sequence crashed the run building the hook's environment, because a window edge there is
a number; and a bare pip install pg-partsmith put a pg-partsmith on PATH that died
with a traceback, where it now says what to install and exits 64. The release itself
would have failed after the PyPI upload, building arm64 on an amd64 machine with nothing
able to run its binaries: the image is now built on each architecture natively, scanned on
each, and finished before anything reaches PyPI.
A container image¶
ghcr.io/bedrock-python/pg-partsmith:<version>, published from the release workflow, so a
Go, Ruby or Node stack runs partition maintenance as a CronJob, a Compose service or an
init container with no Python of its own. The entrypoint is the command itself, it runs as
UID 65532, and the build stage — pip, the build backend, every __pycache__ — is thrown
away rather than shipped.
The library, the CLI and the image carry the same number, always; CI refuses to publish an
image whose --version disagrees with its tag. Tags are the exact version, a moving
minor, and latest for the newest release; a schedule belongs on the minor, since
following latest it would cross a major on its own, at 02:15, with DROP in its
hands. The image size is checked against a budget on every
pull request and a regression fails the build.
The init-container case needs no flag of its own: apply without --allow-destructive
creates and retires nothing, which is exactly what running maintenance at application
startup should do. The documentation states the grants this thing needs and recommends a
dedicated role, because it issues DDL and nobody should have to guess that.
1.1.0 (2026-09-02)¶
Features¶
Bug Fixes¶
- close the review findings on the hook event (38d8c23)
- recognise an attach failure by its SQLSTATE, not by SQLAlchemy's type (5f9185a)
- refuse a mismatched orphan marker, and let the boundaries answer for themselves (1245fd9)
- say when a partition's closed-ness rests on the session timezone (7c44a09)
Documentation¶
- how to query a partitioned table, what a plan costs, and the final report (a82264b)
- mark the two cheat-sheet entries 1.1 superseded (0ddff78)
1.1.0 — one event for every hook¶
This breaks every hook written against 1.0. It is shipped in a minor version on purpose: the library is days old and has no dependants to protect, and carrying the old shape until a major would mean carrying it for years. Ported hooks are two lines of change each; unported ones are refused when the service is constructed, with the fix in the message, rather than failing at their first call in the middle of a maintenance run.
What changed¶
Three of the six phases were handed a rich object and three a bare string, so a
before_drop that archived a partition could not know which period it was archiving —
and could not look it up either, because DETACH clears relpartbound. Every hook method
now takes one PartitionEvent:
| Field | What it is |
|---|---|
phase |
which moment this is (HookPhase) |
config |
the table's configuration — no longer something a hook has to be given separately |
partition |
PartitionInfo: name, bounds, OID, subpartition_type |
window |
the period covered; None for a member of a root HASH or LIST |
operation |
the planned operation: reason, detail, oid, size_bytes, row_estimate, detached_at |
table_name |
the root table, derived from config |
| Before | After |
|---|---|
before_create(config, partition) |
before_create(event) — event.config, event.partition |
after_create(config, partition) |
after_create(event) |
before_detach(table_name, partition) |
before_detach(event) — event.table_name, event.partition |
after_detach(table_name, partition_name) |
after_detach(event) — event.partition.name |
before_drop(table_name, partition_name) |
before_drop(event) — and now event.window, event.operation.reason |
after_drop(table_name, partition_name) |
after_drop(event) |
| — | before_attach(event) / after_attach(event), new: a detached partition going back into the tree |
| — | on_event(event), new: fires for every phase, just before the method named for it |
Also breaking, in the same area¶
PlanExecutor.detach_single_partitionanddrop_single_partitiontake theTablePartitionConfiginstead of the parent's name — it is what the event is built from, and every other entry point already took the config.PartitionLifecycleService.detach_old_partitionsanddrop_detached_partitionstake the config for the same reason:detach_old_partitions(config, partitions),drop_detached_partitions(config, names).
Added¶
PartitionEvent,HookPhase(exported frompg_partsmith), andon_event— one method for an audit trail or a metrics counter across every phase, instead of one identical delegating method per phase, plus another with every phase added. It runs before the method named for the phase, so abefore_*that refuses an operation by raising still leaves the audit trail with the attempt in it.before_attach/after_attach: a partition retention released is re-attached when its window is wanted again — retention grew, or create-ahead reached back to it — and that transition used to happen silently. An export taken while it was detached goes stale the moment it comes back, and this is where a hook is told.- Hook signatures are read when the service is constructed; one still taking the 1.0 arguments is refused there, naming the class, the method and the shape to move to.
1.0.0 (2026-09-01)¶
⚠ BREAKING CHANGES¶
- partition schemes, lifecycle policies and the maintenance plan, in the core and in both the aio and sync packages
Features¶
- batched data movement -- partition_data drains DEFAULT, unpartition moves everything back (9644df3)
- foreign-key aware retention -- Unreferenced() and detach refusals as issues (4257502)
- leaf backends -- local tablespace/storage/privileges and foreign leaves (1cecc57)
- partition schemes, lifecycle policies and the maintenance plan (core + aio/sync, tests pending) (90af1bb)
- sliding LIST progression and Redis lock integration tests (7cf7a4c)
Bug Fixes¶
- close every finding of the external review (F-001..F-016) (927c8e0)
- close the verification round -- referenced rows, pinned detaches, identity sequences (1fed77f)
- count every allocated identity value as possibly cached, not just the newest block (4571ad1)
- decide cycling identity sequences on their whole range, bound the cached region at START (343034e)
- decide every identity column before any sequence moves, and park under a name of our own (9a618c2)
- decide identity sequences on the ids a move carries, not on the destination's own rows (07b9d9a)
- finish interrupted concurrent detaches and accept positional rule arguments (b93adc1)
- hold created relations by OID; transactional FINALIZE; deferred FKs; descending identities (b541b3f)
- hold identities through subtrees, restores and pins; model identity sequences (5fd0bc6)
- measure plain detached tables, re-attach orphans retention wants back, report lifecycle units only (ce14093)
- refuse cached identity blocks, read each identity once, carry it into re-attached subtrees (cc3d72e)
- retention rules combine as predicates, no childless branches, foreign leaves satisfy windows, CreateUntil survives JSON (4e4d955)
Documentation¶
- 1.0 guides, RFC companions, changelog, and the sync mirror scripts (14252f9)
- hand any page over as Markdown, from the page itself (994a06b)
- keep the Markdown emitter inside the linter's rules (4715336)
- leaf backends, sliding LIST, data movement and FK semantics; CI matrix on PostgreSQL 15/16/17 (1235a3e)
- let a page decline the Copy page control (1a76685)
- name every way a sequence outlives the move that set it (ae58bbb)
- one page of context for AI agents, with a map of the rest (3c1474a)
- rebuild the documentation around tutorials, concepts, how-to guides and reference (0baa5b0)
- say on the front page that a page for agents exists (0fbf947)
Miscellaneous Chores¶
- cut 1.0.0 (7a2c6d0)
1.0.0 — the long version¶
Version 1.0 rebuilds the core around four separated concerns — partition scheme, range boundaries, lifecycle policy, and a maintenance plan — after reading how ten production systems (GlitchTip, GitLab, Centrifugo, PGMQ, Hatchet, pg-trx-outbox, Hookdeck Outpost, ColdFront, pg_partman, pg_clickhouse) manage PostgreSQL partitions. See RFC 0001, the OSS research and the verified PostgreSQL semantics.
⚠ BREAKING CHANGES¶
PartitionLifecycleService(repo, metadata, locks, hooks=...)no longer takes aperiod_calculator; the calendar, timezone and codec live in the config'sTimeBoundaries.PostgresMetadataProvider(boundary_codec=...)is only needed foris_partition_closed.- Nested and static topologies are spelled as a scheme:
subpartition=HashSubpartitionSpec(...)→subpartition=HashPartitioning(key=..., modulus=...)orscheme=RangePartitioning(..., child=...);root_layout=...+HASH_BASED/VALUE_BASED→scheme=HashPartitioning(...)/ListPartitioning(...).HashSubpartitionSpec,ListSubpartitionSpec,SubpartitionSpec,root_layoutandauto_attach_after_createare removed (a partition is always attached last, after its subtree). - Hooks:
before_create(config, partition: PartitionInfo)replacesbefore_create(config, partition_name, from_value, to_value). Hooks fire once per partition directly under the root (also for rootHASH/LISTmembers). - Repository protocol:
create_table_like(template, name, partition_by),attach_partition(parent, name, bounds, *, key_arity=1),detach_partition(parent, name, *, mode=DetachMode.AUTO),drop_partition(name, *, expected_oid=None),reconcile_default_rows(..., key_columns=...)replacecreate_partition,create_branch,create_subpartition_table,attach_subpartition,attach_composite_partitionand theconcurrent=flag. Metadata protocol:get_actual_tree,measure,get_partition_tree,get_relation_oid,get_key_high_water_markreplace theNestedPartitionMetadata/CompositeKeyMetadatacapability protocols. pg_partsmith.subpartition_planandpg_partsmith.pruning_rulesare removed;SubpartitionPlan/TopologyFinding/TopologyReason/plan_subpartitionsbecomeMaintenancePlan/Finding/FindingReason/plan_maintenance.service.reconcile_subpartitions()isservice.reconcile().- Removed with their feature:
SubpartitionAction,SubpartitionBoundsandSubpartitionReconcileResult(root imports),SubpartitionSpecBase(pg_partsmith.entities);ListGroupmoved topg_partsmith.schemeand stays exported frompg_partsmith. PeriodandPartitionGranularitymoved topg_partsmith.periods(still re-exported frompg_partsmithandpg_partsmith.entities). Custom calculators implementperiod_at(instant)(current_period()is derived from it; a calculator without it still works, slower).ensure_partitionsreturns the createdPartitionInfos built with their whole subtree;get_partitions_for_pruningalso lists detached orphans past their grace.- Ownership: an attached partition whose bounds are not a window of the scheme's grid (nor
inside one) is reported as
unmanaged_partitionand never detached or dropped. In 0.x any attached partition with an old upper bound was pruned. PartitionStrategygainsNUMERIC_BASED;MaintenanceIssueStepgainsATTACHandMOVE;UnsupportedCapabilityErroris removed with the capability protocols it served.- Repository protocol additions a custom implementation has to provide:
create_table_like(..., physical=) -> intandcreate_foreign_table_like -> int(both return the created relation's OID, read in the creating transaction),move_rows,reconcile_default_rows(..., limit=, expected_target_oid=),attach_partition(..., expected_oid=, expected_parent_oid=),detach_partition(..., expected_oid=),drop_partition(..., drain_into=) -> int; metadata protocol:get_leading_key_minimum,get_relation_kind.partition_existsnow reports foreign tables too.
Added¶
scheme=composition:RangePartitioning,HashPartitioning,ListPartitioningnest to any depth (RANGE → LIST → HASH,LIST → RANGE, rootHASH/LIST), with composite keys, per-level naming and 63-byte name budgets.NumericBoundaries(step, origin): integerRANGEwindows with the cursor read frommax(key)or the serial/identity sequence (CursorSource).LifecyclePolicy: creationCreateAhead,CreateUntil(position),CreateNextIf(predicate); retentionKeepNewest,KeepFor(age),KeepBehind(distance),ExpireIf(predicate)withAllOf/AnyOf/Not; predicatesSizeAbove,RowsAbove,WindowAgeAbove,SqlPredicate,Callback;DetachMode.AUTO/CONCURRENT/BLOCKING;DropAfter(grace, when),DropNever. Facts (sizes, row estimates, SQL answers) are gathered only when a policy asks.service.plan()(read-only, lock-free),service.apply(plan),service.maintain();MaintenancePlanwith typed operations (CreatePartition,AttachPartition,DetachPartition,DropPartition), reasons, sizes, findings with severities,without()/only()filters,describe()and JSON round-trip;MaintenanceResult.plan,attached_count.- Destructive revalidation: detach and drop check the relation's OID (and attachment /
marker) at apply time; a recreated table raises
PlanStaleError. - The orphan marker records the detach instant (
pg-partsmith:detached-at=), which grace periods are measured from; orphans whose window is wanted again are re-attached. ActualTree/PartitionNodecarryoid,relkind(foreign tables are inspected and never touched),detach_pending,facts;DetachedPartitioncarriesdetached_at.EpochBoundaryCodec; codecs and boundaries addressable by name in serialized configs;PartitionTableSettingsacceptsscheme/lifecycleJSON,tz,boundary_codec.- Every lifecycle rule with one defining value takes it positionally:
KeepNewest(12),CreateAhead(3),DropAfter(timedelta(days=7)),ExpireIf(SqlPredicate(...)). - An interrupted
DETACH CONCURRENTLYis completed by the nextmaintain()call, which re-plans under the same lock and re-attaches the finalized table when its window is still wanted — or retires it under the drop policy (detach_finalize). The tree is read with a recursive walk overpg_inheritsinstead ofpg_partition_tree, which omits such a partition and takesACCESS SHAREon every member; inspection now takes no relation lock. - Row moves (DEFAULT reconciliation,
partition_data,unpartition) fail closed under incoming foreign keys (RowMoveRefusedError, recorded as amoveissue): a destructiveON DELETEaction (CASCADE,SET NULL,SET DEFAULT) is refused up front, and a referenced row is refused atomically by PostgreSQL itself — mid-move it is outside the referenced tree, so evenNO ACTIONcannot pass, andSET CONSTRAINTS ALL IMMEDIATEkeeps aDEFERRABLEcheck from escaping to commit unhandled. Generated columns are recomputed rather than copied; identity values survive a move (OVERRIDING SYSTEM VALUE) and the target's identity sequences are advanced past the ids they could still reissue — along the sequence's own path and direction, inside its declared range. A destination whose sequence cycles (any id in its range: a wraparound restarts it at the far end and can move it onto values its increments skipped), would have nothing left to issue, or has handed cached blocks out to sessions — which keep them where nosetvalreaches, while PostgreSQL publishes only the newest allocation, so everything issued sinceSTARTcounts — is refused instead of quietly broken. Every one of those questions is about the ids the move carries, which the move statement hands over as it places them: a row the destination already held came from that sequence itself and never refuses a move. A destination with several identity columns is decided in full before any of its sequences moves, since PostgreSQL does not roll a sequence back with the transaction. unpartitionrefuses a destination that is the root, a member or an orphan of its tree, or a partition of any other table; empties this library's detached partitions too; and — underdrop_emptied— detaches, drains the tail, and lets the drop move whatever arrived late in the same transaction under the drop's lock, counted inrows_moved, so a row committed between the last batch and the drop is moved, never dropped.partition_datafills and re-attaches a matching detached partition instead of giving up on its window, and drains DEFAULT into foreign leaves. Every fill target is held by OID — read in the creating transaction for a fresh partition, the planned orphan's otherwise — verified on every batch (before and, for an unlockable foreign target, after the statement, rolling it back) and re-checked by theATTACHitself, which also verifies the parent it attaches into, so a branch replaced while its subtree is being built never gains the children. A compensating move back into a DEFAULT partition checks both ends. A name that resolves to nothing where an identity was expected is a stale plan, never name-only execution.- An OID-guarded detach fails closed (
detach_partition(expected_oid=)): the blocking form checks identity and attachment in its own transaction and re-checks after the statement, rolling a swap back marker and all; the concurrent form pins the relation (pg_partsmith.aio.repositories.pin) while it is verified and marked, releasing only once the statement's own backend is queued for the partition's lock (the sync mirror pins through a worker thread); a statement that has not got that far when the DDL timeout expires is cancelled and waited for before the pin is released. A foreign relation cannot be pinned and uses the transactional blocking form. Finalizing an interrupted detach is transactional too: lock, checks, marker andFINALIZEcommit or roll back together. Attaches carry the same identity (attach_partition(expected_oid=), re-checked afterATTACHin its transaction), andreconcile_default_rows(expected_target_oid=)verifies the fill target under the move's lock. Re-attaching removes the orphan marker in the attach's transaction, so a later detach starts a fresh grace period instead of inheriting the old instant. - Under
DropNevera matching detached table is never re-attached; a wanted window whose name it holds is reported (name_unusable) instead of recreated over it. - User-authored models —
TablePartitionConfig, schemes, boundaries, leaves, lifecycle rules — refuse unknown fields (extra="forbid"): a misspelledgarceorretentoinno longer validates into a destructive default.KeepFor/WindowAgeAboverefuse negative ages. CreatePartition.capabilitiesandAttachPartition.capabilitiesname theSHARE ROW EXCLUSIVElocks PostgreSQL takes on tables referencing the parent.- Sliding
LIST:ListPartitioning(key, sequence=IntegerSequence(start))is a progression level — one integer value per partition, the cursor read off the newest member (CursorSource.NEWEST_MEMBER), rotated withCreateNextIfor bounded withCreateUntil;ensure_partition(s)accept positions on the root's axis (a value, an instant). - Leaf backends:
config.leaves = LocalLeaves(tablespace, storage_parameters, inherit_privileges)(the default, plain) orForeignLeaves(server, options)— leaves as foreign tables with templated options; refused before any DDL on a parent with a unique index; foreign relations commented withCOMMENT ON FOREIGN TABLEand dropped withDROP FOREIGN TABLE; a foreign partition is managed only under aForeignLeavesconfig. - Batched data movement:
service.partition_data(config, batch_rows, max_batches)drains a DEFAULT partition into lifecycle partitions window by window (created detached, filled, attached);service.unpartition(config, into, drop_emptied)moves everything back into one table.MigrationResult;PlanExecutor.create_partition(fill=...). - Foreign-key aware retention:
Unreferenced()(the condition PostgreSQL enforces onDETACH,23503),FactKind.REFERENCES/PartitionFacts.referenced; a refused detach isPartitionReferencedError, recorded as an issue while the run goes on; measured lock levels on referencing tables onDetachPartition.capabilities. scripts/sync_mirror.pyregeneratespg_partsmith.syncfrompg_partsmith.aio;scripts/sync_tests_mirror.pyregenerates the sync integration suite.- Tests: the Redis lock managers against a real Redis; the integration suites run on
PostgreSQL 15, 16 and 17 in CI (
PG_PARTSMITH_TEST_PG_IMAGE). - Design docs: RFC 0001, OSS research with migration verdicts per project, PostgreSQL semantics verified on 15 and 17.
Changed¶
- Documentation rebuilt around tutorials, concepts, how-to guides and reference pages, with plans and findings captured from a real PostgreSQL 17.
- Existing partitions are matched to windows by their catalog bounds, never by name; names are parsed only to recognise a detached orphan.
- Topology conflicts discovered while applying (
PartitionTopologyError) are recorded as issues and never abort the run; other errors abort unlesscontinue_on_error. is_partition_closed, DEFAULT reconciliation, attach-last subtree creation, hash convergence rules, fail-closed pruning of unreadable bounds, timezone alignment and the orphan marker keep their 0.5 semantics.
0.5.0 (2026-08-28)¶
Features¶
- add boundary codecs for time-sortable partition keys (c10ffdb)
- add ensure_partitions for backfilling explicit periods (2ce6d10)
- add LIST subpartitioning alongside HASH (1267b5b)
- manage HASH and LIST roots that have no time dimension (59321d4)
- manage nested RANGE -> HASH partition trees (0d98360)
- partition trees, boundary codecs, static roots and composite keys (#23) (f150233)
- support composite partition keys (184c55c)
Bug Fixes¶
- answer "is it closed?" instead of raising, and refuse a key we cannot address (188285a)
- count buckets built while finishing a half-built branch (a9ed268)
- exclude identity columns when creating partitions (8f2b985)
- explain a partition that can never report as closed (1acf74a)
- keep doubled quotes from splitting a LIST bound (3f07273)
- keep partition_column in serialized configuration (39c7093)
- leave NULL-keyed rows where PostgreSQL puts them (55c9b75)
- never publish a branch that cannot route its whole keyspace (8b8b1b3)
- quoting hazards that only a hand-built statement can reach (f73dff8)
- read the whole key, every constraint, and the timezone that wrote the bound (5e4ea60)
- spell a partition key as one leading column plus a trailing tuple (a511b44)
- stop the planner abandoning subtrees and planning unusable names (b2bcb96)
- tell a lost race apart from a real conflict, and isolate each branch (cf77fb9)
- tell NULL from 'NULL', and a hidden child from a missing one (46af2e7)
Documentation¶
- add the new boundary and bounds names to the API reference (cfd92b1)
- correct three more claims, and test the one that was only written down (eb6388c)
- correct what the fact-checker falsified, and add what it found missing (5175daf)
- document subpartitioning, boundary codecs and the migration path (fdc4f28)
- name the difference between two fields called partition_type (5ed8863)
- say that issues now fills up on a successful run (d4f37f8)
- say what the database reported, not what was assumed (058801e)
0.4.0 (2026-08-27)¶
⚠ BREAKING CHANGES¶
PostgresPartitionRepository.partition_exists/.is_partition_attachedwere removed — use the identical methods onPostgresMetadataProvider(the repository protocol is write-only by design; the metadata provider is the read API). (#22)MaintenanceIssueStepnow contains only the members that are actually produced:CREATE,DETACH,DROP(theATTACHandHOOK_*members were never emitted).Period.to_date()no longer accepts adayargument.
Added¶
- End-to-end configurable timezone: every calculator accepts
tz(datetime.UTCdefault, or a keyedZoneInfo) — the current period, partition names, and naive boundary literals all follow it; pruning interprets naive catalog boundaries in the calculator's timezone;PartitionLifecycleServicerefuses a calculator/ddl_timezonemismatch so names and real bounds cannot silently drift apart;ddl_timezone=Nonewith a non-UTC calculator logs a warning.HourPeriodCalculatoris UTC-only (local hour names are ambiguous under DST). Defaults are bit-identical to the previous behavior. (#20) - Runtime-checkable
TimezoneAwareCalculator/DdlTimezoneAwareprotocols; new shared pure modulespg_partsmith.partition_bounds,pg_partsmith.pruning_rules,pg_partsmith.catalog_queries;PartitionType.from_partstrat; utils helperscoerce_str,elapsed_ms,describe_exception,is_default_partition_conflict,validate_timezone_alignment;get_period_calculatoris exported frompg_partsmith.strategies;PartitionTableSettings.get_period_calculator(tz=...)forwards the timezone. (#22)
Changed¶
- Library-wide quality pass (behavior-preserving beyond the breaking items above): the
aio/sync mirrors share the pure parsing/pruning/SQL logic instead of hand-maintaining
two copies; repository defaults and SQLSTATE sets live in
pg_partsmith.constants; timezone metadata is discovered via protocols instead ofgetattrsniffing; error/log wording no longer claims recovery where errors propagate;detach_single_partition/drop_single_partitionare documented extension points. (#22)
0.3.0 (2026-08-27)¶
Added¶
- Migration-ergonomics APIs (all mirrored in
aioandsync), extracted from a real migration of a hand-rolled partitioner: service.ensure_partition(config, period)— create and attach the partition for one specific period (idempotent, with DEFAULT reconciliation and attach-race handling); for writers that must guarantee a partition exists before an insert.repository.adopt_partition(table_name, partition_name)— stamp the orphan marker on a legacy detached table so safe-drop accepts it, instead of disabling the guard withdrop_allow_unmanaged.maintain_lifecycle(..., continue_on_error=True)(also on the maintainer andmaintain_partitions) — isolate create/detach/drop failures into the newMaintenanceResult.issues(MaintenanceIssue) instead of aborting the run.metadata.is_partition_closed(partition_name, *, settle_seconds=0)— server-side "the partition's upper bound has passed (+ settle buffer)" check for export pipelines.PartitionInfo.schema_name/PartitionInfo.relnameaccessors;qualify,split_qualified_name, andMaintenanceIssueare exported from the package root.- "Migrating an existing partitioner" documentation guide: retention count-vs-distance, adopting legacy partitions, schema-qualified names, lock ownership of granular calls, per-step error isolation, and export finalization.
0.2.0 (2026-08-26)¶
Added¶
pg_partsmith.sync— synchronous mirror ofpg_partsmith.aiowith the same class names and API, built on the sync SQLAlchemyEngine:PartitionLifecycleService,PartitionMaintainer,maintain_partitions,PostgresPartitionRepository,PostgresMetadataProvider,PostgresAdvisoryLockManager,RedisDistributedLockManager, syncPartitionLifecycleHooks/BasePartitionLifecycleHooks, and sync protocols. Differences from the async package:ddl_timeout_secondsis enforced server-side via PostgreSQLstatement_timeout(per statement), and the Redis lock renews its TTL from a background thread that logs (but cannot cancel maintenance) on renewal failure. (#13)- Hour and quarter partition granularities:
PartitionGranularity.HOUR/.QUARTER,HourPeriodCalculator(table__YYYY_MM_DD_HH, UTC boundaries with hour precision) andQuarterPeriodCalculator(table__YYYY_qN), plushour/quarterfields onPeriodwith validation, arithmetic, and ordering. (#15) Period.to_datetime()— period start as a timezone-aware UTC datetime preserving the hour component; the pruning fallback sort now uses it, so hourly partitions within one day order chronologically.
Changed¶
Periodinternals were restructured around a single per-granularity dispatch; behaviour is unchanged. Built-in calculators now derive names and boundaries fromPerioditself instead of duplicating the formatting and arithmetic. (#16)
Fixed¶
Hardening from a full-library audit (#16) and an external review (#17):
- Pruning fails closed:
infinityupper bounds are treated as unbounded (likeMAXVALUE), and an attached partition whose catalog boundary cannot be interpreted is skipped with a warning instead of being pruned by its name. list_partitionsalways returns schema-qualified partition names taken from the catalog — a partition living in a different schema than its parent can no longer be re-resolved viasearch_pathto an unrelated same-named table.drop_partitionrevalidates attachment and the orphan marker under anACCESS EXCLUSIVElock in the same transaction asDROP TABLE, closing the window where a concurrently reattached or replaced relation could be dropped.- Subpartitioned partitions (
relkind='p') are now recognised by existence checks and orphan discovery, so a detached partitioned child is dropped instead of being silently leaked. - Attach conflict SQLSTATEs (incl.
42809) are only treated as a lost race after verifying the partition is actually attached to the requested parent. - The compensating "return rows to DEFAULT" step now also runs when the attach is interrupted by cancellation (async, shielded) or KeyboardInterrupt (sync).
- A cancellation that lands while awaiting the Redis
SET NXresponse performs a token-checked release, so a server-side-applied SET no longer leaks the lock until TTL. - Detach: a partition left in
inhdetachpendingstate by a cancelledDETACH CONCURRENTLY(e.g. a DDL timeout) is now completed withDETACH PARTITION ... FINALIZEinstead of failing on every subsequent run. - Pruning: partitions with a
MAXVALUEupper bound are never pruned any more — previously the unparseable boundary fell back to name-based ageing, which could drop a catch-all partition holding current data. - DEFAULT reconciliation now runs under the same
SET LOCAL TIME ZONEasATTACH PARTITION, so a non-UTC server timezone no longer moves the wrong row range; if the attach still fails after rows were reconciled, they are moved back to the DEFAULT partition (best effort) instead of being stranded in a detached table. - Attach race handling: SQLSTATE
42809("already a partition", the code PostgreSQL actually raises when a concurrent worker wins the attach) is now tolerated, while55006(partition mid-detach) correctly propagates instead of being mislabelled as "already attached". - DDL statements no longer break when an identifier or literal contains
:(e.g. a pre-existing table comment) — colons are escaped before SQLAlchemytext()parses them as bind parameters. list_partitionsskips (with a warning) partitions whose schema or name contains a dot: such names cannot be addressed safely asschema.relnamestrings and previously produced DDL against the wrong relation.- Boundary parsing only applies to RANGE bound expressions; LIST/HASH bounds no longer yield fabricated from/to values.
- Config validation rejects quoted mixed-case partition columns up front instead of failing later inside reconciliation SQL.
- Lock managers: the per-table acquire rate limit no longer serializes unrelated tables (the delay is now slept outside the shared mutex) and no longer sleeps spuriously on the first acquire after host boot; the Redis lock is released even when the renewal watchdog fails to start; the async Redis lock no longer swallows an external task cancellation during watchdog teardown.
- Maintainer logging: operational
PartitionErrors (e.g. lock contention) are logged as warnings instead of "unexpected exception" errors with tracebacks.
0.1.0 - 2026-05-08¶
First public release of pg-partsmith.
- Time-based partition lifecycle management: create ahead, detach expired, drop orphans.
- Period calculators:
DayPeriodCalculator,WeekPeriodCalculator,MonthPeriodCalculator,YearPeriodCalculatorandBasePeriodCalculatorfor custom strategies. get_period_calculator()— factory function that returns the right calculator for a given granularity.PartitionLifecycleService— orchestrates the full create → detach → drop sequence.PartitionMaintainer— scheduler-friendly wrapper;run_maintenance_safe()never raises.maintain_partitions()— plain async function for APScheduler, Celery Beat, etc.- Lifecycle hooks:
before_create,after_create,before_detach,after_detach,before_drop,after_drop.before_*failures abort the operation;after_*failures are logged as warnings. PostgresPartitionRepository— PostgreSQL DDL implementation.PostgresMetadataProvider— PostgreSQL system catalog queries.PostgresAdvisoryLockManager— session-level advisory locks (no extra dependencies).RedisDistributedLockManager— Redis distributed locks (pg-partsmith[redis-locks]).PartitionTableSettings— pydantic-settings base class for env-driven configuration (pg-partsmith[pydantic-settings]).- Multi-schema support via
schemafield inTablePartitionConfig. - Orphan partition tracking via
COMMENTmarkers on detached tables. - DEFAULT partition reconciliation: moves conflicting rows and retries
ATTACH PARTITION. - TIMESTAMPTZ UTC boundary enforcement (
ddl_timezone="UTC"default). - Safe-drop protection:
UnmanagedPartitionDropErrorguards against dropping unmanaged tables. - All
Protocolclasses are@runtime_checkable— custom implementations can be validated viaisinstance(). - Python 3.11, 3.12, and 3.13 support.