API reference¶
The complete typed public surface. Everything below is re-exported from the top-level
betula_cluster package.
Convenience functions¶
betula_cluster.fit_predict
builtin
¶
fit_predict(
data: _FloatArray,
n_clusters: int = 8,
feature: _Feature = "diagonal",
method: _Method = "gmm",
threshold: float = 0.0,
branching: int = 32,
leaf_cap: int = 32,
max_leaves: int = 2000,
max_iter: int = 100,
n_init: int | None = None,
min_samples: int | Literal["auto"] | None = None,
min_cluster_size: int = Ellipsis,
seed: int = 0,
distance: _Distance = "euclidean",
absorb: _Absorb = "euclidean",
chi2_p: float = 0.95,
chi2_scale: float = 0.0,
n_shards: int | None = None,
normalize: bool = False,
resolution: float = Ellipsis,
covariance_weight: float = Ellipsis,
tangent_weight: float = Ellipsis,
tangent_rank: int = Ellipsis,
projection: _Projection = "none",
projection_dim: int = 64,
projection_max_iter: int = 100,
refine: int = 0,
rank: int = Ellipsis,
graph_degree: int = 0,
balance: float | Literal["auto"] | None = None,
auto_k_max: int = 0,
fuzzifier: float = Ellipsis,
leaf_refit: int = 0,
canonical_order: bool = False,
n_iterations: int = Ellipsis,
) -> NDArray[np.int64]
Cluster the rows of a 2-D float32 or float64 array; returns one int64 label per row (-1 =
noise, produced only by method="hdbscan"). float32 input is clustered in f32 (half the
memory, no upcast). With n_clusters=0 and method="gmm"/"gmm-full" the component count is
selected automatically by BIC.
absorb selects the CF-tree's absorption criterion — the full BIRCH grid plus this crate's own
gate: "euclidean" (D0, the default), "manhattan" (D1), "average" (D2, inter-cluster),
"diameter" (D3, intra-cluster of the merged cell), "ward" (D4, variance increase),
"radius" (R, the merged cell's mean squared radius), and "chi2", a mass-invariant
Mahalanobis-χ² gate at level chi2_p with within-cluster variance chi2_scale (required for
chi2). threshold is read in the chosen criterion's own units — L1 for "manhattan", squared
for the rest, a χ²_dim quantile for "chi2" — so it does not transfer between them.
refine runs BIRCH's Phase 4 — that many Lloyd sweeps over the raw rows, warm-started from the
Phase-3 centres — for the centroid heads ("kmeans", "spherical-kmeans"); other heads ignore
it, having no centre model to sweep. It trades a second pass over the data for a lower
within-cluster sum of squares, which is not the same thing as a better partition: on covtype
scikit-learn's k-means already reaches the lower objective and the worse ARI. Default 0 (off).
n_shards splits the insertion sequence for a shard-and-merge build. It is a model
parameter, not a worker count: each shard summarises its own rows, so two shard counts hold
different point sets and no merge order repairs the difference — measured over the 27 published
cells, labels at 8 shards agree with 1 shard at pairwise ARI 0.46 on average and 0.098 at worst.
Thread count is RAYON_NUM_THREADS and does not enter the answer. None (the default) builds
sequentially; under canonical_order the count is derived from n and passing one is an error
rather than a silent no-op.
betula_cluster.fit_predict_sparse ¶
fit_predict_sparse(
X: Any,
n_clusters: int = 8,
method: _Method = "kmeans",
threshold: float = 0.0,
max_leaves: int = 2048,
max_iter: int = 100,
n_init: int | None = None,
seed: int = 0,
projection: _Projection = "none",
projection_dim: int = 64,
projection_max_iter: int = 100,
auto_k_max: int = 0,
) -> NDArray[np.int64]
One-shot O(nnz) clustering of a scipy.sparse matrix.
Summarises rows into spherical micro-clusters touching only the non-zeros (a flat leader pass
bounded by max_leaves), clusters those with a parametric head (kmeans / gmm /
gmm-full / ward), and labels each row by the head's own point rule. For very
high-dimensional sparse data this avoids the O(d)-per-row cost of the dense path. It uses
the expanded squared-distance form for speed and so does not carry the dense path's
cancellation-free guarantee (accurate for sparse rows far from the dense centroid; see the
library docs). Returns one int64 label per row.
The centre-based heads (kmeans / xmeans / spherical-kmeans) label each row by its
nearest cluster centroid, the rule the dense path uses; kmedoids uses the same scan against
the micro-clusters it chose as medoids, which are not the cluster means. gmm and vmf
label by maximum
posterior: their kernels split over the support of a row, so the density costs O(nnz) rather
than the O(d) this path exists to avoid. The rest — the agglomerative heads (ward /
average / weighted / centroid / median) and gmm-full / gmm-toeplitz* /
mppca / mfa, whose densities read every coordinate — keep the label of the
micro-cluster the summarisation put the row in.
That last group is where the flat summary shows. Once the leader budget is spent the pass has no
proximity gate left, so on a 6 000 × 4 000 block-topic corpus at max_leaves=2048 it force-
assigns 3 952 of 6 000 rows into 544 leaders; a head that models each micro-cluster separately
inherits those mixed leaders, and ward reads ARI 0.068 against the dense tree's 0.987 at the
same compression. Raising threshold does not help (measured flat from 0.5 to 14, worse
above), because near-orthogonal sparse rows give the gate nothing to merge. Prefer a
centre-based head on this path, or reduce first with projection="svd".
projection="svd" turns this into the one-call reduce-then-cluster pipeline for text: the
leaf summary is reduced to projection_dim CF-weighted principal directions, the head
clusters the codes, and each row is labelled by its own code (encoded from its non-zeros).
Clustering the raw high-dimensional geometry directly is the thing to avoid here — see
docs/USAGE.md. auto_k_max bounds the n_clusters=0 search; 0 keeps the default.
betula_cluster.tune ¶
tune(
x: ndarray,
n_clusters: int,
*,
space: dict[str, tuple] | None = None,
y: ndarray | None = None,
objective: str = "calinski_harabasz",
n_trials: int = 30,
sampler: str = "random",
multi_objective: bool = False,
seed: int = 0,
**fixed: Any,
) -> TuneResult
Search betula's CF-representation knobs for the best clustering of x into n_clusters.
Parameters¶
x, n_clusters
Data and the number of clusters to fit each trial with.
space
Search space {param: spec} where each spec is ("int_log", lo, hi) or
("cat", [values]). Defaults to sweeping max_leaves, feature and normalize.
y
Ground-truth labels; required only for objective="ari".
objective
"calinski_harabasz" (default, higher better), "davies_bouldin" (lower better),
"dbcv" (density-based, higher better — use for the HDBSCAN-CF / DbStream density heads,
where the convex metrics mislead), or "ari" (needs y). "simplified_silhouette"
and "medoid_silhouette" (higher better) are scored on the fitted leaf summary
instead of the points, so they cost O(leaves · k · d) per trial and stay affordable
where the point-level metrics do not; read :meth:Betula.validity for what a summary
silhouette is and is not.
n_trials, seed
Search budget and RNG seed.
sampler
"random" (NumPy, no extra deps) or "optuna" (needs betula-cluster[tune]):
TPE for single-objective, NSGA-II for the multi_objective Pareto front.
multi_objective
If true, also return the (quality / memory=n_leaves / speed=fit-time) Pareto front.
**fixed
Extra Betula keyword arguments held constant across trials (e.g. method="gmm").
Returns¶
TuneResult
best_params / best_score / trials (and pareto when multi_objective).
Estimator¶
betula_cluster.Betula ¶
Streaming, scikit-learn-style BETULA estimator.
Parameters are validated lazily — when the engine is built at fit / partial_fit time —
following the scikit-learn convention that __init__ only records its arguments verbatim.
threshold accepts a non-negative float (the CF absorption radius, 0.0 grows it from
scratch) or "auto": a subsample pilot then estimates a warm-start threshold so the full fit
starts near-converged instead of thrashing rebuilds up from zero. "auto" is dense-only.
n_rebuilds_
property
¶
How many times the CF-tree rebuilt under the leaf bound; high ⇒ thrashing.
threshold_
property
¶
Current CF-tree absorption threshold (grows as it rebuilds).
effective_max_leaves_
property
¶
The max_leaves actually used: derived from memory_budget_mb if set, else
configured.
microcluster_centers_
property
¶
Leaf (microcluster) centroids — (n_microclusters, dim).
microcluster_weights_
property
¶
Leaf effective point mass — (n_microclusters,).
microcluster_radii_
property
¶
Leaf RMS radius — (n_microclusters,).
components_
property
¶
NMF parts H — (projection_dim, dim), rows unit-L2, ordered by descending energy.
Every leaf code is a nonnegative combination of these rows, so a row reads directly as a
"topic" over the input features. Requires a "weighted-nmf" / "weighted-nmf-kl"
projection.
reconstruction_err_
property
¶
Relative reconstruction error of the projection, ‖X̃ − W H‖_F / ‖X̃‖_F.
Measured over the leaf centroid matrix the factorizer actually fits — how much of the
compressed data projection_dim parts explain. Requires a projection.
cluster_centers_
property
¶
Macro-cluster centroids — (n_clusters, dim); requires a finalized clustering.
cluster_radii_
property
¶
Macro-cluster RMS radius — (n_clusters,); requires a finalized clustering.
cluster_sizes_
property
¶
Macro-cluster total point mass — (n_clusters,); requires a finalized clustering.
microcluster_proba_
property
¶
Per-microcluster GMM soft responsibilities (n_microclusters, k). GMM heads only.
fit ¶
Fit the CF-tree on X and cluster it; returns self (scikit-learn style).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
X
|
_FloatArray
|
dense float32/float64 array or a |
required |
y
|
Any
|
ignored; present for scikit-learn API compatibility. |
None
|
must_link
|
Any
|
optional |
None
|
cannot_link
|
Any
|
optional |
None
|
fit_predict ¶
fit_predict(
X: _FloatArray,
y: Any = None,
must_link: Any = None,
cannot_link: Any = None,
) -> NDArray[np.int64]
Fit on X; return one int64 label per row (-1 = noise, HDBSCAN head).
Takes the same X / must_link / cannot_link as :meth:fit.
partial_fit ¶
Absorb a chunk; call with no argument to finalize the global clustering.
save ¶
Write the fitted estimator to path as a gzip-framed, version-tagged CBOR file.
The file is a plain gzip member, so gzip -dc model.betula yields the CBOR and the model
stays inspectable without this library. Compression is measured, not assumed: over
digits/covtype/mnist models it takes 2.0–4.8× off the file, and a 14.35 MB mnist model
writes as 5.29 MB in 0.6 s. Codecs that beat gzip beat it by 12–22 % and cost 4–13 s on the
same model, which is the wrong trade for an interactive call.
Compatibility. A saved model is not portable across major versions: load accepts one
schema version and refuses every other, naming the version it found. Re-save a model with
the version that wrote it before upgrading, or keep the training code and re-fit. The
alternative -- a migration path for every past layout -- would pin the internals of the tree
for the whole 1.x line, and the tree is where the research happens.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str
|
destination file. |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
the estimator is not fitted, or the file cannot be written. |
load
classmethod
¶
Read back an estimator written by :meth:save, fitted and ready to predict.
Both framings are accepted: gzip-framed CBOR, and the bare CBOR that 0.8.0 and earlier
wrote. What is not accepted is a different schema version -- see :meth:save for why
there is no migration path.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str
|
a file written by :meth: |
required |
Returns:
| Name | Type | Description |
|---|---|---|
Betula |
Betula
|
the loaded estimator. |
Raises:
| Type | Description |
|---|---|
ValueError
|
the file cannot be read, is not a betula model, or carries a schema version this build does not implement. |
assign_microclusters ¶
Nearest leaf index per row (matches microcluster_centers_ order).
outlier_scores ¶
Per-row deviation from the assigned cluster centroid, over that cluster's own spread.
metric="radius" (the default) divides by the cluster's scalar RMS radius. That radius is
the trace of the pooled covariance, so an elongated cluster's short axis is judged by the
length of its long one — measured on sheared 6-D clusters, ROC-AUC 0.596, and identical at
feature="diagonal" and feature="full" because the per-dimension scatter is never
consulted.
metric="mahalanobis" whitens the deviation by the cluster's full pooled covariance
instead — the parallel-axis pooling of the leaves' own scatter and their spread about the
centroid, whose trace is that RMS radius. The two are therefore calibrated: where the
pooled covariance is isotropic they return the same number, so the refinement moves a score
only where the cluster has a shape. Off-diagonal terms are the point — a cluster can be
elongated along a direction that is not a coordinate axis, and a per-dimension variance
cannot see that — so it costs O(k·d³) once plus O(d²) per row against the scalar
path's O(d), which is worth watching in high dimension.
tree_report ¶
Why the CF-tree looks the way it does — leaf budget, mass concentration, threshold.
Answers "why is my tree collapsing?" with the numbers that decide it: how much of the leaf budget was spent, how much of the mass one leaf ended up holding, and how wide that leaf is against a typical one. A near-full budget with a single leaf carrying most of the points is the size-imbalance pathology of scikit-learn's Birch issue #22854 — the tree spent its leaves resolving the sparse minority and merged the dense majority into one summary.
Whether that costs anything is a separate question, and the heavy leaf's radius answers it: a dense blob that really is point-like is summarized faithfully by one tight leaf, while a heavy leaf as wide as a typical one is a merged region whose internal structure is gone.
Pass X to add an A-BIRCH threshold estimate (:func:~betula_cluster.tuning
.estimate_threshold, Lorbeer et al. 2018) alongside the threshold actually in use. That
estimate is advisory: it assumes well-separated, near-spherical clusters of comparable
size, and diagnosis names every one of those assumptions the data breaks. max_leaves
remains the knob that binds — the threshold is what the rebuild derives from it.
summary ¶
A compact dict describing the dataset's structure (microclusters + macro clusters).
validity ¶
Internal validity indices of the fitted partition, scored on the leaf summary.
Returns calinski_harabasz (higher is better), davies_bouldin (lower is better),
medoid_silhouette and simplified_silhouette (higher is better, capped at 1). All
four cost O(n_leaves · k · d) rather than the O(N²) an exact silhouette over the
points would — the sum of squared distances inside a leaf is S_i + n_i‖μ_i − c‖²
exactly, so no point ever has to be revisited.
Caveats worth reading before using any of them to choose k: Calinski–Harabasz is exact
but undefined at k = 1; Davies–Bouldin is the RMS-dispersion variant, not the classical
mean-distance one; both silhouettes are indices of the summary, not of the points, and no
richer leaf model would change that — two point sets with identical cluster features can
have different pairwise distances, so the exact silhouette is not recoverable from any
summary in this family (research/RESULTS-cf-boundary.md). The two differ in the
representative only: simplified_silhouette measures to the cluster centroid
(Hruschka's simplified silhouette), medoid_silhouette to the leaf nearest it. None of
the four can report "there is no structure here" — for that, fit with n_clusters=0 on a
mixture head and let BIC answer.
summary_mmd ¶
Kernel distance between the leaf summary and the raw sample X. Lower is better.
A label-free, head-independent fidelity number: the leaves are read as the Gaussian mixture
Σ (n_i/N)·N(μ_i, s_i I) with s_i = S_i/(n_i·d) and compared to X by maximum mean
discrepancy under a Gaussian kernel, in closed form — no sampling from the surrogate. Unlike
:meth:validity it needs no labels, no k and no head, so a tree built with
partial_fit alone can be scored.
bandwidth is the kernel's h; None takes the median heuristic on X, which
makes values comparable across leaf budgets of the same data and meaningless across
different data.
Use it to find the leaf budget past which more leaves buy nothing: it flattens where the
clustering stops changing, which mean_sq_radius does not. It is not monotone in the
budget — very coarse leaves can model a smooth blob better than two-point ones — and it
costs O(n_leaves² + n_leaves·N + N²) kernel evaluations, so it is a diagnostic to run at
a few budgets rather than something to put in a loop.
summary_w2 ¶
Mixture-Wasserstein distance between this model's leaf summary and other's.
The drift metric: fit at t1, fit at t2, and read how far the summary moved. It needs
no labels, no shared points and not even the same number of leaves, which is what an
ARI-over-time cannot do. Both leaf summaries are read through the same isotropic surrogate
:meth:summary_mmd uses — leaf i is N(μ_i, (R_i²/d)·I) with mass n_i — so the
two diagnostics agree on what a leaf is.
Zero means the two summaries describe the same density; the value is in the units of the data, so it is comparable across runs on the same feature scale and meaningless across different ones.
find_outliers ¶
find_outliers(
X: _FloatArray,
top_k: int = 100,
metric: _OutlierMetric = "radius",
) -> NDArray[np.int64]
Row indices of the top_k most outlying points (highest score first).
metric is passed through to :meth:outlier_scores.
sample_representatives ¶
For each cluster, the row indices of the k points nearest its centroid.
find_near_duplicates ¶
Groups (row-index arrays) of points sharing a microcluster tighter than radius.
near_duplicate_pairs ¶
near_duplicate_pairs(
X: _FloatArray,
threshold: float = 0.9,
*,
neighbors: int = 0,
) -> NDArray[np.float64]
Scored near-duplicate row pairs by exact cosine similarity, blocked by microcluster.
The CF-tree blocks rows into leaves in O(N); within each (small) leaf, exact pairwise
cosine is computed and pairs scoring >= threshold are kept. The cost is
~O(N * leaf_size) -- the scalable counterpart to an O(N^2) all-pairs scan, and the
scored complement to :meth:find_near_duplicates (which returns unscored groups). Returns
an (m, 3) float64 array of [cos_sim, i, j] with i < j, sorted by similarity
descending.
Recall is bounded by the blocking: a duplicate pair whose two rows were absorbed into
different leaves is invisible to a within-leaf scan. neighbors buys that recall back
without giving up the blocking — each leaf is also scored against its neighbors nearest
leaves by centroid distance, which is the geometry that split the pair in the first place.
0 (the default) is the within-leaf scan unchanged. The added cost is
O(M^2 d) for the leaf-neighbour search plus ~neighbors x the scoring, both still
sub-quadratic in N; a coarser tree (smaller max_leaves) is the other lever, trading
speed for recall by widening the blocks themselves.
mapper ¶
mapper(
lens: _Lens = "density",
resolution: int = 10,
gain: float = 0.3,
link_scale: float = 1.0,
min_node_mass: float = 0.0,
density_k: int = 5,
coordinate: int = 0,
link: _Link = "centroid",
) -> MapperGraph
Build a Mapper topological-skeleton :class:MapperGraph over the fitted microclusters.
TDA Mapper specialised to BETULA: a lens filter ("density" | "radius" |
"l2norm" | "coordinate" | "eccentricity") is covered by resolution bins
overlapping by gain; microclusters in a bin are single-linked at link_scale × the
bin's median nearest-neighbour gap; one node per (bin, component).
It surfaces non-convex structure, branch points and bridges (topic leakage) over the
M << N microclusters — an exploration tool, not a partition. Build the model first.
link chooses what "close" means inside a bin. "centroid" (the default) is the
Euclidean distance between leaf centroids and sees only the gap. "bhattacharyya" divides
that gap by the pair's own spread, using the second moments the cluster features already
carry, which is what stops a thin bridge of sparse microclusters from chaining two dense
regions together: on a dumbbell fixture the centroid rule merges the two lobes at
link_scale=3 on all three seeds and the Bhattacharyya rule merges neither, at any
link_scale measured. It is not free — a leaf that holds one point has no spread, so
every bridge of singletons stays fragmented whether or not it was real structure.
reachability ¶
Build the OPTICS :class:ReachabilityPlot over the fitted microclusters.
A density diagnostic, not a partition: it answers "what does the density structure look
like" rather than "which cluster is this in". min_samples and graph_degree mean what
they do for method="hdbscan" — pass the values that fit used, or the plot describes a
different neighbourhood than the head did. Build the model first.
Reads the same mutual-reachability spanning tree the density head does, so
:meth:ReachabilityPlot.labels_at is that head's hierarchy cut at a height, exactly.
mapper_stability ¶
mapper_stability(
resolutions: Sequence[int] | None = None,
**mapper_kwargs: Any,
) -> list[dict[str, int]]
Sweep Mapper resolution and report how the topology persists across scale.
Returns a list of dicts (one per resolution) with resolution, n_nodes, n_edges,
n_branch_points, n_bridges, n_components (β₀, connected components) and
n_loops (β₁ = edges − nodes + components, the number of independent cycles). Features
constant across many resolutions are real structure; ones that flicker are binning
artefacts — the Mapper analogue of a persistence diagram, without cross-scale node matching.
resolutions defaults to range(4, 30, 2); mapper_kwargs (lens, gain,
link_scale …) pass straight through to :meth:mapper. Build the model first.
export_coreset ¶
export_coreset(
size: int | None = None,
k: int | None = None,
seed: int | None = None,
) -> Coreset
The leaf summary as a weighted-point :class:Coreset, optionally sampled down to
size points with a provable (k, ε) guarantee. Requires a built tree only —
partial_fit is enough, since which head this estimator fitted does not enter it.
With size=None this returns every leaf at its own mass: the streaming summary, exactly
as before, in one O(n_leaves) pass. Pass a size and the leaves are subsampled by
sensitivity sampling (Feldman & Langberg 2011), which costs one weighted k-means over
the leaves and fills in reference_cost / total_sensitivity.
The error has two independent halves, and neither is folded into the other.
Summarization, present in both modes. With Δ = offset = Σᵢ Sᵢ, the summary's cost
ĉost(C) = Σᵢ (Sᵢ + nᵢ‖μᵢ − C‖²) satisfies, for every candidate C and every k::
0 ≤ ĉost(C) − cost(C) ≤ 4·√(Δ · cost(C)) + 4·Δ
— a relative error of 4√ρ + 4ρ at ρ = Δ/cost(C), and cost(C) ≥ OPT_k bounds it
uniformly. :meth:Coreset.summary_epsilon evaluates it, and makes you name the α it
assumes. This is what makes the word "coreset" here a claim rather than a label.
Sampling, present only when size is given. ĉost(C) = Δ + Σᵢ nᵢ‖μᵢ − C‖² and
Δ does not depend on C, so the sample only has to be a coreset of the weighted set
{(μᵢ, nᵢ)} — offset carries the constant instead of losing it. Sensitivity sampling
attains the optimal worst-case size Õ(k·ε⁻²·min(√k, ε⁻²)), matching the STOC 2022 lower
bound, and Õ(k/ε²) on stable instances (arXiv 2405.01339).
size at or above the leaf count returns every leaf exactly, with no sampling error,
rather than a noisy redraw of something already held exactly. k defaults to
n_clusters and seed to this estimator's.
predict_proba ¶
Per-point soft assignment, shape (n, n_components).
The GMM, subspace (mppca / mfa), vMF, Watson and Toeplitz
(gmm-toeplitz / -full / -gs) heads score the point under the fitted mixture,
so predict_proba(X).argmax(1) is exactly :meth:predict. fuzzy-cmeans returns
its own memberships u_j ∝ d_j^{−1/(m−1)}, which also argmax to :meth:predict but
are a partition of unity over the centres, not a posterior — no density is fitted.
k-means / x-means / Ward / HDBSCAN return a heuristic
softmax(−d²/2τ²) over the cluster centroids (τ = mean cluster radius) — a confidence
proxy, not a calibrated posterior. Columns are component indices aligned with
:meth:predict.
hyperbolic raises: that proxy is a Euclidean distance in the ambient coordinates, and
its argmax is not the head's own argmax_c ⟨x, c⟩_L — a confidence that disagrees with
the label it is reported for is worse than none.
assignment_confidence ¶
Per-point confidence in [0, 1] = the max soft-assignment probability (see
:meth:predict_proba); low values flag boundary / ambiguous points.
diagnostics ¶
A richer structural report than :meth:summary — compression, microcluster-radius
percentiles, rebuild count, and (once finalized) cluster mass spread.
representatives ¶
representatives(
X: _FloatArray,
cluster_id: int,
method: _Repr = "medoid",
k: int = 5,
) -> NDArray[np.int64]
Row indices of k representatives of cluster_id. method: medoid (nearest
centroid), boundary (farthest in-cluster), outlier (highest outlier score),
diverse (farthest-point sampling). Empty if the cluster has no predicted members.
cluster_profile ¶
A JSON-able profile of a macro-cluster (size, radius, center, nearest clusters) — feed to an LLM to name it. Geometry only; no data pass needed.
snapshot ¶
A JSON-able snapshot of the current cluster geometry (centers / sizes / radii) for drift
monitoring across time. Requires a finalized clustering; compare two with
:meth:compare_snapshots.
compare_snapshots
staticmethod
¶
Drift report between two :meth:snapshot dicts. Each after cluster is matched to its
nearest before centroid; reports the centroid shift (absolute and in after-radius
units) and the mass ratio per match, plus the cluster counts and the worst shift. Both
snapshots must come from finalized models with ≥ 1 cluster.
active_learning_batch ¶
active_learning_batch(
X: _FloatArray,
n: int = 100,
strategy: _Strategy = "uncertain",
) -> NDArray[np.int64]
Row indices of the n most informative points to review/label. strategy:
uncertain (lowest :meth:assignment_confidence) or outlier (highest
:meth:outlier_scores) — for human-in-the-loop curation / labeling.
Streaming¶
betula_cluster.DenStream ¶
Streaming DenStream density clusterer (Cao et al., SDM 2006) over fading micro-clusters.
For evolving streams where old data should fade: feed chunks with :meth:partial_fit, then
:meth:predict (which finalizes the offline clustering on first call) — or both at once with
:meth:fit / :meth:fit_predict. eps is the micro-cluster radius (tune to the data scale),
decay the fading rate λ, beta the outlier fraction (in (0, 1]) and mu the
macro-cluster weight floor, with beta × mu the promotion/pruning weight (must exceed 1).
All four must be finite and positive; they are checked on the first :meth:partial_fit, where
the model is built. Spherical micro-clusters, float64; -1 labels are noise.
n_microclusters_
property
¶
Number of potential (cluster-eligible) micro-clusters.
drift_
property
¶
Drift diagnostic over the routing distance — {alarms, last_alarm, distance, window}.
The detector (ADWIN; Bifet & Gavaldà, SDM 2007) watches one number per streamed point: how
far the point landed from the nearest micro-cluster, in units of eps. Stationary
data sits near 1 by construction; a distribution moving into space the model does not cover
sends it far higher, and an alarm is raised when that shift is larger than chance explains
at δ = 0.002. distance is the mean over the adaptive window, window its size (it
collapses on a change and regrows while the stream is stationary), last_alarm the stream
time of the most recent report.
It reports; it does not act. An alarm prunes nothing, promotes nothing and relabels
nothing — what to do about a change is the caller's policy. decay and this answer
different questions: decay sets how fast the model forgets, on a fixed schedule, whether
or not anything changed; the detector says whether anything did. An early alarm is the model
warming up, not drift.
microcluster_weights_
property
¶
Potential micro-cluster weights, faded to the current stream time.
partial_fit ¶
Stream a chunk of points into the fading micro-clusters.
cluster ¶
Run the offline step (label the potential micro-clusters) over what has streamed.
predict ¶
Label rows by their nearest potential micro-cluster (-1 = noise); finalizes the
offline clustering first if points have streamed since the last :meth:cluster.
betula_cluster.DbStream ¶
Streaming DBSTREAM density clusterer (Hahsler & Bolaños, 2016) over fading micros.
Like :class:DenStream it fades old data and marks -1 as noise, but it connects
micro-clusters by shared density — the mass of points within radius r of both — rather
than by mere proximity. This recovers arbitrarily-shaped clusters (chained overlapping
micro-clusters) and, unlike a distance rule, keeps two close-but-disconnected dense regions
apart (an empty gap means zero shared density). r is the radius, decay the fading rate,
alpha the shared-density bridge threshold (a pair links when their overlap mass exceeds
alpha * min_weight), and min_weight the weight a micro-cluster needs to form a cluster.
drift_
property
¶
Drift diagnostic over the routing distance — {alarms, last_alarm, distance, window}.
The detector (ADWIN; Bifet & Gavaldà, SDM 2007) watches one number per streamed point: how
far the point landed from the nearest micro-cluster, in units of r. Stationary
data sits near 1 by construction; a distribution moving into space the model does not cover
sends it far higher, and an alarm is raised when that shift is larger than chance explains
at δ = 0.002. distance is the mean over the adaptive window, window its size (it
collapses on a change and regrows while the stream is stationary), last_alarm the stream
time of the most recent report.
It reports; it does not act. An alarm prunes nothing, promotes nothing and relabels
nothing — what to do about a change is the caller's policy. decay and this answer
different questions: decay sets how fast the model forgets, on a fixed schedule, whether
or not anything changed; the detector says whether anything did. An early alarm is the model
warming up, not drift.
microcluster_weights_
property
¶
Micro-cluster weights, faded to the current stream time.
partial_fit ¶
Stream a chunk of points into the fading micro-clusters.
cluster ¶
Run the offline step (label micro-clusters via the shared-density graph).
predict ¶
Label rows by their nearest micro-cluster within r (-1 = noise); finalizes the
offline clustering first if points have streamed since the last :meth:cluster.
Mixed-type data¶
betula_cluster.KPrototypes ¶
k-prototypes clustering of mixed numeric + categorical + directional data.
Huang's k-prototypes (1997) with a third block. categorical lists the integer-coded
categorical column indices of X and directional the columns that together form one
direction; whatever is left is numeric. Distance is::
||Δnum||² + gamma · (categorical mismatch) + gamma_dir · (2 − 2 uᵀc)
The directional columns are L2-normalised per row, so the block reads angle only; a row
whose directional part is all zeros has no direction and its term is the same for every
prototype, which stops it voting rather than making it vote arbitrarily. A cluster's directional
prototype is the normalised resultant R/||R||, not a mean — averaging unit vectors leaves
the sphere.
gamma defaults to half the mean numeric standard deviation (Huang's heuristic).
gamma_dir has no published heuristic at all; it defaults to the mean numeric variance,
so that one unit of ||u − c||² costs one numeric variance. That is a scale-matching
convention, not a result — treat both as parameters to sweep.
Rows are summarised into bounded mixed micro-clusters (a flat leader pass capped at
max_leaves) before clustering, so memory stays bounded. A numeric column is always required,
as is at least one categorical or directional column; a one-column directional block is
rejected, because a direction in one dimension is a sign. float64.
cluster_centroids_
property
¶
Numeric cluster centroids — (n_clusters, n_numeric).
cluster_modes_
property
¶
Categorical cluster modes — (n_clusters, n_categorical) integer codes.
cluster_directions_
property
¶
Directional prototypes — (n_clusters, n_directional) unit vectors.
Each row is its cluster's normalised resultant R/||R||. A cluster whose member
directions cancelled to (near) zero has no direction, and its row is all zeros.
predict ¶
Label rows by their nearest mixed micro-cluster.
Coresets¶
betula_cluster.Coreset
dataclass
¶
A weighted-point coreset: the CF-tree leaf microclusters as (centers, weights, radii).
Each row is one microcluster — a numerically stable summary of the points absorbed into it. The
set is bounded by max_leaves and built in a single streaming pass, so fitting a weighted
clustering / classifier on it is competitive with fitting on the full data at a fraction of the
cost.
cost ¶
Σⱼ wⱼ·d²(xⱼ, C) + offset — this coreset's estimate of the summary's k-means cost.
A method rather than a formula in the docs because offset is easy to forget and
forgetting it understates every cost by the same constant, which looks like nothing until
two costs from different coresets are compared.
summary_epsilon ¶
Relative summarization error 4√ρ + 4ρ at ρ = alpha · offset / reference_cost.
alpha is required, not defaulted. reference_cost is the cost of an α-approximate
solution and therefore upper-bounds OPT_k, so offset / reference_cost
under-states the true ρ = Δ/OPT_k and summary_epsilon(1.0) is an optimistic
reading rather than a certificate. The shipped seeding is k-means++ with greedy trials,
whose O(log k) guarantee holds in expectation, not for the run in hand — so pass the
factor you can defend.
Covers the summary only; sampling error sits on top of it and is what size buys down.
Streaming sketches¶
betula_cluster.KllSketch ¶
Streaming KLL quantile sketch (rank-error). Standalone betula-sketch primitive.
__doc__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__new__
builtin
¶
Create and return a new object. See help(type) for accurate signature.
quantiles
method descriptor
¶
Estimated quantiles for an array of q values.
update_many
method descriptor
¶
Add every value of a 1-D array.
betula_cluster.DdSketch ¶
Streaming DDSketch quantile sketch (relative-error). Standalone betula-sketch primitive.
__doc__
class-attribute
¶
__doc__ = "Streaming **DDSketch** quantile sketch (relative-error). Standalone `betula-sketch` primitive."
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__new__
builtin
¶
Create and return a new object. See help(type) for accurate signature.
quantiles
method descriptor
¶
Estimated quantiles for an array of q values.
update_many
method descriptor
¶
Add every value of a 1-D array.
Robustness¶
betula_cluster.consensus ¶
consensus(
X: _FloatArray,
n_clusters: int,
*,
n_runs: int = 5,
seed: int = 0,
n_jobs: int = 1,
vary: Literal["order", "seed", "both"] = "order",
**fit_kwargs: object,
) -> ConsensusResult
Cluster X n_runs times under a varying nuisance, and return the consensus labelling
with a per-point stability score (see :class:ConsensusResult).
vary names what is allowed to differ between runs, and therefore what the stability score is
a score of:
"order" (default)
the insertion order, with the head's seed held fixed — the CF-tree's own sensitivity, which
is what this function is documented to measure.
"seed"
the head's seed, over one fixed insertion order — the head's initialisation sensitivity
(k-means++ draws, EM restarts); a deterministic head such as ward returns confidence 1.
"both"
both at once. The behaviour before 1.0.0, and the one that cannot say which of the two a
low score came from.
Extra keyword arguments are forwarded to :func:fit_predict (feature / method /
threshold / …). Intended for the partitional heads (kmeans / gmm / ward /
spectral) at a fixed n_clusters; density heads (hdbscan) emit noise / variable
counts the vote cannot align, and are rejected. n_jobs runs the (independent) runs
in parallel threads — the Rust core releases the GIL, so this scales — with <0 meaning all
cores; each run is seeded independently, so the result is identical regardless of n_jobs.
Result types¶
betula_cluster.MapperGraph
dataclass
¶
A Mapper topological-skeleton graph over a fitted model's leaf microclusters.
Each node is a connected group of microclusters inside one cover bin; edges link nodes that
share microclusters (from the cover overlap). branch_points are nodes where the shape splits
(degree ≥ 3); bridges index the edges whose removal would disconnect the graph — thin
links between otherwise separate regions (e.g. leakage between topics in an embedding).
edge_overlap is a per-edge Bhattacharyya coefficient in (0, 1] from the two nodes'
pooled diagonal-Gaussian summaries: a bridge across a sparse neck scores lower than an edge
inside one dense blob, so links are weighted by distributional overlap, not a bare shared count.
persistence ¶
The nerve's 0-D persistence diagram as (k, 2) births/deaths, sorted by persistence.
filtration="overlap" (default) filters by the 1 − edge_overlap gap — a finite bar's
death is the Bhattacharyya depth of a bottleneck, a ranked upgrade of the boolean bridges;
"lens" is the lens sublevel diagram (flares of the shape). Essential (component) classes
carry np.inf in the death column; finite_only=True drops them.
to_networkx ¶
Build a networkx.Graph (requires networkx); nodes carry mass/bin/lens/centroid,
edges carry weight and a boolean bridge flag.
betula_cluster.ReachabilityPlot
dataclass
¶
An OPTICS reachability plot over a fitted model's leaf microclusters.
reachability[i] belongs to order[i], and order[0]'s entry is inf — nothing
reached the first leaf. core_distances and weights are in the leaves' own indexing, not
the sweep's. Plot reachability against its position and clusters appear as valleys, with a
peak at the distance you would have to walk to leave one and enter the next.
It is not an approximation of what method="hdbscan" does: OPTICS with no cutoff is Prim's
algorithm on the mutual-reachability graph, so this is the same spanning tree that head takes
its hierarchy from, written in a different order. Every peak is a merge height in that
hierarchy, and :meth:labels_at reproduces its cut exactly.
One position per leaf, so a valley's width is a leaf count, not a point count. Read
weights before reading the width: three leaves can hold a hundred thousand points.
labels_at ¶
DBSCAN* at eps, per leaf, read off the plot; -1 is noise.
Ankerst et al.'s ExtractDBSCAN-Clustering: walk the order and start a segment wherever the
reachability rises above eps, then drop every leaf whose core distance exceeds it. The
segments are the connected components of the mutual-reachability tree under eps, so this
is DBSCAN* on the summary and not a lookalike — there are no border points, which is the
* in the name.
betula_cluster.TuneResult
dataclass
¶
The outcome of :func:tune: the best single configuration, every trial, and — for
multi_objective=True — the non-dominated (quality / memory / speed) Pareto front.
betula_cluster.ConsensusResult
dataclass
¶
Consensus of several insertion-order-permuted clusterings + a per-point stability score.
The CF-tree is sensitive to insertion order; clustering n_runs random permutations and
voting turns that into a measurable quantity. labels is the majority label per point (input
order); confidence is the fraction of runs agreeing with it — low on an unstable boundary,
high where every insertion order groups the point the same way.
mean_confidence
property
¶
Mean per-point stability across the input — a scalar robustness summary in [0, 1].