API reference¶
The complete typed public surface. Everything below is re-exported from the top-level
betula_cluster package.
Convenience functions¶
betula_cluster.fit_predict
builtin
¶
fit_predict(
data: _FloatArray,
n_clusters: int = 8,
feature: _Feature = "diagonal",
method: _Method = "gmm",
threshold: float = 0.0,
branching: int = 32,
leaf_cap: int = 32,
max_leaves: int = 2000,
max_iter: int = 100,
min_samples: int = 5,
min_cluster_size: int = 5,
seed: int = 0,
distance: _Distance = "euclidean",
absorb: _Absorb = "euclidean",
chi2_p: float = 0.95,
chi2_scale: float = 0.0,
n_jobs: int = 1,
normalize: bool = False,
resolution: float = 1.0,
covariance_weight: float = 0.0,
tangent_weight: float = 0.0,
tangent_rank: int = 2,
projection: _Projection = "none",
projection_dim: int = 64,
projection_max_iter: int = 100,
refine: int = 0,
rank: int = 2,
graph_degree: int = 0,
balance: float | None = None,
) -> NDArray[np.int64]
Cluster the rows of a 2-D float32 or float64 array; returns one int64 label per row (-1 =
noise, produced only by method="hdbscan"). float32 input is clustered in f32 (half the
memory, no upcast). With n_clusters=0 and method="gmm"/"gmm-full" the component count is
selected automatically by BIC.
absorb selects the CF-tree's absorption criterion — the full BIRCH grid plus this crate's own
gate: "euclidean" (D0, the default), "manhattan" (D1), "average" (D2, inter-cluster),
"diameter" (D3, intra-cluster of the merged cell), "ward" (D4, variance increase),
"radius" (R, the merged cell's mean squared radius), and "chi2", a mass-invariant
Mahalanobis-χ² gate at level chi2_p with within-cluster variance chi2_scale (required for
chi2). threshold is read in the chosen criterion's own units — L1 for "manhattan", squared
for the rest, a χ²_dim quantile for "chi2" — so it does not transfer between them.
refine runs BIRCH's Phase 4 — that many Lloyd sweeps over the raw rows, warm-started from the
Phase-3 centres — for the centroid heads ("kmeans", "spherical-kmeans"); other heads ignore
it, having no centre model to sweep. It trades a second pass over the data for a lower
within-cluster sum of squares, which is not the same thing as a better partition: on covtype
scikit-learn's k-means already reaches the lower objective and the worse ARI. Default 0 (off).
betula_cluster.fit_predict_sparse ¶
fit_predict_sparse(
X: Any,
n_clusters: int = 8,
method: _Method = "kmeans",
threshold: float = 0.0,
max_leaves: int = 2048,
max_iter: int = 100,
seed: int = 0,
projection: _Projection = "none",
projection_dim: int = 64,
projection_max_iter: int = 100,
) -> NDArray[np.int64]
One-shot O(nnz) clustering of a scipy.sparse matrix.
Summarises rows into spherical micro-clusters touching only the non-zeros (a flat leader pass
bounded by max_leaves), clusters those with a parametric head (kmeans / gmm /
gmm-full / ward), and labels each row by its nearest micro-cluster. For very
high-dimensional sparse data this avoids the O(d)-per-row cost of the dense path. It uses
the expanded squared-distance form for speed and so does not carry the dense path's
cancellation-free guarantee (accurate for sparse rows far from the dense centroid; see the
library docs). Returns one int64 label per row.
projection="svd" turns this into the one-call reduce-then-cluster pipeline for text: the
leaf summary is reduced to projection_dim CF-weighted principal directions, the head
clusters the codes, and each row is labelled by its own code (encoded from its non-zeros).
Clustering the raw high-dimensional geometry directly is the thing to avoid here — see
docs/USAGE.md.
betula_cluster.tune ¶
tune(
x: ndarray,
n_clusters: int,
*,
space: dict[str, tuple] | None = None,
y: ndarray | None = None,
objective: str = "calinski_harabasz",
n_trials: int = 30,
sampler: str = "random",
multi_objective: bool = False,
seed: int = 0,
**fixed: Any,
) -> TuneResult
Search betula's CF-representation knobs for the best clustering of x into n_clusters.
Parameters¶
x, n_clusters
Data and the number of clusters to fit each trial with.
space
Search space {param: spec} where each spec is ("int_log", lo, hi) or
("cat", [values]). Defaults to sweeping max_leaves, feature and normalize.
y
Ground-truth labels; required only for objective="ari".
objective
"calinski_harabasz" (default, higher better), "davies_bouldin" (lower better),
"dbcv" (density-based, higher better — use for the HDBSCAN-CF / DbStream density heads,
where the convex metrics mislead), or "ari" (needs y).
n_trials, seed
Search budget and RNG seed.
sampler
"random" (NumPy, no extra deps) or "optuna" (needs betula-cluster[tune]):
TPE for single-objective, NSGA-II for the multi_objective Pareto front.
multi_objective
If true, also return the (quality / memory=n_leaves / speed=fit-time) Pareto front.
**fixed
Extra Betula keyword arguments held constant across trials (e.g. method="gmm").
Returns¶
TuneResult
best_params / best_score / trials (and pareto when multi_objective).
Estimator¶
betula_cluster.Betula ¶
Streaming, scikit-learn-style BETULA estimator.
Parameters are validated lazily — when the engine is built at fit / partial_fit time —
following the scikit-learn convention that __init__ only records its arguments verbatim.
threshold accepts a non-negative float (the CF absorption radius, 0.0 grows it from
scratch) or "auto": a subsample pilot then estimates a warm-start threshold so the full fit
starts near-converged instead of thrashing rebuilds up from zero. "auto" is dense-only.
n_rebuilds_
property
¶
How many times the CF-tree rebuilt under the leaf bound; high ⇒ thrashing.
threshold_
property
¶
Current CF-tree absorption threshold (grows as it rebuilds).
effective_max_leaves_
property
¶
The max_leaves actually used: derived from memory_budget_mb if set, else
configured.
microcluster_centers_
property
¶
Leaf (microcluster) centroids — (n_microclusters, dim).
microcluster_weights_
property
¶
Leaf effective point mass — (n_microclusters,).
microcluster_radii_
property
¶
Leaf RMS radius — (n_microclusters,).
components_
property
¶
NMF parts H — (projection_dim, dim), rows unit-L2, ordered by descending energy.
Every leaf code is a nonnegative combination of these rows, so a row reads directly as a
"topic" over the input features. Requires a "weighted-nmf" / "weighted-nmf-kl"
projection.
reconstruction_err_
property
¶
Relative reconstruction error of the projection, ‖X̃ − W H‖_F / ‖X̃‖_F.
Measured over the leaf centroid matrix the factorizer actually fits — how much of the
compressed data projection_dim parts explain. Requires a projection.
cluster_centers_
property
¶
Macro-cluster centroids — (n_clusters, dim); requires a finalized clustering.
cluster_radii_
property
¶
Macro-cluster RMS radius — (n_clusters,); requires a finalized clustering.
cluster_sizes_
property
¶
Macro-cluster total point mass — (n_clusters,); requires a finalized clustering.
microcluster_proba_
property
¶
Per-microcluster GMM soft responsibilities (n_microclusters, k). GMM heads only.
fit ¶
Fit the CF-tree on X and cluster it; returns self (scikit-learn style).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
X
|
_FloatArray
|
dense float32/float64 array or a |
required |
y
|
Any
|
ignored; present for scikit-learn API compatibility. |
None
|
must_link
|
Any
|
optional |
None
|
cannot_link
|
Any
|
optional |
None
|
fit_predict ¶
fit_predict(
X: _FloatArray,
y: Any = None,
must_link: Any = None,
cannot_link: Any = None,
) -> NDArray[np.int64]
Fit on X; return one int64 label per row (-1 = noise, HDBSCAN head).
Takes the same X / must_link / cannot_link as :meth:fit.
partial_fit ¶
Absorb a chunk; call with no argument to finalize the global clustering.
assign_microclusters ¶
Nearest leaf index per row (matches microcluster_centers_ order).
outlier_scores ¶
Per-row distance to its assigned cluster centroid / that cluster's RMS radius.
summary ¶
A compact dict describing the dataset's structure (microclusters + macro clusters).
validity ¶
Internal validity indices of the fitted partition, scored on the leaf summary.
Returns calinski_harabasz (higher is better), davies_bouldin (lower is better) and
medoid_silhouette (higher is better, capped at 1). All three cost
O(n_leaves · k · d) rather than the O(N²) an exact silhouette over the points would
— the sum of squared distances inside a leaf is S_i + n_i‖μ_i − c‖² exactly, so no
point ever has to be revisited.
Caveats worth reading before using any of them to choose k: Calinski–Harabasz is exact
but undefined at k = 1; Davies–Bouldin is the RMS-dispersion variant, not the classical
mean-distance one; the medoid silhouette is the index of the summary, not of the points.
None of the three can report "there is no structure here" — for that, fit with
n_clusters=0 on a mixture head and let BIC answer.
find_outliers ¶
Row indices of the top_k most outlying points (highest score first).
sample_representatives ¶
For each cluster, the row indices of the k points nearest its centroid.
find_near_duplicates ¶
Groups (row-index arrays) of points sharing a microcluster tighter than radius.
near_duplicate_pairs ¶
Scored near-duplicate row pairs by exact cosine similarity within each microcluster.
The CF-tree blocks rows into leaves in O(N); within each (small) leaf, exact pairwise
cosine is computed and pairs scoring >= threshold are kept. The cost is
~O(N * leaf_size) -- the scalable counterpart to an O(N^2) all-pairs scan, and the
scored complement to :meth:find_near_duplicates (which returns unscored groups). Recall
is bounded by the blocking: a pair split across two leaves is missed; use a coarser tree
(smaller max_leaves) to widen blocks and trade speed for recall. Returns an
(m, 3) float64 array of [cos_sim, i, j] with i < j, sorted by similarity
descending.
mapper ¶
mapper(
lens: _Lens = "density",
resolution: int = 10,
gain: float = 0.3,
link_scale: float = 1.0,
min_node_mass: float = 0.0,
density_k: int = 5,
coordinate: int = 0,
) -> MapperGraph
Build a Mapper topological-skeleton :class:MapperGraph over the fitted microclusters.
TDA Mapper specialised to BETULA: a lens filter ("density" | "radius" |
"l2norm" | "coordinate" | "eccentricity") is covered by resolution bins
overlapping by gain; microclusters in a bin are single-linked at link_scale × the
bin's median nearest-neighbour gap; one node per (bin, component).
It surfaces non-convex structure, branch points and bridges (topic leakage) over the
M << N microclusters — an exploration tool, not a partition. Build the model first.
mapper_stability ¶
mapper_stability(
resolutions: Sequence[int] | None = None,
**mapper_kwargs: Any,
) -> list[dict[str, int]]
Sweep Mapper resolution and report how the topology persists across scale.
Returns a list of dicts (one per resolution) with resolution, n_nodes, n_edges,
n_branch_points, n_bridges, n_components (β₀, connected components) and
n_loops (β₁ = edges − nodes + components, the number of independent cycles). Features
constant across many resolutions are real structure; ones that flicker are binning
artefacts — the Mapper analogue of a persistence diagram, without cross-scale node matching.
resolutions defaults to range(4, 30, 2); mapper_kwargs (lens, gain,
link_scale …) pass straight through to :meth:mapper. Build the model first.
export_coreset ¶
export_coreset(
size: int | None = None,
k: int | None = None,
seed: int | None = None,
) -> Coreset
The leaf summary as a weighted-point :class:Coreset, optionally sampled down to
size points with a provable (k, ε) guarantee. Requires a built tree only —
partial_fit is enough, since which head this estimator fitted does not enter it.
With size=None this returns every leaf at its own mass: the streaming summary, exactly
as before, in one O(n_leaves) pass. Pass a size and the leaves are subsampled by
sensitivity sampling (Feldman & Langberg 2011), which costs one weighted k-means over
the leaves and fills in reference_cost / total_sensitivity.
The error has two independent halves, and neither is folded into the other.
Summarization, present in both modes. With Δ = offset = Σᵢ Sᵢ, the summary's cost
ĉost(C) = Σᵢ (Sᵢ + nᵢ‖μᵢ − C‖²) satisfies, for every candidate C and every k::
0 ≤ ĉost(C) − cost(C) ≤ 4·√(Δ · cost(C)) + 4·Δ
— a relative error of 4√ρ + 4ρ at ρ = Δ/cost(C), and cost(C) ≥ OPT_k bounds it
uniformly. :meth:Coreset.summary_epsilon evaluates it, and makes you name the α it
assumes. This is what makes the word "coreset" here a claim rather than a label.
Sampling, present only when size is given. ĉost(C) = Δ + Σᵢ nᵢ‖μᵢ − C‖² and
Δ does not depend on C, so the sample only has to be a coreset of the weighted set
{(μᵢ, nᵢ)} — offset carries the constant instead of losing it. Sensitivity sampling
attains the optimal worst-case size Õ(k·ε⁻²·min(√k, ε⁻²)), matching the STOC 2022 lower
bound, and Õ(k/ε²) on stable instances (arXiv 2405.01339).
size at or above the leaf count returns every leaf exactly, with no sampling error,
rather than a noisy redraw of something already held exactly. k defaults to
n_clusters and seed to this estimator's.
predict_proba ¶
Per-point soft assignment, shape (n, n_components).
The GMM, vMF, and Toeplitz (gmm-toeplitz / -full / -gs) heads score
the point under the fitted mixture, so predict_proba(X).argmax(1) is exactly
:meth:predict. k-means / Ward / HDBSCAN return a heuristic softmax(−d²/2τ²) over
the cluster centroids (τ = mean cluster radius) — a confidence proxy, not a
calibrated posterior. Columns are component indices aligned with :meth:predict.
assignment_confidence ¶
Per-point confidence in [0, 1] = the max soft-assignment probability (see
:meth:predict_proba); low values flag boundary / ambiguous points.
diagnostics ¶
A richer structural report than :meth:summary — compression, microcluster-radius
percentiles, rebuild count, and (once finalized) cluster mass spread.
representatives ¶
representatives(
X: _FloatArray,
cluster_id: int,
method: _Repr = "medoid",
k: int = 5,
) -> NDArray[np.int64]
Row indices of k representatives of cluster_id. method: medoid (nearest
centroid), boundary (farthest in-cluster), outlier (highest outlier score),
diverse (farthest-point sampling). Empty if the cluster has no predicted members.
cluster_profile ¶
A JSON-able profile of a macro-cluster (size, radius, center, nearest clusters) — feed to an LLM to name it. Geometry only; no data pass needed.
snapshot ¶
A JSON-able snapshot of the current cluster geometry (centers / sizes / radii) for drift
monitoring across time. Requires a finalized clustering; compare two with
:meth:compare_snapshots.
compare_snapshots
staticmethod
¶
Drift report between two :meth:snapshot dicts. Each after cluster is matched to its
nearest before centroid; reports the centroid shift (absolute and in after-radius
units) and the mass ratio per match, plus the cluster counts and the worst shift. Both
snapshots must come from finalized models with ≥ 1 cluster.
active_learning_batch ¶
active_learning_batch(
X: _FloatArray,
n: int = 100,
strategy: _Strategy = "uncertain",
) -> NDArray[np.int64]
Row indices of the n most informative points to review/label. strategy:
uncertain (lowest :meth:assignment_confidence) or outlier (highest
:meth:outlier_scores) — for human-in-the-loop curation / labeling.
Streaming¶
betula_cluster.DenStream ¶
Streaming DenStream density clusterer (Cao et al., SDM 2006) over fading micro-clusters.
For evolving streams where old data should fade: feed chunks with :meth:partial_fit, then
:meth:predict (which finalizes the offline clustering on first call) — or both at once with
:meth:fit / :meth:fit_predict. eps is the micro-cluster radius (tune to the data scale),
decay the fading rate λ, and beta × mu the promotion/pruning weight (must exceed 1).
Spherical micro-clusters, float64; -1 labels are noise.
n_microclusters_
property
¶
Number of potential (cluster-eligible) micro-clusters.
microcluster_weights_
property
¶
Potential micro-cluster weights, faded to the current stream time.
partial_fit ¶
Stream a chunk of points into the fading micro-clusters.
cluster ¶
Run the offline step (label the potential micro-clusters) over what has streamed.
predict ¶
Label rows by their nearest potential micro-cluster (-1 = noise); finalizes the
offline clustering first if points have streamed since the last :meth:cluster.
betula_cluster.DbStream ¶
Streaming DBSTREAM density clusterer (Hahsler & Bolaños, 2016) over fading micros.
Like :class:DenStream it fades old data and marks -1 as noise, but it connects
micro-clusters by shared density — the mass of points within radius r of both — rather
than by mere proximity. This recovers arbitrarily-shaped clusters (chained overlapping
micro-clusters) and, unlike a distance rule, keeps two close-but-disconnected dense regions
apart (an empty gap means zero shared density). r is the radius, decay the fading rate,
alpha the shared-density bridge threshold (a pair links when their overlap mass exceeds
alpha * min_weight), and min_weight the weight a micro-cluster needs to form a cluster.
microcluster_weights_
property
¶
Micro-cluster weights, faded to the current stream time.
partial_fit ¶
Stream a chunk of points into the fading micro-clusters.
cluster ¶
Run the offline step (label micro-clusters via the shared-density graph).
predict ¶
Label rows by their nearest micro-cluster within r (-1 = noise); finalizes the
offline clustering first if points have streamed since the last :meth:cluster.
Mixed-type data¶
betula_cluster.KPrototypes ¶
k-prototypes clustering of mixed numeric + categorical data (Huang, 1997).
categorical lists the integer-coded categorical column indices of X; the rest are
numeric. Distance is ||Δnum||² + gamma · (categorical mismatch); gamma defaults to half
the mean numeric standard deviation (Huang's heuristic) when None. Rows are summarised into
bounded mixed micro-clusters (a flat leader pass capped at max_leaves) before clustering, so
memory stays bounded. Both numeric and categorical columns are required; float64.
Coresets¶
betula_cluster.Coreset
dataclass
¶
A weighted-point coreset: the CF-tree leaf microclusters as (centers, weights, radii).
Each row is one microcluster — a numerically stable summary of the points absorbed into it. The
set is bounded by max_leaves and built in a single streaming pass, so fitting a weighted
clustering / classifier on it is competitive with fitting on the full data at a fraction of the
cost.
cost ¶
Σⱼ wⱼ·d²(xⱼ, C) + offset — this coreset's estimate of the summary's k-means cost.
A method rather than a formula in the docs because offset is easy to forget and
forgetting it understates every cost by the same constant, which looks like nothing until
two costs from different coresets are compared.
summary_epsilon ¶
Relative summarization error 4√ρ + 4ρ at ρ = alpha · offset / reference_cost.
alpha is required, not defaulted. reference_cost is the cost of an α-approximate
solution and therefore upper-bounds OPT_k, so offset / reference_cost
under-states the true ρ = Δ/OPT_k and summary_epsilon(1.0) is an optimistic
reading rather than a certificate. The shipped seeding is k-means++ with greedy trials,
whose O(log k) guarantee holds in expectation, not for the run in hand — so pass the
factor you can defend.
Covers the summary only; sampling error sits on top of it and is what size buys down.
Streaming sketches¶
betula_cluster.KllSketch ¶
Streaming KLL quantile sketch (rank-error). Standalone betula-sketch primitive.
__doc__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__new__
builtin
¶
Create and return a new object. See help(type) for accurate signature.
quantiles
method descriptor
¶
Estimated quantiles for an array of q values.
update_many
method descriptor
¶
Add every value of a 1-D array.
betula_cluster.DdSketch ¶
Streaming DDSketch quantile sketch (relative-error). Standalone betula-sketch primitive.
__doc__
class-attribute
¶
__doc__ = "Streaming **DDSketch** quantile sketch (relative-error). Standalone `betula-sketch` primitive."
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__new__
builtin
¶
Create and return a new object. See help(type) for accurate signature.
quantiles
method descriptor
¶
Estimated quantiles for an array of q values.
update_many
method descriptor
¶
Add every value of a 1-D array.
Robustness¶
betula_cluster.consensus ¶
consensus(
X: _FloatArray,
n_clusters: int,
*,
n_runs: int = 5,
seed: int = 0,
n_jobs: int = 1,
**fit_kwargs: object,
) -> ConsensusResult
Cluster X under n_runs random insertion-order permutations; return the consensus
labelling and a per-point stability score (see :class:ConsensusResult).
Extra keyword arguments are forwarded to :func:fit_predict (feature / method /
threshold / …). Intended for the partitional heads (kmeans / gmm / ward /
spectral) at a fixed n_clusters; density heads (hdbscan) emit noise / variable
counts the vote cannot align, and are rejected. n_jobs runs the (independent) permutations
in parallel threads — the Rust core releases the GIL, so this scales — with <0 meaning all
cores; each run is seeded independently, so the result is identical regardless of n_jobs.
Result types¶
betula_cluster.MapperGraph
dataclass
¶
A Mapper topological-skeleton graph over a fitted model's leaf microclusters.
Each node is a connected group of microclusters inside one cover bin; edges link nodes that
share microclusters (from the cover overlap). branch_points are nodes where the shape splits
(degree ≥ 3); bridges index the edges whose removal would disconnect the graph — thin
links between otherwise separate regions (e.g. leakage between topics in an embedding).
edge_overlap is a per-edge Bhattacharyya coefficient in (0, 1] from the two nodes'
pooled diagonal-Gaussian summaries: a bridge across a sparse neck scores lower than an edge
inside one dense blob, so links are weighted by distributional overlap, not a bare shared count.
persistence ¶
The nerve's 0-D persistence diagram as (k, 2) births/deaths, sorted by persistence.
filtration="overlap" (default) filters by the 1 − edge_overlap gap — a finite bar's
death is the Bhattacharyya depth of a bottleneck, a ranked upgrade of the boolean bridges;
"lens" is the lens sublevel diagram (flares of the shape). Essential (component) classes
carry np.inf in the death column; finite_only=True drops them.
to_networkx ¶
Build a networkx.Graph (requires networkx); nodes carry mass/bin/lens/centroid,
edges carry weight and a boolean bridge flag.
betula_cluster.TuneResult
dataclass
¶
The outcome of :func:tune: the best single configuration, every trial, and — for
multi_objective=True — the non-dominated (quality / memory / speed) Pareto front.
betula_cluster.ConsensusResult
dataclass
¶
Consensus of several insertion-order-permuted clusterings + a per-point stability score.
The CF-tree is sensitive to insertion order; clustering n_runs random permutations and
voting turns that into a measurable quantity. labels is the majority label per point (input
order); confidence is the fraction of runs agreeing with it — low on an unstable boundary,
high where every insertion order groups the point the same way.
mean_confidence
property
¶
Mean per-point stability across the input — a scalar robustness summary in [0, 1].