feat(configs): add HDBSCAN benchmark configs

configs/BENCH-CONFIG-SPEC.md

@@ -108,6 +108,7 @@ Configs have the three highest parameter keys:
 |:---------------|:--------------|:--------|:------------|
 | `algorithm`:`estimator` | None |  | Name of measured estimator. |
 | `algorithm`:`estimator_params` | Empty `dict` |  | Parameters for estimator constructor. |
+| `algorithm`:`method_params`:`{method}` | None |  | Parameters for a measured method of estimator. A method listed here is called with these parameters only, without data, which fits methods working off the fitted model such as `HDBSCAN.dbscan_clustering`. |
 | `algorithm`:`batch_size`:`{stage}` | None | Any positive integer | Enables online mode for `{stage}` methods of estimator (sequential calls for each batch). |
 | `algorithm`:`sklearn_context` | None |  | Parameters for sklearn `config_context` used over estimator. `array_api_dispatch` requires `SCIPY_ARRAY_API=1`, which scikit-learn_bench sets by default if it is unset in the environment. |
 | `algorithm`:`sklearnex_context` | None |  | Parameters for sklearnex `config_context` used over estimator. Updated by `sklearn_context` if set. |
@@ -136,10 +137,11 @@ List of available special values:
 | `data`:`dataset` | all | `all_named` | Sets datasets to use as list of all named datasets available in loaders. |
 | `data`:`generation_kwargs`:`n_informative` | all | *float* value in [0, 1] range | Sets datasets to use as list of all named datasets available in loaders. |
 | `bench`:`taskset` | all | Specification of numa nodes in `numa:{numa_node_0}[\|{numa_node_1}...]` format | Sets CPUs affinity using `taskset` utility. |
-| `algorithm`:`estimator_params`:`n_jobs` | sklearn_estimator | `physical_cpus`, `logical_cpus`, or ratio of previous ones in format `{type}_cpus:{ratio}` where `ratio` is float | Sets `n_jobs` parameter to a number of physical/logical CPUs or ratio of them for an estimator. |
+| `algorithm`:`estimator_params`:`n_jobs` | sklearn_estimator | `physical_cpus`, `logical_cpus`, or ratio of previous ones in format `{type}_cpus:{ratio}` where `ratio` is float | Sets `n_jobs` parameter to a number of physical/logical CPUs or ratio of them for an estimator. On Linux, only the CPUs in the process affinity mask (`bench`:`taskset`, numactl) are counted. |
 | `algorithm`:`estimator_params`:`scale_pos_weight` | sklearn_estimator | `auto` | Sets `scale_pos_weight` parameter to `sum(negative instances) / sum(positive instances)` value for estimator. |
 | `algorithm`:`estimator_params`:`n_clusters` | sklearn_estimator | `auto` | Sets `n_clusters` parameter to number of clusters or classes from dataset description for estimator. |
 | `algorithm`:`estimator_params`:`eps` | sklearn_estimator | `distances_quantile:{quantile}` format where quantile is *float* value in [0, 1] range | Computes `eps` parameter as quantile value of distances in `x_train` matrix for estimator. |
+| `algorithm`:`method_params`:`dbscan_clustering`:`cut_distance` | sklearn_estimator | `distances_quantile:{quantile}` format where quantile is *float* value in [0, 1] range | Computes `cut_distance` the same way as `eps` above, so that an HDBSCAN hierarchy can be re-cut at the distance DBSCAN is measured at. |
 
 ## Range of Values
 

configs/common/hdbscan.json

@@ -0,0 +1,48 @@
+{
+    "PARAMETERS_SETS": {
+        "common hdbscan parameters": {
+            "algorithm": {
+                "estimator": "HDBSCAN",
+                "estimator_params": {
+                    "min_cluster_size": 5,
+                    "min_samples": 5,
+                    "metric": "euclidean",
+                    "cluster_selection_method": "eom",
+                    "allow_single_cluster": false,
+                    "store_centers": null,
+                    "copy": false
+                },
+                "estimator_methods": { "training": "fit" }
+            },
+            "data": { "format": "numpy", "order": "C", "dtype": "float64" },
+            "bench": { "n_runs": 3, "time_limit": 1200 }
+        },
+        "sklearn hdbscan parameters": {
+            "algorithm": {
+                "estimator_params": { "n_jobs": "[SPECIAL_VALUE]physical_cpus" }
+            }
+        },
+        "hdbscan brute method": {
+            "algorithm": { "estimator_params": { "algorithm": "brute" } }
+        },
+        "hdbscan kd_tree method": {
+            "algorithm": { "estimator_params": { "algorithm": "kd_tree", "leaf_size": 40 } }
+        },
+        "hdbscan ball_tree method": {
+            "algorithm": { "estimator_params": { "algorithm": "ball_tree", "leaf_size": 40 } }
+        },
+        "hdbscan tree methods": {
+            "algorithm": {
+                "estimator_params": { "algorithm": ["kd_tree", "ball_tree"], "leaf_size": 40 }
+            }
+        },
+        "hdbscan all methods": {
+            "algorithm": {
+                "estimator_params": {
+                    "algorithm": ["brute", "kd_tree", "ball_tree"],
+                    "leaf_size": 40
+                }
+            }
+        }
+    }
+}

configs/experiments/README.md

@@ -2,4 +2,16 @@
 
 `daal4py_svd`: tests performance scalability of `daal4py.svd` algorithm
 
+`hdbscan_datasets`: runs `HDBSCAN` over real datasets, synthetic shapes and large tall or wide data, to cover different data situations rather than parameters.
+
+`hdbscan_gpu_boundaries`: compares `HDBSCAN` on CPU and GPU over `n_samples`, `n_features`, dtypes and metrics, up to sizes stock `sklearn` does not finish.
+
+`hdbscan_parameters`: sweeps the `HDBSCAN` parameter space (metrics, cluster selection, density thresholds, stored centers, dtypes and data formats) over the `sklearn` and `sklearnex` implementations.
+
+`hdbscan_scaling`: tests thread, NUMA, `n_samples` and `n_features` scalability of `HDBSCAN`.
+
+`hdbscan_vs_cuml`: compares `sklearnex` `HDBSCAN` on Intel GPU with `cuml` `HDBSCAN` on NVIDIA GPU.
+
+`hdbscan_vs_dbscan`: compares `HDBSCAN` and its `dbscan_clustering` re-cut with `DBSCAN` at the same distance.
+
 `nearest_neighbors`: tests performance of neighbors search implementations from `sklearnex`, `sklearn`, `raft`, `faiss` and `svs`.

configs/experiments/hdbscan_datasets.json

@@ -0,0 +1,156 @@
+{
+    "INCLUDE": ["../common/sklearn.json", "../common/hdbscan.json"],
+    "PARAMETERS_SETS": {
+        "sklearnex only [cpu,gpu]": [
+            { "algorithm": { "library": "sklearnex", "device": "cpu" } },
+            {
+                "algorithm": {
+                    "library": "sklearnex",
+                    "device": "gpu",
+                    "sklearn_context": { "array_api_dispatch": true }
+                },
+                "data": { "format": "dpnp", "order": "C" }
+            }
+        ],
+        "hdbscan datasets limits": {
+            "bench": { "n_runs": 3, "time_limit": 600 }
+        },
+        "hdbscan brute and kd_tree methods": {
+            "algorithm": { "estimator_params": { "algorithm": ["brute", "kd_tree"], "leaf_size": 40 } }
+        },
+        "moderate real datasets": {
+            "data": [
+                {
+                    "dataset": ["letters", "california_housing", "digits"],
+                    "preprocessing_kwargs": { "normalize": "standard" }
+                },
+                {
+                    "dataset": ["skin_segmentation", "road_network"],
+                    "split_kwargs": { "train_size": 50000 },
+                    "preprocessing_kwargs": { "normalize": "standard" }
+                },
+                {
+                    "dataset": ["covtype", "sensit", "year_prediction_msd"],
+                    "split_kwargs": { "train_size": 20000 },
+                    "preprocessing_kwargs": { "normalize": "standard" }
+                },
+                {
+                    "dataset": "creditcard",
+                    "split_kwargs": { "train_size": 30000 },
+                    "preprocessing_kwargs": { "normalize": "standard" }
+                },
+                {
+                    "dataset": "fashion_mnist",
+                    "split_kwargs": { "train_size": 10000 },
+                    "preprocessing_kwargs": { "normalize": "standard" }
+                }
+            ]
+        },
+        "synthetic shapes": {
+            "data": [
+                {
+                    "source": "make_moons",
+                    "generation_kwargs": { "n_samples": 20000, "noise": 0.05, "random_state": 42 },
+                    "split_kwargs": { "ignore": true }
+                },
+                {
+                    "source": "make_circles",
+                    "generation_kwargs": { "n_samples": 20000, "noise": 0.05, "factor": 0.5, "random_state": 42 },
+                    "split_kwargs": { "ignore": true }
+                },
+                {
+                    "source": "make_blobs",
+                    "generation_kwargs": {
+                        "centers": 50,
+                        "n_samples": 20000,
+                        "n_features": 8,
+                        "cluster_std": [0.5, 8.0],
+                        "random_state": 42
+                    },
+                    "split_kwargs": { "ignore": true }
+                },
+                {
+                    "source": "make_classification",
+                    "generation_kwargs": {
+                        "n_samples": 20000,
+                        "n_features": 20,
+                        "n_informative": 10,
+                        "n_classes": 5,
+                        "random_state": 42
+                    },
+                    "split_kwargs": { "ignore": true }
+                }
+            ]
+        },
+        "large tall datasets": {
+            "data": [
+                {
+                    "dataset": [
+                        "skin_segmentation",
+                        "road_network",
+                        "covtype",
+                        "creditcard",
+                        "year_prediction_msd",
+                        "sensit"
+                    ],
+                    "preprocessing_kwargs": { "normalize": "standard" }
+                },
+                {
+                    "dataset": ["susy", "higgs"],
+                    "split_kwargs": { "train_size": 1000000 },
+                    "preprocessing_kwargs": { "normalize": "standard" }
+                }
+            ]
+        },
+        "large wide datasets": {
+            "data": [
+                { "dataset": ["mnist", "fashion_mnist"], "preprocessing_kwargs": { "normalize": "standard" } },
+                {
+                    "dataset": "cifar",
+                    "split_kwargs": { "train_size": 30000 },
+                    "preprocessing_kwargs": { "normalize": "standard" }
+                }
+            ]
+        }
+    },
+    "TEMPLATES": {
+        "hdbscan real datasets [all methods]": {
+            "SETS": [
+                "sklearn-ex[cpu,gpu] implementations",
+                "common hdbscan parameters",
+                "sklearn hdbscan parameters",
+                "hdbscan all methods",
+                "moderate real datasets",
+                "hdbscan datasets limits"
+            ]
+        },
+        "hdbscan synthetic shapes [all methods]": {
+            "SETS": [
+                "sklearn-ex[cpu,gpu] implementations",
+                "common hdbscan parameters",
+                "sklearn hdbscan parameters",
+                "hdbscan all methods",
+                "synthetic shapes",
+                "hdbscan datasets limits"
+            ]
+        },
+        "hdbscan large tall datasets [kd_tree]": {
+            "SETS": [
+                "sklearnex only [cpu,gpu]",
+                "common hdbscan parameters",
+                "hdbscan kd_tree method",
+                "large tall datasets",
+                "hdbscan datasets limits"
+            ]
+        },
+        "hdbscan large wide datasets [brute, kd_tree]": {
+            "SETS": [
+                "sklearnex only [cpu,gpu]",
+                "common hdbscan parameters",
+                "hdbscan brute and kd_tree methods",
+                "large wide datasets",
+                "hdbscan datasets limits"
+            ]
+        }
+    }
+}