Vol. 1 No. 4 (2026)

Articles

Hierarchical Clustering with Stability-Driven Automatic Cluster-Number Selection Under Noise Perturbations

Published 2026-09-15

  • Hannah Anderson
    • ,
  • Emily Hernandez
    • ,
  • Zoe Reed

Hannah Anderson
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, Urbana, Illinois, USA

Emily Hernandez
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, Urbana, Illinois, USA

Zoe Reed
Department of Electrical and Computer Engineering, Swanson School of Engineering, University of Pittsburgh, Pittsburgh, Pennsylvania, USA

Machine Learning and Applied Artificial Intelligence, Volume 1, Issue 4, 2026 cover

Abstract

The determination of the optimal number of clusters remains one of the most persistent and challenging problems in unsupervised machine learning, particularly within the domain of hierarchical clustering. Traditional validation indices often fail to provide reliable estimates when datasets are subjected to various forms of noise perturbations, as they rely heavily on geometric compactness and separation criteria that are easily disrupted by outliers and measurement errors. This paper introduces a comprehensive framework for stability-driven automatic cluster-number selection, specifically designed to withstand significant noise perturbations. By systematically injecting varying levels of synthetic noise into the original dataset and computing the resultant hierarchical structures, we evaluate the topological consistency of partitions across multiple perturbation iterations. The proposed methodology leverages an advanced consensus-based stability metric that quantifies the structural resilience of each possible cluster count. Extensive theoretical analysis and empirical evaluations on both synthetic and real-world high-dimensional datasets demonstrate that the stability-driven approach consistently outperforms classical heuristics, such as the gap statistic and silhouette coefficient, under low signal-to-noise ratio conditions. The findings presented in this study offer a robust computational paradigm for automated exploratory data analysis, eliminating the need for subjective human intervention in dendrogram interpretation while ensuring high reliability in noisy environments.

Keywords

Hierarchical Clustering, Cluster-Number Selection, Noise Perturbations, Stability Analysis

PDF

Author Biography

Hannah Anderson

Editorial correspondence on behalf of the authors

References

  1. Wolf, F. A., Angerer, P., & Theis, F. J. (2018). SCANPY: Large-scale single-cell gene expression data analysis. Genome Biology, 19, 15.
  2. Liu, W., Gao, S., & Lee, L. H. (2017, December). A multi-objective perspective on robust ranking and selection. In 2017 Winter Simulation Conference (WSC) (pp. 2116-2127). IEEE.
  3. Li, Q., Liu, Z., Xu, T., Luo, T., & Hou, C. (2026). Adaptive Disentangled Representation Learning for Incomplete Multi-View Multi-Label Classification. arXiv preprint arXiv:2601.05785.
  4. Zhang, J., Luo, H., Yang, H., Xu, X., Wu, Q., Shi, Y., ... & Wang, J. (2023). Neuraldome: A neural modeling pipeline on multi-view human-object interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 8834-8845).
  5. Hu, H., Yang, Y., Zhao, Q., & Zhang, C. (2022, June). On the role of discount factor in offline reinforcement learning. In International conference on machine learning (pp. 9072-9098). PMLR.
  6. Jin, H., Tsai, F. S., & Martinez-Vazquez, J. (2025). Optimal expenditure decentralization for sustainable development: evidence from a 52-country panel analysis. Financial Innovation, 11(1), 111.
  7. Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauley, M., et al. (2010). Spark: Cluster computing with working sets. In Proceedings of HotCloud.
  8. Liu, H., & Henao, R. (2025, April). Learning to substitute words with model-based score ranking. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) (pp. 11551-11565).
  9. Sang, Y. (2026, March). Adaptive Quantization Strategies for Robust ML Inference Under Distribution Shift. In Proceedings of the 2026 5th International Conference on Cyber Security, Artificial Intelligence and Digital Economy (pp. 362-368).
  10. Gao, S., & Chen, W. (2015). Efficient subset selection for the expected opportunity cost. Automatica, 59, 19-26.
  11. Gao, F., Shi, Z., Gao, S., & Xiao, H. (2019). Efficient simulation budget allocation for subset selection using regression metamodels. Automatica, 106, 192-200.
  12. Sievers, F., Wilm, A., Dineen, D., Gibson, T. J., Karplus, K., Li, W., et al. (2011). Fast, scalable generation of high-quality protein multiple sequence alignments using Clustal Omega. Molecular Systems Biology, 7, 539.
  13. Lyu, S., Mao, S., & Ma, Y. (2025). Unified Long Video Inpainting and Outpainting via Overlapping High-Order Co-Denoising. arXiv preprint arXiv:2511.03272.
  14. Wang, Z., Kim, E. H., Oh, S. K., Pedrycz, W., Fu, Z., & Yoon, J. H. (2024). Reinforced fuzzy-rule-based neural networks realized through streamlined feature selection strategy and fuzzy clustering with distance variation. IEEE Transactions on Fuzzy Systems, 32(10), 5674-5686.
  15. Xiong, F., Xu, H., Wang, Y., Cheng, R., Wang, Y., & Chu, X. (2025, November). Hs-star: Hierarchical sampling for self-taught reasoners via difficulty estimation and budget reallocation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (pp. 5539-5555).
  16. Li, Q., Luo, T., Jiang, M., Jiang, Z., Hou, C., & Li, F. (2025, April). Semi-supervised multi-view multi-label learning with view-specific transformer and enhanced pseudo-label. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 39, No. 17, pp. 18430-18438).
  17. Jiang, L., Cao, Z., Chen, W., Zheng, X., Lyu, Y., Li, Z., ... & Chen, Y. C. (2026). SAP: Segment Any 4K Panorama. arXiv preprint arXiv:2603.12759.
  18. Zhu, R., Jiang, B., Mei, L., Yang, F., Wang, L., Gao, H., ... & Zhang, D. (2025). Adaptflow: Adaptive workflow optimization via meta-learning. arXiv preprint arXiv:2508.08053.
  19. Xiaoteng Ma, Yiqin Yang, Hao Hu, Qihan Liu, Jun Yang, Chongjie Zhang, Qianchuan Zhao, and Bin Liang. Offline reinforcement learning with value-based episodic memory. arXiv preprint arXiv:2110.09796, 2021b.
  20. Yao, Z., Li, R., Chen, X., Cheng, J., Le, C., Yue, M., ... & Dong, J. (2026). Less Data, Better Alignment: Data-Centric Multi-Evaluator Agreement for Preference Optimization. arXiv preprint arXiv:2607.25136.
  21. Zou, Y., & Chou, C. A. (2022). A combinatorial optimization approach for multi-label associative classification. Knowledge-Based Systems, 240, 108088.
  22. Li, Q., Luo, T., & Liao, J. (2025). Theory-inspired deep multi-view multi-label learning with incomplete views and noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 20706-20715).
  23. Luo, T., Li, Q., Jiang, M., & Hou, C. (2025). Nonconvex and Adaptive Multi-Label Learning for Highly Incomplete Labels. Knowledge-Based Systems, 114784.
  24. Qu, W., Shao, Y., Meng, L., Huang, X., & Xiao, L. (2024, June). A conditional denoising diffusion probabilistic model for point cloud upsampling. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 20786-20795). IEEE.
  25. Sun, L., Hu, J., Li, M., & Peng, H. (2024, July). R-ode: Ricci curvature tells when you will be informed. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 2594-2598).
  26. M. Emani et al., ‘‘A comprehensive evaluation of novel AI accelerators for deep learning workloads,’’ in Proc. IEEE/ACM Int. Workshop Perform. Modeling, Benchmarking Simulation High Perform. Comput. Syst. (PMBS), Nov. 2022, pp. 13–25.
  27. Zhao, J., Zhang, C., Qin, M., & Yang, P. (2025). QuantFactor REINFORCE: mining steady formulaic alpha factors with variance-bounded REINFORCE. IEEE Transactions on Signal Processing, 73, 2448-2463.
  28. Liu, Z., Ye, Z., & Zhang, K. (2025, May). Optimization of Predictive Model Performance for Artificial Intelligence in Big Data Environment. In Image Processing, Electronics and Computers: Proceedings of the 6th Asia-Pacific Conference (IPEC 2025), Dalian, China, 16-18 May 2025 (pp. 1870-1878). 1 Oliver's Yard, 55 City Road, London, EC1Y 1SP: SAGE Publications.
  29. Tian, Y. (2026, January). A Data-Driven Energy Efficiency Improvement and Cost Optimisation Model for New Energy Electric Vehicles. In 2026 IEEE International Conference on Power, Electronics and Green Energy (ICPEGE) (pp. 304-308). IEEE.
  30. Sha, L., Wang, Y., Meng, P., Deng, Y., Chen, T., Zhang, X., ... & Xu, Q. (2025). Pharmacological inhibition of PSPH reduces serine levels and epileptic seizures. Nature Chemical Biology, 21(11), 1742-1753.