Cyber threat management using semi-supervised ensemble learning and enhanced interior search algorithm: applications for illicit marketplace classification in deep/dark web and social platforms
提出一种两阶段半监督方法，结合集成学习和增强型内部搜索算法，用于检测和分类深度/暗网及社交平台上的非法市场，解决了标注数据不足的问题，在销售检测和毒品、武器、凭证分类上取得高F1分数。
Abstract In response to escalating digital crime on the web, our study proposes a two-phase, semi-supervised method to detect and classify illicit marketplaces, addressing the challenge of limited labeled text documents on the deep/dark web, Telegram, Reddit, and Pastebin. Initially, we developed a self-training semi-supervised ensemble model using XGBoost, Random Forest, and AdaBoost to classify sales-related documents accurately. Next, we employ three XGBoost semi-supervised models to further classify “Sale” documents into “Drug”, “Weapon”, and stolen “Credential” categories. To enhance the performance of the classification model, we propose the Enhanced Interior Search Algorithm (EISA), which is an advanced version of the Interior Search Algorithm (ISA) meta-heuristic, for feature selection and hyperparameter tuning. EISA uses different mirror types and Lévy flights to efficiently navigate search spaces and converge on high-quality solutions. Additionally, we propose an application-specific objective function allowing for adjustments based on task requirements. Experimental results indicate that our EISA-aided semi-supervised approach effectively detects and classifies illicit marketplaces, addressing the lack of labeled data, while achieving competitive performance against other models with F1-scores of 0.9451 for “Sale” detection, 0.9173 for “Drug” sales, 0.9250 for “Weapon” sales, and 0.8862 for “Credential” sales, along with Transformed Matthews Correlation Coefficient ( TMCC ) values of 0.9108, 0.9202, 0.9472, and 0.8560. Besides, changes in the objective function’s weights demonstrate the model’s adaptability and applicability in real-world environments. The successful deployment of the EISA-aided semi-supervised model highlights its effectiveness in combating digital crime and confirms its practical value.