Biography

I am pursuing my master’s degree in Control Engineering, Zhejiang University, Hangzhou, China. My major research interests lie in network pruning, quantization, and deployment.

Research and Interests

• Network Pruning
• Neural Network Deployment

Publications

• Siqi Li, Jiangyang Xiang, Jiateng Wei, Chengrui Zhu, Jiangdang Yang, Jun Chen, Jian Yang, Xiaobin Wei, Yunliang Jiang, and Yong Liu. WLR: Well-conditioned linear reconstruction for retraining-free pruning of LLMs. Neural Networks, 200:108840, 2026.
  [BibTeX] [Abstract] [DOI] [PDF]

  Structured pruning is widely recognized as an effective method for reducing the size and computational demands of large language models (LLMs). Traditionally, structured pruning follows a pretrain-prune-retrain framework. Given the high cost of retraining LLMs, recent research has focused on efficient compensation methods to replace the retraining phase. However, many compensation methods introduce additional parameters into the pruned model, which affect deployment and inference. Additionally, many compensation techniques overlook the issue of ill-conditioning caused by outliers in LLMs during the process of solving optimization problems, leading to numerical instability and inefficiency in solutions. To overcome these challenges, we propose Well-conditioned Linear Reconstruction (WLR), a structured pruning compensation method that avoids introducing extra parameters. WLR reconstructs pruned layers using a linear combination of preserved channels while addressing the issue of ill-conditioning during the process. We evaluated our method on LLaMA-V1/V2/V3 and OPT families across multiple language tasks, achieving performance that surpasses the state-of-the-art methods. © 2026 Elsevier Ltd. All rights are reserved, including those for text and data mining, AI training, and similar technologies.

  @article{li2026wlr,
  title = {WLR: Well-conditioned linear reconstruction for retraining-free pruning of LLMs},
  author = {Siqi Li and Jiangyang Xiang and Jiateng Wei and Chengrui Zhu and Jiangdang Yang and Jun Chen and Jian Yang and Xiaobin Wei and Yunliang Jiang and Yong Liu},
  year =2026,
  journal = {Neural Networks},
  volume = 200,
  pages = {108840},
  doi = {10.1016/j.neunet.2026.108840},
  abstract = {Structured pruning is widely recognized as an effective method for reducing the size and computational demands of large language models (LLMs). Traditionally, structured pruning follows a pretrain-prune-retrain framework. Given the high cost of retraining LLMs, recent research has focused on efficient compensation methods to replace the retraining phase. However, many compensation methods introduce additional parameters into the pruned model, which affect deployment and inference. Additionally, many compensation techniques overlook the issue of ill-conditioning caused by outliers in LLMs during the process of solving optimization problems, leading to numerical instability and inefficiency in solutions. To overcome these challenges, we propose Well-conditioned Linear Reconstruction (WLR), a structured pruning compensation method that avoids introducing extra parameters. WLR reconstructs pruned layers using a linear combination of preserved channels while addressing the issue of ill-conditioning during the process. We evaluated our method on LLaMA-V1/V2/V3 and OPT families across multiple language tasks, achieving performance that surpasses the state-of-the-art methods.
  © 2026 Elsevier Ltd. All rights are reserved, including those for text and data mining, AI training, and similar technologies.}
  }

• Jiateng Wei, Quan Lu, Ning Jiang, Siqi Li, Jingyang Xiang, Jun Chen, and Yong Liu. Structured Optimal Brain Pruning for Large Language Models. In The 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 13991-14007, 2024.
  [BibTeX] [Abstract] [DOI]

  language=”eng” data-ev-field=”abstract”>The massive parameters and computational demands hinder the widespread application of Large Language Models (LLMs). Network pruning provides a practical solution to this problem. However, existing pruning works for LLMs mainly focus on unstructured pruning or necessitate post-pruning fine-tuning. The former relies on special hardware to accelerate computation, while the latter may need substantial computational resources. In this paper, we introduce a retraining-free structured pruning method called SoBP (Structured Optimal Brain Pruning). It leverages global first-order information to select pruning structures, then refines them with a local greedy approach, and finally adopts module-wise reconstruction to mitigate information loss. We assess the effectiveness of SoBP across 14 models from 3 LLM families on 8 distinct datasets. Experimental results demonstrate that SoBP outperforms current state-of-the-art methods.

  @inproceedings{wei2024sob,
  title = {Structured Optimal Brain Pruning for Large Language Models},
  author = {Jiateng Wei and Quan Lu and Ning Jiang and Siqi Li and Jingyang Xiang and Jun Chen and Yong Liu},
  year = 2024,
  booktitle = {The 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  pages = {13991-14007},
  doi = {10.18653/v1/2024.emnlp-main.775},
  abstract = {language="eng" data-ev-field="abstract">The massive parameters and computational demands hinder the widespread application of Large Language Models (LLMs). Network pruning provides a practical solution to this problem. However, existing pruning works for LLMs mainly focus on unstructured pruning or necessitate post-pruning fine-tuning. The former relies on special hardware to accelerate computation, while the latter may need substantial computational resources. In this paper, we introduce a retraining-free structured pruning method called SoBP (Structured Optimal Brain Pruning). It leverages global first-order information to select pruning structures, then refines them with a local greedy approach, and finally adopts module-wise reconstruction to mitigate information loss. We assess the effectiveness of SoBP across 14 models from 3 LLM families on 8 distinct datasets. Experimental results demonstrate that SoBP outperforms current state-of-the-art methods.}
  }