Biography

I received the B.S. and the M.S. degrees in control engineering from China University of Geosciences, Wuhan, China, in 2016 and 2019, respectively. I am pursuing a Ph.D degree in engineering with College of Control Science and Engineering, Zhejiang University, Hangzhou, China. My major research interests include process control, data mining, and machine learning.

Research and Interests

• Data Mining
• Process Control

Publications

• Weiwei Liu, Wei Jing, Shanqi Liu, Yudi Ruan, Kexin Zhang, Jian Yang, and Yong Liu. Expert Demonstrations Guide Reward Decomposition for Multi-Agent Cooperation. Neural Computing and Applications, 35:19847-19863, 2023.
  [BibTeX] [Abstract] [DOI] [PDF]

  Humans are able to achieve good teamwork through collaboration, since the contributions of the actions from human team members are properly understood by each individual. Therefore, reasonable credit assignment is crucial for multi-agent cooperation. Although existing work uses value decomposition algorithms to mitigate the credit assignment problem, since they decompose the global value function at multi-agents’ local value function level, the overall evaluation of the value function can easily lead to approximation errors. Moreover, such strategies are vulnerable to sparse reward scenarios. In this paper, we propose to use expert demonstrations to guide the team reward decomposition at each time step, rather than value decomposition. The proposed method computes the reward ratio of each agent according to the similarity between the state-action pair of the agent and the expert demonstrations. In addition, under this setting, each agent can independently train its value function and evaluate its behavior, which makes the algorithm highly robust to team rewards. Moreover, the proposed method constrains the policy to collect data with similar distribution to the expert data during the exploration, which makes policy update more robust. We conduct extensive experiments to validate our proposed method in various MARL environments, the results show that our algorithm outperforms the state-of-the-art algorithms in most scenarios; our method is robust to various reward functions; and the trajectories by our policy is closer to that of the expert policy.

  @article{liu2023edg,
  title = {Expert Demonstrations Guide Reward Decomposition for Multi-Agent Cooperation},
  author = {Weiwei Liu and Wei Jing and Shanqi Liu and Yudi Ruan and Kexin Zhang and Jian Yang and Yong Liu},
  year = 2023,
  journal = {Neural Computing and Applications},
  volume = 35,
  pages = {19847-19863},
  doi = {10.1007/s00521-023-08785-6},
  abstract = {Humans are able to achieve good teamwork through collaboration, since the contributions of the actions from human team members are properly understood by each individual. Therefore, reasonable credit assignment is crucial for multi-agent cooperation. Although existing work uses value decomposition algorithms to mitigate the credit assignment problem, since they decompose the global value function at multi-agents' local value function level, the overall evaluation of the value function can easily lead to approximation errors. Moreover, such strategies are vulnerable to sparse reward scenarios. In this paper, we propose to use expert demonstrations to guide the team reward decomposition at each time step, rather than value decomposition. The proposed method computes the reward ratio of each agent according to the similarity between the state-action pair of the agent and the expert demonstrations. In addition, under this setting, each agent can independently train its value function and evaluate its behavior, which makes the algorithm highly robust to team rewards. Moreover, the proposed method constrains the policy to collect data with similar distribution to the expert data during the exploration, which makes policy update more robust. We conduct extensive experiments to validate our proposed method in various MARL environments, the results show that our algorithm outperforms the state-of-the-art algorithms in most scenarios; our method is robust to various reward functions; and the trajectories by our policy is closer to that of the expert policy.}
  }

• Kexin Zhang, Yong Liu, Yong Gu, Jiadong Wang, and Xiaojun Ruan. Valve Stiction Detection Using Multitimescale Feature Consistent Constraint for Time-Series Data. IEEE-ASME Transactions on Mechatronics, 28:1488-1499, 2023.
  [BibTeX] [Abstract] [DOI] [PDF]

  Using neural networks to build a reliable fault detection model is an attractive topic in industrial processes but remains challenging due to the lack of labeled data. We propose a feature learning approach for industrial time-series data based on self-supervised contrastive learning to tackle this challenge. The proposed approach consists of two components: data transformation and representation learning. The data transformation converts the raw time-series to temporal distance matrices capable of storing temporal and spatial information. The representation learning component uses a convolution-based encoder to encode the temporal distance matrices to embedding representations. The encoder is trained using a new constraint called multitimescale feature consistent constraint. Finally, a fault detection framework for the valve stiction detection task is developed based on the feature learning method. The proposed framework is evaluated not only on an industrial benchmark dataset but also on a hardware experimental system and real industrial environments.

  @article{zhang2023vsd,
  title = {Valve Stiction Detection Using Multitimescale Feature Consistent Constraint for Time-Series Data},
  author = {Kexin Zhang and Yong Liu and Yong Gu and Jiadong Wang and Xiaojun Ruan},
  year = 2023,
  journal = {IEEE-ASME Transactions on Mechatronics},
  volume = 28,
  pages = {1488-1499},
  doi = {10.1109/TMECH.2022.3227960},
  abstract = {Using neural networks to build a reliable fault detection model is an attractive topic in industrial processes but remains challenging due to the lack of labeled data. We propose a feature learning approach for industrial time-series data based on self-supervised contrastive learning to tackle this challenge. The proposed approach consists of two components: data transformation and representation learning. The data transformation converts the raw time-series to temporal distance matrices capable of storing temporal and spatial information. The representation learning component uses a convolution-based encoder to encode the temporal distance matrices to embedding representations. The encoder is trained using a new constraint called multitimescale feature consistent constraint. Finally, a fault detection framework for the valve stiction detection task is developed based on the feature learning method. The proposed framework is evaluated not only on an industrial benchmark dataset but also on a hardware experimental system and real industrial environments.}
  }

• Kexin Zhang, Rongyao Cai, and Yong Liu. Industrial Fault Detection using Contrastive Representation Learning on Time-series Data. In The 22nd World Congress of the International Federation of Automatic Control (IFAC), 2023.
  [BibTeX]

  @inproceedings{zhang2023ifd,
  title = {Industrial Fault Detection using Contrastive Representation Learning on Time-series Data},
  author = {Kexin Zhang and Rongyao Cai and Yong Liu},
  year = 2023,
  booktitle = {The 22nd World Congress of the International Federation of Automatic Control (IFAC)}
  }

• Kexin Zhang, Yong Liu, Yong Gu, Xiaojun Ruan, and Jiadong Wang. Multiple Timescale Feature Learning Strategy for Valve Stiction Detection Based on Convolutional Neural Network. IEEE/ASME Transactions on Mechatronics, 27(3):1478-1488, 2022.
  [BibTeX] [Abstract] [DOI] [PDF]

  This article proposes a valve stiction detection strategy based on a convolutional neural network. Considering the commonly existing characteristics of industrial time-series signals, the strategy is developed to learn features on multiple timescales automatically. Unlike the traditional approaches using hand-crafted features, the proposed strategy can automatically learn representative features on the time-series data collected from industrial control loops. The strategy is composed of two complementary data conversion methods, a mixed feature learning stage and a fusion decision stage, and it has the following merits: 1) the interaction of different pairs of time series can be effectively captured; and 2) the whole process of feature learning is automatic, and no manual feature extraction is needed. The effectiveness of the proposed strategy is evaluated through the comprehensive data, including the International Stiction Data Base, and the real data collected from the real hardware experimental system and the industrial environment. Compared with four traditional methods and three deep-learning-based methods, the experimental results demonstrate that the proposed strategy outperforms the other methods. Besides performance evaluation, we give the implementation procedure of practical application of the proposed strategy and provide the detailed analysis from the perspective of the data conversion methods and the number of timescales.

  @article{zhang2022mtf,
  title = {Multiple Timescale Feature Learning Strategy for Valve Stiction Detection Based on Convolutional Neural Network},
  author = {Kexin Zhang and Yong Liu and Yong Gu and Xiaojun Ruan and Jiadong Wang},
  year = 2022,
  journal = {IEEE/ASME Transactions on Mechatronics},
  volume = {27},
  number = {3},
  pages = {1478-1488},
  doi = {10.1109/TMECH.2021.3087503},
  abstract = {This article proposes a valve stiction detection strategy based on a convolutional neural network. Considering the commonly existing characteristics of industrial time-series signals, the strategy is developed to learn features on multiple timescales automatically. Unlike the traditional approaches using hand-crafted features, the proposed strategy can automatically learn representative features on the time-series data collected from industrial control loops. The strategy is composed of two complementary data conversion methods, a mixed feature learning stage and a fusion decision stage, and it has the following merits: 1) the interaction of different pairs of time series can be effectively captured; and 2) the whole process of feature learning is automatic, and no manual feature extraction is needed. The effectiveness of the proposed strategy is evaluated through the comprehensive data, including the International Stiction Data Base, and the real data collected from the real hardware experimental system and the industrial environment. Compared with four traditional methods and three deep-learning-based methods, the experimental results demonstrate that the proposed strategy outperforms the other methods. Besides performance evaluation, we give the implementation procedure of practical application of the proposed strategy and provide the detailed analysis from the perspective of the data conversion methods and the number of timescales.}
  }

• Kexin Zhang and Yong Liu. Unsupervised Feature Learning with Data Augmentation for Control Valve Stiction Detection. In 2021 IEEE 10th data Driven Control And Learning Systems Conference, pages 1385-1390, 2021.
  [BibTeX] [Abstract] [DOI] [PDF]

  This paper proposes an unsupervised feature learning approach on industrial time series data for detection of valve stiction. Considering the commonly existed characteristics of industrial time series signals and the condition that sometimes massive reliable labeled-data are not available, a new time series data transformation and augmentation method is developed. The transformation stage converts the raw time series signals to 2-D matrices and the augmentation stage increases the diversity of the matrices by performing transformation on different timescales. Then a convolutional autoencoder is used to extract the representative features on the augmented data, these new features are taken as the inputs of the traditional clustering algorithms. Unlike the traditional approaches using hand-crafted features or requiring labeled-data, the proposed strategy can automatically learn features on the time series data collected from industrial control loops without supervision. The effectiveness of the pro-posed approach is evaluated through the International Stiction Data Base (ISDB). Compared with the traditional machine learning methods and deep learning based methods, the experimental results demonstrate that the proposed strategy outperforms the other methods. Besides performance evaluation, we provide a visualization process of feature learning via principal component analy-sis.

  @inproceedings{zhang2021unsupervisedfl,
  title = {Unsupervised Feature Learning with Data Augmentation for Control Valve Stiction Detection},
  author = {Kexin Zhang and Yong Liu},
  year = 2021,
  booktitle = {2021 IEEE 10th data Driven Control And Learning Systems Conference},
  pages = {1385-1390},
  doi = {https://doi.org/10.1109/DDCLS52934.2021.9455535},
  abstract = {This paper proposes an unsupervised feature learning approach on industrial time series data for detection of valve stiction. Considering the commonly existed characteristics of industrial time series signals and the condition that sometimes massive reliable labeled-data are not available, a new time series data transformation and augmentation method is developed. The transformation stage converts the raw time series signals to 2-D matrices and the augmentation stage increases the diversity of the matrices by performing transformation on different timescales. Then a convolutional autoencoder is used to extract the representative features on the augmented data, these new features are taken as the inputs of the traditional clustering algorithms. Unlike the traditional approaches using hand-crafted features or requiring labeled-data, the proposed strategy can automatically learn features on the time series data collected from industrial control loops without supervision. The effectiveness of the pro-posed approach is evaluated through the International Stiction Data Base (ISDB). Compared with the traditional machine learning methods and deep learning based methods, the experimental results demonstrate that the proposed strategy outperforms the other methods. Besides performance evaluation, we provide a visualization process of feature learning via principal component analy-sis.}
  }