RQ1: To what extent can the prediction model accurately predict the system safety?

The prediction model plays a vital role in our online repair scheme; it determines whether to initiate the repair process and provides guidance for generating repaired actions. Therefore, this RQ aims to investigate whether the constructed prediction model can precisely estimate the future STL sores of the subject CPSs.

Setup

During data collection, we introduce action noises to DRL control policies to overcome the imbalance between safe and unsafe trajectories. Compared to the original policies, the success rates of the noised policies under standard specifications are presented. The results demonstrate an improved balance between safe and unsafe trajectories due to the injected noises. In this work, we collect 5000 trajectories for each task and DRL controller, each consisting of 300 timesteps. These datasets, collected with action noises and standard specifications, are then used to train the prediction models.

To evaluate the accuracy of the prediction model, we divide the collected dataset into training, validation, and testing subsets in a 7:1:2 ratio.  Considering the detection of unsafe actions as a binary classification problem, we utilize four metrics to assess the effectiveness of the prediction model: accuracy, F1-score, mean square error (MSE), and the area under the receiver operating characteristic curves (AUC).  The threshold for the classification is set to 0 for all tasks. While the accuracy and the F1-score measure the precision of the binary classification, MSE and AUC provide insights into the numerical accuracy of the prediction scores.

Result

Table shows the performance of the trained prediction models for each manipulation task with PPO and TRPO control policies. It can be observed that, for all prediction models, the classification accuracy ranges from 0.83 to 0.94, and the F1-score varies from 0.80 to 0.94. Such high values indicate that the prediction model is capable of making accurate predictions, enabling an efficient determination of unsafe actions.