Machine Learning Digital Twin Applied to Hybrid Vehicle Emission Test—A Multimetric Evaluation Approach
DOI:
https://doi.org/10.63385/sriic.v1i2.348Keywords:
Artificial Intelligence, Machine Learning, Dynamic Vehicle Simulation, Vehicle Electrification, Multimetric EvaluationAbstract
This article proposes and illustrates a multimetric evaluation template for digital twins based on machine learning in critical engineering applications using an example as a specific testbed for discussing unified assessment principles. We analyze this concept through the lens of a particular and relevant example: the performance of a hybrid vehicle during homologation tests for transient emissions. In this scenario, ML models must not only optimize efficiency but also ensure strict compliance with environmental regulations in dynamic, operating modes. The case illustrates the complexity that arises when attempting to unify requirements for accuracy, fault tolerance, adaptability, and regulatory compliance, providing a framework for exploring how a unified evaluation system can lead to a more consistent and reliable integration of ML into critical systems. Test data of emission tests on a Hybrid vehicle was used to train a Random Forest model. Different sets of input parameters illustrate some of the capabilities and limitations of using AI. Shapley values were used to discuss some of the AI model characteristics and limitations. Using as input parameters only Vehicle speed, Acceleration, and Battery State of Charge (SoC) allowed the digital twin to achieve R2 0.80. Inclusion of Internal Combustion Engine oil temperature increased the model R2 to 0.97 and curiously changed the ranking of the other input parameters. SoC, which was the most influential in previous cases, went down to almost negligible impact on the model results. Despite reaching a quite high R2 of 0.97, the model can miss important aspects of the physical system without Human expert supervision.
References
[1]International Organization for Standardization, & International Electrotechnical Commission, 2022. ISO/IEC 23053:2022—Framework for Artificial Intelligence (AI) Systems Using Machine Learning (ML). ISO: Geneva, Switzerland. Available from: https://www.iso.org/standard/74438.html
[2]Tabassi, E., 2023. Artificial Intelligence Risk Management Framework (AI RMF 1.0) (No. NIST AI 100-1). National Institute of Standards and Technology (U.S.): Gaithersburg, MD, USA. DOI: https://doi.org/10.6028/NIST.AI.100-1
[3]Raja, V.J., M, D., Solaimalai, G., et al., 2024. Machine Learning Revolutionizing Performance Evaluation: Recent Developments and Breakthroughs. In Proceedings of the 2nd International Conference on Sustainable Computing and Smart Systems (ICSCSS), Coimbatore, India, 10 July 2024; pp. 780–785. DOI: https://doi.org/10.1109/ICSCSS60660.2024.10625103
[4]Anžel, A., Heider, D., Hattab, G., 2023. Interactive polar diagrams for model comparison. Computer Methods and Programs in Biomedicine. 242, 107843. DOI: https://doi.org/10.1016/j.cmpb.2023.107843
[5]Carvalho, J.B.S., Rodriguez, V.J., Torcinovich, A., et al., 2025. Rethinking Robustness in Machine Learning: A Posterior Agreement Approach. arXiv preprint. arXiv:2503.16271. DOI: https://doi.org/10.48550/ARXIV.2503.16271
[6]Hassija, V., Chamola, V., Mahapatra, A., et al., 2024. Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence. Cognitive Computation. 16(1), 45–74. DOI: https://doi.org/10.1007/s12559-023-10179-8
[7]Sewada, R., Jangid, A., Kumar, P., et al., 2023. Explainable Artificial Intelligence (XAI). Journal of Nonlinear Analysis and Optimization. 13(1), 41–47. DOI: https://doi.org/10.36893/JNAO.2022.V13I02.041-047
[8]Mothilal, R.K., Sharma, A., Tan, C., 2020. Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, Barcelona, Spain, 27 January 2020; pp. 607–617. DOI: https://doi.org/10.1145/3351095.3372850
[9]Hanselle, J., Heid, S., Fürnkranz, J., et al., 2024. Probabilistic Scoring Lists for Interpretable Machine Learning. arXiv preprint. arXiv: 2407.21535. DOI: https://doi.org/10.48550/arXiv.2407.21535.
[10]Devarasetty, N., 2024. Optimizing data engineering for AI: improving data quality and preparation for machine learning application. Research and Analysis Journal. 7(3), 1–29. DOI: https://doi.org/10.18535/raj.v7i03.397
[11]Kamrud, A., Borghetti, B., Schubert Kabban, C., 2021. The Effects of Individual Differences, Non-Stationarity, and the Importance of Data Partitioning Decisions for Training and Testing of EEG Cross-Participant Models. Sensors. 21(9), 3225. DOI: https://doi.org/10.3390/s21093225
[12]Ferrara, E., 2023. Fairness and Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, and Mitigation Strategies. Sci. 6(1), 3. DOI: https://doi.org/10.3390/sci6010003
[13]Cloots, A.S., 2019. Blockchain and the Law: The Rule of Code. By Primavera De Filippi and Aaron Wright. The Cambridge Law Journal. 78(1), 213–217. DOI: https://doi.org/10.1017/S0008197319000084
[14]Apicella, A., Isgrò, F., Prevete, R., 2025. Don’t push the button! Exploring data leakage risks in machine learning and transfer learning. Artificial Intelligence Review. 58(11), 339. DOI: https://doi.org/10.1007/s10462-025-11326-3
[15]Wolff, R.F., Moons, K.G.M., Riley, R.D., et al., 2019. PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies. Annals of Internal Medicine. 170(1), 51–58. DOI: https://doi.org/10.7326/M18-1376
[16]Hu, H., Liu, M., Yuan, D., et al., 2017. A block based encoding approach for improving sliding window network coding in wireless networks. In Proceedings of the 3rd IEEE International Conference on Computer and Communications (ICCC), Chengdu, China, December 2017; pp. 300–304. DOI: https://doi.org/10.1109/CompComm.2017.8322560
[17]Jiang, W., 2021. Applications of deep learning in stock market prediction: Recent progress. Expert Systems with Applications. 184, 115537. DOI: https://doi.org/10.1016/j.eswa.2021.115537
[18]Khiari, J., Olaverri-Monreal, C., 2023. Uncertainty-Aware Vehicle Energy Efficiency Prediction Using an Ensemble of Neural Networks. IEEE Intelligent Transportation Systems Magazine. 15(5), 109–119. DOI: https://doi.org/10.1109/MITS.2023.3268032
[19]Tomanik, E., Jimenez-Reyes, A.J., Tomanik, V., et al., 2023. Machine-Learning-Based Digital Twins for Transient Vehicle Cycles and Their Potential for Predicting Fuel Consumption. Vehicles. 5(2), 583–604. DOI: https://doi.org/10.3390/vehicles5020032
[20]Ganesh, A.H., Xu, B., 2022. A review of reinforcement learning based energy management systems for electrified powertrains: Progress, challenge, and potential solution. Renewable and Sustainable Energy Reviews. 154, 111833. DOI: https://doi.org/10.1016/j.rser.2021.111833
[21]SAE International Technical Standard, 2023. Recommended Practice for Measuring the Exhaust Emissions and Fuel Economy of Hybrid-Electric Vehicles, Including Plug-in Hybrid Vehicles. SAE International Technical Standard: Warrendale, PA, USA. DOI: https://doi.org/10.4271/J1711_202302
[22]European Union, 2018. Regulations: Commission Regulation (EU) 2018/1832. Official Journal of the European Union. L 301/1. Available from: https://www.stradalex.eu/en/se_src_publ_leg_eur_jo/toc/leg_eur_jo_3_20181127_301/doc/ojeu_2018.301.01.0001.01
[23]International Council on Clean Transportation (ICCT), 2017. Real-driving Emissions Test Procedure for Exhaust Gas Pollutant Emissions of Cars and Light Commercial Vehicles in Europe. International Council on Clean Transportation (ICCT): Washington, DC, USA. Available from: https://theicct.org/publication/real-driving-emissions-test-procedure-for-exhaust-gas-pollutant-emissions-of-cars-and-light-commercial-vehicles-in-europe/?gad_source=1&gad_campaignid=22639629046&gbraid=0AAAAA_pFlefpgv0teP7NMaKgMc-h4PbHr&gclid=CjwKCAiA-__MBhAKEiwASBmsBIdx60BAmW1OXrawYvufoH2OjTNdDBaHQEReNGxhz7uWr0WwjWzlFhoCmXwQAvD_BwE
[24]Scikit-learn developers, n.d. RandomForestRegressor. Available from: https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html (cited 2 January 2025).
[25]Tomanik, E., Miedviedieva, N., Tomanik, V., et al., 2025. Use of Digital Twins to Analyze and Predict CO2 and Emissions on Hybrid Vehicles. In: Slavinska, O., Danchuk, V., Kunytska, O., et al. (Eds.). Intelligent Transport Systems: Ecology, Safety, Quality, Comfort, Lecture Notes in Networks and Systems. Springer Nature: Cham, Switzerland. pp. 135–147. DOI: https://doi.org/10.1007/978-3-031-87379-9_13
[26]Maria, T.M., Tomanik, E., Ivanytska, A., et al., 2025. Engine Emissions Test Analysis Model Based on Instantaneous OBD Reading and AI. In: Slavinska, O., Danchuk, V., Kunytska, O., et al. (Eds.). Intelligent Transport Systems: Ecology, Safety, Quality, Comfort, Lecture Notes in Networks and Systems. Springer Nature: Cham, Switzerland. pp. 161–171. DOI: https://doi.org/10.1007/978-3-031-87379-9_15
[27]Lundberg, S.M., Lee, S.I., 2017. A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems. Curran Associates, Inc.: Red Hook, NY, USA. Available from: https://www.bibsonomy.org/bibtex/172c147d69a1d43d7b64860b04a3295c6
[28]Argonne National Laboratory, n.d. Downloadable Dynamometer Database. Available from: https://www.anl.gov/es/downloadable-dynamometer-database (cited 3 February 2025).
[29]Toyota Motor Corporation, 2020. RAV4 PRIME RAV4 Plug-in Hybrid—Gasoline-Electric Hybrid Synergy Drive—Hybrid Vehicle Dismantling Manual. Toyota Motor Corporation: Toyota, Japan. Available from: https://content.instructables.com/FSJ/JHT2/LDREAKBB/FSJJHT2LDREAKBB.pdf
[30]Krysmon, S., Claßen, J., Pischinger, S., et al., 2023. RDE Calibration—Evaluating Fundamentals of Clustering Approaches to Support the Calibration Process. Vehicles. 5(2), 404–423. DOI: https://doi.org/10.3390/vehicles5020023
PDF