• Research & Development

FOR IMMEDIATE RELEASE No. 3774

Comparison between standard AI performance evaluation and formal verification

Comparison between standard AI performance evaluation and formal verification

TOKYO, February 26, 2025 - Mitsubishi Electric Corporation (TOKYO: 6503) announced today that it has developed "rapid formal verification technology for AI," targeting AI models known as decision tree ensembles.1 This technology aims to reduce the risk of AI malfunctions and help realize a society where AI can be used with confidence. Its development is a direct consequence of the development of Mitsubishi Electric's Maisart®.2

To address the rapid advance of AI technology and its increasing use globally, the world's first comprehensive legal framework for AI, the "EU Artificial Intelligence Act," came into effect on August 1, 2024. Countries around the world are establishing similar laws and guidelines to address AI risks, requiring businesses that develop and provide AI to manage these risks appropriately. The reliability of AI is particularly crucial in systems where safety is critical, such as autonomous control systems, power and social infrastructure systems, and cybersecurity systems, due to the significant damage that can be caused by errors.

Typically, AI reliability is evaluated using a finite set of test data not used in the model's training, assessing metrics such as accuracy. However, AI behavior is highly complex and even if the test results are 100% accurate, the risk of errors with untested data cannot be eliminated. To address this issue, a formal verification method has been proposed, which involves setting the expected behavior of AI in advance and rigorously verifying that it operates as expected without omissions. However, formal verification has been limited due to the enormous time required to verify large AI models and the difficulty in determining the priority of addressing the risk of malfunctions.

To address these issues, Mitsubishi Electric has developed a new algorithm aimed at efficiently conducting formal verification on decision tree ensembles, widely used for tasks like numerical data prediction, and an interactive verification tool utilizing this algorithm. By leveraging this technology, AI developers can rapidly cycle through formal verification processes, thereby reducing the risk of AI errors. This enhancement of AI reliability will help realize a society where AI can be used with greater confidence.


  1. 1A method that improves prediction accuracy by combining multiple decision trees, which are models that make predictions based on data partitioned according to specific conditions.
  2. 2Brand name ("Mitsubishi Electric's AI creates the State-of-the-ART in technology") of AI for making devices smarter. Maisart

Inquiry

Media contact

Customer Inquiries

Related articles

  • Research & Development