Intelligent Machine Health Monitoring and Maintenance System

 

Project Duration: October 2001 – March 2003

Sponsor: Ohio Aerospace Institute

Collaborator: Allison Advanced Development Company, B. F. Goodrich Aerospace, and Aircraft Braking Systems

Subcontractor: VerTech LLC

 

Synopsis

The goal of this project is to develop a preliminary intelligent machine health monitoring and maintenance system and demonstrate its applicability and effectiveness via an industrial case study. Some aircraft component failures can lead to consequences ranging from flight delay or cancellation to personal injury, a significant and increasingly visible problem for the traveling public. Technologies to anticipate maintenance and repair requirements will provide the opportunity to schedule maintenance in ways to minimize cost and flight scheduling disruption. Machine health models that can drill down to subassembly or component levels in failure prediction offer the additional opportunity of enabling maintenance to be targeted to specific components, rather than requiring replacement of complex and expensive assemblies.

 

Quarter I (10/1/01 – 12/31/01) Activity Summary

  1. Project kickoff.  The project kickoff meeting was held on November 13 at Allison Advanced Development Company (AADC).  Samuel Huang from the University of Cincinnati (UC) and Bill VerDuin from VerTech LLC made presentations to AADC Managers and Engineers.  The proposed methodology for machine health monitoring and maintenance was introduced.  Case study requirements were discussed.  Mark Rinehart from Rolls Royce Corporation (RRC) introduced their engine condition monitoring (ECM) program and the collaboration with Data Systems & Solutions (DSS) for data collection and analysis. It was decided that a follow-up meeting be held at AADC to determine whether ECM can be an appropriate case study.
  2. Case study determination. A follow-up meeting was held on December 10 at AADC.   Peter Withers from DSS demonstrated the use of COMPASS software tool in ECM.  Details on the ECM data collected by COMPASS were discussed.  It was determined that automatic diagnosis of engine failure mode based on datasets collected by COMPASS is a suitable case study for this project.  Subsequently, RRC provided 13 data samples for model building and 22 for testing.  A preliminary feasibility study was carried out and completed in early January 2002 (see Quarter II Activity Summary).
  3. Modeling technique evaluation.  Professor Samuel Huang and his student Ranganath Kothamasu have previously developed a neural-fuzzy technique called Adaptive Mamdani Fuzzy Model (AMFM) for data driven model building.  The applicability of AMFM for intelligent condition-based maintenance (ICBM) was evaluated.  It was determined that AMFM meets the requirement of ICBM since it is lucid, generalized, and accurate.

 

Kickoff Presentation

COMPASS Overview

AMFM Modeling Technique

 

Quarter II (1/1/02 – 3/31/02) Activity Summary

  1. ECM case study.  Engine performance data, including inter-turbine temperature, fuel flow, shaft speed and vibration, is used to determine the status of the engine.  The status could be Normal or experiencing problems (failure modes) including Turbine Deterioration and Compressor Bleed Leak.  Three linguistic rules were extracted from 13 training data samples and tested in 22 previously not seen data samples.  The diagnosis generated using the three linguistic rules was compared with actual engine conditions provided by RRC.  An accuracy of 89.5% was achieved, and hence established the feasibility of our proposed neural-fuzzy technique for machine health monitoring and maintenance.  This report is proprietary and is available only to the participating collaborators upon request.
  2. Hard turning case study.  While AMFM meets the general requirement of ICBM, its effectiveness needs to be validated through a controllable case study.  The hard turning case study consists of experimental data relating the cutting forces to process parameters such as speed and feed of the cutting operation.  Simulated datasets were developed relating the flank wear and the forces to machine faults such as bearing wear and fixture misalignment.  Artificial noise was generated at different levels and a comparison was made between AMFM and the traditional regression method based on statistics.  AMFM performed significantly better than regression in all the test cases, in terms of missed hits and false alarms.
  3. Initial ICBM system architecture.  Literature review on condition based maintenance and reliability analysis was conducted.  Based on the review, a high level ICBM system architecture was developed.

 

Hard Turning Case Study

ICBM System Architecture

 

 Quarter III (4/1/02 – 6/31/02) Activity Summary

 

1.      Feature Extraction from Time-Series Data.  In machine health monitoring and maintenance, a large number of data samples are collected over time.  It is essential to extract features, such as trends and peaks, from time-series data to identify patterns associated with the status of a machine.  We have reviewed a number of general feature extraction techniques.  We have also realized that customized feature extraction techniques are needed to deal with specific problems.

2.      Engine Condition Monitoring Follow-up Study.  In the initial study, our model achieved an 89.5% accuracy.  In an attempt to improve model accuracy, a follow-up study was conducted.  We used more sophisticated feature extraction techniques and used subtractive clustering for rule extraction.  The model is better than the initial model, yet we are still unable to achieve 100% diagnosis accuracy.  A closer examination of the data showed that in some cases, AADC engineers were not absolutely sure about engine statuses.  There is no doubt that our model can still be improved.  Nonetheless, we believe that our model is very competitive.  We are looking forward to a controlled case study to benchmark our technology with human experts or existing computer systems.  A detailed report of our follow-up study is available upon request, subject to AADC approval.

3.      Condition-based Maintenance Survey. We started a survey on existing machine health monitoring and maintenance research.  A list of researchers in the area has been identified.

 

Feature Extraction Techniques

List of Researchers

 

 Quarter IV (7/1/02 – 9/30/02) Activity Summary

 

1.      Rule extraction.  Our initial rule extraction method requires parameter discretization followed by neural network training.  It is well known that neural network training is more of an art than science, and there is a potential danger of overfitting the training data because of the large number of connections (free parameters).  Therefore, we have developed two other rule extraction methods, one based on decision tree induction and the other based on clustering analysis.

2.      AMFM modeling.  There were a few bugs with our original AMFM (Adaptive Mamdani Fuzzy Model) model implementation in Microsoft Visual C ++, because of the difficulty (mainly visualization) in debugging.  We have revisited the design of AMFM and implemented the model in MATLAB, because of the build-in graphical display capability that facilitates debugging.  Currently, our AMFM model only considers Gaussion type (closed) fuzzy membership function.  We are currently working on implementing Sigmoid type (open left or open right) membership functions.

 

Rule Extraction

AMFM Rule Tuning

 

 Quarter V (10/1/02 – 12/31/02) Activity Summary

  1. Business Survey and Competitive Analysis.  We have conducted a survey on companies that engage in the business of machine health monitoring and maintenance.  It appears that Smart Signal is the leader in terms of capability and technology.  We have compared our modeling technology with those employed by these companies. Our technology covers wider modeling aspects and is more advanced than that of Smart Signal; thus, indicating commercial competitiveness.
  2. Analysis of MIMOSA compliance. MIMOSA (Machinery Information Management Open Systems Alliance) is an industrial consortium that deals with enterprise asset optimization, in which machine health monitoring and maintenance is a critical component.  MIMOSA has developed an information model, CRIS (Common Relational Information Schema), that aims at eliminating the need of custom mapping between different proprietary software systems.  We have reviewed MIMOSA’s schema and it is the same as our ICBM system functions.  We believe that the best approach to achieve 100% MIMOSA compliance is to develop proprietary database based on our modeling technology and then develop MIMOSA gateways.
  3. AMFM Model Improvement.  We have completed the implementation of Sigmoid type membership functions in AMFM.

 

Company Survey

Technology Comparison

MIMOSA Overview

 

Quarter VI (1/1/03 – 3/31/03) Activity Summary

  1. Other Machine Health Monitoring Applications.  We were hoping to conduct a controlled case study provided by DSS to benchmark our technology with human experts and existing computer systems to clearly demonstrate its effectiveness and practical utility.  However, due to the shift of business focus, RRC/AADC was not able to make this happen during the project period.  While the effort of obtaining a controlled case study from DSS is ongoing, we have initiated contacts with other companies (GEAE, Parker Hannifin, Timken, Cincinnati Machine, and Kennametal) including to identify other machine health monitoring applications.  A common issue is data availability.  Although all of the companies regard machine health monitoring as an important problem, only few of them are collecting data systematically.  None of them have a sophisticated data collection and processing system like that of DSS; and thus, not capable of providing the data we need.  Nonetheless, we did identify a power generator engine monitoring application at GEAE.  The challenge is to screen and process large amount of raw data before model development.
  2. Identification of Key Research Issues. Our existing modeling technology (AMFM) requires that the model parameters (both input and output) be clearly identified and the data should contain input/output pairs without missing information.  It turns out that this is quite a strict requirement in real-world applications.  Based on our study on real-world applications, two key research issues are identified.  The first issue is dimensionality reduction to identify the few critical parameters (from hundreds of potential input parameters) that are needed for modeling.  The second issue is treatment of missing data to generate complete input/output pairs of data for model building/tuning.  Literature review and preliminary study on these two issues are currently underway.

 

Summary

            This project was carried out in conjunction with a parallel SBIR (small business innovation research) Phase I project entitled “Development of an Intelligent Condition-Based Maintenance System,” which was awarded to VerTech LLC by the National Institute of Standards and Technology (NIST).  The SBIR project was from August 2001 to February 2002 with a total value of $70K.  By pulling the resources together, we were able to successfully complete two case studies (engine condition monitoring and hard turning) in early February 2002 and submit a Phase II proposal to NIST.   Since we clearly demonstrated the feasibility of intelligent condition-based maintenance, NIST awarded the Phase II proposal from August 2002 to February 2004 with a total value of $300K.  Because this OAI CCRP project was leveraged by the NIST funding, we were able to conduct both theoretical modeling research and practical case studies.  Results of this project are summarized as follows:

·        Modeling Technology: We adopted a knowledge-based modeling approach where IF-THEN rules are used as model elements.  Two techniques for rule extraction from data are developed, namely, decision-tree induction and clustering-based rule extraction.  A proprietary AMFM model is developed for rule tuning and optimization.

·        Application: A hard turning case study (tool wear prediction and identification of bearing wear and fixture misalignment) using simulated dataset has demonstrated the accuracy and noise immunity of our modeling technology; while its practical utility is demonstrated via a real-world aircraft engine fault diagnosis application.  Details of both applications are available upon request.

·        Commercialization Assessment: We have conducted literature-based business survey and competitive analysis.  We also visited a number of companies to assess the commercial viability of machine health monitoring and maintenance service.  In addition to RRC/AADC, GEAE provided power generator engine data and Kennametal provided cutting tool wear data.  Both datasets are been analyzed under the NIST support.  The following companies also provided data: Parker Hannifin, Timken, Cincinnati Machine, Swagelok, and Everady.  However, these data are not suitable for this particular project either because the way data is collected or the application focus.  A number of companies also expressed their interest in machine health monitoring but have no data ready available: Goodrich, Aircraft Braking Systems, and Caterpillar.

·        Future Development: A number of research issues have been identified for future development, including dimensionality reduction, treatment of data with missing elements, and model adaptation to a changing environment.  A couple of proposals have been submitted to federal funding agencies.