A Scalable and Adaptive Tool for Rapid Process Modeling

 

Project Duration: October 2003 – April 2005

Sponsor: Ohio Aerospace Institute

Collaborator: Parker Hannifin, B. F. Goodrich Aerospace

 

Synopsis

The goal of the proposed project is to develop a software tool that will allow engineers to accurately model complex processes in a short time and with low costs.  The need for process modeling exists in many aspects of an industrial organization, including product design, manufacture, and service.  The ability to quickly and accurately model a process will lead to improved quality and productivity, reduced costs, and shortened time to market.

 

Quarter I (10/15/03 – 1/14/04) Activity Summary

1.     Project kickoff.  Because of the long collaborative relation with Parker Hannifin, conversation on identifying a target application started in August 2003, before this project officially started.  After several teleconferences with Michael Benjamin and Ira Rubel.  It was determined that probe array based fuel measurement would be the target application.  Subsequently, Rich Orbell at Parker Hannifin’s Electronic Systems Division provided background information and initial data for model building.  In mid-November 2003, initial results were presented to Rich Orbell and it was determined that additional data needs to be collected, marking the official start of the project.

2.     Preliminary model based on small samples. It was determined that the first task is to investigate the ability of a conventional neural network based model for fuel measurement. It was found that all probes (including redundant probes) need to be used to achieve required measurement accuracy on the training data.  However, the model cannot achieve required accuracy on the testing data, as the testing data contains certain level of noise.  Click here for a report.

3.     Follow-up model based on large samples.  To improve accuracy on noisy data, a larger amount of training samples were generated.  We also investigated the use of different inputs (use pitch and roll in addition to probe and use total wet length of all probes).  The results showed that a conventional neural network based model was not able to achieve the desired accuracy of 0.16% error on average and 0.8% error maximum.  Click here for a report.  The question is whether the desired accuracy can be achieved using Parker Hannifin’s current table look-up method.

Quarter II (1/15/04 – 4/14/04) Activity Summary

  1. Assessment of possible model accuracy.  It is apparent that neural network model accuracy is adversely affected by noise in the data.  Therefore, it is necessary to establish an appropriate expectation of model accuracy with noisy data.  The table look-up method has a high accuracy, so the noisy data should be tested with the table look-up method to establish a benchmark.  Parker Hannifin’s assistance is needed to establish such a benchmark.  The high accuracy of the table look-up method is partially attributed to that fact that it uses the sum of wetted probe length as an input, which tends to cancel out white noise.  Although this cannot be done with a conventional neural network, it inspires the investigation of alternative modeling approaches. 
  2. Investigation of alternative modeling approaches. We have designed a “reverse modeling” approach.  The basic idea is as follows.  The probe readings (wetted length) are a result of the fuel volume and pitch and roll of the tank.  Therefore, a “forward model” can be build with volume, pitch and roll and input and probe readings as output.  As pitch and roll of the tank can be obtained separately (without resorting to triangulation of wetted probes), fuel measurement can be viewed as a “reverse modeling” problem by finding the volume that produces matching probe readings.  This approach can also handle the situation when some probes malfunction and produces erroneous or no readings.  Click here for details of this modeling approach.
  3. Development of reserve modeling approach.  We are currently implementing the reverse modeling approach, which requires two algorithms.  The first algorithm is to allow building the forward model with “don’t care” outputs, namely, the probes have no readings because they are not wetted.  The second algorithm is to search for the input volume that produces “matching” probe readings.  There is considerable flexibility in defining “matching,” e.g., the sum of wetted probe length so white noise can be cancelled out.

 

Quarter III (4/15/04 – 7/14/04) Activity Summary

  1. Implementation of reverse modeling algorithms.  Three different reverse modeling algorithms were developed.  The first one, LMR, is base on the Levenberg-Marquardt optimization method with a regular quadratic objective function (sum of squared error).  The second, GDR, is based on first order gradient descend search with a quadratic objective function.  The third, GDR1, is based on first order gradient descend search with a linear objective function (sum of error).  The rationale for developing the third algorithm is that white noises tend to cancel out when summed.  There three algorithms have been implemented in Microsoft Visual C++.  Click here for a description of these three algorithms.
  2. Brainstorming Additional Modeling Methods.  Preliminary results with reverse modeling algorithms do not indicate superior performance.  This is because reverse modeling is highly dependent on the accuracy of the forward model.  A neural network forward model is difficult to achieve optimal accuracy.  Therefore we brainstormed additional modeling methods.  One is to apply reverse modeling to the table lookup model itself, as it is the most accurate model.  This would be achieved using stochastic optimization techniques such as particle swarm or genetic algorithms.  Another is to develop different neural network models for different flight situations, since it was observed that probe readings behave different in severe flight situation.  The other is to use historical information (progressive decrease of fuel volume during flight) to remove noise from probe readings.  Click here for a summary of additional modeling methods.

 

Quarter IV (7/15/04 – 10/14/04) Activity Summary

  1. Reverse modeling results.  The three reverse modeling algorithms were applied.  For noise free test data, reverse modeling actually produces slightly inferior results compared with the original neural network model.  For noisy test data, the GDR algorithm improves the original neural network model results slightly.  The advantage of reverse modeling is more evident when redundant probes are removed.  However, we should note that none of the reverse modeling algorithms produces results that meet the accuracy requirements, while the original neural network model meets the accuracy requirements when applied to noise free test data.  This may be caused by the lack of access to an accurate forward model (the forward model used for reverse modeling is developed using neural networks and has inherent inaccuracy).  Click here for a summary of reverse modeling results.
  2. Study of model robustness.  Since reverse modeling shows promise when redundant probes are removed, we believe it would functional well in the presence of faulty probes.  Dealing with faulty probes is a strong motivation for starting this project since the table lookup method is very accurate and handles noisy data reasonably well.  Its disadvantage is that it would not produce good results when there are faulty probes.  There was an expectation that a neural network model can deal with faulty probes gracefully.  However, this was found not to be true.  The performance of the neural network model deteriorates quickly even with only two faulty probes.  On the other hand, particle swarm optimization (PSO) proves to be very robust.  Gradient search based reverse modeling can be used to fine tune the result of PSO to achieve slight performance improvement.  Click here for a summary of model robustness study.

 

Quarter V (10/15/04 – 1/14/05) Activity Summary

  1. Comparison of different modeling techniques.  We have explored different modeling techniques and it is time to consolidate the efforts.  Reverse modeling requires the use of pitch and roll, whereas forward modeling may or may not use pitch and roll as input.  Therefore, we consolidated all previous efforts to develop three models; namely, (1) forward model without pitch and roll as input (FMO), (2) forward model with pitch and roll as input (FMI), and (3) reverse model (REM).  We evaluate the performance of these three models with noise-free data and noisy data and under the assumption of the presence of faulty probes.  Click here for a summary of result comparison.
  2. Methods to improve model accuracy and robustness.  Since none of the models meet accuracy requirements when data contains noise, we believe data preprocessing methods must be developed to reduce noise.  One such method is to sample multiple readings from a probe and take the average as model input.  We also believe that Parker’s table look-up method can be modified to deal with faulty probes gracefully.  This requires knowing pitch and roll and constructing a look-up table for each probe.  We will discuss the feasibility of such an approach with Parker engineers.  

 

Quarter VI (1/15/05 – 4/15/05) Activity Summary

  1. Fuel measurement project wrap-up.  We discussed project findings and recommendations with Parker engineers.  Our findings showed that simply applying a neural network modeling approach does not lead to an algorithm that tolerates noise and probe failure.  This supports Parker engineers' skepticism on some competitors' unsubstantiated claims.  Our recommendations are also consistent with Parker engineers' knowledge.  The fuel measurement project was successfully concluded.  Click here for a summary of the project.
  2. Modeling software tool development.  In parallel to this project, we worked with VerTech LLC to apply rapid process modeling in machinery diagnosis and prognosis under a SBIR (Phase I and II) project supported by the National Institute of Standards and Technology.  A software tool, DP, was developed.  An installation package of the software tool is available upon request.  Click here for the instruction manual.