Knowledge-based Modeling: An Expert Centered Data Driven Approach

Knowledge-based Modeling: An Expert Centered Data Driven Approach

Motivation

The key to solving engineering problems is to develop accurate models. For many complex engineering problems, first principle models cannot be developed effectively. In such cases, the traditional approach is to develop an empirical model based on experimental or historical data through techniques such as multiple linear regression, neural networks, or support vector machine. The drawback is that prior knowledge cannot be effectively incorporated into the model building process. The resultant model is strongly biased towards the data collected and its robustness is questionable when data is sparse. An alternative is to develop a symbolic expert system by interviewing domain experts and convert their knowledge into IF-THEN rules. Since expert systems are not developed based on actual data, they are biased towards the experts’ heuristics and are usually rigid and brittle (adversely affected by noise). In addition, acquisition of domain knowledge is a tedious and time consuming process, commonly known as the knowledge acquisition bottleneck. We believe the integration of knowledge embedded in data and those possessed by experts can lead to a superior modeling approach. This motivates us to develop a knowledge-based modeling approach, where automatic data analysis tools are constructed to assist experts in building and optimizing models.

Goal

The goal is to develop a generic framework for knowledge-based modeling and a set of enabling tools that allow experts to quickly build an accurate and robust model in specific domains. We have selected IF-THEN linguistic rules (originated from fuzzy logic) as a unified knowledge representation scheme because of their compatibility to expert heuristic reasoning and their adaptability to actual data. The task is then two-fold:

(1) Extract easy to understand IF-THEN linguistic rules from dataset so experts can incorporate their knowledge by adding and revising these rules, thus achieve knowledge integration and alleviate the knowledge acquisition bottleneck.

(2) Automatically tune the integrated rules to obtain a knowledge-based model with optimal accuracy and interpretability, thus ensure model robustness and its user acceptance.

Technical Approach

The framework for knowledge-based modeling is shown in Figure 1. The set of enabling tools, including cluster analysis, rule extraction, approximate reasoning, and rule tuning, are collectively maintained by a digital knowledge assistant (DKA). The model building process is as follows. It starts with experts identifying input and output parameters of the problem domain. The DKA then conducts an initial analysis of the collected dataset to characterize input and output parameters in terms of cluster analysis. This allows the experts to visualize the data distribution. They may refine the clusters and provide linguistic terms (such as small, medium, and large) to describe the parameters based on their expertise and the observed clusters. The DKA subsequently extract linguistic IF-THEN rules from the dataset and present them to the experts. The experts may add, delete, or revise these rules to form an integrated knowledge base. Now, the DKA can utilize an approximate reasoning technique (originated from fuzzy inference) to operate on the rules and compare the result to collected data. It then uses non-linear optimization techniques to fine tune these rules based on accuracy and interpretability requirements specified by the experts. The enabling techniques are briefly summarized as follows.

Figure 1. Generic framework for knowledge-based modeling.

Cluster Analysis

The purpose of clustering is to separate a dataset into a number of groups (called clusters) based on some measure of similarity, in order to observe interesting structures of the dataset and draw conclusions. Cluster analysis is a well-established research area where many algorithms are available. Commonly used clustering algorithms include k-means clustering and fuzzy c-means clustering. Theses algorithms require the number of clusters to be determined. A recent algorithm that does not have this restriction is Yager and Filev’s (1994) mountain clustering, which was further improved by Chiu (1995) and termed subtractive clustering. The basic idea of subtractive clustering is as follows. First, a mountain function is established such that a data point will have a higher mountain value if it has more nearby data points. The data point with the highest mountain value is then treated as the first cluster center. Mountain values for all data points are then updated to bring down the mountain value of the first cluster center and that of its nearby points. In this way, the second cluster center (which has the maximum updated mountain value) will be found somewhere else. This procedure repeats until the current maximum mountain value falls below a certain level compared to the original maximum mountain value. It is possible to use a single adjustable parameter, i.e., cluster radius, to control the number of clusters in subtractive clustering.

Rule Extraction

Rule extraction can be done in a number of different ways. A popular method is decision tree induction. The principle is as follows. The root of the decision tree is established and an input parameter, say P, is selected based on a certain criterion (e.g., information gain). Say P is described using n linguistic terms. Then n edges are established, with each edge corresponds to one linguistic term. The original dataset is thus partitioned into n new datasets based on A. If the data samples within a new dataset all have identical outcomes, then stop growing that branch of the tree. Otherwise, establish a new node and select another input parameter to grow the tree. Each leaf in the constructed decision tree would represent a rule. An alternative is to use the result of cluster analysis, namely, represent each cluster with a prototype and treat it as a rule. When this approach is used, care must be taken to control linguistic term/cluster mapping in order to ensure rule interpretability.

Approximate Reasoning

Approximate reasoning, also known as fuzzy inference, operates on qualitative linguistic rules to generate quantitative results. It requires each linguistic term to be quantified using a membership function so that quantitative inputs can be fuzzified to invoke linguistic rules. It also requires a defuzzification mechanism to convert rule inference results to quantitative outputs. Although fuzzy inference follows a standard process (Jang et al. 1997), there are considerable freedom in choosing membership functions and the deffuzification mechanism. To ensure computation efficiency, the weighted average method is used for deffuzification. To introduce greater flexibility in quantifying linguistic terms, we use an intersected sigmoid membership function. It is a unique combination of two sigmoid functions, one open left and the other open right. Compared to the commonly used Gaussian membership function, it can be adjusted to approximate a much wider data distribution.

Rule Tuning

Rule tuning is accomplished through the use of an Adaptive Mamdani Fuzzy Model (AMFM) as shown in Figure 2. Taken a leaf out of neuro-fuzzy modeling, AMFM converts the approximate reasoning process into a five-layer network and map the rules into the network architecture. Membership functions and rules are then automatically adjusted through error back-propagation and non-linear optimization.

Figure 2. Adaptive Mamdani fuzzy model for rule tuning.

In addition to these enabling techniques, we are also working on data processing techniques including data cleansing and dimensionality reduction. Data cleansing refers to filling in missing values for incomplete data samples so they can be included in model building, thus maximizing data utilization. Dimensionality reduction is the determination of critical parameters that are necessary and sufficient for model building, which is critical to improve model accuracy and robustness, especially when data is sparse.

Applications

The knowledge-based modeling approach has been proven successfully in various real-world design and manufacturing applications. These applications are briefly summarized in Table 1.

Table 1. Summary of applications.

Application	Input	Output	Result	Collaborator
Engine Nacelle Forming	Part Geometry, Part Material Type, Die Type	Process Method: Bare Punch, Blow Down, Blow Up	Extracted rules for forming process planning to shorten new operator learning curve from 6 months to 1 month (estimated)
Atomizer Design	Orifice Diameter, Ratio of Length to Width of Slots, Ratio of Spin Chamber to Orifice Diameter	Fuel Flow, Spray Angle, Sauter Mean Diameter	Similar response surfaces compared to those produced by a regression model, with improved R² (2% in average) and F-statistics (68% in average) after data cleansing
Mesh Generation	2-Dimensional Part Geometry	Quadrilateral Multi-Block	A software tool that allows users to automatically generate multi-block for quadrilateral meshes
Seamless Tubing	Disc Speeds, Motor Loads, Temperature, Disc Position, Plug, Reeler, Tube Weight, Tube Length	Process Yield	Reduced yield prediction model parameters by 80% (from 10 to 2) while maintaining a 93% predictive accuracy
Thermal Paint Calibration	Thermal image mapped into R/G/B values	Surface Temperature	Achieved 97.75% accuracy in mapping image to temperature
Temperature Profiling	Oven Power, Plastic Preform Depth	Preform Temperature	Generated temperature profile comparable to that obtained using theoretical model with dramatically reduced model development time

Technical Contact

Dr. Samuel H. Huang, Associate Professor and Director

Intelligent CAM Systems Laboratory

Department of Mechanical, Industrial and Nuclear Engineering

University of Cincinnati

Cincinnati, OH 45221

Phone: (513)556-1154 Fax: (513)556-3390

E-mail: sam.huang@uc.edu

Project Team

Member	Title	Time
John Shi	Post-Doc	January 2002 – present
Ranganath Kothamasu	Ph.D.	September 1999 – present
Kai Hu	M.S.	March 2004 – present
Hao Xing	Ph.D.	September 1998 – August 2003
Saurabh Dwivedi	M.S.	January 2001 – December 2003
Niharika Rapur	M.S.	September 2002 – August 2003
Sumit Maloo	M.S.	January 2001 – December 2001
Nikhil Pujari	M.S.	January 2001 – December 2001
Guruprasad Banbekar	M.S.	September 1999 – August 2001
Yogesh Shiralkar	M.S.	January 2000 – December 2000

Acknowledgement

We would like to thank the Ohio Aerospace Institute (OAI), NASA, and the Ohio Department of Development (ODOD) for supporting the following projects:

A Scalable and Adaptive Tool for Rapid Process Modeling (OAI)
Automated FEA/CFD Hexahedral Mesh Generation Using an Integrated Neural Network/Rule-based Method (OAI)
Parallel Distributed Manufacturing Knowledge Acquisition: A Neural Network/Fuzzy Logic Based Approach, Phase I and II (OAI)
Development of a Knowledge Agent for Intelligent Design and Manufacturing (ODOD through OAI)
Improving Accuracy in Space-based Automatic Target Recognition: An Integrated Neuro-Fuzzy Approach (NASA through the Ohio Space Grant Consortium)

We would also like to thank the following companies for research collaboration through student internships and engineering support for real-world applications.

Goodrich Corporation
Parker Hannifin
Rolls-Royce Corporation
Lorain Tubular
Plastics Technology Inc.

Publications

1. S. Dwivedi, S. H. Huang, J. Shi, and W. H. VerDuin, “Yield Prediction for Seamless Tubing Processes: A Computational Intelligence Approach,” accepted by Computers in Industry.

2. S. H. Huang, “Dimensionality Reduction in Automatic Knowledge Acquisition,” IEEE Transactions on Knowledge and Data Engineering, Vol. 15, No. 6, 2003, pp. 1364-1373.

3. H. Xing, S. H. Huang, and J. Shi, “Rapid Development of Knowledge-based System Via Integrated Knowledge Acquisition,” Artificial Intelligence for Engineering Design, Analysis and Manufacturing, Vol. 17, No. 3, 2003, pp. 221-234.

4. S. H. Huang, R. Kothamasu, Y. C. Shiralkar, and D. Bogstad, “Prediction of Plastic Preform Temperature Profile and Modeling Perspective,” International Journal of Manufacturing Science and Technology, Vol. 4, No. 2, 2003, pp. 56-83.

5. S. H. Huang and H. Xing, “Extract Intelligible and Concise Fuzzy Rules from Neural Networks,” Fuzzy Sets and Systems, Vol. 132, No. 2, 2002, pp. 233-243.

6. S. H. Huang, H. Xing, and M. Benjamin, “Automated Knowledge Acquisition for Design and Manufacturing: The Case of Micromachined Atomizer,” Journal of Intelligent Manufacturing, Vol. 12, No. 4, 2001, pp. 377-391.

7. S. H. Huang, H. Xing, and G. Wang, “Intelligent Classification of Drop Hammer Forming Process Method,” International Journal of Advanced Manufacturing Technology, Vol. 18, No. 2, 2001, pp. 89-97.

8. S. H. Huang and M. R. Endsley, “Providing Understanding of the Behavior of Feedforward Neural Networks,” IEEE Transactions on Systems, Man, and Cybernetics, Vol. 27, No. 3, 1997, pp. 465-474.

9. S. H. Huang, H.-C. Zhang, S. Sun, and H. Li, “Function Approximation and Neural-Fuzzy Approach to Machining Process Selection,” IEEE Transactions on Components, Packaging, and Manufacturing Technology - Part C: Manufacturing, Vol. 19, No. 1, 1996, pp. 9-18.

10. S. H. Huang and H.-C. Zhang, “Neural-Expert Hybrid Approach for Intelligent Manufacturing: A Survey,” Computers in Industry, Vol. 26, No. 2, 1995, pp. 107-126.

11. H.-C. Zhang and S. H. Huang, “Applications of Neural Networks in Manufacturing: A State-of-the-Art Survey,” International Journal of Production Research, Vol. 33, No. 3, 1995, pp. 705-728.

12. H.-C. Zhang and S. H. Huang, “A Fuzzy Approach for Process Plan Selection,” International Journal of Production Research, Vol. 32, No. 6, 1994, pp. 1265-1279.

13. S. H. Huang and H.-C. Zhang, “Artificial Neural Networks in Manufacturing: Concepts, Applications, and Perspectives,” IEEE Transactions on Components, Packaging, and Manufacturing Technology - Part I, Vol. 17, No. 2, 1994, pp. 212-228.

14. S. H. Huang, J. Shi, S. Maloo, and E. Steinthorsson, “Automatic Generation of Quadrilateral Multi-Block Topology for FEA/CFD Applications,” 2002 IEEE International Conference on Industrial Technology, Bangkok, Thailand, December 11-14, 2002.

15. R. Kothamasu, S. H. Huang, and I. Marinescu, “Intelligent Tool Wear Estimation for Hard Turning: Neural-Fuzzy Modeling and Model Evaluation,” 2002 International CIRP Design Seminar, Hong Kong, May 16-18, 2002.

16. Marinescu, R. Felix, R. Kothamasu, S. H. Huang, “Prediction of 3-D Surface Texture Using Neuro-Fuzzy Techniques,” 2002 International CIRP Design Seminar, Hong Kong, May 16-18, 2002.

17. S. H. Huang, R. Kothamasu, and H. Xing, “Automatic Generation of Pragmatic and Intelligible Fuzzy Rules,” IEEE Systems, Man and Cybernetics 2001, Tuscon, AZ, October 8-10, 2001.

18. S. H. Huang, H. Xing, and R. Kothamasu, “An Integrated Approach to Manufacturing Knowledge Acquisition: With Application to Process Planning for Drop Hammer Forming,” IJCAI-01 Workshop on Artificial Intelligence and Manufacturing, Seattle, WA, August 5, 2001.

19. P. Svenmarck, S. H. Huang, and E. P. Blasch, “Transparent Neurofuzzy Models of Human Performance,” Proceedings of the Fifth Annual Conference on Human Interaction with Complex Systems (HICS), April 30-May 2, Beckman Institute, University of Illinois, 2000.

20. S. H. Huang and H. Xing, “Heuristic Manufacturing Knowledge Modeling: A Neural Network/Fuzzy Logic Hybrid Approach,” the 4th International Conference on Frontiers of Design and Manufacturing, pp. 80-90, June 16-19, 2000, Hangzhou, P. R. China.

21. S. H. Huang and H.-C. Zhang, “On the Use of Knowledge-Based Connectionist Model in CAPP systems,” Manufacturing Science and Engineering, PED-Vol. 68-1, ASME, 1994, pp. 65-74.

22. S. H. Huang and H.-C. Zhang, “Neural Networks in Manufacturing,” Robotics and Manufacturing: Proceedings of the Fifth International Symposium on Robotics and Manufacturing, pp. 743-748, August, 1994

23. S. H. Huang and H.-C. Zhang, “Overview of Neural Networks in Manufacturing,” Proceedings of 1993 IEEE/CHMT International Electronics Manufacturing Technology Symposium, Santa Clara, CA, pp. 177-190, October, 1993.

24. S. H. Huang and H.-C. Zhang, “A Neural-Expert Hybrid Knowledge Acquisition Model for Process Planning,” AUTOFACT'93 Conference Proceedings, November 8-11, 1993, Chicago, IL, pp. 25-1 - 25-16.