Yong Chen - Industrial Data Analytics for Diagnosis and Prognosis

Shiyu Zhou

University of Wisconsin – Madison

Yong Chen

University of Iowa

Copyright © 2021 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008.

Limit of Liability/Disclaimer of Warranty:

While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herin may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services please contact our Customer Care Department with the U.S. at 877-762-2974, outside the U.S. at 317-572-3993 or fax 317-572-4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print, however, may not be available in electronic format.

Library of Congress Cataloging-in-Publication Data:

Names: Zhou, Shiyu, 1970- author. | Chen, Yong (Professor of industrial and systems engineering), author.

Title: Industrial data analytics for diagnosis and prognosis : a random effects modelling approach / Shiyu Zhou, Yong Chen.

Description: Hoboken. NJ : John Wiley & Sons, Inc., 2021. | Includes bibliographical references and index.

Identifiers: LCCN 2021000379 (print) | LCCN 2021000380 (ebook) | ISBN 9781119666288 (hardback) | ISBN 9781119666295 (pdf) | ISBN 9781119666301 (epub) | ISBN 9781119666271 (ebook)

Subjects: LCSH: Industrial engineering--Statistical methods. | Industrial management--Mathematics. | Random data (Statistics) | Estimation theory.

Classification: LCC T57.35 .Z56 2021 (print) | LCC T57.35 (ebook) | DDC 658.0072/7--dc23

LC record available at https://lccn.loc.gov/2021000379

LC ebook record available at https://lccn.loc.gov/2021000380

Cover image: © monsitj/ iStock/Getty Images

Cover design by Wiley

Set in 9.5/12.5pt STIX Two Text by Integra Software Services, Pondicherry, India.

To our families :

Yifan and LauraJinghui, Jonathan, and Nathan

1 Cover

2 Title page Industrial Data Analytics for Diagnosis and Prognosis A Random Effects Modelling Approach Shiyu Zhou University of Wisconsin – Madison Yong Chen University of Iowa

3 Copyright

4 Dedication

5 Preface

6 Acknowledgments

7 Acronyms

8 Table of Notation

9 Chapter 1: Introduction 1.1 Background and Motivation1.2 Scope and Organization of the Book1.3 How to Use This BookBibliographic Notes

10 Part 1 Statistical Methods and Foundation for Industrial Data Analytics Chapter 2: Introduction to Data Visualization and Characterization 2.1 Data Visualization2.1.1 Distribution Plots for a Single Variable2.1.2 Plots for Relationship Between Two Variables2.1.3 Plots for More than Two Variables2.2 Summary Statistics2.2.1 Sample Mean, Variance, and Covariance2.2.2 Sample Mean Vector and Sample Covariance Matrix2.2.3 Linear Combination of VariablesBibliographic NotesExercises Chapter 3: Random Vectors and the Multivariate Normal Distribution 3.1 Random Vectors3.2 Density Function and Properties of Multivariate Normal Distribution3.3 Maximum Likelihood Estimation for Multivariate Normal Distribution3.4 Hypothesis Testing on Mean Vectors3.5 Bayesian Inference for Normal DistributionBibliographic NotesExercises Chapter 4: Explaining Covariance Structure: Principal Components 4.1 Introduction to Principal Component Analysis4.1.1 Principal Components for More Than Two Variables4.1.2 PCA with Data Normalization4.1.3 Visualization of Principal Components4.1.4 Number of Principal Components to Retain4.2 Mathematical Formulation of Principal Components4.2.1 Proportion of Variance Explained4.2.2 Principal Components Obtained from the Correlation Matrix4.3 Geometric Interpretation of Principal Components4.3.1 Interpretation Based on Rotation4.3.2 Interpretation Based on Low-Dimensional ApproximationBibliographic NotesExercises Chapter 5: Linear Model for Numerical and Categorical Response Variables 5.1 Numerical Response – Linear Regression Models5.1.1 General Formulation of Linear Regression Model5.1.2 Significance and Interpretation of Regression Coefficients5.1.3 Other Types of Predictors in Linear Models5.2 Estimation and Inferences of Model Parameters for Linear Regression5.2.1 Least Squares Estimation5.2.2 Maximum Likelihood Estimation5.2.3 Variable Selection in Linear Regression5.2.4 Hypothesis Testing5.3 Categorical Response – Logistic Regression Model5.3.1 General Formulation of Logistic Regression Model5.3.2 Significance and Interpretation of Model Coefficients5.3.3 Maximum Likelihood Estimation for Logistic RegressionBibliographic NotesExercises Chapter 6: Linear Mixed Effects Model 6.1 Model Structure6.2 Parameter Estimation for LME Model6.2.1 Maximum Likelihood Estimation Method6.2.2 Distribution-Free Estimation Methods6.3 Hypothesis Testing6.3.1 Testing for Fixed Effects6.3.2 Testing for Variance–Covariance ParametersBibliographic NotesExercises

11 Part 2 Random Effects Approaches for Diagnosis and Prognosis Chapter 7: Diagnosis of Variation Source Using PCA 7.1 Linking Variation Sources to PCA7.2 Diagnosis of Single Variation Source7.3 Diagnosis of Multiple Variation Sources7.4 Data Driven Method for Diagnosing Variation SourcesBibliographic NotesExercises Chapter 8: Diagnosis of Variation Sources Through Random Effects Estimation 8.1 Estimation of Variance Components8.2 Properties of Variation Source Estimators8.3 Performance Comparison of Variance Component EstimatorsBibliographic NotesExercises Chapter 9: Analysis of System Diagnosability 9.1 Diagnosability of Linear Mixed Effects Model9.2 Minimal Diagnosable Class9.3 Measurement System Evaluation Based on System DiagnosabilityBibliographic NotesExercisesAppendix Chapter 10: Prognosis Through Mixed Effects Models for Longitudinal Data 10.1 Mixed Effects Model for Longitudinal Data10.2 Random Effects Estimation and Prediction for an Individual Unit10.3 Estimation of Time-to-Failure Distribution10.4 Mixed Effects Model with Mixture Prior Distribution10.4.1 Mixture Distribution10.4.2 Mixed Effects Model with Mixture Prior for Longitudinal Data10.5 Recursive Estimation of Random Effects Using Kalman Filter10.5.1 Introduction to the Kalman Filter10.5.2 Random Effects Estimation Using the Kalman FilterBiographical NotesExercisesAppendix