Stephen Winters-Hilt - Informatics and Machine Learning

1 Cover

2 Title Page

3 Copyright Page

4 Dedication Page

5 Preface

6 1 Introduction 1.1 Data Science: Statistics, Probability, Calculus … Python (or Perl) and Linux 1.2 Informatics and Data Analytics 1.3 FSA‐Based Signal Acquisition and Bioinformatics 1.4 Feature Extraction and Language Analytics 1.5 Feature Extraction and Gene Structure Identification 1.6 Theoretical Foundations for Learning 1.7 Classification and Clustering 1.8 Search 1.9 Stochastic Sequential Analysis (SSA) Protocol (Deep Learning Without NNs) 1.10 Deep Learning using Neural Nets 1.11 Mathematical Specifics and Computational Implementations

7 2 Probabilistic Reasoning and Bioinformatics 2.1 Python Shell Scripting 2.2 Counting, the Enumeration Problem, and Statistics 2.3 From Counts to Frequencies to Probabilities 2.4 Identifying Emergent/Convergent Statistics and Anomalous Statistics 2.5 Statistics, Conditional Probability, and Bayes' Rule 2.6 Emergent Distributions and Series 2.7 Exercises

8 3 Information Entropy and Statistical Measures 3.1 Shannon Entropy, Relative Entropy, Maxent, Mutual Information 3.2 Codon Discovery from Mutual Information Anomaly 3.3 ORF Discovery from Long‐Tail Distribution Anomaly 3.4 Sequential Processes and Markov Models 3.5 Exercises

9 4 Ad Hoc , Ab Initio , and Bootstrap Signal Acquisition Methods 4.1 Signal Acquisition, or Scanning, at Linear Order Time‐Complexity 4.2 Genome Analytics: The Gene‐Finder 4.3 Objective Performance Evaluation: Sensitivity and Specificity 4.4 Signal Analytics: The Time‐Domain Finite State Automaton (tFSA) 4.5 Signal Statistics (Fast): Mean, Variance, and Boxcar Filter 4.6 Signal Spectrum: Nyquist Criterion, Gabor Limit, Power Spectrum 4.7 Exercises

10 5 Text Analytics 5.1 Words 5.2 Phrases – Short (Three Words) 5.3 Phrases – Long (A Line or Sentence) 5.4 Exercises

11 6 Analysis of Sequential Data Using HMMs 6.1 Hidden Markov Models (HMMs) 6.2 Graphical Models for Markov Models and Hidden Markov Models 6.3 Standard HMM Weaknesses and their GHMM Fixes 6.4 Generalized HMMs (GHMMs – “Gems”): Minor Viterbi Variants 6.5 HMM Implementation for Viterbi (in C and Perl) 6.6 Exercises

12 7 Generalized HMMs (GHMMs) 7.1 GHMMs: Maximal Clique for Viterbi and Baum–Welch 7.2 GHMMs: Full Duration Model 7.3 GHMMs: Linear Memory Baum–Welch Algorithm 7.4 GHMMs: Distributable Viterbi and Baum–Welch Algorithms 7.5 Martingales and the Feasibility of Statistical Learning (further details in Appendix) 7.6 Exercises

13 8 Neuromanifolds and the Uniqueness of Relative Entropy8.1 Overview 8.2 Review of Differential Geometry [206, 207] 8.3 Amari’s Dually Flat Formulation [113–115] 8.4 Neuromanifolds [113–115] 8.5 Exercises

14 9 Neural Net Learning and Loss Bounds Analysis 9.1 Brief Introduction to Neural Nets (NNs) 9.2 Variational Learning Formalism and Use in Loss Bounds Analysis 9.3 The “sinh −1(ω)” link algorithm (SA) 9.4 The Loss Bounds Analysis for sinh −1( ω ) 9.5 Exercises

15 10 Classification and Clustering 10.1 The SVM Classifier – An Overview 10.2 Introduction to Classification and Clustering 10.3 Lagrangian Optimization and Structural Risk Minimization (SRM) 10.4 SVM Binary Classifier Implementation 10.5 Kernel Selection and Tuning Metaheuristics 10.6 SVM Multiclass from Decision Tree with SVM Binary Classifiers 10.7 SVM Multiclass Classifier Derivation (Multiple Decision Surface) 10.8 SVM Clustering 10.9 Exercises

16 11 Search Metaheuristics 11.1 Trajectory‐Based Search Metaheuristics 11.2 Population‐Based Search Metaheuristics 11.3 Exercises

17 12 Stochastic Sequential Analysis (SSA) 12.1 HMM and FSA‐Based Methods for Signal Acquisition and Feature Extraction 12.2 The Stochastic Sequential Analysis (SSA) Protocol 12.3 Channel Current Cheminformatics (CCC) Implementation of the Stochastic Sequential Analysis (SSA) Protocol 12.4 SCW for Detector Sensitivity Boosting 12.5 SSA for Deep Learning 12.6 Exercises

18 13 Deep Learning Tools – TensorFlow 13.1 Neural Nets Review 13.2 TensorFlow from Google 13.3 Exercises

19 14 Nanopore Detection – A Case Study 14.1 Standard Apparatus 14.2 Controlling Nanopore Noise Sources and Choice of Aperture 14.3 Length Resolution of Individual DNA Hairpins 14.4 Detection of Single Nucleotide Differences (Large Changes in Structure) 14.5 Blockade Mechanism for 9bphp 14.6 Conformational Kinetics on Model Biomolecules 14.7 Channel Current Cheminformatics 14.8 Channel‐Based Detection Mechanisms 14.9 The NTD Nanoscope 14.10 NTD Biosensing Methods 14.11 Exercises

20 Appendix A: Python and Perl System Programming in LinuxA.1 Getting Linux and Python in a Flash (Drive) A.2 Linux and the Command Shell A.3 Perl Review: I/O, Primitives, String Handling, Regex

21 Appendix B: PhysicsB.1 The Calculus of Variations

22 Appendix C: MathC.1 Martingales [102] C.2 Hoeffding Inequality

23 References

24 Index

25 End User License Agreement

1 Chapter 3 Table 3.1 (tag) Gap sizes, with bin size 100. Table 3.2 (aaa) Gap sizes, with bin size 100.

2 Chapter 5Table 5.1 High frequency word counts from Il principle.Table 5.2 Keyword types (I power; II opportunity; III parties; IV actions).Table 5.3 The three highest frequency words.Table 5.4 The proximate high frequency wordsTable 5.5 High frequency up to first word that is not subjunctive+ or romant...Table 5.6 High frequency words given in terms of six categories: heart, powe...Table 5.7 Sample sentiment table values.Table 5.8 Shakespeare insult kit (internet author anonymous).

3 Chapter 10Table 10.1 Performance comparison table for the different SVM methods.Table 10.2 Sequential chunking using different DNA hairpin datasets.Table 10.3 Multi‐threaded chunking using different DNA hairpin datasetsTable 10.4 Sequential chunking with the Absdiff kernel.Table 10.5 Multi‐threaded chunking with the Absdiff kernel.Table 10.6 Performance comparison of the different SVM methods.

4 Chapter 14Table 14.1 Comparative analysis of the translocation/dwell‐time (T/TD) and n...Table 14.2 Sensitivity limits for detection in the streptavidin‐biosensor mo...

1 Chapter 1 Figure 1.1 A Penrose tiling. A non‐repeating tiling with two shapes of tiles... Figure 1.2 The Viterbi path. (Left) The Viterbi path is recursively defined,... Figure 1.3 Chunking on a dynamic table. Works for a HMM using a simple join ... Figure 1.4 Edge feature enhancement via HMM/EM EVA filter. The filter “proje... Figure 1.5 (Left) The general stochastic sequential analysis flow topology. ...

2 Chapter 2 Figure 2.1 The Norwalk virus genome (the “cruise ship virus”). Figure 2.2 The start of the E. coli genome file, FASTA format. Figure 2.3 The Geometric distribution, P ( X = k ) = (1 − p ) (k−1) p , with... Figure 2.4 The Gaussian distribution, aka Normal, shown with mean zero and v...

3 Chapter 3 Figure 3.1 Codon structure is revealed in the V. cholera genome by mutual in... Figure 3.2 ORF encoding structure is revealed in the V. cholera genome by ga... Figure 3.3 (a) Topology index histograms shown for the V. cholerae CHR. I ge... Figure 3.4 Topology‐index histograms are shown for the Chlamydia trachomatis... Figure 3.5 Hash interpolated Markov model (hIMM) and gap/hash interpolated M...

4 Chapter 4 Figure 4.1 Schematic for the finite state automaton used for acquisition of ... Figure 4.2 Sensitivity (SN) and Specificity (SP). For the predictor evaluato... Figure 4.3 Sensitivity (SN) and Specificity (SP) for other two conventions (... Figure 4.4 FSA with alternating SP:SN optimized tuning. Step 1: Acquire sign... Figure 4.5 Tuning on “start_drop_value for a collection of blockade signals ... Figure 4.6 Robust Spike feature extraction: radiated DNA. A time‐domain FSA ... Figure 4.7 SVM classification results with and without spike analysis. Addin... Figure 4.8 FSA acquisition flowchart.Figure 4.9 Prokaryotic gene structure discovered thus far.Figure 4.10 Two types of “stop” codon.Figure 4.11 Hypothesized splice signal upstream.Figure 4.12 Hypothesized splice signal downstream of stop upstream from true...