Applications of Artificial Neural Networks

307 Pages·2017·5.76 MB·English

Applications of Artificial Neural Networks (ANNs)
in
exploring materials property-property correlations

Xiaoyu Cheng

School of Engineering and Materials Science
Queen Mary, University of London

Submitted in partial fulfilment of the requirements of the Degree of
Doctor of Philosophy at University of London
February 2014
QUEEN MARY, UNIVERSITY OF LONDON
ABSTRACT
PHD THESIS
APPLICATIONS OF ARTIFICIAL NEURAL NETWORKS (ANNS) IN
EXPLORING MATERIALS PROPERTY-PROPERTY CORRELATIONS
Xiaoyu Cheng
The discoveries of materials property-property correlations usually require prior
knowledge or serendipity, the process of which can be time-consuming, costly,
and labour-intensive. On the other hand, artificial neural networks (ANNs) are
intelligent and scalable modelling techniques that have been used extensively to
predict properties from materials’ composition or processing parameters, but are
seldom used in exploring materials property-property correlations. The work
presented in this thesis has employed ANNs combinatorial searches to explore the
correlations of different materials properties, through which, ‘known’ correlations
are verified, and ‘unknown’ correlations are revealed. An evaluation criterion is
proposed and demonstrated to be useful in identifying nontrivial correlations.
The work has also extended the application of ANNs in the fields of data
corrections, property predictions and identifications of variables’ contributions. A
systematic ANN protocol has been developed and tested against the known
correlating equations of elastic properties and the experimental data, and is found
to be reliable and effective to correct suspect data in a complicated situation where
no prior knowledge exists. Moreover, the hardness increments of pure metals due
to HPT are accurately predicted from shear modulus, melting temperature and
Burgers vector. The first two variables are identified to have the largest impacts
on hardening. Finally, a combined ANN-SR (symbolic regression) method is
proposed to yield parsimonious correlating equations by ruling out redundant
variables through the partial derivatives method and the connection weight
approach, which are based on the analysis of the ANNs weight vectors. By
applying this method, two simple equations that are at least as accurate as other
models in providing a rapid estimation of the enthalpies of vaporization for
compounds are obtained.
CONTENTS
ABSTRACT ...................................................................................................................... i
CONTENTS ...................................................................................................................... i
LIST OF TABLE ......................................................................................................... viii
LIST OF FIGURE ........................................................................................................ xii
DECLARATION OF AUTHORSHIP ........................................................................ xv
ACKNOWLEDGEMENTS ......................................................................................... xvi
DEFINITIONS AND ABBREVIATIONS ................................................................ xvii
1. Introduction .............................................................................................................. 1
1.1 Aims and objectives ........................................................................................... 1
1.2 Thesis outline...................................................................................................... 3
1.3 Academic contribution ....................................................................................... 3
1.3.1 Journal article .................................................................................................. 3
1.3.2 Conference ...................................................................................................... 4
1.4 Literature review ................................................................................................ 4
1.4.1 Property correlations of materials ................................................................... 5
1.4.2 Data mining .................................................................................................... 5
1.4.3 Artificial intelligence ...................................................................................... 6
1.4.4 Artificial neural networks (ANNs) ................................................................. 7
1) Introduction to ANNs ..................................................................................... 7
2) Types of ANNs ............................................................................................... 8
3) Learning rules and backpropagation algorithm .............................................. 9
4) Application of ANNs .................................................................................... 10
i. A brief history of ANNs ............................................................................ 10
ii. General applications .................................................................................. 11
iii. Applications in materials science .............................................................. 13
1.5 Capturing property correlations through a combinatorial ANN search ........... 14
1.6 Verification of the elastic properties of the elements ....................................... 16
i
1.6.1 Elementary definitions .................................................................................. 19
1.6.2 Isotropic and anisotropic ............................................................................... 21
1.6.3 The relationship between E, G, K and ν ....................................................... 28
1.6.4 Static and dynamic measurements ................................................................ 31
1) Static approaches .......................................................................................... 31
i. Tension test ............................................................................................... 32
ii. Torsion test ................................................................................................ 33
iii. Flexure test ................................................................................................ 34
2) Dynamic approaches ..................................................................................... 36
i. Wave propagation methods ....................................................................... 37
ii. Vibration methods ..................................................................................... 37
1.7 Capturing materials properties correlations using artificial neural networks: an
example in hardening of pure metals by high pressure torsion ................................... 39
1.8 The discovery of materials properties correlations by artificial neural networks
and symbolic regression ............................................................................................... 41
1.8.1 The enthalpy of vaporization ........................................................................ 43
1.8.2 Genetic programming and symbolic regression ........................................... 44
1.8.3 ANNs to identify the contributions of input variables .................................. 46
2. ANN methodology and configuration ................................................................... 50
2.1 Inputs and outputs of ANNs ............................................................................. 54
2.2 Early stopping ................................................................................................... 55
2.3 Bayesian regularization .................................................................................... 55
2.4 General evaluation criteria ............................................................................... 56
3. Capturing property correlations through a combinatorial ANN search .......... 58
3.1 Introduction ...................................................................................................... 58
3.2 Experiment ....................................................................................................... 58
3.2.1 Data collection .............................................................................................. 58
3.2.2 Input variables and output variables ............................................................. 59
3.2.3 Evaluation criteria ......................................................................................... 60
ii
3.3 Results and discussion ...................................................................................... 63
3.3.1 Top binary order correlations ....................................................................... 64
1) Cohesive energy and heat of vaporization at the normal boiling point ........ 64
2) Atomic weight and specific heat capacity .................................................... 70
3.3.2 Top ternary order correlations ...................................................................... 73
1) Cohesive energy, boiling point and heat of vaporization ............................. 73
2) Heat of vaporization, surface energy and molar volume .............................. 76
3) Shear modulus, bulk modulus, and Poisson's ratio ....................................... 78
3.3.3 Top quaternary order property correlations .................................................. 81
1) Surface energy, thermal conductivity, lattice parameter a and work function
...................................................................................................................... 82
3.4 Conclusion ........................................................................................................ 86
4. Verification of the elastic properties of the elements through ANNs ................ 88
4.1 Introduction ...................................................................................................... 88
4.2 Experiment ....................................................................................................... 88
4.2.1 Data discrepancy in handbooks and databases ............................................. 89
4.2.2 Data pre-treatment ........................................................................................ 89
1) Annotation removed ..................................................................................... 89
2) SI unit conversion ......................................................................................... 90
3) Data distribution information ....................................................................... 90
4.2.3 ANN methodology ........................................................................................ 90
1) ANN constructions ....................................................................................... 94
2) ANNs simulations ......................................................................................... 96
3) Employ ANNs to verify data ...................................................................... 104
4.3 Result and discussion ..................................................................................... 109
4.3.1 Validation of ANNs .................................................................................... 109
1) Valid inputs for ANN constructions ........................................................... 109
2) Correlations captured by ANNs .................................................................. 114
iii
4.3.2 Comparisons of elastic properties predicted by ANNs and the correlating
equations ................................................................................................................. 115
4.3.3 Original experimental value........................................................................ 120
1) Gd and Nd (Inconsistent G) ........................................................................ 121
2) Er and Ir (Inconsistent K) ........................................................................... 121
i. Erbium (Er) ............................................................................................. 121
i. Iridium (Ir) .............................................................................................. 122
3) Li, Pr and Tc (Inconsistent E and G) .......................................................... 124
i. Lithium (Li) ............................................................................................. 124
ii. Praseodymium (Pr) ................................................................................. 125
iii. Technetium (Tc) ...................................................................................... 126
4) Rh (Inconsistent E and K) ........................................................................... 127
5) Cs, Os, Re, Ru, Sc, Tm and Y (Inconsistent K and ν) ................................ 128
i. Cesium (Cs) ............................................................................................. 128
ii. Osmium (Os) and Ruthenium (Ru) ......................................................... 129
iii. Rhenium (Re) .......................................................................................... 131
iv. Scandium (Sc) ......................................................................................... 131
v. Thulium (Tm) .......................................................................................... 132
vi. Yttrium (Y) ............................................................................................. 133
6) Th (Inconsistent E and ν) ............................................................................ 134
7) Ho and Zr (Inconsistent E, K and ν) ........................................................... 135
i. Holmium (Ho) ......................................................................................... 135
ii. Zirconium (Zr) ........................................................................................ 136
8) Hf, Na and Pu (Inconsistent E, G and ν) .................................................... 137
i. Hafnium (Hf) ........................................................................................... 137
ii. Sodium (Na) ............................................................................................ 138
iii. Plutonium (Pu) ........................................................................................ 139
9) Ce, In, K and Tl (Inconsistent E, G and K) ................................................ 140
i. Cerium (Ce) ............................................................................................. 140
iv
ii. Indium (In) .............................................................................................. 141
iii. Potassium (K) .......................................................................................... 142
iv. Thallium (Tl) ........................................................................................... 143
10) Be, Cd, Eu, Ga, La, Lu, Rb, Sm, U and Yb (Inconsistent E, G, K and ν) .. 144
i. Beryllium (Be) ........................................................................................ 144
ii. Cadmium (Cd) ......................................................................................... 145
iii. Europium (Eu) ......................................................................................... 146
iv. Gallium (Ga) ........................................................................................... 147
v. Lanthanum (La) ....................................................................................... 147
vi. Lutetium (Lu) .......................................................................................... 150
vii. Rubidium (Rb) ........................................................................................ 150
viii. Samarium (Sm) ....................................................................................... 152
ix. Uranium (U) ............................................................................................ 152
x. Ytterbium (Yb) ........................................................................................ 153
4.3.4 Factors that influence elastic properties...................................................... 154
1) Theoretical factors ...................................................................................... 154
2) Experimental factors ................................................................................... 155
i. Purity ....................................................................................................... 155
ii. Temperature ............................................................................................ 155
iii. Mechanical processing ............................................................................ 156
iv. Static or dynamic measurements ............................................................. 156
4.4 Conclusion ...................................................................................................... 157
5. Capturing materials properties correlations using artificial neural networks:
an example in hardening of pure metals by high pressure torsion ........................ 159
5.1 Introduction .................................................................................................... 159
5.2 Methodology................................................................................................... 159
5.2.1 Data collection ............................................................................................ 159
5.2.2 The inputs and output ................................................................................. 161
5.2.3 Neural network analysis method ................................................................ 162
v
5.2.4 Knowledge extraction ................................................................................. 163
5.3 Results and discussion .................................................................................... 164
5.3.1 Comparison of ANN curves ....................................................................... 164
5.3.2 Modelling with a limited data supply ......................................................... 169
5.3.3 Factors that affect the accuracy in the prediction ....................................... 171
5.3.4 Underlying physical principle of parameters extracted by ANNs .............. 174
5.3.5 Comparison with the physical model ......................................................... 174
5.3.6 The applicability of the forward selection method ..................................... 176
5.4 Conclusion ...................................................................................................... 176
6. The discovery of materials properties correlations through artificial neural
networks and symbolic regression ............................................................................. 178
6.1 Introduction .................................................................................................... 178
6.2 Experiment ..................................................................................................... 178
6.2.1 The contribution of the different variables in ANNs .................................. 179
1) The 'PaD' method ........................................................................................ 179
2) The 'CW' method ........................................................................................ 180
6.2.2 SR modelling to obtain the mathematical expression ................................. 181
6.3 Results and discussion .................................................................................... 182
6.3.1 ANN combinatorial search ......................................................................... 182
6.3.2 The 'PaD' 'and 'CW' method ....................................................................... 183
6.3.3 The important input variables - Tb, Tc and Pc ............................................. 187
6.3.4 SR model analysis ....................................................................................... 189
1) SR models using all five input variables .................................................... 189
2) Mathematical expression using variables selected by ANNs ..................... 191
6.4 Conclusion ...................................................................................................... 193
7. General conclusion and future work .................................................................. 195
7.1 General conclusion ......................................................................................... 195
7.2 Original contribution of the thesis .................................................................. 197
7.3 Future work .................................................................................................... 199
vi
References .................................................................................................................... 201
Appendix ...................................................................................................................... 231
Appendix I .............................................................................................................. 231
Appendix II ............................................................................................................. 237
Appendix III ........................................................................................................... 239
Appendix IV ........................................................................................................... 241
Appendix V ............................................................................................................ 243
Appendix VI ........................................................................................................... 246
Appendix VII .......................................................................................................... 248
Appendix VIII ........................................................................................................ 251
Appendix IX ........................................................................................................... 255
Appendix X ............................................................................................................ 258
Appendix XI ........................................................................................................... 261
Appendix XII .......................................................................................................... 264
Appendix XIII ........................................................................................................ 266
Appendix XIV ........................................................................................................ 270
Appendix XV ......................................................................................................... 275
Appendix XVI ........................................................................................................ 276
Appendix XVII ....................................................................................................... 277
Appendix XVIII ..................................................................................................... 279

vii
LIST OF TABLE

Table 1-1 Summary of stiffness matrixes with independent elastic constants, and
simplified VRH formulas for the typical pure metals at room
temperature. ........................................................................................... 24
Table 1-2 Relations between the elastic properties. .............................................. 30
Table 1-3 Annotations for the correlating equations listed in Table 1-2. .............. 30
Table 1-4 Relative merits of the dynamic and static approaches. ......................... 36
Table 3-1 The 24 properties used in the ANN combinatorial search. ................... 60
Table 3-2 The binary and ternary order correlations between cohesive energy,
boiling point and heat of vaporization. ................................................. 74
Table 3-3 The binary and ternary order correlations between heat of vaporization,
surface energy and molar volume. ........................................................ 78
Table 3-4 The binary and ternary order correlations between shear modulus, bulk
modulus and Poisson's ratio. ................................................................. 80
Table 3-5 The binary, ternary and quaternary order correlations between work
function, surface energy, thermal conductivity and lattice parameter a
for the 37 elements. ............................................................................... 86
Table 4-1 Unit conversion. .................................................................................... 90
Table 4-2 Systematic methodology for error corrections. ..................................... 91
Table 4-3 Dataset used to build the ANNs includes the consistent values of 5
elements (Co, Dy, Fe, Np, Ta and Tb) and the most common values of
22 elements (Ag, Al, Au, Ba, Bi, Ca, Cr, Cu, Mg, Mn, Mo, Nb, Ni, Pb,
Pd, Pt, Sn, Sr, Ti, V, W and Zn). ........................................................... 95
Table 4-4 A summary of ANN simulations of the elastic properties correlations
that are constituted by E, G, K and ν..................................................... 96
Table 4-5 Correlation for error checking in the source pool. .............................. 104
viii