On my 2nd Phd project, I have constructed a complex trait-population model to describe how species evolve and assemble in a community under natural selection. However, people may naturally ask if such complexity is really needed. What is the discrepancy between the model with and without population dynamics and variable trait variance? To answer this question, a model selection/classification should be performed on the data generated under those candidate models. As what we used in parameter inference, an Approximate Bayesian Computation-Sequential Monte Carlo method is able to handle this work. However, a huge computational demanding is the bottle neck. To avoid to waste time on fitting each empirical data set by ABC-SMC, a method that only uses the feature of data sets would be a better choice. This reminds me the existing and fast-developed took, i.e. machine learning. So, from this post on, I would like to share my experience of learning Machine Leaning, how to construct a neural network, how to derive backwards propagation algorithm for leaning parameters of a neural network and how to use tensorflow to simply use machine learning, or even deep learning (a multiple-layer neural network algorithm), to do model selection/classification on the topic I mentioned above.
A fast illustration
Presenting application at first would be concrete for people understand what I am talking about. Here in this simple example, I generated 200K data sets for each of the four models, i.e. Brownian Motion model, Ornstein Uhlenbeck (OU) model, Trait-population model, anti-Ornstein Uhlenbeck model. As a preliminary test, I just adjusted the parameter values for \(\gamma\) and \(\alpha\) to stand for 4 models, i.e. when \(\gamma=0,\alpha=0\) it stands for a Brownian Motion model; when \(\gamma>0,\alpha=0\) it stands for an OU model; when \(\gamma=0,\alpha>0\) it stands for an anti-OU model; when \(\gamma>0,\alpha>0\) it stands for a trait-population evolution (TP) model. I didn’t remove population dynamics from the BM and OU model. Therefore, the data generated by those two processes would be more alike to TP model than by the standard BM and OU model. The logic behind is that if the learning neural network can distinguish TP from the other 3 models, it is able to pick TP out from model pile with standard BM and OU model. Note that the anti-OU model is similar to Drury et al. model in 2018.
When the parameter is nonzero, I simply randomly chose values as I assume we don’t know any prior information from the empirical data.
I used in total around 1000K data sets that are randomly generated under 4 models but labeled with correct model names to train a simply three-layer neural network. It is super fast to train such small network and can be done within few minutes. Note that fitting data via ABC-SMC may coat a few days with 30 iterations and 200K particles for each iteration. Then I generated 100 data sets to feed the trained neural network. The generating models are known in advance. The purpose of this test is to check if the trained network is able to recognize the generating models.
A preliminary result
The result is shown below:
Each chart contains four bars indicating the probability of each model (B: BM; O: OU; T: TP; A: Anti-OU). Color red denotes that the prediction of the network is wrong. Otherwise, the highest bar correctly implies the generating model. The results reveal that BM and OU model are confusing to the trained neural network. This makes sense that because of the normalization of traits the output of these two models show no significant discrepancy, i.e. almost all traits are evenly distributed in the trait space. Only the range of the traits would provide some information, i.e. BM model has wider range for the traits than OU model. But the normalization neglects it. The other two models, TP and Anti-OU are nicely distinguishable from the model pile. This indicates that the trait-population model do have significant pattern to express itself. Thus our complexity plays a role in describing trait patterns.
Future plan
Here in the example, I only exploited a 3-layer neural network but already got a nice result. The issues of confusing BM and OU model may be resolved if a deep neural network is applied. Anyway, the power of machine learning on biology is shown and there are more improvement space waiting for further study.
I do believe that machine learning can be useful in biology. But so far as I know, few literature present this tech on biology field. In fact, data analysis and pattern recognition are also key component in biological research besides modeling. I hope machine learning could attract biologist’s attention and thrive here like what it develops in industry and computer science.
The end
As planed at the beginning, the following posts would be on machine learning. Coming soon!