Volume 5 Supplement 3
State Space Model with hidden variables for reconstruction of gene regulatory networks
© Wu et al. 2011
Published: 23 December 2011
State Space Model (SSM) is a relatively new approach to inferring gene regulatory networks. It requires less computational time than Dynamic Bayesian Networks (DBN). There are two types of variables in the linear SSM, observed variables and hidden variables. SSM uses an iterative method, namely Expectation-Maximization, to infer regulatory relationships from microarray datasets. The hidden variables cannot be directly observed from experiments. How to determine the number of hidden variables has a significant impact on the accuracy of network inference. In this study, we used SSM to infer Gene regulatory networks (GRNs) from synthetic time series datasets, investigated Bayesian Information Criterion (BIC) and Principle Component Analysis (PCA) approaches to determining the number of hidden variables in SSM, and evaluated the performance of SSM in comparison with DBN.
True GRNs and synthetic gene expression datasets were generated using GeneNetWeaver. Both DBN and linear SSM were used to infer GRNs from the synthetic datasets. The inferred networks were compared with the true networks.
Our results show that inference precision varied with the number of hidden variables. For some regulatory networks, the inference precision of DBN was higher but SSM performed better in other cases. Although the overall performance of the two approaches is compatible, SSM is much faster and capable of inferring much larger networks than DBN.
This study provides useful information in handling the hidden variables and improving the inference precision.
Microarrays can simultaneously measure the expression of thousands of genes. In the past decade or so, many time series experiments have employed microarrays to profile the temporal change of gene expression. For instance, one can retrieve many time-course gene expression datasets from the Gene Expression Omnibus database (http://www.ncbi.nlm.nih.gov/geo/). These datasets usually have much smaller numbers of time points, compared to the large number of genes. Here we focus on how to infer gene regulatory networks (GRNs) from time series microarray datasets.
Any effective GRN inference method has to cope well with the large number of genes and the small number of time points that characterize microarray datasets. During the past few decades, many methods have been developed, such as Dynamic Bayesian Network (DBN) [1, 2] and Probability Boolean Network (PBN) . However, DBN and PBN cannot be used to infer large networks with hundreds of genes due to computational overhead. Thus, there is a need to study different approaches to improving inference accuracy and reducing computational cost.
A State Space Model (SSM) [4–8] has been developed for GRN inference in recent years. It has attracted much attention because it has a much higher computational efficiency and can handle noise well. The variables in SSM can be divided into two groups, hidden variables and observed variables. Observed variables are expression levels of genes measured by microarray experiments. Hidden variables include aspects of the evolution process.
In this study, we investigated the performance of SSM and addressed the effect of the number of hidden variables on inference accuracy. An intuitive way is to let the number of hidden variables equal that of observed variables, but SSM may not be convergent. To make it feasible to infer a large network from a limited number of time points, we need to determine the number of hidden variables in SSM. [4, 6, 7] used Bayesian Information Criterion (BIC),  used cross-validation and [9, 10] used Principal Component Analysis (PCA) to determine the number of hidden variables. These methods give a unique value for the number of hidden variables under their corresponding optimal definitions. However, since we are mostly interested in inference of GRNs, one should use accuracy of inferred GRNs to define the optimal criteria. That is, the optimal number of hidden variables that leads to the highest accuracy. It is found that PCA and BIC approaches do not necessarily produce an optimal number of hidden variables. Instead, simply setting the number of hidden variables may give a better or compatible accuracy in SSM. To evaluate the overall performance of SSM with hidden variables, we inferred a number of GRNs using synthetic datasets with different numbers of genes and time points generated from GeneNetWeaver .
In this section, we briefly present the SSM method and two approaches (BIC and PCA) for determining the number of hidden variables in GRN inference.
State Space Model
Bayesian Information Criterion
P(x t ,y t |θ) is probability given parameter θ; N θ is the number of parameters; N is the number of data points. BIC can be calculated with a given number of hidden variables. The number of hidden variables that has the largest BIC will be adopted as the optimal solution.
Principal Component Analysis
Because the number of time points is usually much smaller than the number of genes, a microarray dataset y t (t = 1,...T) has redundant information. From another aspect of view, all measurements for i-th gene form a vector of length T, g i . g i (i = 1,...l) form a linear space, whose dimension is less than or equal to min(l,T) . Vectors g i and g j (i ≠ j) may not be orthogonal. Here inner product is defined as covariance between those two vectors. One can find a new set of orthogonal bases, z k (k = 1,... min(l,T)), and g i can be expressed as linear combination of z k , since they belong to the same linear space. If one only uses a fraction of new bases, for example, z k (k = 1,...m, m < min(l,T), then g i cannot be fully recovered. However, one can choose the most important z k , to let the error be minimized. This can be done by using PCA [9, 10, 12]. Roughly speaking, the error . λ k is eigenvalue of covariance matrix of dataset g i . If one throws away those bases z k whose λ k are small, then the dimension of microarray dataset is reduced. One must notice that, this method of dimension reduction is approximate due to the small amount of time points. For example, if there are 10 genes with only 1 time point, then one possible way to extract GRN is that, if the expression levels of gene i and j both are large, then one expect there is a regulatory relationship between them. This means that the dimension of linear space of g i (i = 1,...l) is 1, even though the real dimension is not 1. Due to the lack of time points, treating the dimension as 1 is the best way to extract a GRN.
SSM uses the same idea as PCA does . The second equation of (1) contains dimension reduction. The dimension of hidden variables x t is less than y t . BIC [4, 6, 7] and PCA [9, 10] can be used to determine the dimension of x t .
Results and discussion
Two types of synthetic datasets generated by using GeneNetWeaver  were used as test cases in this paper, one for E. coli and the other for yeast. We generated 10 GRNs with 30 genes and 41 time points for each of them. The purpose of generating 10 GRNs is to eliminate errors due to particular network topology or attributes, since a GRN inference algorithm may perform better for some GRNs than for the others.
It is worthwhile to note that when the number of hidden variables is small, some regulations are bidirectional in GRNs obtained by SSM, which means gene i regulates gene j and in the same time gene j regulates gene i. This is because the number of hidden variables in SSM is small (= 2 here).
Determining the number of hidden variables in SSM is important in GRN inference. Our results using synthetic time series gene expression datasets of E. coli and yeast, generated by GeneNetWeaver, show that the existing BIC and PCA approaches may not be able to determine the optimal number of hidden variable in SSM. None of them can lead to a better performance than simply setting a fixed number of hidden variables (between1 and 5). In all the tested cases, the average precision scores of GRNs inferred by SSM are mostly better than or compatible with that of DBN. SSM is much more computationally efficient than DBN, enabling the inference and analysis of larger GRNs.
List of abbreviations used
State Space Model
Dynamic Bayesian Networks
Gene regulatory networks
Bayesian Information Criterion
Principle Component Analysis
Probability Boolean Network.
This work was supported by the US Army Corps of Engineers Environmental Quality Program under contract #W912HZ-08-2-0011 and the NSF EPSCoR project "Modeling and Simulation of Complex Systems" (NSF #EPS-0903787). Permission was granted by the Chief of Engineers to publish this information.
This article has been published as part of BMC Systems Biology Volume 5 Supplement 3, 2011: BIOCOMP 2010 - The 2010 International Conference on Bioinformatics & Computational Biology: Systems Biology. The full contents of the supplement are available online at http://www.biomedcentral.com/1752-0509/5?issue=S3.
- Murphy K, Mian S: Modeling gene expression data using dynamic Bayesian networks. Technical Report, Computer Science Division, University of California, Berkeley, CA. 1999
- Zou M, Conzen SD: A new dynamic Bayesian network (DBN) approach for identifying gene regulatory networks from time course microarray data. Bioinformatics. 2005, 21: 71-79. 10.1093/bioinformatics/bth463.View ArticlePubMed
- Shmulevich I, Dougherty ER, Kim S, Zhang W: Probabilistic Boolean networks: A rule-based uncertainty model for gene regulatory networks. Bioinformatics. 2002, 18 (2): 261-274. 10.1093/bioinformatics/18.2.261.View ArticlePubMed
- Wu FX: Gene regulatory network modelling: a state-space approach. Int J Data Mining and Bioinformatics. 2008, 2 (1): 1-14. 10.1504/IJDMB.2008.016753.View Article
- Rangel C, Angus J, Ghahramani Z, Lioumi M, Sotheran E, Gaiba A, Wild DL, Falciani F: Modeling T-cell activation using gene expression profiling and state space modeling. Bioinformatics. 2004, 20 (9): 1361-1372. 10.1093/bioinformatics/bth093.View ArticlePubMed
- Wu F, Zhang W, Kusalik A: Modeling gene expression from microarray expression data with state-space equations. Pac Symp Biocomput. 2004, 9: 581-592.
- Osamu H, Ryo Y, Seiya I, Rui Y, Tomoyuki H, Charnock-Jones DS, Cristin P, Satoru M: Statistical inference of transcriptional module-based gene networks from time course gene expression profiles by using state space models. Bioinformatics. 2008, 24: 932-942. 10.1093/bioinformatics/btm639.View Article
- Kojima K, Rui Y, Seiya I, Mai Y, Masao N, Ryo Y, Teppei S, Kazuko U, Tomoyuki H, Noriko G, Satoru M: A state space representation of VAR models with sparse learning for dynamic gene networks. Genome Inform. 2009, 22: 56-68.
- Holter NS, Maritan A, Cieplak M, Fedoroff NV, Banavar JR: Dynamic modeling of gene expression data. Proc Natl Acad Sci. U S A. 2001, 98: 1693-1698. 10.1073/pnas.98.4.1693.PubMed CentralView ArticlePubMed
- Raychaudhuri S, Stuart JM, Altman RB: Principal components analysis to summarize microarray experiments: application to sporulation time series. Pac Symp Biocomput. 2000, 455-466.
- Marbach D, Schaffter T, Mattiussi C, Floreano D: Generating realistic in silico gene networks for performance assessment of reverse engineering methods. J Comput Biol. 2009, 16 (2): 229-239. 10.1089/cmb.2008.09TT.View ArticlePubMed
- Bishop CM: Pattern Recognition and Machine Learning. 2006, Springer
- Kalman RE: A new approach to linear filtering and prediction problems. Transaction of the ASME--Journal of Basic Engineering. 1960, 35-45.
- Li P: Inferring gene regulatory networks from time series microarray data. PhD thesis. 2009, The University of Southern Mississippi, School of Computing
- Dempster AP, Laird AN, Rubin DB: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society Series B. 1977, 39 (1): 1-38.
- Li P, Zhang CY, Perkins EJ, Gong P, Deng YP: Comparison of probabilistic boolean network and dynamic bayesian network approaches for inferring gene regulatory networks. BMC Bioinformatics. 2007, 8 (Suppl 7): S13-10.1186/1471-2105-8-S7-S13.PubMed CentralView ArticlePubMed
- Shannon P, Markiel A, Ozier O, Baliga NS, Wang JT, Ramage D, Amin N, Schwikowski B, Ideker T: Cytoscape: a software environment for integrated models of biomolecular interaction networks. Genome Res. 2003, 13: 2498-2504. 10.1101/gr.1239303.PubMed CentralView ArticlePubMed
- Fawcett , Tom : ROC Graphs: Notes and Practical Considerations for Researchers. 2004, Netherlands: Kluwer Academic Publishers
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.