Volume 5 Supplement 3
The 2010 International Conference on Bioinformatics and Computational Biology (BIOCOMP 2010): Systems Biology
State Space Model with hidden variables for reconstruction of gene regulatory networks
 Xi Wu^{1},
 Peng Li^{2},
 Nan Wang^{1},
 Ping Gong^{3},
 Edward J Perkins^{4},
 Youping Deng^{5} and
 Chaoyang Zhang^{1}Email author
DOI: 10.1186/175205095S3S3
© Wu et al. 2011
Published: 23 December 2011
Abstract
Background
State Space Model (SSM) is a relatively new approach to inferring gene regulatory networks. It requires less computational time than Dynamic Bayesian Networks (DBN). There are two types of variables in the linear SSM, observed variables and hidden variables. SSM uses an iterative method, namely ExpectationMaximization, to infer regulatory relationships from microarray datasets. The hidden variables cannot be directly observed from experiments. How to determine the number of hidden variables has a significant impact on the accuracy of network inference. In this study, we used SSM to infer Gene regulatory networks (GRNs) from synthetic time series datasets, investigated Bayesian Information Criterion (BIC) and Principle Component Analysis (PCA) approaches to determining the number of hidden variables in SSM, and evaluated the performance of SSM in comparison with DBN.
Method
True GRNs and synthetic gene expression datasets were generated using GeneNetWeaver. Both DBN and linear SSM were used to infer GRNs from the synthetic datasets. The inferred networks were compared with the true networks.
Results
Our results show that inference precision varied with the number of hidden variables. For some regulatory networks, the inference precision of DBN was higher but SSM performed better in other cases. Although the overall performance of the two approaches is compatible, SSM is much faster and capable of inferring much larger networks than DBN.
Conclusion
This study provides useful information in handling the hidden variables and improving the inference precision.
Introduction
Microarrays can simultaneously measure the expression of thousands of genes. In the past decade or so, many time series experiments have employed microarrays to profile the temporal change of gene expression. For instance, one can retrieve many timecourse gene expression datasets from the Gene Expression Omnibus database (http://www.ncbi.nlm.nih.gov/geo/). These datasets usually have much smaller numbers of time points, compared to the large number of genes. Here we focus on how to infer gene regulatory networks (GRNs) from time series microarray datasets.
Any effective GRN inference method has to cope well with the large number of genes and the small number of time points that characterize microarray datasets. During the past few decades, many methods have been developed, such as Dynamic Bayesian Network (DBN) [1, 2] and Probability Boolean Network (PBN) [3]. However, DBN and PBN cannot be used to infer large networks with hundreds of genes due to computational overhead. Thus, there is a need to study different approaches to improving inference accuracy and reducing computational cost.
A State Space Model (SSM) [4–8] has been developed for GRN inference in recent years. It has attracted much attention because it has a much higher computational efficiency and can handle noise well. The variables in SSM can be divided into two groups, hidden variables and observed variables. Observed variables are expression levels of genes measured by microarray experiments. Hidden variables include aspects of the evolution process.
In this study, we investigated the performance of SSM and addressed the effect of the number of hidden variables on inference accuracy. An intuitive way is to let the number of hidden variables equal that of observed variables, but SSM may not be convergent. To make it feasible to infer a large network from a limited number of time points, we need to determine the number of hidden variables in SSM. [4, 6, 7] used Bayesian Information Criterion (BIC), [5] used crossvalidation and [9, 10] used Principal Component Analysis (PCA) to determine the number of hidden variables. These methods give a unique value for the number of hidden variables under their corresponding optimal definitions. However, since we are mostly interested in inference of GRNs, one should use accuracy of inferred GRNs to define the optimal criteria. That is, the optimal number of hidden variables that leads to the highest accuracy. It is found that PCA and BIC approaches do not necessarily produce an optimal number of hidden variables. Instead, simply setting the number of hidden variables may give a better or compatible accuracy in SSM. To evaluate the overall performance of SSM with hidden variables, we inferred a number of GRNs using synthetic datasets with different numbers of genes and time points generated from GeneNetWeaver [11].
Methods
In this section, we briefly present the SSM method and two approaches (BIC and PCA) for determining the number of hidden variables in GRN inference.
State Space Model
We used expectationmaximization (EM) [12, 15] to infer parameters in SSM.
Bayesian Information Criterion
P(x_{ t },y_{ t }θ) is probability given parameter θ; N_{ θ } is the number of parameters; N is the number of data points. BIC can be calculated with a given number of hidden variables. The number of hidden variables that has the largest BIC will be adopted as the optimal solution.
Principal Component Analysis
Because the number of time points is usually much smaller than the number of genes, a microarray dataset y_{ t }(t = 1,...T) has redundant information. From another aspect of view, all measurements for ith gene form a vector of length T, g_{ i }. g_{ i }(i = 1,...l) form a linear space, whose dimension is less than or equal to min(l,T) [12]. Vectors g_{ i } and g_{ j }(i ≠ j) may not be orthogonal. Here inner product is defined as covariance between those two vectors. One can find a new set of orthogonal bases, z_{ k }(k = 1,... min(l,T)), and g_{ i } can be expressed as linear combination of z_{ k }, since they belong to the same linear space. If one only uses a fraction of new bases, for example, z_{ k }(k = 1,...m, m < min(l,T), then g_{ i } cannot be fully recovered. However, one can choose the most important z_{ k }, to let the error be minimized. This can be done by using PCA [9, 10, 12]. Roughly speaking, the error $d=\sum _{k=m+1}^{\mathsf{\text{min}}\left(l,T\right)}{\lambda}_{k}$. λ_{ k } is eigenvalue of covariance matrix of dataset g_{ i }. If one throws away those bases z_{ k } whose λ_{ k } are small, then the dimension of microarray dataset is reduced. One must notice that, this method of dimension reduction is approximate due to the small amount of time points. For example, if there are 10 genes with only 1 time point, then one possible way to extract GRN is that, if the expression levels of gene i and j both are large, then one expect there is a regulatory relationship between them. This means that the dimension of linear space of g_{ i }(i = 1,...l) is 1, even though the real dimension is not 1. Due to the lack of time points, treating the dimension as 1 is the best way to extract a GRN.
SSM uses the same idea as PCA does [12]. The second equation of (1) contains dimension reduction. The dimension of hidden variables x_{ t } is less than y_{ t }. BIC [4, 6, 7] and PCA [9, 10] can be used to determine the dimension of x_{ t }.
Results and discussion
Two types of synthetic datasets generated by using GeneNetWeaver [11] were used as test cases in this paper, one for E. coli and the other for yeast. We generated 10 GRNs with 30 genes and 41 time points for each of them. The purpose of generating 10 GRNs is to eliminate errors due to particular network topology or attributes, since a GRN inference algorithm may perform better for some GRNs than for the others.
It is worthwhile to note that when the number of hidden variables is small, some regulations are bidirectional in GRNs obtained by SSM, which means gene i regulates gene j and in the same time gene j regulates gene i. This is because the number of hidden variables in SSM is small (= 2 here).
Conclusions
Determining the number of hidden variables in SSM is important in GRN inference. Our results using synthetic time series gene expression datasets of E. coli and yeast, generated by GeneNetWeaver, show that the existing BIC and PCA approaches may not be able to determine the optimal number of hidden variable in SSM. None of them can lead to a better performance than simply setting a fixed number of hidden variables (between1 and 5). In all the tested cases, the average precision scores of GRNs inferred by SSM are mostly better than or compatible with that of DBN. SSM is much more computationally efficient than DBN, enabling the inference and analysis of larger GRNs.
List of abbreviations used
 SSM:

State Space Model
 DBN:

Dynamic Bayesian Networks
 GRNs:

Gene regulatory networks
 BIC:

Bayesian Information Criterion
 PCA:

Principle Component Analysis
 PBN:

Probability Boolean Network.
Declarations
Acknowledgements
This work was supported by the US Army Corps of Engineers Environmental Quality Program under contract #W912HZ0820011 and the NSF EPSCoR project "Modeling and Simulation of Complex Systems" (NSF #EPS0903787). Permission was granted by the Chief of Engineers to publish this information.
This article has been published as part of BMC Systems Biology Volume 5 Supplement 3, 2011: BIOCOMP 2010  The 2010 International Conference on Bioinformatics & Computational Biology: Systems Biology. The full contents of the supplement are available online at http://www.biomedcentral.com/17520509/5?issue=S3.
Authors’ Affiliations
References
 Murphy K, Mian S: Modeling gene expression data using dynamic Bayesian networks. Technical Report, Computer Science Division, University of California, Berkeley, CA. 1999
 Zou M, Conzen SD: A new dynamic Bayesian network (DBN) approach for identifying gene regulatory networks from time course microarray data. Bioinformatics. 2005, 21: 7179. 10.1093/bioinformatics/bth463.View ArticlePubMed
 Shmulevich I, Dougherty ER, Kim S, Zhang W: Probabilistic Boolean networks: A rulebased uncertainty model for gene regulatory networks. Bioinformatics. 2002, 18 (2): 261274. 10.1093/bioinformatics/18.2.261.View ArticlePubMed
 Wu FX: Gene regulatory network modelling: a statespace approach. Int J Data Mining and Bioinformatics. 2008, 2 (1): 114. 10.1504/IJDMB.2008.016753.View Article
 Rangel C, Angus J, Ghahramani Z, Lioumi M, Sotheran E, Gaiba A, Wild DL, Falciani F: Modeling Tcell activation using gene expression profiling and state space modeling. Bioinformatics. 2004, 20 (9): 13611372. 10.1093/bioinformatics/bth093.View ArticlePubMed
 Wu F, Zhang W, Kusalik A: Modeling gene expression from microarray expression data with statespace equations. Pac Symp Biocomput. 2004, 9: 581592.
 Osamu H, Ryo Y, Seiya I, Rui Y, Tomoyuki H, CharnockJones DS, Cristin P, Satoru M: Statistical inference of transcriptional modulebased gene networks from time course gene expression profiles by using state space models. Bioinformatics. 2008, 24: 932942. 10.1093/bioinformatics/btm639.View Article
 Kojima K, Rui Y, Seiya I, Mai Y, Masao N, Ryo Y, Teppei S, Kazuko U, Tomoyuki H, Noriko G, Satoru M: A state space representation of VAR models with sparse learning for dynamic gene networks. Genome Inform. 2009, 22: 5668.
 Holter NS, Maritan A, Cieplak M, Fedoroff NV, Banavar JR: Dynamic modeling of gene expression data. Proc Natl Acad Sci. U S A. 2001, 98: 16931698. 10.1073/pnas.98.4.1693.PubMed CentralView ArticlePubMed
 Raychaudhuri S, Stuart JM, Altman RB: Principal components analysis to summarize microarray experiments: application to sporulation time series. Pac Symp Biocomput. 2000, 455466.
 Marbach D, Schaffter T, Mattiussi C, Floreano D: Generating realistic in silico gene networks for performance assessment of reverse engineering methods. J Comput Biol. 2009, 16 (2): 229239. 10.1089/cmb.2008.09TT.View ArticlePubMed
 Bishop CM: Pattern Recognition and Machine Learning. 2006, Springer
 Kalman RE: A new approach to linear filtering and prediction problems. Transaction of the ASMEJournal of Basic Engineering. 1960, 3545.
 Li P: Inferring gene regulatory networks from time series microarray data. PhD thesis. 2009, The University of Southern Mississippi, School of Computing
 Dempster AP, Laird AN, Rubin DB: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society Series B. 1977, 39 (1): 138.
 Li P, Zhang CY, Perkins EJ, Gong P, Deng YP: Comparison of probabilistic boolean network and dynamic bayesian network approaches for inferring gene regulatory networks. BMC Bioinformatics. 2007, 8 (Suppl 7): S1310.1186/147121058S7S13.PubMed CentralView ArticlePubMed
 Shannon P, Markiel A, Ozier O, Baliga NS, Wang JT, Ramage D, Amin N, Schwikowski B, Ideker T: Cytoscape: a software environment for integrated models of biomolecular interaction networks. Genome Res. 2003, 13: 24982504. 10.1101/gr.1239303.PubMed CentralView ArticlePubMed
 Fawcett , Tom : ROC Graphs: Notes and Practical Considerations for Researchers. 2004, Netherlands: Kluwer Academic Publishers
Copyright
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.