- Research article
- Open Access
- Published:

# Probing the role of stochasticity in a model of the embryonic stem cell – heterogeneous gene expression and reprogramming efficiency

*BMC Systems Biology*
**volume 6**, Article number: 98 (2012)

## Abstract

### Background

Embryonic stem cells (ESC) have the capacity to self-renew and remain pluripotent, while continuously providing a source of a variety of differentiated cell types. Understanding what governs these properties at the molecular level is crucial for stem cell biology and its application to regenerative medicine. Of particular relevance is to elucidate those molecular interactions which govern the reprogramming of somatic cells into ESC. A computational approach can be used as a framework to explore the dynamics of a simplified network of the ESC with the aim to understand how stem cells differentiate and also how they can be reprogrammed from somatic cells.

### Results

We propose a computational model of the embryonic stem cell network, in which a core set of transcription factors (TFs) interact with each other and are induced by external factors. A stochastic treatment of the network dynamics suggests that NANOG heterogeneity is the deciding factor for the stem cell fate. In particular, our results show that the decision of staying in the ground state or commitment to a differentiated state is fundamentally stochastic, and can be modulated by the addition of external factors (2i/3i media), which have the effect of reducing fluctuations in NANOG expression. Our model also hosts reprogramming of a committed cell into an ESC by over-expressing OCT4. In this context, we recapitulate the important experimental result that reprogramming efficiency peaks when OCT4 is over-expressed within a specific range of values.

### Conclusions

We have demonstrated how a stochastic computational model based upon a simplified network of TFs in ESCs can elucidate several key observed dynamical features. It accounts for (i) the observed heterogeneity of key regulators, (ii) characterizes the ESC under certain external stimuli conditions and (iii) describes the occurrence of transitions from the ESC to the differentiated state. Furthermore, the model (iv) provides a framework for reprogramming from somatic cells and conveys an understanding of reprogramming efficiency as a function of OCT4 over-expression.

## Background

Understanding the molecular networks which give rise to pluripotency in embryonic stem (ES) cells is crucial for among other things developing reprogramming strategies. Recent work has shed light on several key aspects of the underlying network and its interaction with external factors, in particular the chemical media which maintain the cells [1]. The current understanding is that ESCs occupy a multiplicity of sub-states, with stochastic transitions between them. One aim is to understand the molecular interactions that maintain cells in a pluripotent state, destabilize this state leading to commitment, as well as allow a return to the pluripotent state from a committed state. Given the substantial experimental efforts currently underway to understand these mechanisms, a computational systems biology approach seems like a way forward within which such questions could be formulated [2–6]. As in many other biomedical problem areas, a computational approach would here allow diverse experimental results to be absorbed into the formulation of the model, but more importantly, could serve as a hypothesis generator to test mechanisms through further experimentation. The recognition that states of a ES cell are read out by the gene expression of key regulators, has lead to a simple hypothesis regarding the pluripotent nature of the ESC [7]. An ES cell can be in a “ground state”, in which it is neutral to any developmental specification. However, it is possible for the cell to transition to a differentiated state. Here we explore the dynamics of a simplified network model representing key elements of ESC transcription factor and signaling regulators to suggest mechanisms for such a transition state picture.

At the heart of the pluripotency network lies the triad OCT4, SOX2 and NANOG [8–10], where OCT4 and SOX2 act together as a heterodimer regulating several genes including NANOG, OCT4 and SOX2 [11]. There are additional TFs that also impact pluripotency. The exact regulatory mechanisms in the network with impact on pluripotency remain to be fully understood. However, it appears that self-reinforcing mechanisms through feedback of these key regulators upon themselves seem to be instrumental. Interacting with these key components in mice are external factors like Leukemia Inhibiting Factor (LIF), which can substitute for feeders by activating the transcription factor STAT3 that inhibits ES differentiation [12, 13]. Another factor, Bone Morphogenetic Protein (BMP4), has been shown to inhibit the differentiation proteins and thus can be used as a replacement for serum [14]. There are corresponding factors active in humans. The common media for maintaining stem cells in cultures is LIF plus serum or BMP4. It has been shown that serum/BMP4 can be replaced by small molecules which inhibit FGF4 receptor tyrosine kinases and the ERK cascade (2i/3i medium) [15]. The 2i/3i (two or three types of differentiation inhibiting molecules) medium is used successfully to maintain stem cells *in vitro* in combination with or without LIF.

Biochemical systems naturally exhibit stochastic fluctuations due to random interaction processes, gene transcription and translation as well as degradation. Recent studies have explored the role of stochastic fluctuations in a variety of organisms ranging from bacteria to mammalian cells [16, 17]. In ESCs, it was shown that the expression of some transcription factors important for pluripotency are heterogeneous when cells are maintained in the “classical” environment i.e. LIF plus BMP4 or serum. Stochasticity or heterogeneity has been observed in key stem cell TFs such as NANOG [18–20], REX1 [21], STELLA [22]. Based upon these observations, it appears that stem cells exist in a multitude of sub-states, where each sub-state represents a certain multi-distribution of TF concentrations. In particular, NANOG shows more heterogeneity than OCT4 and SOX2 [18, 20]. Cells expressing lower levels of NANOG are more prone to differentiate [18, 23], thereby conferring a stochastic component to the ability of the cell to self-renew. Hence, the state space of ESCs is intricately woven into the heterogeneous gene expression of some of the key regulators of the network.

Underlying the ability of NANOG to act as a “gatekeeper” of pluripotency [24], is the fact that OCT4-SOX2 also induces FGF4, a differentiation promoting growth factor [7]. The ES cell requires OCT4 and SOX2 to maintain it in a pluripotent state, while at the same time pushing it towards differentiation. NANOG is thought to prevent differentiation, and hence when it reaches low levels, the probability to commit increases. How FGF4 fits into this network has so far not been computationally explored. Mouse ESCs can be maintained in a pluripotent state, through introduction of small molecule inhibitors. Ying et al. [15] discovered two different sets of small molecule inhibitors; 3i – FGF receptor inhibitor, Mitigen activated protein (MAP) kinase/ERK kinase - MEK inhibitor and GSK3 inhibitor, 2i – MEK inhibitor and a GSK3 inhibitor. Wray et al. [25] established that the expressions of NANOG and REX1 within the mouse ES cultures under 2i conditions were not heterogeneous i.e. only NANOG high or REX high are present, suggesting the existence of cells in a state that is intrinsically less fluctuating. This could be denoted a true “ground” state, which they suggested is an inherent stable pluripotency network governed by OCT4, SOX2 and NANOG, but, which is perturbed by Erk signaling acting through the FGF receptors.

It follows that a quantitative analysis of network dynamics could improve our understanding of the multiple states of the ESC. Previous purely deterministic studies have explored the dynamics of the OCT4-SOX2-NANOG regulatory network, as well as its role in determining the cell fate, i.e the final lineage: epiblast, trophectoderm and endoderm [26, 27]. However, neither of these computational studies analyzed heterogeneity in NANOG expression. Kalmar et al. [20] suggested by stochastic modeling of a simplified stem cell network based upon observations, how NANOG fluctuations could make the stem cell state transition between multiple states. Their model involved feedbacks, both positive and negative between OCT4 and NANOG which lead to NANOG levels cycling between high and low levels as an excitable system. Subsequently Glauche et al. [28] further studied the nature of such stochastic transitions with two different model scenarios. In one model NANOG, which is induced by OCT4-SOX2 can act as a bistable switch, and can transition between high and low levels. In the other model, which is based upon an activator-repressor mechanism, NANOG can oscillate on a fixed limit cycle, and can recapitulate the observed heterogeneity in NANOG levels. Hence, several types of mechanisms could lead to NANOG heterogeneity. It is also suggested how NANOG can act as a gatekeeper by suppressing any differentiation signals which would ultimately make the cell transition into a differentiated cell. However, in [28], the signal to differentiate is external, and cells therefore cannot differentiate spontaneously as observed.

In this work we build upon these ideas by further analyzing how fluctuations in NANOG play a role in both allowing cells to transition between ES sub-states and then to finally exit irreversibly into a differentiated state. However, this occurs in a spontaneous fashion. Key to our approach, which is different from that of refs. [20, 28], is the development of a self-organized network, in which the pluripotent network governed primarily by OCT4-SOX2-NANOG interacts with a differentiation pathway gene denoted by “G”. Candidates for G are for example GATA6 and SOX17. It is the stochastic dynamics of this network in which several types of feedbacks give rise to the observed stochastic stem cell fate. The noise therefore is internal to the network, with external stimuli controlling the strength of the fluctuations. Hence, stem cells can spontaneously change fate in accordance with observations. This also allows us to answer the second question as to how reprogramming can be simulated in our model. In [29] it has been shown that over-expression of OCT4 can lead to reprogramming a somatic cell to an ESC. However, the efficiency is maximal for the levels of OCT4 within a certain window [30]. Our model can reproduce this result, and we show how the interaction between OCT4, NANOG and the differentiation pathway gene G lead to this result.

## Results and discussion

### A simplified computational model of the ESC

Our simplified ESC network model considers a combination of positive and negative feedbacks between OCT4-SOX2 and NANOG and G. With stochastic simulations we demonstrate the permissive nature of this self-contained network – most cells retain pluripotency except for a fraction that get pushed towards differentiation. This model, is based upon an epigenetic effect by which OCT4 regulates NANOG, is also employed for reprogramming somatic cells into ESC.

The heterodimer OCT4-SOX2 is known to serve as an activator of OCT4, SOX2 and NANOG [11]. As in [28], we simplify the interaction of OCT4 and SOX2 with NANOG as shown in Figure 1. The feedback between NANOG and OCT4-SOX2 must be weak. Otherwise it would be inconsistent with low levels of NANOG and high levels of OCT4 and SOX2 as pointed out in [20, 28]. Hence, we do not explicitly have NANOG inducing OCT4 and SOX2 in contrast to refs. [26, 27]. To describe both the embryonic as well as the differentiated state, we include G in the circuit. One candidate for G is Sox17, which was shown to play a role in the control of differentiation of ESCs into extra embryonic endoderm [31]. SOX17 interferes with the self-renewal program by inhibiting SOX2, OCT4 and NANOG [31]. Another candidate for G is GATA6, which is responsible for endoderm formation and also mutually antagonizes NANOG [27]. In [28], the authors assumed an external signal promoting differentiation. However, in our approach the gene G is regulated by the ESC circuit itself, and hence is part of the network which determines the cell fate. Our circuit also includes the differentiation promoting autocrine growth factor FGF4, which is shown schematically in Figure 1 to repress NANOG [32–34]. It has been suggested that FGF4 acts upstream in the induction of differentiation [32, 35]. The Fgf4 gene is expressed in mouse embryonic stem cells and only OCT-SOX complexes are able to promote its transcriptional activation [36]. Inhibition of FGF4 along with GSK3 consolidates the ESC self-renewal and pluripotency [15, 25]. As in [27], we have assume mutual antagonism between NANOG and the differentiation gene G, as well as activation of G by OCT4-SOX2. As shown in [27], over-expression of OCT4 could either lead to the establishment or loss of the stem cell fate, depending on the level of OCT4. We have also considered epigenetic regulation of NANOG as it has been suggested that OCT4 activates the histone demethylases Jmjd2c, which in turn exposes key pluripotent genes to regulation, among them NANOG [37]. Including this mechanism in the model results in sharpening the non-linearity of the OCT4-NANOG interaction.

### Exploring the ground state of the ESC

#### Commitment – transition from the stem cell state to a differentiated state

We first compute the steady states of the system for different values of LIF using the deterministic rate equations (Eq. 1) for the circuit in Figure 1 with the parameters given in Table 1.

With dynamics resulting from the interactions between G, NANOG and OCT4-SOX2, there are basically two states of the system: (i) the stem cell state, when OCT4-SOX2 and NANOG are ON and G is OFF, and (ii) vice versa for the somatic state.

In the somatic state G is high and both OCT4-SOX2 and NANOG are suppressed and hence OFF. This state remains even when increasing LIF since the model for the NANOG gene regulatory function is based upon a simplified epigenetic mechanism. For Nanog to be activated, the Nanog promoter must be bound by OCT4 along with any of its activators OCT4, NANOG (autoregulation), LIF. To be repressed, Nanog must be bound by OCT4 along with its repressors FGF4 and G.

Adding LIF, has no effect on NANOG if OCT4 is OFF, since LIF cannot access NANOG. However, if initially the cell is in a stem cell state with high OCT4-SOX2, then OCT4-SOX2 exposes NANOG, which allows LIF to induce NANOG. This in turn leads to suppression of G, which finally relieves the suppression on OCT4-SOX2. These sequential negative interactions implement a positive feedback loop between NANOG and OCT4-SOX2 (analogous to a different mechanism suggested in [28]). Additional file 1: Figure S1A displays the two states of the cell. The regulation of NANOG occurs through a feed-forward loop [38], in which OCT4 directly activates NANOG and indirectly represses NANOG through FGF4. Additional file 1: Figure S1B shows that adding 2i/3i to the media leads to suppression of FGF4 (magenta), and hence relieves NANOG from repression.

So far we have described a deterministic approach. However, chemical reactions are necessarily stochastic, and hence protein levels fluctuate in time [39–41]. We assume that all of the stochasticity originates from within the network, i.e internal noise, as it is entirely due to random events of protein production and degradation for each of the molecular components with no external noise [42]. Since this noise is generated by the network itself, it could be regarded as “permissive”, which has been conjectured to be the source of hematopoietic commitment [43, 44]. To study the effects of stochasticity, we used a Gillespie approach where the deterministic equations (see Methods) provide transition rates for a master equation. The latter is simulated by a Monte Carlo procedure to provide the time evolution of the concentration levels [45].

#### Stochastic dynamics under LIF conditions

In Figure 2A, we show the time series of OCT4-SOX2 and NANOG concentrations for a stochastic simulation of Equation 1 with *LIF*=85 for the parameters in Table 1. Not shown are the G and FGF4 time series; G fluctuates around extremely low levels and FGF4 is similar to OCT4-SOX2. Although OCT4-SOX2 remains at a fairly high level, NANOG displays a larger fluctuation. The corresponding distributions obtained from several Monte-Carlo runs (Figure 2C), show a tail for low NANOG levels with a peak at higher levels. OCT4-SOX2 displays less heterogeneity. This recapitulates the observed NANOG heterogeneity [18–20]. NANOG regulation occurs due to the competition between OCT4-SOX2, which directly induces NANOG, and suppression by FGF4, which itself is induced by OCT4-SOX2. This type of regulation implements an incoherent feed-forward loop [38]. It is the delay between the noisy OCT4-SOX2 induction of NANOG and its subsequent suppression through induction of FGF4, which itself is fluctuating, that creates the excess fluctuations observed for NANOG.

It has been shown that NANOG expression fluctuations reaching very low levels lead to irreversible commitment [18, 24]. Hence we have built into our model the possibility of leaving the stem cell state by NANOG interactions with the differentiation gene G. Figure 2E shows NANOG fluctuations from a typical simulation. Should the NANOG expression hit a low level, G is relieved from the suppressive effects of NANOG, and is turned ON. Then G shuts OFF OCT4-SOX2 and hence the pluripotent state is transformed into differentiated one. Before this transition occurs, OCT4-SOX2 is at high levels but NANOG could be either high or low. It is only when NANOG reaches a very low level, through several consecutive degrading events in NANOG or OCT4-SOX2, and/or coupled with increase in FGF4 or G, that the switch to a differentiated state occurs. The above results which suggest the role of increased heterogeneity in NANOG as responsible for the fate of the stem cell, were obtained for the parameter set displayed in Table 1. To show that these results are robust to changes in parameter values we computed the fluctuations in NANOG and compared them with the fluctuations in OCT4, using the Linear Noise Approximation (LNA) [46–48] for a wide range of parameter sets (see Methods). In Figure 3, in each panel, we see the distribution of NANOG and OCT4 fluctuations for random parameter sets, for changes in parameters in increasing order (5*%*,15*%* and 50% respectively). For each distribution in parameter space, in the majority of the cases, we see that the highest fluctuations occur in NANOG expression. However, there are cases marked by the oval *A*, in the middle and last subplots, where NANOG and OCT4 fluctuations are extremely low. These represent those cases where the state of the cell is in the differentiated state, and hence the fluctuations in *G* would be highest. In the last subplot, the oval *B* represents those cases where the parameter sets corresponded to: weak NANOG auto-regulation, strong suppression of NANOG by FGF4, G and weak suppression of OCT4 by G. This therefore resulted in higher noise in OCT4 than NANOG, since the latter was strongly suppressed, at the same time G was unable to fully turn OCT4 off. However, the above results indicate that NANOG in most cases experiences the highest fluctuations for a wide range of parameters, thereby supporting its role as the “gatekeeper” of the stem cell state.

The pluripotent state has high levels of OCT4-SOX2 which are less heterogeneous than those of NANOG. The continuum of NANOG values spans both, high and low NANOG values. In [28] the NANOG distribution resulted in a bimodal one, due to a bistable switch-like mechanism. A critical point is that NANOG functions as a gatekeeper regardless of its exact distribution – at low values it is unable to repress G and hence causes a transition. Our simulations are consistent with the experimental observation that although LIF-BMP4 maintains ESCs, a low amount of differentiated cells are nevertheless produced. Simulations also show that increasing LIF improves the maintenance of stem cell cultures, i.e. the mean time that a cell, which is initialized as a stem cell, remains a stem cell increases with LIF value (see Figure 2F).

#### Stochastic dynamics under 2i/3i conditions

Recently, it was shown that ESCs can be maintained in 2i/3i [25] media, with the interesting result that heterogeneity in NANOG is lost. Our model assumes that the effect of small molecules in the 2i/3i medium is to suppress FGF4. This would relieve the suppression on NANOG. As it is shown in Additional file 1: Figure S1B, the system now exhibits a higher level of NANOG.

Figure 2B and Figure 2D show time series and distributions of OCT4-SOX2 and NANOG concentrations under 2i/3i conditions. They both fluctuate at high levels, with lower NANOG heterogeneity. Although LIF is not present, we assume that the stem cell state was initialized with G low and NANOG and OCT4-SOX2 high. Suppression of FGF4 leads to higher induction of NANOG and hence increased positive feedback between OCT4-SOX2 and NANOG through G, which ensures that their levels remain high. Hence, our simulations agree with the experimental observation of loss of NANOG heterogeneity with cells cultured in 2i/3i media. In Additional file 2: Figure S2 we display the mean and standard deviation of NANOG fluctuations using the LNA (see Methods). The figure shows that increasing 2i/3i while increasing the mean levels of NANOG decrease its fluctuations, while still being higher than fluctuations in OCT4. As mentioned earlier, although OCT4-SOX2 maintains pluripotency, it also induces FGF4, which pushes cells to differentiate. However, since FGF4 receptor signaling and GSK3 are inhibited in 2i/3i media, NANOG is not repressed (see Figure 1) and hence the NANOG high state is observed. This would be the true “ground state”, which requires no feeders or serum, except the small molecule inhibitors, which prevent differentiation as well as provide improved bio-synthetic environment for cell growth.

### Reprogramming – transition from somatic to iPS cells

Ectopic expression of the pluripotency transcription factors OCT4, SOX2, KLF4 enables the transition from somatic cells to iPS cells (ES like cells) [49]. One notes that NANOG is not required for reprogramming despite its hub role in the architecture. Reprogramming with only these three factors is inefficient. Hence, substantial efforts are made to overcome this drawback. Initially, over-expression of c-MYC was used but the addition of this factor increased not only the efficiency of reprogramming but also the tumorigenicity of the cells [49]. Recently, it has been shown that c-MYC can be replaced by GLIS1, which does not have the same tumorigenic effect [29]. However, the reprogramming process still remains inefficient and more understanding of the process on the molecular level is needed.

Our minimal dynamical model elucidates the reprogramming process when only OCT4, SOX2 and KLF4 are over-expressed and identifies the obstacles to overcome: The differentiation gene G antagonizes OCT4 and NANOG, and since it feeds back positively upon itself, once ON, it ensures that OCT4-SOX2 and NANOG are OFF. When OCT4-SOX2 is OFF, NANOG cannot be induced since OCT4 is unable to fulfill its epigenetic role of exposing the NANOG promoter for transcription. Hence, NANOG stays OFF. NANOG is also repressed by FGF4, which in this case would be low, since its inducer OCT4-SOX2 is OFF. Hence, over-expression of OCT4 is the key.

#### Deterministic analysis

The parameter *α* in Equation 1 governs the OCT4 over-expression. In Figure 4A, the bifurcation diagram shows how the network components change with increasing *α* keeping the other parameters fixed. Three regions can be identified, approximately given by:

(i) *α*< 0.001, (ii) 0.001 <*α*> 0.2, (iii) *α*> 0.2. When going from (i) to (ii) OCT4 exposes NANOG and the high value of LIF (KLF4 over-expression [50, 51]) induces NANOG, raising its expression to a sufficiently high level. The antagonism between NANOG and G, reduces G to low levels. This removes the G repression of OCT4-SOX2, which further induces NANOG. This implements the positive feedback loop, which makes the transition from the somatic state to the stem cell state. We note that with OCT4-SOX2 at high levels, FGF4 is induced, which does repress NANOG. However, this suppression is not strong enough to counter the induction from *LIF*, to switch it OFF. In addition, with 2i/3i media present, this effect can be significantly reduced.

As OCT4-SOX2 over-expression is further increased and reaches *α*≃0.2 (iii) another bifurcation occurs. Now, a new state emerges – a differentiated state in which NANOG can be at low levels while G is high. This new state is possible because over-expression of OCT4 leads to a relatively slow increase in G, as it can also be induced by OCT4 in the model (it is slow because it is also suppressed by NANOG which itself is strongly induced by OCT4). Once OCT4 crosses a certain threshold, G can overcome NANOG, leading to a differentiated state.

Our deterministic model analysis indicates that the stem cell circuit activation must be conditioned by OCT4-SOX2 over-expression. It is known that over-expression of OCT4 is mandatory for obtaining iPS cells in the laboratory [29]. However, too large an over-expression leads to a differentiated state [52, 53]. Hence OCT4 has to be within a set of thresholds to achieve reprogramming [30]. It should be pointed out though that the latter result was obtained for human ESC.

#### Stochastic simulations

In Figure 4B we present the results of a corresponding stochastic simulation when the reprogramming was successful. Initially, the differentiation gene G has high expression whereas NANOG and OCT4-SOX2 have low expression values – the circuit is in the somatic state. Over-expression of OCT4-SOX2 with an optimal dosage *α*≃0.05 ensures that NANOG is strongly activated by OCT4 and hence the stem cell circuit switches to a stem cell state.

A major challenge is to increase the efficiency of the reprogramming process. We used our model to study the variation of reprogramming efficiency when the value of OCT4-SOX2 over-expression (*α*) is varied. Figure 4C shows the success rates obtained from multiple Monte Carlo simulations where the value of *α* was varied in the [0,0.4] range. Our results suggest that the dosage of reprogramming transcription factors has an important impact on reprogramming efficiency. This has been experimentally shown in a study on optimization of iPS cells generation [30]. We identified an optimal range for OCT4 added expression value (*α*) to be [0.001,0.1]. Such a success interval should be considered when reprogramming is conducted under LIF and BMP4 medium conditions. If the values of reprogramming factor expression is lower than the values in the reprogramming success range then reprogramming does not take place as OCT4 is not significantly expressed and it does not strongly induce NANOG. If the values of OCT4 are too high then they would correspond to values that lead to endoderm differentiation and reprogramming is not successful. Figure 4C shows a 10-fold decrease of the percentage of obtained iPS cells for values of *α* outside success intervals.

Our results show that the stem cell medium where the somatic cells are maintained after transduction also plays an important role in reprogramming efficiency. When reprogramming is successful, the differentiation gene G is OFF while NANOG and OCT4-SOX2 are at high values. The latter induces FGF4 which represses NANOG. The NANOG suppression by FGF4 influences negatively the reprogramming outcome. Thus, repression of FGF4 should have a positive impact on reprogramming efficiency. Indeed, when increasing 2i/3i concentration, an increase in iPS cell generation efficiency is observed (see Figure 4C). The percentage of iPS cells in this study represents the outcome from a minimalistic model and there must be additional factors not considered here which might modify the percentages. However, such factors would equally influence every scenario under consideration.

Nevertheless, our results demonstrate that setting the degrees of over-expression and choosing the iPS cell medium should be considered for optimizing reprogramming efficiency.

For completeness we performed similar analyses for a modified network topology without the differentiation gene G (see Additional file 3: Figure S3 and Additional file 4).

## Conclusions

Our computational model of the transcriptional dynamics of the embryonic stem cell suggests mechanisms in the simplified network feedback structure which allow cells to make a stochastic decision to exit from a stem cell state to a differentiated one. Such an event is random and occurs due to the internal noise of network components. In particular, we explicitly showed how NANOG heterogeneity enables such transitions. NANOG integrates several noisy signals. OCT4 both directly activates NANOG, as well as suppresses it through FGF4. When NANOG falls below a certain threshold, G gets activated, leading to shutdown of NANOG and OCT4. FGF4 can be suppressed by the 2i/3i media which leads to reduction of NANOG heterogeneity (specifically at higher levels) and hence to stability of the stem cell state, i.e the “ground state”. Our model could explain how the absence of the 2i/3i media, can result in the experimentally observed “leakage” to differentiated cells even under ideal culture conditions, since stochastic transitions of NANOG to relatively low levels can occur, in this case. The spontaneous commitment picture emerging from our model studies is consistent with the “permissive” scenarios suggested in the context of hematopoiesis [43, 44]. One might speculate that this ESC property allows cells to form tissues in the natural environment of the embryo through a low rate of regulated differentiation events. Finally we studied the reprogramming scenario of somatic cells due to OCT4 over-expression. Our model was able to explain why reprogramming efficiency is biphasic with respect to OCT4 levels. Once reprogramming occurs, the external stimuli provide optimal conditions to maintain for the stem cell state.

Our simplified model could be expanded as more links in this network are explored. For example, recent work [54, 55] suggests that NANOG is epigenetically modified by Ezh2, and as discussed in [56], this could have interesting consequences for a model seeking to describe NANOG fluctuations. It is expected that future experiments will discover additional network componentsand external media implicated to govern stem cell fate and reprogramming, which could be included into our current model. We have not explored the consequences of external noise due to the environmental signals: LIF and 2i/3i, which will be explored in another work. One area of immediate interest is to immerse our single cell stochastic dynamics in a spatial context of growing and dividing cells [57] with the aim to understand how noise in gene expression couples with mechanics and cell fate in the living embryo.

## Methods

### Model ingredients

NANOG is induced by OCT4-SOX2.

NANOG dimerizes and regulates itself positively [58].

The differentiation gene G positively auto-regulates itself and is repressed by NANOG.

LIF induces NANOG, presumably through Klf4 via the Stat3 pathway.

G is induced by OCT4-SOX2 and suppresses both NANOG and OCT4-SOX2.

Even though FGF4 is a growth factor, it is induced by OCT4-SOX2 [36]. Furthermore, FGF4 signaling was shown to suppress NANOG.

The 2i/3i medium has the effect of suppressing FGF4.

### Network dynamics

For the circuit in Figure 1, we obtain the following set of differential equations from a thermodynamic approach [59–63]. The equations describe the behavior of NANOG, OCT4-SOX2, FGF4 and differentiation gene G (SOX17), with concentration levels denoted by [N], [OS], [FGF] and [G]. The concentrations of LIF and small molecules in the 2i/3i medium are denoted as *LIF* and*I*_{3}respectively.

In [37], an epigenetic effect was implicated by which OCT4 regulates the NANOG region by regulating the histone demethylases Jmjd2c. Here we implement such an effect for NANOG by assuming that that all TF’s which can bind to the Nanog promoter, do so only when the OCT4-SOX2 heterodimer [OS] is first bound to it. This functional form is motivated by the need to have OCT4-SOX2 make NANOG available for transcription. The parameter values used for the simulations are displayed in Table 1. Using the above deterministic equations we can obtain their steady state values as a function of the parameters. We also use the reaction rates from Equation 1 to write down a master equation, which has been simulated using the Gillespie algorithm to obtain the results in Figures 2 and 4.

We have performed Linear Noise Approximation analysis to prove the robustness of our results, as described below.

### Robustness analysis for NANOG fluctuations using the LNA

A second order expansion of the master equation, obtained from the transition rates in Equation 1, is called as the linear noise approximation (LNA) [46–48]. The assumption is that at steady state each network component fluctuates about its mean level, given by solving Equation 1, and is described by a Gaussian distribution. The fluctuations are described by a covariance matrix *C*. The diagonal components of *C*, describe the variances in each component, and the off diagonal components describe the cross correlations between the various species fluctuations. *C* is obtained at steady state by solving the Lyapunov equation given by,

where *J* is the Jacobian matrix, and *D* the effective diffusion matrix, which is obtained from Equation 1. We compute *C*, for a given parameter set and obtain the standard deviations for NANOG, OCT4 etc. This is then repeated for 500 randomly generated parameter sets. Each randomly generated parameter set is obtained by varying each of the parameters within a uniform distribution around the fiducial parameter set in Table 1 by ±5%, 15% and 50%. We did not vary the common degradation parameter *γ*, since the major effect would only lead to rescaling of time.

The bifurcation analysis was performed using JDesigner [64], Oscill8 [65]. Matlab (The Mathworks) was used to solve the differential equations, stochastic Gillespie simulations and the covariances of the network components using the LNA.

## References

- 1.
Nichols J, Smith A: The origin and identity of embryonic stem cells. Development. 2011, 138 (1): 3-8. 10.1242/dev.050831.

- 2.
Peltier J, Schaffer DV: Systems biology approaches to understanding stem cell fate choice. Iet Syst Biol. 2010, 4 (1): 1-11. 10.1049/iet-syb.2009.0011.

- 3.
MacArthur BD, Ma’ayan A, Lemischka IR: Systems biology of stem cell fate and cellular reprogramming. Nat Rev Mol Cell Biol. 2009, 10 (10): 672-681.

- 4.
Enver T, Pera M, Peterson C, Andrews PW: Stem Cell States, Fates, and the Rules of Attraction. Cell Stem Cell. 2009, 4 (5): 387-397. 10.1016/j.stem.2009.04.011.

- 5.
Huang S: Systems biology of stem cells: three useful perspectives to help overcome the paradigm of linear pathways. Philos Trans R Soc Lond B Biol Sci. 2011, 366 (1575): 2247-59. 10.1098/rstb.2011.0008.

- 6.
Artyomov MN Meissner A Chakraborty AK: A Model for Genetic and Epigenetic Regulatory Networks Identifies Rare Pathways for Transcription Factor Induced Pluripotency. PLoS Comput Biol. 2010, 6 (5): e1000785-10.1371/journal.pcbi.1000785.

- 7.
Silva J, Smith A: Capturing pluripotency. Cell. 2008, 132 (4): 532-6. 10.1016/j.cell.2008.02.006.

- 8.
Mitsui K, Tokuzawa Y, Itoh H, Segawa K, Murakami M, Takahashi K, Maruyama M, Maeda M, Yamanaka S: The homeoprotein Nanog is required for maintenance of pluripotency in mouse epiblast and ES cells. Cell. 2003, 113 (5): 631-42. 10.1016/S0092-8674(03)00393-3.

- 9.
Boyer L, Lee T, Cole M, Johnstone S, Levine S, Zucker J, Guenther M, Kumar R, Murray H, Jenner R, Gifford D, Melton D, Jaenisch R, Young R: Core transcriptional regulatory circuitry in human embryonic stem cells. Cell. 2005, 122 (6): 947-56. 10.1016/j.cell.2005.08.020.

- 10.
Niwa H: How is pluripotency determined and maintained?. Development. 2007, 134 (4): 635-46. 10.1242/dev.02787.

- 11.
Chew J, Loh Y, Zhang W, Chen X, Tam W, Yeap L, Li P, Ang Y, Lim B, Robson P, Ng H: Reciprocal transcriptional regulation of Pou5f1 and Sox2 via the Oct4/Sox2 complex in embryonic stem cells. Mol Cell Biol. 2005, 25 (14): 6031-46. 10.1128/MCB.25.14.6031-6046.2005.

- 12.
Smith A, Heath J, Donaldson D, Wong G, Moreau J, Stahl M, Rogers D: Inhibition of pluripotential embryonic stem cell differentiation by purified polypeptides. Nature. 1988, 336: 688-690. 10.1038/336688a0.

- 13.
Smith A: Embryo-derived stem cells: Of mice and men. Annu Rev Cell Dev Biol. 2001, 17: 435-462. 10.1146/annurev.cellbio.17.1.435.

- 14.
Ying Q, Nichols J, Chambers I, Smith A: BMP induction of Id proteins suppresses differentiation and sustains embryonic stem cell self-renewal in collaboration with STAT3. Cell. 2003, 115 (3): 281-92. 10.1016/S0092-8674(03)00847-X.

- 15.
Ying Q, Wray J, Nichols J, Batlle-Morera L, Doble B, Woodgett J, Cohen P, Smith A: The ground state of embryonic stem cell self-renewal. Nature. 2008, 453: 1027-1032.

- 16.
Balazsi G, van Oudenaarden, Collins JJ: Cellular Decision Making and Biological Noise: From Microbes to Mammals. Cell. 2011, 144 (6): 910-925. 10.1016/j.cell.2011.01.030.

- 17.
Eldar A, Elowitz MB: Functional roles for noise in genetic circuits. Nature. 2010, 467 (7312): 167-73. 10.1038/nature09326.

- 18.
Chambers I, Silva J, Colby D, Nichols J, Nijmeijer B, Robertson M, Vrana J, Jones K, Grotewold L, Smith A: Nanog safeguards pluripotency and mediates germline development. Nature. 2007, 450: 1230-1234. 10.1038/nature06403.

- 19.
Singh AM, Hamazaki T, Hankowski KE, Terada N: A Heterogeneous Expression Pattern for Nanog in Embryonic Stem Cells. Cell Stem Cells. 2007, 25: 2534-2542.

- 20.
Kalmar T, Lim C, Hayward P, Muoz-Descalzo S, Nichols J, Garcia-Ojalvo J, Martinez Arias A: Regulated Fluctuations in Nanog Expression Mediate Cell Fate Decisions in Embryonic Stem Cells. PLoS Biol. 2009, 7 (7): e1000149-10.1371/journal.pbio.1000149.

- 21.
Toyooka Y, Shimosato D, Murakami K, Takahashi K, Niwa H: Identification and characterization of subpopulations in undifferentiated ES cell culture. Development. 2008, 135: 909-918. 10.1242/dev.017400.

- 22.
Hayashi K, Chuva de SousaLopesS: Dynamic Equilibrium and Heterogeneity of Mouse Pluripotent Stem Cells with Distinct Functional and Epigenetic States. Cell Stem Cell. 2008, 3 (4): 391-401. 10.1016/j.stem.2008.07.027.

- 23.
Osorno R, Chambers I: Transcription factor heterogeneity and epiblast pluripotency. Phil Trans R Soc B. 2011, 366: 2230-2237. 10.1098/rstb.2011.0043.

- 24.
Silva J, Nichols J, Theunissen T, Guo G, van Oosten, Barrandon O, Wray SJ, amd Yamanaka: Nanog Is the Gateway to the Pluripotent Ground State. Cell. 2009, 138 (4): 722-737. 10.1016/j.cell.2009.07.039.

- 25.
Wray J, Kalkan T, Smith AG: The ground state of pluripotency. Biochem Soc Trans. 2010, 38: 1027-1032. 10.1042/BST0381027.

- 26.
Chickarmane V, Troein C, Nuber UA, Sauro H, Peterson C: Transcriptional Dynamics of the Embryonic Stem Cell Switch. PLoS Comput Biol. 2006, 2 (9): e123-10.1371/journal.pcbi.0020123.

- 27.
Chickarmane V, Peterson C: A computational model for understanding stem cell, trophectoderm and endoderm lineage determination. PLoS One. 2008, 3 (10): e3478-10.1371/journal.pone.0003478.

- 28.
Glauche I, Herberg M, Roeder I: Nanog Variability and Pluripotency Regulation of Embryonic Stem Cells - Insights from a Mathematical Model Analysis. PLoS One. 2010, 5 (6): e11238-10.1371/journal.pone.0011238.

- 29.
Maekawa M, Yamaguchi K, Nakamura T, Shibukawa R, Kodanaka I, Ichisaka T, Kawamura Y, Mochizuki H, Goshima N, Yamanaka S: Direct reprogramming of somatic cells is promoted by maternal transcription factor Glis1. Nature. 2011, 474: 225-229. 10.1038/nature10106.

- 30.
Papapetrou EP, Tomishimac MJ, Chambers SM, Micae Y, Reed E, Menona J, Tabara V, Mog Q, Studer L, Sadelain M: Stoichiometric and temporal requirements of Oct4,Sox2, Klf4, and c-Myc expression for efficient human iPSC induction and differentiation. Proc Natl Acad Sci U S A. 2009, 31: 12759-112764.

- 31.
Niakan K, Ji H, Maehr R, Vokes S, Rodolfa K, Sherwood R, Yamaki M, Dimos J, Chen A, Melton D, McMahon A, Eggan K: Sox17 promotes differentiation in mouse embryonic stem cells by directly regulating extraembryonic gene expression and indirectly antagonizing self-renewal. Genes Dev. 2010, 24: 312-326. 10.1101/gad.1833510.

- 32.
Kunath T, Saba-El-Leil MK, Almousailleakh M, Wray J, Meloche S, Smith A: FGF stimulation of the Erk1/2 signalling cascade triggers transition of pluripotent embryonic stem cells from self-renewal to lineage commitment. Development. 2007, 134: 2895-2902. 10.1242/dev.02880.

- 33.
Masui S: Pluripotency maintenance mechanism of embryonic stem cells and reprogramming. Int J Hematol. 2010, 91: 360-372. 10.1007/s12185-010-0517-9.

- 34.
Lanner F, Rossant J: The role of FGF/Erk signaling in pluripotent cells. Development. 2010, 137 (20): 3351-60. 10.1242/dev.050146.

- 35.
Stavridis MP, Lunn JS, Collins BJ, Storey KG: A discrete period of FGF-induced Erk1/2 signalling is required for vertebrate neural specification. Development. 2007, 134: 2889-2894. 10.1242/dev.02858.

- 36.
Yuan H, Corbi N, Basilico C, Dailey L: Developmental-specific activity of the FGF-4 enhancer requires the synergistic action of Sox2 and Oct-3. Genes Dev. 1995, 9: 2635-2645. 10.1101/gad.9.21.2635.

- 37.
Loh YH, Zhang W, Chen X, George J, Ng HH: Jmjd1a and Jmjd2c histone H3 Lys 9 demethylases regulate self-renewal in embryonic stem cells. Genes and Dev. 2007, 21: 2545-2557. 10.1101/gad.1588207.

- 38.
Mangan S, Alon U: Structure and function of the feed-forward loop network motif. Proc Natl Acad Sci U S A. 2003, 100: 11980-11985. 10.1073/pnas.2133841100.

- 39.
Kaern M, Elston TC, Blake WJ, Collins JJ: Stochasticity in gene expression: from theories to phenotypes. Nat Rev Genet. 2005, 6 (6): 451-64. 10.1038/nrg1615.

- 40.
Raj A, van Oudenaarden A: Nature, nurture, or chance: stochastic gene expression and its consequences. Cell. 2008, 135 (2): 216-26. 10.1016/j.cell.2008.09.050.

- 41.
Shahrezaei V, Swain PS: The stochastic nature of biochemical networks. Curr Opin Biotechnol. 2008, 19 (4): 369-74. 10.1016/j.copbio.2008.06.011.

- 42.
Swain PS, Elowitz MB, Siggia ED: Intrinsic and extrinsic contributions to stochasticity in gene expression. Proc Natl Acad Sci U S A. 2008, 99 (20): 12795-800.

- 43.
Enver T, Heyworth CM, Dexter TM: Do Stem Cells Play Dice?. Blood. 1997, 92: 2348-351.

- 44.
Graf T, Enver T: Forcing cells to change lineages. Nature. 2009, 462: 587-594. 10.1038/nature08533.

- 45.
Gillespie DT: Exact Stochastic Simulation of Coupled Chemical Reactions. J Physical Chem. 1977, 81 (25): 2340-2361. 10.1021/j100540a008.

- 46.
Kampen NV: Stochastic Processes in Physics and Chemistry. Institute of Theoretical Physics, University of Utrecht,. 1992, North Holland, The Netherlands

- 47.
Elf J, Ehrenberg M: Fast Evaluation of Fluctuations in Biochemical Networks With the Linear Noise Approximation. Genome Res. 2003, 13 (11): 2475-2484. 10.1101/gr.1196503.

- 48.
Paulsson J: Summing up the noise in gene networks. Nature. 2004, 427: 415-418. 10.1038/nature02257.

- 49.
Takahashi K, Yamanaka S: Induction of Pluripotent Stem Cells from Mouse Embryonic and Adult Fibroblast Cultures by Defined Factors. Cell. 2006, 126 (4): 663-676. 10.1016/j.cell.2006.07.024.

- 50.
Bourillot P, Aksoy I, Schreiber V, Wianny F, Schulz H, Hummel O, Hubner N, Savatier P: Novel STAT3 target genes exert distinct roles in the inhibition of mesoderm and endoderm differentiation in cooperation with Nanog. Stem Cells. 2009, 27 (8): 1760-71. 10.1002/stem.110.

- 51.
Bourillot PY, Savatier P: Krüppel-like transcription factors and control of pluripotency. BMC Biol. 2010, 8: 125-10.1186/1741-7007-8-125.

- 52.
Niwa H, Miyazaki J, Smith AG: Quantitative expression of Oct-3/4 defines differentiation, dedifferentiation or self-renewal of ES cells. Nat Genet. 2000, 24 (4): 372-376. 10.1038/74199.

- 53.
Okita K, Yamanaka S: Induced pluripotent stem cells: opportunities and challenges. Phil Trans R Soc. 2011, 366: 2198-2207. 10.1098/rstb.2011.0016.

- 54.
Villasante A, Piazzolla D, Li H, Gomez-Lopez G, Djabali M, Serrano M: Epigenetic regulation of Nanog expression by Ezh2 in pluripotent stem cells. Cell Cycle. 2011, 10 (9): 1488-98. 10.4161/cc.10.9.15658.

- 55.
Hankowski KE, Terada N: An Ezh way to turn off Nanog. Cell Cycle. 2011, 10 (14): 2253-4. 10.4161/cc.10.14.16388.

- 56.
Herberg M, Roeder I: Epigenetic Nanog regulation and the role of functional heterogeneity. Cell Cycle. 2011, 14: 2252-2253.

- 57.
Krupinski P, Chickarmane V, Peterson C: Simulating the Mammalian Blastocyst - Molecular and Mechanical Interactions Pattern the Embryo. PLoS Comput Biol. 2011, 7 (5): e1001128-10.1371/journal.pcbi.1001128.

- 58.
Wang J, Levasseur D, Orkin S: Requirement of Nanog dimerization for stem cell self-renewal and pluripotency. Proc Natl Acad Sci U S A. 2008, 105 (17): 6326-31. 10.1073/pnas.0802288105.

- 59.
Shea M, Ackers G: The OR control system of bacteriophage λ. A physical chemical model for gene regulation. J Mol Biol. 1985, 181: 211-230. 10.1016/0022-2836(85)90086-5.

- 60.
Buchler N, Gerland U, Hwa T: On schemes of combinatorial transcription logic. Proc Natl Acad Sci U S A. 2003, 100: 5136-41. 10.1073/pnas.0930314100.

- 61.
Bintu L, Buchler N, Garcia H, Gerland U, Hwa T, Kondev J, Phillips R: Transcriptional regulation by the numbers: models. Curr Opin Genet Dev. 2005, 15: 116-124. 10.1016/j.gde.2005.02.007.

- 62.
Hasty J, Isaacs F, Dolnik M, McMillen D, Collins J: Designer gene networks: Towards fundamental cellular control. Chaos. 2001, 1: 201-220.

- 63.
Narula J, Gottgens B, Igoshin OA, M SA: Modeling Reveals Bistability and Low-Pass Filtering in the Network Module Determining Blood Stem Cell Fate. PLoS Comput Biol. 2010, 6: e1000771-10.1371/journal.pcbi.1000771.

- 64.
JDesigner. [http://sourceforge.net/projects/jdesigner/]

- 65.
Oscill8, Bifurcation Program. [http://sourceforge.net/projects/oscill8/]

## Acknowledgements

This work was funded by the Swedish Foundation for Strategic Research Senior Individual Grant (A3 06:215) and the Swedish Research Council (Vr 621-2008-3074). VC would like to acknowledge Prof Elliot M Meyerowitz for support. We thank an anonymous reviewer for constructive comments.

## Author information

### Affiliations

### Corresponding author

## Additional information

### Competing interests

The authors declare that they have no competing interests.

### Authors’ contributions

VC, VO, CP conceived and developed the project. VC, VO developed the simulation and analysis programs. VC, VO, CP analyzed the data and wrote the manuscript. All authors read and approved the final manuscript.

## Electronic supplementary material

**(A)**

Additional file 1: Figure S1. Steady state analysis of the stem cell circuit as functions of LIF and _{I 3}concentrations. The dashed lines indicate unstable states. **(A)** The steady state values of NANOG, OCT4-SOX2 and G as functions of LIF concentration. The plots display two states of the cell. (i) the stem cell state – high OCT4-SOX2/NANOG and low G and (ii) the differentiated state with low OCT4-SOX2/NANOG and high G **(B)** Similar graphs for small molecule differentiation inhibitors _{I 3} concentration (here for 100). Although there are still two states, NANOG is still higher here than in the previous case. Shown also is the level of FGF4 (magenta), which is at ≃20, which is much lower than in the previous case. (EPS 678 KB)

**not**

Additional file 3: Figure S3. Time series of [N] and [OS] for the stochastic dynamics of the alternative architecture with no differentiation gene G for *LIF*=100. A differentiation event; NANOG (red) and OCT4-SOX2 (blue). The OCT4-SOX2 expression is **not** lost in a differentiated cell in the model without G. (EPS 106 KB)

## Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

## Rights and permissions

## About this article

### Cite this article

Chickarmane, V., Olariu, V. & Peterson, C. Probing the role of stochasticity in a model of the embryonic stem cell – heterogeneous gene expression and reprogramming efficiency.
*BMC Syst Biol* **6, **98 (2012). https://doi.org/10.1186/1752-0509-6-98

Received:

Accepted:

Published:

### Keywords

- Stem cells
- Heterogeneity
- Stochasticity
- Computational model
- Differentiation
- Reprogramming