# Constructing stochastic models from deterministic process equations by propensity adjustment

- Jialiang Wu
^{1}, - Brani Vidakovic
^{2}and - Eberhard O Voit
^{2, 3}Email author

**5**:187

**DOI: **10.1186/1752-0509-5-187

© Wu et al; licensee BioMed Central Ltd. 2011

**Received: **18 July 2011

**Accepted: **8 November 2011

**Published: **8 November 2011

## Abstract

### Background

Gillespie's stochastic simulation algorithm (SSA) for chemical reactions admits three kinds of elementary processes, namely, mass action reactions of 0^{th}, 1^{st} or 2^{nd} order. All other types of reaction processes, for instance those containing non-integer kinetic orders or following other types of kinetic laws, are assumed to be convertible to one of the three elementary kinds, so that SSA can validly be applied. However, the conversion to elementary reactions is often difficult, if not impossible. Within deterministic contexts, a strategy of model reduction is often used. Such a reduction simplifies the actual system of reactions by merging or approximating intermediate steps and omitting reactants such as transient complexes. It would be valuable to adopt a similar reduction strategy to stochastic modelling. Indeed, efforts have been devoted to manipulating the chemical master equation (CME) in order to achieve a proper propensity function for a reduced stochastic system. However, manipulations of CME are almost always complicated, and successes have been limited to relative simple cases.

### Results

We propose a rather general strategy for converting a deterministic process model into a corresponding stochastic model and characterize the mathematical connections between the two. The deterministic framework is assumed to be a generalized mass action system and the stochastic analogue is in the format of the chemical master equation. The analysis identifies situations: where a direct conversion is valid; where internal noise affecting the system needs to be taken into account; and where the propensity function must be mathematically adjusted. The conversion from deterministic to stochastic models is illustrated with several representative examples, including reversible reactions with feedback controls, Michaelis-Menten enzyme kinetics, a genetic regulatory motif, and stochastic focusing.

### Conclusions

The construction of a stochastic model for a biochemical network requires the utilization of information associated with an equation-based model. The conversion strategy proposed here guides a model design process that ensures a valid transition between deterministic and stochastic models.

## Background

Most stochastic models of biochemical reactions are based on the fundamental assumption that no more than one reaction can occur at the exact same time. A consequence of this assumption is that only elementary chemical reactions can be converted directly into stochastic analogues [1]. These include: 1) zero-order reactions, such as the generation of molecules at a constant rate; 2) first-order reactions, with examples including elemental chemical reactions as well as transport and decay processes; and 3) second-order reactions, which include heterogeneous and homogeneous bimolecular reactions (dimerization). Reactions with integer kinetic orders other than 0, 1 and 2 are to be treated as combinations of sequential elementary reactions. The advantage of the premise of non-simultaneous reaction steps is that the stochastic reaction rate can be calculated from a deterministic, equation-based model with some degree of rigor, even though the derivation is usually not based on first physical principles but instead depends on other assumptions and on macroscopic information, such as a fixed rate constant in the equation-based model. The severe disadvantage is that this rigorous treatment is not practical for modelling larger biochemical reaction systems. The reasons include the following. First, in many cases, elementary reaction rates are not available. Secondly, even in the case that all reaction parameters are available, the computational expense is very significant when the system involves many species and reactions, and this fact ultimately leads to a combinatorial explosion of required computations. Within a deterministic modelling framework, the common practice in this situation is to fit the transient and steady-state experimental data with a phenomenological, (differential) equation-based model, which explicitly or implicitly eliminates or merges some intermediate species and reactions. The best-known examples are probably Michaelis-Menten and Hill rate laws, which are ultimately explicit, but in truth approximate a multivariate system of underlying chemical processes.

Similar model reduction efforts have been carried out for stochastic modelling. For instance, the use of a complex-order function (which corresponds to a reduced equation-based model) was shown to be justified for some types of stochastic simulations. A prominent example is again the Michaelis-Menten rate law, which can be reduced from a system of elementary reactions to an explicit function by means of the *quasi-steady-state assumption* (see Result section and [2, 3]). However, model reduction within the stochastic framework has proven to be far more difficult than in the deterministic counterpart. The difficulties are mainly due to the fact that the reduction must be carried out on the chemical master equation (CME). This process is nontrivial and has succeeded only in simple cases.

In general, the construction of a stochastic model for a large biochemical network requires the use of information available from an equation-based model. In the past, several strategies have been proposed for this purpose and within the context of Gillespie's exact stochastic simulation algorithm (SSA; [1]) and its variants [4]. For example, Tian and Burrage [5] proposed that a stochastic model could be directly formulated from the deterministic model through a Poisson leaping procedure. However, a rigorous mathematical justification for such a conversion is lacking. Typical moment-based approaches [6–8] derive ODEs for the statistical moments of the stochastic model from an equation-based model where the 0^{th}, 1^{st} and 2^{nd} order reactions follow mass action rate laws. More recently the moment method was extended to cover models consisting of rational rate laws [9]. Moreover, it was realized that the moment method is complementary to, but cannot fully replace, stochastic simulations, because it does not cover situations like genetic switches [6, 10].

In this article, we explore the mathematical connection between deterministic and stochastic frameworks for the pertinent case of Generalized Mass Action (GMA) systems, which are frequently used in Biochemical Systems Theory (BST; [11–13]). Specifically, we address two questions: First, under what conditions can a deterministic, equation-based model be converted directly into a stochastic simulation model? And second, what is a proper way of implementing this conversion? We will develop a method to answer these questions and demonstrate it for functions in the canonical power-law format of GMA systems. However, the results are applicable to other functions and formats as well, as we will demonstrate with several examples.

### Representations of systems of biochemical reactions

*N*

_{ s }different chemical species ${\left\{{S}_{s}\right\}}_{s=1}^{{N}_{s}}$, interact through

*N*

_{ r }unidirectional reaction channels ${\left\{{R}_{r}\right\}}_{r=1}^{{N}_{r}}$. Each reaction channel can be characterized as

*S*

_{ s }consumed and produced due to reaction

*R*

_{ r }, respectively, and

*k*

_{ r }is the rate constant. The changed amount of ${S}_{s}{v}_{rs}={\stackrel{\u0304}{v}}_{rs}-{\underset{-}{v}}_{rs}$, which is due to the firing of reaction

*R*

_{ r }, defines the stoichiometric coefficient of

*S*

_{ s }in

*R*

_{ r }. The stoichiometric coefficients of all species can be summarized according to each reaction

*R*

_{ r }in the stoichiometric vector

The size of the system is defined as Φ = *AU*, where *A* is the Avogadro number and *U* is the reaction volume.

The modelling of biochemical reaction networks typically uses one of two conceptual frameworks: deterministic or stochastic. In a deterministic framework, the state of the system is given by the a non-negative vector $\left[\mathbf{X}\left(t\right)\right]={\left[\left[{X}_{1}\left(t\right)\right],\dots ,\left[{X}_{{N}_{s}}\left(t\right)\right]\right]}^{T}\in {\mathbb{R}}^{{N}_{s}}$, where component [*X*_{
s
}(*t*)] represents the concentration of species *S*_{
s
}, measured in moles per unit volume. The temporal evolution of the state of the system is modelled by a set of ordinary differential equations, which in our case are assumed to follow a generalized mass action (GMA) kinetic law. By contrast, in a stochastic framework, the state of the systems is characterized by a vector $\mathbf{x}\left(t\right)={\left[{x}_{1}\left(t\right),\dots ,{x}_{{N}_{s}}\left(t\right)\right]}^{T}\in {\mathbb{Z}}^{{N}_{s}}$, whose values are non-negative integers. Specifically, *x*_{
s
}(*t*) = Φ [*X*_{
s
} (*t*)] is the count of *S*_{
s
} molecules, which is a sample value of the random variable *X*_{
s
}(*t*). The system dynamics of this process is typically described with the chemical master equation (CME). Both GMA and CME will be discussed in detail in the following sections.

### Motivation for the power-law formalism: reactions in crowded media

Power-law functions with non-integer kinetics have proven very useful in biochemical systems analysis, and forty years of research have demonstrated their wide applicability (*e.g*., see [11–13]). Generically, this type of description of a biochemical reaction can be seen either as a Taylor approximation in logarithmic space or as a heuristic or phenomenological model that has been applied successfully hundreds of times and in different contexts, even though it is difficult or impossible in many situations to trace it back to first mechanistic principles. A particularly interesting line of support for the power-law format can be seen in the example of a bimolecular reaction occurring in a spatially restricted environment. Savageau demonstrated that the kinetics of such a reaction can be validly formulated as a generalization of the law of mass action, where non-integer kinetic orders are allowed [14, 15]. Neff and colleagues [16–18] showed with careful experiments that this formulation is actually more accurate than alternative approaches.

*S*

_{1}and

*S*

_{2}is given as $k{\left[{X}_{1}\left(t\right)\right]}^{{f}_{1}}{\left[{X}_{2}\left(t\right)\right]}^{{f}_{2}}$. Here,

*k*is the

*rate constant*and

*f*

_{1}and

*f*

_{2}are real-valued

*kinetic orders*, which are no longer necessarily positive integers as it is assumed in a mass action law. As an example, consider the reversible bimolecular reaction ${S}_{1}+{S}_{2}\underset{{k}_{b}}{\overset{{k}_{f}}{\rightleftharpoons}}{S}_{3}$. Like Neff and colleagues [17], we begin by formulating a discrete update function for the population of

*S*

_{3}molecules as

The first term on the right-hand side of this equation, *f* ([*X*_{1}], [*X*_{2}])Δ*t x*_{1}*x*_{2}, describes the production of *S*_{3}: it depends on the totality of possible collisions *x*_{1} *x*_{2} and also on some fraction *f* ([*X*_{1}], [*X*_{2}])Δ*t* that actually reacts and forms the product. In a dilute environment, *f* ([*X*_{1}], [*X*_{2}]) equals a traditional rate constant, and the reaction obeys the law of mass action, while in a spatially restricted environment, such as the cytoplasm, one needs to take crowding effects into account. As shown in Savageau [14, 15], the desired fraction of a reaction in a crowded environment becomes a *rate function* that depends on the current concentrations of *S*_{1} and *S*_{2}. The second term, *g* ([*X*_{3}]) Δ*tx*_{3}, describes the fraction *g* ([*X*_{3}]) Δ*t* of species *S*_{3} that dissociates back into *S*_{1} and *S*_{2}. This fraction may depend on some functional form of [*X*_{3}] because in a crowded environment the complex may not be able to dissociate effectively. Thus, rate constants in the generalized mass action setting become rate functions (*cf*. [17]).

*f*and

*g*in the logarithmic space (log [

*X*

_{1}], log[

*X*

_{2}]) around some operating point (

*a*,

*b*). The result for

*f*is

*k*

_{ f },

*α*, and

*β*are constants related to the chosen operating point (

*a*,

*b*). The final step is achieved by ignoring all higher order terms (HOT) beyond the constant and linear terms. Transformation back to the Cartesian space yields

*g*([

*X*

_{3}]) ≈

*k*

_{ d }[

*X*

_{3}]

^{ γ }. By combining constants we arrive at a power-law representation for the dynamics of species

*S*

_{3}as

where *a* = *α* + 1, *b* = *β* + 1, and *c* = *γ* + 1. As long as *k*_{
f
}, *k*_{
d
}, *a*, *b* and *c* remain more or less constant throughout a relevant range, the power-law model is mathematically well justified. In actual applications, the values of rate constants and kinetic orders can be estimated from experimental data [19]. When the functions *f* and *g* are originally not in power-law format, they can be locally approximated by power-law functions with a procedure similar to the one shown above (Equations (3) to (5)). An illustration will be given in the example section.

### The Generalized Mass Action (GMA) format

*de novo*or as an approximation of some other nonlinear rate laws. GMA models characterize the time evolution of the system state given that the system was in the state

*X*(

*t*

_{0}) at some initial time

*t*

_{0}. Generically, the state of the system is changed within a sufficiently small time interval by one out of the

*N*

_{ r }possible reactions that can occur in the system. The reaction velocity through reaction channel

*R*

_{ r }is:

*s*= 1, ...,

*N*

_{ s }. As shown in the example of a bimolecular reaction, the kinetic order

*f*

_{ rs }associated with species

*S*

_{ s }captures the effects of both reactant properties (such as the stoichiometric coefficient

*v*

_{ rs }) and environmental influences (such as temperature, pressure, molecular crowding effects, etc.). Therefore

*f*

_{ rs }does not necessary equal an integer

*v*

_{ rs }, which is assumed to be the case in mass action kinetics, but is possibly real-valued and may be negative. Summing up the contributions of all reactions, one obtains a GMA model describing the dynamics of

*S*

_{ s }as

for every *s* = 1, ..., *N*_{
s
}. Each reaction contributes either a production flux or a degradation flux to the dynamics of a certain species. Positive terms (*v*_{
rs
} > 0) represent the production of *S*_{
s
}, while negative terms (*v*_{
rs
} < 0) describe degradation. If *f*_{
rs
} is positive, then *S*_{
s
} accelerates the reaction *R*_{
r
}; a negative value represents that *S*_{
s
} inhibits the reaction, and *f*_{
rs
} = 0 implies that *S*_{
s
} has no influence on the reaction. The rate constant *k*_{
r
} for reaction *R*_{
r
}, is either positive or zero. Both, the rate constant and the kinetic order, are to be estimated from data.

### Proper use of equation-based functions for stochastic simulations

*α*(

**X**), and

*α*(

**X**)

*dt*describes the probability that a reaction will change the value of a system variable within the next (infinitesimal) time interval (

*t*,

*t*+

*dt*). While a formal definition will be given later (Equation 18), it is easy to intuit that the propensity function is in some sense analogous to the rate in the corresponding deterministic model. In fact, the propensity function is traditionally assumed to be

*α*(

**X**) =

*f*

_{ s }(

**X**), if the deterministic model is

*X*

_{ s }' =

*f*

_{ s }(

**X**,

*t*),

*s*= 1, ...,

*N*

_{ s }. However, a proper justification for this common practice is by and large missing. Indeed, we will show that the direct use of a rate function as the propensity function in a stochastic simulation algorithm requires that at least one of the following assumptions be true:

- 1)
*f*is a linear function; - 2)
the reaction is monomolecular;

- 3)
all

*X*_{ i }in the system are noise-free variables,*i.e*., without (or with ignorable) fluctuations, which implies that the covariance of any two participating reactants is zero (or close to zero).

Each of these assumptions constitutes a sufficient condition for the direct use of a rate function as the propensity function and applies, in principle, to GMA as well as other systems. The validity of these conditions will be discussed later. Specifically, the first condition will be addressed in the Results section under the headings "0^{th}-order reaction kinetics" and "1^{st}-order reaction kinetics, " while the second condition will be discussed under the heading "Real-valued order monomolecular reaction kinetics." The third condition will be the focus of Equations (29-36) and their associated explanations.

In reality, the rates of reactions in biochemical systems are commonly nonlinear functions of the reactant species, and fluctuations within each species are not necessarily ignorable. Therefore, to the valid use of an equation-based model in a stochastic simulation mandates that we know how to define a proper propensity function. The following section addresses this issue. It uses statistical techniques to characterize estimates for both the mean and variance of the propensity function, and these features will allow an assessment of the validity of the assumption *α*(**X**) = *f*_{
s
}(**X**) and prescribe adjustments if the assumption is not valid.

## Methods

### Deriving the mean and variance of a power-law function of random variables

*X*

_{ s }with the format $PL\left(\mathbf{X}\right)=k\prod _{s=1}^{{N}_{s}}{X}_{s}^{{f}_{s}}$. Estimates of its mean

*μ*

_{ PL }and variance

*σ*

_{ PL }are given as

(for details, see Additional file 1). Here,

*μ*

_{ s }= E[

*X*

_{ s }] and ${\sigma}_{s}^{2}=\mathsf{\text{E}}\left[{\left({X}_{s}-{\mu}_{s}\right)}^{2}\right]$ are the mean and variance of random variable

*X*

_{ s }, respectively. If we choose to express cov [log

*X*

_{ i }, log

*X*

_{ j }] as a function of

*μ*

_{ s },

*σ*

_{ s }

^{2}and covariance

*σ*

_{ ij }= cov [

*X*

_{ i },

*X*

_{ j }], using a Taylor approximation, we obtain

*X*

_{1}, ...,

*X*

_{ s })is log-normally distributed (

*i.e*., (log

*X*

_{1}, ..., log

*X*

_{ s }) is normally distributed). In such a case, a simpler alternative way to calculate cov [log

*X*

_{ i }, log

*X*

_{ j }] is

The approximation formulae for *μ*_{
PL
} and *σ*_{
PL
}^{2} in eqns. (8)-(10) provide an easy numerical implementation if observation data are available to estimate cov [log*X*_{
i
}, log*X*_{
j
}]. Furthermore, Equations (11)-(13) demonstrate how *μ*_{
PL
} and *σ*_{
PL
}^{2} are related to *μ*_{
s
}, *σ*_{
s
}^{2} and *σ*_{
ij
}; however, the price of this insight is paid by the possible inaccuracy introduced through the Taylor approximation. Equations (15)-(17) also provide a functional dependence of *μ*_{
PL
} and *σ*_{
PL
}^{2} on (*μ*_{
s
}, *σ*_{
s
}^{2}, *σ*_{
ij
}), but it is only valid if the additional assumption of log-normality is acceptable.

### Deriving proper propensity functions for stochastic simulations from differential equation-based models

where Φ is the system size as defined above.

*R*

_{ r }stochastically, one needs the state update vector

**v**

_{ r }and must characterize the quantity of molecules flowing through of reaction channel

*R*

_{ r }during a small time interval. The key concept of this type of description is the propensity function

*α*

_{ r }(

**x**), which is defined as

**X**(t) is no longer deterministic, and the result is instead stochastic and based on the transition probability

Updating CME requires knowledge of every possible combination of all species counts within the population, which immediately implies that it can be solved analytically for only a few very simple systems and that numerical solutions are usually prohibitively expensive [24]. To address the inherent intractability of CME, Gillespie developed an algorithm, called the *Stochastic Simulation Algorithm* (SSA), to simulate CME models [1]. SSA is an exact procedure for numerically simulating the time evolution of a well-stirred reaction system. It is rigorously based on the same microphysical premise that underlies CME and gives a more realistic representation of a system's evolution than a deterministic reaction rate equation represented by ODEs. SSA requires knowledge of the propensity function, which however is truly available only for elementary reactions. These reactions include: 1) 0^{th} order reactions, exemplified with the generation of a molecule at a constant rate; 2) 1^{st} order monomolecular reactions, such as an elemental chemical conversion or decay of a single molecule; 3) 2^{nd} order bimolecular reactions, including reactive collisions between two molecules of the same or different species. The reactive collision of more than two molecules at exactly the same time is considered highly unlikely and modelled as two or more sequential bimolecular reactions.

*R*

_{ r }is computed as the product of a stochastic rate constant

*c*

_{ r }and the number

*h*

_{ r }of distinct combinations of reactant molecules, i.e.

Here ${h}_{r}\left(\mathbf{x}\right)=\left\{\begin{array}{c}\hfill \prod _{s=1}^{{N}_{s}}\left(\begin{array}{c}\hfill {x}_{s}\hfill \\ \hfill {\underset{-}{v}}_{rs}\hfill \end{array}\right)\approx \frac{\prod _{s=1}^{{N}_{s}}{x}_{s}^{{\underset{-}{v}}_{rs}}}{\prod _{s=1}^{{N}_{s}}{\underset{-}{v}}_{rs}!},\mathsf{\text{for}}\phantom{\rule{2.77695pt}{0ex}}{x}_{s}\ge {\underset{-}{v}}_{rs}>0\hfill \\ \hfill 0,\phantom{\rule{2.77695pt}{0ex}}\mathsf{\text{otherwise}}\hfill \end{array}\right.$, where *x*_{
s
} is the sample value of random variable *X*_{
s
}. The approximation is invoked when *x*_{
s
} is large and (*x*_{
s
} - 1), ..., (*x*_{
s
} - v_{
rs
} + 1) are approximately equal to *x*_{
s
}.

In Gillespie's original formulation [1] *c*_{
r
} is a constant that only depends on the physical properties of the reactant molecules and the temperature of the system, and *c*_{
r
}*dt* is the probability that a particular combination of reactant molecules will react within the next infinitesimally small time interval (*t*, *t* + *dt*). The constant *c*_{
r
} can be calculated from the corresponding deterministic rate constants, if they are known.

_{ r }is not a constant but a function of the reactant concentrations. Thus, we denote c

_{ r }as a

*stochastic rate function*, while retaining the definition of

*h*

_{ r }as above. Knowing that any positive-valued differentiable function can be approximated locally by a power-law function, we assume the functional form of the stochastic rate function as

*κ*

_{ r }and

*ε*

_{ rs }are constants that will be specified in the next section, and

*r*= 1, ...,

*N*

_{ r }. Note that

*ε*

_{ rs }are now real-valued. Once the stochastic rate function is determined (see below), the propensity function can be calculated as

**x**and summing over all

**x**, we obtain

*X*

_{ s }(

*t*) is given as

The details of these derivations are shown in Additional file 1.

*X*

_{ s },

*v*

_{ rs },

*k*

_{ r }and

*f*

_{rs' }are again the stoichiometric coefficients, rate constants, and kinetic orders, respectively. By substituting $\left[{X}_{s}\right]=\frac{E\left[{X}_{s}\right]}{\mathrm{\Phi}}$ from Equation (18) into this GMA model, we obtain a "particle-based" equation of the format

- 1)adopt a zero-covariance assumption as was done in [25], which implies ignoring random fluctuations within every species as well as their correlations. This assumption is only justified for some special cases such as monomolecular and bimolecular reactions under the thermodynamic limit (cf. [4, 6]), but is not necessary valid in generality. Here the thermodynamic limit is defined as a finite concentration limit which the system reaches when both population and volume approach infinity. Under this assumption, the left hand side of (29) becomes$\begin{array}{cc}\hfill E\left[{\alpha}_{r}\left(\mathbf{x}\right)\right]& =E\left[\frac{{\kappa}_{r}}{\prod _{s=1}^{{N}_{s}}{\underset{-}{v}}_{rs}!}\prod _{s=1}^{{N}_{s}}{x}_{s}^{{\underset{-}{v}}_{rs}+{\epsilon}_{rs}}\right]\hfill \\ =\frac{{\kappa}_{r}}{\prod _{s=1}^{{N}_{s}}{\underset{-}{v}}_{rs}!}\prod _{s=1}^{{N}_{s}}E{\left[{X}_{s}\right]}^{{\underset{-}{v}}_{rs}+{\epsilon}_{rs}}\hfill \end{array}$(30)

*r*= 1, ...,

*N*

_{ r }, and Equation (24) yields

Here, the index *r*_0 is used to distinguish this 0-covariance propensity function from a second type of propensity in the next section.

for every *s* = 1, ..., *N*_{
s
}.. Note that this result is exactly equivalent to the equation-based model (27).

*α*

_{r_0 }and obtain mean and variance as

for every *s* = 1, ..., *N*_{
s
}. These expressions demonstrate that even with large numbers of molecules the mean of CME does not always converge to the GMA model. Indeed, the convergence is only guaranteed in one of the following special situations: 1) the reaction is of 0^{th} order; 2) the reaction is a real value-order monomolecular reaction, with 1^{st} order reaction as a special case; 3) the covariance contribution in (34) is sufficiently small to be ignored for all participating reactant species of a particular reaction channel. Except for these three special situations, the covariance as shown in (34) significantly affects the mean dynamics. Therefore, stochastic simulations using zero-covariance propensity functions will in general yield means different from what the deterministic GMA model produces. How large these differences are cannot be said in generality. Under the assumption that the GMA model correctly captures the mean dynamics of every species, this conclusion means that *α*_{r_0 }is not necessarily an accurate propensity function for stochastic simulations, and the direct conversion of the equation-based model into a propensity function must be considered with caution.

- 2)We again assume that the GMA model is well defined, which implies that information regarding the species correlations and fluctuations has been captured in the parameters of the GMA model on the left hand size of Equations (7) and (28). To gain information regarding correlations, we use Taylor expansion to approximate the propensity function (see Additional file 1 for details):$\begin{array}{c}E\left[{\alpha}_{r}(X(t))\right]=E\left[\frac{{\kappa}_{r}}{{\displaystyle \prod _{s=1}^{{N}_{s}}{\underset{\xaf}{v}}_{rs}}!}{\displaystyle \prod _{s=1}^{{N}_{s}}{X}_{s}^{{\underset{\xaf}{v}}_{rs}}{}^{+{\epsilon}_{rs}}}\right]\\ \approx \frac{{\kappa}_{r}}{{\displaystyle \prod _{s=1}^{{N}_{s}}{\underset{\xaf}{v}}_{rs}!}}{\displaystyle \prod _{s=1}^{{N}_{s}}E{\left[{X}_{s}\right]}^{{\underset{\xaf}{v}}_{rs}+{\epsilon}_{rs}}}\\ \times \mathrm{exp}\left({\displaystyle \sum _{i<j}^{{N}_{s}}\left({\underset{\xaf}{v}}_{ri}+{\epsilon}_{ri}\right)\left({\underset{\xaf}{v}}_{rj}+{\epsilon}_{rj}\right)}\text{}\text{cov}\left[1\text{og}{X}_{i},\mathrm{log}{X}_{j}\right]\right)\end{array}$(37)

**x**of the system at time

*t*, the stochastic rate function of reaction

*R*

_{ r }is

Here it is important to understand that although the random variables {*X*_{
s
}}_{s∈S}appear in the expression *c*_{
r
}(**x**), *c*_{
r
}(**x**) is not a function of random variables but a deterministic function. The reason is that the cov [log*X*_{
i
}(*t*), log*X*_{
j
}(*t*)] in the composition of *c*_{
r
}(**x**), which as the numerical characteristic of the random variables {*X*_{
s
}}_{s∈S}, is deterministic. Therefore, the stochastic rate function *c*_{
r
}(**x**) is a well-justified deterministic function that is affected by both the state of the system $\left[{x}_{1},\dots ,{x}_{{N}_{s}}\right]$ and cov [log*X*_{
i
}(*t*), log*X*_{
j
}(*t*)], the numerical characteristic of fluctuations in the random variables {*X*_{
s
}}_{s∈S}.

*c*

_{ r }(

**x**), the propensity function is

*R*

_{ r }. For simplicity of discussion, we define the

*propensity adjustment factor*(paf) of reaction

*R*

_{ r }as

*paf*is a function of time

*t*and represents the contribution of the reactants to correlations among species in the calculation of the propensity function for reaction

*R*

_{ r }. We denote the propensity function in (39), which accounts for the contribution of the covariance, as

*α*

_{r_cov}, in order to distinguish it from the propensity function

*α*

_{r_0 }(32), which is based on the assumption of zero-covariance,

*i.e*.,

Remembering that cov [log*X*_{
i
}(*t*), log*X*_{
j
}(*t*)], which is a component in both the stochastic rate function *c*_{
r
}(**x**) and now in the function *paf*(*t*), is a deterministic function rather than a function of random variables, *paf*(*t*) is a deterministic correction to the kinetic constant *k*_{
r
} in the construction of *α*_{r_cov }in (41), which corrects the stochastic simulation toward the correct average.

*α*

_{r_0},

*α*

_{r_cov }leads to accurate stochastic simulations. To illustrate this difference, we analyze $\frac{d}{dt}E\left[{X}_{s}\left(t\right)\right]$ as follows: We apply the approximation techniques in eqns. (9)-(11) in order to obtain the mean and variance of the propensity function

*α*

_{r_cov}:

for every *s* = 1, ..., *N*_{
s
}, which is equivalent in approximation to the GMA model (28). In the other words, the mean of every molecular species obtained by using *α*_{r_cov }in the CME derived equation (27) is approximately identical to the corresponding macroscopic variable in the GMA model.

**Calculation of** cov [log*X*_{
i
}(*t*), log*X*_{
j
}(*t*)**]**

When data in the form of multiple time series for all the reactants are available, it is possible to compute cov [log*X*_{
i
}(*t*), log*X*_{
j
}(*t*)**]** directly from these data. Once this covariance is known, the function *paf*, *α*_{r_cov }and the mean dynamics can all be assessed. Alas, the availability of several time series data for all reactants under comparable conditions is rare, so that cov [log*X*_{
i
}(*t*), log*X*_{
j
}(*t*)**]** must be estimated in a different manner.

If one can validly assume that the covariance based on *α*_{r_0 }does not differ significantly from the covariance based on *α*_{r_cov}, one may calculate cov [log*X*_{
i
}(*t*), log*X*_{
j
}(*t*)**]** by one of following methods.

Method 1:

One uses *α*_{r_0 }to generate multiple sets of time series data of all reactants and then computes cov [log*X*_{
i
}(*t*), log*X*_{
j
}(*t*)**]**.

Method 2:

*X*

_{ i }(

*t*), log

*X*

_{ j }(

*t*)

**]**is expressed as a function of mean and covariance in one of the following ways; either as

The first functional expression of cov [log*X*_{
i
}(*t*), log*X*_{
j
}(*t*)**]** is achieved by Taylor approximation, whereas the second expression is obtained by the additional assumption that the concentrations (*X*_{1}, ..., *X*_{
s
}) are log-normally distributed [8, 23]. The consideration of a log-normal distribution is often justified by the fact that many biochemical data have indeed been observed to be log-normally distributed (*e.g*., [20–22]).

Second, one uses *α*_{r_0 }to approximate the mean and covariance either by direct simulation, as shown in method 1, or by a moment-based approach, which is explained in Additional file 2, and which yields the differential equations

Here for *r* = 1, ..., *N*_{
r
}, and *s*, *m*, *n* = 1, ..., *N*_{
s
}, $\mu ={\left({\mu}_{1},\dots ,{\mu}_{{N}_{s}}\right)}^{T}$, (*V*)_{
rs
}= *v*_{
rs
}, $\alpha ={\left({\alpha}_{1},...,{\alpha}_{{N}_{r}}\right)}^{T}$, ${\alpha}^{\u2033}={\left({\alpha}_{r}^{{}^{\u2033}},.\phantom{\rule{2.77695pt}{0ex}}.\phantom{\rule{2.77695pt}{0ex}}.\phantom{\rule{2.77695pt}{0ex}},{\alpha}_{{N}_{r}}^{{}^{\u2033}}\right)}^{T}$, ${\left({\alpha}_{r}^{{}^{\u2033}}\right)}_{mn}=\frac{{\partial}^{2}{\alpha}_{r}\left(\mathbf{X}\right)}{\partial {X}_{m}\partial {X}_{n}}$, ${\alpha}_{r}^{{}^{\u2033}}\odot \sigma =\sum _{m,n=1}^{{N}_{s}}\frac{{\partial}^{2}{\alpha}_{r}\left(\mathbf{X}\right)}{\partial {X}_{m}\partial {X}_{n}}{|}_{\mathbf{X}=\mu}{\sigma}_{mn}$, ${\alpha}^{{}^{\u2033}}\odot \sigma \triangleq {\left({\alpha}_{1}^{{}^{\u2033}}\odot \sigma ,\dots ,{\alpha}_{{N}_{r}}^{{}^{\u2033}}\odot \sigma \right)}^{T}$, ${\alpha}^{\prime}=\left({{\alpha}_{1}}^{\prime},\dots ,{{\alpha}_{{N}_{r}}}^{\prime}\right)$, ${{\alpha}_{r}}^{\prime}={\left(\frac{\partial {\alpha}_{r}\left(\mathbf{\mu}\right)}{\partial {X}_{1}},\dots ,\frac{\partial {\alpha}_{r}\left(\mathbf{\mu}\right)}{\partial {X}_{{N}_{s}}}\right)}^{T}$, and Λ is a diagonal matrix with ${\left(\mathrm{\Lambda}\right)}_{rr}={\alpha}_{r}\left(\mu \right)+\frac{1}{2}\sum _{m,n=1}^{{N}_{s}}\frac{{\partial}^{2}{\alpha}_{r}\left(\mu \right)}{\partial {X}_{m}\partial {X}_{n}}{\sigma}_{mn}$.

### Statistical criteria for propensity adjustment

Suppose an equation-based model captures the average behavior of a stochastic system and one intends to find the propensity function for a stochastic simulation that will reproduce that means. One can use the 95% confidence interval to evaluate the need for a propensity adjustment. Specifically, for stable systems that will reach a steady state, we use the reversible reaction model as an example. If the steady state of the ODE *x*_{
st
} is within the 95% confidence interval of *n* runs of stochastic simulations, i.e. ${x}_{st}\in \left[{\mu}_{st}-1.96\frac{{\delta}_{st}}{\sqrt{n}},{\mu}_{st}+1.96\frac{{\delta}_{st}}{\sqrt{n}}\right]$, then the rate function in the original ODEs can be used as the propensity without adjustment; otherwise propensity adjustment is needed. Here *μ*_{
st
} and *δ*_{
st
} can be attained from either a moment-base method or from *n* independent runs of stochastic simulations using propensity without adjustment. An example discussing a reversible reaction with feedback controls can be found in the results section.

For other systems that do not reach a steady state, but where instead transient characteristics are of the highest interest, one can judge the need of propensity adjustment by whether the pertinent characteristics of the ODEs are within the 95% confidence interval of the corresponding characteristic, which is given by a prediction from the moment-based method or from *n* runs of stochastic simulations. The Repressilator example in the result section will serve as a demonstration.

## Results

### Generic special cases

*e.g*., Equations (40)-(44)) is not needed, and in some other cases it becomes relatively simple.

- 1)
0

^{th}-order reaction kinetics

*s*= 1, ...,

*N*

_{ s },

*f*

_{ rs }= 0. According to Equations (40)-(44), one obtains

^{th}-order reaction, its rate equation can be taken directly as the propensity function in stochastic simulations.

- 2)
1

^{st}-order reaction kinetics

*f*

_{ rs }=

*δ*

_{ sj },

*i*,

*j*= 1, ...,

*N*

_{ s }. Therefore, according to Equations (40)-(44)

^{st}-order reactions, the rate equation can again be taken directly as the propensity function in stochastic simulations.

- 3)
Real-valued order monomolecular reaction kinetics

*f*

_{ rj }≠ 0,

*f*

_{ rs }= 0, for any

*s*≠

*j*,

*s*= 1, ...,

*N*

_{ s }. Equations (40)-(44) lead to

- 4)
2

^{nd}-order reaction kinetics

*i*,

*j*∈ {1, ...,

*N*

_{ s }},

*i*≠

*j*,

*f*

_{ ri }=

*f*

_{ rj }= 1, and

*f*

_{ rs }= 0, for all

*s*≠

*i*,

*j*. Therefore, according to Equations (40)-(44)

^{nd}-order reactions is different from the rate equation. The difference can be ignored only if the contribution from the covariance is insignificant. In general, the rate equation yields only an approximate propensity function for stochastic simulations, and the approximation quality must be assessed on a case-by-case basis.

- 5)
Bimolecular reaction with real-valued order kinetics

*i*,

*j*∈ {1, ...,

*N*

_{ s }},

*i*≠

*j*,

*f*

_{ ri },

*f*

_{ rj }≠ 0, and

*f*

_{ rs }= 0, for all

*s*≠

*i*,

*j*. According to Equations (40)-(44) we obtain

For bimolecular reactions of complex order, the propensity function is different from the rate equation. The difference can be ignored only if the contribution from the covariance is insignificant.

### Power-law representation of a reversible reaction with feedback controls

Here *S*_{3} feeds back to inhibit the forward reaction and *S*_{1} feeds back on the reverse reaction and accelerates it. The task is to develop a stochastic model whose performance converges to that of the deterministic GMA model. We can see from equations (52) that three variables *x*_{1}, *x*_{2} and *x*_{3} contribute to the forward flux ${k}_{f}{\mathrm{\Phi}}^{1-{f}_{1}-{f}_{2}-{f}_{3}}{x}_{1}^{{f}_{1}}{x}_{2}^{{f}_{2}}{x}_{3}^{{f}_{3}}$ and two variables *x*_{1} and *x*_{3} contribute to the backward flux ${k}_{b}{\mathrm{\Phi}}^{1-{g}_{1}-{g}_{3}}{x}_{1}^{{g}_{1}}{x}_{3}^{{g}_{3}}$. Because several variables are involved, their covariance has the potential of affecting the forward and the backward propensity functions in a stochastic simulation. To obtain the covariance information, we formulate the moment equations (53) from the ODE model (52).

Here *μ* = (*μ*_{1}, *μ*_{2}, *μ*_{3})^{
T
}, $V=\left[\begin{array}{ccc}\hfill -1\hfill & \hfill -1\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill 1\hfill & \hfill -1\hfill \end{array}\right]$, $\alpha ={\left({\alpha}_{1},{\alpha}_{2}\right)}^{T}=\left[\begin{array}{c}\hfill {k}_{f}{\mathrm{\Phi}}^{1-{f}_{1}-{f}_{2}-{f}_{3}}{x}_{1}^{{f}_{1}}{x}_{2}^{{f}_{2}}{x}_{3}^{{f}_{3}}\hfill \\ \hfill {k}_{b}{\mathrm{\Phi}}^{1-{g}_{1}-{g}_{3}}{x}_{1}^{{g}_{1}}{x}_{3}^{{g}_{3}}\hfill \end{array}\right]$.

Moreover, for *r* = 1, 2 and *m*, *n* = 1, 2, 3, ${\left({\alpha}_{r}^{{}^{\u2033}}\right)}_{mn}=\frac{{\partial}^{2}{\alpha}_{r}\left(\mathbf{X}\right)}{\partial {x}_{m}\partial {x}_{n}}$, *α*" = (*α*_{1}", *α*_{2}")^{
T
}, $\sigma =\left[\begin{array}{ccc}\hfill {\sigma}_{11}\hfill & \hfill {\sigma}_{12}\hfill & \hfill {\sigma}_{13}\hfill \\ \hfill {\sigma}_{21}\hfill & \hfill {\sigma}_{22}\hfill & \hfill {\sigma}_{23}\hfill \\ \hfill {\sigma}_{31}\hfill & \hfill {\sigma}_{32}\hfill & \hfill {\sigma}_{33}\hfill \end{array}\right]$, ${\alpha}_{r}^{{}^{\u2033}}\odot \sigma \triangleq \sum _{m,n=1}^{3}\frac{{\partial}^{2}{\alpha}_{r}\left(\mathbf{X}\right)}{\partial {x}_{m}\partial {x}_{n}}{|}_{\mathbf{X}=\mu}{\sigma}_{mn}$, *α*"⊙*σ* ≜ (*α*_{1}"⊙ *σ*, *α*_{2}"⊙ *σ*)^{
T
}, *α*' = (*α*_{1}', *α*_{2}'), ${{\alpha}_{r}}^{\prime}={\left(\frac{\partial {\alpha}_{r}\left(\mu \right)}{\partial {x}_{1}},\frac{\partial {\alpha}_{r}\left(\mu \right)}{\partial {x}_{2}},\frac{\partial {\alpha}_{r}\left(\mu \right)}{\partial {x}_{3}}\right)}^{T}$, and $\mathrm{\Lambda}=\left[\begin{array}{cc}\hfill {\alpha}_{1}\left(\mu \right)+\frac{1}{2}\sum _{m,n=1}^{3}\frac{{\partial}^{2}{\alpha}_{1}\left(\mu \right)}{\partial {x}_{m}\partial {x}_{n}}{\sigma}_{mn}\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill {\alpha}_{2}\left(\mu \right)+\frac{1}{2}\sum _{m,n=1}^{3}\frac{{\partial}^{2}{\alpha}_{2}\left(\mu \right)}{\partial {x}_{m}\partial {x}_{n}}{\sigma}_{mn}\hfill \end{array}\right]$.

### Repressilator

*Repressilator*[27], which is a three-component genetic circuit where each component represses its downstream neighbour. More specifically (as shown in Figure 3), gene

*G*

_{1}codes for protein

*x*

_{1}, whose dimer

*y*

_{1}subsequently represses the transcription of the gene

*G*

_{2}. Similarly,

*y*

_{2}, the dimer of gene

*G*

_{2}'s protein product

*x*

_{2}, represses the transcription of gene

*G*

_{3}, and

*y*

_{3}, the dimer of gene

*G*

_{3}'s protein product

*x*

_{3}, represses the transcription of gene

*G*

_{1}. The corresponding differential equation model following mass action kinetics is given by [28]

*i*= 1, 2, 3;

*j*= 2, 3, 1;

*k*= 3, 1, 2; the rate constants are explained in the diagram below

[28]. Here Φ = 1, $p\left({x}_{i}\right)=1+4{c}_{p}{x}_{i}+\frac{4{c}_{d}{c}_{p}d{x}_{i}}{{\left(1+{c}_{d}{c}_{p}{x}_{i}^{2}\right)}^{2}}$, *c*_{
p
} = *κ*_{+}/*κ*_{-}, *c*_{
d
} = *k*_{+}/*k*_{-} and *d* = *d*_{0, i}+ *d*_{
r, i
} for *i* = 1, 2, 3. It has been shown that the simplified ODEs rather accurately approximate the transient dynamics of the full system by retaining the original oscillation period and amplitude.

*x*

_{ i }and

*m*

_{ i }are reduced to half (Figure 4). The reason is that, in the stochastic simulation, the oscillation period is very sensitive to the ratio of

*x*

_{ i }and

*m*

_{ i }, which has been altered by the scaling operation. Therefore, in general one needs to pay attention to how scaling may affect the stochastic performance when the model is generated through the conversion of an ODE model.

We can see from equations (55) that two variables *x*_{
i
} and *m*_{
i
} contribute to the production of *x*_{
i
}; hence, their covariance could affect the propensity function of *x*_{
i
} in the production reaction of a stochastic simulation. Similar to the example of a reversible reaction (Equation 52), it is therefore necessary to evaluate covariance effects and to judge whether the propensity function needs adjusting. Thus, we need to compare the difference between the dynamics of the phenomenological model (55) and the dynamics under the influence of covariance, which can be produced by either stochastic simulation or the moment approach.

^{nd}and the 4

^{th}panels of Figure 5. Obtaining the covariance-influenced dynamics with the moment-based approach is more complicated, and we need to discuss some implementation issues.

*p*(

*x*

_{ i })

^{-1}, which have rather complicated functional forms. To simplify the calculation, we replace the function

*p*(

*x*

_{ i })

^{-1}with an approximating power-law function. Specifically, suppose the original parameter values are

*κ*

_{+}=

*k*

_{+}= 5,

*κ*

_{-}=

*k*

_{-}= 100 and

*d*= 20. Plotting the data (

*x*

_{ i },

*p*(

*x*

_{ i })

^{-1})in log-log space (Figure 5) indicates that the original function is represented well by a straight line:

*x*

_{ i }∈ [30, 300]. In Cartesian space, this line corresponds to the power-law function

which models the original function very well (see Figure 5). For *x*_{
i
} ∈ [1, 30], this power-law function does not fit the original function precisely; the effect of this imprecision can be evaluated later at after we use this power-law function in the moment-based method. Moreover, using the truncated moment equations to estimate the mean and variance involves multiple approximations: First, the function *p* (*x*_{
i
})^{-1} on the right-hand side of (55) is replaced by a power-law function (see Figure 5). Second, the result is approximated by Taylor expansion to the second order. Third, similar to the example of a reversible reaction, the central moment of the third degree is assumed to be zero, which leads to a closed-form ODE for the first two moments.

*t*= 400 seconds. The oscillation period and amplitude in the stochastic simulation based on the GMA approximation (without adjustment) are almost identical to those of the original ODEs. Therefore, a propensity adjustment for the GMA approximation is not needed, and the GMA approximation can be used as a propensity function for stochastic simulations. In other words, a stochastic model for the Repressilator system can be generated by using the scheme in (32) without propensity adjustment. Moreover, the imprecision caused by the power-law approximation can be tolerated when its corresponding moment-based mean matches the original ODEs sufficiently well with respect to the features of highest interest.

### Enzymatic reaction using a quasi-steady state assumption (QSSA)

*E*]

_{0}≜ [

*E*] + [

*ES*] is assumed to be constant. In addiction, by making the so-called

*quasi steady state assumption*(QSSA) [29, 30], assuming that the complex ES is essentially in steady state, we can assert $\frac{d\left[ES\right]}{dt}\approx 0$. As it has been discussed many times in the literature, QSSA reduces the system and leads to the approximate form

which is known as *Michaelis-Menten kinetics* [30]. The characterizing parameters are *V*_{max} = *k*_{2}[*E*]_{0} and *K*_{
m
} = (*k*_{-1} + *k*_{2})/*k*_{1}.

*S*→

*P*is

where the volume was scaled so that Φ = 1 and the lower-case letter *s* denotes the molecule count of species S. Instead of reviewing the relatively complicated manipulations with CME, we show in the following that the techniques described above lead directly from the equation-based model to the propensity function for the reduced system.

*T*] ≜

*K*

_{ m }+ [

*S*]. The result,

*S*]

_{0}and [

*T*]

_{0}=

*K*

_{ m }+ [

*S*]

_{0}. The corresponding stochastic model has only one reaction channel and the propensity function is

*T*is a function of

*s*and its covariance with

*s*is therefore 1. By applying

*t*=

*K*

_{ m }+

*s*, the propensity function can be simplified as

Thus, we arrive at the propensity function for the reduced system, which is identical to the result of Rao and Arkin obtained through manipulations of CME.

In the above derivation, we used the simplest type of recasting, where a new, auxiliary variable simply consists of an old variable plus a constant. This reformulation of the Michaelis-Menten process as a pair of GMA equations is a special case of a much more general recasting technique that permits the equivalent conversion of any system of ordinary differential equations into a power-law format [31]. However, this equivalence transformation imposes constraints on the variables of the GMA equations, and it is at this point unclear whether there are mathematical warranties ensuring that the proposed transition from differential to stochastic equations in general preserves these constraints in all cases. This question will require further investigation.

### Stochastic Focusing

Stochastic focusing [26] describes the phenomenon that the fluctuations of a chemical species can drive the system to reach a different steady state than what a deterministic ODE model predicts. To demonstrate the utility of propensity adjustment, we derive a stochastic model which produces consistent results with those of the deterministic model.

*k*

_{1}from some source

*Φ*and degrades with rate

*k*

_{4}through the catalysis with signalling molecule S; the end product P is converted from species I at rate

*k*

_{2}and degrades at rate

*k*

_{3}; the signalling molecule S is produced and degrades at rates

*k*

_{5}and

*k*

_{6}, respectively. Moreover, the value of

*k*

_{5}is reduced to half at a certain time point to achieve a significant divergence effect. In order to capture the average dynamics of the system accurately, we use a power-law model in GMA format instead of the mass action rate law in [32].

*i*and

*s*contribute to the degradation of

*I*and that their covariance could therefore affect the propensity function of

*I*in the degradation reaction of a stochastic simulation. To calculate the propensity adjustment function

*paf*

_{4}(

*t*) = exp (-

*f*

_{ I }

*f*

_{ S }cov [log

*I*(

*t*), log

*S*(

*t*)]) for reaction ${R}_{4}:I+S\underset{}{\overset{{k}_{4}}{\to}}S$, we formulate equations (

*cf*. (60)) for the moments as

Here *μ* = (*μ*_{
I
}, *μ*_{
P
}, *μ*_{
S
})^{
T
}, $V=\left[\begin{array}{ccc}\hfill 1\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill -1\hfill & \hfill 1\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill -1\hfill & \hfill 0\hfill \\ \hfill -1\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill -1\hfill \end{array}\right]$, $\alpha =\left[\begin{array}{c}\hfill {\alpha}_{1}\hfill \\ \hfill {\alpha}_{2}\hfill \\ \hfill {\alpha}_{3}\hfill \\ \hfill {\alpha}_{4}\hfill \\ \hfill {\alpha}_{5}\hfill \\ \hfill {\alpha}_{6}\hfill \end{array}\right]=\left[\begin{array}{c}\hfill {k}_{1}\hfill \\ \hfill {k}_{2}i\hfill \\ \hfill {k}_{3}p\hfill \\ \hfill {k}_{4}{i}^{{f}_{I}}{s}^{{f}_{S}}\hfill \\ \hfill {k}_{5}\hfill \\ \hfill {k}_{6}s\hfill <\end{array}\right]$