The inference of gene regulatory networks is a key issue for genomic signal processing. This paper addresses the inference of probabilistic Boolean networks (PBNs) from observed temporal sequences of network states. Since a PBN is composed of a finite number of Boolean networks, a basic observation is that the characteristics of a single Boolean network without perturbation may be determined by its pairwise transitions. Because the network function is fixed and there are no perturbations, a given state will always be followed by a unique state at the succeeding time point. Thus, a transition counting matrix compiled over a data sequence will be sparse and contain only one entry per line. If the network also has perturbations, with small perturbation probability, then the transition counting matrix would have some insignificant nonzero entries replacing some (or all) of the zeros. If a data sequence is sufficiently long to adequately populate the matrix, then determination of the functions and inputs underlying the model is straightforward. The difficulty comes when the transition counting matrix consists of data derived from more than one Boolean network. We address the PBN inference procedure in several steps: (1) separate the data sequence into “pure ” subsequences corresponding to constituent Boolean networks; (2) given a subsequence, infer a Boolean network; and (3) infer the probabilities of perturbation, the probability of there being a switch between constituent Boolean networks, and the selection probabilities governing

Abstract—Probabilistic Boolean networks (PBNs) have been recently introduced as a paradigm for modeling genetic regulatory networks. One of the objectives of PBN modeling is to use the network for the design and analysis of intervention strategies aimed at moving the network out of undesirable states, such as those associated with disease, and into desirable ones. To date, a number of intervention strategies have been proposed in the context of PBNs. However, all these techniques assume perfect knowledge of the transition probability matrix of the PBN. Such an assumption cannot be satisfied in practice since the presence of noise and the availability of limited number of samples will prevent the transition probabilities from being accurately deter-mined. Moreover, even if the exact transition probabilities could be estimated from the data, mismatch between the PBN model and the actual genetic regulatory network will invariably be present. Thus, it is important to study the effect of modeling errors on the final outcome of an intervention strategy and one of the goals of this paper is to do precisely that when the uncertainties are in the entries of the transition probability matrix. In addition, the paper develops a robust intervention strategy that is obtained by minimizing the worst-case cost over the uncertainty set. Index Terms—Control of biological networks, estimation errors, robust dynamic programming, robust minimax control, perturba-tion bounds. I.

Abstract—A prime objective of modeling genetic regulatory net-works is the identification of potential targets for therapeutic inter-vention. To date, optimal stochastic intervention has been studied in the context of probabilistic Boolean networks, with the con-trol policy based on the transition probability matrix of the as-sociated Markov chain and dynamic programming used to find optimal control policies. Dynamical programming algorithms are problematic owing to their high computational complexity. Two additional computationally burdensome issues that arise are the potential for controlling the network and identifying the best gene for intervention. This paper proposes an algorithm based on mean first-passage time that assigns a stationary control policy for each gene candidate. It serves as an approximation to an optimal control policy and, owing to its reduced computational complexity, can be used to predict the best control gene. Once the best control gene is identified, one can derive an optimal policy or simply utilize the approximate policy for this gene when the network size precludes a direct application of dynamic programming algorithms. A salient point is that the proposed algorithm can be model-free. It can be directly designed from time-course data without having to infer the transition probability matrix of the network. Index Terms—Dynamic programming, genetic regulatory networks, mean first-passage time, probabilistic Boolean networks, stochastic optimal control. I.

An approximate representation for the state space of a context-sensitive probabilistic Boolean network has previously been proposed and utilized to devise therapeutic intervention strategies. Whereas the full state of a context-sensitive probabilistic Boolean network is specified by an ordered pair composed of a network context and a gene-activity profile, this approximate representation collapses the state space onto the gene-activity profiles alone. This reduction yields an approximate transition probability matrix, absent of context, for the Markov chain associated with the context-sensitive probabilistic Boolean network. As with many approximation methods, a price must be paid for using a reduced model representation, namely, some loss of optimality relative to using the full state space. This paper examines the effects on intervention performance caused by the reduction with respect to various values of the model parameters. This task is performed using a new derivation for the transition probability matrix of the context-sensitive probabilistic Boolean network. This expression of transition probability distributions is in concert with the original definition of context-sensitive probabilistic Boolean network. The performance of optimal and approximate therapeutic strategies is compared for both synthetic networks and a real case study. It is observed that the approximate representation describes the dynamics of the context-sensitive probabilistic Boolean network through the instantaneously random probabilistic Boolean network with similar parameters.

Abstract—There is an ongoing effort to design optimal inter-vention strategies for discrete state-space synchronous genetic regulatory networks in the context of probabilistic Boolean net-works; however, to date, there has been no corresponding effort for asynchronous networks. This paper addresses this issue by postulating two asynchronous extensions of probabilistic Boolean networks and developing control policies for both. The first extension introduces deterministic gene-level asynchronism into the constituent Boolean networks of the probabilistic Boolean network, thereby providing the ability to cope with temporal context sensitivity. The second extension introduces asynchronism at the level of the gene activity profiles. Whereas control policies for both standard probabilistic Boolean networks and the first proposed extension are characterized within the framework of Markov decision processes, asynchronism at the profile level results in control being treated in the framework of semi-Markov decision processes. The advantage of the second model is the ability to obtain the necessary timing information from sequences of gene-activity profile measurements. Results from the theory of stochastic control are leveraged to determine optimal intervention strategies for each class of proposed asynchronous regulatory networks, the objective being to reduce the time duration that the system spends in undesirable states. Index Terms—Asynchronous genetic regulatory networks, op-timal stochastic control, semi-Markov decision processes, transla-tional genomics. I.

A key objective of gene network modeling is to develop intervention strategies to alter regulatory dynamics in such a way as to reduce the likelihood of undesirable phenotypes. Optimal stationary intervention policies have been developed for gene regulation in the framework of probabilistic Boolean networks in a number of settings. To mitigate the possibility of detrimental side effects, for instance, in the treatment of cancer, it may be desirable to limit the expected number of treatments beneath some bound. This paper formulates a general constraint approach for optimal therapeutic intervention by suitably adapting the reward function and then applies this formulation to bound the expected number of treatments. A mutated mammalian cell cycle is considered as a case study.

Abstract—A salient purpose for studying gene regulatory networks is to derive intervention strategies to identify potential drug targets and design gene-based therapeutic intervention. Optimal and approximate intervention strategies based on the transition probability matrix of the underlying Markov chain have been studied extensively for probabilistic Boolean networks. While the key goal of control is to reduce the steady-state probability mass of undesirable network states, in practice it is important to limit collateral damage and this constraint should be taken into account when designing intervention strategies with network models. In this paper, we propose two new phenotypically constrained stationary control policies by directly investigating the effects on the network long-run behavior. They are derived to reduce the risk of visiting undesirable states in conjunction with constraints on the shift of undesirable steady-state mass so that only limited collateral damage can be introduced. We have studied the performance of the new constrained control policies together with the previous greedy control policies to randomly generated probabilistic Boolean networks. A preliminary example for intervening in a metastatic melanoma network is also given to show their potential application in designing genetic therapeutics to reduce the risk of entering both aberrant phenotypes and other ambiguous states corresponding to complications or collateral damage. Experiments on both random network ensembles and the melanoma network demonstrate that, in general, the new proposed control policies exhibit the desired performance. As shown by intervening in the melanoma network, these control policies can potentially serve as future practical gene therapeutic intervention strategies. Index Terms—Gene regulatory networks, probabilistic Boolean networks, network intervention, Markov chain, stationary control policy, melanoma. Ç

Abstract In this paper, we study optimal control policies for Probabilistic Boolean Networks (PBNs) with hard constraints. Boolean Networks (BNs) and PBNs are useful and effective tools for modelling genetic regulatory networks. A PBN is essentially a collection of BNs driven by a Markov chain process. It is well-known that the control/intervention of a genetic regulatory network is useful for avoiding undesirable states associated with diseases like cancer. Therefore both optimal finite-horizon control and infinite-horizon control policies have been proposed to achieve the purpose. Actually the optimal control problem can be formulated as a probabilistic dynamic programming problem. In many studies, the optimal control problems did not consider the case of hard constraints, i.e., to include a maximum upper bound for the number of controls that can be applied to the PBN. The main objective of this paper is to introduce a new formulation for the optimal finite-horizon control problem with hard constraints. Experimental results are given to demonstrate the efficiency of our proposed formulation.

Abstract—External control of a genetic regulatory network is used for the purpose of avoiding undesirable states such as those associated with a disease. Certain types of cancer therapies, such as chemotherapy, are given in cycles with each treatment being followed by a recovery period. During the recovery period, the side effects tend to gradually subside. In this paper, it is shown how an optimal cyclic intervention strategy can be devised for any Markovian genetic regulatory network. The effectiveness of optimal cyclic therapies is demonstrated through numerical studies for random networks. Furthermore, an optimal cyclic policy is derived to control the behavior of a regulatory model of the mammalian cell-cycle network. Index Terms—Cyclic therapy, dynamic programming, genetic regulatory networks, probabilistic Boolean networks (PBNs), stochastic optimal control. I.

Abstract—Intervention in gene regulatory networks in the context of Markov decision processes has usually involved finding an optimal one-transition policy, where a decision is made at every transition whether or not to apply treatment. In an effort to model dosing constraint, a cyclic approach to intervention has previously been proposed in which there is a sequence of treatment windows and treatment is allowed only at the beginning of each window. This protocol ignores two practical aspects of therapy. First, a treatment typically has some duration of action: adrugwillbeeffectiveforsomeperiod, after which there can be a recovery phase. This, too, might involve a cyclic protocol; however, in practice, a physician might monitor a patient at every stage and decide whether to apply treatment, and if treatment is applied, then the patient will be under the influence of the drug for some duration, followed by a recovery period. This results in an acyclic protocol. In this paper we take a unified approach to both cyclic and acyclic control with duration of effectiveness by placing the problem in the general framework of multiperiod decision epochs with infinite horizon discounting cost. The time interval between successive decision epochs can have multiple time units, where given the current state and the action taken, there is a joint probability distribution defined for the next state and the time when the next decision epoch will be called. Optimal control policies are derived, synthetic networks are used to investigate the properties of both cyclic and acyclic interventions with fixed-duration of effectiveness, and the methodology is applied to a mutated mammalian cell-cycle network. Index Terms—Acyclic intervention, cyclic intervention, drug scheduling, gene regulatory network, genomic signal processing, optimal control. I.