### Citations

1845 |
Markov Decision Processes: Discrete Stochastic Dynamic Programming
- Puterman
- 2005
(Show Context)
Citation Context ... · at−1 and Q0 · · · , Qt, while that is Markov if at only depends on Qt. A policy is stationary if at1 = at2 when Qt1 = Qt2 for some t1, t2. In general, a policy belongs to one of the following sets =-=[15]-=-: • ΠHR: a set of randomized history dependent policies; • ΠMR: a set of randomized Markov policies; • ΠSR: a set of randomized stationary policies; • ΠSD: a set of deterministic stationary policies. ... |

473 |
Dynamic Programming, Deterministic and Stochastic Models
- Bertsekas
- 1987
(Show Context)
Citation Context ...o clear methodology to find optimal policies for the problems that possess the proprieties of infinite horizon, average cost optimization, and have a countably infinite state space. Indeed, reference =-=[18]-=- remarks that it is difficult to analyze and obtain optimal policies for such problems. The works in [19–22] contribute to the analysis of MDPs with countably infinite state space. Moreover, reference... |

346 | Introduction to Stochastic Dynamic Programming - Ross - 1983 |

170 |
Control Techniques for Complex Networks
- Meyn
(Show Context)
Citation Context ...by applying the stationary and deterministic threshold policy to the MDP is positive recurrent, i.e., the stationary distribution exists. Proof: (Sketch) The proof is based on Foster-Lyapunov theorem =-=[25]-=- associated with the Lyapunov function L(x, y) = x2 + y2. We realize that if E[A2i ] < ∞ and E[Ai] < 1 for i = 1, 2, then there exists a stationary threshold type policy that is average-optimal and ca... |

100 |
Stochastic dynamic programming and the control of queueing systems
- Sennott
- 1999
(Show Context)
Citation Context ...ero to begin with. Our goal is to characterize and obtain the average-optimal policy, i.e., the policy that minimizes V (θ). It is not hard to see (as shown in [15]) that ΠSD ⊂ ΠSR ⊂ ΠMR ⊂ ΠHR. As in =-=[15, 17]-=- there might not exist a SR or SD policy that is optimal, in what regime does the average-optimal policy lie? We first describe the probability law for our MDP and then in subsequent sections develop ... |

70 | Dynamic algorithms for multicast with intra-session network coding
- Ho, Viswanathan
- 2005
(Show Context)
Citation Context ...e intersession network coding is allowed. Reddy et al. [8] have showed how to design coding-aware routing controllers that would maximize coding opportunities in multihop networks. References [9] and =-=[10]-=- attempt to schedule the network coding between multiplesession flows. Xi and Yeh [11] propose a distributed algorithm that minimizes the transmission cost of a multicast session. References [12–14] a... |

44 |
Trading Structure for Randomness
- Chachulski, Jennings, et al.
- 2007
(Show Context)
Citation Context ...nique for wireless networks has been considered by Katti et al. [3]. They propose an architecture, referred to as COPE, which contains a special network coding layer between the IP and MAC layers. In =-=[4]-=-, an opportunistic routing protocol is proposed, referred to as MORE, that randomly mixes packets that belong to the same flow before forwarding them to the next hop. Sagduyu et al. [5] characterize t... |

39 | A tiling approach to network code design for wireless networks
- Effros, Ho, et al.
- 2006
(Show Context)
Citation Context ...tially supported by the NSF under grants CNS0954153, CNS-0963818, CNS-0904520 and CNS-1149458, by the AFOSR under contract No. FA9550-13-1-0008 and the DTRA under grant HDTRA113-1-0030. Effros et al. =-=[1]-=- introduced the strategy of reverse carpooling that allows two opposite information flows share bandwidth along a shared path. Fig. 1(b) shows an example of two connections, from n1 to n4 and from n4 ... |

32 | Average optimality in dynamic programming with general state space - Schal - 1993 |

31 |
Convex Analytic Methods in Markov Decision Processes
- Borkar
- 2002
(Show Context)
Citation Context ...heorem 6. For the MDP{(Qt, at), t ≥ 0}, there exists a stationary and deterministic policy θ∗ that minimizes V (θ) if E[A2i ] <∞ and E[Ai] < 1 for i = 1, 2. Proof: See Appendix A. According to Borkar =-=[24]-=-, it is possible to find the randomized policy that is closed to the average-optimal by applying linear programming methods for an MDP of a very generic setting, where randomized stationary policies a... |

29 |
Average cost optimal stationary policies in infinite state Markov decision processes with unbounded costs
- Sennott
- 1989
(Show Context)
Citation Context ...rministic. We begin by considering the infinite horizon α-discounted cost case, where 0 < α < 1, which we then tie to the average cost case. This method is typically used in the MDP literature (e.g., =-=[22]-=-), where the conditions for the structure of the average-optimal policy usually rely on the results of the infinite horizon α-discounted cost case. For our MDP{(Qt, at), t ≥ 0}, the total expected dis... |

28 | Cross-layer optimization for wireless multihop networks with pairwise intersession network coding
- Khreishah, Chih-chun, et al.
- 2009
(Show Context)
Citation Context ...s in the network coding enabled networks. Sagduyu and Ephremides [6] focus on the network coding in the tandem networks and formulate related cross-layer optimization problems, while Khreishah et al. =-=[7]-=- devise a joint codingscheduling-rate controller when the pairwise intersession network coding is allowed. Reddy et al. [8] have showed how to design coding-aware routing controllers that would maximi... |

27 | Distributed algorithms for minimum cost multicast with network coding
- Xi, Yeh
- 2005
(Show Context)
Citation Context ...oding-aware routing controllers that would maximize coding opportunities in multihop networks. References [9] and [10] attempt to schedule the network coding between multiplesession flows. Xi and Yeh =-=[11]-=- propose a distributed algorithm that minimizes the transmission cost of a multicast session. References [12–14] analyze the similar trade-off between power consumption and packet delays from differen... |

20 |
Multimedia Wireless Transmission with Network Coding
- Nguyen, Nguyen, et al.
- 2007
(Show Context)
Citation Context ...perspectives. Ciftcioglu et al. [12] propose a threshold policy using the Lyapunov technique. The threshold policy in [12] is an approximate solution with some performance guarantees. Nguyen and Yang =-=[13]-=- present a basic Markov decision process (MDP) framework for the problem at hand. Huang et al. [14] analyze the performance of the transport protocols over meshed networks as well as several implement... |

17 | Cross-layer optimization of MAC and network coding in wireless queueing tandem networks
- Sagduyu, Ephremides
- 2008
(Show Context)
Citation Context ...r broadcast networks with erasure channels. In addition, several works, e.g., [6–11], investigate the scheduling and/or routing problems in the network coding enabled networks. Sagduyu and Ephremides =-=[6]-=- focus on the network coding in the tandem networks and formulate related cross-layer optimization problems, while Khreishah et al. [7] devise a joint codingscheduling-rate controller when the pairwis... |

15 | Control of Markov *chains with long-run average cost criterion: the dynamic programming equations - BORKAR - 1989 |

13 | Multipath wireless network coding: a population game perspective
- Reddy, Shakkottai, et al.
- 2010
(Show Context)
Citation Context ... formulate related cross-layer optimization problems, while Khreishah et al. [7] devise a joint codingscheduling-rate controller when the pairwise intersession network coding is allowed. Reddy et al. =-=[8]-=- have showed how to design coding-aware routing controllers that would maximize coding opportunities in multihop networks. References [9] and [10] attempt to schedule the network coding between multip... |

13 | Monotonicity in markov reward and decision chains: Theory and applications - Koole - 2006 |

7 |
Control of multi-hop communication networks for inter-session network coding
- Eryilmaz, Lun, et al.
- 2011
(Show Context)
Citation Context ... pairwise intersession network coding is allowed. Reddy et al. [8] have showed how to design coding-aware routing controllers that would maximize coding opportunities in multihop networks. References =-=[9]-=- and [10] attempt to schedule the network coding between multiplesession flows. Xi and Yeh [11] propose a distributed algorithm that minimizes the transmission cost of a multicast session. References ... |

6 | Product form stationary distributions for diffusion approximations to a flow-level model operating under a proportional fair sharing policy
- Kang, Kelly, et al.
- 2007
(Show Context)
Citation Context ...ntly different from Bernoulli. As expected, for the single relay network, the QLT policy has the optimal performance and the QL+WT policy does not have any advantage. Moreover, there are results (see =-=[26]-=-) that indicate that the independent arrivals model is accurate under heavy traffic for multi-hop networks. Hence, our characterization of the optimal policy does have value in a more general case. Ou... |

4 |
Capacity and stable throughput regions for the broadcast erasure channel with feedback: An unusual union,” Information Theory
- Sagduyu, Georgiadis, et al.
- 2013
(Show Context)
Citation Context ...MAC layers. In [4], an opportunistic routing protocol is proposed, referred to as MORE, that randomly mixes packets that belong to the same flow before forwarding them to the next hop. Sagduyu et al. =-=[5]-=- characterize the capacity region for the similar broadcast networks with erasure channels. In addition, several works, e.g., [6–11], investigate the scheduling and/or routing problems in the network ... |

4 | Cost-delay tradeoffs for two-way relay networks
- Ciftcioglu, Sagduyu, et al.
- 2011
(Show Context)
Citation Context ...hat minimizes the transmission cost of a multicast session. References [12–14] analyze the similar trade-off between power consumption and packet delays from different perspectives. Ciftcioglu et al. =-=[12]-=- propose a threshold policy using the Lyapunov technique. The threshold policy in [12] is an approximate solution with some performance guarantees. Nguyen and Yang [13] present a basic Markov decision... |

1 | Comparing Recent Assumptions for the Existence - Cavazos-Cadena, Sennott - 1992 |

1 |
He was with the Center for Information and Communications Technology in the National Chiao Tung University, as a research assistant between July 2007 and August 2009. Since October 2014, he has been a joint postdoctoral fellow with the Massachusetts Insti
- Resnick
- 1992
(Show Context)
Citation Context ...ransit to the states in the CSi and CSi−1. Case (ii): p(1)0 + p (1) 1 = 1 and p (2) 0 + p (2) 1 < 1. Note that θ̃ results in a reducible Markov chain. That is, there are several communicating classes =-=[27]-=-. We define the classes CS1 = {(a, b) : a = 0, 1 and b ∈ N ∪ {0}} and CSi = {(a, b) : a = i, b ∈ N∪{0}} for i ≥ 2, as shown in Fig. 9. Then each CSi is a communicating class under the policy θ̃. The s... |