Stochastic Control of Two Competing Queues

We consider optimal server time allocation to two parallel queues. The server has available complete past observations of the queue sizes for his decisions. The infinite time discounted version of the problem is analyzed here. It is shown that the optimal strategy is stationary. The optimal value function is shown to be the unique solution of the Bellman equation. Finally, analysis of degenerate Bellman equations, of the type appearing in this problem is presented. Numerical methods of solution can be derived from the results presented here.