Modeling and Analysis of Stochastic Systems

Contents

    1. Introduction
      • What in the World Is a Stochastic Process?
      • How to Characterize a Stochastic Process?
      • What Does One Do with a Stochastic Process?
    2. Discrete-Time Markov Chains: Transient Behavior
      • Definitions and Characterization
      • Examples
      • DTMC in Other Fields
      • Marginal Distributions
      • Occupancy Times
      • Computation of Matrix Powers
    3. DTMCs: First Passage Times
      • Definitions
      • Cumulative Distribution Function of T
      • Absorption Probabilities
      • Expectation of T
      • Generating Function and Higher Moments of T
    4. DTMCs: Limiting Behavior
      • Exploring the Limiting Behavior by Examples
      • Irreducibility and Periodicity
      • Recurrence and Transience
      • Determining Recurrence and Transience: Infinite DTMCs
      • Limiting Behavior of Irreducible DTMCs
      • Examples: Limiting Behavior of Infinite State-Space Irreducible DTMCs
      • Limiting Behavior of Reducible DTMCs
      • DTMCs with Costs and Rewards
      • Reversibility
    5. Poisson Processes
      • Exponential Distributions
      • Poisson Process: Definitions
      • Event Times in a Poisson Process
      • Superposition and Splitting of Poisson Processes
      • Non-Homogenous Poisson Process
      • Compound Poisson Process
    6. Continuous-Time Markov Chains
      • Definitions and Sample Path Properties
      • Examples
      • Transient Behavior: Marginal Distribution
      • Transient Behavior: Occupancy Times
      • Computation of P(t): Finite State-Space
      • Computation of P(t): Infinite State-Space
      • First-Passage Times
      • Exploring the Limiting Behavior by Examples
      • Classification of States
      • Limiting Behavior of Irreducible CTMCs
      • Limiting Behavior of Reducible CTMCs
      • CTMCs with Costs and Rewards
      • Phase-Type Distributions
      • Reversibility
    7. Queueing Models
      • Introduction
      • Properties of General Queueing Systems
      • Birth and Death Queues
      • Open Queueing Networks
      • Closed Queueing Networks
      • Single Server Queues
      • Retrial Queue
      • Infinite Server Queue
    8. Renewal Processes
      • Introduction
      • Properties of N(t)
      • The Renewal Function
      • Renewal-Type Equation
      • Key Renewal Theorem
      • Recurrence Times
      • Delayed Renewal Processes
      • Alternating Renewal Processes
      • Semi-Markov Processes
      • Renewal Processes with Costs/Rewards
      • Regenerative Processes
    9. Markov Regenerative Processes
      • Definitions and Examples
      • Markov Renewal Process and Markov Renewal Function
      • Key Renewal Theorem for MRPs
      • Extended Key Renewal Theorem
      • Semi-Markov Processes: Further Results
      • Markov Regenerative Processes
      • Applications to Queues
    10. Diffusion Processes
      • Brownian Motion
      • Sample Path Properties of BM
      • Kolmogorov Equations for Standard Brownian Motion
      • First Passage Times
      • Reflected SBM
      • Reflected BM and Limiting Distributions
      • BM and Martingales
      • Cost/Reward Models
      • Stochastic Integration
      • Stochastic Differential Equations
      • Applications to Finance

Epilogue

Appendix A: Probability of Events

Appendix B: Univariate Random Variables

Appendix C: Multivariate Random Variables

Appendix D: Generating Functions

Appendix E: Laplace–Stieltjes Transforms

Appendix F: Laplace Transforms

Appendix G: Modes of Convergence

Appendix H: Results from Analysis

Appendix I: Difference and Differential Equations

Answers to Selected Problems

References

Index

Exercises appear at the end of each chapter.