Renewal theory is the branch of probability theory that generalizes the Poisson process for arbitrary holding times. Instead of exponentially distributed holding times, a renewal process may have any independent and identically distributed (IID) holding times that have finite mean. A renewal-reward process additionally has a random sequence of rewards incurred at each holding time, which are IID but need not be independent of the holding times.

A renewal process has asymptotic properties analogous to the strong law of large numbers and central limit theorem. The renewal function ${\displaystyle m(t)}$ (expected number of arrivals) and reward function ${\displaystyle g(t)}$ (expected reward value) are of key importance in renewal theory. The renewal function satisfies a recursive integral equation, the renewal equation. The key renewal equation gives the limiting value of the convolution of ${\displaystyle m'(t)}$ with a suitable non-negative function. The superposition of renewal processes can be studied as a special case of Markov renewal processes.

Applications include calculating the best strategy for replacing worn-out machinery in a factory and comparing the long-term benefits of different insurance policies. The inspection paradox relates to the fact that observing a renewal interval at time t gives an interval with average value larger than that of an average renewal interval.

Renewal processes

Formal definition

Sample evolution of a renewal process with holding times S_i and jump times J_n.

Let ${\displaystyle (S_{i})_{i\geq 1}}$ be a sequence of positive independent identically distributed random variables with finite expected value

${\displaystyle 0<\operatorname {E} [S_{i}]<\infty .}$

We refer to the random variable ${\displaystyle S_{i}}$ as the "${\displaystyle i}$-th holding time".

Define for each n > 0 :

${\displaystyle J_{n}=\sum _{i=1}^{n}S_{i},}$

each ${\displaystyle J_{n}}$ is referred to as the "${\displaystyle n}$-th jump time" and the intervals ${\displaystyle [J_{n},J_{n+1}]}$ are called "renewal intervals".

Then ${\displaystyle (X_{t})_{t\geq 0}}$ is given by random variable

${\displaystyle X_{t}=\sum _{n=1}^{\infty }\operatorname {\mathbb {I} } _{\{J_{n}\leq t\}}=\sup \left\{\,n:J_{n}\leq t\,\right\}}$

where ${\displaystyle \operatorname {\mathbb {I} } _{\{J_{n}\leq t\}}}$ is the indicator function

${\displaystyle \operatorname {\mathbb {I} } _{\{J_{n}\leq t\}}={\begin{cases}1,&{\text{if }}J_{n}\leq t\\0,&{\text{otherwise}}\end{cases}}}$

${\displaystyle (X_{t})_{t\geq 0}}$ represents the number of jumps that have occurred by time t, and is called a renewal process.

Renewal-reward processes

Sample evolution of a renewal-reward process with holding times S_i, jump times J_n and rewards W_i

Let ${\displaystyle W_{1},W_{2},\ldots }$ be a sequence of IID random variables (rewards) satisfying

${\displaystyle \operatorname {E} |W_{i}|<\infty .\,}$

Then the random variable

${\displaystyle Y_{t}=\sum _{i=1}^{X_{t}}W_{i}}$

is called a renewal-reward process. Note that unlike the ${\displaystyle S_{i}}$, each ${\displaystyle W_{i}}$ may take negative values as well as positive values.

The random variable ${\displaystyle Y_{t}}$ depends on two sequences: the holding times ${\displaystyle S_{1},S_{2},\ldots }$ and the rewards ${\displaystyle W_{1},W_{2},\ldots }$ These two sequences need not be independent. In particular, ${\displaystyle W_{i}}$ may be a function of ${\displaystyle S_{i}}$.

Renewal function

We define the renewal function as the expected value of the number of jumps observed up to some time ${\displaystyle t}$:

${\displaystyle m(t)=\operatorname {E} [X_{t}].\,}$

Key renewal theorem

Let X be a renewal process with renewal function ${\displaystyle m(t)}$ and interrenewal mean ${\displaystyle \mu }$. Let ${\displaystyle g:[0,\infty )\rightarrow [0,\infty )}$ be a function satisfying:

• ${\displaystyle \int _{0}^{\infty }g(t)\,dt<\infty }$
• g is monotone and non-increasing

The key renewal theorem states that, as ${\displaystyle t\rightarrow \infty }$:^[3]

${\displaystyle \int _{0}^{t}g(t-x)m'(x)\,dx\rightarrow {\frac {1}{\mu }}\int _{0}^{\infty }g(x)\,dx}$

Asymptotic properties

Renewal processes and renewal-reward processes have properties analogous to the strong law of large numbers, which can be derived from the same theorem. If ${\displaystyle (X_{t})_{t\geq 0}}$ is a renewal process and ${\displaystyle (Y_{t})_{t\geq 0}}$ is a renewal-reward process then:

${\displaystyle \lim _{t\to \infty }{\frac {1}{t}}X_{t}={\frac {1}{\operatorname {E} [S_{1}]}}}$^[6]

${\displaystyle \lim _{t\to \infty }{\frac {1}{t}}Y_{t}={\frac {1}{\operatorname {E} [S_{1}]}}\operatorname {E} [W_{1}]}$

almost surely.

Proof

First consider ${\displaystyle (X_{t})_{t\geq 0}}$. By definition we have: ${\displaystyle J_{X_{t}}\leq t\leq J_{X_{t}+1}}$ for all ${\displaystyle t\geq 0}$ and so ${\displaystyle {\frac {J_{X_{t}}}{X_{t}}}\leq {\frac {t}{X_{t}}}\leq {\frac {J_{X_{t}+1}}{X_{t}}}}$ for all t ≥ 0. Now since ${\displaystyle 0<\operatorname {E} [S_{i}]<\infty }$ we have: ${\displaystyle X_{t}\to \infty }$ as ${\displaystyle t\to \infty }$ almost surely (with probability 1). Hence: ${\displaystyle {\frac {J_{X_{t}}}{X_{t}}}={\frac {J_{n}}{n}}={\frac {1}{n}}\sum _{i=1}^{n}S_{i}\to \operatorname {E} [S_{1}]}$ almost surely (using the strong law of large numbers); similarly: ${\displaystyle {\frac {J_{X_{t}+1}}{X_{t}}}={\frac {J_{X_{t}+1}}{X_{t}+1}}{\frac {X_{t}+1}{X_{t}}}={\frac {J_{n+1}}{n+1}}{\frac {n+1}{n}}\to \operatorname {E} [S_{1}]\cdot 1}$ almost surely. Thus (since ${\displaystyle t/X_{t}}$ is sandwiched between the two terms) ${\displaystyle {\frac {1}{t}}X_{t}\to {\frac {1}{\operatorname {E} [S_{1}]}}}$ almost surely.[3] Next consider ${\displaystyle (Y_{t})_{t\geq 0}}$. We have ${\displaystyle {\frac {1}{t}}Y_{t}={\frac {X_{t}}{t}}{\frac {1}{X_{t}}}Y_{t}\to {\frac {1}{\operatorname {E} [S_{1}]}}\cdot \operatorname {E} [W_{1}]}$ almost surely (using the first result and using the law of large numbers on ${\displaystyle Y_{t}}$).

Renewal processes additionally have a property analogous to the central limit theorem:^[6]

${\displaystyle {\frac {X_{t}-t/\mu }{\sqrt {t\sigma ^{2}/\mu ^{3}}}}}$

Inspection paradox

The renewal interval determined by the random point t (shown in red) is stochastically larger than the first renewal interval.

A curious feature of renewal processes is that if we wait some predetermined time t and then observe how large the renewal interval containing t is, we should expect it to be typically larger than a renewal interval of average size.

Mathematically the inspection paradox states: for any t > 0 the renewal interval containing t is stochastically larger than the first renewal interval. That is, for all x > 0 and for all t > 0:

${\displaystyle \operatorname {P} (S_{X_{t}+1}>x)\geq \operatorname {P} (S_{1}>x)=1-F_{S}(x)}$

where F_S is the cumulative distribution function of the IID holding times S_i. A vivid example is the bus waiting time paradox: For a given random distribution of bus arrivals, the average rider at a bus stop observes more delays than the average operator of the buses.

The resolution of the paradox is that our sampled distribution at time t is size-biased (see sampling bias), in that the likelihood an interval is chosen is proportional to its size. However, a renewal interval of average size is not size-biased.

Proof

Observe that the last jump-time before t is ${\displaystyle J_{X_{t}}}$; and that the renewal interval containing t is ${\displaystyle S_{X_{t}+1}}$. Then ${\displaystyle {\begin{aligned}\operatorname {P} (S_{X_{t}+1}>x)&{}=\int _{0}^{\infty }\operatorname {P} (S_{X_{t}+1}>x\mid J_{X_{t}}=s)f_{J_{X_{t}}}(s)\,ds\\[12pt]&{}=\int _{0}^{\infty }\operatorname {P} (S_{X_{t}+1}>x|S_{X_{t}+1}>t-s)f_{J_{X_{t}}}(s)\,ds\\[12pt]&{}=\int _{0}^{\infty }{\frac {\operatorname {P} (S_{X_{t}+1}>x\,,\,S_{X_{t}+1}>t-s)}{\operatorname {P} (S_{X_{t}+1}>t-s)}}f_{J_{X_{t}}}(s)\,ds\\[12pt]&{}=\int _{0}^{\infty }{\frac {1-F(\max\{x,t-s\})}{1-F(t-s)}}f_{J_{X_{t}}}(s)\,ds\\[12pt]&{}=\int _{0}^{\infty }\min \left\{{\frac {1-F(x)}{1-F(t-s)}},{\frac {1-F(t-s)}{1-F(t-s)}}\right\}f_{J_{X_{t}}}(s)\,ds\\[12pt]&{}=\int _{0}^{\infty }\min \left\{{\frac {1-F(x)}{1-F(t-s)}},1\right\}f_{J_{X_{t}}}(s)\,ds\\[12pt]&{}\geq \int _{0}^{\infty }(1-F(x))f_{J_{X_{t}}}(s)\,ds=1-F(x)=\operatorname {P} (S_{1}>x),\\[12pt]\end{aligned}}}$ since both ${\displaystyle {\frac {1-F(x)}{1-F(t-s)}}}$ and ${\displaystyle 1}$ are greater than or equal to ${\displaystyle 1-F(x)}$ for all values of s.

Superposition

Unless the renewal process is a Poisson process, the superposition (sum) of two independent renewal processes is not a renewal process.^[7] However, such processes can be described within a larger class of processes called the Markov-renewal processes.^[8] However, the cumulative distribution function of the first inter-event time in the superposition process is given by^[9]

${\displaystyle R(t)=1-\sum _{k=1}^{K}{\frac {\alpha _{k}}{\sum _{l=1}^{K}\alpha _{l}}}(1-R_{k}(t))\prod _{j=1,j\neq k}^{K}\alpha _{j}\int _{t}^{\infty }(1-R_{j}(u))\,{\text{d}}u}$

where R_k(t) and α_k > 0 are the CDF of the inter-event times and the arrival rate of process k.^[10]

Example application

Eric the entrepreneur has n machines, each having an operational lifetime uniformly distributed between zero and two years. Eric may let each machine run until it fails with replacement cost €2600; alternatively he may replace a machine at any time while it is still functional at a cost of €200.

What is his optimal replacement policy?

Solution

The lifetime of the n machines can be modeled as n independent concurrent renewal-reward processes, so it is sufficient to consider the case n=1. Denote this process by ${\displaystyle (Y_{t})_{t\geq 0}}$. The successive lifetimes S of the replacement machines are independent and identically distributed, so the optimal policy is the same for all replacement machines in the process. If Eric decides at the start of a machine's life to replace it at time 0 < t < 2 but the machine happens to fail before that time then the lifetime S of the machine is uniformly distributed on [0, t] and thus has expectation 0.5t. So the overall expected lifetime of the machine is: ${\displaystyle {\begin{aligned}\operatorname {E} [S]&=\operatorname {E} [S\mid {\text{fails before }}t]\cdot \operatorname {P} [{\text{fails before }}t]+\operatorname {E} [S\mid {\text{does not fail before }}t]\cdot \operatorname {P} [{\text{does not fail before }}t]\\[6pt]&=0.5t({\frac {t}{2}})+t({\frac {2-t}{2}})\end{aligned}}}$ and the expected cost W per machine is: ${\displaystyle {\begin{aligned}\operatorname {E} [W]&=\operatorname {E} [W\mid {\text{fails before }}t]\cdot \operatorname {P} ({\text{fails before }}t)+\operatorname {E} [W\mid {\text{does not fail before }}t]\cdot \operatorname {P} ({\text{does not fail before }}t)\\[6pt]&=2600({\frac {t}{2}})+200({\frac {2-t}{2}})=1200t+200.\end{aligned}}}$ So by the strong law of large numbers, his long-term average cost per unit time is: ${\displaystyle {\frac {1}{t}}Y_{t}\simeq {\frac {\operatorname {E} [W]}{\operatorname {E} [S]}}={\frac {4(1200t+200)}{t^{2}+4t-2t^{2}}}}$ then differentiating with respect to t: ${\displaystyle {\frac {\partial }{\partial t}}{\frac {4(1200t+200)}{t^{2}+4t-2t^{2}}}=4{\frac {(4t-t^{2})(1200)-(4-2t)(1200t+200)}{(t^{2}+4t-2t^{2})^{2}}},}$ this implies that the turning points satisfy: ${\displaystyle {\begin{aligned}0&=(4t-t^{2})(1200)-(4-2t)(1200t+200)=4800t-1200t^{2}-4800t-800+2400t^{2}+400t\\[6pt]&=-800+400t+1200t^{2},\end{aligned}}}$ and thus ${\displaystyle 0=3t^{2}+t-2=(3t-2)(t+1).}$ We take the only solution t in [0, 2]: t = 2/3. This is indeed a minimum (and not a maximum) since the cost per unit time tends to infinity as t tends to zero, meaning that the cost is decreasing as t increases, until the point 2/3 where it starts to increase.

See also

Notes

References

• Cox, David (1970). Renewal Theory. London: Methuen & Co. p. 142. ISBN 0-412-20570-X.
• Doob, J. L. (1948). "Renewal Theory From the Point of View of the Theory of Probability" (PDF). Transactions of the American Mathematical Society. 63 (3): 422–438. doi:10.2307/1990567. JSTOR 1990567.
• Feller, William (1971). An introduction to probability theory and its applications. Vol. 2 (second ed.). Wiley.
• Grimmett, G. R.; Stirzaker, D. R. (1992). Probability and Random Processes (second ed.). Oxford University Press. ISBN 0198572220.
• Smith, Walter L. (1958). "Renewal Theory and Its Ramifications". Journal of the Royal Statistical Society, Series B. 20 (2): 243–302. JSTOR 2983891.
• Wanli Wang, Johannes H. P. Schulz, Weihua Deng, and Eli Barkai (2018). "Renewal theory with fat-tailed distributed sojourn times: Typical versus rare". Phys. Rev. E. 98 (4): 042139. arXiv:1809.05856. Bibcode:2018PhRvE..98d2139W. doi:10.1103/PhysRevE.98.042139. S2CID 54727926.{{cite journal}}: CS1 maint: multiple names: authors list (link)