Last modified on May 25th, 2024

chapter outline

 

Jensen’s Inequality

Jensen’s inequality relates the expected value of a convex or concave function to the function of their expected values.

It states that if ‘X’ is an integrable random variable and f: ℝ → ℝ is a convex function such that Y = f(X) is also integrable, then E[f(X)] ≥ f(E[X])

Thus, if a convex function is applied to the mean of a variable, it is less than or equal to the mean of that function applied to the variable. However, if ‘f’ is a concave function, the inequality reverses.

Jensen’s Inequality

Proof

Let us consider a convex function f(x).

Then, for any point x0, the graph of f(x) lies entirely above the tangent at the point ‘x0

f(x) ≥ f(x0) + b(x – x0), where ‘b’ is the slope of the tangent 

Now, considering x = X and x0 = E[X], we get

f(X) ≥ f(E[X]) + b(X – E[X])

Taking the expected values of both sides, we get

E[f(X)] ≥ E[f(E[X]) + b(X – E[X])]

By using the linear property of expected values, we get

E[f(X)] ≥ f(E[X]) + b(E[X] – E[X])

⇒ E[f(X)] ≥ f(E[X]) …..(i)

If the function f(x) is concave, then -f(x) is convex, and from the inequality (i), we get

E[-f(X)] ≥ -f(E[X])

Multiplying both sides by (-1), we get

E[f(X)] ≤ f(E[X]) …..(ii)

Thus, Jensen’s inequalities for convex and concave functions are proved.

Alternative Form

However, there is an alternative form of this inequality, where the expected value or mean E[X] is expressed as E[X] = a1x1 + a2x2 + … + anxn.

It implies:

f(E[X]) = f(a1x1 + a2x2 + … + anxn) and  E[f(X)] = a1f(x1) + a2f(x2) + … + anf(xn), for all a1, a2, …, an ≥ 0

In particular, if a1, a2, …, an ≥ 0 and ${\sum ^{n}_{i=1}a _{i}=1}$ for all λi, then

${\sum ^{n}_{i=1}a _{i}f\left( x_{i}\right) \geq f\left( \sum ^{n}_{i=1}a _{i}x_{i}\right)}$ 

In general, if a real-valued function ‘f’ is convex on the interval ‘I,’ and x1, x2, …, xn Є I, then for a1, a2, …, an ≥ 0,

${\dfrac{a_{1}f\left( x_{1}\right) +a_{2}f\left( x_{2}\right) +\ldots +a_{n}f\left( x_{n}\right) }{a_{1}+a_{2}+\ldots +a_{n}}\geq f\left( \dfrac{a_{1}x_{1}+a_{2}x_{2}+\ldots +a_{n}x_{n}}{a_{1}+a_{2}+\ldots +a_{n}}\right)}$

Conditions to Follow

To apply Jensen’s inequality, we use the following conditions of calculus.

If f: I → ℝ is a twice-differentiable function, then

  • ‘f’ is convex on ‘I,’ if and only if f”(x) ≥ 0, for all x Є I
  • ‘f’ is concave on ‘I,’ if and only if f”(x) ≤ 0, for all x Є I

Solved Examples

Use Jensen’s inequality to find the minimum of f(X) = 3X2 for a random variable X with the mean μ = 5

Solution:

Here, the function f(x) = 3X2 
Differentiating the function with respect to x, we get
f’(x) = 6X
f”(x) = 6 > 0
As we know, f”(x) ≥ 0 ⇔ f(x) is a convex function
Now, to find the minimum value of f(X) = 3X2, we find the lower bound on the expected value  E[f(X)].
Using Jensen’s inequality, we get
E[f(X)] ≥ f(E[X]) 
⇒ E[3X2] ≥ f(E[3X2])
⇒ E[3X2] ≥ 3(E[X2])
 ⇒ E[3X2] ≥ 3(5)2 (since the mean μ = E[X] = 5)
⇒ E[3X2] ≥ 75

Thus, the minimum value of f(X) = 3X2 for a random variable X is 75

If p, q, r, s > 0 and p + q + r + s = 1, then find the minimum of 
${\left( p+\dfrac{1}{p}\right) ^{5}+\left( q+\dfrac{1}{q}\right) ^{5}+\left( r+\dfrac{1}{r}\right) ^{5}+\left( s+\dfrac{1}{s}\right) ^{5}}$

Solution:

As we know, p, q, r, s > 0 and p + q + r + s = 1
Thus, 0 < p, q, r, s < 1. 
Let f (x) = ${\left( x+\dfrac{1}{x}\right) ^{5}}$, x on the interval I = (0, 1), 
Now, f’(x) = ${5\left( x+\dfrac{1}{x}\right) ^{4}\left( 1-\dfrac{1}{x^{2}}\right)}$
f”(x) = ${5\left( 4\left( x+\dfrac{1}{x}\right) ^{3}\left( 1-\dfrac{1}{x^{2}}\right) ^{2}+\dfrac{2\left( x^{2}+1\right) ^{4}}{x^{7}}\right)}$ 
Here, f”(x) > 0, then f is strictly convex on I
By Jensen’s inequality, we get 
${f\left( \dfrac{p+q+r+s}{4}\right) \leq \dfrac{f\left( p\right) +f\left( q\right) +f\left( r\right) +f\left( s\right) }{4}}$
⇒ ${4f\left( \dfrac{p+q+r+s}{4}\right) \leq f\left( p\right) +f\left( q\right) +f\left( r\right) +f\left( s\right)}$
⇒ ${4\left( \dfrac{1}{4}+4\right) ^{5}\leq f\left( p\right) +f\left( q\right) +f\left( r\right) +f\left( s\right)}$
⇒ ${\dfrac{17^{5}}{4^{4}}\leq f\left( p\right) +f\left( q\right) +f\left( r\right) +f\left( s\right)}$
Thus, the minimum is ${\dfrac{17^{5}}{4^{4}}}$, attained when p = q = r = s = ${\dfrac{1}{4}}$

Last modified on May 25th, 2024