Jensen’s Inequality

Jensen’s inequality relates the expected value of a convex or concave function to the function of their expected values.

It states that if ‘X’ is an integrable random variable and f: ℝ → ℝ is a convex function such that Y = f(X) is also integrable, then E[f(X)] ≥ f(E[X])

Thus, if a convex function is applied to the mean of a variable, it is less than or equal to the mean of that function applied to the variable. However, if ‘f’ is a concave function, the inequality reverses.

Proof

Let us consider a convex function f(x).

Then, for any point x_0, the graph of f(x) lies entirely above the tangent at the point ‘x₀’

f(x) ≥ f(x₀) + b(x – x₀), where ‘b’ is the slope of the tangent 

Now, considering x = X and x₀ = E[X], we get

f(X) ≥ f(E[X]) + b(X – E[X])

Taking the expected values of both sides, we get

E[f(X)] ≥ E[f(E[X]) + b(X – E[X])]

By using the linear property of expected values, we get

E[f(X)] ≥ f(E[X]) + b(E[X] – E[X])

⇒ E[f(X)] ≥ f(E[X]) …..(i)

If the function f(x) is concave, then -f(x) is convex, and from the inequality (i), we get

E[-f(X)] ≥ -f(E[X])

Multiplying both sides by (-1), we get

E[f(X)] ≤ f(E[X]) …..(ii)

Thus, Jensen’s inequalities for convex and concave functions are proved.

Alternative Form

However, there is an alternative form of this inequality, where the expected value or mean E[X] is expressed as E[X] = a₁x₁ + a₂x₂ + … + a_nx_n.

It implies:

f(E[X]) = f(a₁x₁ + a₂x₂ + … + a_nx_n) and  E[f(X)] = a₁f(x₁) + a₂f(x₂) + … + a_nf(x_n), for all a₁, a₂, …, a_n ≥ 0

In particular, if a₁, a₂, …, a_n ≥ 0 and ${\sum ^{n}_{i=1}a _{i}=1}$ for all λ_i, then

${\sum ^{n}_{i=1}a _{i}f\left( x_{i}\right) \geq f\left( \sum ^{n}_{i=1}a _{i}x_{i}\right)}$ 

In general, if a real-valued function ‘f’ is convex on the interval ‘I,’ and x₁, x₂, …, x_n Є I, then for a₁, a₂, …, a_n ≥ 0,

${\dfrac{a_{1}f\left( x_{1}\right) +a_{2}f\left( x_{2}\right) +\ldots +a_{n}f\left( x_{n}\right) }{a_{1}+a_{2}+\ldots +a_{n}}\geq f\left( \dfrac{a_{1}x_{1}+a_{2}x_{2}+\ldots +a_{n}x_{n}}{a_{1}+a_{2}+\ldots +a_{n}}\right)}$

Conditions to Follow

To apply Jensen’s inequality, we use the following conditions of calculus.

If f: I → ℝ is a twice-differentiable function, then

• ‘f’ is convex on ‘I,’ if and only if f”(x) ≥ 0, for all x Є I
• ‘f’ is concave on ‘I,’ if and only if f”(x) ≤ 0, for all x Є I

Solved Examples