Inequalities

Jensen’s Inequality. Let $X$ be a convex set in a real vector space, $\varphi : X \to \R$ a convex function, $\{x_i\}_{i=1}^n \subseteq X$ and $\{\lambda_i\}_{i=1}^n \subseteq [0, \infty)$ such that $\sum_{i=1}^n \lambda_i = 1$. Then

$$\varphi\left(\sum_{i=1}^n \lambda_i x_i\right) \le \sum_{i=1}^n \lambda_i \varphi(x_i)$$

If $\varphi$ is otherwise concave, then

$$\varphi\left(\sum_{i=1}^n \lambda_i x_i\right) \ge \sum_{i=1}^n \lambda_i \varphi(x_i)$$

Proof. It suffices to only prove the result for convex functions, as if it is concave, then $-\varphi$ is convex and

$$-\varphi\left(\sum_{i=1}^n \lambda_i x_i\right) \le -\sum_{i=1}^n \lambda_i \varphi(x_i)$$

so now just multiply both sides by -1. The proof follows by induction. $n=1$ is trivial since $\varphi(\lambda_1 x_1) = \varphi(x_1) = \lambda_1 \varphi(x_1)$, and $n=2$ follows directly from the definition of convexity.

$$\varphi(\lambda_1 x_1 + \lambda_2 x_2) \le \lambda_1 \varphi(x_1) + \lambda_2 \varphi(x_2)$$

If true for $n$, let $\{x_i\}_{i=1}^{n+1}$ and $\{\lambda_i\}_{i=1}^{n+1}$. If needed, reorder these terms so that $\lambda_{n+1} < 1$. Then

$$y = \sum_{i=1}^{n} \frac{\lambda_i x_i}{1 - \lambda_{n+1}} \implies \sum_{i=1}^{n+1} \lambda_i x_i = (1 - \lambda_{n+1})y + \lambda_{n+1}x_{n+1}$$

By convexity, $y \in X$ and by the inductive step,

$$\varphi\left(\sum_{i=1}^{n+1} \lambda_i x_i\right) \le (1 - \lambda_{n+1}) \varphi(y) + \lambda_{n+1} \varphi(x_{n+1}) \le \sum_{i=1}^n \lambda_i\varphi(x_i) + \lambda_{n+1} \varphi(x_{n+1}) = \sum_{i=1}^{n+1} \lambda_i x_i \quad \square$$

AM-GM inequality. For any positive real numbers $x_1,x_2,\dots,x_n$,

$$\frac{x_1 + x_2 + \cdots + x_n}{n} \ge \sqrt[n]{x_1 x_2 \cdots x_n}$$

Proof. Note that $\log(x)$ is increasing and $\frac{d^2}{dx^2} \log(x) = \frac{-1}{x^2} < 0$ implies it is strictly concave. Therefore, if we apply the Jensen’s inequality to $\lambda_i = 1/n$, and then apply $e^x$ on both sides (which doesn’t change the direction of the inequality)

$$\log\left(\sum_{i=1}^n \frac{x_i}{n}\right) \ge \sum_{i=1}^n \frac{\log(x_i)}{n} \implies \frac{x_i}{n} = \left(\prod_{i=1}^n x_i\right)^{1/n} \quad \square$$

Young’s inequality. If $a,b \ge 0$ and $p,q > 1$ such that $p^{-1} + q^{-1} = 1$, then

$$ab \le \frac{a^p}{p} + \frac{b^q}{q}$$

Proof. Using the $\log$ trick as above, Jensen’s inequality states

$$\log\left(\frac{a^p}{p} + \frac{b^q}{q}\right) \ge \frac{\log(a^p)}{p} + \frac{\log(b^q)}{q} = \log(ab) \quad \square$$

Hölder’s inequality. Let $(X, \Sigma, \mu)$ be any measure space and $p,q > 1$ with $p^{-1} + q^{-1} = 1$. For any measurable functions $f,g : X \to \C$,

$$\int \abs{f(x)g(x)} \: dx \le \left(\int \abs{f(x)}^p \: dx\right)^{1/p}\left(\int \abs{g(x)}^q \: dx\right)^{1/q}$$

Proof. Denote $\norm{f}_p = (\int \abs{f(x)}^p \: dx)^{1/p}$. We wish to prove $\norm{fg}_1 \le \norm{f}_p \norm{g}_q$. By Young’s inequality,

$$\frac{\abs{f(x)}}{\norm{f}_p} \frac{\abs{g(x)}}{\norm{g}_q} \le \frac{\abs{f(x)}^p}{p\norm{f}_p^p} + \frac{\abs{g(x)}^q}{q\norm{g}_q^q}$$

Since these are both positive for all $x$, we can integrate both sides and get

$$\frac{\norm{fg}_1}{\norm{f}_p\norm{g}_q} \le \frac{\norm{f}_p^p}{p\norm{f}_p^p} + \frac{\norm{g}_q^q}{q\norm{g}_q^q} = \frac{1}{p} + \frac{1}{q} = 1 \implies \norm{fg}_1 \le \norm{f}_p\norm{g}_q \quad \square$$

Minkowski inequality. Let $(X, \Sigma, \mu)$ be any measure space and $p \ge 1$. For any measurable functions $f,g : X \to \C$,

$$\left(\int \abs{f(x) + g(x)}^p \: dx\right)^{1/p} \le \left(\int \abs{f(x)}^p \: dx\right)^{1/p} + \left(\int \abs{g(x)}^p \: dx\right)^{1/p}$$

Proof. For $p=1$, this is just the triangle inequality for $\C$ applies at every $x$ and then integrating both sides. Once again, if we use the $\norm{f}_p$ notation as above, we wish to prove $\norm{f + g}_p \le \norm{f}_p + \norm{g}_p$. Assuming $p > 1$, compute

$$\norm{f + g}_p^p = \int \abs{f(x) + g(x)}\abs{f(x) + g(x)}^{p-1} \: dx \le \int \abs{f(x)}\abs{f(x) + g(x)}^{p-1} \: dx + \int \abs{g(x)}\abs{f(x) + g(x)}^{p-1} \: dx$$

We can apply Hölder’s inequality to the functions $f(x)$ and $(f(x)g(x))^{p-1}$, with $q = p/(p-1)$, to get

$$\int \abs{f(x)}\abs{f(x) + g(x)}^{p-1} \: dx \le \left(\int \abs{f(x)}^p \: dx\right)^{1/p}\left(\int \abs{f(x) + g(x)}^p\right)^{1 - \frac{1}{p}} = \norm{f}_p \frac{\norm{f + g}_p^p}{\norm{f + g}_p}$$

By doing the same thing for the second term with $g(x)$, we get

$$\norm{f + g}_p^p \le \norm{f}_p \frac{\norm{f + g}_p^p}{\norm{f + g}_p} + \norm{g}_p \frac{\norm{f + g}_p^p}{\norm{f + g}_p} \implies \norm{f + g}_p \le \norm{f}_p + \norm{g}_p \quad \square$$

Cauchy-Schwarz inequality for $L^p$ spaces. For any $f,g \in L^p$,

$$\norm{\inner{f, g}}^2 \le \inner{f, f} \inner{g, g}$$

Proof. The inner product in $L^p$ space is defined as

$$\inner{f, g} \coloneqq \left(\int f(x)\cl{g(x)} \: dx\right)^{1/2}$$

The Hölder inequality immediatelly tells us that for $p = q = 2$,

$$\int f(x)\cl{g(x)} \: dx \le \int \abs{f(x)g(x)} \: dx \le \left(\int \abs{f(x)}^2 \: dx\right)^{1/2}\left(\int \abs{g(x)}^2 \: dx\right)^{1/2} = \inner{f, f} \inner{g, g} \quad \square$$