Functions With Lipschitz Continuous Derivatives Example List

2. Is this function Lipschitz continuous?

Is this function Lipschitz continuous?

Solution 1

Since this looks like a homework, I will just outline the proof.

1. First note that this is the same as proving $$f(x)=e^{-\|x\|}x, x\in \mathbb{R}^n$$ is Lipschitz.

2. Each component $f_i$ of $f$ is a $C^1$ function on $\mathbb{R}^n$ with partial derivative given by $$\frac{\partial f_i}{\partial x_i}=\left\{\begin{array}{ll}e^{-\|x\|}\left(1-\frac{x_i^2}{\|x\|}\right) & \mbox{ if }x\neq 0\\1&\mbox{ if }x= 0\end{array}\right.$$ and for $j\neq i$, $$\frac{\partial f_i}{\partial x_j}=\left\{\begin{array}{ll}-e^{-\|x\|}\frac{x_ix_j}{\|x\|} & \mbox{ if }x\neq 0\\0&\mbox{ if }x= 0\end{array}\right.$$

3. Note that all partial derivatives of $f_i$ are also bounded on $\mathbb{R}^n$.

4. Now you can apply mean value theorem for functions from $\mathbb{R}^n$ to $\mathbb{R}^n$ to prove that $f$ is Lipschitz.

Solution 2

Here is a long-winded answer. I didn't even think to check differentiability of $e^{-\|x\|}x_1$ at $x=0$ (because $x \mapsto e^{-\|x\|}$ is not differentiable at $x=0$). See @TCL's answer for a much more succinct approach.

First prove that $\phi_1(x) = e^{-\|x\|}x_1$ is locally Lipschitz continuous. Let $K$ be a convex, compact set, then the scalar function $t \mapsto e^{-t}$ has a bounded derivative on $K$. Let the bound be $L_K$, hence $\|e^{-\|x\|} - e^{-\|y\|}\| \le L_K \left| \|x\|-\|y\| \right| \le L_K \|x-y\|$ for $x,y \in K$. The function $x \mapsto x_1$ is also Lipschitz continuous with rank $1$. Hence we have $|\phi_1(x) -\phi_1(y) | \le |e^{-\|x\|}x_1 - e^{-\|x\|}y_1 + e^{-\|x\|}y_1 -e^{-\|y\|}y_1 | \le |x_1-y_1| + L_K|y_1| \|x-y\|$ Since $K$ is bounded, we see that $\phi_1$ is Lipschitz on any convex, compact set.

Note that $\phi_1$ is clearly smooth except for possibly $x=0$.

Now we will show that $\phi_1$ is globally Lipschitz continuous. Choose $x,y$ and let $\eta(t) = \phi_1(x+t(y-x))$. Since the set $\{x+t(y-x)\}_{t \in [0,1]}$ is convex and compact, $\eta$ is Lipschitz, hence absolutely continuous. It follows that $\eta$ is differentiable a.e. and $\eta(1)-\eta(0) = \int_0^1 \eta'(t) dt$. In particular, if $|\eta'(t)| \le B$ a.e., then $|\eta(1)-\eta(0)| \le B$. We see that if $x \neq 0$, then $\eta'(t) = \frac{\partial \phi_1(x+t(y-x))}{\partial x} (y-x)$.

A quick calculation shows that $\frac{\partial \phi_1(x)}{\partial x} = e^{-\|x\|} e_1^T -x_1 e^{-\|x\|} \frac{x^T}{\|x\|}$, which gives $\| \frac{\partial \phi_1(x)}{\partial x} \| \le 1+ |x_1|e^{-\|x\|} \le 1+ e^{-1}$. It follows that $|\eta'(t)| \le (1+e^{-1}) \|x - y \|$ a.e., and so $|\eta(1)-\eta(0)| = |\phi_1(x)-\phi_1(y)| \le (1+e^{-1}) \|x - y \|$. Since $x,y$ were arbitrary, we see that $\phi_1$ is globally Lipschitz continuous, with rank $\le 1+e^{-1}$.

Exactly the same analysis applies to $\phi_k$ defined by $\phi_k(x) = e^{-\|x\|}x_k$. It follows that the function $\phi(x) = e^{-\|x\|}x$ is globally Lipschitz continuous.

Since $f(x) = \sum_k \phi(\mu_k-x)$, it follows that $f$ is globally Lipschitz continuous.

Note that in the proof above, the function is proved to be locally Lipschitz continuous in order to show that the restriction is absolutely continuous. This allows us to focus on the derivative of the restriction instead, which we show is bounded, hence the function is globally Lipschitz continuous.

Related videos on Youtube

18 : 00

What is a Lipschitz condition?

06 : 04

Every Lipschitz Function is Uniformly Continuous Proof

10 : 36

Supplemental Video: Lipschitz-continuity

10 : 25

09 : 31

Lipschitz Functions: Intro and Simple Explanation for Usefulness in Machine Learning

Comments

• Let $\mu \in \mathbb R^d$ be given. Is the function $f:\mathbb R^d \to \mathbb R^d$ defined as $f(x) := \exp(-\|x- \mu\|) (\mu - x)$ Lipschitz continuous? More specifically, for any $x, y \in \mathbb R^d$, is there a $D\in \mathbb R$ such that $$ | \exp(-\|x- \mu\|) (\mu - x) - \exp(-\|y- \mu\|) (\mu - y)] | \leq D\|x-y\|? $$ $\| \|$ is the Euclidean norm. Thanks!

  • What is $i$? Did you intend for there to be a norm or an inner product there?

  • @MichaelGrant: Thanks. My typo, there is no $i$.

  • Suggestion: Try showing that the gradient is bounded.

  • @NateEldredge: Is it that an everywhere differentiable function $f : \mathbb R^d → \mathbb R$ is Lipschitz continuous if and only if $\|f'\|$ is bounded?

  • Right, basically the mean value inequality. There may be some extra work to do at $x=0$.

  • @NateEldredge: (1) you mean at $x=\mu$? At $x=\mu$, $f$ isn't differentiable, is it? Can we still apply the conclusion in my last comment? (2) at $x\neq \mu$, $f'(x)$ is too complicated for verifying if it is bounded, even if $d =1$?

• +1 Nice. Never occurred to me that $f$ would be differentiable at $x=0$.

• Thanks! (1) Why are all partial derivatives of $f_i$ also bounded on $\mathbb R^n$? For example, why are $x_i^2/\|x\|$ and $x_ix_j/\|x\|$ bounded? (2) I also failed to see $f_i$ is differentiable at $x=0$, because its partial derivatives at $x\neq 0$ aren't defined at $x+0$. So how do you know the partial derivative of $f_i$ at $x=0$ is $1$ and $0$?

• (1) $|x_i|\le \|x\|$. (2). Try to find the partial derivative at $x=0$ by definition.

• Thanks, @TCL! Why does $|x_i| \leq \|x\|$ imply $x_i^2 /\|x\|$ and $x_ix_j/\|x\|$ are bounded?

• They aren't bounded, but multiplied by $e^{-\|x\|}$ they are.

• @TCL, Why after being multiplied by $e^{-\|x\|}$ they are bounded? By the way, my questions are not from homework. They are from my previous questions.

• Their absolute values are less than $\|x\|e^{-\|x\|}$, and $re^{-r}$ is bounded.

Recents

hernandezhatimor.blogspot.com

Source: https://9to5science.com/is-this-function-lipschitz-continuous

Share this post