For example, in matlab, norm (A,2) gives you induced 2-norm, which they simply call the 2-norm. So in that sense, the answer to your question is that the (induced) matrix 2-norm is $\le$ than Frobenius norm, and the two are only equal when all of the matrix's eigenvalues have equal magnitude.
Notice that the intersection of all L^p spaces is not necessarily L^infty. Hence, for the argument to work you need a-priori for the L^infty norm to be finite.
In spatial euclidean vector spaces norm is an intuitive concept: It measures the distance from the null vector and from other vectors. The generalization to function spaces is quite a mental leap ...
This proof is really a way of saying that the topology induced by a norm on a finite-dimensional vector space is the same as the topology defined by open half-spaces; in particular, all norms define the same topology and all norms are equivalent. There are other ways to prove that using the Hahn-Banach theorem.
In number theory, the "norm" is the determinant of this matrix. In that sense, unlike in analysis, the norm can be thought of as an area rather than a length, because the determinant can be interpreted as an area (or volume in higher dimensions.) However, the area/volume interpretation only gets you so far.
Continuity with respect to the norm $\Vert \cdot\Vert$ follows from the fact that addition and negation are $\Vert \cdot \Vert$-continuous, that the norm itself is continuous and that sums and compositions of continuous functions are continuous. Remark.
The $p$ -norm on $\mathbb R^n$ is given by $\|x\|_ {p}=\big (\sum_ {k=1}^n |x_ {k}|^p\big)^ {1/p}$. For $0 < p < q$ it can be shown that $\|x\|_p\geq\|x\|_q$ (1, 2).
I have no background in gradients. You are correct, the answer for L0-norm is discontinuous. And what is a coordinate? Can you point to me a link on all these?
The $1$-norm and $2$-norm are both quite intuitive. The $2$-norm is the usual notion of straight-line distance, or distance ‘as the crow flies’: it’s the length of a straight line segment joining the two points.
The operator norm is a matrix/operator norm associated with a vector norm. It is defined as $||A||_ {\text {OP}} = \text {sup}_ {x \neq 0} \frac {|A x|_n} {|x|}$ and different for each vector norm. In case of the Euclidian norm $|x|_2$ the operator norm is equivalent to the 2-matrix norm (the maximum singular value, as you already stated). So every vector norm has an associated operator norm ...
