#math
As defined in the linear algebra aspect, in the book Neural Network Primer

The jacobian is a way to represent directional gradient of a function's output with respect to it's input in the form of a matrix.

Suppose a function $f$ which transforms input vector $\mathbf{x}$~$(d)$ into an output $\mathbf{y}$~$(o)$
Then the Jacobian ~ $(o,d)$ is defined as

$$\partial f(\mathbf{x}) = \begin{bmatrix} \frac{\partial y_{1}}{\partial x_{1}} & \dots & \frac{\partial y_{1}}{\partial x_{d}} \ \vdots & \ddots & \vdots \ \frac{\partial y_{o}}{\partial x_{1}} & \dots & \frac{\partial y_{d}}{\partial x_{d}} \end{bmatrix}$$

On the dimensionality of Jacobians

When we work with a linear function $\mathbf{W}$ of the form $$\mathbf{y} = \mathbf{W}\cdot \mathbf{x}$$with the standard input dimension of $d$ and output dimension of $o$,

We can represent Jacobian with respect to $\mathbf{x}$ as simply as $$\partial_{\mathbf{x}}[\mathbf{W}\cdot \mathbf{x}] = \mathbf{W}$$and the dimensionality of the jacobian is simply $(o,d)$

However, when looked upon as a function of $\mathbf{W}$ and with $\mathbf{x}$ as the function itself, the Jacobian takes the dimensions of $(o,o,d)$.

Then, we can always take the isomorphic map of $\mathbf{W}$ and represent it as a vector as $\mathrm{vec}(\mathbf{W})$~$(od)$, and thus the jacobian is of the dimemsions $(o,od)$.

This is like a squashing of the weight space into something more representable and digestible.

Later on in more readings, jacobian notation may hide the true dimensions of matrices.