Mathematics desk
< April 23	<< Mar \| April \| May >>	April 25 >

Welcome to the Wikipedia Mathematics Reference Desk Archives
The page you are currently viewing is a transcluded archive page. While you can leave answers for any questions shown below, please ask new questions on one of the current reference desk pages.

April 24

derivative of constrained function is sum of unconstrained partials?

Hello, while studying neural networks I came upon what was described by the lecturer as a "math trick" to solve a particular type of optimization problem using gradient descent. Basically, when optimizing a neural net that requires two parameters to be equal, you can replace the partial derivative for each constrained parameter by the sum of the partials with respect to each constrained parameter. So if you have $f(x_{1},x_{2})$ , then the derivative of $f(x,x)$ with respect to $x$ is the same as the sum of the partial derivatives of $f(x_{1},x_{2})$ with respect to $x_{1}$ and $x_{2}$ . So far I have not been able to find a counter example, but I also do not know how to prove it. If anyone has pointers, clues, or proofs please help! Sorry if its obvious, and thanks! Brusegadi (talk) 01:09, 24 April 2017 (UTC)[reply]

It's a particular case of the multivariable chain rule; it's easier to see what's going on from the more general form

{\frac {d}{dt}}(f(x(t),y(t)))={\frac {\partial f}{\partial x}}{\frac {dx}{dt}}+{\frac {\partial f}{\partial y}}{\frac {dy}{dt}}

. --JBL (talk) 01:52, 24 April 2017 (UTC)[reply]

Thank you so much! Brusegadi (talk) 02:08, 24 April 2017 (UTC)[reply]

How about a slightly different situation involving total derivative of a function G of constrained independent variables x_i with constant sum, for instance 1. Can the partial derivative with respect to x_i exist by keeping the other x_j constant even if only the sum of x_i is constant, not every x_i? Is this due to fact that dx_i is around zero? Thanks.--82.137.9.214 (talk) 23:49, 24 April 2017 (UTC)[reply]

The partial derivative is a feature of the function, not of the combination of function and constraint. So yes, the partial derivative can exist regardless of what context the function will be used in. You can take the total differential of G, which in the n=2 case is

dG={\frac {\partial G}{\partial x_{1}}}dx_{1}+{\frac {\partial G}{\partial x_{2}}}dx_{2}.

Then if you impose

x_{1}+x_{2}=k

and hence

dx_{1}+dx_{2}=0

hence

dx_{2}=-dx_{1},

you get

dG={\frac {\partial G}{\partial x_{1}}}dx_{1}+{\frac {\partial G}{\partial x_{2}}}\times (-dx_{1})=\left({\frac {\partial G}{\partial x_{1}}}-{\frac {\partial G}{\partial x_{2}}}\right)dx_{1}.

Loraof (talk) 16:16, 25 April 2017 (UTC)[reply]

But if n>2, not enough information has been provided—for the last step, we need to know how the offset of

dx_{1}

is distributed among

dx_{2},\,dx_{3},

etc. Loraof (talk) 16:21, 25 April 2017 (UTC)[reply]

But can x₂ be held constant when taking the partial derivative in respect to x₁ as requested by the definition of the partial derivative? Similar question for the other independent variable x₁ to be held constant when taking the partial derivative in respect to x₂. Isn't the situation a bit stretched because strictly one independent variable cannot be made constant when the partial derivative is taken in respect to the other indepedent variable in such cases where only the sum of independent variables can be constant? Or it is about quasi-constancy of independent variables which is satisfactory in this situation?--82.137.14.76 (talk) 00:17, 26 April 2017 (UTC)[reply]

The original question does not ask for a partial derivative so it's hard to see the relevance of this query. Maybe you should ask a new question instead, where you can make your hypotheses clear. --JBL (talk) 01:07, 26 April 2017 (UTC)[reply]

Johnson's SU-distribution

Is Johnson's SU-distribution and "shepherd's crook" and Johnson Curve all the same thing? Thanks. Anna Frodesiak (talk) 06:25, 24 April 2017 (UTC)[reply]

I want to know because of [1] and [2] that I did. Anna Frodesiak (talk) 10:24, 24 April 2017 (UTC)[reply]

Face value numbers

When we count, for example, coins or banknotes by their face value rather than actual quantity expressed by natural numbers, are those still natural numbers or some other kind? Thanks.--212.180.235.46 (talk) 16:59, 24 April 2017 (UTC)[reply]

Natural numbers don't include decimals, and most currencies have a sub-denomination (like cents for dollars), which makes it a decimal number. Yen is one that doesn't, so, for that case, I suppose you could use natural numbers for prices, in most cases (with exceptions for buying in quantity, where they might break the price down by tenths of a yen, etc.) StuRat (talk) 17:38, 24 April 2017 (UTC)[reply]

Perhaps not everyone will agree, but I would consider natural numbers to be dimensionless. Currency is given units of whatever denomination it is, so $5 is not the same as 5 even though there's no decimals being used. --RDBury (talk) 05:21, 25 April 2017 (UTC)[reply]

Yep as we all know time is money so definitely not dimensionless :) Dmcq (talk) 10:25, 28 April 2017 (UTC)[reply]