Talk:Sparsity-of-effects principle

There is clearly some confusion here about the difference between the sparsity-of-effects principle and what I have learned as either the main effects principle or the hierarchical ordering principle. My recent edit to this page is an attempt to begin a reconciliation. I have sourced my reasoning. Ideally I think this page should be forked into two pages, one for the main effects principle and one for the hierarchical ordering principle. It would probably also be a good idea to bring in some examples of both of these principles. --Statwizard 20:57, 22 March 2006 (UTC)[reply]

Coordinate-system invariance?

I come not from a design-of-experiments background but from a numerical optimization background. It seems as though the sparsity-of-effects principal is saying (in optimization terminology) that the Hessian matrix of the cost function at the solution is likely to be "almost diagonal": that for each row and column, only one off-diagonal entry is likely to be close in magnitude to the corresponding diagonal entry. Said another way, I think that's saying that the eigenvectors of the Hessian corresponding to the strongest curvature will tend to lie on the planes of the coordinate system (i.e., to have the magnitude of one or two components dominate the magnitude of the other components). Also, this implies that it is trivial to permute the degrees of freedom to produce an approximate Hessian that is a tridiagonal matrix: For each DoF, i, find the DoF, j > i, that it interacts most with; swap DoF i+1 and j to move the i–j interaction terms to the off-diagonal.

It seems notable that this principle then seems to not be invariant under change of coordinates. That is, if instead of an experiment having physically-meaningful "knobs" to turn, the knobs were first transformed by a matrix, then this principal would break down. In contrast, for many optimization problems, coordinate-system invariance is assumed. As such, it seems like to use this principal one has to be careful to select variables that are physically meaningful in the context of the underlying mechanisms. Does that sound right? —Ben FrantzDale (talk) 13:12, 19 September 2014 (UTC)[reply]

More sources

This article needs updating. Here's some sources I found I will be adding later: https://www.tandfonline.com/doi/abs/10.1080/08982112.2011.553760 https://onlinelibrary.wiley.com/doi/abs/10.1002/cplx.20123 https://www.amazon.com/Design-Analysis-Experiments-Douglas-Montgomery/dp/1119386101/ref=dp_ob_title_bk Furthermore, the Hunter textbook doesn't seem to discuss the topic at all. I could not find a copy of Hamada. Deleet (talk) 15:02, 20 February 2019 (UTC)[reply]