Talk:Dilution (neural networks)

	This article is within the scope of WikiProject Computing, a collaborative effort to improve the coverage of computers, computing, and information technology on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.ComputingWikipedia:WikiProject ComputingTemplate:WikiProject ComputingComputing articles
???	This article has not yet received a rating on the project's importance scale.

Robotics

	This article is within the scope of WikiProject Robotics, a collaborative effort to improve the coverage of Robotics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.RoboticsWikipedia:WikiProject RoboticsTemplate:WikiProject RoboticsRobotics articles
???	This article has not yet received a rating on the project's importance scale.

Possible expansion from Convolutional Neural Network article

This article, apart from its disputed factual accuracy, is quite short. There seems to be more information in the Convolutional Neural Networks article (some of the information in the Dropout section of that article is rather general and could be applied to help improve this article). Mocl125 (talk) 20:50, 10 June 2020 (UTC)[reply]

Relation between strong/weak dilution and dropout

In response to @Jeblad's note in the article about strong/weak dilution, I looked into one of the sources on this topic (from 1991) that Jeblad mentions, Introduction to the Theory of Neural Computation by Hertz, Krogh, and Palmer. Unfortunately, the section about dilution requires a level of understanding about earlier neural networks that I do not have yet. I would really appreciate if someone with more experience than me could include some reference to dilution (and/or to the even earlier predecessors of this idea, if they exist) in this article, if relevant. Many thanks! Mocl125 (talk) 05:08, 11 June 2020 (UTC)[reply]

I don't think it matters. Have you ever once, in your life, heard it called "dilution" instead of "dropout"? I read an Arxiv DL paper a day and have for a decade, and I have never once heard of it outside of this WP entry, and you'll have to search long and hard for a DL researcher who knows what 'dilution' is but not 'dropout'... It's nice that someone or other somewhere back in the 1980s or whatever invented something that if you squint looks like dropout, but no one cares, and that should be stuffed into a 'History' section or something (with a big caveat 'it had no influence on anyone at all and was later reinvented'), for those who do care about playing Trivial Pursuit with citations. As the article stands, it seems like a mix of WP:POVTITLE and WP:UNDUE to me, and it should be renamed back to 'Dropout (neural networks)' and refocused on what people actually know dropout for, like regularization and ensembling. --Gwern (contribs) 18:07 8 December 2022 (GMT) 18:07, 8 December 2022 (UTC)[reply]