Talk:Leela Chess Zero

WP:NPOV?

Unlike high-end chess programs such as Stockfish, which basically do a bunch of calculations on the CPU, Leela Chess Zero, also known as LCZero, uses a neural net running on a GPU.

As explained here,[1] you can run LCZero on the GPU on a PC, on Google Colab, or on Google Cloud.

The question of whether LCZero is stronger that Stockfish is of great interest to many people. We should not imply that it is stronger unless the sources support that conclusion. And of course we may need to change what we say about this as both programs go though new revisions.

According to [2] in a Blitz Chess LCZero is in third place, behind Stockfish and Komodo.

Nonetheless, our article says "Leela beats the world champion Stockfish engine despite a 1-pawn handicap", citing [3]].

This implies that Leela is stronger than Stockfish, but the sources do not support that claim.

The above "despite a 1-pawn handicap" is especially misleading. It implies that the starting position gave Stockfish one more pawn than Leela

The actual source says:

"The Lc0 engine got off to a fast start in the bonus games, beating the reigning Computer Chess Champion Stockfish despite Lc0 starting down a pawn in the odds-chess position." with the actual moves listed (you can see them on the board by clicking on the arrows).

As can clearly be seen by replaying the game (you can replay the moves on the page I linked to by clicking on the arrows), They started out even, then LCZero blundered and lost a pawn on the second move. So the article is misleading.

More importantly, by focusing on the one game, instead of the result of the entire tournament, the article implies that LCZero is stronger than Stockfish.

Another problem is that is isn't clear from the source whether LCZero was running on the same hardware as Stockfish. Yes, I am aware that one other editor of this page thinks he knows, but until we see an actual source that says what the hardware was, I stand by my conclusion that it is unclear.

Normally, these tournaments are run on PCs with fast CPUs and lots of memory, but without an epecially powerful GPU -- most chess programs use a lot of CPU, not GPU. In suchs a case, Stockfish would have un unfair advantage.

Some chess programs run hardware back at the lab and send their moves over the Internet (earlier programs like Deep Blue used a telephone and a modem for this).

So it isn't clear whether Stockfish and LCZero were running on the same hardware or whether LCZero was running on Google Cloud, which is far more powerful than any PC if you pay enough.

I tried to address some of these issues, but as you can see from the page history I keep getting reverted.

So my question is this: Does the current wording neutrally present what the sources say about the relative strength of Stockfish and LCZero? Or does it give one game too much prominence instead of focusing on the overall win-loss ratio? --Guy Macon (talk) 23:03, 23 November 2018 (UTC)

Several things -

1. It's true that which is stronger between Leela and Stockfish commands strong public interest, but like is common in computer chess, the stronger engine depends on the hardware used. Of course in an AB engine vs. AB engine comparison, you can run both of them on the same hardware. But you can't run Stockfish and Leela on the same hardware. Stockfish uses CPU, Leela uses GPU. If they If you pit Stockfish against Leela on a 176-core 2.3 GHz CPU with no graphics card, Stockfish wins easily. If you do the same test with Stockfish on a Pentium 486 but with two GTX 1080 Ti cards, Leela wins easily.
2. If you demand that Leela run on CPU, then yeah, Stockfish crushes Leela without a contest. But that wouldn't be a fair fight, since Leela crushes Stockfish if both are run on GPU (in fact as far as I'm aware Stockfish can't even run on GPU).
3. So back to this article. The game which Leela won against Stockfish is under a section heading "Notable games". Is this game notable? I'd say without question it is. Stockfish is the unchallenged best AB engine right now, so the fact that Leela can win at pawn odds is definitely notable. Of course this doesn't show that Leela is better than Stockfish - in fact in many of the competitions, using consensus-fair hardware, Stockfish is still stronger than Leela right now. The section shouldn't be read as such.
4. All the tournaments in which SF finished ahead of Leela are already in the article.

If there's anything to change here it would be to remove the line about "internal tests", which right now even sources a Google Docs page, for the above reasons that internal tests depend on hardware used. Banedon (talk) 01:34, 24 November 2018 (UTC)

	a	b	c	d	e	f	g	h
8									8
7									7
6									6
5									5
4									4
3									3
2									2
1									1
	a	b	c	d	e	f	g	h

Typical initial setup for pawn odds: White (the stronger player) starts without the f-pawn. See Handicap (chess). Not to be confused with blundering and losing a pawn on the second move.

One problem. Your statement "...the fact that Leela can win at pawn odds" is a lie. Actually winning at pawn odds is an indication of strength. Blundering away your pawn on move two is an indication of weakness.

Focusing on one game where Leela did well is a violation of WP:NPOV and WP:WEIGHT. You are trying to mislead the reader into thinking that Leela is something other than a chess engine that is currently weaker than Komodo and Stockfish.

Another example of misleading language: "surpassing the strength of early releases of Rybka, Stockfish, and Komodo, despite evaluating orders of magnitude fewer positions". More accurate would be "Stockfish, and Komodo surpass the strength of Leela despite having significantly less sophisticated evaluation functions". It violates NOPOV to compare the current release of Leela with early releases of Rybka, Stockfish, and Komodo. It violates NPOV to pretend that how many positions a chess engine evaluates is somehow significant. It is well know among chess engine programmers that there is a trade-off between looking at more positions but spending less time evaluating each one and looking at fewer positions but spending more time evaluating each one. Which the developers choose isn't significant. But the article (there is a theme to the NPOV violations here) makes it sound like making a different decision on more positions vs better evaluation somehow implies that Leela is superior. --Guy Macon (talk) 07:13, 24 November 2018 (UTC)

I don't appreciate you calling what I wrote a lie. I don't appreciate you insinuating that I am pro-Leela either. I will point out to you that I added this edit which explicitly said Leela did not perform well; the text is still in the article. That's all. Banedon (talk) 08:14, 24 November 2018 (UTC)

So, is it your position that Leela was playing at pawn odds? --Guy Macon (talk) 09:35, 24 November 2018 (UTC)

@Guy Macon: I'm afraid there are a couple misconceptions here. First, Leela didn't blunder a pawn. The pawn sacrifice was hard-coded into the game. It was part of a "bonus" round of games separate from the normal tournament. Each player in the bonus games gave up a pawn at the beginning of the game and then moved their other pieces back to their starting positions. You can't just start with a pawn missing as most chess engines don't support that. Second, nothing in this article has implied that Leela is a better chess engine than Stockfish. The reason that specific bonus game is considered notable is precisely because it was against the world's best chess engine. This is the same reason that Judit Polgár's win against Gary Kasparov is highlighted as a "Notable game" at Judit Polgár, not to mislead the reader into thinking that Judit Polgár is a better chess player than Gary Kasparov, but because beating the world's best chess player is notable. Finally, the game wasn't just picked out of thin air by Wikipedia editors. It was highlighted here and here. I hope that clears things up a bit. Kaldari (talk) 09:55, 24 November 2018 (UTC)

If you don't believe me, look at the end of move 4 in the game. The positions are exactly as in the diagram you give above. Kaldari (talk) 09:59, 24 November 2018 (UTC)

Thanks for clearing that up. I retract everything I wrote above about pawn odds, and apologize for the misunderstanding. --Guy Macon (talk) 10:16, 24 November 2018 (UTC)

Why 'she' and not 'it' ?

Is this standard practice to refer to chess engines this way? J mareeswaran (talk) 11:09, 4 February 2019 (UTC)

No but for whatever reason, it's become standard terminology to refer to Leela as "she". As far as I can tell, it's because "Leela" is a female name. Banedon (talk) 14:59, 4 February 2019 (UTC)

Article Evaluation (Wiki Ed)

Hello,

I have recently evaluated this article for a class assignment. I think this article is good but more could be added to make it better. Some of the notes I have may help in the bettering of this article.

• "despite evaluating orders of magnitude fewer positions while using MCTS.": quite confusing, could be changed to be more clear. Some things are unclear.

• Says that version 0.20 was released after the paper on Alpha Zero, but does not specify the date when the paper or version was released.

• I think things could be updated or be more specified (Last paragraph of the competition results in particular).

• I think there could be a section about how, specifically, Leela Chess Zero was created.

• Some of the information in the sources are not used or not specified in the article. For example, it could contain pictures or brackets of the standings.

• The articles could give more about the standings of Leela in the tournaments like the amount of points scored.

LingschLuke (talk) 17:15, 7 February 2019 (UTC)

@LingschLuke: Thanks for the great suggestions! I've just rewritten the paragraph about the influence of the new AlphaZero paper per your suggestion. Kaldari (talk) 14:59, 14 February 2019 (UTC)

Explanation of How it Works

I've added a little bit to the lead mentioning that it's neural network based. But maybe it would be a good idea to have a bit more in the lead + a separate section going over how Leela works? ArguMentor (talk) 13:12, 14 February 2019 (UTC)

Citation issue

The article says "See the season 14 archives at http://tcec.chessdom.com/archive.php " but that URL doesn't lead directly to the season 4 archives. You are expected to figure out what to click. Is there way way to create a link that goes directly to the section? --Guy Macon (talk) 08:01, 16 February 2019 (UTC)

Do we refer to Leela as "she" or "it"?

In view of the recent edit switching from "she" to "it", I think this should be discussed. I have a slight preference for "she" (this is also the gender used by Leela developers, e.g. here) but don't mind "it". Pinging @Haifadude: and @J mareeswaran: as probably interested in this question.

Either is fine with me, but we should be consistent. Kaldari (talk) 17:15, 12 August 2019 (UTC)

it. It's not human. – ishwar  (speak) 20:48, 12 August 2019 (UTC)

Does Leelenstein deserve its own article?

Leelenstein just beat Stockfish in CCC 10. Does it deserve its own article? It's a separate engine from Lc0, but only in the sense of a different neural network - the binary that executes the search is mostly the same (as far as I know the only difference is that Leelenstein uses trade penalty). I'm favoring a separate article, but asking here first. Banedon (talk) 12:55, 23 September 2019 (UTC)

Personally, I doubt there are enough independent reliable sources to make a real article about Leelenstein. You might want to start it here and see if grows big enough to split off. Kaldari (talk) 13:54, 23 September 2019 (UTC)

Indeed. ZBalling (talk) 02:10, 23 October 2019 (UTC)

Who holds the copyright to the Lc0 logo?

There's a black-and-red Lc0 logo that can be seen on top of the Lc0 blog. I think it'd be nice to include in this article. Is it in the public domain? If not, who holds the copyright to it? Does anyone know? Banedon (talk) 07:04, 18 November 2019 (UTC)

@Banedon: The Lc0 logo isn't copyrightable. Feel free to upload it to Commons and use the {{textlogo}} license template. Kaldari (talk) 17:24, 19 November 2019 (UTC)

@Banedon: I went ahead and added it. Thanks for the suggestion. Kaldari (talk) 17:45, 19 November 2019 (UTC)

AllieStein as a Leela Chess Zero variant

I don't think it's fair to say AllieStein is a Leela Chess Zero variant. Both the binary and the net are different. One could say it's an AlphaZero variant, but not Lc0 variant. Ping @Ramzuiv:. Banedon (talk) 07:26, 21 April 2020 (UTC)

I'm inclined to agree with you regarding AllieStein and Leela in direct comparison. However, when considering Allie and Stein separately, both projects can more fairly be viewed as variants of Leela- both projects use the same code for evaluating NNs as Lc0, Allie uses the same nets as Lc0, and the Stein network was originally created with Lc0 in mind as a backbone. The modular nature of NN engines certainly gives rise to a Ship of Theseus situation.

It's worth reflecting on the central role the NN evaluator has in all three projects- it allows Allie, which originally used Lc0's net, to run the Stein network (which was originally paired with the Lc0 search algorithm), whereas AlphaZero, to the best of my knowledge would not be able to use the Stein network, nor could Allie use AZ's network. Certainly the one common core of Allie, Stein, and Leela all using the Lc0 evaluator isn't enough to justify calling AllieStein an Lc0 variant, but it does make them part of a larger project and heritage, which is distinct from the (ideologically similar) AlphaZero -Ramzuiv (talk) 07:54, 21 April 2020 (UTC)

That's kind of the point right? NN engines require a binary and a neural network. Allie is a binary, Stein is a network, and Lc0 has both its own binary and network. Therefore you could have AllieStein, but you could also have AllieZero or Leelenstein. They're all variants of AlphaZero because they all use the same methodology, but I don't see how Allie and Stein are variants of Lc0. Banedon (talk) 12:12, 21 April 2020 (UTC)

So what do you propose doing? -Ramzuiv (talk) 18:07, 21 April 2020 (UTC)

I think AllieStein needs its own article. Only problem is it doesn't have enough sources to write on (i.e. might not meet WP:GNG). Banedon (talk) 00:34, 22 April 2020 (UTC)

Self-Play Elo section

The "Self-Play Elo" section seems like a lengthy digression that isn't especially relevant to the article, IMO. It also doesn't have any sources and seems likely to be original research. Kaldari (talk) 22:48, 23 July 2020 (UTC)

What is this?

"temperature-based variation" ? Bubba73 ^{You talkin' to me?} 02:36, 6 December 2021 (UTC)

Temperature is a technical concept that makes Leela's play more random. High temperature effectively means Leela more often makes moves other than what it thinks are the best moves. It's a setting that is only meant to be used during training, to help encounter unfamiliar positions and therefore learn them. In real games, there is still a "temp decay" feature to make Leela play more openings; the temperature decays over the course of the game so later Leela always plays the "best" moves. Banedon (talk) 02:46, 6 December 2021 (UTC)

"Leela chess Zero" or "Leela Chess Zero"?

Is it "Leela chess Zero" or "Leela Chess Zero"? The article is inconsistent in this respect, including the title. Which one is it? --Mortense (talk) 16:11, 6 December 2021 (UTC)

It's Leela Chess Zero, with a capital C, as can be seen from their website at https://lczero.org/ and GitHub repo at https://github.com/LeelaChessZero/ . I've updated the article accordingly. -- The Anome (talk) 16:29, 6 December 2021 (UTC)