Wikipedia talk:Wikipedia Signpost/2023-07-17/Recent research

Discuss this story

Presumably the preprint about WiCE, after giving the example quoted above, goes on to discuss the problems with both the sentence from the article Santa Maria della Pietà, Prato ("13th-century icon" is not supported by the source) and the "sub-claims" GPT-3 generated from it (clearly the "icon" can't be both 13th-century and from 1638)? If so, what does it say? I think the original source has misunderstood that the 14th-century image itself (attributed to Giovanni Bonsi), as opposed to a "depiction of the miraculous event" (unspecified, but it occurred in the 17th century), is the fresco at the centre of the later altarpiece (painted by Mario Balassi in 1638, and on canvas rather than in fresco according to the Italian Wikipedia article), so that doesn't help. Ham II (talk) 11:23, 17 July 2023 (UTC)[reply]

Yes, indeed! Was this passage deliberately selected as all screwed up (because of misunderstanding the Italian original), or was it just chance? I'm not sure what this proves, other than the old maxim GIGO. All three of the "Sub-claims predicted by GPT-3" are factually incorrect! Johnbod (talk) 03:46, 19 July 2023 (UTC)[reply]

@Johnbod Looks like a case of Kozierok's First Law: "The apparent accuracy of a Wikipedia article is inversely proportional to the depth of the reader's knowledge of the topic."

Andreas JN 466 09:56, 19 July 2023 (UTC)[reply]

Thanks and small correction on Wikipedia ChatGPT plugin

Thanks for covering this work! One small correction RE:

"The plugin works by first performing a Google site search on Wikipedia to find articles matching the user's query"

This was true of the earliest version of the plugin, but for production we've switched to leveraging the Wikimedia Search API to find articles matching the user's query. We'll update the docs/README to reflect this (our quick R&D on this outpaced our technical documentation, but catching up now)! Maryana (WMF) (talk) 22:10, 17 July 2023 (UTC)[reply]

Thanks Maryana! Yes, the writeup here was solely based on the Wikimedia Foundation's public documentation and thus dependent on its correctness. (In the future I would suggest double-checking and if necessary fixing documentation in advance of such high profile public announcements.)

Just to understand correctly, I assume you mean that the plugin continues to be developed to be able to use both Google site search and the Wikimedia Search API (with a feature flag [1] that is currently set to "google_search_is_enabled"), but that the latter is selected as the preferred search provider right now in the settings. (Feel free to correct me as I may have misread the code.)

Anyway, that's good to know - I had in fact been wondering about that aspect, also in the light of some concerns about proprietary services that were voiced recently (not by me) in the Wikimedia AI Telegram group.

Regards, HaeB (talk) 02:53, 18 July 2023 (UTC)[reply]