Talk:Generative artificial intelligence

Definition

The New York Times definition is Silicon Valley has been gripped by a frenzy over start-ups working on “generative” A.I., technologies that can generate text, images and other media in response to short prompts.

The Pinaya et al. definition is Generative AI refers to a set of artificial intelligence techniques and models designed to learn the underlying patterns and structure of a dataset and generate new data points that plausibly could be part of the original dataset.

The NYT definition can be objected to on the grounds that GAN is generative but does not take prompts as input. NYT was probably conflating the broader category of Generative model with the specific category of Generative pre-trained transformer and similar transformer-based architectures, which happened to become popular in 2022. Furthermore the "text, images and other media" part can be objected to on the grounds that a Generative model can generate outputs such as robot actions and industrial HVAC control which are not creative media.

The Pinaya et al. definition can be objected to on the grounds that it doesn't mean much to a Wikipedia reader who is not an expert in the subject. The common usage of "Generative AI" in 2023, and the reason readers will be looking it up, is to refer to systems like ChatGPT, Midjourney, and so on.

To find a balance, I'm editing the lede to refer first to the more general academic/historical definition (essentially a synonym for Generative model), and then to the more specific common modern usage. Lwneal (talk) 12:17, 11 August 2023 (UTC)

I think we should remove the link to Generative model since that article's discussion of the term is limited to statistical classification; clearly not what most intend when discussing generative AI. 80.2.247.44 (talk) 11:53, 4 July 2024 (UTC)

It may sound surprising, but statistical classification is exactly what ChatGPT and Sora are doing every time you use them! Generative AI models are neural networks built using Machine learning, which is the field of study of applied statistical modeling. Neural networks can be trained with a discriminative objective or a generative objective.

At the risk of oversimplifying, the practical difference between "Deep Learning" in the 2010s and "Generative AI" in the 2020s is that the old systems were trained with discriminative objectives (using supervised learning and human-labeled data) and the new systems are trained with generative objectives (using unsupervised learning).

This is the difference between looking at a picture and recognizing that it is a cat (discrimination), and looking at the word "Cat" and drawing a picture of a cat (generation). The History section could be a good place to explain this distinction. Lwneal (talk) 18:15, 4 July 2024 (UTC)

Nice explanation. Can you incorporate it into the article somehow? Maybe there should be a section that compares/contrasts the various related fields, including such as procedural generation. Michaelmalak (talk) 18:24, 4 July 2024 (UTC)

People have different interpretations of what "generative AI" means, it's a pretty fuzzy term. I don't know if that's the best one, but from my experience, the most common interpretation is "an AI model that generates something based on a prompt". I agree however that one issue with this definition is that it doesn't include things like GANs, which I suppose most people would consider part of generative AI.

I agree that the linked article generative model brings more confusion than clarity about what generative AI is. The definition provided in the link ("a generative model is a model of the conditional probability of the observable X, given a target y, symbolically, P(X|Y=y)") is appropriate for models that predict likely input features based on the actual output, but it's more dubious notably for generative pretrained transformers, which predict the next word. I also think that GPT models do more than superficial statistics. They also process semantics, so the term "statistical classification" isn't ideal to describe them, as it depends on how broad your definition of "statistics" is.

So I would be in favor of removing " using generative models" from the first sentence. It sounds trivial but it's unclear whether that's objectively true. But if there is no consensus for removing it, I will not insist. Alenoach (talk) 20:46, 4 July 2024 (UTC)

References

Synthetic Media vs. Generative Artificial Intelligence

It has been suggested that the article Synthetic media be merged into this one.

• Per that article, "Synthetic media" is "a catch-all term for the artificial production... [of] media by automated means". This includes mechanical, symbolic, and procedural generation systems which have never been referred to as "Generative AI"

• Specifically, the first example given in the Synthetic media article is Maillardet's_automaton, which clearly predates any generative AI system (or indeed any AI system)

• Conversely, the most notable generative AI systems are large language models, which are not mentioned in Synthetic media. I can find no usage connecting LLMs to that term in any of either article's citations.

• Per Google Trends, the phrase "Generative AI" was almost unknown before the year 2022, but became increasingly common starting in October 2022 (the month after the release of Stable_Diffusion). The search term "Generative AI" appears to be at least 100 times more common than "Synthetic Media" as of 2023.

Based on these facts I suggest that the two articles not be merged, and that "Generative AI" continue to refer to the specific type of AI that became extremely well-known starting in late 2022: the type of AI that uses a neural network to understand natural language prompts. Lwneal (talk) 17:09, 19 May 2023 (UTC)

I agree MensaGlobetrotter (talk) 21:23, 24 July 2023 (UTC)

merge and redirect to the other article to keep history or whatever Sebbog13 (talk) 17:19, 28 July 2023 (UTC)

I agree that we should work on merging these and other related articles. I just left a similar comment on the Computational creativity talk page, saying: Part of the issue here is that there are multiple articles -- Artificial intelligence art‎, Computational creativity, Generative artificial intelligence, Synthetic media, Synthography, and more -- that all cover roughly the same topic, which divides editors' time, leading to issues with all of these articles. It's worth trying to merge some or all of these articles so we have fewer articles to focus on. Elspea756 (talk) 15:44, 3 August 2023 (UTC)

The following is my summary:

• The term "Deepfake" came into usage in 2018, referring to a controversial type of image manipulation technology which uses deep neural networks to convincingly edit the faces of people in video, notably for pornographic or propagandistic purposes. It is a common search term.
• The term "Generative AI" came into usage in 2022, referring to a newer type of technology (the pretrained transformer) which uses deep neural networks to translate natural language commands ("prompts") into text, images, video, etc. This technology is also controversial, but for different reasons. It is a common search term.
• The term "Synthetic media" is not in common usage outside of Wikipedia. Several sources cited in that article define it as a synonym for Deepfake. However, the article covers not just deepfakes, but a range of topics including Artificial intelligence art, Procedural generation, Maillardet's automaton, etc.
• "Artificial intelligence art" is a very common search term and could refer to any kind of art created by AI, but in practice today usually refers to Text-to-image model.
• "Synthography" is not in common usage outside of Wikipedia and appears to be a recently-coined neologism synonymous with Artificial intelligence art. Several sources cited in Synthography do not themselves contain the word "Synthography".
• Computational creativity covers a broad and difficult-to-define topic, but does cite sources. The article is well-researched but somewhat out of date. It is not a common search term.
• "Artificial imagination" is not in common usage outside of Wikipedia. The article is not particularly well researched and cites sources that do not contain the phrase "Artificial imagination".
• Generative art, Algorithmic art, Cybernetic art, Computer art, Artificial intelligence art and Digital art all seem to overlap to some extent.

Based on WP:NEO and WP:REDUNDANT it seems to me that Synthography could be merged into Artificial intelligence art and Artificial imagination could be merged into Computational creativity.

Do these seem reasonable? Lwneal (talk) 18:05, 4 August 2023 (UTC)

Yes, the mergers you have proposed sound like a good start! I'd say that there are a few problems with this analysis of these article, because it is largely based on the articles themselves that have their own problems. For example, "Generative artificial intelligence" has been a common concept in the field for decades. "Generative Art" has been a concept since at least the1960s, quickly looking at my books I've got an MIT Press book that talks about "generative artificial intelligence" that came out in 2009, "Generative Adversarial Networks" have been around since 2014, but here on Wikipedia we have an article written on "Generative artificial intelligence" that makes it sound like this didn't exist before 2023, likely because that's when the majority of the article's contributors first learned about it. This is part of the problem that hopefully merging some of these articles can help fix, where editor knowledge isn't split across so many similar articles, with some of these articles showing glaring problems because relatively few editors are looking at them. So, yes, the mergers you have proposed sound like a good start! Thanks, Elspea756 (talk) 19:35, 4 August 2023 (UTC)

Synthography has been merged and redirected into Artificial intelligence art to start with.

I suspect a problem with many of the articles in this space is one of overlapping definitions. For example, the word "Generative":

• "Generative" the statistical machine learning term which means "models the joint probability distribution of the training dataset", opposite of "discriminative"
• "Generative" the dictionary word which means "creative, capable of originating or producing"

The nerds who invented GAN and GPT probably intended the former definition. But the public at large increasingly uses it in the latter sense- especially because the most famous "generative" systems to date can output art and poetry, which seems closely related to creativity.

This gets tricky when AI that models joint probability is used for non-creative tasks like robot control, and when AI that doesn't (for example, decision tree AI) is used for creative tasks like painting pictures. Which one is "generative"? Both?

That 2009 book sounds interesting. Maybe looking back to the earliest published instances of the phrase "Generative artificial intelligence" can decide the issue. Lwneal (talk) 17:31, 10 August 2023 (UTC)

For reference, pageviews during the last year for many of the terms compared by Lwneal: [1] --LaukkuTheGreit (Talk•Contribs) 08:33, 20 September 2023 (UTC)

no deleting to keep the revision history Sebbog13 (talk) 19:56, 4 August 2023 (UTC)

Right, yes, nothing would be deleted from revision histories when merging, and there will be redirects from the previous articles to the article they've been merged into, so that existing links still work and people searching for the previous title will find the article it has been merged into. Elspea756 (talk) 20:12, 4 August 2023 (UTC)

I agree. Keep them separate. They can reference each other. 193.116.103.233 (talk) 03:16, 3 September 2023 (UTC)

Comment Keep them as separate articles. 2001:8003:9100:2C01:D015:BF4E:5E4C:B752 (talk) 07:51, 10 December 2023 (UTC)

References

Wiki Education assignment: Research Process and Methodology - SU23 - Sect 200 - Thu

This article was the subject of a Wiki Education Foundation-supported course assignment, between 24 May 2023 and 10 August 2023. Further details are available on the course page. Student editor(s): NoemieCY (article contribs).

— Assignment last updated by NoemieCY (talk) 13:08, 14 July 2023 (UTC)

Added Benefits and Risks

I added benfits and risks of generative AI. This is based on an expert briefing of the UN Security Council in July 2023. MensaGlobetrotter (talk) 21:24, 24 July 2023 (UTC)

Thank you for your contribution, it adds valuable context. However, it appears to have been written in the style of ChatGPT's default "helpful assistant" mode, which is excessively wordy and repetitive. Because of this, the new section takes up approximately half of the entire article.

If you did use an LLM to assist in this summary, would you consider re-prompting with a different system prompt? For example, "Summarize the above text. Do not repeat yourself. Be concise." Or you might summarize the briefing in your own words, using the LLM only as a reading tool. See https://en.wikipedia.org/wiki/Wikipedia:Using_neural_network_language_models_on_Wikipedia Lwneal (talk) 20:43, 25 July 2023 (UTC)

Thanks for your comments. I had used LLM to hone in on and split out benefits and risks. These were spread across a very long article, and I was worried that I would miss content or introduce bias. I have reworked the sections in question with much more manual effort.MensaGlobetrotter (talk) 21:46, 25 July 2023 (UTC)

I've removed this section that was largely written by ChatGPT. The entire section was mostly or completely untrue and had nothing to do with generative AI, instead describing other types of AI. The source says it is describing "generative and other artificial intelligence." For example, generative AI is not being used for "autonomous weapons systems." Elspea756 (talk) 12:19, 26 July 2023 (UTC)

You do make some valid points. However, please consider the possibility that your definition of generative AI is too narrow. Please also consider deleting just the points you find problematic. The first risk noted was that generative AI is "capable of autonomously creating content, raises concerns about the spread of disinformation and hate speech, potentially becoming a defining moment for such threats.". Is this not a risk of generative AI? MensaGlobetrotter (talk) 14:11, 26 July 2023 (UTC)

Did the Hubble Space Telescope run on Generative AI?

In the 1980's and 1990's, the term "Generative Planning" or "Generative AI Planning" was used in some academic AI literature such as this article published by a NASA team in 1997:

"Our deliberator is a traditional generative AI planner based on the HSTS planning framework (Muscettola 1994)"

A definition of HSTS from the cited paper:

"HSTS (Heuristic Scheduling Testbed System) is a representation and problem solving framework that provides an integrated view of planning and scheduling... In the paper we describe an integrated planner and scheduler for short term scheduling of the Hubble Space Telescope"

Does this mean the Hubble Space Telescope was running an early version of ChatGPT?

Reading that article, it is clear that generative AI planner refers to Automated planning and scheduling. The word "generative" here has a meaning closer to generative grammar or Generative Modelling Language, and refers to the practice of generating many possible sequences of actions in a hand-coded state space, using trial-and-error to solve a combinatorial optimization problem.

The "generative planning" of the Hubble Telescope was certainly a type of artificial intelligence- but it was Symbolic AI or GOFAI, not involving a generative model, a neural network or machine learning. Lwneal (talk) 21:46, 22 September 2023 (UTC)

References

Extremely unhappy because the Generative AI concept has been hijacked by large language models

I am extremely unhappy with the current framing of generative AI as being derived from / being a variant of transformers and large language models. The fact that LLMs generate paragraphs of text is not a sufficient argument. Historically, the generative AI movement stems from Generative Adversarial networks. The seminal article by Goodfellow (60k citations) is not even mentioned in this Wikipedia entry. This is a rewriting of history and is an usurpation by text people of fundamental findings from pixel people. Yuck. It reminds me of my student who said in a presentation that 'neural networks started with Krishevsky'. 129.125.178.72 (talk) 11:59, 2 October 2023 (UTC)

I agree that this article should include more detail on pre-2020 unsupervised deep learning. Generative Adversarial Nets should get their due (and perhaps also Boltzmann machines, Deep Belief nets, etc). Edits are welcome.

I disagree with the characterization of pixel people as antagonistic to text people. They're often the same people! Ian Goodfellow and Ilya Sutskever were co-authors[2] even before the original GAN paper. Alec Radford, after DCGAN[3], went on to work with Sutskever on GPT-2 and GPT-3[4]. Vaswani and Parmar wrote "Attention Is All You Need"[5], and they also wrote "Image Transformer"[6].

If there ever was a distinction between text people and pixel people in AI, it ended when the text people were all fired. Most of the key deep learning researchers of the 2010s, whether they trained on image or text data, used the same methods and were associated through co-authorship to one community. In fact, many of them were students of the same three professors. Lwneal (talk) 23:34, 2 October 2023 (UTC)

Wiki Education assignment: Technology and Culture

This article was the subject of a Wiki Education Foundation-supported course assignment, between 21 August 2023 and 15 December 2023. Further details are available on the course page. Student editor(s): Nadpnw, AdvaitPanicker, Ferna235 (article contribs).

— Assignment last updated by Thecanyon (talk) 05:32, 12 December 2023 (UTC)

Wiki Education assignment: Intro to Technical Writing

This article was the subject of a Wiki Education Foundation-supported course assignment, between 19 February 2024 and 18 March 2024. Further details are available on the course page. Student editor(s): Tskhattab (article contribs).

— Assignment last updated by Tskhattab (talk) 22:37, 21 February 2024 (UTC)

Energy consumption?

Shouldn't there be a section about the very high energy consumption required to create and run generative IT? Rosieredfield (talk) 18:44, 28 March 2024 (UTC)

Totally agree! Maybe it's time to make my first ever edit... Ogre.Yoga (talk) 02:12, 2 April 2024 (UTC)

Relevance of AlphaFold

Is AlphaFold really considered generative AI, as suggested in the subsection "Molecules"? My impression is that it predicts the 3D structure of folded proteins, but does not really generate something in the usual sense. Alenoach (talk) 19:04, 30 April 2024 (UTC)

I believe technically it generates predictions. It is strange, however, that on their website I am unable to find any mention of generative AI (outside of "you can generate your own AlphaFold predictions using") but I know anecdotally AlphaFold is seen as genAI. Sojoelous (talk) 22:43, 8 May 2024 (UTC)

Misuse in journalism

Some of the content of the subsection "Misuse in journalism" is not necessarily misuse, and could arguably be moved to the article Applications of artificial intelligence, especially given that this subsection is quite lengthy. There isn't yet an article about "Artificial intelligence in journalism", but creating one would be another option. Alenoach (talk) 02:15, 27 May 2024 (UTC)

Subsection "Training Data Collapse"

"Training data collapse" doesn't seem to be a widespread concern.

Model "collapse" occurs when recursively reusing synthetic data, from models that are worse and worse; where each model is imperfectly trained from the synthetic data of the previous generation. For example, when training model 2 with the output of model 1, the model 2 will be less good. And model 3 trained from model 2 will be worse. But when it's not recursive, it doesn't lead to a collapse. And you can always filter out bad quality text or AI-generated images when scrawling the web. It's still a concern, but much less widespread than the other ones listed in the "Concerns" section.

Perhaps move this section or some of its content to a more specific article, for example Synthetic data? Alenoach (talk) 05:28, 27 August 2024 (UTC)

Actually, I merged "Training Data Collapse" into the newly added section "Content quality". So it's less of a problem now, although it's still possible to migrate part of its content to another article like Synthetic data. Alenoach (talk) 05:51, 27 August 2024 (UTC)