DescriptionAlgorithmically-generated landscape artwork of forest with Shinto shrine.png |
Demonstration of the usage of negative prompting on algorithmically-generated artworks created using the Stable Diffusion V1-4 AI diffusion model. The purpose of a negative prompt is to instruct the AI to omit certain objects, motifs or visual elements when generating an image, as opposed to a positive prompt which instructs the AI to include such things.
This image aims to illustrate the process in which negative prompting within Stable Diffusion can be used to fine-tune the output of an AI generated image based on the desires of the user, as one part out of three images showing each step of the procedure.
- Procedure/Methodology
All artworks created using a single NVIDIA RTX 3090. Front-end used for the entire generation process is Stable Diffusion web UI created by AUTOMATIC1111.
A single 768x512 image was generated with txt2img using the following prompts:
Prompt: Hakurei Shrine in distance, Gensokyo, nature landscape, landscape art, far view from distance, traditional Japanese architecture in distance, Shinto shrine in distance, forests, mountains, rivers, art style of Craig Mullins and jordan grimmer and tyler edlin and darek zabrocki and raphael lacoste
Steps: 50, Sampler: Euler a, CFG scale: 7, Seed: 1411213889, Size: 768x512
From there, two additional images were generated using the same seed and positive prompt, however this time using negative prompts:
- Second image
Prompt: Hakurei Shrine in distance, Gensokyo, nature landscape, landscape art, far view from distance, traditional Japanese architecture in distance, Shinto shrine in distance, forests, mountains, rivers, art style of Craig Mullins and jordan grimmer and tyler edlin and darek zabrocki and raphael lacoste
Negative prompt: green trees
Steps: 50, Sampler: Euler a, CFG scale: 7, Seed: 1411213889, Size: 768x512
- Third image
Prompt: Hakurei Shrine in distance, Gensokyo, nature landscape, landscape art, far view from distance, traditional Japanese architecture in distance, Shinto shrine in distance, forests, mountains, rivers, art style of Craig Mullins and jordan grimmer and tyler edlin and darek zabrocki and raphael lacoste
Negative prompt: round stones, round rocks
Steps: 50, Sampler: Euler a, CFG scale: 7, Seed: 1411213889, Size: 768x512
Afterwards, for all three images, the image was extended by 128 pixels on both the left and right sides using a single pass of the "Outpainting mk2" script within img2img. This was done using the same seed value of 1411213889 earlier, along with a setting of 100 sampling steps with Euler a, denoising strength of 0.8, CFG scale of 7, mask blur of 25, fall-off exponent value of 1.8, colour variation set to 0.03. The prompts used were identical to those utilised during the first step. This subsequently increases the image's dimensions to 1024x512, while also revealing additional foilage and architectural elements which were previously absent from the original AI-generated image.
Then, two passes of the SD upscale script using "Real-ESRGAN 4x plus anime 6B" were run within img2img. The first pass used a tile overlap of 64, denoising strength of 0.3, 50 sampling steps with Euler a, and a CFG scale of 7, using an identical seed of 482112941 for all three images. The second pass used a tile overlap of 128, denoising strength of 0.1, 30 sampling steps with Euler a, and a CFG scale of 7, using an identical seed of 3320472043 for all three images. |
Permission (Reusing this file) |
- Output images
As the creator of the output images, I release this image under the licence displayed within the template below.
- Stable Diffusion AI model
The Stable Diffusion AI model is released under the CreativeML OpenRAIL-M License, which "does not impose any restrictions on reuse, distribution, commercialization, adaptation" as long as the model is not being intentionally used to cause harm to individuals, for instance, to deliberately mislead or deceive, and the authors of the AI models claim no rights over any image outputs generated, as stipulated by the license.
- Addendum on datasets used to teach AI neural networks
Artworks generated by Stable Diffusion are algorithmically created based on the AI diffusion model's neural network as a result of learning from various datasets; the algorithm does not use preexisting images from the dataset to create the new image. Ergo, generated artworks cannot be considered derivative works of components from within the original dataset, nor can any coincidental resemblance to any particular artist's drawing style fall foul of de minimis. While an artist can claim copyright over individual works, they cannot claim copyright over mere resemblance over an artistic drawing or painting style. In simpler terms, Vincent van Gogh can claim copyright to The Starry Night, however he cannot claim copyright to a picture of a T-34 tank painted with similar brushstroke styles as Gogh's The Starry Night created by someone else. |