Prompts are the key to unlocking Stable Diffusion’s capabilities. Mastering the art of Stable Diffusion Prompts involves learning formats, structures, techniques, iterations, and nuances for composing effective prompts that produce your desired AI art.
This comprehensive guide covers everything you need to know to create amazing AI generated art with Stable Diffusion through expert prompting.
Table of Contents
Introduction to Prompting Stable Diffusion
A prompt tells Stable Diffusion what kind of AI generated art you want it to create. Prompts activate connections and associations in the AI to produce new images matching your description. Even the same prompt yields diverse creations when sampled multiple times.
For example: “A wise old wizard reading an ancient tome”
Prompting is a learnable skill that improves through experimentation. Follow this guide to level up your skills in generating AI art through effective prompting.
Starting Your Stable Diffusion Prompting Journey
Before diving into mastering prompts, your first step is gaining access to Stable Diffusion to generate images from your prompts.
Thankfully, there are straightforward ways to start prompting online for free using web apps like the official Stable Diffusion demo, NightCafe Creator, and more.
Get started with online prompting by visiting our list of free websites for generating images. Or reference our Quick Start Guide for setup details.
For those wanting more flexibility, we have a comprehensive guide covering additional options like local installations and cloud computing.
With your platform ready, it’s time to immerse yourself in prompting and start crafting AI generated art.
Now, let’s explore the essential techniques for creating effective prompts that will enhance your artistic journeys.
Essential Prompt Formats
Stable Diffusion supports multiple prompt formats:
- Plain text prompts – Freeform descriptive sentences specifying your vision.
- Example: “A wise old wizard standing in a mystical forest casting a spell”
- Metadata tags – Special prompts enclosed in brackets that define styles, mediums, etc.
- Example: [medium: oil painting], [fantasy theme]
- Weighted prompts – Repeating words in your prompt to increase their influence.
- Example: “A very very old wise wizard casting a spell”
- Negative prompts – Exclude unwanted elements by prefixing them with “no”, “avoid”, etc.
- Example: No beard.
- Embedded prompts – Prompt text surrounded by <<>> that associates with a section of the image.
- Example: <<In the background is a towering gothic castle on a hill>>
- Sampling prompts – Additional prompt provided when sampling an existing image to create variations.
- Example: Make the wizard look younger with brown hair.
- Prompt sequences – Series of incremental prompts building on each other.
- Example: “A wizard in a forest, The wizard is old with a beard, He is casting a spell with glowing hands, The spell glows blue and lights up the forest”
- Templates – Reusable preset prompt structures.
- Example: [An elderly [character] standing in a [setting] casting [magic effect] with [lighting/color] effects]
Structuring Effective Prompts
Crafting a well-structured prompt involves more than just listing descriptive details. Arrange your prompt to tell a logical, cohesive story that guides Stable Diffusion through the key elements and relationships to generate your desired image.
Well-structured long prompts should:
- Summarize the essence first.
- Example: “A wise old wizard”
- Separate into logical sections.
- Example: Physical description, environment, action.
- Use complete sentences.
- Vary sentence length.
- Employ vivid, descriptive language.
- Example: “Shimmering azure spell” instead of just “blue magic”.
- Repeat key terms.
- Example: Wizard, old, mystical.
- Establish relationships between elements.
- Example: The wizard’s spell lit up the dark forest.
- Tell a cohesive story.
- Use select salient details.
- Example: Pointed wizard’s hat instead of describing full outfit
- Restate the core idea at the end.
- Example: “The ancient wizard harnessed mystical energies.”
For example: “A stooped old wizard stands amidst the ancient trees, focused on his spellcasting. His gnarled hands cup an orb of light, conjured magical energies illuminating his wrinkled face and long white beard. The swirling mist around him glows from the spell’s azure radiance”
Strategies for Exploring the Prompt Space
Take an iterative approach to systematically explore variations of your Stable Diffusion prompt. Make incremental changes to keywords, styles, attributes, details, tone and more to uncover new creative directions for Stable Diffusion to render.
Systematically explore the prompt space by:
- Trying different keywords and descriptors.
- Example: “Sinister”, “friendly”.
- Varying art styles and mediums specified.
- Example: “Impressionist”, “geometric”.
- Altering attributes like color, lighting, composition.
- Example: “At sunset”, “low angle perspective”.
- Adding or removing details.
- Example: “Has a staff”, “no beard”.
- Shifting emotional tones.
- Example: “Confident”, “unsure”.
- Combining prompt snippets.
- Example: “A joyful cartoon wizard in watercolor style.”
- Adjusting weights and negatives.
- Example: “A very very happy wizard. No darkness.”
- Interpolating between prompts.
- Example: “Halfway between wizard and robot.”
- Following unexpected tangents.
- Example: “cyberpunk wizard”
For example: “A joyful painterly cartoon wizard”
Guiding the Image Generation
When providing prompts to Stable Diffusion, you can choose to closely match your description or diverge from it. Tailor your prompt details and parameters based on your goals for guiding the AI image generation.
To closely match prompts:
- More details – Exact hat shape, beard length
- Repeat key terms
- Increase CFG scale to 9+
- Specify styles – [gothic engraving style]
- Sample extensively – Generate 50+ samples
- Inpaint divergent parts – Edit with img2img
To diverge from your prompts:
- Use abstract metaphors. Example: “Conjurer of secrets”.
- Introduce ambiguity. Example: Just “a wizard” vs “a friendly wizard”.
- Decrease CFG scale to 7
- Provide minimal guidance. Example: “A character.”
- Add noise – [noise: 0.5]
For example:
Detailed prompt: “An elderly bearded wizard in a black robe and pointed hat casting a glowing spell.”
Abstract: “A sage harnessing the primordial powers.”
Directing Image Composition
Carefully guiding image composition is key to ensuring Stable Diffusion generates images that match your creative vision. Layout, framing, perspective and other compositional elements set the tone and direct the viewer’s eye through the image.
Guide composition by:
- Explicit terms – “low angle”, “rule of thirds”
- Positioning – “Wizard on right side”
- Relative sizes – “Enormous glowing spell”
- Lead eye through story – Describe visual path
- Embedded prompts – <<castle in distance>>
For example: “[Low angle view] In the foreground the wizard gestures, staff pointed at mountains. Wizard in sharp focus on [rule of thirds right]. <> Smaller mountains underneath twilight sky.”
Varying Artistic Style
One of the most exciting aspects of AI image generation is the ability to easily explore different artistic styles. Keep the main subjects consistent, but vary the overall style by specifying different art movements, mediums, transfers, vectors and creative outliers.
Keep subjects consistent but vary style by:
- Art movements – Cubist, art deco
- Mediums – Charcoal, vector graphics
- Style transfers – Lego brick wizard
- Style vectors – Interpolate with Ghibli style
- Style outliers – Unique creations through sampling
For example: “A renaissance portrait of a wise wizard”
Creating Variations
Generating multiple variations of an image is a key benefit of AI image generation. Explore different versions by altering details, style, and generations.
Generate variations by:
- Altering details – Different clothes, staves, spells
- Changing style – lighting, color, mood, composition
- Generating many samples – Create 50+ samples
- Incremental edits – Slowly zoom out over generations
- Controlled noise – [noise: 0.5], [variation: 5]
For example: “A wizard reading a book in a candle-lit library.”
Essential Stable Diffusion Prompts Writing Tips
Crafting effective prompts is an art that takes practice. Follow these key tips when writing prompts tailored for Stable Diffusion to improve your image results:
- Be specific but don’t overdefine.
- Example: “A tall wizard” vs “A tall elderly male human wizard wearing a pointed blue hat and flowing robes”
- Use vivid descriptive language.
- Example: “Ethereal azure energy” instead of “blue energy”
- Structure logically.
- Example: Appearance > Setting > Action
- Repeat key terms.
- Example: Wizard, spell, magic, mystical
- Remove ambiguity.
- Example: “A friendly wizard” vs just “A wizard”
- Balance brevity and detail.
- Example: “A bearded wizard casting a spell”
- Specify relationships between elements.
- Example: The spell lit the forest.”
- Guide mood and tone.
- Example: “A sinister wizard standing ominously” vs “A friendly wizard smiling”
- Iterate and refine.
- Example: Try 5 variations of a prompt before selecting one
- Check grammar and spelling.
Thoughtfully honing your prompt writing skills for Stable Diffusion will allow you to better guide the AI creations.
Controlling Artistic Style with Stable Diffusion Prompts
Mention:
- Medium like oil painting, watercolor, vector art, clay sculpture
- Example: [medium: graphite sketch], [oil painting style], [clay sculpture]
- Specific artists or movements like Picasso, Van Gogh, Art Nouveau
- Example: [Van Gogh color and texture], [Picasso cubism], [Art nouveau framing]
- Websites like DeviantArt, ArtStation, Pixiv
- Example: [DeviantArt themes], [ArtStation photorealism], [Pixiv anime style]
- Color palettes and lighting
- Example: [muted earth tones], [high contrast lighting], [pastel palette]
- Textures and level of detail
- Example: [rough brushstrokes], [smooth blending], [highly detailed]
- Time periods and cultures
- Example: [60’s psychedelia], [Mayan influences], [Victorian wallpaper motifs]
- Adjectives describing mood and style
- Example: [ominous atmosphere], [glitch distortions], [magical wonder]
Also consider:
- Aspect ratio like [panoramic wideshot]
- Frames like [polaroid border], [ornate gilded frame]
- Camera effects like [tilt-shift focus], [lens flares]
- Film effects like [overexposed], [monochrome]
- Distressing like [glitch distortion], [peeling paint]
- Textures like [canvas texture], [stone overlay]
- Collage elements like [newspaper clippings]
- Abstract effects like [kaleidoscope]
- Blending modes like [screen layering], [multiply]
When controlling style, provide both general terms like “cubism” and more specific details like “geometric shapes.” Recommend combinations like [ArtStation concept art] + [cinematic lighting]. Explain context and purpose for styles. Start with the most important elements, and give guidance on usage.
Conclusion
This complete guide covers the formats, structures, principles, iterations, and examples needed to craft effective prompts tailored to your creative vision. With practice, you can master prompting to unlock Stable Diffusion’s full artistic potential.
