ComfyUI Misc

• 1. ComfyUI Wildcards
  • 1.1. About Wildcards
  • 1.2. gkr-wildcards
    • 1.2.1. Usage
    • 1.2.2. Availability on CivitAI
  • 1.3. Wildcards: How To
• 2. ComfyUI-Workflow
• 3. SafeTensor Cleaner

1. ComfyUI Wildcards

I recently discovered by chance that it is possible to use a structured wildcards YAML file to generate prompts for Stable Diffusion.

1.1. About Wildcards

Wildcards are placeholders inside our prompts that get replaced automatically with words or phrases from a list.

A wildcards YAML file defines reusable lists of words or phrases that ComfyUI can randomly insert into prompts. Each entry is a key followed by a list of options. When a prompt uses a wildcard such as __hair_color__, ComfyUI picks one item from the matching list in the YAML. It is possible to refer multiple wildcards to create a meta-wildcard, for example __hair_color__, __eye_color__, __face_shape__ to create a __face__ wildcard.

Using this method, we can create a large number of prompts with a small number of wildcards.

This allows fast variation, style mixing, and structured prompt generation without writing many separate prompts.

I strarted creating a small YAML structure and used LLMs to generate prompts for a simple theme, then I refined the LLM system prompts manually. After a few iterations and tests, I was able to generate a few themed wildcards (present in the gkr-wildcards folder).

Those wildcards are known to work with ComfyUI.

I used those with ComfyUI-Nvidia-Docker to generate images, placing the generated files in the wildcards folder of the ComfyUI installation within the basedir/custom_nodes/comfyui-impact-pack/wildcards/ folder to be used with ComfyUI-Impact-Pack (note that restarting the container is needed for Impact Pack to pick up the new wildcards).

1.2. gkr-wildcards

This folder contains the YAML files that were generated using LLMs (after multiple iterations of refinement) but before I used an LLM to generate a generic template for the wildcards, as present in the howto folder.

The themes selected reflect my personal interests and the kind of images I am interested in creating.

1.2.1. Usage

The theme of each wildcard is in its name (e.g. gkr-anime.yaml is for anime prompts). Each file follows a common structure, usually when using a wildcard file, we will be able to select the sub-wildcards we want to use in the prompt. For example, __gkr_japan/spotlight_kimono__ will be replaced by a random item from the gkr_japan/spotlight_kimono list.

The most useful keywords in the wildcard files are combo, random and spotlight, as they allow to create more complex prompts.

• combo will build a structured-but-random prompt from some specific relevant to the combo sub-wildcards.
• random will build a random prompt from a list of sub-wildcards.
• spotlight provide a list of randomly generated prompts that were asked to produce the best results for a specific theme.

When using a workflow that supports them, we can type the base theme (e.g. gkr_ttrpg) and should see the list of available sub-wildcards, to select the most appropriate ones.

Combination of combo and random keywords from different themes allow to create various prompts, for example __gkr_ttrpg/random_hero_portrait__, __gkr_cyberpunk/random_backgrounds__ might provide a good combination for a cyberpunk Tabletop RPG hero portrait 😊

A single negative_prompt entry is also present in the files, it can be used to specify a theme specific negative prompts for workflows that support it.

1.2.2. Availability on CivitAI

• gkr_anime
• gkr_comics
• gkr_cyberpunk
• gkr_gothic
• gkr_japan
• gkr_popculture
• gkr_scifi
• gkr_steampunk
• gkr_ttrpg
• gkr_videogame
• gkr_wasteland

1.3. Wildcards: How To

This folder contains a method to generate wildcards YAML files using LLMs. It was generated by using LLMs to analysis the common structure I had defined in the various themes and then using the LLMs to provide a "generic" YAML file that can be used to generate prompts for any theme, with embedded instructions.

See the howto/README.md for more information.

2. ComfyUI-Workflow

This folder contains a "Combined Wokflow" for ComfyUI that I use to generate images using the Wildcards.

See the ComfyUI-Workflows/README.md for more information.

3. SafeTensor Cleaner

This folder contains a script to clean SafeTensor files.

See the SafeTensor-Cleaner/README.md for more information.