jc-builds/sd2-inpainting-coreml

Stable Diffusion 2.0 Inpainting - CoreML

Warning: This model has known quality issues. For production use, we recommend Realistic Vision Inpainting instead.

CoreML conversion of Stable Diffusion 2.0 Inpainting for Apple Silicon devices (iPhone, iPad, Mac).

Model Details

Property	Value
Base Model	Stable Diffusion 2.0 Inpainting
Resolution	512x512
UNet Channels	9 (latent + mask + masked image)
Prediction Type	V-Prediction
Attention	SPLIT_EINSUM (optimized for ANE)
Safety Checker	Not included

Known Issues

This model may produce suboptimal results due to:

1. V-Prediction: SD 2.0 uses v-prediction instead of epsilon-prediction (SD 1.5), requiring different scheduler math
2. Quality: Some users report blurry or inconsistent outputs
3. Prompt Following: May not follow prompts as accurately as SD 1.5-based models

Recommended Alternative

For better results, use Realistic Vision Inpainting:

• Based on SD 1.5 (epsilon-prediction)
• Higher quality photorealistic output
• Includes NSFW safety checker
• Better prompt adherence

Files

File	Size	Description
sd2-inpainting-coreml.zip	2.39 GB	Full model bundle

Bundle Contents

Resources/
├── TextEncoder.mlmodelc      # CLIP text encoder
├── Unet.mlmodelc             # 9-channel inpainting UNet
├── VAEDecoder.mlmodelc       # Latent to image decoder
├── VAEEncoder.mlmodelc       # Image to latent encoder
├── vocab.json                # Tokenizer vocabulary
└── merges.txt                # BPE merges

Usage Notes

If using this model, your inpainting pipeline must handle v-prediction correctly:

# V-prediction formula (SD 2.0)
x₀ = αₜ · xₜ - σₜ · v

# vs Epsilon-prediction formula (SD 1.5)
x₀ = (xₜ - σₜ · ε) / αₜ

Ensure your scheduler implements v-prediction math, otherwise output will be corrupted.

License

This model is released under the CreativeML Open RAIL-M License.

You CAN:

• Use commercially
• Redistribute
• Modify and create derivatives

You MUST:

• Include license and attribution
• Not use for illegal purposes
• Not generate content exploiting minors
• Not use for harassment or deception

Attribution

• Original Model: Stable Diffusion 2.0 Inpainting by Stability AI
• CoreML Conversion: Using Apple ml-stable-diffusion

Conversion Details

Converted using Apple's ml-stable-diffusion toolkit:

python -m python_coreml_stable_diffusion.torch2coreml \
  --model-version stabilityai/stable-diffusion-2-inpainting \
  --convert-unet \
  --convert-text-encoder \
  --convert-vae-decoder \
  --convert-vae-encoder \
  --attention-implementation SPLIT_EINSUM \
  --bundle-resources-for-swift-cli \
  -o output

Related

• Realistic Vision Inpainting (Recommended)
• Apple ml-stable-diffusion
• Stable Diffusion 2.0 Inpainting