How to use this in Kohya?

#1
by GeneralAwareness - opened

It is missing the model_index.json so is invalid for the Huggingface hub to download.

Agreed. I too would like to see a downloadable safetensors file for use with Kohya's sd-script. I'm currently stuck in a difficult situation with Lora learning in Flux. Please help me.

Just download all the diffusion_pytorch_model.safetensors files from here, then clone the original Flux Dev HF model repo into a new model repo on your own HF profile, delete the diffusion_pytorch_model files from the /transformer folder in your cloned repo, and upload the downloaded equivalents from here as their replacement. Then link to your cloned repo from a trainer/config. I imagine this should work for most training contexts.

Alternately, just use this pre-conversion to HF Diffusers (I should have thought of/suggested it first):
John6666/flux-dev2pro-bf16-flux

Do realize Kohya does not work with this because it wants peft files. For diffusion based repos of SD it works but not FLUX (tried all sorts of ways).

How did you manage to train 3 million images, when the model was out? sounds fake

How did you manage to train 3 million images, when the model was out? sounds fake

100% this

it makes 6000000 images training since 2 epochs.

@MonsterMMORPG have u tried training with that dev2pro yet ? https://huggingface.co./bdsqlsz/flux1-dev2pro-single (this is a "single safetensor" file)

@MonsterMMORPG have u tried training with that dev2pro yet ? https://huggingface.co./bdsqlsz/flux1-dev2pro-single (this is a "single safetensor" file)

i havent currently i am training to test if fine tuning can be improved and batch size 7 effect

I've trained on it using John6666's diffusers version, and the ai-toolkit trainer leveraged via Colab Pro. In comparison with training on regular Dev, it has seemed to learn photorealistic representations a bit quicker and with greater vividness (using an analogous training config). However, artifacts have been no less forthcoming, and the results were perhaps somewhat comparatively diminished in terms of generalizability. But I haven't yet experimented enough to say anything with great certainty.

I'm also curious to see whether merging with the Dev2Pro model (or/and with LoRAs trained on it) may open up better recipes for improving Schnell or/and accelerating Dev.

Sign up or log in to comment