Flux is a new 12-billion parameter transformer model for text-to-image generation built by Black Forest Labs, competing in the same arena as Stable Diffusion, MidJourney, and DALL-E 3. Most image comparisons don’t really pose much of a challenge to the current batch of generative models, focusing on aspects like aesthetics, image quality, and stylism without stress-testing arguably the most important quality of a model—prompt adherence, aka how closely the model follows the user’s requested prompt.
Here are four of my examples designed to “stress test” the ability of these models to follow ‘relatively’ complicated instructions. They are ranked in order of perceived difficulty, at least with older models.
This is straight text-to-image—we are not considering other features such as weighting, inpainting, ControlNets, etc.
Easy
a fantasy creature with the body of a dragon and a beachball for a head, hybrid, best quality, shadows and lighting, fantasy illustration muted, earthy color palette and detailed textures, influenced by Art Nouveau
Technically we have to give the win to DALL-E though aesthetically the Flux image is decidely cooler.
Medium
A country panicking over meteorological reports showing a complete lack of wind (like doldrums) orders their civilians to do their civic duty and use leaf blowers to keep the wind turbines moving, satirical
It took quite a few iterations (from both models) to generate an appliance that didn’t resemble an oversized blow dryer.
Hard
a cylindrical piano that wraps around a pianist who sits in the center on a spinning stool, photography
When training data around a certain concept is largely expressed in a very singular manner, it can be quite difficult to alter that notion. A great example is a piano. A piano actually exhibits another challenging issue for these models: images that contain “periodic data” (the alternating black/white keys, even fingers to a certain degree).
I’m super impressed with Flux - 99% of the time you end up with images very similar to DALL-E with an extra piano in the image itself.
Impossible
A dog walking a human on a leash, role reversal, absurdism, far side comic style
This idea is so utterly alien to the training data that it’s basically impossible to get this to generate correctly to the point where I wonder if it’s because any potential training data that might include a “human being walked” is likely to be rather saucy and thus is ignored.
Conclusion
Considering that the Black Forest lab schnell model is only 12b parameters and is publicly available, it’s hugely impressive to see it be competitive against the juggernaut that is OpenAI. With some time, it’s not unreasonable that it might be possible to run this locally on a 3090/4090 RTX card with 24GB vram as well.
If you’d like to play with them yourself, they are available on the Replicate playground.
Author’s note: Although these prompts ran against the faster model Schnell, they were re-tested against dev/pro of the Black Forest Lab models with varying degrees of guidance and did not show significant improvement to prompt adherence.
Here is such an example:
