AI art generation has become a daily tool I leverage in my companies as an entrepreneur. In this article, I'll discuss the evolution of AI image generation through 3 years of trial and error to achieve my own personalized consistant branding.
Murmurings
January 5th 2021 OpenAI released their first image model - DALL·E. This model’s debut sketches were charming, but failed to produce variety when it came to real world images.
Sometime later, OpenAI researchers created GLIDE - OpenAI’s first stable-diffusion model. It leveraged OpenAI's embedding model CLIP to use ranking so that image output was more correlated to it's prompt, known as re-ranking.
How Diffusion Models Work:
- 1
Start with random noise (like static on a TV screen)
- 2
A model is trained to reverse the noise addition process by iteratively refining the noisy data, bringing out image-like structures.
- 3
Over time, a coherent image emerges.
How Re-ranking Works with Images:
- 1
Start with multiple generated images (different versions of the same prompt).
- 2
A model evaluates each image based on how well it matches the prompt and the desired visual quality.
- 3
Each image receives a score, and the images are ranked from best to worst.
- 4
The highest-ranked image is selected as the final output.
The same concepts around GLIDE was then reused in OpenAI's DALL·E 2: now with Stable diffusion, re-ranking and 4x greater resolution. Generating realistic images were no longer an issue. It was around September 2022, just 2 months the birth of ChatGPT, that OpenAI released DALLE-2 to the public.
The two other competing models at this time were Stable-Diffusion 1.4 by Stability AI and Midjourney v1 by Midjourney, Inc. Stable-Diffusion 1.4 was an open-source diffusion model. Midjourney was a closed-source model which worked similarly as Stable-Diffusion 1.4, but used a proprietary, curated dataset that focused on artistic output.
Precursor models such as DALLE-1 were seen as a proof-of-concept, but these newer models now showed great promise for the generations to follow.
Fine-tuning
AI image gen's output at this point is limited to the boundries of their training dataset. For instance, if we train a model on images of fishes strictly, then ask the same model to draw a dog, we'll get a picture that more closely resembles a fish than dog.
This is where fine-tuning on top of a pre-trained model is advantageous. Instead of training a new separate dog model, let's try fine-tining by expose the fish trained model to new pictures and descriptions of dogs.
Multi-millionaire indie hacker Pieter Levels, @levelsio on Twitter/X, created his most lucrative business to date as PhotoAI using Dreambooth. The Dreambooth paper by Google Research was a fine-tuning technique for diffusion image models. This open-sourced technique combined with Diffusers, a framework for working with diffusion models by Huggingface, made it accessible for the average developer to create custom AI image models.
It was at this point in time, I realized that I could wave a prevriblal wand, type a bit of text and out pops whatever image I want. Magic. It was this article that introduced me to Dreambooth in December 2022.
Fine-tuning in practice
I started off with 12 images of myself. The photos were taken in various angles, backgrounds, with a couple of full body shots. I had a Macbook Air M1 which didn't have the GPU power to run these models, let alone train one. As a work around, I used a Jupyter notebook with Google Colab to access a few hours of low level GPUs in the cloud connected to my Google Drive as a file store - costing less than $10[1]. It was fairly straightforward adding images, downloading the model files and waiting about an hour for the fine-tuning process to complete.
My new fine-tuned model was now outputting pictures of myself like magic. Performance was drastically better than the base model, but these photos wouldn't fool anyone that knew me. Below shows a before picture taken that month, and after with the best photo of 100s I created. Most of them looked like this producing images that stretched my features or changed a major feature completely.
Textual Inversion
After lerking various Reddit r/StableDiffusion discussions. I learned that modifying a bit of text used in the process could help focus the output sharper. This bit of text is referred to as a token and is encoded with its correlated image during fine-tuning to create a relationship between the image and the text. It's used like this photo of {token} person.
It's best to pick a token that isn't a real word. To show why let's say pick I pick cherry as my token - photo of cherry person - the output is more likely to show a photo of a cherry than me. This is because there's far more photos of cherries in the base model's training than there are of me in the fine-tuned model.
The same applies with celebrities' names, they are likely to occur frequently in the base model's training dataset. I noticed if I pick a token that is close to the famous person's name, just not exact, and your image output will be weighted towards faces that looks like that celebrity. This is known as celebrity tokens. I picked a celebrity I have been told I look like photo of cara-delevinge-person person, there's also starbyface.com you can use.
The new model using the celebrity token technique worked wonderfully now!
Fair to say I had fun creating what ever crossed my mind. To my eye there were tell-tell signs it was AI generated, but would it fool someone I knew that was inexperienced with AI? Well it even fooled my mom, even if it was only for a few minutes!
I wanted a memento from all this effort and thought to generate a painting of myself, a kind of modern day self-portrait. This would become my profile picture2 on various platforms for years to come.
In-painting
Fast forward 2 years, a lot had changed including my hair. I cut my hair drastically short and no longer matched the woman in my profile picture. The portrait still held significance for me when I looked at it so I wanted to repurpose it and only change the hair to match my new look. And what a perfect use case for in-painting!
Here's what I looked like around that time in a clip of a video comparing Claude 3.5 against OpenAI's first reasoning model o3:
In-painting works similar to photoshopping an image to modify the original image. The way it works is just like the diffusion models we've discussed so far, with the exception that parts of the image are locked from being updated, the locked pixels are also used to influence what the model outputs.
I was able to accomplish the new look first locking all pixels outside the hair and adding a prompt of straight fringed soft mullet.
Necessity breeds innovation
It was March 16th 2025, I had a handful of blog posts on my site. It was a good time to start adding meta tag images for when links are posted to social media. These make links more attractive to click with an image. The style I was after was of a vintage greyscale comic.
Replicate made it easy to connect to models where they worry about the model infrastructure and you can pay per request. I dusted off my account and started experimenting with the leading AI image models at the time creating about 80 images, spending $2.61 and out came a single image I could see myself using.
My luck would soon change on a ski trip just 10 days later. I was checking yesterday's headlines before hitting the slopes and read OpenAI had released a new noteworthy model - GPT‑4o image generation. The model's announcement was particularly impressive for it's ability to analyze an uploaded image and output a similar style along with ability to generate images with precise, stylized, and coherent text.
Here's my chat with ChatGPT from that morning. In the chat, I tried to create the image I was after with the Replicate hosted models. After 4 prompts I had a blog hero in the style I wanted that actually resembled me! All without having to train a custom model - I was beyond impressed!
Image-to-video
Now that I had blog images in a branded style, I wanted my profile picture to match. With a single prompt GPT-4o did a great job at style matching.
Though the new image lost a bit of my resemblance in the process. I wondered if adjusting the character's pose in the self-portrait using an image-to-video model could improve the resemblance.
Image-to-video models are used as the name implies, pass it an image with a prompt out, and out comes a video. Instead of making one picture at a time and chaining them together, the model creates a short sequence of frames all at once in its memory, similar to sketching several pages in a flipbook notebook. It uses the starting image and its knowledge of common movements, like walking or flowing water, to shape how the video plays out. Currently, models can only output seconds worth of a video due to how demanding this all is computationally along with challenges of making the characters and objects stay the same as they move.
Here's the video and frame I end up using from LTX-Video which I ran twice for a whopping price of $0.03:
I tried the one of the top model at the time named Hailuo, which was 33x more expensive at $0.50. These seem to be best suited for realistic video generation so my use case wasn't a good use particularly since I only needed a single frame.
ControlNet
It was July 2025, I had tried numerous times using GPT-4o to switch my hair to blonde as it had been for 6 months an would be for the foreseeable future. Surprisingly this one of the hardest challenges I've come across with the evolution of my profile image where I was transformed into having longer hair, a masculine woman or a man completely. I perhaps my prompting was to blame. I suspect this is more likely due to GPT-4o's training data lacking women with short hair like me.
To remedy the face changing so much I wanted to give ControlNet a try. ControlNet adds an extra layer when using AI image generation allowing control of the structure of the image. You could give it a stick figure pose, and the AI will draw a person in that exact pose. You could give it a sketch, and it will turn that sketch into a detailed image. ControlNet also has depth control, but I won't cover that in this since I'm not creating 3d images.
Now by this point, my 4th generation self-portrait has a lower resolution and noisy texture than the original. This is fine for human eye, but makes it incredibly difficult to use for computer vision. To demonstrate this, I'll create an create an outline from my profile images using image processing known as a Canny edge detection.
I tried to fix this using a real photo of myself photoshopped to the original self-portrait and got a good canny outline to use. I experimented using the canny outline with GPT-4o and it followed the outline precisely! Though to my knowledge GPT-4o doesn't officially support ControlNet.
Interestingly, OpenAI has updated their image model as of the time of writing and it no longer follows canny outlines precisely - bummer. Here's a video comparison to demonstate the GPT-4o's output from July 2025 and GPT-4o's output from today using the same prompt and outline -
Ultimately, I had to abandon this technique all together as I couldn't get latest self-portrait's style to be replicated using this technique to produce an updated self-portrait.
Seeds
Back at the drawing board trying to get my self-portrait's jet black hair closer to my current blonde hair. I used a technique called image-to-prompt with ChatGPT to get a prompt that would match style portrait.
Then using that prompt I used in-painting to lock parts of the self-portrait I wanted to keep along with the style prompt of:
Monochrome stippled portrait of an androgynous person looking down, wearing a turtleneck and jacket, moody expression, digital ink drawing style, dramatic lighting, vintage texture, light shaggy mullet hair, photo should be in black and white.
Once a style fit what I wanted of a greyscale light hair in a hair style close to mine - I locked the model's seed number. A seed number controls the randomness of the generative process. Using the same seed number, prompt and settings the same image will be produced reliably. So I could keep the same seed and make small changes to the text prompt, which would can generate new images that maintain the style and composition. For this I updated the prompt to remove a single word mullet and out came the image I wanted.
Wrap up
Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away.— Antoine de Saint-Exupéry
In less than a decade, AI image generation has gotten incredibly good. It was an interesting journey to follow while solutions were being shared publically. More common image generation AI models have moved into proprietary commercial products with the likes of digital waterstamping. If I'm being honest, I would now reach for a commercial product first. But for times when that falls short, I have the tools I need ready.