Working with a creative AI on a practical application

As a commercial digital designer, I have been subscribing to an image-generating AI called MidJourney that process images based on natural language input. And this is my experience with the process after several hundred generated images.

this is a perception of the concept, from a professional standpoint, not a technical analysis of how the neural network processes its loops.

Key takeaways: Good with organic and abstract, bad with objects and accuracy. use as inspiration...

( for now, basic training and sourcing can fix a lot of what is currently lagging )

Note: At the moment there are several similar image rendering AI's out there, all of them with different qualities. The most talked about Dall-e is currently perceived to be one of the best. based on training and ways of working.

Generally, it is amazing how far we have come with this type of technology. And it's incredible to see some of the fantastic art pieces this AI can produce in seconds.

How MidJourney renders in visual steps :

1. What you expect versus what you see

the AI is acting like a very creative artist that is very bad at fulfilling an assignment with a brief.

If you are looking for something specific or have an image in your head, of how you expect it to look. Then you will be looking for a long time. Because in 9/10 times the AI comes up with something you haven't thought about, and that might not serve your purpose or the visions you had set off with. In most cases, there are weird anomalies in crucial parts of the image. Because the AI is constructing the images organically and in most cases (for now) a lot of things blend into each other. Things that realistically are seen apart.

This is also why the AI is better at organic shapes rather than items or manufactured objects. Because naturally blending objects look more real in organic settings. This is also how Photoshop's algorithm that removes elements from a scene is working best. It can recreate a forest but is not good with man-made objects.

When you look at what other people generate. Most of the output looks great. But yet far from what most users are looking for, most users are refining their output several times to get closer to what they actually expect. This is my own experience as well. And very often you come to realize that you are not getting closer either because it does not understand context or phrases or it does not have enough training on different objects or concepts.

2. Be open-minded about the outcome and use it for inspiration

The AI generates images almost like a dream: You don't really get all the details right, a lot of things blend into each other and weird elements pop in and out. It can be very dreamlike.

As with artists versus designers, one can really blow your mind and one can stick to an assignment and deliver. This AI blows my mind several times per day. scrolling through the open log of generated imagery. The AI can take in styles from "photorealistic" down to a specific author or artist known for a certain style. and it reproduces it with very detailed precision. On top of that you can apply width, height, aspect ratio or in general, just have it generate more details with the HD command. or turn up the rendering quality and time which cost more computing power.

the AI tends to like warm and cold in a composition where no colors are explained or if the sentence is barely understood. if the AI is told to render "me" or "self-portrait" it renders either a dark-haired woman or a slim man with a mustache.

I wouldn't say it's conscious or aware. Based on all the defaults I see in many images (like flat top mountains, blue and red colors, horror styled Lovecraft art styles), it simply looks like it has a very simple set of defaults if very few details are described which might also be a sign that it needs more training and more thus more variety.

but it is getting really good at creating human faces besides the iris that it very rarely gets right. that it often creates two iris in one eye or gives the character a cross-eyed look because it doesn't know the relationship between the eyes, that the eyes should focus in the same direction.

3. the process is usually:

you generate some fast drafts they come in a 2x2 grid. from that, you then pick one or more or tell it to generate 4 new ones. from your picks, you can choose to reproduce them in 4 new ways. and in that way narrow in. when you have one draft that you like you can then scale it up and in that process, a ton more details are being added. in every step of this process, the AI generates something new based on the art you chose to proceed with.

you can also choose to start off with an image and then tell the ai to add stuff to that image style or set an image weight that tells the AI what it should focus on the image or the prompt. You can also set a ratio for every sentence to tell it how much to focus on each element in your prompt.

normal render time is about 1 min...

The practical application I was trying to apply this service to

was a board game cover art that would resemble some of the game components inside. which were a cryptex combination lock, a set of playing cards, a die, a maze, and a puzzle. This was an impossible task for the AI to handle because it had very little training in perceiving most objects.

More normal results looked like this:

there is defiantly a trick to getting good results. and it can be argued if that is in a "Human language" direction or a more command-centric way of optimizing your results.

What about NFT's?

As an NFT artist, I can also recognize a movement within the AI image generation scene, where NFT art is very often the target of a lot of the users that use the "generate new variation" function, on a good recipe for interesting art for the cryptocurrency scene.

In NFT art you sell "versions" of the same art style and figure. on a marketplace, as an online blockchain token, and thus securing your profit in sales and resales going forward because your name is at the end of the "chain" of ownership. If AI image generation is being put into this mix I can only see a fall in general profit for most artists going forward. This concerns me.

I hope you found my brief reflections on the topic informative. feel free to drop a comment if am missing any angles.