DALL·E 2 and MidJourney

You may have heard about the recently launched AI (Artificial Intelligence) tools, MidJourney and DALL·E 2. They are the new cool tools in the design industry to create terrific concepts. Both tools generate images out of the text. Using a huge database of categorized photos, MidJourney and DALL·E 2 generate realistic photos from different art styles to highly complex forms as 2D images, only by writing in their command bar. You can write a few words or a huge paragraph and still get amazing results. Despite many issues with the authorship of these generated images, which is also a huge controversy among the opposers of the tools, the number of designers using them is increasing. The technology is very new, and it will take some time to become mature until to the point that it can be accepted as a unique tool for creating inspiring conceptual images.

What is Midjourney

“Midjourney is a research lab and the name of the lab’s artificial intelligence program that creates images from textual descriptions, similar to OpenAI’s DALL-E. The tool is currently in open beta. The British magazine The Economist used the program to create the front cover for an issue in June 2022.” As described on Wikipedia.

What is DALL·E 2

“DALL·E 2 is a new AI system that can create realistic images and art from natural language descriptions. DALL·E 2 can make realistic edits to existing images from a natural language caption. It can add and remove elements while considering shadows, reflections, and textures.” Check out more about DALL·E 2 on openai.com.

In this article, young and talented architect William Garner discusses the abilities of DALL·E 2 in architectural conceptualization and the results of combining it with Midjourney after testing for a month. William thinks it is safe to conclude that each DALL·E 2 and Midjourney has distinct advantages. However, at this point, Midjourney can be more “creative” and provide the ability to explore more imaginative thinking. Still, DALL·E 2 feels more literal and has the power to take control through targeted editing.

Make sure to check William Garner‘s Instagram profile for more examples.

While DALL·E 2 is used for architectural purposes, it seems good at “imitating” what it has learned. Still, the reuse and application of that knowledge into more “imaginative” designs are limited. For instance, the prompt “front view of a brutalist clay structure with lush greenery, soft light” was used to create the following image.

And this one: “Front view of a traditional clay house with lush vegetation and soothing light.” It could be difficult to determine that the design proposals produced by all of the runs are not actual structures because they are all fairly tenable and realistic. Although stunning, I believe these results are a little too resolved and confined to be helpful in the architectural design process. That being said, perspective and context are important factors.

DALL·E 2 currently gains strength in performing targeted edits (inpainting and outpainting). This is where we can edit parts of the image and prompt it with new changes. For example, this could be useful in painting designs onto a site photo in the future or changing parts of a building’s materials.

DALL·E 2’s capacity to make precise edits is now at its strongest level (inpainting and outpainting). Here, we can alter certain portions of the image and give it new instructions. This can be helpful, for instance, if designs need to be painted later on a site shot or if the materials used in a structure need to be altered.

On the other hand, it seems that Midjourney frees the designers to be far more focused on creativity and imagination in their ideas. For instance, it easily generates the choice “ethereal Zaha Hadid circular bedroom interior made of algae + futuristic circular roof light”

I have been experimenting with utilizing Midjourney to initiate the creative process of ideas and DALL·E 2 to generate variations and carry out revisions to discover a workflow that combines the best of both worlds like the following projects. This gives us considerably more control over the creative exploration process for the time being. Still, it also enables us to make specific decisions about the designs shown to us.

Another illustration of this is the project that follows, originally designed in Midjourney as a building beside a canal but was later modified using DALL·E 2 to include a neon sign, alter the facade, and alter the roof.

Additionally, I used DALL·E 2 to create several variations. As you can see, it kept the formal concept and mirrored the image’s color and the lighting’s effect but could not pick up the detail in the image. OpenAI has stated natural variations are limited currently, so this is perhaps why.

For the time being, it appears that to meet creative artists’ needs, it will be necessary to continue stacking several AIs, much like traditional software workflows. But as these tools expand, it will be fascinating to see how they interact and how fresh approaches to creative exploration take shape.

The pace of the human x machine design loop traveling between machine-generated possibilities and human intuition has been decelerated by most of my prior experience, concentrating on using DIY Google Colab Notebooks. These brand-new AI “apps” undoubtedly move AI closer to widespread adoption and toward becoming a genuine real-time creative design collaborator.