Down the Rabbit Hole: World in MidJourney. Episode 2. Making a 360 and Showing it Off!

InkPromptu
8 min readOct 22, 2023

--

My little steampunk world at this point was a sorry 30 sec video. I wanted to make it into a real place, where one can look around, or even go for a walk. MJ is able to create infinite number of unique worlds. And I wanted to show as many of them as possible!

In order to do that my world need to be saved as a 360 panoramic image. Then I will be able to view and share it in the 360 viewer as a 360 spheres. Apparently it is really not that hard! Let’s go step by step.

A Bit About 360 Panoramas

The 360 panoramas can be equirectangular, cubical, cylinder. First one is the most popular format. The Equirectangular projection is the default output format of a scanning panorama camera equipped with a fisheye lens — 180° fisheye giving a full sphere at 360° rotation. Scanning cameras sometimes cover more than one 360° turn but software often assumes that equirectangular images cover 360° horizontally and 180° vertically.

Fo us, important part it that our future panorama should have aspect ratio of 2:1. Panoramic images in the viewer are created by mapping rectangular image onto a sphere. Naturally, a lot of distortion occurs. Picture on the right gives us some idea of equirectangular distortion. Note that areas around equator stay pretty much intact. Distortion quickly grows towards the future poles. As you see on the right image, when viewed with a pano viewer, this map becomes a cube.

Equirectangular map image (left) and same image rendered in the 3D viewer.

Let’s take a look on the real equirectangular panorama taken with a camera. The one below is flawless. Look how real the room looks in the viewer! You will find this and other brilliant renderings from the same artist here.

Equirectangular image of the room and same room imagerendered in 3D viewer.

How Do I get a Distorted Image?

The beautiful precise image of the living room was created with a fish-eye camera. Mine will be created with AI, so it wont have equirectangular projection by default. It would be nice to have a converter from a normal (perspective) image into the equirectangular. Apparently it is not that easy. I tried more than a dozen, I cannot recommend either of them. We will be creating our distorted images manually, unless you guys have a tool in mind. I don’t.

While creating your images, keep in mind the folowing:

1. Upper and lower edge of your images should be free from details: they will be pinched in the panoramic viewer.
2. Main content should be vertically centered on equator level and the field of view should strictly horizontal.
3. Important details should be in the middle third of the image
4. If necessary, you can distort the edges manually, until they appear just fine in the viewer. I used Transform > Wrap command in Photoshop.

Also, no worries. We are creating fantasy illustration here. So what if our houses and rooms will be a bit crooked? It will add to the charm.

This photo is here for no reason :)

Preparing the Pano Source

This is the first square image. It was made in in Episode 1. Our goal is to expand this image into a 2:1 panorama, keeping the interesting content near equator, where it will be most visible to the viewer. The edges of the image (up and down on the future sphere) will be heavily distorted, let’s keep the content there as vague as possible. We also need details in the foreground, otherwise the scene will look dull — but those details will also be distorted, so we will need to choose wisely.

Creating the Panoramic image in MidJourney

Pan the Content

I planned to make several pans, forming a long panoramic image, then zoom it out to 2:3 final. It worked. Problem with this approach is that is is very hard to keep the style same throughout the panning. In order to do so, I first minimized the influence of the MJ own style adding — raw parameter by default. Then I started with adding image references.

Uploaded photos, links to online images, link to a MJ image, and even uploaded screenshot of a specific detail — all these methods can be used for image prompting. Read about image prompting here.

I also ended up dropping all artistic references, since they kept producing artist-specific artifacts in a variety of styles, and the subject (“sad monster”). In this case MJ produced rather bland panned content in the same style as the main generation. Don’t forget to add the “pano”keyword. Below is how my result looked like superimposed onto equirectangular map.

Use additional images to get to the desired content and style.
Panning Result.
Meaningful content should be around equator.

left and right edges of the panorama should be as similair as possible so that they can be easily stitched later. I guided MJ by makin a screenshot one edge and giving as a visual prompt while panning the other edge.

Zoom the Edges

Next step, I needed MJ to create the upper and lower edges. It is easily done with a Custom Zoom. You might need to try several zoom factors to find the one that works.

Zooming out
Selected image

Alternative approach: Pan, Square, Repeat

The generated image was ok, but rather boring. I experimented a bit more, combining panning with Square function. From my experience, square outputs results that are fully in sync with the initial image style and visually more engaging than just panning all the way. Look for yourself:

Upscaling

Starting October 21st, MJ can upscale X 4, making our life much easier. Upscale the result 4X and download it. In order to get the upscaled image you need to open it in the browser. Upscales are in webp format, you will need to save it as JPG or PNG.

Just in case you still need to upscale your image outside MidJourney, this is the best tool for the job. Before MJ upscaling was introduced, we will use it to upscale our pano. It is not needed for the job anymore, but it is still abriliant tool if you need ven larger images. This is the link to it. Try it out, you will be amazed. It worth every penny.

Alternative Approach: Equirectangular Prompt in MidJourney

Midjourney has some understanding of equirectangular projection, so it can be added in the beginning of the prompt. The results are not very predicatble and very unstable, but sometimes extremely cool. Here is one sample of this approach

Create a seed image.
Pan it and make a square
Resulting panorama

Photoshop Processing

There are probably millions of Photoshop guides on image processing. I won’t be describing it here in any detail. In short, your approach will be as follows:

1. Resample the image to 6000X3000 — default size for the 3D panos.
2. Edit your panoramic image mo make a horizontal tile from it. It is done with Offset filter.
3. Stitch the image using any tool from the vast Photoshop toolbox.

Left part of the image is much darker. Using gradient tool, select it and adjust curves.
Select Filters> Other Offset and offset the image horizontally wrapping pixels around. Now when left and right come together, they will align flawlessly. Use guides to make sure your scene is still straight on equator
Image 1. Using content-aware tools edit the stitch. I also touched up the lower edge of the image with Photoshop 3D pano tools.
Image 2, also after Photoshop processing.

Before Photoshop version 2024 i would also highly recommend the 3D pano tool. They were marvelous and allowed to touch up the pano edges. Unfortunately, they are discontinued, but if you are using earlier versions of Photoshop, you might want to take a look at this video.

Adding the Metadata for Facebook

In order to view and share your panoramas, you need a viewer. First viewer we will be using is Facebook.

In order for FB viewer to show an image as a panorama, it needs to have the right metadata. If you are creating your pano with a camera. this metadata is in the file already. In our case it is not, but it is easy to add it. Download the XMP metadata file and add it to the the File Info (File>File Info > Raw Data >Import ). If you need more detailed instruction. use the link below, this is where I learned this tip.

Cool. I can show off my brave new world in Facebook now. But what about my website? Can I show it there — without all the Facebook nonsense, just my stunning design? What about the linked panoramas? Custom design, custom controls, limited field of view? All this require a stand-alone viewer, and I was going to find a perfect one.

Stand-alone Viewers and Tours.

The best in class viewer I found is called Pannellum. It is an open source project allowing to generate, edit and show stand-alone panoramas and 360 tours. It requires a server to run, any server. Offline python one is good, just spin it off and do all your testing before uploading your work online.

Check it out. It is just as advertised, lightweight and powerful. Got to Episode 3 to learn more :)

--

--

InkPromptu

Successfully using Artificial Intelligence in lieu of natural stupidity