Down the Rabbit Hole: World in MidJourney. Episode 2. Making a 360 and Showing it Off!
My little steampunk world at this point was a sorry 30 sec video. I wanted to make it into a real place, where one can look around, or even go for a walk. MJ is able to create infinite number of unique worlds. And I wanted to show as many of them as possible!
In order to do that my world need to be saved as a 360 panoramic image. Then I will be able to view and share it in the 360 viewer as a 360 spheres. Apparently it is really not that hard! Let’s go step by step.
A Bit About 360 Panoramas
The 360 panoramas can be equirectangular, cubical, cylinder. First one is the most popular format. The Equirectangular projection is the default output format of a scanning panorama camera equipped with a fisheye lens — 180° fisheye giving a full sphere at 360° rotation. Scanning cameras sometimes cover more than one 360° turn but software often assumes that equirectangular images cover 360° horizontally and 180° vertically.
Fo us, important part it that our future panorama should have aspect ratio of 2:1. Panoramic images in the viewer are created by mapping rectangular image onto a sphere. Naturally, a lot of distortion occurs. Picture on the right gives us some idea of equirectangular distortion. Note that areas around equator stay pretty much intact. Distortion quickly grows towards the future poles. As you see on the right image, when viewed with a pano viewer, this map becomes a cube.
Let’s take a look on the real equirectangular panorama taken with a camera. The one below is flawless. Look how real the room looks in the viewer! You will find this and other brilliant renderings from the same artist here.
How Do I get a Distorted Image?
The beautiful precise image of the living room was created with a fish-eye camera. Mine will be created with AI, so it wont have equirectangular projection by default. It would be nice to have a converter from a normal (perspective) image into the equirectangular. Apparently it is not that easy. I tried more than a dozen, I cannot recommend either of them. We will be creating our distorted images manually, unless you guys have a tool in mind. I don’t.
While creating your images, keep in mind the folowing:
1. Upper and lower edge of your images should be free from details: they will be pinched in the panoramic viewer.
2. Main content should be vertically centered on equator level and the field of view should strictly horizontal.
3. Important details should be in the middle third of the image
4. If necessary, you can distort the edges manually, until they appear just fine in the viewer. I used Transform > Wrap command in Photoshop.
Also, no worries. We are creating fantasy illustration here. So what if our houses and rooms will be a bit crooked? It will add to the charm.
Preparing the Pano Source
This is the first square image. It was made in in Episode 1. Our goal is to expand this image into a 2:1 panorama, keeping the interesting content near equator, where it will be most visible to the viewer. The edges of the image (up and down on the future sphere) will be heavily distorted, let’s keep the content there as vague as possible. We also need details in the foreground, otherwise the scene will look dull — but those details will also be distorted, so we will need to choose wisely.
Creating the Panoramic image in MidJourney
Pan the Content
I planned to make several pans, forming a long panoramic image, then zoom it out to 2:3 final. It worked. Problem with this approach is that is is very hard to keep the style same throughout the panning. In order to do so, I first minimized the influence of the MJ own style adding — raw parameter by default. Then I started with adding image references.
Uploaded photos, links to online images, link to a MJ image, and even uploaded screenshot of a specific detail — all these methods can be used for image prompting. Read about image prompting here.
I also ended up dropping all artistic references, since they kept producing artist-specific artifacts in a variety of styles, and the subject (“sad monster”). In this case MJ produced rather bland panned content in the same style as the main generation. Don’t forget to add the “pano”keyword. Below is how my result looked like superimposed onto equirectangular map.
left and right edges of the panorama should be as similair as possible so that they can be easily stitched later. I guided MJ by makin a screenshot one edge and giving as a visual prompt while panning the other edge.
Zoom the Edges
Next step, I needed MJ to create the upper and lower edges. It is easily done with a Custom Zoom. You might need to try several zoom factors to find the one that works.
Alternative approach: Pan, Square, Repeat
The generated image was ok, but rather boring. I experimented a bit more, combining panning with Square function. From my experience, square outputs results that are fully in sync with the initial image style and visually more engaging than just panning all the way. Look for yourself:
Upscaling
Starting October 21st, MJ can upscale X 4, making our life much easier. Upscale the result 4X and download it. In order to get the upscaled image you need to open it in the browser. Upscales are in webp format, you will need to save it as JPG or PNG.
Just in case you still need to upscale your image outside MidJourney, this is the best tool for the job. Before MJ upscaling was introduced, we will use it to upscale our pano. It is not needed for the job anymore, but it is still abriliant tool if you need ven larger images. This is the link to it. Try it out, you will be amazed. It worth every penny.
Alternative Approach: Equirectangular Prompt in MidJourney
Midjourney has some understanding of equirectangular projection, so it can be added in the beginning of the prompt. The results are not very predicatble and very unstable, but sometimes extremely cool. Here is one sample of this approach
Photoshop Processing
There are probably millions of Photoshop guides on image processing. I won’t be describing it here in any detail. In short, your approach will be as follows:
1. Resample the image to 6000X3000 — default size for the 3D panos.
2. Edit your panoramic image mo make a horizontal tile from it. It is done with Offset filter.
3. Stitch the image using any tool from the vast Photoshop toolbox.
Before Photoshop version 2024 i would also highly recommend the 3D pano tool. They were marvelous and allowed to touch up the pano edges. Unfortunately, they are discontinued, but if you are using earlier versions of Photoshop, you might want to take a look at this video.
Adding the Metadata for Facebook
In order to view and share your panoramas, you need a viewer. First viewer we will be using is Facebook.
In order for FB viewer to show an image as a panorama, it needs to have the right metadata. If you are creating your pano with a camera. this metadata is in the file already. In our case it is not, but it is easy to add it. Download the XMP metadata file and add it to the the File Info (File>File Info > Raw Data >Import ). If you need more detailed instruction. use the link below, this is where I learned this tip.
Cool. I can show off my brave new world in Facebook now. But what about my website? Can I show it there — without all the Facebook nonsense, just my stunning design? What about the linked panoramas? Custom design, custom controls, limited field of view? All this require a stand-alone viewer, and I was going to find a perfect one.
Stand-alone Viewers and Tours.
The best in class viewer I found is called Pannellum. It is an open source project allowing to generate, edit and show stand-alone panoramas and 360 tours. It requires a server to run, any server. Offline python one is good, just spin it off and do all your testing before uploading your work online.
Check it out. It is just as advertised, lightweight and powerful. Got to Episode 3 to learn more :)
