Future of Product contents generation — Tech note Planningo
TL;DR
- Planningo researched WebGL, Backend realistic Rendering, Image Generation AI
- We have problem that we have to solve
- We will solve it in these way
If you are planning to sell a product, do you know what you need to prepare right now?
If you are planning to sell an actual product, you need to start by selecting an online store to sell your products. You also need to prepare product photos for your online store, create attractive product content and descriptions, and engage in marketing efforts such as SNS marketing or blog marketing to promote your products.
These processes can be costly. To create product content and photos, you need to compare prices and quality from various studios. To promote your products, you may need to write promotional posts on SNS or blogs, or hire marketing agencies.
At Planningo, our vision is to offer a solution that can manage the entire process of creating product content, including the promotion of products, through our services.
We have accumulated the following technologies to achieve this vision.
Utilization of WebGL technology
Through WebGL technology, we have launched the VIVAR service. This service allows product sellers to view their products in 3D and apply various options. Additionally, users can place virtual products in real environments using augmented reality (AR).
Using WebGL, we can create a 3D viewer for products that allows users to change various options. We also offer a product editor for virtual assembly of modular products, as well as a 3D product assembly guide that shows the sequence of assembly steps.
Furthermore, we have the capability to apply various shaders to images using WebGL. This includes image post-processing with the application of depth estimation for lighting effects.
Backend photorealistic rendering
Using the Blender cycle engine, we have the ability to perform photorealistic rendering of 3D scenes.
Based on this technology, we launched the Snapic service, which provides photorealistic rendering of product models using 3D scenes created through a web browser.
While photorealistic rendering is primarily used on the client side to provide images or videos for product content, if many 3D models and scenes are available, it can also be used as data for image-to-video AI.
Tesla uses virtual rendering data from 3D spaces to train the AI for their autonomous vehicles’ cameras. We also see the 3D space as a new breakthrough for data preparation. We are currently working on preparing this technology.
AI image generation
We have also obtained the capability to generate product content using AI. Our service, Photio, offers a pipeline that allows users to generate product background photos that closely resemble their desired references(called remix). Currently, we are testing the beta version of the image generation service. Additionally, we are developing AI technology to modify specific parts of generated or user images.
Furthermore, we are expanding our AI technologies to cover other aspects of AI product content creation. This includes background removal AI, image-to-text AI, and embedding-based similarity search. We have also begun utilizing frame interpolation AI models to create smooth transitions between images.
What technologies will we research in the future?
To achieve our vision of providing a solution that makes it easy to create high-quality product content, we need to research and develop new technologies. We currently have some challenges that need to be addressed.
Two main challenged are these
User is not an professional photographer
When users take product photos with their smartphones, the appearance of the products is often distorted or the colors are altered compared to when the photos are taken in a studio. Through WebGL technology, we have confirmed that we can transform user product photos into cleaner, undistorted images.
However, since users cannot change the distortion or color alteration themselves, we need to develop an AI model or algorithm to correct the product images. Additionally, we plan to develop an application that allows users to adjust the angle of the product and analyze the environment during shooting using AR technology, in order to guide users in capturing high-quality product photos.
Users do not effectively utilize the images they create.
Therefore, we believe that instead of providing users with materials to create content, we need to provide them with finished content that they can immediately use.
One of the representative finished products of product content is the product introduction page. Through the product photo shooting application that solves the first challenge, we can obtain multiple high-quality photos for the product introduction page. We can use these images to create detailed product pages and generate the layout and description for the product introduction page using AI.
Another essential part of product content is promotional videos. If users simply input their products into our service, we can generate promotional videos about the products. We envision providing the content as it is to users.
The required video content for our products includes shots of the product from various angles, different scenes featuring the product or other props, and a compilation of videos or images lasting about one minute.
Using LLM, we can generate the content for each time frame and create frames based on the appearance over time, which can be used to create a video. This is currently the most realistic and achievable technology, as it is based on LLM, which is also used in Google’s VideoPoet model.
We can also utilize the technologies we have in Planningo to fill in the assets for videos or images. The most crucial aspect we need is the ability to generate images from various angles and create AI modules that can create videos based on camera movements.
We plan to complete the development of these technologies for addressing the two challenges by July 2024. We will then observe how users utilize our services, identify any difficulties, and continue our research to provide a solution that makes it easy to create product content.
