Date and Venue
Refers to the official registration site below. You may find presentation slides there as well.
Date: 2019/8/18 12:00~15:00
Venue: Mercari, Inc. @ Roppongi, Tokyo, Japan.
Hashtag: #xRTech
Topic 1: 3D Digital archive of architectures using wide-area Photogrammetry
広域フォトグラメトリによる建築デジタルアーカイブ by 龍 lilea @lileaLab
Ryuu-san: 3D Specialist, Photogrammetrist at HoloLab, Inc.
- Precaution — Photogrammetry requires a powerful computer. Don’t try it on a laptop or your battery will inflate!
Usage of 3D Digital Archives
- Virtual guide for famous sightseeing and cultural spots
- VR Pilgrimage — experience the real walking course in VR
- WebAR — summon the architecture on the AR space for education, with no need for any dedicated application. (File format: .usdz/.gITF)
- Simulation of VR space on the virtual office
- DiGITAL ARTISAN — Digital event space for seamless virtual & real meetup on VRChat
- Archive — preserving the memory of buildings to be destroyed.
Steps to create Photogrammetry
- Use a drone to take photos from the sky, on a very high detail
- Details on the bottom side where drones can’t fly in will be added using photos from DSLR camera
- Software — Reality Capture, CINEMA4D
Know-hows
- Actually, photogrammetry is weak against thin objects. Don’t use the backside of the objects in that case. Or fix it using DCC (Digital Content Creation) tools like CINEMA4D.
- You need to fix the details manually. Cut the unnecessary meshes, fill in the gaps or sometimes create a new 3D object to fill in the blank.
- “Align” is very important. You need to create a Control Point on each photo to tell Reality Capture that each point represents the same thing on different angles to improve capture accuracy.
- Use PL filter — to decrease reflection to preserve details on highlighted objects.
- Beware of flickering. Kanto region uses 50Hz light, Kansai uses 60Hz light.
- Prepare shooting equipment — pictures from 360 camera can be used as Skybox so bring both DSLR and 360 camera. The laser measurement device is also good to have since object size from the photo is different from the real size.
Topic 2: Introduction to Diminished Reality
Diminished Reality 入門 by sotan @sotanmochi
What is Diminished Reality (DR)?
- Removing objects from reality
- The opposite of AR. AR appends objects to the real space. DR removes objects from the real space. Both of them are being done in real-time in virtual space.
- Usages— Remove hands when writing to see what’s being written, See through the wall, Remove billboards from a sightseeing spot, remove current furniture to place a new one using AR, etc.
- Adobe Photoshop content-aware fill is the best example of DR application.
Previous Researches
- Pre-Observation Based DR (POB-DR) — Pre-take the background and compose them with the object.
- Real-time Observation Based DR (ROB-DR) — Use many cameras to compose photos.
- Image Inpainting Based DR (IB-DR) — Use algorithms to fill in images (like Photoshop content-aware fill), this approach requires deep learning.
DR Applications
- DR + AR = Virtual YouTuber in real space!? In the future, you can even change service robots appearance to humanoid characters
- However, Image Inpainting requires a lot of processing power and there is almost no research about inpainting moving objects in real-time. So it’s still a question if this can be done without a powerhouse GPU/prerendering.
- The most difficult issue is how to improve the performance to be able to diminish reality in real-time.
Topic 3: Merits and demerits of using Babylon.js to build a WebVR application for Oculus Quest
Babylon.jsでOculus Quest向けWebVRアプリを作るときのメリットとデメリット by Limes @WheetTweet
Steps to build WebVR app for Oculus Quest
- Babylon.js is a browser framework for rendering 3DCG, based on JavaScript.
- Developer tools — Text Editor & Browser (anything except Internet Explorer…)
- Server — Local = Express/Websock, Remote= Playground/Github Pages
- Accessing the URL from the Oculus Quest browser and try your app!
Merits
- No need for headset-specific SDK (support Quest/Go/Rift/VIVE/WinMR)
- No need to build app = fast development
- Project size is very small, compared to a Unity project
Demerits
- Need to learn JavaScript
- GUI for development is not good. However, you can use
scene.debugLayer.show()for a simple GUI inspector. - Only a few tips for VR development. Gaze, Teleport, Grab, Button inputs are supported. You need to create a pull request for improvement yourself.
- Only a few information in Japanese. On the other hand, there’s a lot of chance to participate in OSS.
Summary
As of today, sophisticated tools like Unity are still better for VR development but Babylon.js is also catching up quickly. Also, there’s a lot of chances for a contribution so you should give it a try!
Topic 4: xR for Expression (VRChat x Shader)
表現としてのxR by ほたて猫魔人 @HhotateA_xR
Hotate-san is a creator in VRChat. This talk is about how shaders are being used in VRChat.
- VR doesn’t have a distinct line between academic research and creation
- Many events are being held in VRChat.
- Shaders can be used in VRChat. Since shaders can be used, you can change any visual expression inside the system.
- Change real-like avatar to hologram-like avatar to overcome the uncanny valley
- The speaker-developed VR x AR walking app is now for sale — https://booth.pm/ja/items/1509057
Topic 5: Building the economic system on virtual space by blockchain
ブロックチェーンで仮想空間上に経済圏を作りたい話 by 佐藤 直人 @sn___2309 / く゚ @qiwdu / 牧野 賢士 @MacKen196
“Frontier” — a group of 3 high school members talks along with funny Boke-Tsukkomi hahaha.
- Use blockchain to redefine digital items
- The biggest issue of digital items is that they can be copied without permission from the owner. So it is very difficult to create a digital market on VR.
- How to solve it? use Etherium token to trace digital items. Non-Fungible Tokens (NFT)
- However, digital data outside the Etherium system can still be duplicated so it can’t be protected from being copied.
- Reverse thinking: let’s use NFT to create an access key for using data. So even if you can duplicate data, but if you don’t have the right then you can’t use it. The group is now designing the specification for the access key.
- If this happens, you can trade digital assets and it should have the potential to grow into a new economic system.
Topic 6: VR physical training system (Kendama and golf)
VR物理トレーニングシステム（けん玉やゴルフなど） by 川崎 仁史 @VRkendama / 久保田 悟 @kendama121
A talk from Kendama specialists! They are from CanR, Inc., a company to help people do what they can’t do in the real world in the VR world.
- First product = Kendama VR. 982 people were able to play Kendama after training with the system. Second product = VR Golf.
- Start: They presented the idea in the Tokyo XR startups 5th incubating program. The president of the program said it was interesting but the market is too small. They need to rethink how to make the product Interesting, Make money and Come into fashion (buzz).
- After finding what people want to pay for simulators to improve their skills, they found that “Golf’ is a perfect choice. They decided to create a golf training system in VR and received funding finally.
- Pass-through issue— when you move objects quickly in VR, it passes through other objects without collision. They solve this by calculating the angle and power of the club and the ball programmatically, without relying on the psychic engine.
- Achievement — Users can copy the swing from pro players and play. Everyone experienced the software “CanGolf” showed improvement of head swing speed.
I actually tried this app “CanGolf” during the exhibition time. It was so cool that you can improve yourself by imitating the pro’s swing guideline. This is not possible using a traditional video system since neither you can see the pro swinging right in front of you nor you can see the swing guideline when you twist yourself to the right.
VR app enables us to have a personal trainer who we can align ourselves with. So we can learn from him in such an intuitive way. I really feel that this will change how we train ourselves to learn any physical skill in the near future.
Topic 7: Full-body 3D scan system kit with Raspberry Pi
全身3Dスキャン システムキット — Raspberry Pi 使用 by リアルアバター制作 — 岩山 幸洋 @HappyOcean
- The speaker uses RealityCapture to create a 3D model from 80 cameras
- His vision is to create a future where people from over the world can meet anyone at any place at any time.
- In order to achieve the vision, a real avatar is an important asset to represent a person’s identity.
- The required tools for creating a 3D model are the camera and the controller. So Raspberry Pi and a mobile camera module can be adapted for this use. This system is far cheaper than using multiple DSLR cameras connecting to a PC.
- However, since photogrammetry requires you to take a sharp and clear photo to be able to stitch, a small camera module attached to Raspberry Pi may post a problem for computer vision. Need to manually adjust the camera.
[Ads] You can now make your 3D avatar for only 7,800 Yen. Visit https://real-avatar.com
Topic 8: A story of a web engineer who supported VTuber at Niconico Chokaigi
webエンジニアがニコニコ超会議でVtuberをサポートした話 by へいきゅーぶ @hey_cube
A talk from a web engineer, not xR engineer. He built an application to take a photo with Ibara Hiyori, an Ibaraki prefecture-approved official virtual YouTuber.
- A web application that randomly chooses Hiyori-chan picture with Ibaraki prefecture sightseeing spot. You then take your photo and the application will combine your portrait with Hiyori-chan and the spot. With this, you can get a virtual memorial picture instantly.
- This app idea is born from an internal Hackathon. A speaker is working in an advertising agency. A department in charge of Hiyori-chan PR wanted an idea to promote her. A good example of how Hackathon creates new value.
remove.bgis a free API to remove the background from the picture. Very good.- Achievement — over 150 of 1000 people coming to the booth has taken the pictures with Hiyori-chan
- Cost — about 40 hours of development.
- Final note: even without using xR technology, you can contribute to appealing xR contents!
Exhibition Pit
This is the last session where exhibitors come to introduce their app/service in 1 minute. Those interested in any app can try it during the free time after talk session.
Final words
I hope you enjoyed this meetup report. xR Tech Tokyo is a really great meetup event for those residing in Japan who are interested in VR/AR/MR development so I suggest you check the latest info at https://vrtokyo.connpass.com/ for the future events.
This is my first time taking event notes and reports in English so I apologize if there is any mistake. Feel free to comment if you have any questions! Also if you are Thai, you can check the 15th meetup report at my blog here. Thank you for reading!
