People in Los Angeles was an experiment on street photogrammetry. I was capturing people on the streets and in public spaces with my smartphone for about 3 months during my stay in Southern California. The aim of that article is to share this experience and what I’ve learned from it about photogrammetry solutions, street photography, privacy, distance and the nature of our memories.
Once I came to LA the friend of mine asked me to describe her how the city really is. That was a moment when I had already started to experiment with photogrammetry but her respond encouareged me to try this in the city space.
Here’s the note from the beggining of my stay:
People who want to occupy more space. They want more of themselves. They are louder, but not necessarily loud. They are expressive, although not always in contrast. They are boldly themselves. They boldly can or can not. THEY ARE, with all the intensity. As much as possible. They entertain, work, what everyone does. Just as yesterday and tomorrow, but today and now I am trying the hardest — to be. They’ve decided to be. They occupy spaces. They make choices. They articulate what they want or need. Or, they are not speaking at all.
As you can see, before any image or description of the city appeared in my mind, there was something more striking and much more relecant— the people. So I took the challenge of capturing them in their natural environment and daily situations thereby maintaining the impression of space occupied by them.
Photogrammetry — technically
Photogrammetry is basically the method of 3D reconstruction from multiple images. Dating to the mid-19th century, it has already been in use in topographic mapping, architecture, engineering and police investigations. Although the method itself is not particularly innovative, lidars and 3d scanners introduced new possibilities in the field as capturing dense geometry even in real-time. There are also some photogrammety software solutions which make the method accessible even for those with no specific hardware. I’ve been experimenting with those:
Agisof PhotoScan is the one that can be used on 30-day trial version.
3D Zephyr has its free version, but there is no option to actually export the mesh.
Capturing Reality is the one that I finally stick with as I’ve found it the fastes and the most accurate. You can try it for free but there is no option to export the mesh in that case. You can make renders and model’s video (with the watermark).
No matter which of those programs you decide to use, the first thing you will probably learn about photogrammetry is that it is limited. Every tutorial goes with a certain list of DOs and DON’Ts, which I really reccomend to follow so you are likely to avoid a lot of frustration. In other words — to achieve good results you need to follow software rules. It means i.a.: all sorts of movements are highly inadvisable. It makes people not perfect theme for what I would call “casual photogrammetry” at all.
However, to achieve high-quality results in capturing people for all sort of commercial purposes there are used rigs of more than 100 high-end DSLRs that allows to take all the shoots at once from different perspectives and proceed the result in a while with the highest quality. At the beginnings of photography, photographed person was supported on a special structure to remain motionless for the exposure time. Being motionless is quite unnatural for us. Think about the first photographed people and how rigid their poses were.
There’s the way to go for photogrammetry. A lot of good work has already been made by Az Balabanian whose works I really apreciate. The other model representing this still unique photography-like approach towards photogrametry is Eddy Adams with Maria sitting on the couch which is my personal favourite.
Models of People in Los Angeles were mostly taken with a smartphone Samsung Galaxy S6 and one with Panasonic HMD camera. The point was to keep it as simple as possible and make use of what I had available at that time. The whole series consists of 20 portraits created over 3 months. My photogrammetry practice was sketch-like, attempting to represent situations that happen literally in a moment, sometimes even in the blink of an eye. But they really happened and I hope that all their rough, glitchy surfaces stands for their original reality, the human being.
To avoid misunderstaiding, going into question with “the rules” was not my goal at any point. Unintentionally, it was the result of the uniqness of the situation, life as it really is was always before technology. I didn’t want to limit myself to the scenes that I’m sure I can get good results. So I worked with captured material no matter what the result promise to be.
Photogrammetry requires the photographed person to remain motionless for a long time, entrust himself or herself to the photographer in a particular way. Photogrammetry is the process that takes time. It’s not a single shot, it’s the situation that lasts for a while with an incredible tension between the photographed person and me (what sometimes cause those 40 seconds last forever!). In was the experiment on the distance that we perform in different sorts of social relations. There was always a desire to “take a closer look” but at the same time be careful not to break the border of the person’s comfort zone who entrusted me with her/his image and the body. I was interested how we can use technology to investigate those reletions and mark the territories of our own bodies, human appearance in the city space.
Not thinking much about the limitations, I was focused on life that was there around me and how to capture it, save and share — to give other people better understanding what life really in that particular space. Our existance has spatial dimension and to represent the world to speak to our bodies and spatial experience is very important for me. Think about the world where we can no more limit our memories and understanding of the world, to images which are still, once fixed. I do not personally like the feeling when I realize the memory that was just evoked in my mind is just the pictures, is literally the image — not a memory.
The memory-as-image can look like this:
I like that picture a lot and it always makes me think — what is the girl’s view on the situation, what remained in her memory from that meeting with “muscle man”? I argue if that is just a single image, if that is really sharp, solid, stable, sure, confident, single perspective. For me memories are often blured, multi-dimensional, multimodal or even uncertain.
Photogrammetry makes it possible to change the perspective on the scene.
or even give the observer first-person view like the one here:
Imagine placing yourself in 3 dimensional memory-as-scene where you how can re-think, reconsider facts, feel and understand the whole situation again as an embodied experience.
Let me share with you some of my best practices based on those experiments. While sketching itself usually took me no more than a minute, real work starts in the studio. I had raw videos that looked as if they were literally made with zero regard to all photogrammetry shooting rules. But — as there was no chance to re-take the pictures I felt like it is worth spending any time needed to make the model as good as it is possible.
First step was always to extract frames from the videos. I used to work with 50–200 images for a single scene.
alignement > settings
Reality Capture has couple of different alignment settings which I always underestimated but when I was really forced to get the result from my material (and there was no chance to re-take it) then I was literally forced to play with that and to learn how to get better results. I’ll highly encourage you to play with it no matter what is the first result.
Sometimes it happened that RC use only a few pictures which results in partial reconstruction. First try go to the:
scene > component
and see if there is more than one point cloud generated (conponent 0, component 1, etc.). Check if there is any that consists of more (or maybe all) the images.
In that case RC managed to align all the images (see the green dots in images corners).
When there is still an issue, you can always define control points to help the software recognise certain pixels (even then they do not match color at every image). Here is the tutorial how to work with control points.
Once you make sure the point cloud is good enough you can move further and create the mesh.
reconstruction > normal detail or workflow > calculate model
I always use reconstruction in normal detail, but it’s up to you and the results you’re expecting.
The results you got are sometimes really, really raw and there is often certain “background”, which was not recognized properly.
Choose the area that interests you by:
reconstruction > define reconstruction area (automatically/manually — I usually use auto option and then just change the
the other thing you can do is to define ground plane, to make sure your model is situated in the position you want it. You can centralize it and
reconstruction > define ground plane
I usually use all of the simplify and unwrap options to optimise the model.
Last step is texturing. In my case it works better when I just select a few the best images for texturing.
[choose 2D view in the left panel and select all images and from the menu below choose] > weight in texturing > 0
then select few images of chose and give it value 0.5/1 (one is the maximum so you can differ if you’d like all the images to have the same weight in texturing or you’d like to differ them instance to give the best face texture the priority and avoid afecting it )
The next steps for the project are:
– build solution to display 3D content on web that will fit the project need
– prototype VR experience that will let the observer angage with models (and make use of sound which is at the moment by-product of taking videos)
– experiment with street-photogrammetry hardware solution to take street-photogrammetry sketches/portraits more accurately
There article is the first one for me where I attempted to share some of more technical parts of the work. Please, ask if anything is unclear.I look forward to share more of the experience.