Everyday Objects as Dynamic Creator Tools in XR Spaces

Humans are tool makers

I use both physical and digital tools to build prototypes. I have always been curious about how we can bring the best aspects of the physical world into the creation processes of the digital space. I have been thinking about this question within the language of existing 3D tools; and the exciting possibilities with the rise of spatial compute. The following is a collection loose thoughts in this area.

Imagine tools and interactions crafted specifically for the at-home artist, the avid crafts lover, and the DIY enthusiast. How might we allow ourselves to continue building tools in a mixed reality setting? Thereby, encouraging us to continue being the tool makers that we are.

Tool Recognition and Usage

Navigating the learning curve to engage with a digital 3D building environment can be daunting, creating a barrier for many. What if it didn’t need to be? How can we draw inspiration from our present, mundane, everyday interactions to reimagine the way we use 3D tools?

I propose using computer vision and image recognition to capture objects in our everyday lives and use them as tools for digital creation.

Lets use the example of a spoon. Through computer vision and image recognition, the spoon would be used as a tool or brush for sculpting. The brush/tool would take the form of how the spoon is shaped, curved, bent and textured to create patterns.

Spoon as a Sculpting Tool Using Gesture Recognition and Marker Based Tracking

Spoon as a Sculpting Tool Using Gesture Recognition and Marker Based Tracking

Forks as a Sculpting Tool Using Gesture Recognition and Marker Based Tracking

Through tracking our movements, gestures, and perhaps even natural language processing, our ideas of physical building and creating would become more resourceful, this way, everything around us becomes a tool. A conduit for creation.

An example of how a piece of a broken hair clip can be used in various different ways to create patterns, brush strokes and shapes.

For creators, especially those hesitant to transition from hands-on craftsmanship to digital 3D realms, purposeful interactions and technology integration can serve as a guiding bridge, enabling them to unleash their creativity. Imagine physically walking around their creations while actively editing them with tools found in their homes or studios. Every location would become a creator space, then it’s only a matter of being more resourceful.

Currently, to sculpt (in digital 3d spaces), I would create custom brushes by using patterns and stamps. What if we could skip a few steps and create a “pattern” through existing objects we see around us.

In this context, everything in our surroundings transforms into sculpting tools for the digital clay within the software.

Every object around us can be seen as a tool to sculpt with, to create with.

More thoughts

Taking speculative exploration a step further, consider harnessing the scanned objects or tools as active contributors to problem-solving.

For example, I want to clip on a second pipe to my sink, and decided to 3D print the attachment. I would be able to show the application the sink and using the same technology like above, through spatial image recognition, it would tell me the dimensions of the pipe. I would then proceed to build my attachment in the same interface using the sink as reference for my attachment, ensuring I am using the right measurements and that it fits.

The potential housing for this application is uncertain. Could it extend beyond the confines of a laptop, possibly integrating with smart glasses, or necessitate a distinct device? Let’s explore the possibilities!

In conclusion, the fusion of physical and digital tools in the realm of prototyping unveils a rich tapestry of possibilities. As we contemplate ways to seamlessly integrate the best aspects of the physical world into digital creation processes, the emergence of spatial computing introduces an exciting dimension to our explorations. Here is a quick recap of the ideas mentioned:

1. Ubiquitous Creator Spaces: The concept of every location becoming a creator space may evolve into a ubiquitous reality. With tools seamlessly integrated into our surroundings, individuals could effortlessly transition between physical and digital creation modes, fostering a continuous and accessible creative environment.
2. Intuitive and Natural Creation Processes: The integration of computer vision, image recognition, and gesture tracking could lead to more intuitive and natural creation processes. Creators might find themselves physically interacting with their digital designs, offering a tactile and immersive experience. This approach may bridge the gap for those hesitant to adopt digital tools, making the transition more fluid.
3. Problem-Solving Through Digital Creation: The application of spatial computing and image recognition to solve everyday problems may become commonplace. From designing custom solutions for household needs to addressing practical challenges through augmented reality, individuals could leverage digital creation tools for practical problem-solving in their daily lives.
4. Expanded Scope of 3D Printing: The integration of spatial image recognition for 3D printing projects could revolutionize how we approach manufacturing at a personal level. Users may easily scan and measure objects, ensuring precise dimensions for customized 3D-printed solutions. This could democratize the 3D printing process, empowering individuals to create tailored objects for their specific needs.

In essence, the future could witness a profound shift in the way we perceive, interact with, and create within the digital landscape. As these concepts evolve, they have the potential to redefine not only how we craft digital content but also how we approach problem-solving and innovation in our everyday lives.

For a deeper understanding of spatial computing and its workings, here is a resource: https://medium.com/antaeus-ar/spatial-computing-101-a-comprehensive-introduction-2f1005f5a235