NOTE: Telling the Story from the Beginning
NOTE: 0019 — Monday 23 October 2017
It has been a while since I have posted on Medium in regards to the process of my Graduation Thesis Research Project. With lack of time to slowly build up this documentation, I’ve decided to compile a timeline within this post to encapsulate the process from its early stage up until today.
To start, the initial idea in terms of an end goal was to create a BIM reporting tool that could redefine how documentation could be viewed when decentralised onto a platform for model interrogation. An example of such a tool was inspired by the work of CL3VER — an interactive 3D presentation software.
Above is an screenshot of the final presentation from using Cl3VER which requires the use of their software to realise the interaction and storytelling process.
Instead of using the actual program they provide (which requires subscription to), I would aim to create a free alternative in creating the same results on the web as there are alternatives to generating interactive models.
With this idea in mind, I looked heavily into creating such scenes using open-source libraries as A-Frame to present models on the web. Prerequisites for learning A-Frame include:
- HTML
- CSS
- JavaScript
- 3D modelling software (to understand exporting modelling file types)
I’ve posted my process earlier in this publication as shown:
- NOTE: A-Frame Basics
- NOTE: The Prototyping Process
- NOTE: Setting a Scene in A-Frame
- NOTE: Building Interactivity in A-Frame
- NOTE: Adding Data to Objects in A-Frame — Part A
- NOTE: Adding Data to Objects in A-Frame — Part B
- NOTE: Building Buttons for A-Frame
The process had led me to understand how to build interactive A-Frame scenes which could attach data to objects when clicked to view. Time was still well spent as it was considered the first of two parts in my first prototype exploration.
Prototype 1-A (A-Frame scene) was still without a doubt very useful for presentation related outcomes for architects to tell their story. HOWEVER, what I realised throughout this process was that such tools, even the one CL3VER provides, are useful for final collective scenarios of collating data but are not ideal in real world scenarios. More specifically what I mean by this is that a project within the architecture, engineering and construction (AEC) industry revolves the matter of constantly re-framing, reiterating and redesigning the end building. In order to keep up with this process, it is essential to look for much faster and effective ways of documentation.
In the next post, I referred to this idea and elaborated on potential ways to solve this issue: NOTE: Where to from here?
The successful outcomes and problems of the A-Frame scene were noted down in this particular post and derived the need for real-time changes to be the underlying factor to achieve a fundamental shift in architectural practice. This led to understanding how Flux.io can play a role in assisting with achieving real-time changes to 3D geometry on the web.
Below was the result of a conversation I had with my industry partners in an attempt to resolve the issue with implementation of Flux.
The diagram is broken down as such:
- A database is used to store all the objects inside a building information model where each item is catalogued with attributes
- The object’s geometry data is passed through to Flux and presented on a web application built using A-Frame (my role in the process) and the end user is able to view the attributes from the database as well as edit the data/geometries directly in the interface. This will be synchronised through Flux and updated in the database (a CRUD spreadsheet — create, read, update and delete).
It was from the discussion that I attempted to follow Flux’s SDK documentation to build their default application (as a starting off point) into realising the real-time relationships of updated geometries.
In essence, it was to allow for the data of geometries to be piped directly from Flux into A-Frame for real-time editing.
This is demonstrated in the post: NOTE: Starting with Flux
The result of this exploration — Prototype 1B (shown above) proved that we could without a doubt develop the product further for collaboration and interaction (as it was more of a model viewer who can update integer data values.
The concept of hosting live geometry on the web was derived in this workflow diagram:
What is happening in the diagram is a breakdown process of sending a model from an external server to the web via Flux. Through that process a script will continuously loop to break down the model into data that is readable for the web platform as well as the database. The changes made by the user on the web platform are tracked and synchronised back to the database through Flux. Flux has the ability to synchronise back to the original model on the external server.
What was realised later on from this workflow diagram is the idea that the model is still seen as an external product. When sending back to the model, you are in fact duplicating the model and the changes made on the model externally aren’t synchronised instantaneously — which is a problem for real-time collaboration.
What do I mean by this?
We are all familiar with the collaborative example of Google Docs.
It is a web application that allows for multiple users to edit a word document on their web browser. Any changes made by a user, others will see too at the same time. This is what makes the internet very powerful.
What if the 3D model was a collaborative document?
My research project aims to suggest the possibility that this should exist. It already does in other forms such as live documentation (Google Docs) as well as online multiplayer games (Minecraft). Technologies such as verbal communication, emails and file sharing services are the alternatives practitioners use to close the communication gap about model changes. This is an ineffective and time consuming process that results with the duplication of files and repetitive method of synchronising parts of a model to be up to date.
The next step from having understood the successes and faults of the initial prototype (Prototype 1A — A-Frame Scene + Prototype 1B — Flux Default Application) was through another conversation with Kyle from Flux.io. The conversation started because from exploring Flux previously, we’ve noted down that the viewport they use, uses Three.js library. Three.js is a library that can create 3D geometries on the web and is the backbone of A-Frame. Flux had actually used the open-source library (Three.js) and developed it further to customise their standard look and interaction called ‘Flux Viewport’ which allowed them to also create their own JSON schema.
Let me rephrase that paragraph by starting off with how we should look at this scenario:
- 3D geometries on the web are made up of JSON strings (that hold properties of a geometry — such as dimensions, origins, vertices, colours, etc)
- The JSON string of a geometry is different to that of Flux because of the fact that Flux had included their own JSON schema in order to simplify how it reads the data. This is explained further in this post: NOTE: Flux Sample App Review
- What we wanted to do from here was revert it back to the Three.js JSON so that we could pipe it straight into a Three.js / A-Frame scene because we would then be able to customise the interaction of the geometries in our own viewport
- This was when I emailed Kyle from Flux in regards to being able to generate the Three.js JSON of the same geometry. He provided a react application which he put together found here: https://github.com/flux-labs/flux-three-json and successfully demonstrated the possibilities
After reviewing the application, I needed to regain a sense of how the application was put together. Through my research of learning React which is seen here, NOTE: Learn React — Part 1, it turns out that React was the perfect platform to create a web application because:
React really shines when your data changes over time.
I then built sample elements through following tutorials for React, which I further described in NOTE: Interactivity in React— Part 2.
After understanding how to build the parts of React, I used Kyle’s Flux-Three-JSON React application as the basis for the entire application. What I had learnt from the React tutorials also allowed me to create elements for my Flux projects and keys, which are displayed on the left of my application (shown below).
The initial step when developing further the React Application that Kyle — the Flux engineer put together, was to successfully implement the same features as defined in the Flux SDK tutorial. It was an unusual process when translating HTML and JavaScript to React elements. The interesting part of the process was determining states within the elements. This being the underlying feature that allowed me to stay in control of the application.
Once the elements were in place, it was a matter of correlating the submission buttons and input values to speak with Flux.
Below are some of the functions implemented to each of the elements which were prevalent in the SDK tutorial. So there is some similar processes because React does use the exact same JavaScript functions for its elements.
Now that I was able to view Flux geometries, create keys as well as update the value of its keys (as shown below), it meant that I could start to edit the data.
More specifically, through developing the default Flux Application using the Flux SDK tutorial, I picked up the possibility of adding custom attributes to any geometry by including it in it’s JSON string.
I decided to test it out by editing the sphere’s JSON string in the same project using Flux Flow (shown below). It can store the edited data into a new key, but is this necessary? In fact do we need Flux Flow altogether to be able to edit the JSON string?
After discussing further with my industry partners, it is clear that what would be really beneficial is to make the application as user friendly as possible and be able to add/edit/delete additional data to an object. By looking directly at modifying the geometry’s JSON string, we could potentially resolve this issue. More specifically, we could break down a JSON string into a data table that is editable for users of the application (sketch shown below).
What I hypothesised the process to be was:
- Obtain JSON string of geometry from Flux
- Display the geometry as a table in the web application
- Edit the data table in the web application
- Push the newly edited table as a new JSON string to Flux
- Override/update JSON data — updates geometry in Flux
Below is an interactive application that allows the selected geometry’s JSON string to be shown as a table.
http://www.encodedna.com/javascript/populate-json-data-to-html-table-using-javascript.htm
Using the above link as a guide, I was able to populate a HTML table in the application, so that whenever any geometry was selected in the viewport, it would generate the selected geometry’s JSON string into the table. However, a problem with this method is that the nested data could not be displayed. Instead it appears as [object object] and this was a problem if I wanted to edit it. It is also a HTML element and not a React element, which is problematic for setting and updating states.
After reviewing possible ways of viewing and editing JSON through React, I came across a few JSON tree editors:
riek — kaivi
Riek’s demo was what drew me in to try it out. However, as I had very little knowledge in incorporating other people’s components, I found riek difficult as it did not include an example script.
react-json-view — prodhan29
https://github.com/prodhan29/react-treeview-component
This particular component lacked documentation for me to successfully implement the feature. I realised when I applied react-json-view that I understood the problem — requires a callback function.
react-treeview-component — mac-s-g
https://github.com/mac-s-g/react-json-view#onedit-onadd-and-ondelete-interaction
The one linked above was the component I implemented successfully in the application. I had success with this particular one because of the amount of effort put into documenting how to use it.
I was able to view the selected Key’s data on the JSON tree as well as edit it, but I could not edit the data of that JSON until I pushed the edited JSON back. This was the next step.
The script below is a callback function, when the JSON tree is edited, this function is called to update the state of the data and the object (which is the JSON tree’s input JSON value), the Flux cell value is also updated with the edited JSON string as well as the Flux viewport’s geometry.
The limitations of this process of using Flux is its inability to create geometry directly from the application. My understanding for this is that:
- You can input data into the Flux Flow
- However it must be serialized as a JSON string and parsed as a JSON object as shown below before it can be a Flux geometry
Once the data is set up in Flux, the JSON is readily editable on the web application.
The ending result is a web application that is able to access Flux:
- Your Flux projects
- Your Flux Keys
- Your Flux Data
Once we were able to access the JSON data of the geometry from the keys and pipe that through into an editable JSON tree viewer, we could override the attributes and update it back to Flux.
This proof of concept shows that we can edit models in real time as collaborative documents on the web.
© Emily Y Leung and Project Redback, 2017. Unauthorized use and/or duplication of this material without express and written permission from this site’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Emily Y Leung and Project Redback with appropriate and specific direction to the original content.
Last modified on Monday 23 October 2017
