Blender, Python and Your Custom Data Part 1: loading data into attributes

In this blog we will use scientific data to make a 3D visualization in Blender 3.5.0. A nice way to use your own data in Blender is to use attributes. Once you have your data as an attribute in Blender you can use it in Geometry Nodes and in your shaders. As an example, we will use data from the particle accelerator at CERN and make the visualization shown in Fig. 1a and 1b.

Figure 1a: a visualization of particle showers created by collisions in the CERN particle accelerator. The small blocks making up the ‘showers’ indicate detectors surrounding the collision and the color of the blocks indicate the energy reading of the detector. The data shown is generated by the GEANT4 toolkit based on the Linear Collider Detector. We only show data from the calorimeter that is sensitive to particles that interact through the electromagnetic force (ECAL readings). Thus the ‘showers’ you see (4 in this case) could be from for example photons or electrons being absorbed in the calorimeter.

I will explain how to create Fig. 1a and 1b in a series of three blogs. The first part, this blog, covers how to get the data into attributes in Blender. The second part will show how to use attributes in Blender Geometry Nodes. And lastly, the third part will dive into using your attributes in your shaders.

Figure 1b: animated version of Fig. 1a showing several different showers.

The data

For this blog we use generated data from the particle accelerator at CERN*. The data is from simulations where electrons and positrons (the electron anti-particle) are collided with each other. Out of such collisions come multiple particles. These particles are absorbed by detectors surrounding the collision, called calorimeters. These detectors absorb the particle to measure for example their energies. Calorimeters do not absorb the particle in one go, but stop and interact with the particles through multiple layers of material and detectors. This causes a so called ‘shower’ of absorptions inside the many detectors that make up the calorimeter. The many detectors are located around the point of the original collision. The structure of the data coming from one calorimeter is a cube of dimensions 51x51x25 and each point in this data cube represents a detector that holds an energy value. Fig. 1a shows a representation of four ‘showers’ detected in four calorimeters surrounding a collision.

You can get more information on the data from this recent paper by Khattak et al. 2022. And the data can be downloaded from zenodo here. The data is stored in the Hierarchical Data Format. In order to open these .h5 files in Python you can use the h5py package. To open the file and select one shower in Python would look something like this:

import h5py
with h5py.File(filename, "r") as f:
  # This can be handy in order to see the keys available
  print("Keys: %s" % f.keys())
  # You can select the energies from one shower (with index 10) like this
  one_shower = f['ECAL'][10] # one_shower will be of shape 51x51x25

Scripting

Now, lets get right to it and look at the script that puts the data into a Blender attribute. First we import numpy and bpy. Next we use numpy to load a data cube for one shower.

import os
import numpy as np
import bpy
 
# We load the data using numpy
nr = "8_9_5"
shower = np.load(os.path.join(directory, "example_shower_"+str(nr)+".npy"))

If we would run print(shower.shape) we can see the data cube has dimensions (51, 51, 25). Here the last index, with 25 entries, is the one in the radial, outwards direction of the shower. We will use the indices of the element in the data cube as vertices. Thus our model of one shower will contain 51x51x25 vertices.

One cube is data from one shower detected in one calorimeter, but the collision center is surrounded by a cylinder made up of multiple calorimeters. We need to translate the coordinates in de cube (local_coords) to coordinates relative to the center line of the collider (called vertices). We first center the coordinate system of the cube to its center by shifting it by (-25, -25, 0). Also we rotate the data cube to point in the radial outwards direction by an angle phi (if you want the details on how the data is oriented, please see the paper). Lastly we translate the data radially outwards by R_tube to make room for the inner cyliner where the collision takes place. And we must not forget to save the energy values from the data cube in energiesFloat.

# This is the angle from the x axis. We take this random
phi = (random.uniform(0,360)/360) * 2*math.pi
# Distance from the origin for the shower
R_tube = 0.5 

# The local_coords are the coordinates in the data cube
# before rotation
local_coords, vertices, energiesFloat = [], [], []
for index, elem in np.ndenumerate(shower):
    # We save the energy of the current detector
    energiesFloat.append(elem)
    # We use the indices of the cube as local coordinates
    # of the data. We center the data around the middle of
    # the cube.
    local_z = index[2]
    local_y = index[1] -25
    local_x = index[0] -25
    # This next bit is to rotate the data cube and transform
    # the local coordinates to the global coordinates of the
    # scene. Please see the paper if you want to understand 
    # this in detail.
    R = math.sqrt(local_x**2 + local_z**2)
    if R == 0:
      z = R_tube*math.cos(phi)
      x = R_tube*math.sin(phi)
      else:
        if local_x >= 0:
            phi_local = math.acos(local_z/R)        
        else:
            phi_local = -1*math.acos(local_z/R)   
        
        z = R_tube*math.cos(phi) + R*math.cos(phi + phi_local)
        x = R_tube*math.sin(phi) + R*math.sin(phi + phi_local)
      
      y = local_y
      # And save the coordinates of the vertices.
      vertices.append((x,y,z))

We can now make the object and mesh in Blender and use from_pydata() to create the geometry. We also link the new object to a collection. If you are new to making geometry through Python code, see this previous blog.

# Create a new mesh
ob_name = “shower_”+str(nr)
mesh = bpy.data.meshes.new(ob_name + “_mesh”)
 
# Create a new object and add the mesh
ob = bpy.data.objects.new(ob_name, mesh)
 
# Add the geometry to the mesh
mesh.from_pydata(vertices, [], [])
 
# Link the object to the collection
shower_collection_name = ‘Showers’
bpy.data.collections[shower_collection_name].objects.link(ob)

And lastly we will add the energy values as attributes to the Blender object. The Blender docs describe attributes as: “An attribute is a generic term to describe data stored per-element in a geometry data-block.”. Examples of attributes you might know are: Vertex groups, UV maps and Color Attributes. But, as we will do now, you can also create your own custom attributes.

The way I think about it is that attributes allow you to save data (for example values or vectors) for every element (say per vertex, edge or for example face) of an object. In our example every detector reading is a vertex and for every vertex we want to save the energy readout as a float. In script, the attributes are stored in ob.data.attributes and a new attribute can be created by calling ob.data.attributes.new. The type of the attribute will be float and its domain will be point. The point domain means we store a data value for every vertex. For example, other domains let you store values per edge or face.

# Create a new attribute for the object
ob.data.attributes.new(name=’myAttributeFloat’, type=’FLOAT’, domain=’POINT’)

# Store the energies in the new attribute
ob.data.attributes['myAttributeFloat'].data.foreach_set('value', energiesFloat)

The data in ob.data.attributes[‘myAttributeFloat’] is of the type bpy_prop_collection. The elements of the collection are of the type bpy_struct and in our case where we have floats it is of the type FloatAttributeValue(bpy_struct). Because of this, setting the elements of a collection is somewhat involved and the preferred way to do it is through the foreach_set method. Have a look at the documentation for collections to learn more. Now you can run the script and you should see something like Fig. 2.

Figure 2: the vertices generated from the 51x51x25 sized data cube.

Inspecting Attributes in Blender

There are atleast two easy ways to inspect attributes in Blender. The first is through the Properties Editor and its Object Data Properties tab (see Fig. 3). There you see a list of standard attributes like Vertex Colors, but under Attributes you see the attribute myAttributeFloat that we just made. You also see that per vertex it stores a float.

Figure 3: you can inspect the attributes of your model in the Properties Editor and the Object Data Properties tab.

If you want to see the values stored in the attribute, there is another way to inspect attributes. This is through the Spreadsheet editor which is by default shown in the Geometry Nodes Workspace. In Fig. 4 you can see the spreadsheet and the attribute myAttributeFloat that we just made. You can see that it has a value for every vertex in the cube. We will talk more about this spreadsheet in the second part of this blog serie which covers using Attributes in Geometry Nodes.

Figure 4: you can inspect attributes in Blender in the Spreadsheet Editor, by default found in the Geometry Nodes Workspace.

Closing

This was the first blog in a series of three. We have learned how we can create attributes from our own custom data. In Fig. 1a we actually show four showers from four different data cubes, instead of one as in Fig. 2.

In the next blog we will learn how to use the attributes in Geometry Nodes in order to create, alter and remove geometry. This wil be the next step in making Fig. 2 look like Fig. 1a! Hope this was helpfull.

Cheers, Ben