Unveiling Snowflake’s Stealthy Upgrade : A Deep Dive into Enhanced Worker Node Specs
In my recent investigation into Snowflake’s worker nodes, I discovered several significant updates that appear to have been made without formal announcements.
- The local disk capacity has increased from 200GB/node to 400GB/node
- The CPU architecture has changed from x86_64 to aarch64.
Current Specifications of Worker Nodes
The queries are now executed on the following machine specifications per node:
- Instance Type: C6g.2xlarge
- Platform: Elastic Kubernetes Service
- Operating System: Amazon Linux 2 (Linux Kernel 5.4.181)
- CPU Cores: 8 cores/node
- CPU Architecture: aarch64 (arm64)
- Disk Capacity: 400GB (392156184576 bytes)/node
In support of this, I found a statement from last May’s financial results where the CFO mentioned that they had already migrated to Graviton 2
.
We’ve fully migrated all of our customers in AWS to Graviton 2. [1]
Specifications of Previous Worker Nodes
According to the report from Select Corporation, the c5d.2xlarge
was being used as of November 2022.
it’s fairly well known that on AWS each node is a c5d.2xlarge EC2 instance, with 16GB of RAM and a 200GB SSD. [2]
To better understand these changes, I compared the single node specifications from November 2022 and January 2024
Information Gathering Procedure
Python UDF
Using Python’s platform module, I was able to output the information of the nodes.
import platform
system_info= {
'system': platform.system(),
'release': platform.release(),
'version': platform.version(),
'architecture': platform.architecture(),
'machine': platform.machine(),
'processor': platform.processor(),
'python_version': platform.python_version(),
'python_compiler': platform.python_compiler(),
'python_build': platform.python_build(),
'os_version': platform.uname(),
'disk_usage': shutil.disk_usage("/"),
'environment_variables': dict(os.environ),
}
Executed Result :
{
"system": "Linux",
"release": "5.4.181-99.354.amzn2.aarch64",
"version": "#1 SMP Wed Mar 2 18:50:47 UTC 2022",
"architecture": [
"64bit",
""
],
"machine": "aarch64",
"processor": "",
"python_version": "3.11.5",
"python_compiler": "GCC 11.2.0",
"python_build": [
"main",
"Sep 11 2023 13:15:12"
],
"os_version": [
"Linux",
"PYTHON-UDF",
"5.4.181-99.354.amzn2.aarch64",
"#1 SMP Wed Mar 2 18:50:47 UTC 2022",
"aarch64",
""
],
"disk_usage":[
392156184576,
6392868864,
385763315712
],
"environment_variables":{
"USER":"udf",
"LOGNAME":"udf",
"HOME":"/home/udf",
"TZ":"America/Los_Angeles",
"OMP_NUM_THREADS":"1",
"OPENBLAS_NUM_THREADS":"1",
"MKL_NUM_THREADS":"1",
"BLIS_NUM_THREADS":"1",
"VECLIB_MAXIMUM_THREADS":"1",
"NUMBA_NUM_THREADS":"1",
"NUMEXPR_NUM_THREADS":"1",
"CONDA_PREFIX":"/usr/lib/python_udf/xxxxxxxxxxxxxxxxx",
"PROJ_LIB":"/usr/lib/python_udf/xxxxxxxxxxxxxxxxx/share/proj",
"GDAL_DATA":"/usr/lib/python_udf/xxxxxxxxxxxxxxxxx/share/gdal",
"GDAL_DRIVER_PATH":"/usr/lib/python_udf/xxxxxxxxxxxxxxxxx/lib/gdalplugins",
"GEOTIFF_CSV":"/usr/lib/python_udf/xxxxxxxxxxxxxxxxx/share/epsg_csv"
},
}
The Python runtime is conda
. The default timezone seems to be Los Angeles
. The processor name appears to be hidden
. Each worker node is limited to a single thread.
Investigating /proc
I wanted to know the processor name, so I investigated /proc
.
In Linux, you can collect system information by looking under the /proc folder. The information about the CPU is stored in /proc/cpuinfo
, but I didn’t have read permission.
However, I did manage to look at the following files, which I had read access to:
/proc/self/exe
/proc/self/stat
/proc/self/maps
/proc/self/exe
indicates the process name of the UDF, which was /bin/python_udf_server_3.11.
/proc/self/maps
is a memory map file. It’s too long to detail here, but it’s interesting because it shows the loaded Python standard libraries, among other things.
/proc/self/stat
shows the state of the process itself, looking like this:
4858 (python_udf_serv) S 4857 4858 4788 0 -1 4194560 5530 601 0 0 9 1 0 0 25 5 2 0 9957 57290752 6351 18446744073709551615 2097152 16947760 281473996413216 0 0 0 0 17305601 17640 0 0 0 17 7 0 0 0 0 0 17475264 17517840 318431232 281473996417949 281473996423137 281473996423137 281473996423137 0
Of interest was the CPU number. The “7”
in “17 7 0”
indicates the number of the CPU executing the process, starting from 0.
I ran the UDF multiple times and it was always between 0 to 7
, leading me to deduce it’s an 8-core
CPU.
Conclusion
Snowflake has improved their infrastructure continuously.
The doubling of local disk space at no extra cost is a welcome improvement !