Neural Networks with JMP
Building neural networks to determine body fat percentages based on various health measures
Digging into JMP once again, this time we’ll look at neural networks as we attempted to build a model predicting the body fat percentage in men. Data was collected from participants by measuring various aspects of their physical bodies. These variables include age, weight, height, and circumference of the body at the neck, chest, abdomen, and other areas. The dataset ultimately includes observations from 252 participants, which help provide a basis for our models. We’ll look at two neural networks, a basic and a complex version, and compare the predictions of these with our benchmark Least-Squares model.
The dataset with scripts for the models below can be found on GitHub (.jmp file).
Methodology
Using JMP, a number of options are available when building a neural network. The hidden layer, or “black box” layer, of the model can use 1–3 transformation functions to manipulate the data: TanH, Linear, and Gaussian. Each has its own strengths, so some models may benefit from certain functions more than other models. For this study, we created both a “base” and “complex” neural network. The base model requests 3 nodes in the first layer using only the TanH function. Our complex model uses 3 nodes each for TanH, Linear, and Gaussian activation functions, and repeats the structure in a second layer.
The other specification we adjusted for our models was the number of tours. JMP and other software tend to model neural networks to the point where they become too complex and start modeling random noise. To help prevent this issue, JMP applies a penalty function so the model stops growing once the model’s predictions stop improving. Increased tours helps us build multiple models in effort to narrow down the best-fitting model averaged among each network. For both our neural networks, we’ll use 20 tours.
Baseline Model
As usual, we first want to develop a basic least-squares regression model. This gives us a benchmark to compare predictions with the neural networks later. We’ll also need to split the dataset into train/validate/test sets for cross-validation. This is especially important when working with neural networks as JMP will use the validation set to check predictions as the model is built to ensure its performance is improving. We used a 60/20/20 split, giving us 151 observations in the training set, 50 in validation, and 51 in the testing set. Working with ordinary least-squares gives us a model with an R-square value of 0.6657 on the testing set. While this is a good starting point, the neural networks will hopefully improve our predictive accuracy
Neural Networks: Base Model
After creating a benchmark, we can move on to our first neural network. As mentioned above, we’ll use two approaches to defining our NN. The first opts to use only 3 nodes in the first layer using the TanH hyperbolic tangent function. We left the penalty method as using the squared value and set the number of tours to 20. The results of our first model can be seen below. An R-square value of 0.7049 on the test set is an improvement over the OLS, and a lower RMSE also indicates this is the stronger model of the two. Next, we’ll look at a more complex neural network.
Neural Networks: Complex Model
The third model we’ll build is a neural network using all 3 functions on two layers of the model. This can give us a more accurate model, but it also runs the risk of creating a model which relies on too much randomness. We opted to use 3 nodes each for the TanH, Linear, and Gaussian functions on both the first and second layer, ultimately resulting in 18 nodes in the hidden layer of our model. The images below compare the difference in complexity between the basic NN (left) and the complex model (right).
With our new, complex model, we can see an increase in model accuracy from the previous two approaches. On the testing set, our complex NN has an R-square value of 0.7524, which is easily the highest of the 3. It also has a slightly lower RMSE and AAE than the base network. Thus, we can more forward assuming this is the best-fitting model for our data.
Going Deeper
Settling on the complex neural network as our chosen model, we now want to take a look at variable importance within the model. Abdomen circumference has a significantly higher influence on the model than any other variable, with over 92% of the percentage body fat being affected by this variable. The next strongest influencer is weight, which only has a 0.056 total effect on the model. This is followed by hip and thigh circumference with 0.047 and 0.038, respectively. Our other two models have similar reports, indicating this is not too strange for our data.
Now that we’ve determined the best model, we can try it on a brand new observation. Using the following data, we should be able to make a prediction with reasonable accuracy:
Age (years)=50
Weight (lbs)= 167
Height (inches)= 67.75
Neck circumference (cm)= 38.8
Chest circumference (cm)=100.4
Abdomen circumference (cm)=89
Hip circumference (cm)= 93.2
Thigh circumference (cm)= 57.0
Knee circumference (cm)=34.8
Ankle circumference (cm)= 20.6
Biceps (extended) circumference (cm)= 33.9
Forearm circumference (cm)= 28.3
Wrist circumference (cm)= 18.0
Entering those values into our model gives us an estimated percentage body fat of about 18.69%. For comparison, we also tested a calculator provided by an external website (www.calculator.net/body-fat-calculator.html). According to this source, a male with these measures would have around 19.8% body fat. Our model’s estimate is off by 1.11%, for an error of about 5% based on the website’s calculator.
