Computational Science— virtual doors of perception
“Essentially all models are wrong but some are useful”, George BOX.
Prologue
Computational Engineering more commonly known as Computational Mechanics (CM) or Engineering Simulation is a sub-discipline of Computational Science also known as Scientific Computing or Scientific Computation (SC). To better understand Computational Mechanics, one first needs to have a clear image of how it fits into the broader field of Computational Science. Hence, an introduction to Computational Science is not only of interest but essential. Computational Science is an interdisciplinary growing body of knowledge concerned with the use of advanced computing techniques to simulate complex models representing physical and natural events and processes. The objective of such simulation is to observe and foresee the emerging patterns in and evolution of a given complex system or process helping the observer to gain insight to the observed system. Successful simulation studies do more than compute numbers. They make use of a variety of techniques to draw inferences from these numbers. Simulations make creative use of calculational techniques that can only be motivated extra-mathematically and extra-theoretically. As such, unlike simple computations that can be carried out on a computer, the results of simulations are not automatically reliable. Much effort and expertise goes into deciding which simulation results are reliable and which are not. Computational Science is very often confused with Computer Science, although Computer Science is one of the three main pillars elevating Computational Science. The concept of Computational Science as we know today refers to the whole procedure of establishing the mathematical models, developing the numerical algorithms required to solve the mathematical models, transforming the algorithms into computer codes and finally running the codes and visualizing the results on a computer. This procedure is more broadly addressed as Modeling and Simulation M&S. The latter definition may leave the reader with more questions than answers with regards to Computational Science. To help the curious reader to gain a simple yet insightful understanding of Computational Science and eventually Computational Mechanics a retrospective review is of utmost value.
Some history
Computational Science has come a long way to morph into what it is today. Although the major momentum of the discipline took place during the mid twentieth century onward, one can trace the earliest forms of computation back to Babylonians around 1800 B.C. in Mesopotamia. They used what we call today Pythagorean Theorem in their alleged land survey and etched the results on clay tablets. Although a very simple model by today’s standards, it shows the long history of mathematical modeling, its intellectual provenance and field applicability in human society. Mathematical modeling is essentially representing a real-world scenario in the form of an abstract mathematical equation. Galileo Galilei believed “The laws of nature are written in the language of mathematics”, however, one should notice that the mathematical models do not fit exactly the real world but a refined version of it. In an attempt to decode the nature’s blueprint, mathematics and computation were fervently utilized to translate nature into a more intelligible configuration for human cognitive capacity. Modelling the real life problems by establishing a relationship between the governing quantities of interest in the form of mathematical equations persists to this day and is one of the founding principles of Computational Science. But the construction of the mathematical model is just the gambit of a whole intricate although fascinating procedure of mimicking the world around us. Once the mathematical model is established it needs to be solved in order to obtain numerical values which are the sought after unknowns of a modeled phenomenon. Finding the numerical values of the mathematical models might seem trivial to the twenty first century educated reader but the long development process of the corresponding methods to resolve such models is of historical significance for the science and enthusiasts of history of science. To put it in the context let’s take a look at the famous Pythagorean Theorem as one of the earliest forms of modeling.
The Pythagorean theorem establishes a model representing the relation between the sides of a right triangle shown in the image above in the following mathematical form
Equation 1 is a model, a mathematical one, equivalent to the geometry of any right triangle. Babylonians used this model — although using different notations — in their land survey activity to draw accurate boundaries for their properties. There is evidence that the model we today call Pythagorean Theorem was established by Chinese and Indians long before Pythagoras, how long? no one knows for sure but the first tangible proof of the model is that of Babylonian clay tablets which was also long before Pythagoras. A thousand years after Babylonians, came Pythagoras with his demonstration of the equation and to this day we call it Pythagorean Theorem.
Now back to the model in Equation 1, so far we merely have an abstract model representing a real-world scenario by establishing the relationship between its governing variables of interest, here the sides of a right triangle, in the form of a mathematical equation. Considering a and b as known values, the unknown of the equation i.e. x can be expressed in an explicit form
At this point, mathematically one can consider the problem as solved since we have an explicit formulation for our unknown x, but for practical purposes one needs to solve the problem to obtain numerical values of x. For instance Babylonians needed to ascribe a value to x, or quantity of interest, otherwise the model was of no real use for their land survey activity.
Pythagorean theorem is a polynomial of second degree (considering x as unknown, a and b as known, otherwise it changes its type), now consider a polynomial of the general form
Today we know that the solution to the above polynomial can always be expressed in an explicit way for linear (n=1), quadratic (n=2), cubic (n=3) and quartic (n=4) forms, although for quintic (n=5) and higher degrees no explicit expression for the unknown x exists, this is known as Abel-Ruffini theorem or Abel’s impossibility theorem. For those higher order polynomials one needs to resort to what is called Numerical Methods where one hopefully gets an accurate enough approximation for x.
As discussed there doesn’t always exist an explicit solution to a mathematical model and in fact in majority of cases the explicit solution is out of reach. This is not the whole issue though, one should notice that even for those cases where the explicit solution is achievable, finding the solution in terms of numerical values is a whole new challenge. To demonstrate this, let’s go back to the Pythagorean theorem and have a look at Equation 2. Let’s say the values of a and b are 3 and 5 respectively then the value of x will be the square root of 16 which is simply identified as 4, but what if the values of a and b are 7 and 8, then how do we find the square root of 15? Babylonians evidently found a way to calculate the square root of imperfect square integers with a great precision. Nobody knows how exactly Babylonians approximated values such as square root of 15, although they allegedly used an iterative algorithm. We know that the first documented presentation of an iterative numerical algorithm is referenced in the book of Heron of Alexandria called Metrica which is why the method is also called Hero’s method. An iterative method refers to a procedure where one starts with an initial guess then using the initial guess computes a better approximate and repeats the procedure until the sequence of numbers converges to a finite limit which is the correct answer. The iterative methods and the concept of convergence are very typical of numerical methods and of significant importance. As one can observe despite the existence of an explicit solution to a quadratic polynomial such as Equation 2, finding the numerical value of such simple models already requires numerical algorithms in order to obtain some exploitable results.
The use of modeling and numerical algorithms to approximate the unknown quantities was seen again, centuries after Babylonians by the work of Archimedes in finding the value of 𝝅 which is the relation between the circumference of a circle and its radius. Archimedes proposed different methods in finding the value of 𝝅. One method is that he considered a circle with inscribed and circumscribed polygons inside and outside the circle and observed that the inscribed circle has a shorter perimeter than that of the circle’s, and the circumscribed circle has a perimeter longer than that of the circle’s hence, the circumference of the circle is bound between the perimeter of the inscribed and the circumscribed polygons (see Image 2).
He started his approximation with hexagons and for more accuracy he went up to 96-side polygons where he estimated 𝝅 to be 3.1408 which is impressively accurate enough for his time. Archimedes stopped at 96-side polygons, and for a good reason, the computation becomes more and more cumbersome as the number of sides of the polygon increases.
The method of Archimedes in finding the circumference of a circle captures the quintessence of Computational Science or numerical modeling whose two main features are, first, an abstract representation of a perfect physical quality, and second, approaching the exact solution of the problem to an admissible approximation. For a very long time after the Greek era there was not much documented trace of any significant advancement related to computation, although there have been attempts in finding the square roots and 𝝅 around the world by different curious individuals, for instance Chinese were quite active and successful in their attempt to find square roots and the value of 𝝅 but their work remained obscure to the western world due to the lack of communication.
As the natural science advanced, the need for more extensive mathematical models describing sophisticated natural phenomena rose. During fifteenth and sixteenth centuries by astronomical discoveries of Kopernikus and Kepler a new computational momentum occurred. The need for modeling the movement of celestial bodies required more advanced models than the quadratic polynomial model presenting a right triangle used for land survey. By Newton and Leibniz in sixteenth century, emerged the concept of differential calculus which presents a whole new type of mathematical models namely differential equations. These are equations involving derivatives and are of several types among which the most well known types are Ordinary Differential Equations and Partial Differential Equations. Except from very simple problems differential equations are impossible to solve analytically hence, the necessity for constructing numerical methods for producing approximate solutions transpired. The emergence of such challenging and complicated problems, led scholars such as Leibniz, Newton, Euler and Gauss to develop new mathematical methods which in turn commenced a whole new era for Computational Science. These individuals were not merely interested in the mathematical models but they wanted numerical results, hence the development of new numerical methods in turn took a new more advanced shape. Today we know of Euler’s finite difference method in solving Ordinary Differential Equations (ODEs) exhibiting the concept of discretization, where the unknown variable is represented at grid-points. There is Newton’s method also known as Newton-Raphson method for finding the root of non-linear polynomials, this method is actually the more general form of Hero’s method. There also exists Gauss method for solving large systems of linear algebraic equations.
By the beginning of the twentieth century the study of numerical methods became an academic topic, the topic today we know as Numerical Analysis. For many years since the dawn of computation in Mesopotamia and during the Greek era, all the way to twentieth century the main stimulus for the evolution of the domain was pretty much exclusively astronomy and geometry but by the 1940s and the need for weapon development came even more sophisticated physical phenomena to be modeled and solved. This happened simultaneously with the advent of electronic computers which truly shaped the Computational Science as an established field of science of strategic importance right next to experiment and theory. Today Computational Science is an essential field of study in many disciplines of science and engineering. New numerical algorithms along with advanced High Performance Computers allow the resolution of complex systems with millions of unknowns. Computational Science had an astonishing boom during a very short period of time and today it’s a dominant player in almost every scientific and engineering discipline. Despite its widespread use in science and engineering, it is a rather misunderstood and is to some extent unknown to a significant portion of the scientific community. This lack of clarity may come from its interdisciplinary nature and its metamorphosis during a long period of history.
Today as an interdisciplinary field, Computational Science sweeps a broad field of technical and scientific domains e.g. Engineering, Physics, Biology, Chemistry, Astrophysics, Climate Research, Finances and Social Sciences, to name a few. The following diagram visualizes three main constituents of Computational Science.
As one may conclude from the above diagram the “Problem Domain” is where the distinction between the sub-disciplines of Computational Science arises.
Computational Mechanics — the art of simulation
Although in the realm of Computational Science it is difficult to draw clear solid lines between its sub-disciplines one can safely consider Computational Mechanics the sub-discipline where the “Problem Domain” is defined by the principles of mechanics and laws of physics. If one looks up the definition of Computational Mechanics in different sources, summary of all different definitions might come down to: “Computational Mechanics is an interdisciplinary engineering major concerned with the use of computational methods in mimicking the behavior of physical phenomena governed by the principles of mechanics/physics”. To expand upon this definition, Computational Mechanics is the discipline where computational methods and devices are used to study Theoretical Mechanics in order to apply to problems of interest in engineering and applied sciences, where Theoretical Mechanics is concerned with the study of motion of bodies under the action of forces. As a quick reminder one should know that Theoretical Mechanics is a vast field of science encompassing areas such as mathematical physics as well as Classical Mechanics including elements of quantum mechanics, molecular dynamics and the theories underlying solid mechanics, fluid mechanics and materials science.
What we call today Computational Mechanics is widely associated with solving the relevant mathematical models on a digital computer in order to gain insight into the mechanism of physical phenomena. It is called Modeling and Simulation M&S or simply Simulation. Modeling refers to the establishment of the mathematical equations which is an ancient practice since Babylonians. These models are usually expressed in a standard form called Boundary Value Problem (BVP). Boundary Value Problem or BVP has two main attributes; first, a differential equation and second, boundary conditions. The differential equation represents the behavior of the phenomenon in a specific domain. The domain is also called computational domain. Boundary conditions are another important aspect in BVP. In structures and solid mechanics the boundary conditions are displacement and force and in heat transfer we see temperature and convection as boundary conditions. The boundary conditions can further be divided into two groups of natural boundary conditions and essential boundary conditions but for the sake of brevity we don’t discuss the reason behind this terminology, as the avid reader will find an extensive discussion of the matter in literature.
Computational Mechanics has been of enormous help in enriching our understanding of complex natural phenomena and in the design of countless engineering systems. In order to be accurately modeled most of physical phenomena and mechanical processes require Partial Differential Equations (PDE) which are the dominant differential types appearing in Boundary Value Problems. PDEs are studied in a mathematical discipline called calculus and deal with continuous change e.g. the heat flowing through a medium (the rate of change of the temperature through time and space). Along with PDEs there exists Ordinary Differential Equations (ODE) the solution to which was already discussed by Leonard Euler. At the time of Euler hand calculations were the only possibility and that’s probably why the efforts were limited to Ordinary Differential Equations. Now, what about the solution to PDEs? PDEs are of fundamental importance in modeling of physical phenomena and its mathematical foundation is interwoven with Computational Mechanics. Towards the end of nineteenth century there was an abundance of mathematical models of PDE type representing various physical and natural phenomena. The use of difference methods for PDEs as done for ODEs presented enormous challenge as a discretization in time and space giving any reasonable accuracy required huge computational effort, that it was virtually impossible to realize it for any practical application. In the beginning of twentieth century Walther Ritz (1878–1909) a Swiss theoretical physicist proposed a variational form of the PDE model representing the deformation of an elastic plate caused by a given force. This method was later used by the English physicist Lord Rayleigh (1842–1919) in his work on finding eigen-frequencies of a vibrating string. Eventually Boris Galerkin (1871–1945) a Russian engineer generalized the method and applied it to many different engineering problems. Although Galerkin in his paper refers to the method as Ritz method, today we know it as Ritz-Galerkin or Rayleigh-Ritz or simply Ritz method.
Ritz-Galerkin method is the corner stone of one of the most impactful numerical algorithms in science and engineering i.e. Finite Element Method (FEM). Finite Element Method is arguably the epitome of Computational Mechanics and for good reason; despite the utility of Ritz-Galerkin method in solving PDEs it presents certain limitations which are surmounted in FEM. Richard Courant a German mathematician first discussed Finite Element Method in his paper in 1943. Courant’s article doesn’t contain a single equation and is explained in plain text, however in its annex the method is applied to a practical problem. He discusses how one can discretize the domain into rectangles and then each rectangle is transformed into a triangle by drawing an orthogonal which allows defining a linear function on each triangle. This is how Finite Element Method has been introduced to and transformed the field of Computational Mechanics. Although the first complete formulation of the method, together with significant applications solved on digital computers, appeared in a paper by John Turner, Ray Clough, Harold Martin, and L. J. Topp in 1956. Today Finite Element Method is arguably the most widely used numerical method in different applications in finding an approximate solution to natural phenomena modeled using Partial Differential Equations, and Computational Mechanics in particular is where one studies most of these phenomena.
Some of the most recurring PDEs in engineering science are the following equations — for the sake of brevity the parameters are not declared, although the kin reader can look them up with details in literature —
Static equilibrium equations in three dimensional solid mechanics:
Navier-Stokes equations in fluid mechanics:
And Maxwell’s equations in electromagnetic:
One may wonder: how do mathematical models such as the ones presented in Equations 4 to 6 come to existence? or to be more specific how does one abstract a physical phenomenon through a mathematical equation? The way that a mathematical representation of a physical phenomenon comes to existence is through mechanical principles — at least in the realm of Computational Mechanics — such as equilibrium conditions or constitutive laws and also fundamental laws of physics such as conservation of linear momentum, conservation of mass and conservation of energy. That’s how for instance Equation 4 is built (a combination of Cauchy stress theory, equilibrium principle, etc.). These laws define the behavior of a mechanical/physical system by representing the evolution of the system’s state which in turn is conceptualized in the model.
Today most of Boundary Value Problems are smoothly handled by commercial software suits as well as advanced open source solvers. Because of the widespread use of the Finite Element Method in commercial and open source solvers we elaborate briefly on this method and how it approximates the solution to a Boundary Value Problem. The basic premise of Finite Element Method is altering the computational domain into a collection of small, overlapping simple geometries called finite elements. This step is called discretization which allows for the model to be solved over simple geometries rather than on the entire domain which might be in a very complex form. Recall that the model is a mathematical equation of the PDE type and the answer to a PDE is a function, this function is approximated over the finite elements in the form of polynomials using interpolation theory, although the approximation is not required to be of polynomial form. The interpolation is done using the discrete values found at specific locations called nodes. The last step is to assemble the element-wise approximations and construct the global resolution over the domain which gives a numerical analog of the original mathematical model. The three steps of discretization, element-wise approximation and assembly are the three main steps in finite element procedure and are the projection of a main important difference between Finite Element Method and traditional variational methods such as Ritz-Galerkin, that is variational methods present serious shortcomings among which the most important one is constructing the approximation function. The approximation functions are arbitrary and there is no systematic way of constructing such functions, this particularly becomes an issue knowing that the approximation functions need to satisfy certain features, such as linear independence, continuity and essential boundary conditions, now respecting all such conditions on a geometrically complex shape makes traditional variational methods computationally ineffective. This is where Finite Element Method comes to rescue by presenting a systematic way for constructing approximation function on an element level. One should recall that numerical methods such as Ritz-Galerkin, approximate the solution of a PDE on the whole domain and there is no discretization involved in the process.
In Computational Science in general and Computational Mechanics in particular since the numerical solution to the problem is typically an approximation of the exact answer, one needs to validate and verify the accuracy of the results at the end of the process, this last part is called validation and verification or V&V. Verification is said to be the process of determining whether the output of the simulation approximates the true solutions to the differential equations of the original model. Validation, on the other hand, is said to be the process of determining whether the chosen model is a good enough representation of the real-world system for the purpose of the simulation.
The reasons for which engineers and scientists need to study a numerical method, notably finite element method can be manifold:
- Practical problems involve complicated domains (both geometry and material constitution), loads and nonlinearities preventing an analytical solution, therefore, the only alternative is an approximation solution using numerical methods.
- Numerical methods along with electronic computers can be used in order to investigate the effects of various parameters (e.g. geometry, material parameters or loads) of the system on its response to gain a better understanding of the process/system being analyzed. It is cost effective and saves time and resources compared to the multitude of physical experiments needed to gain the same level of understanding.
- The power of numerical methods coupled with electronic computers allow us to take into account all relevant features in a mathematical model of a physical phenomenon without worrying about its solution by exact means.
Moreover the finite element method and its generalizations are the most powerful computer oriented methods ever devised to resolve engineering and scientific problems. Major industries such as automobile, aerospace, pharmaceutical, electronics as well as emerging technologies such as nanotechnology and biotechnology rely on the finite element to simulate complex physical processes at different scales.
In a nutshell
Human insatiable thirst for deciphering the laws of nature and the way it functions led him to an abstract representation of physical phenomena in the form of mathematical models. Most common types of mathematical equations representing physical phenomena are Ordinary Differential Equations and Partial Differential Equations. Although it is one thing to develop such mathematical equations to represent natural phenomena; it is quite another thing to find numerical results to satisfy such equations. The long history of computational methods to solve such models dates back at least to Babylonians although the practice of mathematical modeling persists to this day, to which engineers and scientists resort in order to gain insight to phenomena and systems they wish to understand. The insight gained using the models may in one or more ways contribute to human convenience and bring comfort to human life.
Resolving the mathematical models by hand is a tedious and time consuming task. By the introduction of electronic computers during twentieth century, human’s natural mathematical abilities received a formidable enhancement. Such enhancement significantly increased the speed with which human can perform certain mathematical tasks thus allowing the expansion of the problem complexity from a restricted range of simple topological structures to structures of far greater complexity or even different type. Digital computers and informatique techniques also allowed the conversion of numerical results into graphical form, this in turn allows the easier access to massive data sets by shifting from one representational mode to another. The realm of Computational Science and Computer Simulation not only impacted human life from a technological and mundane stand point but also raised the bar from an intellectual point of view for philosophers of science. The fact that much of the successes of the modern physical sciences are due to calculation, brought Computational Science to the center of the attention of the philosophers of science and how the electronic computers along with numerical methods extend human native inferential abilities in ways where human memory capacity is enhanced. Human’s congenital mathematical aptitude have been supplemented by computational devices enabling us, in certain areas, to move beyond what our psychological capacities can achieve. This enhancement can greatly increase the speed with which we can perform certain mathematical operations, thus altering the time scale in ways analogous to the way that optical telescopes alter spatial scales.
Computational methods today play a central role in applied science, engineering and technology. From astronomy, physics and biology to the design of computer chips, cars and aircraft, computer simulation and computational methods are standard practice. Computational Science in general and Computational Mechanics in particular unite computational simulation, scientific experimentation, geometry, mathematical models, visualization, and high performance computing to address some of the “grand challenges” of computing in the sciences and engineering. Advanced graphics and parallel architectures enable scientists to analyze physical phenomena and processes, providing a virtual microscope for inquiry at an unprecedented level of detail.
Computational Science opened new doors of perception into a different perspective of natural world, both in physical and intellectual terms. As computers become more pervasive in our lives, simulation is considered the epistemological engine of our time. Simulation has achieved this honor because of its ability to extend our conceptual models with the speed and efficiency that is beyond our cognitive abilities.
At the end it is entertaining to observe, despite the fact that Modeling and Simulation M&S scaled up our inferential abilities in dissecting and deciphering the natural world, one shouldn’t overlook the fact that none of the models used in Computational Science represents reality as it is. When we talk about a real world referent of a natural system, we are actually talking about the concept that we have, utilizing symbols we agreed upon. This formal representation of the system is merely a symbol of the natural system. In M&S, our model is the conceptualization of the naturalization, and our simulation becomes the symbol standing for it. In this sense we never acquire knowledge of the natural system itself, but of the representing formal system that we used to describe the natural system.
It is then enlightening to close this content with an allusion to the observation of Aldous Huxley, “However expressive, symbols can never be the things they stand for”, as he expressed in his essay “The doors of perception”.
References:
- “Scientific Computing, a historical perspective” by Bertil Gustafsson.
- “Extending Ourselves: Computational Science, Empiricism, and Scientific Method” by Paul Humphreys.
- “An Introduction to the Finite Element Method” 3rd Edition by J.N. Reddy.
- “Epistemology of modeling and simulation” by Andreas Tolk et al.
