SAM Underworld project

Overview

Underworld 2 is a python-friendly version of the Underworld code which provides a programmable and flexible front end to all the functionality of the code running in a parallel HPC environment. This gives signficant advantages to the user, with access to the power of python libraries for setup of complex problems, analysis at runtime, problem steering, and multi physics coupling. While Underworld2 embraces Jupyter Notebooks as the preferred modelling environment, only standard python is required.

The Underworld2 development team is based in Melbourne, Australia at the University of Melbourne and at Monash University led by Louis Moresi. We would like to acknowledge AuScope Simulation, Analysis and Modelling for providing long term funding which has made the project possible. Additional funding for specific improvements and additional functionality has come from the Australian Research Council (http://www.arc.gov.au). The python toolkit was funded by the NeCTAR eresearch_tools program. Underworld was originally developed in collaboration with the Victorian Partnership for Advanced Computing.

Numerical methods

Description

Underworld is a Lagrangian integration point finite element code. This is a modernization of the original particle-in-cell concept from the 1960s in which a structured mesh and an unstructured particle swarm co-exist. The mesh is used to solve diffusion-dominated parts of the problem and the particle swarm is used to track advected quantities. In the finite element context, the mapping from mesh to particles is through the usual basis functions of the elements and the mapping from particles to mesh is through the integration scheme used to build up the stiffness matrices etc.

The applications of the method are mainly in modelling of complex fluids where very large strains occur but the material also has a memory of the entire strain / strain-rate history. In geosciences this occurs due to the visco-elasticity of rocks at lithospheric temperature and their tendency to develop fabric (lattice preferred orientation and stress/strain-dependent grain size). Problems with material interfaces which undergo severe distortion during the deformation are also naturally handled by this method provided there is no slip on the interface.

Background

The method has been published in detail in Moresi et al (2002, 2003)1. These papers dealt exclusively with 2D applications but in recent years, we have introduced a number of improvements in the method to enable us to scale the problem to 3D. For example we developed a fast discrete Voronoi method to compute the integration weights of the particle-to-mesh mapping efficiently 2. We have also concentrated on extremely robust solvers / preconditioners which are necessary because the material variations and geometrical complexity are both large and unpredictable at the start of the simulation.

The benefit of this approach is associated with the separation of the computational mesh from the swarm of points which track the history. This allows us to retain a much more structured computational mesh than the deformation / material history would otherwise allow. We can take full advantage of the most efficient geometrical multigrid solvers and there is no need to preserve structure during any remeshing operations we undertake (for example if we do need to track a free surface or an internal interface). Although there are several complexities introduced by enforcing this separation, we find that the benefits, for our particular class of problems, are significant.

Implementation and parallelism

Underworld is implemented using the StGermain framework. This provides the essential infrastructure to manage i/o, meshes, particle swarms, finite element operations, in a parallel (domain decomposition, message passing) environment. The numerical solvers are based around the PETSc software suite which focuses on delivering good parallel scalability (up to thousands-of-cores). Our experience to date shows good scalability for thermal problems to 10000+ cores.

References

Discover our user manual, installation process, source code and blog.

Also, our cited research list and latest activities on Facebook and Twitter.

Technical references

John Mansour, Owen Kaluza, Julian Giordani, Romain Beucher, Rebecca Farrington, Gareth Kennedy, Louis Moresi, Mirko Velic, Adam Beall, Dan Sandiford. (2018, October 31). underworldcode/underworld2: v2.6.0b (Version v2.6.1b). Zenodo. http://doi.org/10.5281/zenodo.1475861

Louis Moresi, John Mansour, Julian Giordani, Rebecca Farrington, Owen Kaluza, Steve Quenette, Robert Woodcock and Geoffrey Squire, (2017), Underworld - Bringing a Research Code to the Classroom, Abstract [ED41B-0283] presented at 2017 Fall Meeting, AGU, New Orleans, LA, 11-15 Dec.

Moresi, L., Quenette, S., Lemiale, V., Mériaux, C., Appelbe, W., Mühlhaus, 2007, Computational approaches to studying non-linear dynamics of the crust and mantle: Phys. Earth Planet. Inter, v. 163, p. 69–82, doi: 10.1016/j.pepi.2007.06.009.

Moresi, L., Dufour, F., and Muhlhaus, H.B., 2002, Mantle convection modeling with viscoelastic/brittle lithosphere: Numerical methodology and plate tectonic modeling: Pure And Applied Geophysics, v. 159, no. 10, p. 2335–2356, doi: 10.1007/s00024-002-8738-3.

Moresi, L., Dufour, F., and Muhlhaus, H.B., 2003, A Lagrangian integration point finite element method for large deformation modeling of viscoelastic geomaterials: Journal of Computational Physics, v. 184, no. 2, p. 476–497.
Quenette, S., Moresi, L. N., Sunter, P. D., & Appelbe, W. F. (2007). Explaining StGermain: An aspect oriented environment for building extensible computational mechanics modeling software. Presented at the HIPS 2007 Workshop, Parallel and Distributed Processing Symposium, 2007. Proceedings. 19th IEEE International.

Velić, M., D. A. May, and L. N. Moresi (2009), A fast robust algorithm for computing discrete voronoi diagrams, Journal of Mathematical Modelling and …, doi:10.1007/s10852-008-9097-6.

SAM PROGRAM LEADER
Prof. Louis Moresi
Australian National University

Return to the SAM main page

Explore our other programs