Sie sind auf Seite 1von 63

Strömungstechnik

Innovative CFD Methoden für Studien zur Wasserstoff- Sicherheit mit High-Performance Supercomputing (HPSC) - Weiterentwicklung, Validierung und Performance Analyse

Werner Rehm

Juli 2001

Kurzfassungsbericht über EU/HGF-CFD/DDT Projekte

CFD with HPC

CFD with HPC T h e A u t h o r : Dr. rer. nat.

The Author: Dr. rer. nat. Werner Rehm, Year of Birth 1942

Physicist in natural science and PhD at RWTH Aachen working as research scientist, project and work group leader at the Research Centre Jülich (FZJ), who has gained long-term experience in various aspects of computational fluid dynamics (CFD) in the aerea of safety research under severe accident conditions, especially for conventional and nuclear energy systems, including fire prevention and explosion protection (e.g. HTR-, LWR-, and H2-safety technology). His current interest is applied computational fluid dynamics (ACFD) with higher order turbulence and combustion models using high-performance supercomputing (HPSC) of engineering flows in complex geometries. He is a member of the ERCOFTAC Pilot Center Germany West, of the CFD-Society in Canada and of various programme committees. He has performed several international projects in the field of hydrogen safety technology with relevance in research and practice. In this context, he has presented many CFD-related publications at conferences and in journals.

CFD with HPC

Innovative CFD Methoden für Studien zur Wasserstoff- Sicherheit mit High-Performance Supercomputing (HPSC) - Weiterentwicklung, Validierung und Performance Analyse

Kurzfassung

In diesem Report werden die F+E-Arbeitsergebnisse dokumentiert, die im Rahmen von EU/HGF-Projekttätigkeiten zur numerischen Simulation von reaktiven Strömungen in komplexen Geometrien durchgeführt werden. Das Ziel ist die Weiterentwicklung der Methoden zur numerischen Fluiddynamik (CFD), um mit Hilfe von High-Performance Computing (HPC), die Explosionsprozesse in technischen Systemen genauer analysieren zu können. Die Anwendungsbeispiele betreffen konventionelle und nukleare Energiesysteme, insbesondere die Sicherheitsaspekte einer zukünftigen Wasserstofftechnik. Der Schwerpunkt liegt bei der Modellierung des störfallbedingten Verhaltens von Wasserstoff in Sicherheitseinschlüssen hinsichtlich Verteilung und Verbrennung von Gasgemischen, die bei langsamen, schnellen und überschnellen Flammen eine Rolle spielen. Für den Brand- und Explosionsschutz werden Modelle und Kriterien entwickelt, um Übergänge von Deflagrationen in Detonationen (DDT) sicherheitstechnisch mit adäquaten Gegenmaßnahmen kontrollieren zu können. Dabei wird das physikalische Mixing-Konzept mit Dilution- und Inertisation Medien verwendet und vorgeschlagen.

CFD with HPC

Innovative CFD Methods for Hydrogen Safety Studies Using High-Performance Supercomputing (HPSC) - Improvement, Validation and Performance Analysis

Abstract

This report describes the R&D work performed within the scope of EC/HGF project activities concerning the numerical simulation of reacting flow in complex geometries. The aim is the refinement of numerical methods used in computational fluid dynamics (CFD) by introducing high-performance computations (HPC) to analyse explosion processes in technical systems in more detail. Application examples concern conventional and nuclear energy systems, especially the safety aspects of future hydrogen technology. The project work is mainly focused on the modelling of the accident- related behaviour of hydrogen in safety enclosures regarding the distribution and combustion of burnable gas mixtures, ranging from slow to fast or even rapid flames. For fire and explosion protection, special models and criteria are being developed for the assessment of adequate safety measures to control deflagration- to-detonation transition (DDT) processes. Therefore, the physical mixing concept with dilution and inertization media is applied and recommended.

CFD with HPC

Contents of the Report

Preface

6

Overview

8

Synopsis

9

Technical Summary

11

Conclusion and Outlook

42

References

44

Acknowledgments

47

Annex: Field Codes and Supercomputing

48

Nomenclature

61

Curriculum Vitae

63

5

Preface

CFD with HPC

The assessment of the integrity of safety enclosures under severe accident conditions is an important safety issue for conventional and nuclear power plants, especially safety studies for severe accident conditions exploring innovative safety trends. These studies are under consideration to improve the safety behaviour of technical systems and to reduce the consequences of accidents for the environment as far as possible. To limit the increasing global climate problem and the greenhouse effect, nuclear power and hydrogen technology are potential future energy options. However, increasing safety requirements have to be fulfilled to minimize industrial risks and to harmonize the safety culture. In the case of severe accidents in water-cooled nuclear reactors, for instance, significant quantities of hydrogen and steam could be released into the atmosphere of the containment, locally reaching flammability or even detonability limits. To control accidental hydrogen combustion, different safety measures have been developed as part of the defence-in-depth concept with several accident management strategies. However, concerning flame acceleration or hot jet mecha- nisms, for example, a transition from deflagration to detonation (DDT) cannot be excluded a priori and may produce very high explosion loads jeopardizing safety equipment and enclosure systems. In this context, several research projects based on computational fluid dynamics (CFD) have been established to study the effects of DDT on hydrogen explosion loads in large-scale enclosures. These joint projects involved a close collaboration between research centers, universities, and industry, e.g. FZJ, FZK, FUB, RWTH, AEA, CDL, INCAS, and UPM, with know-how transfer leading to synergy and spin-off effects. The R&D effort was supported by the HGF senate

6

CFD with HPC

in Bonn and the EC directorate in Brussels within the scope of joint hydrogen

safety projects based on an international cooperation.

This productive cooperation was focused on hydrogen distribution,

combustion, flame acceleration and DDT processes, including CFD code impro-

vement with validation and the exploration of mitigation or prevention features.

Main aspects are related to scientific high-performance supercomputing (HPSC)

for large-scale simulations of reactive flows in complex geometries, using vector

and parallel processing capabilities, including aspects of fire protection and

explosion prevention for innovative energy systems, especially for nuclear safety

and hydrogen safety with relevance to industrial safety. Besides the optimization

of modern CFD methods with supercomputing for safety analyses, further studies

were performed for hydrogen control systems, which are used in conventional fire

protection and industrial explosion prevention related to process and

environmental safety. Therefore, several industrial companies were contacted,

e.g. ESMG, INBUREX, FIKE, BINDER, TOTAL WALTHER, KIDDE-

DEUGRA, and HYDROGAS.

In this context, we also would like to thank foreign partners from Europe,

Canada, USA, and Japan for their helpful discussions in the field of fluid dynamic

safety analyses and hydrogen safety concepts for nuclear and conventional

applications relevant in research and practice.

Werner Rehm, July 2001

Project Leader and Coordinator c/o

FZJ Research Centre Jülich, Germany

ISR Institute for Safety Research and Reactor Technology

Email: w.rehm@fz-juelich.de or w.rehm@t-online.de

http://www.fz-juelich.de/isr/

http://isr230.isr.kfa-juelich.de

7

Overview

CFD with HPC

The final documentation of the HGF-H2 Project in FZJ report Part 2: CFD

Software Optimization and Special Application consists of the following reports:

1. Final Report Part 1: Advanced CFD methods with supercomputing - improvement, validation, and performance, by W. Rehm, C. Nae, et al., Aug. 2001, FZJ Report

2. Detailed Report Part 2: Recent DDT simulations and criteria based on experimental and numerical results, by B. Wang, W. Rehm, Sep. 2001, FZJ Report

3. Analysis Report Part 3: Benchmark calculations and performance analysis for special application examples, by W. Jahn, W. Rehm, Oct. 2001, FZJ Report

4. Summary Report Part 4: Evaluation and application plan for fluid dynamic safety analysis, by W. Rehm, W. Jahn, C. Nae, B. Wang, Nov. 2001, FZJ Report

Foundation: The basis of the final documentaion of Part 2 is the EC project on H2DDT and the further development required within the scope of the HGF-H2 Project based on the project proposal and the positive vote of the HGF reviewer for the FZJ’s work programme outlined in the presentation for the evaluation. During the working period between 1998 and 2001, both EC and HGF projects were performed in parallel and have been documented and the results published in several progress reports and presented at international conferences. In the final report Part 1, we present in selected chapters the main project work and major results that have been obtained together with our project partners, s. Ref. [27].

8

Synopsis

CFD with HPC

In this short summary report, we present the R&D work that is being performed within the framework of applied computational fluid dynamics (ACFD) with high-performance supercomputing clusters (HPSC) for joint EC/HGF-funded CFD/DDT projects, aimed at the numerical simulation of turbulent reacting flows in complex geometries at the so-called GGG computing level to gain high numerical resolution in space and time. As a result, it is shown that scientific-technical high-performance supercomputing networking (HPSN) has the capability to enhance a close interdisciplinary collaboration between universities, research centres, and industry. Major aspects are related to the prediction of physical-chemical phenomena using a modern field code cluster (MFCC), solving the reactive Navier-Stokes/Euler equations with higher order turbulence and combustion models on a powerful platform, including code verification and validation of numerical results by experimental observation. In this context, special CFD project results are outlined in five selected chapters relating to hydrogen behaviour - distribution and combustion - as well as the transition from deflagration to detonation (DDT) in explosive hydrogen mixtures using CRAY supercomputing systems and different parallelization strategies. The report focuses on CFD optimization and application studies. The main task is to obtain a sufficiently high numerical resolution of reactive flows in complex geometries using supercomputing to study fire and explosion loads. In addition, it is shown in the appendix that each grid size has an optimum number of processor nodes for parallel processing. This was analysed in more detail for large-scale multi-dimensional simulations of non/reacting flows based on several distribution and combustion test cases. Furthermore, these project studies are relevant with regard to the maintenance of scientific competence and expertise at

9

CFD with HPC

FZ Jülich to improve the safety culture for innovative energy systems by combining basic research work and end user practice for the industrial safety of technical systems, including fire protection and explosion prevention in conventional and nuclear power plants, especially for hydrogen technology as a future energy option. Besides the modern CFD methodology developed with supercomputing, a major effort was made in the field of industrial safety to control fire and explosions inside and outside of safety enclosures, studying the various safety techniques for process safety and industrial explosion protection, including the state of the art and new development directions with federal regulation trends. The H2-CFD/DDT-related research work has been performed during the past few years at the Institute for Safety Research and Reactor Technology in the ISR1 Department of Research Centre Jülich (FZJ). We gratefully acknowledge the ISR directory with the critical guideline and we appreciate the scientific reorganization of the ISR, which was initiated by the board of the directors. The author is grateful to all partners for supporting and funding this project work, especially the HGF senate in Bonn and the EC directorate in Brussels for their valuable consultations. The EU-H2DDT project work was successfully completed with seven European partners and is documented in FZJ-ISR Report Vol. 9, reaching extensive consensus. The following German HGF project on H2 safety for nuclear reactors was performed by FZ Karlsruhe and FZ Jülich consisting of Part 1, Part 2, and Part 3. In Part 2 we obtained recent results with our partners which are different from Part 1 and Part 3, apart from a harmonized compromise in the controversial debate about hydrogen safety features.

10

Technical Summary

CFD with HPC

Safety Aspects: Generally, fire and explosion protection is an important safety issue in most areas of engineering. For instance, the prediction of hydrogen explosion loads in safety enclosures (reinforced concrete containment or steel vessel systems) under severe accident conditions is relevant for nuclear reactor containments and hydrogen demonstration plants with fuel cells or hydrogen- powered vehicles. Whether or not a transition from deflagration to detonation (DDT) could occur in safety enclosures depends on the specific plant design and the anticipated accident scenario. Explosion phenomena are under investigation worldwide due to the high danger potential, as experimental observation has shown. The DDT phenomena have been extensively studied so that most of them are qualitatively understood. However, quantitative methods have not yet been established for objective assessment in large-scale enclosures. DDT is a very complex process associated with turbulent reacting flows and shock wave interactions where rigorous closure solutions are still in the distant future so that practical approaches are needed. The major problems are related to the different time and length scales of the physical-chemical processes involved, which are difficult to predict in realistic safety geometries for ambient conditions. It is well known that the likelihood of DDT increases with the geometrical scale and that for the numerical treatment huge computer resources are necessary to gain a sufficiently high numerical resolution both in space and time. (Note: It is an ambitious task to define detonation limits in lean hydrogen-air mixtures for large- scale geometries). Therefore, it is reasonable to subdivide the potential DDT processes into essential stages, e.g. for slow, fast, and rapid flames in the sub/super-sonic regime, and to develop suitable software tools capable of numerically defining DDT conditions or criteria to control the resulting combustion mode and the load response.

11

CFD with HPC

Objectives and Aims: The HGF project on reactor safety addresses innovative methods of the analysis and control of hydrogen behaviour in core meltdown accidents. The project consists of three parts. Part 2 of the project is performed by FZJ-ISR1 and is related to: CFD Software Optimization with Nuclear and Non-nuclear (Special) Applications. The prediction of hydrogen behaviour under severe accident conditions is an important safety concern for the energy systems of nuclear and conventional power plants to accommodate increasing safety regulations by advanced safety methods thus minimizing the hydrogen risk for industrial safety. In general, hydrogen behaviour consists of two classical phenomena, namely hydrogen distribution and hydrogen combustion. Both phenomena are associated with turbulent flows. The description of turbulent flows is one of the most difficult problems to solve in continuum physics. Depending on the information needed for the details of the flow field, different physical models and numerical techniques have been developed in the literature to solve the continuum equations with the conservation quantities in the form of so-called Navier-Stokes equations. Therefore, the prediction of hydrogen behaviour has, in principle, two main aspects. The first one is a more scientific aspect pursued in the research work of universities and the second one is a more engineering aspect focused on the end- user work of industry. For this reason, in the past few years a wide range of predictive tools have been developed for various application aims, ranging from lumped parameter codes to multi-dimensional field codes. Following the project aim of developing innovative methods for the prediction of hydrogen behaviour, the underlying basic idea of Part 2 was, as a working hypothesis, to combine selected computational fluid dynamics (CFD) tools in a modern field code cluster (MFCC) with supercomputing capabilities, which are useful and suitable for research work as well as for end-user practice, representing the state of the art of

12

CFD with HPC

non-reacting flow simulations (distribution) and reacting flow simulations (combustion) in complex geometries. The selection and evaluation of the basic CFD codes used at FZJ in Part 2 was made by expert consulting, professional assistance from academics or commercial companies, and a comprehensive literature review, e.g. Ref. [1-17], based on our own experience already gained in the EU-H2DDT project to fulfil the tasks outlined in the project proposal and proven by the positive vote of the HGF reviewer committee. The main tasks include: improvement, validation, performance and optimization of numerical simulations with supercomputing systems for non-reacting flows (turbulent fluid), reacting flows (turbulent combustion), including transition from deflagration to detonation (DDT) regarding hydrogen safety analyses with fire and explosion protection for technical systems. Proof tests and test cases are more closely related to non- nuclear applications (hydrogen technology) and less to nuclear applications (reactor technology), performing special single effect studies from a generic safety point of view on the basis of the specific R&D programme of FZ Jülich.

Work Tasks: Within the scope of Part 2, special subtasks were planned in more detail and described in the project proposal with the following topics :

Improvement, optimization, and application of modern field codes for Fluid Dynamic Safety Analysis in the FZJ for technical systems, especially for hydrogen safety.

A probabilistic safety analysis (PSA) was not performed since the working

group moved to a different department. However, PSA studies were performed by NNC Knutsford as a partner in the EU-H2DDT project so that these results are also available. A safety analysis of structural mechanics could not be performed because of the closing period of the working group. But the finite element program

13

CFD with HPC

DYNA was ported, parallelized and tested on the T3E in a separate FZJ project and is now also ready to use.

As a consequence, the project work of Part 2 was focused on Applied

Computational

Computing (HPSC) for the simulation of reactive flows in complex geometries, including multi-fluid flows, particle transport and fluid-structure interactions. In principle, the project team was able to determine the basic items and issues, which were outlined in the HGF-H2 work programme and specified in the FZJ tasks for detailed CFD/DDT studies. Concerning the H2 analysis, the R&D work was mostly performed for dry hydrogen-air mixtures as worst case considerations. With respect to H2 control, the mixing concept was selected with dilution/inertization media as suppression agents. Considerable progress was thus made in analysing hydrogen safety conditions and defining safety criteria for innovative power plants as a future energy option, which are useful for limiting the increasing climate problem and fulfilling advanced safety requirements minimizing industrial risks.

(ACFD)

Super-

Fluid

Dynamics

with

High-Performance

Contents: The final report consists of five selected chapters. The first chapter outlines applied computational fluid dynamics (ACFD) using high- performance supercomputer clusters (HPSC). The second chapter describes large-scale simulations for complex engineering flows with recent solvers, including LES methods. The third chapter summarizes experimental and numerical studies of explosion phenomena in hydrogen-air mixtures under severe accident conditions. The fourth chapter highlights reactive flow simulations in complex geometries with high-performance supercomputing. The fifth chapter gives an overview of application examples for hydrogen safety technology, including safety systems for fire protection and explosion prevention used in

14

CFD with HPC

process and industrial safety. Besides the conclusion with outlook, an extensive bibliography has also been compiled. Finally, the appendix collects field code equations, the main results of parallel processing with a large number of performance and analysis tests, including representative input data for some CFD codes. Great effort was involved in preparing the input data sets for the several field codes used in various test cases, including the handling of input/output data for pre- and post-processing with visualization or animation.

Basic Equations and Field Codes: The fluid motion can be described by field codes solving the conservation equations for mass, momentum, and energy, which form together with the equation of state a closed mathematical system, i.e. the so-called Navier-Stokes (Na-St.) equations. The governing equations are formulated, as follows:

(1)

/ t + div (

u) = 0;

(2)

(

u j u) = - grad p + div(

grad u j ) + S M ;

(3)

(4)

u j ) / t + div( i) / t + div(

p = p ( , T) and i = i ( , T).

(

i u) = - p div u + div(k grad T) + S i

;

These governing equations for rate of change + convection - diffusion = source term can be expressed for a general variable ( ) in the generic form, as to:

∂ ( ) ∂ t + ( u - ) = S
∂ (
) ∂ t +
(
u -
) = S

with the closure terms:

(a) Standard turbulence- and combustion models are based on the eddy

viscosity ( ) and a two-equation turbulence model (k-eps): T = C k 2 / with

the eddy life time e = k / .

15

CFD with HPC

(b) A turbulent combustion model

Damköhler number being larger than unity (Da = e / ch > 1).

(c) A shock compression heating (S C ) with a reaction model: S C = A c ( /

ch ) for T > T ig using the chemical induction time ( ch ) and the ignition

temperature (T ig .).

The basic partial differntial equations (PDEs) are integrated in a computing

domain, using the conservative form of finite volume methods (FVM) with

various accurate differencing schemes and robust solver techniques for the

convection, diffusion and source terms with ambient initial and boundary

conditions. The resulting set of algebraic equations is solved with direct or

indirect algorithms (i.e. iterative or coupled solver), as to: a(p) Q(p) = a(w) Q(w)

+ a(e) Q(e) for the residual reduction to the defined values and aimed at grid-

independent solutions. Here, we select from verification of codes versus

validation of results, as to an independent end-user point of view.

The basic equations contain all of the physics of a given turbulent flow, due

to the fact that turbulence is a continuum phenomenon. Turbulence modelling is

one of three key elements in CFD using RANS, LES or DNS methods, (Fig. 1).

Mathematical theories have been developed for the other two key elements, i.e.

grid generation and algorithm development. An other issue is related to the

combustion modelling of premixed flames, (Fig.2).

For the numerical simulation of transient, compressible, turbulent, chemi-

cally reacting flows, we have developed special field codes for specific studies

providing new versions of reactive Navier-Stokes and on reactive Euler flow sol-

ver codes. The software and hardware systems of the modern field code cluster

are based on an innovative methodology for CFD predictions of engineering

flows in complex geometries using high-performance supercomputing facilities.

and the

(S T ): S T = - A T

( /k ) m lim

16

CFD with HPC

Present Flow Solvers and Combustion Models

Computing Effort

high

low

 

Research

Development

Standard

     
       

RNS

Resolution

DNS

DNS

LES

LES

Subgrid Scale Models

VLES

VLES

Mean Values

 
   

RANS

RNS

 
       

Energy Spectrum

 
Mean Values       RANS RNS           Energy Spectrum  
Mean Values       RANS RNS           Energy Spectrum  
Mean Values       RANS RNS           Energy Spectrum  

small Eddies

large Eddies

Kolmogorov

Integral

resolved

Scales

Fig. 1: Overview on Navier-Stokes flow solver methods with DNS, LES, VLES, and RANS techniques used in research, development and standard applications.

Mixed-is-Burnt Model PDF Model DDT Combustor Eddy Dissipation Model Engine Flamelet Model lam. gewinkelte Flammen
Mixed-is-Burnt Model
PDF Model
DDT
Combustor
Eddy
Dissipation
Model
Engine
Flamelet Model
lam. gewinkelte Flammen

Fig. 2 Combustion models for premixed flames as to the Borghi diagram, i.e.

flame

velocity

ratio versus

flame

length

scale

Damköhler and Karlovitz numbers.

17

ratio

with

the

parameters

CFD with HPC

Modern Software and Hardware Infrastructure

Deflagration PVP System MPP System Detonation CFX-PVM T90 DET-HPF T3E-600 T3E-1200 Reactive 10 Proc. Reactive
Deflagration
PVP System
MPP System
Detonation
CFX-PVM
T90
DET-HPF
T3E-600
T3E-1200
Reactive
10 Proc.
Reactive
512
Proc.
512
Proc.
8 GBytes
Navier-Stokes
64 GBytes
262
GBytes
Euler
18 GFLOPS
300
GFLOPS
614
GFLOPS
Solver
Solver
MFCC-MPP: D3UNS, CFX, AIXCO, DET, (COM, DYNA)
DEC-WS
J90
HPSC- ERCO-DDT
Pre- and Post-
16 Proc.
SHOCKIN
Semi-Implicit
Processing
8
GBytes
Flame-Shock
IFSAS
Networking
3
GFLOPS
Na-St. Solver
BUILD, FIEDVIEW
Fluid Structure Dynamics
File Server
Transition

Fig. 3: Modern field code cluster (MFCC) with high-performance super- computing on the CRAY computer complex at FZJ for the simulation of reactive flows in complex geometries with parallel processing.

Tab. 1: Main CFD/DDT-related computer models used in hydrogen safety.

Distribution Mode (Concentration): Non-reactive Na-St. Solver D2UNS-MPI (FZJ) Unstructured Navier-Stokes Flow Solver with LES Modelling

Deflagration Mode (Flame Acceleration): Reactive Na-St Solver CFX-3D/MPI

(AEA) Higher Order Turbulence and Combustion Models with Unstructured Grids

DDT Mode (Auto-Ignition): Reactive Na-St Solver AIXCO-2D/MPI (RWTH) Flame Front Tracking/Shock Wave Capturing Schemes and Reduced Reactions

DDT Mode (Shock-Ignition): Reactive Euler Solver SHOCKIN-2D/AG (FZJ) Unstructured Reactive Euler Flow Solver with Reduced and Detailed Chemistry

Detonation Mode (Propagation): Reactive Euler Solver DET-2D/HPF (FZK) Hydrodynamics with Chemical Source Term and One-Step Reaction Kinetics

Fluid-Structure Analysis (Loads): Reactive Euler Solver IFSAS-3D/AG (CDL) Shock/Detonation Ignition Models with Adaptive Grids for Explosion Loads

Structure Mechanics (Response): Finite Element Code DYNA-3D/PVM for static and dynamic deformations of safety enclosures (CONDAT)

18

CFD with HPC

Innovative CFD Methodology for Hydrogen Safety

Slow Flame

Fast Flame

Rapid Flame

Slow Flame Fast Flame Rapid Flame Deflagration Explosion/DDT Detonation Modelling     Phenomena
Slow Flame Fast Flame Rapid Flame Deflagration Explosion/DDT Detonation Modelling     Phenomena
Slow Flame Fast Flame Rapid Flame Deflagration Explosion/DDT Detonation Modelling     Phenomena

Deflagration

Explosion/DDT

Detonation

Modelling

   

Phenomena

Slow Flame Fast Flame Rapid Flame Deflagration Explosion/DDT Detonation Modelling     Phenomena

Flame

 

Scenario

Codes

   

Deflagration

CFX

AIXCO

SHOCKIN

 

Explosion

DET

IFSAS

 

Detonation

IFSAS

DYNA

Prediction of Explosion

 

AICC

Pressure

Acceleration

 

Shock

Pressure

Transition DDT

 

CJ

Pressure

Propagation

 

Hazard

Potential

Loads in Safety Enclosures

Slow Flame

subsonic

Fast Flame

turbulent, sonic

Rapid Flame

supersonic

Combustion Loads Pressure, Impulse

Flame supersonic Combustion Loads Pressure, Impulse Hydrogen Flames and Modern CFD Codes Using Supercomputing

Hydrogen Flames and Modern CFD Codes Using Supercomputing

for the Prediction of Explosion Loads in Complex Geometries

Figs. 4: Hydrogen flame scenario (above) and reactive field codes for hydrogen safety technology (below) of technical energy systems.

19

CFD with HPC

Results: In Part 2 of the HGF-H2 project, we developed modern CFD methods for the analysis of hydrogen combustion studies using supercomputing capabilities related to software optimization and special applications including improvement, validation, and performance tests. The modern field code cluster (MFCC) developed consists of several reactive Navier-Stokes/Euler flow solvers:

D2UNS, CFX, AIXCO, SHOCKIN, DET, and IFSAS with an advanced platform for pre- and post-processing performed with the PATRAN, BUILD and AVS, FIELDVIEW tools on fast workstations. Characteristics are hybrid grids, higher

order turbulence and

combustion models

with advanced

solver techniques,

including supercomputing with parallel vector processing (PVP) and massively parallel processing (MPP), which is executed on the CRAY supercomputer complex (J90, T90, and T3E) of the FZJ, (Fig. 3 and Tab. 1). Our modern field code cluster is capable of numerically simulating non/reacting flows of slow, fast, and rapid hydrogen flames in complex geometries with resulting loads, (Fig. 4), and consists of the following powerful software and hardware infrastructure:

D2UNS Code: We developed a new unstructured Navier-Stokes solver

for non-reacting flow simulations (hydrogen distribution) consisting of several classical turbulence modelling and large eddy simulations (LES) for compressible flows, including subgrid scale (SGS) modelling based on a Smagorinsky model with wall damping, wall laws and dynamic modelling by Germano. The code works quite well on the T3E in the MPP mode and was nowadays successfully tested for benchmark test cases reported in the literature (e.g. channel flow around obstacles). Presently, 3-dimensional versions are under development together with multi-fluid models and options for reacting source terms. The code is a sophisticated research tool for the large-scale simulation of engineering flows, especially for turbulent mixing with inertization, making full use of mesh

20

CFD with HPC

refinement and adaptation using flow field parameters. Special information is highlighted in the annex.

SHOCKIN Code: For the numerical simulation of reacting flows

related to hydrogen (auto)self-ignition (i.e. spontaneous thermal ignition), we developed a new adaptive reactive Euler solver with a two-step reaction model, including reduced or detailed chemistry for the influence of active/passive additives. The code was validated for various H2-DDT experiments performed at SWL of RWTH Aachen showing an excellent agreement between experimental and numerical results and providing detailed insights into shock wave behaviour. For the first time, H2-DDT test simulations were extensively performed for self- ignition conditions in a large-scale containment model geometry with about 50.000 m**3 volume. The code is an advanced research tool for analysing DDT self-ignition conditions for complex geometries and for various hydrogen-air mixtures, including dilution and inertization effects to prevent explosions. Specific information is outlined in the annex.

(hydrogen

distribution and combustion) in un/confined geometries, we significantly improved the CFX software. This software consists of CFX-4 version (structured grids) and CFX-5 version (unstructured grids) with body-fitted coordinates. CFX- 4 includes a variety of physical-chemical models, whereas CFX-5 was extended for new wall functions and eddy dissipation combustion models (EDM) with flame-quenching modelling. Various classical turbulence models can be applied (e.g. k-epsilon, k-omega, RNG, etc.). We implemented a new LES model in a first test version for in/compressible flows based on a Smagorinsky model with modified wall functions for non-reacting flows, which was tested for tube and

CFX

Code:

For

the

prediction

of

non/reacting

flows

21

CFD with HPC

channel flows with obstacles. Throughout this approach, the subgrid viscosity is computed by the subgrid scale model ( s ) which can be written in the form of:

s = (c s D

) 2 ( 2 S ij S ij ) 1/2

with the S ij resolved strain rate tensor using the grid cell, D the wall daming, and c s as model constant. The CFX-4 version was ported to the T90 for the PVP mode and CFX-5 was optimized on the T3E for the MPP mode. Performance tests showed a good scalability with increasing grid cells and number of processor nodes. Both new versions of CFX-4/5 were extensively used and validated in experiments for hydrogen distribution and combustion tests, especially for flame acceleration with repeated obstacles (RUT tests), for hot jet ignition in multi-room tests (BMC tests), and for flame-quenching tests (ENEL test). Empirical parameters were fine tuned for the selected test cases indicating that the experimental results agreed quite well with the numerical simulations. The CFX-4/5 code system is a comprehensive design and analysis tool for numerical simulations of industrial flows including fire and explosion protection. For instance, Fig. 5 displays the numerical simulation of a turbulent H2-flame acceleration in the channel with repeated obstacles into the cavity of the large- scale explosion RUT facility with about 500 m**3 volume.

AIXCO Code: For the numerical simulation of hydrogen combustion

(flame acceleration and DDT processes), we essentially improved the reactive Navier-Stokes code AIXCO. This code is based on the flamelet combustion model with flame front tracking and shock wave capturing schemes. The flame front is described as a discontinuity between the burnt and unburnt gas, using a level set formulation ( G equation ) with appropriate burning laws ( s ) which are

22

CFD with HPC

derived from combustion theory or experimental results. The associated level set function evolution equation can be written, as follows:

G t + ( v + s n )

G = 0; and |

G | = 1

with G ( x, t ) = flame front level set, s = normal burning velocity, and v = flow velocity. The effects of turbulence can be taken into account through classical two-equation models (e.g. k-eps.). In addition, a two-step reaction model is implemented to study self-ignition characteristics of various hydrogen-air mixtures based on reduced chemical reaction schemes. Furthermore, the code is able to simulate the transition from a deflagration wave to a detonation wave, using a switching criterion defined by the critical ignition temperature. We ported, parallelized and optimized the AIXCO code on the CRAY-T3E, obtaining good performance tests and reasonable speedup factors with increasing grid cells and processor nodes up to 500 processors (600 Mhz). The code has now been tested in small-scale laboratory DDT tests provided by SWL at RWTH Aachen and large-scale explosion tests performed in RUT-KI, Moscow, reviewing the DDT modelling. Finally, the code was extended for very large eddy simulation (VLES) using two-scale turbulence modelling with down- and up-scaling between the integral and the subgrid scales for non-reacting flows, e.g. mixing layers, jets, wakes, and channel flows with obstacles. Recently, the VLES turbulence model is adapted in combination with the flamelet combustion model for the numerical simulation of turbulent reacting flows. In conclusion, the AIXCO code is a modern research tool for the prediction of turbulent combustion, based on RANS, VLES, or DNS flow solver options used for combustion engines and hydrogen safety. For example, Fig, 6 demonstrates the numerical simulation of a H2-flame interaction with shock wave focusing in the corner of the RUT test cavity reaching auto-ignition conditions in the unburnt gas ahead of the flame with DDT pre-conditioning.

23

CFD with HPC

Temperature Field: 300 to 1600 K Fast Flame
Temperature Field: 300 to 1600 K
Fast Flame
Pressure Field: 1 to 5 bar Shock Focusing
Pressure Field: 1 to 5 bar
Shock Focusing

Fig. 5: CFX (k-e, EDM) test example for flame acceleration (FA) in a channel with temperature and pressure contours (11% H2-air at NTP).

burnt gas unburnt gas Density (kg/m**3) Time = 0.178 s
burnt gas
unburnt gas
Density (kg/m**3)
Time = 0.178 s

Fig. 6: AIXCO (k-e, FC) test case for DDT flame-shock wave interaction in a cavity with density contour (15% H2-air at NTP).

in out Cavity DDT Formation of Detonation at 20 ms Pressure (bar) 0.3 36
in
out
Cavity
DDT
Formation of Detonation at 20 ms
Pressure (bar)
0.3
36

Fig.7: IFSAS (1-step, AG) test case for shock wave focusing in a corner with DDT self-ignition and detonation formation in an explosive hydrogen-air mixture as to pressure contour (15% H2-air at NTP).

24

CFD with HPC

IFSAS Code: For the detailed analysis of explosion phenomena with

relevance to industrial safety, we implemented a new version of the Integrated Fluid Structure Analysis Software (IFSAS) with the SuperSTATE module concerning shock ignition, detonation propagation, multi-phase flows, dust explosion, fluid-structure analysis, and load response. For resolved computations and for detonation wave simulation a standard Arrhenius law is normally used in the fuel consumption process as described:

(

f ) t +

(

v f ) = -

f b exp ( - E / RT ) with T = p /

R

or a more sophisticated chemical rate law can be employed. The code uses adaptive grids for static and dynamic mesh refinement, including slanted boundaries. The code was tested and validated for representative explosion processes (SWL/RUT tests), especially for single-effect studies associated with shock ignition, detonation propagation over obstacles with resulting loadings in confined and partly confined geometries for various explosive mixtures and ambient conditions. For practical applications, an extended version is under development. Indeed, a typical test case is presented in Fig. 7 showing the numerical simulation of shock wave focusing with self-ignition and detonation formation of an explosive H2-air mixture in the RUT test cavity with high DDT pressure peaks.

Supercomputing: To analyse the different aspects related to hydrogen

safety, these special CFD/DDT codes were developed for specific studies with a high numerical resolution. We ported the modules of the modern field code cluster (MFCC) on several computer platforms, ranging from fast workstations to powerful supercomputers, with respect to meta-computing to increase the available computer resources, depending on the required details of the

25

CFD with HPC

non/reacting flow field. Most of the modules of the MFCC software system were optimized for parallel processing on the CRAY supercomputer complex at FZJ as a center of excellence. The CRAY complex consists of the J90, T90, and T3E, which are integrated in a high-speed network operating in the upper GGG- computing class, whereas the teraflops class is already under investigation for even more challenging tasks. In this context, we studied several parallelization strategies, focusing on massively parallel processing (MPP) using multi-blocks, domain decomposition and message passing tools (HPF, PVM, and MPI). The performance results were compared with the theoretical values for linear speeding (Sp) defined by the Amdahl‘s law, as to:

Sp (a) = p / (1 + (p - 1) a ) with p = no. of processors and a = sequential part.

We will therefore use results obtained in other projects, such as UNICORE (access), KOJCEK (performance), GIGABIT (test bed west), for high-speed networking and clustering in Europe based on EC/OECD-initiated project plans. Recently, we peform large-scale simulations of the order of some million grid points with about 500 processor nodes obtaining a reasonable performance, scalability, and efficiency above 50 %, especially for large eddy simulations (LES) of engineering flows for more challenging tasks, (Fig. 8).

prevention, or protection measures in various technical applications. For fire protection and explosion prevention, effective suppression agents have been studied recently and already being used in industrial safety, public buildings, and high-tech equipment (Fig. 9). The effectiveness depends on the special plant design and the specific requirements, which are under investigation in further project activities with relevance to industrial safety, Ref. [14].

H2

Safety:

Hydrogen

control

systems

are

based

on

mitigation,

26

CFD with HPC

CFD with HPC Fig. 8: Speedup for massively parallel processing on the CRAY-T3E with the CFX-LES

Fig. 8: Speedup for massively parallel processing on the CRAY-T3E with the CFX-LES test case in channel flows over obstacles using 3.000.000 grid points.

in channel flows over obstacles using 3.000.000 grid points. Fig. 9: Fire protection system with FM-200

Fig. 9: Fire protection system with FM-200 spreeding in the control tower of the airport at Düsseldorf by KD.

27

CFD with HPC

Collaboration: The project work of Part 2 was performed in close collabaration with the ITM of RWTH in Aachen concerning the further development of the flamelet combustion code AIXCO code, especially for the extension of VLES turbulence modelling with new turbulent combustion models together with powerful parallel processing. Another close collaboration was established with AEA Technology in Munich (Otterfing) regarding the general purpose software CFX with special emphasis on LES models in combination with advanced eddy dissipation combustion models and massively parallel processing, which have recently also been used for engineering studies. Finally, we worked together with CDL Canada to improve the explosion software for industrial safety studies. With these productive collaborations it was possible to obtain access to findings from related research work and to involve recent developments in our HGF-H2 project work in Part 2.

Cooperation: The HGF-H2 project was performed in cooperation between FZ Jülich and FZ Karlsruhe, the latter being the coordinator. We received the deflagration code COM and the detonation code DET from the coordinator for some test cases performed in Part 2. For this purpose, the detonation code DET2D was rewritten in Fortran 90 for parallel vector processing (PVP) on the T90 and using High-Performance Fortran (HPF) for massively parallel processing (MPP) on the T3E, obtaining quite good performance and scalability for large numbers of grid sizes and processor nodes. The detonation code DET-2D was adapted in parameter studies for DDT conditions of mild and strong ignition by shock wave focusing effects in the RUT test geometry. The deflagration code COM-3D was used for performance tests on the T3E with varying results and applied for the numerical simulation of flame acceleration tests with repeated obstacles performed in the FZK tube. Experimental and numerical results showed

28

CFD with HPC

tentative agreement, but we used the COM code more or less as a black box, due

to the curtailed cooperation time and lack of data needed for further studies.

Findings: We developed a modern field code cluster (MFCC) with high- performance supercomputing (HPSC) using the CRAY complex at FZJ, which is

a suitable constellation for improved hydrogen safety analysis of technical

systems related to distribution and combustion effects including explosion aspects

of the transition from deflagration to detonation (DDT). The modern field code

cluster consists of special modules for specific tasks combining applied CFD methods for practical applications as design or analysis tools together with future CFD methodology for the next generation of codes as research tools. Therefore, we also provided higher order turbulence and combustion models, including large eddy simulations (LES). It is thus possible to obtain a high numerical resolution of non/reacting flows in complex geometries, depending on the required details of the flow field. The prototype versions developed were verified (code-to-code) and validated (code-to-experiment) in benchmark and experimental tests with fine tuning of the model parameters, obtaining considerable agreement between experimental and numerical results. Coarse grid simulations can be performed on fast workstations with single-processing, whereas fine grid simulations preferably need multi-processing. Moderate parallel processing is well established, but we also realized a remarkable performance and scalability of our reactive Navier-Stokes/Euler codes for massively parallel processing (MPP) on the CRAY-T3E in various performance and optimization tests.

Comprehensive application test cases for hydrogen distribution and combustion showed proper iteration and convergence behaviour with sufficient reduction of the residual to the defined values. Parameter and sensitivity studies outlined the uncertainties associated with grid-independent solutions for turbulent

29

CFD with HPC

mixing and turbulent combustion. We found that the eddy dissipation combustion model generally resulted in appropriate results, but the model constants must be fitted to the special combustion mode, for example, related to flame acceleration or hot jet ignition. We assumed that for the flamelet combustion models fewer empirical contants would be required using standarized burning laws, [Ref. 18,

19].

However, predictions with scaling of turbulent combustion and DDT processes are still a very complex task and remain an open question for hydrogen safety applications in real geometries. Concerning DDT conditions, our modern field code cluster offers the ability to analyse DDT conditions in complex geometries with regard to critical flame speeds, critical shock Mach number, critical self-ignition temperatures, onset of detonation formation and propagation (without the cell structure). More progress has been made in simulating DDT mode A ahead of the flame front with shock wave focusing and self-ignition in the unburnt gas compared with DDT mode B in the flame brush with turbulent mixing and re-ignition triggering DDT processes, which would need further investigations and huge computer resources. In any case, the numerical simulation of DDT conditions allows a more objective evaluation basis than in the past in contrast to the use of simple DDT criteria in a more subjective way (e.g. only based on detonation cell size classes and geometrical ranking). Such simple DDT criteria must be carefully used in realistic safety studies, where it is not quite clear whether or not simple criteria are a conservative approach and where the real mechanisms are not taken into account. DDT results have shown that very high reflected detonation pressure peaks can be produced in explosive H2-air mixtures in the order of about 100 bar and more, which need suitable hydrogen control systems to limit the resulting combustion loads inside and outside safety enclosures, Fig. 10. The hydrogen control options are related to mitigation, prevention, and protection, as discussed below with some pros and cons.

30

CFD with HPC

Inherent H2 Safety Features for Explosion Protection

Recombiners Lean Mixtures 4 % H2 Igniters Deflagration: CFX Code (D2UNS) 9 % H2 Slow
Recombiners
Lean Mixtures
4
% H2
Igniters
Deflagration: CFX Code
(D2UNS)
9
% H2
Slow
Flame
Explosion, DDT:
Hazard
AIXCO, SHOCKIN
Regime
Fast
Flame
18
% H2
Inertization
Steam
Detonation
N2, CO2, He
Rapid
Flame
DET, IFSAS
Mixing Media
Micro Sprays
58 % H2
Limiting
Conditions
Loads: IFSAS Code
(DYNA)
70
% H2
Hot Jet Injection
H2-Air-Steam
75 % H2
Rich Mixtures

Fig. 10: Hydrogen safety features for fire and explosion protection in technical systems, consisting of safety measures, including recombiners, igniters, or inertization to control slow, fast or rapid flames, especially deflagration-to- detonation transition (DDT) for anticipated events in severe accident conditions, including predictive CFD methodology for safety assessment

31

CFD with HPC

Mitigation: The hydrogen safety problem in LWR reactors is associated with the production of significant quantities of hydrogen-air-steam mixtures (besides the reactants CO, CO2, etc.) in the case of an extreme core meltdown accident in the reactor containment due to metal-water and core-concrete reactions (i.e. in- and ex-vessel reactions). Depending on the specific reactor design and the anticipated accident scenario, it is to be expected that slow, fast, or rapid hydrogen flames could develop, jeopardizing the safety equipment and the safety enclosure. Severe conditions for the reactor containment would occur during an unlikely transition from deflagration to detonation, initiated by hot jet ignition, multiple ignition, fan- induced transition or flame acceleration with shock-wave amplification and detonation formation with end-wall reflection. To analyse and control hydrogen combustion events, several safety methods and various safety devices have been studied worldwide as part of the defence-in- depth concept to reduce the hydrogen risk under severe accident conditions. For instance, recombiners and/or igniters have been proposed as part of the so-called dual concept to reduce the amount of free hydrogen or to ignite the flamable hydrogen at a low concentration level. However, recombiners have a limited capacity, may be poisoned or could be superheated and igniters could trigger an explosion under unusual conditions. As a consequense, it is questionable whether the two safety devices would always be safety-orientated (fail-safe) in realistic accident sequences. Safety-orientation is a basic principle in nuclear safety technology! Anyway, recombiners and igniters have special failure rates, which should be evaluated by fault tree and event tree analysis based on probabilistic safety analysis before a scientific-technical proven statement could be made about the efficiency of such devices in reducing the hydrogen risk in comparison with other core meltdown risks. In contrast to the recombiner/igniter devices, the dilution/inertization measures of sensitive hydrogen mixtures act as fire- and explosion-protection

32

CFD with HPC

systems which are strongly safety-orientated and should be considered for further investigation and optimization. The diluation/inertization system can be designed as an active and/or passive safety system, as already used in conventional safety technology. The amount of inertization defines the quenching of rapid, fast, or slow hydrogen flames. Furthermore, a cold injection of post-inertization lowers the containment pressure as part of the accident management strategy (including water sprays or filtered venting). Finally, an insensitive containment atmosphere can be handled more easily and safely than a sensitive one. For dilution/inertization measures, experimental data are available, e.g. for CO2, N2, He, etc., or modern mixture media for fire protection and explosion prevention. There exist several safety reasons for partial- or post-inertization measures in PWR types, e.g. with CO2, whereas N2 pre-inertization is already used in BWR types during operation (but not in the startup and shutdown period). Last but not least, dilution/inertization measures make the hydrogen safety problem less complex and it is therefore more or less reduced to the question of hydrogen distribution and mixing quality to exclude hydrogen combustion loads in containments. More recently, modern CFD methods with supercomputing have been established to predict hydrogen distribution and inertization mixing rates with a high numerical resolution in complex geometries. Therefore, we improved our modern field code cluster (MFCC) with high-performance supercomputing (HPSC) with special emphasis on using classical turbulence modelling for the mean flow quantities of standard applications as well as introducing large eddy simulations (LES) for more detailed flow quantities of engineering flows. LES developments are now powerful numerical tools for fluid flow predictions, which will become more effective with increasing supercomputer resources towards the teraflops class in the near future so that more precise numerical simulations will be possible.

33

CFD with HPC

Prevention: To control the hydrogen safety problem in light-water reactors, prevention measures should have the highest priority! Therefore, the reactor should be designed so that the volume to power ratio is optimized in such a way that the mean hydrogen concentration in the containment dome is less than about 10 % H2 vol., i.e. below the upper flammability limit, so that DDT events with self-ignition are normally not to be expected. In this case, global detonations would be very unlikely as a result of experimental observation with direct ignition of detonations (e.g. SNL tests). However, non-uniform mixing could cause local explosions. Therefore, the containment should be gas-tight and capable of withstanding combustion loads resulting from the complete combustion pressure (constant volume pressure) for the wet containment atmosphere in the order of the static design pressure of about 10 bar. As a result, such a passive hydrogen safety concept would limit maximal possible combustion loads under severe accident conditions in a credible and convincing manner for a far-reaching consensus from a generic safety point of view.

Protection: For fire protection and explosion prevention, besides N2/CO2 inertization several new mixing media have been developed more recently and are already being used in industrial safety, public buildings, and high-tech equipment to mostly physically suppress fires or explosions. Inertization is a neutralization of oxygen by partial or total inerting. Modern examples are FM-200, ARGONITE, INERGEN, MICROSPRAYS, etc. The effectiveness depends on the special plant design and the specific requirements. For example, FM-200 is a very effective mixing medium with fast reaction for fire extinction in several application classes, which would be safe for personal and has been environmentally proven as well as tested by regulatory authorities. Modern fire protection and explosion prevention systems could be designed with inherent safety features as pre-, partial- or post-inertization in technical systems, using fast

34

CFD with HPC

active or passive injection, which should be also considered for nuclear safety and hydrogen technology, especially for the combination of nuclear and chemical power plants (e.g. HTR type with H2 production). Finally, microwater spray systems can effectively suppress fires and explosions depending on the droplet size, as experimental observations have shown. Last but not least, water is relatively cheap and normally available in sufficient quantities. Presently, we studying recent fire and explosion control systems for more inherent safety (i.e. a new approach in contrast to risk reduction as acceptably low) in further project activities.

Conclusion: H2 control with recombiners and igniters (i.e. dual concept) has special probabilistic failure rates and specific deterministic weak points initiating unlikely fires or explosions. Self-ignition sources must be avoided as far as possible, according to the basic explosion protection rules. The major problem is that the dual system is not absolutely safety-orientated, which is an underlying principle in nuclear safety. Therefore, it is not quite clear whether the H2 risk is really reduced by the dual concept. In addition, the dual concept with catalytic recombiners is always a slow-acting system (in about 24 hours) and it is not very smart to allow the distribution of a sensitive mixture and to wait for deliberate ignition thus producing a fire or even an explosion with uncertain consequences which are more or less unpredictable. Therefore, the dual concept is NOT an ideal solution to be recommended in nuclear reactor containments for severe accident conditions without critical review and additional safety measures. Small- scale laboratory tests and optimistic numerical simulations with simple lumped parameter codes or simplified CFD simulations on coarse grid levels are not very representative of the real accident scenario with regard to the completeness of the accident sequences and scaling uncertainties. As a consequense, appropriate hydrogen control systems with active and/or passive safety devices should be

35

CFD with HPC

further established, preferably in combination WITH dilution/inertization measures using modern mixing media for fire protection and explosion prevention, to handle the hydrogen safety problem for technical systems in a more suitable, reliable and inherently safe manner.

Recommendations: Based on our R&D work performed in the scope of Part 2 of this HGF project for hydrogen safety studies of technical systems, the following issues should be investigated in more detail with regard to subsequent project activities:

Probabilistic safety analysis: We recommend performing a probabilistic

safety analysis based on fault tree and event tree techniques to evaluate the failure

probability and weak points of recombiners and/or igniters under severe reactor accident conditions in comparison to the estimation of the reliability of modern dilution or inerting systems providing a scientific-technically proven basis with quantitative and qualitative assessments of the efficiency of hydrogen control systems. Probabilistic and deterministic safety studies are therefore proposed using objective methodologies as far as possible, including worst case considerations. The results should be used to compare, optimize and harmonize hydrogen control systems. which are under consideration for various applications to minimize the probability of a hydrogen risk.

Fluid dynamic safety analysis: Concerning deterministic safety analysis, we recommend performing a detailed uncertainty analysis for the prediction of hydrogen distribution and combustion modes under severe reactor accident conditions based on systematic parameter and sensitivity studies, including scaling effects (e.g. turbulent combustion) regarding the completeness of the accident scenario. To estimate the safety margin of enclosures for combustion

36

CFD with HPC

loads with the resulting uncertainty band, it is proposed that best-estimate as well as conservative assumptions should be used. For this reason, a huge number of large-scale simulations will be necessary, taking into account coarse to fine grid techniques and modern CFD flow solver algorithms, which should be preferably executed with high-performance computing and parallel processing to obtain a sufficiently high numerical resolution. Advanced CFD methods with supercomputing are already in use for various applications worldwide. Also pre-, partial- and post-inertization systems with modern mixing media should be studied in research and practice for fire protection and explosion prevention with relevance to industrial safety to reduce the accident consequences inside the plant with a hydrogen risk.

we

recommend that structure mechanical safety analysis should be performed for the resulting combustion loads (temperature and pressure-impulse history) with uncertainties under severe reactor accident conditions of safety equipment and safety enclosures, using finite element (FE) codes. It is suggested that appropriate CFD codes should be coupled with modern FE codes, which should both be executed in the massively parallel mode using supercomputing. Nowadays, the software and hardware infrastructure has been made available for improved safety studies of nuclear and conventional systems to limit hazard consequences outside plants with a hydrogen risk.

Structure

mechanical

safety

analysis:

Regarding

the

safety

margin,

Comments: The working period of Part2 of the HGF project on hydrogen safety was from 1998 to 2001. During this time, two projects were performed, namely the EU-H2DDT project together with the HGF-H2 project, which were executed by the same CFD working group. The EU-H2DDT project was completed with a final report in 2000. For the HGF-H2 project, a manpower of

37

CFD with HPC

2.5 PJ/a was required. During the following working period of the HGF-H2 project, the ISR1 was reorganized at FZ Jülich, which constrained the project work of Part 2. The reduced working group (1-2 PJ/a) was supplemented by a guest scientist (8 ma mo) and a post-doc (1.5 PJ) so that the project team was now, in principle, capable of undertaking the essential working tasks with some modifications and restrictions. The project work of Part 2 was involved in another FZJ project for CFD simulations with CRAY supercomputing so that we received the computer resources for the T90 (50 CPU-h/mo) and for the T3E (2500 CPU- h/mo), which were sufficient for test cases, but limited for large-scale simulations. Hence, we performed large-scale simulations in the order of 1 to 10 million grid points with maximal processor nodes and an effective performance in the MPP mode. For the scientific-technical results obtained in Part 2, a special application plan for further R&D work in research and practice was proposed in the field of fire protection and explosion prevention for technical systems, e.g. for hydrogen technology, to continue and apply the findings in a specific project group for applied computational fluid dynamics (ACFD) with high-performance supercomputing clusters (HPSC) in FZ Jülich (s. Tab. 2, 3). In this context, the modern field code cluster (MFCC) developed for supercomputing, networking and clustering in Europe forms the basis of two recently accepted EC proposals, i.e. THERMO-NET (nuclear safety) and EXPRO (industrial safety), s. Tab. 4. The overall project aims are to proceed with the successful project work using scientific-technical high-performance supercomputing for innovative numerical simulations of non/reacting flows in complex geometries with modern CFD flow solvers for the next generation of codes, ranging from the GGG computing level towards the teraflops computer class. Main results of the EC/HGF-CFD/DDT project work have been published in international journals and conference papers, e.g. Ref. [20 - 27].

38

CFD with HPC

Tab. 2: Recent R&D Work of the ACFD Project Group

Scientific-Technical Flow and Combustion Research

Higher Order Turbulence and Combustion Models (LES-EDC)

Fine Structure Models with Massively Parallel Processing (MPP)

Basic Codes: CFX-LES and AIXCO-VLES with AEA and RWTH

Research Codes: D2UNS-LES and SHOCKIN-CHEMKIN with INCAS

Basic Scientific Research for Fire and Explosion Protection

Multi-Fluid Models with Multi-Phase Flows and Particle Transport

Complex Geometries in Near- and Far-Reaching Fields with Dust Explosion

Basic Codes: CFX-5 and IFSAS-3D with AEA and CDL

Complex Load Response in Safety Enclosure and Environment

Fluid-Structure Interaction of Components (IFSAS Code)

Explosion Loads Inside and Outside Safety Concepts (DYNA Code)

Industrial Safety of Innovative Energy Systems

Plant Safety: Deflagration-to-Detonation (DDT) Inside and Outside Safety Buildings for Conventional und Nuclear Energy Systems

Environmental Safety: Fire/Explosion Protection for Plant Designs (H2)

Scientific Relevance and Practical Competence Centre of Excellence

Supercomputing with Parallel Processing (CRAY Complex in FZJ)

Research Clustering (ERCOFTAC Center, CFD Society Canada)

Application Competence (EC Research, Siemens, Opel, BMW)

Funding Resources: EC Projects and Proposals (THERMO-NET, EXPRO)

39

CFD with HPC

Tab. 3: Special ACFD Project Group with Structure

Applied Computer Fluid Dynamics (ACFD) in Research and Practice with Supercomputing Scientific-Technical Flow and Combustion Research

Computer Fluid Dynamics (CFD)

Non/Reactive Flow Simulations Intern. Cluster Projects: EC/HGF/DFG/BMBF Basic Scientific Research for Fire and Explosion Protection

Fluid Structure Dynamics (FSD)

Interaction and Component Behaviour Detailed Deformation Analysis and Load Response Complex Load Calculations in Safety Enclosures

Application Fields (AF)

Advanced Safety Analysis of the Hydrogen Behaviour in Technical Plant Systems and the Environment Industrial Safety of Innovative Energy Systems

Cooperation (EC)

Universities, Research Centres, Industry RWTH Aachen, FZ Karlsruhe, AEA Technology, CDL Canada CEA Grenoble, UPM Madrid, NNC Knutsford, INCAS Bucharest, Scientific Relevance and Practical Competence

Collaboration Plan (CP)

Education, Training and Competence Know-how Transfer and Spin-off/Synergy Effects Federal Prevention Measures for Accident and Climate Protection

Organization Plan (OP)

FZJ-NIC Competence Group, IWV Project H2 Fuel Cells Interdisciplinary CFD Collaboration with Supercomputing Internal/External Coordination of Computer Fluid Dynamics

40

CFD with HPC

Tab.4: EC/HGF-CFD/DDT Related Project Activities

Project

Title

Content

Partner

EC Project: Next CFD Codes Generation and Methodology

 

H2 DDT

Pilot Project

FZJ, et al. Funding: DM 0.150 mio. FUB, FZJ, et al. Funding: DM 0.250 mio. UC3, FZJ, et al. Required: DM 0.400 mio. CEA, FZJ, et al. Required: DM 0.250 mio.

Status:

Finished 1995

H2 DDT

Main Project

Status:

Finished 1999

EXPRO

CFD-LES

Status:

Started 2001

Thermo-Net

Networking

Status:

Started 2001

 

EC Brussels

Project: HPC Parallelization Strategies and Performance

 

CFD-DDT

Supercomputing

VSR, ISR, ZAM T90/T3E Account

Status:

Since 1995

 

FZ Jülich

HGF Project: CFD Software Improvement and Application

 

H2 Safety

Optimization

FZK, FZJ Required: DM 1.5 mio.

Status:

Finished 2001

 

HGF Bonn

41

CFD with HPC

Conclusion and Outlook

In general, fire and explosion protection is an important safety issue in most areas of the engineering. For instance, the prediction of hydrogen explosion loads in safety enclosures under severe accident conditions is relevant for nuclear reactor containments and hydrogen demonstration plants with fuel cells or hydrogen-powered vehicles. Contextually, special verification tests of reactive field codes and validation tests of calculations have been performed for specific reacting flows in complex geometries. The turbulent combustion modes considered are related to deflagration, transition (DDT), and detonation in explosive hydrogen-air mixtures. Hereby, experimental and numerical results were compared on integral- and laboratory scales, obtaining a high temporal and spatial resolution of complex flows in the numerical simulations. Therefore, a modern field code cluster (MFCC) was established with new versions of the reactive Navier-Stokes/Euler flow solver codes: D3UNS, CFX, AIXCO, for fast flames (deflagrations) and DET, IFSAS, SHOCKIN for rapid flames (detonations), including vector and parallel processing capabilities. For benchmark calculations, most of these codes have been ported successfully to the heterogeneous CRAY-J90/T90/T3E supercomputer complex at FZ Jülich, running in the sequential, moderate parallel or massively parallel mode with reasonable performances and good speedup factors. As a result, the reduction of the computing time per allocated processor allows appropriate mesh refinement with unstructured or adaptive grids and robust algebraic multi-grid solvers together with higher order turbulence- and combustion models of multi-fluid flows and chemical source terms. Consequently, we have provided RANS/VLES/LES/ flow solver algorithms in combination with the flamelet concept (FC), the eddy dissipation model (EDM)

42

CFD with HPC

or probability density functions (PDF). In the scope of joint research project activities funded by the HGF in Bonn and the EC in Brussels, academic and commercial software systems (e.g. AIXCO and CFX-5) have been developed for massively parallel processing (MPP) on the CRAY-T3E with maximum number of processor nodes (600 MHz) for high-performance supercomputing (HPSC), based on domain decomposition with message passing tools, using MPI and PVM or HPF routines. Hence, we use multi-block un/structured pre-processing (PATRAN/BUILD) as well as online post-processing (AVS/FIELDVIEW) for meta-supercomputing clusters with huge computer resources, solving the in- put/output bottleneck of large computing domains within practicable computing times. In summary, the applied computational fluid dynamics (ACFD) with supercomputing makes it possible to explore the scientific-technical aspects of fire prevention and explosion protection in more detail for realistic safety enclosures of nuclear and conventional energy systems, especially for the hydrogen safety technology as a future energy option. Last but not least, the interdisciplinary collaboration of centers of excellence via high-performance computer networking (HPCN) contributes to enhance innovative trends of turbulent combustion simulations in research and practice for engineering flows in complex geometries. In this context, we have established a modern field code cluster with hybrid grids, higher order turbulence and combustion models together with high- performance computations using supercomputer capabilities at FZJ for detailed hydrogen safety studies. A major effort and significant progress was made in introducing large eddy simulations (LES) relevant to industrial safety with the support of further EC-funded project activities.

43

References

CFD with HPC

[1] Technical Aspects of Hydrogen Control and Combustion in Severe Light- Water Reactor Accidents, Ed. by N.C. Rasmussen, et al., National Academy Press, Washington, 1987. [2] Hydrogen in Water-Cooled Nuclear Power Reactors, Ed. By EC Brussels and IAEA Vienna, EUR 14037 EN, 1991. [3] Hydrogen Behaviour and Mitigation in Water-Cooled Nuclear Power Reactors, Ed. By E.D. Loggia, EC Brussels, EUR 14039 EN, 1992. [4] Computational Fluid Dynamics on Parallel Systems, Ed. by S. Wagner, Notes on Numerical Fluid Mechanics, Volume 50, Vieweg Verlag, 1993. [5] Large Eddy Simulations of Complex Engineering and Geophysical Flows, Ed. by S.A. Orszag, et al., Cambridge University Press, 1993. [6] Turbulent Reacting Flows, Ed. by F.A. Williams, et al., Academic Press, London, 1994. [7] Solarwasserstoff – Energieträger der Zukunft, Ed. by P. Hennicke, Wuppertal Texte, Birkhäuser Verlag, Berlin, 1995. [8] Compilers for Parallel Computers, Ed. by M. Gerndt, Vol. 21, Forschungszentrum Jülich, 1996. [9] Wissenschaftliche Grundlagen des Brand- und Explosionsschutzes, Ed. by S. Bussenius, Hohlhammer Verlag, Stuttgart, 1996. [10] EU Research on Severe Accidents, Ed. by G. Van Goethem, et al., Symposium FISA 95-97-99-01, EUR 16896 EN, 1996. [11] Combustion, Ed. by I. Glassman, Academic Press, London, 1996. [12] Fluid Dynamics and Transport of Droplets and Sprays, Ed. by W.A. Sirignano, Cambridge University Press, 1999.

44

CFD with HPC

[13] Supercomputing for Nuclear Applications, Ed. by H. Kaburaki, Intern. Conference SNA-2000, Tokyo, Sept. 2000. [14] Process Safety and Industrial Explosion Protection, Intern. ESMG

Symposium 2001, Nuremberg, March 2001. [15] Numerische Berechnung turbulenter Strömungen in Forschung und Praxis, Ed. by W. Rodi, University of Karlsruhe, Hochschulkurs 1998. [16] CFD-2001, Annual Conference of the CFD Society Canada, Waterloo, June

2001.

[17] Computational Physics, Intern. Conference, Aachen, Sept. 2001. [18] N. Peters: Turbulent Combustion, Edition, RWTH-ITM, Aachen, Oct. 1999. [19] M. Herrmann: Numerical Simulation of Premixed Turbulent Combustion Based on a Level Set Flamelet Model, PhD Thesis, RWTH Aachen, 2001. [20] R. Klein, W. Rehm (Eds.): Models and Criteria for Prediction of Deflagration-to-Detonation Transition in Hydrogen-Air-Steam Systems under Severe Accident Conditions, Vol. 9, Forschungszentrum Jülich, 2000. [21] W. Rehm, M. Gerndt, W. Jahn, R. Vogelsang, B. Binninger, M. Herrmann, H. Olivier, M. Weber: Reactive Flow Simulations in Complex Geometries with High-Performance Supercomputing, Proc. Intern Conf. on Supercomputing for Nuclear Application, SNA-2000, Tokyo, Sept. 2000. [22] W. Rehm: Angewandte Fluiddynamik mit High-Performance Supercomputing, ATW Vol. 45 (2000), No. 10, Oct. 2000, Inforum Bonn. [23] J. Eyink, W. Rehm, et al.: Research on Hydrogen Risk Mitigation Resulting from Hypothetical Severe Accident, ATW Vol. 45 (2000), No. 12, Dec. 2000, Inforum Bonn. [24] W. Rehm, R. Klein: Evaluation of Deflagration-to-Detonation Transition in Hydrogen-Air Mixtures: DDT-Related Experimental and Numerical Studies, ATW Vol. 46 (2001), No. 1, Jan. 2000, Inforum Bonn.

45

CFD with HPC

[25] W. Rehm, C. Nae, B. Wang, W. Jahn: Applied Computational Fluid Dynamics (ACFD) Using High-Performance Supercomputing Clusters (HPSC),

Proc. Intern. Conf. on Nuclear Reactor Technology, KTG-Forum, Dresden, May

2001.

[26] W. Rehm, C. Nae, W. Jahn, , R. Vogelsang, B. Wang: CFD Simulations of

Turbulent Reactive Flows with Supercomputing for Hydrogen Safety, Proc. Intern. Conf. on Computational Physics, CCP-2001, Aachen, Sept. 2001. [27] W. Rehm, et al.: Advanced CFD Methods for Hydrogen Safety Technology with Supercomputing - Software Optimization and Special Application with Improvement, Validation and Performance, Report FZJ-ISR1-8/2001, Jülich, August 2001.

46

Acknowledgments

CFD with HPC

This R&D work was performed within the scope of European projects on H2-DDT (Model and Criteria Verification), as well as a FZJ-VSR project on supercomputing (Reactive Flow Simulations) and of a German HGF-H2 project (CFD Optimization and Application).

We acknowledge the support of the FZJ (VS, ISR and ZAM). The author is grateful to all project participants and in particular to the EC in Brussels (Dr. G. Van Goethem), FU Berlin (Prof. R. Klein), FZ Karlsruhe (Dr. W. Breitung), and RU-Bochum (Prof. H. Unger) for their cooperation, especially to RWTH- ITM/SWL Aachen (Prof. N. Peters, Dr. B. Binninger, Dr. H. Herrmann, Prof. H. Olivier) as well as to AEA Technology (Dr. G. Scheuerer, Dr. F. Unger, Dr. Kuntz, Dr. Forkel) for their services, including CDL in Canada (Dr. P. Thibault). We would like to thank everybody involved at FZJ (Dr. M. Gerndt, DI W. Jahn, Dr. C. Nae, Dr. R. Vogelsang, Dr. B. Wang) for their valuable contributions, including the administration and infrastucture at FZ Jülich. Finally, we also thank the ESMG group and KIDDE-DEUGRA GmbH for their helpful information and material.

Last but not least, I would like to thank cordially my wife Leni-Keil-Rehm for her permant patience and great interest in preparing the project work with the documentation for publication, especially all other persons involved for many productive discussions about further R&D developments.

47

CFD with HPC

ANNEX: Reactive Field Codes with Supercomputing

Applied Computational Fluid Dynamics (ACFD) Using High-Performance Supercomputing Clusters (HPSC)

W. Rehm, C. Nae, B. Wang, W. Jahn, FZ Jülich, Germany

Objectives: The overall project aim is to develop modern computational fluid dynamic methods for hydrogen combustion studies using high-performance supercomputing clusters with field code optimization and special application, including refinement, validation and performance analysis. The modern field code cluster developed with parallel processing capability aims to obtain a high numerical resolution of non/reacting flows in complex geometries. Special application are related to hydrogen safety and explosion loads under severe accident conditions for nuclear and conventional energy systems, including transition from deflagration to detonation processes (DDT), s. Tab. 1 and Figs. 1 to 12, as follows.

Summary: We have developed a modern field code cluster (MFCC) with high-performance supercomputing (HPSC) using the CRAY computer complex (T90, T3E, J90) at FZ Jülich with parallel vector processing (PVP) and massively parallel processing (MPP). The MFCC software system consists of Euler / Navier-Stokes flow solvers: D3UNS, CFX, AIXCO, SHOCKIN, IFSAS and DET, which are suitable for numerically simulating slow, fast, and rapid hydrogen flames with resulting combustion loads inside and outside of safety enclosures. The MFCC software system with supercomputing is used as an advanced research and analysis tool for fire and explosion protection of innovative energy systems with respect to nuclear reactor safety and conventional industrial safety (e.g. for hydrogen technology as a future energy option to limit the increasing climate problem). The R&D work for hydrogen safety at FZ Jülich is performed in the framework of joint projects supported by the EC in Brussels and the HGF in Bonn, whose cooperations are gratefully acknowledged.

48

CFD with HPC

(1) Main Features of the Modern Field Code Cluster

Global formulation for the Navier-Stokes equations in the conservative variables, including multifluid non/reacting formulation;

RANS formulation and spatial filtering for LES using macro redefinitions;

Implementation of various turbulence models with special wall/near wall treatment;

Complex boundary condition treatment for real industrial applications;

Dedicated extensions for special flows (jets, mixing and share layers, free boundaries, flames);

Structured/unstructured domain discretization based on multiblocking and domain decomposition using static/dynamic load balancing;

Robust mixed finite volume/finite element spatial integration techniques;

Implicit/explicit time integration techniques using convergence acceleration and adaptive criteria;

High level of parallelization and increased efficiency on MPP;

Integrated solver and pre/postprocessing tools for mesh adaptation and refinement using flowfield parameters in a closed loop algorithm.

Important CFD criteria and tools implemented in the MFCC for the numerical analysis and solution assessment (e.g. D3UNS code):

Controlled level of numerical diffusion and/or dissipation of the numerical scheme;

True estimator for numerical modeling of viscosity using a global indicator and distribution analysis;

Error estimation based on static and/or dynamic definitions using flowfield parameters;

Wall and near wall assessment of the numerical solution based on complex spatial/time averaged dedicated analysis;

Special evaluation and validation of the numerical solution on the boundaries based on the method of characteristics;

Complex time/space averaged surface analysis and correlation with experimental data;

Complex time dependent data analysis;

Complex and detailed postprocessing tools for solution visualization;

High level of data exchange with other CFD code packages and general CAD tools;

49

CFD with HPC

(2) Basic Field Code Equations of the Continuum Physics

∂ ∂ ( ~ + ◊ u j ) = 0 ∂ t ∂ x
(
~
+
◊ u
j )
= 0
∂ t
∂ x
j
(
~
)
(
)
2
~
u
+
◊ u ~
◊ u ~
=
+
(
+
)
2
◊ S ~
◊ S
i
i
j
t
ij
ii
ij
t
∂ x
∂ x
x
3
j
i
j
(
~
)
[(
)
]
2
~
e
+
◊ e ~
+
◊ u ~
=
u ~
◊ ◊
2
◊ S ~
◊ S
+
(
k
+
)
k
j
j
ij
3 ii
ij
t
t
∂ x
∂ x
∂ x
∂ x
j
i
j
j
(
~
)
(
)
Y
+
◊ u
=
◊ ∂ Y ~
+
(x t)
,
j ◊ Y ~
Y
,
j
Y
∂ t
∂ x
∂ x
Sc
x
j
j
Y
j
where :
C
p
=
R
1
k =
=
p
D
ii
Pr
3
1
e ~ =
C
+
u ~ u ~
~
1
1
C
p
t
v
i
i
=
T
D
2
=
ii k
t
2
C
v
Pr
t

and :

M = ◊ u ◊ u ◊ u ~ ◊ u ~ = T +
M
= ◊ u
◊ u
◊ u ~
◊ u ~
=
T
+
D
~
1
~
~
ij
i
j
i
j
ij
ij
=
u
+
u
S ij
i
j
2
x
x
1
~
j
i
T
=
M
◊ M
=
◊ P
ij
ij
kk
ij
t
ij
3
~
~
2
~
P
=
2
◊ S
◊ S
1
ij
ij
ij
kk
2
3
D
=
◊ ◊ M
◊ p
ij
SGS
ij
3
with:
F U
(
,n
)(
+
F U
,n
)
ROE
(
U ,U
,n
)
I
IJ
J
IJ
ROE
=
◊ d
(
U ,U
,n
)
I
J
IJ
solver
I
J
IJ
2

r

(

,

NUM

SGS

,

=

=

U I U

IJ

U

U

J JI

=

=

U

I

U

J

+

{(
2

1

1

1 {(
2

1

)

()

U

C

)

()

U

C

+

+

and:

C

)

=

NUM

1 1 1 { ( 2 1 ) () ◊ U C ) () ◊ U

,

1

V

S

(

u , v

,

SGS

w

)

(

u

,

v

,

w

)

T

upwind

t

2

S

ij

2 S

w ) ( ◊ u , v , w ) T upwind t ◊ 2 ◊

3 kk

ij

2

(

=

C

50

() }

U

D

I

n

IJ

() }

U

D

J

n

IJ

=

S

1

V

)

2

( ◊ u ◊ + v ◊ u v ~ 2 ◊ S ◊ 2
(
u
+
v
u
v
~
2
S
2
◊ S
ij
3

+

w

S

kk

w

ij

)

upwind

2

CFD with HPC

(3) Reactive Field Code Equations of Fluid Dynamics

The governing continuum equations of turbulent reactive fluid flows are written in the

following two-dimensional conservative form:

U ++ FG += HR ++ S Q tx y xy u v v 2
U
++ FG
+= HR
++ S
Q
tx
y
xy
u
v
v
2
u
u
+ p
vu
vu
i
2
2
where
U =
v
,
F =
uv
,
G =
v
+
p
,
H =
v
,
y
E
uE
+
up
vE
+
vp
vE
vp
+
Y
uY
vY
vY
0
0
0
Y
1
1
0
xx
xy
Y
2
R =
, S =
,
Q =
0
,
Y =
xy
yy
2
,
=
,
uv +
+
T
uv +
+
T
0
xx
xy
x
xy
yy
y
Y
N
N
0
0
and
=+()2
u
+
v
,
==
(u
+ v
),
= (
+ 2 )v
+
u
.
xx
x
y
xy
yx
y
x
yy
y
x
Here p, , T, E, u and v denote pressure, density, temperature, total specific energy and

velocity vector components in the Cartesian coordinates, respectively. Y j and j represent the

mass fraction and production rate of jth species. and indicate coefficients of viscosity, and

the thermal conductivity. In the above equations i = 0 and i = 1 correspond to a planar and

an axis-symmetric flow. For the application of a detailed kinetic scheme, the production rate (

j ) of the jth species is given by:

where

r

jm

and

l

jm

m

=

Nr

m =1

(

r

jm

l

jm

) (

fm

bm

)

are the coefficients of the jth species in the mth reaction on the right- and

left-hand side, respectively. The forward and backward reaction rates of the mth reaction, fm

and bm , must be defined by the user.

51

CFD with HPC

The total specific energy is expressed as:

E = e +

1

2

(

u

2

+ v

2

) ,

where e denotes the specific internal energy, which can be calculated according to the

thermodynamic database.

In case of utilising the two-step kinetic model,

Y = and = 1 2
Y
=
and
=
1
2

.

and

(0 <

and

< 1) represent the remainder of the progress of the induction reaction and that of the

exothermal reaction. The increasing rates of

and are expressed as:

 

=

d

 

k

 

E 1 ),

 
 

1

dt

=

1

exp(

 

RT

 

=

d

k

2

(

2

exp(

 

E

2

)

(1

 

)

2

exp(

E

2

+

q

 

=

p

       

2

dt

2

and the total specific energy is determined by

RT

       

RT

 
 

E =

 

p

+

 

+

 

2

+

 

2

)

 

(

 

1)

 

q

 

1 (
2

u

v

     

where indicate the specific heat ratio.

))

The governing equations can be resolved, e.g. by a second-order extension of Godunov’s

approach, including the Piecewise Linear Method. The details of the applied numerical

approaches for resolving the gas dynamic problem are described in detail by /Yu/. For the

calculations of chemical kinetic and thermodynamic terms, the subroutines of a chemical

kinetic package can be applied. A detailed description of the package is given by /Kee/. In this

context, see for additional reading the following references:

/Yu/: Q. Yu: Experimental and Numerical Study of Unsteady Shock and Detonation Waves,

Doctoral thesis, Stoßwellenlabor, RWTH Aachen, 1996.

/Kee/: R.J. Kee, F.M. Rupley, J.A. Miller CHEMKIN-II: a FORTRAN Chemical Kinetics

Package for the Analysis of Chemical Kinetics, SAND89-8009, 1989.

/B. Wang, W. Rehm/: Numerical Studies of Shock Reflection and Ignition, Intern. Colloqium

on Dynamics of Explosions ICDERS 2001, Seattle, USA, Aug. 2001.

52

CFD with HPC

Tab. 1: CFD/DDT-related computer models used for hydrogen safety studies at FZJ.

Distribution Mode (Concentration): Non-reactive Na-St. Solver D3UNS/MPI Unstructured flow solver with classical and LES turbulence modelling

Deflagration Mode (Flame Acceleration): Reactive Na-St Solver CFX-3D/MPI Higher Order Turbulence and Combustion Models with Unstructured Grids

DDT Mode (Auto-Ignition): Reactive Na-St Solver AIXCO-2D/MPI Flame Tracking/Shock Wave Capturing Schemes and Reduced Reactions

DDT Mode (Shock-Ignition): Reactive Euler Solver SHOCKIN-2D/AG Unstructured Flow Solver with Reduced and Detailed Chemistry

Detonation Mode (Propagation): Reactive Euler Solver DET-2D/HPF Hydrodynamics with Chemical Source Term and One Step Reaction Kinetics

Fluid-Structure Analysis (Loads): Reactive Euler Solver IFSAS-3D/AG Shock/Detonation Ignition Models with Adaptive Grids and Explosion Loads

Structure Mechanics (Response): Finite Element Code DYNA-3D/PVM for static and dynamic deformations of safety enclosures (CONDAT)

Modern CFD Methodology in Fluid Mechanics Deflagration PVP System MPP System Detonation HPSC- Clusters CFX-PVM
Modern CFD Methodology in Fluid Mechanics
Deflagration
PVP System
MPP System
Detonation
HPSC- Clusters
CFX-PVM
T90
DET-HPF
T3E-600
T3E-1200
UNICORE Interface
Reactive
10 Proc.
Reactive
512
Proc.
512
Proc.
8 GBytes
Navier-Stokes
64 GBytes
262
GBytes
Euler
18 GFLOPS
300
GFLOPS
614
GFLOPS
Solver
Solver
FZJ-T3E
(1) State
GMD-SP2
MFCC: D2UNS, CFX, COM, AIXCO, DET, DYNA
DEC-WS
FUB-T3E
J90
HPSC- ERCO-DDT
Pre- and Post-
16 Proc.
SHOCKIN
Semi-Implicit
Processing
8
GBytes
Flame-Shock
IFSAS
Networking
3
GFLOPS
Na-St. Solver
BUILD, FIEDVIEW
RUS-T3E
Fluid Structure Dynamics
File Server
Transition
(2) Plan
Using FZJ Projects: UNICORE (Access), KOJAK (Performance), and GIGABIT TESTBED
Meta-Computing

Fig. 1: Modern field code cluster with high-performance supercomputing on the CRAY computer complex at FZJ for the simulation of reactive flows in complex geometries (e.g. CFX-PVM and AXICO-MPI on the T3E with massively parallel processing).

53

CFD with HPC

D ET-2D /T3E/H PF H igh-S pee d F low w ith 500.000 C ells
D ET-2D /T3E/H PF
H igh-S pee d F low w ith 500.000 C ells
250 250
T
200 200
i
m
150 150
e
100 100
50 50
s
0 0
10
20
30
40
60
80
100
Num ber of Processors

Fig. 2a: Parallel performance of the reactive Euler solver DET-2D/HPF on the CRAY-T3E for a detonation test case in a model geometry with obstacles using 500.000 cells.

CFX-3D/T3E/PVM

Channel Flow 700.000 Cells

800 800 600 600 400 400 200 200 0 0 32 64 96 Time (s)
800
800
600
600
400
400
200
200
0
0
32
64
96
Time (s)

Processor Nodes

Fig. 2b: Parallel performance of the reactive Na-St. solver CFX-3D/PVM on the CRAY-T3E for a complex flow test case in a tube geometry using 700.000 cells.

A IX C O - 2 D /T 3 E /M P I

B e nc hm a rkte s t 400 400 300 300 200 200 100
B e nc hm
a rkte s t
400
400
300
300
200
200
100
100
0
0
8
16
24
48
T im e
(s )

N um ber of P rocessors

Fig. 2c: Parallel performance of the reactive Na-St. solver AIXCO-2D/MPI on the CRAY- T3E for a deflagration test case in a channel geometry with obstacles using 50.000 cells.

54

CFD with HPC

(4) Non-Reacting Flows (RANS/LES Tests)

CFD with HPC (4) Non-Reacting Flows (RANS/LES Tests) Fig. 3: Unsteady iso-vortex representation using RANS code

Fig. 3: Unsteady iso-vortex representation using RANS code and k-eps turbulence model (Mach = 0.2, Reynolds = 22.000)

´
´

Fig. 4: Domain partitioning using load balancing for MPP

´ Fig. 4: Domain partitioning using load balancing for MPP Fig. 5: Iso-temperature profile for a

Fig. 5: Iso-temperature profile for a 2D jet using LES turbulence model (Mach = 0.10, Reynolds = 5.000)

55

CFD with HPC

CFD with HPC F i g . 6 a : Instantaneous LES t / l ratio

Fig. 6a: Instantaneous LES t /

l ratio distribution (Mach = 0.15, Reynolds = 22.000)

r c rite ria distribution 1.E+06 beta = 0.05 beta = 0.10 1.E+05 1.E+04 1.E+03
r c rite ria distribution
1.E+06
beta = 0.05
beta = 0.10
1.E+05
1.E+04
1.E+03
1.E+02
1.E+01
1.E+00
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
r

Fig. 6b: Global statistics for the r criteria distribution for compressible LES simulations

56

CFD with HPC

CFD with HPC Fig. 7: Complex analysis and postprocessing for a 3D LES test case with

Fig. 7: Complex analysis and postprocessing for a 3D LES test case with obstacles

and postprocessing for a 3D LES test case with obstacles Fig. 8: Instantaneous iso-temperature distribution for

Fig. 8: Instantaneous iso-temperature distribution for accident simulations in open air

57

CFD with HPC

(5) Reacting Flows (CFD/DDT Tests)

Validation of the SHOCKIN code was performed for experimental results with a series of

schlieren pictures which were obtained in a lean H 2 -air mixture in a square shock tube with a

long gap in the flat end wall. Unsteady high temperature and density gradients are induced by

shock reflections and by the interaction between a shock wave and a rarefaction fan. The

ignition in such flow fields is an ideal example of validating the reaction model for a wide range

of applications.

20 s
20
s
40 s
40
s
60 s
60
s
80 s
80
s

Fig. 9: Resolved experi-mental schlieren pictures of shock-induced ignition at a gap entrance (mixture: 15%H 2 +85%air; initial pressure: 4 kPa; initial temperature: 293 K; incident shock Mach no.: 2.87; gap: hight 10 mm).

Upper wall Gap End wall 20 s Lower wall
Upper wall
Gap
End wall
20
s
Lower wall

Shock waves

40 s
40
s

Reaction wave

Reaction wave

60 s
60
s
80 s
80
s

Explosion wave

Detonation wave
Detonation wave

Fig. 10: Numerically simulated schlieren pictures produced with the reactive fluid dynamics combustion code SHOCKIN (conditions and time step correspond to Fig. 9). The figure shows the self-ignition process and the formation of detonation.

58

CFD with HPC

The SHOCKIN code was applied to a containment geometry for DDT studies using a two- step reaction model where a high pressure region is located at the lower corners of the dome.

pressure region is located at the lower corners of the dome. Fig: 11 : Instantaneous temperature

Fig: 11 : Instantaneous temperature and pressure contours inside the dome (Mach = 2.85; mixture: 15%H 2 +85%air; p o : 0.1 MPa; T o : 293 K)

(6) Hydrogen Distribution (CFD/BMC Tests)

CFX calculation for a helium injection and distribution test in a multi-compartment geometry (Batelle Modell Containment). BMC description :

Volume: 600m³

Air:

Helium Injection:

Diameter: 95mm Speed : 42m/s

Rate:

Computational details :

Code : CFX with k-eps

Blocks : 91

Elements : 68.522

Points :

Code : CFX with k-eps Blocks : 91 Elements : 68.522 Points : Fig. 12a: BMC

Fig. 12a: BMC model geometry

1 bar, 304K

ca. 0.049 kg/s

99.883

59

CFD with HPC

CFD with HPC Figs. 12b: Light gas distribution during the first 220 seconds after injection. Figs.
CFD with HPC Figs. 12b: Light gas distribution during the first 220 seconds after injection. Figs.

Figs. 12b: Light gas distribution during the first 220 seconds after injection.

distribution during the first 220 seconds after injection. Figs. 12c: Light gas distribution during the first
distribution during the first 220 seconds after injection. Figs. 12c: Light gas distribution during the first

Figs. 12c: Light gas distribution during the first 512 seconds after injection.

References

/C. Nae/: Efficient LES using a Betta-Gamma Scheme and Wall Laws, International Conference on Computational Fluid Dynamics ICFD 2001, Oxford, U.K., 2001. /B. Wang, W. Jahn, W. Rehm/: Application of a CFD Code for Reactive Flows in H2-Air Mixtures, CFD 2001 Conference - A CFD Odyssey , Waterloo, Canada, 2001.

60

Nomenclature

CFD with HPC

a

Iteration coefficient

b

Pre-exponential factor

c

Model constant

c 0

Speed of sound

f

Fuel fraction

k

Turbulent kinetic energy

m lim

Minimal mixture fraction

n

Normal vector

p

Pressure

r

Reaction coefficient

s

l

Laminar flame velocity

s

t

Turbulent flame velocity

t

Time

u

Fluid velocity

u'

Velocity intensity

v j

Velocity vector

x

Space coordinate

AEA

AEA Technology GmbH

A

A

C

T

Combustion constant

Burning constant

BR

Block ratio

CFDS

CFD Society Canada

CDL

Combustion Dynamics Ltd.

C

Flow constant

D

Diffusion coefficient

Da

Damköhler number

DNS

Direct numerical simulation

E

Energy

EC

European Commission

EDC

Eddy dissipation concept

ERCOFTAC

European Research on Turbulent Flow and Combustion

61

CFD with HPC

F

FC

FZJ

FZK

FUB

G

ISR

ITM

Ka

M S

LES

Q

R

Re

RANS Reynolds-averaged numerical simulation

RWTH Rheinisch-Westfälische Technische Hochschule

S

SWL

S

S

T

Flux vector

Flamelet concept

Forschungszentrum Jülich

Forschungszentrum Karlsruhe

Freie Universität Berlin

G equation

Institute for Safety Research and Reactor Technology

Institute for Technical Mechanics

Karlovitz number

Shock Mach number

Large eddy simulation

Iteration variable

Gas constant

Reynolds number

Source term

Shock Wave Laboratory

Turbulent burning rate

Shock heating rate

Temperature

Ignition temperature

Velocity vector

University Carlos 3, Madrid

University Politecnica, Madrid

Volume

Progress variable

Central Institute for Applied Mathematics

Dissipation rate

Turbulent viscosity

T

C

T ig

Viscosity

f Schmidt-Prandtl No,

E /

Density

Time

Variable

Diffusion term

Volume fraction of hydrogen

62

Curriculum Vitae

CFD with HPC

Name:

Werner Rehm

Date and place of birth:

10 June 1942 in Nürnberg (Nuremberg)

Marital status:

Married to Leni Keil-Rehm

Nationality:

German

School qualification:

Abitur (University matriculation examination)

University education:

Weiden (Opf.), June 1964 Studies of physics (Diplom-Physiker)

Special field:

RWTH Aachen, October 1970 Solid state physics and reactor technology

Diploma work:

Numerical integration of heat equation

Professional career:

with special consideration of temperature-dependent heat conductivity Research scientist at FZJ Jülich from Jan. 1971

in the institutes: ZFR, ISF und ISR Position: Group and project leader of EC/FZJ/HGF projects

Special area:

Fluid dynamic safety analysis of technical systems:

Dissertation:

hydrogen safety, LWR and HTR with process heat PhD at RWTH Aachen, Dr. rer. nat., Nov. 1979

Working field:

on hypothetical accident behaviour of HTRs Computer fluid dynamics with hydrogen behaviour

Special interest:

in nuclear and conventional systems (H2 safety) Higher order turbulence and combustion models, ith high-performance supercomputing cluster (HPSC)

Membership of scientific associations:

ERCOFTAT

CFD Society Canada (CFD-SC), Reviewer (NED), international programme committee SNA-2003

pilot centre Germany West,

63