Table Of Content

Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 3. DATES COVERED 2003 2. REPORT TYPE 00-00-2003 to 00-00-2003 4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER NAVO MSRC Navigator. Spring 2003 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION Naval Oceanographic Office (NAVO),Major Shared Resource Center REPORT NUMBER (MSRC),1002 Balch Boulevard,Stennis Space Center,MS,39522 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S) 11. SPONSOR/MONITOR’S REPORT NUMBER(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited 13. SUPPLEMENTARY NOTES 14. ABSTRACT 15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF ABSTRACT OF PAGES RESPONSIBLE PERSON a. REPORT b. ABSTRACT c. THIS PAGE Same as 32 unclassified unclassified unclassified Report (SAR) Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18 The Director’s Corner Steve Adamec, NAVO MSRC Director Changing to Better Serve You In the past several months we have seen substantial change and progress here at the NAVO MSRC. We've brought on board our new MSRC technical services provider, Lockheed Martin (more in the article that follows on page 4), who brings substantial expertise and enthusiasm to their support of this MSRC and the DoD HPC Modernization Program (HPCMP). We're also completing several major Center enhancements, designated as Technology Insertion for Fiscal Year 2003 (TI-03), across several major technology areas within the MSRC. These include substantial upgrades to the IBM POWER4 HPC (MARCELLUS) system, the Remote Storage Facility (RSF), and the internal MSRC networking capability. When complete, these enhancements will provide almost 10 teraflops of aggregate peak computing capability with commensurately balanced storage and networking capabilities. This enormous computational capability will continue to enable unparalleled advances in the DoD science and technology areas served by the HPCMP. As I've mentioned in past issues of the Navigator, we recognize that it is critically important for us to redouble our efforts in assessing and implementing common user environments, practices, and tools within and across the Centers. Your individual and collective user feedback through the User Advocacy Group makes it clear that you consider this to be one of your highest priorities for us. In response, we've formed new internal teams whose primary goal is to strengthen our linkage and participation with the HPCMP Programming Environment and Training (PET) program elements that emphasize user environment, tools, and productivity. My staff and I look forward to seeing you in June at the 2003 HPCMP Users' Conference in Bellevue, Washington. As always, please take every opportunity to let us know how we can better serve you. Your feedback is critically important to us and to the HPCMP. ABOUT THE COVER: This image shows the temperature variable in a dataset created by a computational model run on the IBM POWER4 (MARCELLUS) in support of the Airborne Laser Challenge Project II. The data were visualized using Alias|Wavefront Maya 4.5 on a Windows 2000 workstation. A volumetric surface rendering technique was used for data elements where temperature was in the top 20 percent of the data range, 0.8 to 1.0. Temperature data values from 0.65 to 0.8 were rendered using a volume cloud technique. See “Data Visualization with Alias|Wavefront Maya 4.5,” page 22, for further information about how this image was created. 2 SPRING 2003 NAVO MSRC NAVIGATOR Contents The Naval Oceanographic Office (NAVO) Major Shared Resource Center (MSRC): Delivering Science to the Warfighter The NAVO MSRC provides Department of Defense (DoD) scientists and engineers with high performance computing (HPC) resources, The Director’s Corner including leading edge computational systems, large-scale data storage and archiving, scientific 2 Changing to Better Serve You visualization resources and training, and expertise in specific computational technology areas (CTAs). 4 A New Teammate Joins the NAVO MSRC These CTAs include Computational Fluid Dynamics (CFD), Climate/Weather/Ocean Modeling and Simulation (CWO), Environmental Feature Articles Quality Modeling and Simulation (EQM), Computational Electromagnetics and Acoustics 5 Lattice-Boltzmann Large-Eddy Simulation of Turbulent (CEA), and Signal/Image Processing (SIP). Jet Flows NAVO MSRC 9 High Performance Computing and Simulation for Code N7 Advanced Armament Propulsion 1002 Balch Boulevard 16 Clear Air and Optical Turbulence in a Jet Stream in Stennis Space Center, MS 39522 the Airborne Laser Context 1-800-993-7677 or [email protected] High Performance Computing 11 Largest NAVO MSRC System Becomes Even Bigger and Better 11 Using the smp Queue on MARCELLUS Programming Environment and Training NAVO MSRC Navigator www.navo.hpc.mil/Navigator 13 Environmental Quality Modeling Activities Under PET 24 NAVO MSRC PETUpdate NAVO MSRC Navigator is a biannual technical publication designed to inform users of the news, 24 PET Distance Learning: Ready to Serve 24/7 events, people, accomplishments, and activities of the Center. For a free subscription or to make 25 A Consistent, Well-Documented Computational address changes, contact NAVO MSRC at the Environment for the DoD HPC Centers above address. 27 PET Climate/Weather/Ocean (CWO) Modeling and EDITOR: Simulation—A Brief Review Gioia Furness Petro, [email protected] Scientific Visualization DESIGNERS: Cynthia Millaudon, [email protected] 22 Data Visualization with Alias|Wavefront Maya 4.5 Kerry Townson, [email protected] Lynn Yott, [email protected] The Porthole Any opinions, conclusions, or recommendations in 23 Visitors to the Naval Oceanographic Office this publication are those of the author(s) and do not necessarily reflect those of the Navy or NAVO Major Shared Resource Center MSRC. All brand names and product names are trademarks or registered trademarks of their Navigator Tools and Tips respective holders. These names are for information purposes only and do not imply endorsement by the Navy or NAVO MSRC. 29 Programming TotalView and Vampir Upcoming Events Approved for Public Release 31 Conference Listings Distribution Unlimited NAVO MSRC NAVIGATOR SPRING2003 3 A New Teammate Joins the NAVO MSRC Linda Wise Pyfrom, Program Manager, Lockheed Martin at NAVOMSRC On 15 January 2003, Lockheed Martin (LM) became the provided hardware and software management and technical newest member of the Naval Oceanographic Office Major direction to global Fortune 500 companies. Shared Resource Center (NAVO MSRC) team. LM is excited Ms. Pyfrom authored and instructed the Trusted Software about providing Technical Operations and User Support Methodology for the National Security Agency Cryptologic Services to the NAVO MSRC and supporting the Center and assisted in the transition of this methodology to NAVOCEANO High Performance Computing (HPC) team. the Carnegie Mellon University Software Engineering LM brings a wealth of HPC expertise to the NAVO MSRC Institute. She was Principal Systems Engineer in support of through leading research into, and participation in the the Strategic Defense Initiative for Martin Marietta and development of, next-generation HPC systems. LM provides General Electric. Ms. Pyfrom began her career as a Test HPC hardware and software services for large-scale Engineer for Ford Aerospace serving the U.S. Air Force at its computational users and utilizes HPC resources in the design Cheyenne Mountain Space Defense Operations Center. of advanced technology products. Whether producing HPC hardware with a five order-of-magnitude performance CHARLIEROBERTSON- MANAGER, TECHNICALOPERATIONS improvement at Sandia National Laboratories or linking the Charlie Robertson has more than 40 years of management, LM team with customer laboratories in real-time simulations technical, and supervisory experience. He has served as site for the Joint Strike Fighter Program, LM is an innovative manager for HPC facility management services and site member of the HPC community. manager for military command and control software LM brings a highly experienced group to the NAVO MSRC development projects. Most recently, he was Program team that is honored to work with the Operational and Director for Technical Operations for the NAVO MSRC. Prior Research users of NAVO MSRC services. The LM team is to that position, he served as the Program Director of the committed to continuing the evolution of the NAVO MSRC U.S. Navy Primary Oceanographic Prediction System HPC capabilities in the 21st century. (POPS). Key management team members include: JEFFGOSCINIAK- MANAGER, USERSERVICES LINDAWISEPYFROM - LM PROGRAMMANAGER Jeff Gosciniak has 20 years of experience in the leadership Linda Wise Pyfrom brings to the NAVO MSRC team of software development efforts, including the development extensive experience in large-scale Information Technology of Highly Available Enterprise System Architectures. Most (IT), strategic direction, program management, engineering, recently, he served as Manager, Information Systems and process implementation for commercial, defense, and Engineering and Security for LM on the Consolidated Space civil government customers. Operations Contract (CSOC) for NASA. In this position he She has served as Director of Information Technology and served as the Chief Systems Architect for the CSOConline Director of Information Systems for LM in support of NASA, computing infrastructure. Mr. Gosciniak also served more the Navy, and other government customers. As Program than 10 years for LM Aeronautics, where he played an Manager, Ms. Pyfrom led a 186-person team in the instrumental role in the IT efforts that supported the LM win development of the Naval Standard Integrated Personnel of the Joint Strike Fighter contract. Prior to joining LM, Mr. System (NSIPS), the newly operational system supporting Gosciniak worked in the Technology Laboratories for the Navy active service personnel and retirees. Ms. Pyfrom has Eveready Battery Company. The new LM team leaders (L-R): Jeff Gosciniak, Linda Wise Pyfrom, and Charlie Robertson. 4 SPRING 2003 NAVO MSRC NAVIGATOR Lattice-Boltzmann Large-Eddy Simulation of Turbulent Jet Flows S. Menon and H. Feiz, School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA Sponsored by Army Research Office Active control using fuel modulation (by employing embedded micro- synthetic jets inside fuel injectors) has been experimentally shown to be an effective approach to control combustion instability in gas turbine engines. Numerical simulations can help in the design cycle if the dynamics of the interaction between the actuators and the combustion process can be properly modeled. However, this involves resolving motion over a wide range of scales. For example, a typical fuel injector orifice can be as small as 1-5 millimeters (mm), and the embedded microscale actuators are even smaller. On the other hand, the typical scale of Figure 1. Computational domain for the synthetic and forced jet simulations. In both cases, the flow at the entrance of the inlet pipe is a combustor is around 30 centimeters forced at the same frequency. But, in the jet case only, a mean flow is (cm). The resolution requirement to also added to the inflow condition. The flow at the exit plane of the resolve the microjets and the flow orifice evolves naturally in both cases. outside in the combustor is too severe for any single numerical method. finite-difference schemes because it collision process allows the recovery of recovers the Navier-Stokes equations the nonlinear macroscopic advection The Lattice-Boltzmann (LB) method, and is computationally very efficient, through multi-scale expansions. Second, when combined with the conventional more stable, and easily parallelizable. because the macroscopic properties of Finite-Volume Large-Eddy Simulation Additionally, the LB method solves a the flow field are not solved directly, (FV-LES), has the potential to provide single continuous particle distribution the LB method avoids solving the a collaborative resolution to this multiscale problem. (which is analogous to the particle Poisson equation, which is numerically distribution function in kinetic theory) difficult in most finite difference In this approach, the LB-LES in a lattice (or grid). methods. Third, the macroscopic approach is employed to resolve regions inside the microjets and fuel The introduction of the Bhatnager- properties are obtained from the injector while FV-LES is employed Gross-Krook (BGK) single relaxation microscopic particle distributions elsewhere in the combustor. This time model for the collision operator through simple arithmetic integration. article reports on the ability of the further simplifies the algorithm and In this model, a new, second-order, LB-LES approach to capture eliminates the lack of Galilean accurate Three Dimensional (3D) LB complex dynamics in jet flows in a invariance and the dependence of method has been developed using a 3D computationally efficient manner. The pressure on velocity. Solving the LB cubic lattice model with the 19-bit coupled LB-LES and FV-LES multi- Equation (LBE) instead of the Navier- velocity discretization (used here to scale approach is currently being Stokes equation has three distinct recover the Navier-Stokes equation). validated and will be described in advantages: First, due to the kinetic This model has been extended to the near future. nature of the LB method, the deal with complex geometries and The LB method is considered an convection operator is linear. Simple to include a variable grid without loss attractive alternative to conventional convection in conjunction with a of accuracy. NAVO MSRC NAVIGATOR SPRING2003 5 breakdown as the flow expands downstream of the orifice. Dissipation is maximum in the high strain regions that typically reside in the braid regions and in the regions surrounding the vortices. The square jet also shows an axis- switching behavior seen in the experiments as well. Axis switching is indicated by the crossover of the Additionally, spreading rate of the jet in the two to enhance its a planes. In the near-field region of the applicability to high jet exit, the vortex structures at the Reynolds number flow, an LES corners are formed farther version of this model has been downstream with respect to the developed whereby a localized sides. This triggers the axis switching dynamic subgrid model is employed since it results in the formation of to compute an additional subgrid THE SQUARE SYNTHETIC AND nonplanar vortex structure. relaxation time in the BGK model of FORCED JETS Comparisons with data from Feiz et the LBE. The dynamic evaluation The dimensions of the square jet al.1 show reasonable agreement with eliminates the need to specify any ad computational domain are shown in past experiments. hoc parameters since all model Figure 1. The grid is stretched from coefficients evolve naturally as a part the high resolution in the orifice SQUARE JET IN CROSS-FLOW of the simulation. region, but the stretching is The computational domain for the test To expedite the turnaround time, the maintained below 10 percent to case shown in Figure 3 is resolved LBE-LES solver is implemented in ensure accuracy is not compromised. using 200x150x100 for the cross-flow parallel using the Message Passing The inlet region is resolved using domain and 50X50X100 for the jet Interface (MPI). The computational 170x170x52, the nozzle is resolved section. The Reynolds number is efficiency of the LB-LES solver is using 66x66x7, and the outflow 4700, based on the jet velocity and considerable and achieves 4.42 x10-9 region is resolved using 202x202x234. the nozzle width D, and the jet cross- Central Processing Unit (CPU) Figures 2a and 2b show, respectively, flow velocity ratio is 0.5. The cross- seconds per time step, per grid point, typical visualization of the vortex flow flow velocity profile is initialized with a per processor on the IBM SP4. For a generated by the synthetic jet and boundary layer thickness of 2D. typical simulation of 20 forcing forced square jet. The forcing Figure 3 also shows a comparison of cycles, using 11 million grid points, approximately 2,000 single-processor frequency for both cases is the same, predicted mean velocity and total hours are needed on the IBM POWER and the main difference between kinetic energy with data at a specified 4 machine (MARCELLUS). these two cases is that there is no mean flow A key feature of all the studies b in the synthetic jet reported here is that the inlet pipe is case. fully resolved so that the flow at the jet exit plane evolves naturally. This is A key feature in contrast to many past studies where observed in both is the the jet exit plane profile is typically effect of vortex specified as a boundary condition. stretching and Figure 2. Flow visualization of (a) forced square jet and (b) synthetic jet. The color iso-surfaces indicate values of constant vorticity. Green indicates azimuthal vorticity, and red/blue indicates streamwise vorticity of equal and opposite sign. Initially, azimuthally coherent vortices are shed from the orifice, but undergo vortex switching and stretching, eventually leading to breakdown in more randomly oriented streamwise vortices. 6 SPRING 2003 NAVO MSRC NAVIGATOR Figure 4. Flow visualization of the jet in cross flow. The formation of the hanging vortices and the formation of the counter-rotating pair in the downstream direction is clearly seen. Recirculation downstream of these structures also forms, as indicated by the streamlines. location. Very good agreement is obtained here and also at other locations.1 A jet in cross-flow generates a complex flow topology due to the highly 3D nature of this flow. Past studies have identified two structural featuresin this flow: a horseshoe (or kidney-shaped) structure and a Figure 4 also shows how the jet rolls which allows the jet exit profile to Counter-Rotating Vortex Pair (CRVP) up and creates the recirculation evolve naturally. form in this flow. region: an important mechanism for Good agreement with established The current LB-LES captures both the mixing of jet and the cross-flow. data was obtained in the present these features and also explains the Finally, Figure 5 shows a time study. These results establish LBE- dynamics of the formation of these sequence of the formation of these LES as an alternate method for structures and their subsequent flow features as the jet exits from the simulating turbulent shear flows. breakdown. Figure 4 shows these orifice and is turned downstream by features quite clearly. Future application of this LBE-LES the cross-flow. approach will be in a hierarchical The horseshoe vortices are tubelike In summary, a new LES implementation simulation approach whereby structures that form directly above the exit on the lateral edges of the jet of the LBE method has been conventional finite-volume LES and extend around the jet body developed and used to simulate a methods will be used to resolve the and up along the lee side of the jet, 3D square jet and a 3D square jet large-scale flow features in the approximately matching the path of in cross-flow. A localized dynamic combustor, while the LBE-LES the jet. These tubes coincide with the subgrid closure is used to close the approach will be used to resolve the location where the jet shear layer LES version of the LBE model. In finer scale features as in the folds and eventually contribute to the these simulations the inflow is applied embedded synthetic jet and/or the circulation of the CRVP. far upstream of the jet exit plane, flow inside the fuel injector. X / D = 0 U (EXP) V W 2 K U (LBE) V W K 1 0 -1 0 1 2 Figure 3. Schematic of the jet in cross-flow and comparison with experimental data. NAVO MSRC NAVIGATOR SPRING2003 7 1 2 3 4 5 6 Figure 5. Time sequence of the formation of the jet in cross-flow and the shedding of the hanging vortices as the flow propagates downstream. References 1.Feiz, H., H. Soo, and S. Menon, "LES of Turbulent Jets Using the Lattice Boltzmann Approach," AIAA Paper No. 2003-0780, 41st AIAA Aerospace Sciences Meeting and Exhibit, Reno, NV, January 2003. Contact Information For further information contact Suresh Menon at [email protected] 8 SPRING 2003 NAVO MSRC NAVIGATOR High Performance Computing and Simulation for Advanced Armament Propulsion Michael J. Nusca, U.S. Army Research Laboratory (ARL), Aberdeen Proving Ground, MD The Army is exploring a variety of and repeatedly ignite the high-energy Computing (HPC) facilities. However, armament propulsion options for and HLD solid propellant charge. the gun propulsion-modeling indirect- and direct-fire weapons As modular and HLD propelling environment has historically been (guns) for the legacy force and Future charges are being developed, one in which separate codes Combat Systems (FCS). optimized, and ultimately mated to (some one-dimensional, some two- As it transforms, the Army has systems such as indirect-fire cannon dimensional (2D)) are used, with no identified requirements for and the continually evolving FCS, single multidimensional code able hypervelocity projectile launch there is a critical need to have a to address the truly 3D details of all systems for strategic Army missions. single, validated, maintainable these weapons systems. Among these systems are those that computer code based on state-of-the- This unfortunate situation renders use solid propellant—granular form art Computational Fluid Dynamics comparison of ballistic performance loaded in modules (indirect-fire) or (CFD) as an evaluation and cumbersome and inconclusive. In disk and strip form for High-Loading- performance analysis tool. contrast, the multiphase continuum Density (HDL) cartridges (direct-fire) It has long been recognized that the equations that represent the physics —augmented by ElectroThermal- availability of such a tool would of gun propulsion comprise a set Chemical (ETC) technology. Two such provide the Army with the unique of general equations universally armament propulsion systems are the capability to simulate current and applicable to all solid propellant Army's Modular Artillery Charge emerging gun propulsion systems armament propulsion systems. System (MACS) and HLD charges for using computer simulations. These In direct response to this situation the the FCS. simulations would serve to both ARL began a development program The MACS is being developed for streamline testing and aid in the about eight years ago to revolutionize indirect fire cannon on current 155 optimization of weapon performance. the Army's ability to use HPC to milimeter (mm) systems (e.g., the Indeed, such a tool would dovetail simulate propelling charges. The M109A6 Paladin and M198 Towed nicely with the Army's initiative in the current author at ARL, with Howitzer). The efficiency of the creation of national High Performance consultation from noted industry/ MACS charge is dependent on proper academic experts, has worked on flamespreading through the this project. The result is the propellant modules, a process that has been repeatedly demonstrated a Army's "next-generation," computer scaleable, 3D, multiphase, in gun firings, successfully CFD code for armament propulsion photographed using the Army modeling. Research Laboratory (ARL) 155 mm ballistics simulator, and The ARL NGEN3 code represents the b numerically modeled using the sole Department of Defense (DoD) ARL Next Generation Three computer tool that is able to simulate Dimensional (3D) interior ballistics the highly complex physics associated code (NGEN3). The FCS requires with indirect- and direct-fire guns. c weapons systems exhibiting NGEN3 code development and increased range, accuracy, application to the FCS is a and highly repeatable projectile DoD HPC Challenge Project (FY01- launch performance. 03) and is being exercised regularly Figure 1. (a) Porosity Contours (red is One of the technologies under with priority access to the Cray dense material) at Initial Time, investigation to achieve these goals is SV1ex at the Naval Oceanographic (b) Porosity Contours at 6 ms, and the ETC concept, in which electrically (c) Propellant Temperature Contours Office Major Shared Resource Center generated plasma is injected into the (red is fully ignited propellant at 440K) (NAVO MSRC). gun chamber in order to efficiently at 6 ms. NAVO MSRC NAVIGATOR SPRING2003 9