GPU accelerated data analysis in Python

93 Pages·2017·13.71 MB·English

by  Giuseppe Di Bernardo

X-ray Crystallography: a short overview Accelerated Big Data Analytics with Python 3D Advanced Visualization
GPU accelerated data analysis in Python:
a study case in Materials Science
Giuseppe Di Bernardo
Max Planck Computing and Data Facility (MPCDF)
Garching bei München, Germany
[email protected]
@jose_dibernardo
Firenze
April, 19th - 22nd 2018
Giuseppe Di Bernardo (MPCDF) GPU Crystallography with Python April, 19-22 2017 1 / 59
Basic Research at cutting-edge;
strictly curiosity-driven and quality oriented;
Flexible, dynamic, interdisciplinary MPIs;
Autonomy → scientists decide upon science;
Long-term trust systems;
Quality assurance by peers.
Sites of Max Planck Research Institutes
and Associated Institutes
MAX PLANCK INSTITUTES ABROAD
ITALY
BibliothecaHertziana, Rome
Art HistoryInstitute, Florence
THE NETHERLANDS
Max Planck Institute forPsycholinguistics, Nijmegen
USA, FLORIDA
Max Planck Florida, Jupiter
LUXEMBOURG
Max Planck Institute Luxembourg forInternational,
European andRegulatoryProceduralLaw,
Luxembourg
M A X -P L A N C K -G E S E L L S C H A F T | NAME, ABTEILUNG, KURZTITEL TT.MM.JJJJ | SEITE ‹Nr.›
X-ray Crystallography: a short overview Accelerated Big Data Analytics with Python 3D Advanced Visualization
Outline
1 X-ray Crystallography: a short overview
2 Accelerated Big Data Analytics with Python
3 3D Advanced Visualization of APT Crystallography
Giuseppe Di Bernardo (MPCDF) GPU Crystallography with Python April, 19-22 2017 4 / 59
X-ray Crystallography: a short overview Accelerated Big Data Analytics with Python 3D Advanced Visualization
Outline
1 X-ray Crystallography: a short overview
Atom Probe Tomography
APT Crystallography: implementation on a GPU
2 Accelerated Big Data Analytics with Python
PyCUDA: a Python-based approach to GPU computing
PyNX: a GPU-based Fourier analysis in a Python workbench
3 3D Advanced Visualization of APT Crystallography
Python Plotting for Exploratory Data Analysis
Getting started with ParaView
Giuseppe Di Bernardo (MPCDF) GPU Crystallography with Python April, 19-22 2017 5 / 59
X-ray Crystallography: a short overview Accelerated Big Data Analytics with Python 3D Advanced Visualization
Atom Probe Tomography (APT)
A new playground for Big Data analysis1
Atom probe tomography:
Retrieve high-quality/crystalline data
Iterativily use the Fourier analysis for
reconstruction of APT data;
Need of the state-of-the art data mining and
visualization
Baptiste Gault & Shyam Katnagallu BDDMS -APT
Fourier Transforms on billion points data!!
1by courtesy of S. Katnagallu, B. Gault, L. Stephenson & D. Raabe (MPIE)
and M. Rampp (MPCDF)
Giuseppe Di Bernardo (MPCDF) GPU Crystallography with Python April, 19-22 2017 6 / 59
X-ray Crystallography: a short overview Accelerated Big Data Analytics with Python 3D Advanced Visualization
Scattering computing from an atomistic model
Theory: X-ray and neutron scattering
Scattering vector
Scattering density, at position r
Z
A(S) = ρ(r) exp(2iπ S · r)dV = FT[ρ(r)]
V
Scattered amplitude
divide et impera
Our approach: Direct calculation
FFT is very fast: O(N logN)
N−1
P
A(s) = f (s)exp(2iπS · r )
coordinates in reciprocal space j j
j=0
(hkl) are imposed by the
O(N2)
electron density grid
Giuseppe Di Bernardo (MPCDF) GPU Crystallography with Python April, 19-22 2017 7 / 59
X-ray Crystallography: a short overview Accelerated Big Data Analytics with Python 3D Advanced Visualization
GPU-accelerated computing of large X-ray data
Direct calculation in 3D reciprocal space (hkl): atomic
1
positions (r ) & scattering factors (f ):
i i
N−1
P
A(s) = f (s)e2iπS·rj
j
j=0
computing any point in (hkl) from any structural model!
In crystallography usually we have:
2
many atoms, e.g. N O(≥1 × 108)
atoms
many reflections (N ) in hkl, e.g. N O(128 × 128 × 128)
hkl khl
It requires massive parallelism
3
N ≈ 10 × N × N : O(≥1 × 1015) Flop
flop atoms hkl
modern GPU: 1013 Flop/s, CPU server: 1012 Flop/s
algorithm well suited for GPU computations, scalable across
multiple GPUs → O(minutes)
Giuseppe Di Bernardo (MPCDF) GPU Crystallography with Python April, 19-22 2017 8 / 59
X-ray Crystallography: a short overview Accelerated Big Data Analytics with Python 3D Advanced Visualization
GPU-accelerated computing of large X-ray data
Direct calculation in 3D reciprocal space (hkl): atomic
1
positions (r ) & scattering factors (f ):
i i
N−1
P
A(s) = f (s)e2iπS·rj
j
j=0
computing any point in (hkl) from any structural model!
In crystallography usually we have:
2
many atoms, e.g. N O(≥1 × 108)
atoms
many reflections (N ) in hkl, e.g. N O(128 × 128 × 128)
hkl khl
It requires massive parallelism
3
N ≈ 10 × N × N : O(≥1 × 1015) Flop
flop atoms hkl
modern GPU: 1013 Flop/s, CPU server: 1012 Flop/s
algorithm well suited for GPU computations, scalable across
multiple GPUs → O(minutes)
Giuseppe Di Bernardo (MPCDF) GPU Crystallography with Python April, 19-22 2017 8 / 59
X-ray Crystallography: a short overview Accelerated Big Data Analytics with Python 3D Advanced Visualization
GPU-accelerated computing of large X-ray data
Direct calculation in 3D reciprocal space (hkl): atomic
1
positions (r ) & scattering factors (f ):
i i
N−1
P
A(s) = f (s)e2iπS·rj
j
j=0
computing any point in (hkl) from any structural model!
In crystallography usually we have:
2
many atoms, e.g. N O(≥1 × 108)
atoms
many reflections (N ) in hkl, e.g. N O(128 × 128 × 128)
hkl khl
It requires massive parallelism
3
N ≈ 10 × N × N : O(≥1 × 1015) Flop
flop atoms hkl
modern GPU: 1013 Flop/s, CPU server: 1012 Flop/s
algorithm well suited for GPU computations, scalable across
multiple GPUs → O(minutes)
Giuseppe Di Bernardo (MPCDF) GPU Crystallography with Python April, 19-22 2017 8 / 59