Skip to content

Commit 1ddef3d

Browse files
authored
Merge pull request SPECFEM#1237 from danielpeter/devel
updates doc
2 parents e70c9c8 + 7d49b76 commit 1ddef3d

13 files changed

+151
-26
lines changed

doc/USER_MANUAL/02_getting_started.tex

Lines changed: 95 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,6 @@ \chapter{Getting Started}\label{cha:Getting-Started}
99
git clone --recursive --branch devel https://github.com/SPECFEM/specfem2d.git
1010
\end{verbatim}
1111

12-
Note: for people who would like to run the package on Windows rather than on Unix machines, you can install Docker or VirtualBox (installing a Linux in VirtualBox in that latter case) and run it easily from inside that.
13-
14-
We recommend that you add \texttt{ulimit -S -s unlimited} to your \texttt{.bash\_profile} file and/or \texttt{limit stacksize unlimited} to your \texttt{.cshrc} file to suppress any potential limit to the size of the Unix stack.
1512

1613
Then, to configure the software for your system, run the
1714
\texttt{configure} shell script. This script will attempt to guess
@@ -32,22 +29,24 @@ \chapter{Getting Started}\label{cha:Getting-Started}
3229

3330
You can replace the GNU compilers above (gfortran and gcc) with other compilers if you want to; for instance for Intel ifort and icc use FC=ifort CC=icc instead.
3431

35-
Before running the \texttt{configure} script, you should probably edit file \texttt{flags.guess} to make sure that it contains the best compiler options for your system. Known issues or things to check are:
36-
32+
Before running the \texttt{configure} script, you should probably edit file \texttt{flags.guess} to make sure that it contains the best
33+
compiler options for your system. Known issues or things to check are:
3734
\begin{description}
38-
\item [Intel ifort compiler] See if you need to add \texttt{-assume byterecl} for your machine.\newline
39-
35+
\item [{\texttt{Intel ifort compiler}}] See if you need to add \texttt{-assume byterecl} for your machine.
4036
In the case of that compiler, we have noticed that initial release versions sometimes have bugs or issues that can lead to wrong results when running the code, thus we \emph{strongly} recommend using a version for which at least one service pack or update has been installed.
41-
In particular, for \red{version 17} of that compiler, users have reported problems (making the code crash at run time) with the \texttt{-assume buffered\_io} option; if you notice problems,
37+
In particular, for version 17 of that compiler, users have reported problems (making the code crash at run time) with the \texttt{-assume buffered\_io} option; if you notice problems,
4238
remove that option from file \texttt{flags.guess} or change it to \texttt{-assume nobuffered\_io} and try again.
43-
\item [IBM compiler] See if you need to add \texttt{-qsave} or \texttt{-qnosave} for your machine.
44-
\item [Mac OS] You will probably need to install \texttt{Xcode}.
45-
\item [IBM Blue Gene machines] Please refer to the manual of SPECFEM3D\_Cartesian, which contains detailed instructions on how to run on Blue Gene.
39+
40+
\item [{\texttt{IBM compiler}}] See if you need to add \texttt{-qsave} or \texttt{-qnosave} for your machine.
41+
42+
\item [{\texttt{Mac OS}}] You will probably need to install \texttt{Xcode}.
43+
44+
%\item [{\texttt{IBM Blue Gene machines}] Please refer to the manual of SPECFEM3D\_Cartesian, which contains detailed instructions on how to run on Blue Gene.
4645
\end{description}
4746

4847
The SPECFEM2D software package relies on the SCOTCH library to partition meshes.
49-
The SCOTCH library \citep{PeRo96}
50-
provides efficient static mapping, graph and mesh partitioning routines. SCOTCH is a free software package developed by
48+
The SCOTCH library \citep{PeRo96} provides efficient static mapping,
49+
graph and mesh partitioning routines. SCOTCH is a free software package developed by
5150
Fran\c{c}ois Pellegrini et al. from LaBRI and Inria in Bordeaux, France, downloadable from the web page \url{https://gitlab.inria.fr/scotch/scotch}.
5251
In case no SCOTCH libraries can be found on the system, the configuration will bundle the version provided with the source code for compilation.
5352
The path to an existing SCOTCH installation can to be set explicitly with the option \texttt{-{}-with-scotch-dir}.
@@ -68,7 +67,13 @@ \chapter{Getting Started}\label{cha:Getting-Started}
6867

6968
When compiling the SCOTCH source code, if you get a message such as: "ld: cannot find -lz",
7069
the Zlib compression development library is probably missing on your machine and you will need to install it or ask your system administrator to
71-
do so. On Linux machines the package is often called "zlib1g-dev" or similar. (thus "sudo apt-get install zlib1g-dev" would install it)
70+
do so. On Linux machines the package is often called "zlib1g-dev" or similar (thus "sudo apt-get install zlib1g-dev" would install it).\newline
71+
72+
You can add \texttt{-{}-enable-vectorization} to the configuration options to speed up the code in the fluid (acoustic) and elastic parts.
73+
This works fine if (and only if) your computer always allocates a contiguous memory block for each allocatable array;
74+
this is the case for most machines and most compilers, but not all.
75+
To check if that option works fine on your machine, run the code with and without it for an acoustic/elastic model and make sure the seismograms are identical.\newline
76+
7277

7378
You may edit the \texttt{Makefile} for more specific modifications. Especially, there are several options available:
7479
%
@@ -83,7 +88,7 @@ \chapter{Getting Started}\label{cha:Getting-Started}
8388
make
8489
\end{verbatim}
8590
%
86-
to create all executables which will be placed into the folder \texttt{./bin/}.
91+
to create all executables which will be placed into the folder \texttt{./bin/}.\newline
8792

8893
By default, the solver runs in single precision. This is fine for most application, but if for some reason
8994
you want to run the solver in double precision, run the \texttt{configure} script with option ``\texttt{-{}-enable-double-precision}''.
@@ -94,23 +99,91 @@ \chapter{Getting Started}\label{cha:Getting-Started}
9499
\texttt{replace\_use\_mpi\_with\_include\_mpif\_dot\_h.pl} in the root directory to replace all of them with \texttt{include `mpif.h'} automatically.
95100

96101
If you have problems configuring the code on a Cray machine, i.e. for instance if you get an error message from the \texttt{configure} script, try exporting these two variables:
97-
\texttt{MPI\_INC=\${CRAY\_MPICH2\_DIR}/include and FCLIBS=" "}, and for more details if needed you can refer to the \texttt{utils/Cray\_compiler\_information} directory.
102+
\texttt{MPI\_INC=\${CRAY\_MPICH2\_DIR}/include and FCLIBS=" "}, and for more details if needed you can refer to the \texttt{utils/infos/Cray\_compiler\_information} directory.
98103
You can also have a look at the configure script called\newline
99-
\texttt{utils/Cray\_compiler\_information/configure\_SPECFEM\_for\_Piz\_Daint.bash}.
104+
\texttt{utils/infos/Cray\_compiler\_information/configure\_SPECFEM\_for\_Piz\_Daint.bash}.\newline
105+
106+
For people who would like to run the package on Windows rather than on Unix machines, you can install Docker or VirtualBox (installing a Linux in VirtualBox in that latter case) and run it easily from inside that.\newline
100107

108+
We recommend that you add \texttt{ulimit -S -s unlimited} to your \texttt{.bash\_profile} file and/or \texttt{limit stacksize unlimited} to your \texttt{.cshrc} file to suppress any potential limit to the size of the Unix stack.\newline
109+
110+
111+
%------------------------------------------------------------------------------------------------%
112+
\section{Using the GPU version of the code}
113+
%------------------------------------------------------------------------------------------------%
114+
115+
\noindent
116+
SPECFEM2D supports GPU acceleration by CUDA.
117+
When compiling for GPU cards, you can enable the CUDA version with:
118+
\begin{verbatim}
119+
./configure --with-cuda ..
120+
\end{verbatim}
121+
or
122+
\begin{verbatim}
123+
./configure --with-cuda=cuda9 ..
124+
\end{verbatim}
125+
where for example \texttt{cuda4,cuda5,cuda6,cuda7,..} specifies the target GPU architecture of your card,
126+
(e.g., with CUDA 9 this refers to Volta V100 cards), rather than the installed version of the CUDA toolkit.
127+
Before CUDA version 5, one version supported basically one new architecture and needed a different kind of compilation.
128+
Since version 5, the compilation has stayed the same, but newer versions supported newer architectures.
129+
However at the moment, we still have one version linked to one specific architecture:
130+
\begin{verbatim}
131+
- CUDA 4 for Tesla, cards like K10, Geforce GTX 650, ..
132+
- CUDA 5 for Kepler, like K20
133+
- CUDA 6 for Kepler, like K80
134+
- CUDA 7 for Maxwell, like Quadro K2200
135+
- CUDA 8 for Pascal, like P100
136+
- CUDA 9 for Volta, like V100
137+
- CUDA 10 for Turing, like GeForce RTX 2080
138+
- CUDA 11 for Ampere, like A100
139+
- CUDA 12 for Hopper, like H100
140+
\end{verbatim}
141+
So even if you have the new CUDA toolkit version 11, but you want to run on say a K20 GPU, then you would still configure with:
142+
\begin{verbatim}
143+
./configure --with-cuda=cuda5
144+
\end{verbatim}
145+
The compilation with the cuda5 setting chooses then the right architecture (\texttt{-gencode=arch=compute\_35,code=sm\_35} for K20 cards).\newline
146+
147+
148+
%------------------------------------------------------------------------------------------------%
149+
\section{Adding OpenMP support in addition to MPI}
150+
%------------------------------------------------------------------------------------------------%
151+
152+
OpenMP support can be enabled in addition to MPI. However, in many
153+
cases performance will not improve because our pure MPI implementation
154+
is already heavily optimized and thus the resulting code will in fact
155+
be slightly slower. A possible exception could be IBM BlueGene-type
156+
architectures.\newline
157+
158+
\noindent
159+
To enable OpenMP, add the flag \texttt{-{}-enable-openmp} to the configuration:
160+
\begin{verbatim}
161+
./configure --enable-openmp ..
162+
\end{verbatim}
163+
This will add the corresponding OpenMP flag for the chosen Fortran compiler.
164+
Please note that only the elastic domain solver has OpenMP support for now.\newline
165+
166+
167+
%------------------------------------------------------------------------------------------------%
101168
\section{Visualizing the subroutine calling tree of the source code}
169+
%------------------------------------------------------------------------------------------------%
102170

103171
Packages such as \texttt{doxywizard} can be used to visualize the subroutine calling tree of the source code.
104172
\texttt{Doxywizard} is a GUI front-end for configuring and running \texttt{doxygen}.
105173

106-
\section{Becoming a developer of the code, or making small modifications in the source code}
174+
\noindent
175+
To visualize the call tree (calling tree) of the source code, you can see the Doxygen tool available in directory \texttt{doc/call\_trees\_of\_the\_source\_code}.
107176

108-
If you want to develop new features in the code, and/or if you want to make small changes, improvements, or bug fixes, you are very welcome to contribute.\newline
109177

110-
To do so, i.e. to access the development branch of the source code with read/write access (in a safe way, no need to worry too much about breaking the package, there is a robot called BuildBot that is in charge of checking and validating all new contributions and changes), please visit this Web page:\newline
178+
%------------------------------------------------------------------------------------------------%
179+
\section{Becoming a developer of the code, or making small modifications in the source code}
180+
%------------------------------------------------------------------------------------------------%
181+
182+
If you want to develop new features in the code, and/or if you want to make small changes, improvements, or bug fixes, you are very welcome to contribute!
183+
To do so, i.e. to access the development branch of the source code with read/write access (in a safe way, no need to worry too much about breaking the package,
184+
there are CI tests based on BuildBot, Travis-CI and Jenkins in place that are checking and validating all new contributions and changes),
185+
please visit this Web page:\newline
111186
\url{https://github.com/SPECFEM/specfem2d/wiki}\newline
112187

113-
\noindent
114-
To visualize the call tree (calling tree) of the source code, you can see the Doxygen tool available in directory \texttt{doc/call\_trees\_of\_the\_source\_code}.
115188

116189

doc/USER_MANUAL/03_mesh_generation.tex

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -162,7 +162,9 @@ \section*{Notes about \texttt{DATA/Par\_file} parameters}
162162
\end{figure}
163163

164164

165+
%------------------------------------------------------------------------------------------------%
165166
\section{How to use Gmsh to generate an external mesh}
167+
%------------------------------------------------------------------------------------------------%
166168

167169
Gmsh%
168170
\footnote{freely available at the following address : \url{http://gmsh.info/}}
@@ -267,7 +269,10 @@ \section{How to use Gmsh to generate an external mesh}
267269
In addition, four files like \texttt{free\_surface\_file} corresponding
268270
to the sides of the model are generated.
269271

272+
273+
%------------------------------------------------------------------------------------------------%
270274
\section{How to use Cubit/Trelis to generate an external mesh}
275+
%------------------------------------------------------------------------------------------------%
271276
Trelis (that was known as Cubit)%
272277
\footnote{available at \url{http://www.csimsoft.com/}}
273278
is a 2D/3D finite element grid generator distributed by Csimsoft which can be
@@ -309,7 +314,9 @@ \section{How to use Cubit/Trelis to generate an external mesh}
309314
This tab will allow you to play the scripts one line after another directly in Cubit/Trelis.
310315
With this you should be able to understand how to create meshes and export them under SPECFEM2D format.
311316

317+
%------------------------------------------------------------------------------------------------%
312318
\subsection{Note about Cubit/Trelis built-in Python}
319+
%------------------------------------------------------------------------------------------------%
313320
Beware, there are some (annoying) differences between cubit built-in Python and the actual Python langage:
314321
\begin{itemize}
315322
\item \texttt{"aString" + 'anotherString'} can cause problems even after being stored: \newline
@@ -334,6 +341,7 @@ \subsection{Note about Cubit/Trelis built-in Python}
334341
\end{itemize}
335342
And probably many others! Think about that before getting mad.
336343

344+
337345
%------------------------------------------------------------------------------------------------%
338346
\section{Notes about absorbing PMLs}
339347
%------------------------------------------------------------------------------------------------%
@@ -449,6 +457,7 @@ \section{Notes about absorbing PMLs}
449457
\end{figure}
450458
%%
451459

460+
452461
%------------------------------------------------------------------------------------------------%
453462
\section{Controlling the quality of an external mesh}
454463
%------------------------------------------------------------------------------------------------%
@@ -471,6 +480,7 @@ \section{Controlling the quality of an external mesh}
471480
%
472481
This tool is useful to estimate the mesh quality and to see it evolve along the successive corrections.
473482

483+
474484
%------------------------------------------------------------------------------------------------%
475485
\section{Controlling how the mesh samples the wave field}
476486
%------------------------------------------------------------------------------------------------%

doc/USER_MANUAL/04_running_the_solver.tex

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@ \chapter{Running the Solver xspecfem2D}
5050

5151
\end{itemize}
5252

53+
5354
%------------------------------------------------------------------------------------------------%
5455
\section*{Notes about \texttt{DATA/Par\_file} parameters}
5556
%------------------------------------------------------------------------------------------------%
@@ -96,7 +97,6 @@ \section*{Notes about \texttt{DATA/Par\_file} parameters}
9697
\section*{Notes about \texttt{DATA/SOURCE} parameters}
9798
%------------------------------------------------------------------------------------------------%
9899

99-
100100
The \texttt{SOURCE} file located in the \texttt{DATA/} directory should be edited in the following way:
101101
%
102102
\begin{description}
@@ -230,7 +230,7 @@ \section{How to run elastic wave simulations}
230230

231231
An optional useful Python script called \texttt{SEM\_save\_dir.py} is provided.
232232
It allows one to automatically save all the parameters and results of a given simulation.
233-
%------------------------------------------------------------------------------------------------%
233+
234234

235235
%------------------------------------------------------------------------------------------------%
236236
\section{How to run axisymmetric wave simulations}
@@ -312,7 +312,7 @@ \section{How to run axisymmetric wave simulations}
312312
This simple code is useful to learn how the spectral-element method works in both plane-strain and axisymmetric geometries.
313313
Have a look to it if interested. Once in its directory, type \texttt{./make\_Fortran\_2D\_axisymmetric.csh} and then \texttt{./xspecfem2D}
314314
to compile and run. The bug discussed above is not present in this small code.
315-
%------------------------------------------------------------------------------------------------%
315+
316316

317317
%------------------------------------------------------------------------------------------------%
318318
\section{How to run anisotropic wave simulations}
@@ -454,6 +454,7 @@ \section{How to run poroelastic wave simulations}
454454
several sources with different frequencies and if you consider anistropic
455455
permeability.
456456

457+
457458
%------------------------------------------------------------------------------------------------%
458459
\section{How to run electromagnetic wave simulations}
459460
%------------------------------------------------------------------------------------------------%
@@ -610,6 +611,7 @@ \section{How to choose the time step}
610611
% is used to refer this table in the text
611612
\end{table}
612613

614+
613615
%------------------------------------------------------------------------------------------------%
614616
\section{How to set plane waves as initial conditions}
615617
%------------------------------------------------------------------------------------------------%
@@ -631,6 +633,7 @@ \section{How to set plane waves as initial conditions}
631633
\end{itemize}
632634
\end{description}
633635

636+
634637
%------------------------------------------------------------------------------------------------%
635638
\section{Note on the viscoelastic model used}
636639
%------------------------------------------------------------------------------------------------%
@@ -644,6 +647,7 @@ \section{Note on the viscoelastic model used}
644647
and thus the code outputs the band in which the approximation is very good, outside of that range it can be less accurate.
645648
The logarithmic center of that frequency band is the \texttt{f0} parameter defined (in Hz) in input file \texttt{DATA/SOURCE}.
646649

650+
647651
%------------------------------------------------------------------------------------------------%
648652
\section{Note on viscoelasticity in the 2D plane strain approximation}
649653
%------------------------------------------------------------------------------------------------%

doc/USER_MANUAL/05_adjoint_simulations.tex

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,10 @@ \section{How to obtain finite sensitivity kernels}
6262

6363
\end{enumerate}
6464

65+
66+
%------------------------------------------------------------------------------------------------%
6567
\section{Remarks about adjoint runs and solving inverse problems}
68+
%------------------------------------------------------------------------------------------------%
6669

6770
% This from Carl Tape:
6871
SPECFEM2D can produce the gradient of the misfit function for a
@@ -88,7 +91,9 @@ \section{Remarks about adjoint runs and solving inverse problems}
8891
``\texttt{ac}'' denotes acoustic (``\texttt{el}'' for elastic),
8992
``\texttt{kl}'' means kernel (and you may find ``\texttt{k}'' as well, which is the interaction at each time step, i.e., before doing time integration).
9093

94+
%------------------------------------------------------------------------------------------------%
9195
\section{Caution}
96+
%------------------------------------------------------------------------------------------------%
9297

9398
Please note that:
9499
%

0 commit comments

Comments
 (0)