This repository has been archived by the owner on Apr 4, 2023. It is now read-only.
-
-
Notifications
You must be signed in to change notification settings - Fork 30
/
Copy pathfaq.html
190 lines (185 loc) · 10.6 KB
/
faq.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Frequently asked questions — IPython v0.9.1 documentation</title>
<link rel="stylesheet" href="_static/default.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
VERSION: '0.9.1',
COLLAPSE_MODINDEX: false,
FILE_SUFFIX: '.html'
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="contents" title="Global table of contents" href="contents.html" />
<link rel="index" title="Global index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="top" title="IPython v0.9.1 documentation" href="index.html" />
<link rel="next" title="History" href="history.html" />
<link rel="prev" title="IPython.kernel.core.notification blueprint" href="development/notification_blueprint.html" />
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="modindex.html" title="Global Module Index"
accesskey="M">modules</a> |</li>
<li class="right" >
<a href="history.html" title="History"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="development/notification_blueprint.html" title="IPython.kernel.core.notification blueprint"
accesskey="P">previous</a> |</li>
<li><a href="index.html">IPython v0.9.1 documentation</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section">
<h1 id="frequently-asked-questions"><span id="faq"></span>Frequently asked questions<a class="headerlink" href="#frequently-asked-questions" title="Permalink to this headline">¶</a></h1>
<div class="section">
<h2 id="general-questions">General questions<a class="headerlink" href="#general-questions" title="Permalink to this headline">¶</a></h2>
</div>
<div class="section">
<h2 id="questions-about-parallel-computing-with-ipython">Questions about parallel computing with IPython<a class="headerlink" href="#questions-about-parallel-computing-with-ipython" title="Permalink to this headline">¶</a></h2>
<div class="section">
<h3 id="will-ipython-speed-my-python-code-up">Will IPython speed my Python code up?<a class="headerlink" href="#will-ipython-speed-my-python-code-up" title="Permalink to this headline">¶</a></h3>
<p>Yes and no. When converting a serial code to run in parallel, there often many
difficulty questions that need to be answered, such as:</p>
<ul class="simple">
<li>How should data be decomposed onto the set of processors?</li>
<li>What are the data movement patterns?</li>
<li>Can the algorithm be structured to minimize data movement?</li>
<li>Is dynamic load balancing important?</li>
</ul>
<p>We can’t answer such questions for you. This is the hard (but fun) work of parallel
computing. But, once you understand these things IPython will make it easier for you to
implement a good solution quickly. Most importantly, you will be able to use the
resulting parallel code interactively.</p>
<p>With that said, if your problem is trivial to parallelize, IPython has a number of
different interfaces that will enable you to parallelize things is almost no time at
all. A good place to start is the <tt class="docutils literal"><span class="pre">map</span></tt> method of our <tt class="xref docutils literal"><span class="pre">MultiEngineClient</span></tt>.</p>
</div>
<div class="section">
<h3 id="what-is-the-best-way-to-use-mpi-from-python">What is the best way to use MPI from Python?<a class="headerlink" href="#what-is-the-best-way-to-use-mpi-from-python" title="Permalink to this headline">¶</a></h3>
</div>
<div class="section">
<h3 id="what-about-all-the-other-parallel-computing-packages-in-python">What about all the other parallel computing packages in Python?<a class="headerlink" href="#what-about-all-the-other-parallel-computing-packages-in-python" title="Permalink to this headline">¶</a></h3>
<p>Some of the unique characteristic of IPython are:</p>
<ul class="simple">
<li>IPython is the only architecture that abstracts out the notion of a
parallel computation in such a way that new models of parallel computing
can be explored quickly and easily. If you don’t like the models we
provide, you can simply create your own using the capabilities we provide.</li>
<li>IPython is asynchronous from the ground up (we use <a class="reference" href="http://www.twistedmatrix.com">Twisted</a>).</li>
<li>IPython’s architecture is designed to avoid subtle problems
that emerge because of Python’s global interpreter lock (GIL).</li>
<li>While IPython’s architecture is designed to support a wide range
of novel parallel computing models, it is fully interoperable with
traditional MPI applications.</li>
<li>IPython has been used and tested extensively on modern supercomputers.</li>
<li>IPython’s networking layers are completely modular. Thus, is
straightforward to replace our existing network protocols with
high performance alternatives (ones based upon Myranet/Infiniband).</li>
<li>IPython is designed from the ground up to support collaborative
parallel computing. This enables multiple users to actively develop
and run the <em>same</em> parallel computation.</li>
<li>Interactivity is a central goal for us. While IPython does not have
to be used interactivly, it can be.</li>
</ul>
</div>
<div class="section">
<h3 id="why-the-ipython-controller-a-bottleneck-in-my-parallel-calculation">Why The IPython controller a bottleneck in my parallel calculation?<a class="headerlink" href="#why-the-ipython-controller-a-bottleneck-in-my-parallel-calculation" title="Permalink to this headline">¶</a></h3>
<p>A golden rule in parallel computing is that you should only move data around if you
absolutely need to. The main reason that the controller becomes a bottleneck is that
too much data is being pushed and pulled to and from the engines. If your algorithm
is structured in this way, you really should think about alternative ways of
handling the data movement. Here are some ideas:</p>
<ol class="arabic simple">
<li>Have the engines write data to files on the locals disks of the engines.</li>
<li>Have the engines write data to files on a file system that is shared by
the engines.</li>
<li>Have the engines write data to a database that is shared by the engines.</li>
<li>Simply keep data in the persistent memory of the engines and move the
computation to the data (rather than the data to the computation).</li>
<li>See if you can pass data directly between engines using MPI.</li>
</ol>
</div>
<div class="section">
<h3 id="isn-t-python-slow-to-be-used-for-high-performance-parallel-computing">Isn’t Python slow to be used for high-performance parallel computing?<a class="headerlink" href="#isn-t-python-slow-to-be-used-for-high-performance-parallel-computing" title="Permalink to this headline">¶</a></h3>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<h3><a href="index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference" href="">Frequently asked questions</a><ul>
<li><a class="reference" href="#general-questions">General questions</a></li>
<li><a class="reference" href="#questions-about-parallel-computing-with-ipython">Questions about parallel computing with IPython</a><ul>
<li><a class="reference" href="#will-ipython-speed-my-python-code-up">Will IPython speed my Python code up?</a></li>
<li><a class="reference" href="#what-is-the-best-way-to-use-mpi-from-python">What is the best way to use MPI from Python?</a></li>
<li><a class="reference" href="#what-about-all-the-other-parallel-computing-packages-in-python">What about all the other parallel computing packages in Python?</a></li>
<li><a class="reference" href="#why-the-ipython-controller-a-bottleneck-in-my-parallel-calculation">Why The IPython controller a bottleneck in my parallel calculation?</a></li>
<li><a class="reference" href="#isn-t-python-slow-to-be-used-for-high-performance-parallel-computing">Isn’t Python slow to be used for high-performance parallel computing?</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="development/notification_blueprint.html" title="previous chapter">IPython.kernel.core.notification blueprint</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="history.html" title="next chapter">History</a></p>
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="_sources/faq.txt">Show Source</a></li>
</ul>
<h3>Quick search</h3>
<form class="search" action="search.html" method="get">
<input type="text" name="q" size="18" /> <input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="modindex.html" title="Global Module Index"
accesskey="M">modules</a> |</li>
<li class="right" >
<a href="history.html" title="History"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="development/notification_blueprint.html" title="IPython.kernel.core.notification blueprint"
accesskey="P">previous</a> |</li>
<li><a href="index.html">IPython v0.9.1 documentation</a> »</li>
</ul>
</div>
<div class="footer">
© Copyright 2008, The IPython Development Team.
Last updated on Sep 14, 2008.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 0.5.
</div>
</body>
</html>