Skip to content

Commit e0caf56

Browse files
committed
Write guide for parallel processing trials.
This was discussed at CiwPython#206. Note that this adds a script to `docs/_static`. This is because I do not believe the parallel processing can be doctested.
1 parent c9a3a95 commit e0caf56

File tree

5 files changed

+100
-1
lines changed

5 files changed

+100
-1
lines changed

.github/workflows/tests.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,3 +45,7 @@ jobs:
4545
- name: Run docts
4646
run: |
4747
python doctests.py
48+
49+
- name: Test parallel processing script
50+
run: |
51+
python docs/_static/script_for_parallel_processing/main.py

docs/Guides/index.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,4 +28,5 @@ Contents:
2828
deadlock.rst
2929
process_based.rst
3030
from_file.rst
31-
behaviour/index.rst
31+
behaviour/index.rst
32+
parallel_process.rst

docs/Guides/parallel_process.rst

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
.. _parallel_process:
2+
3+
=========================
4+
How to Parallelise Trials
5+
=========================
6+
7+
It is possible to repeat a simulation in parallel using the cores available on a
8+
given computer. This can lead to decreases in computational time as instead of
9+
running each successive simulation :ref:`one after the other <tutorial-iv>` they
10+
can be run at the same time.
11+
12+
As an example consider the following simulation network::
13+
14+
>>> import ciw
15+
>>> N = ciw.create_network(arrival_distributions=[ciw.dists.Exponential(rate=0.2)],
16+
... service_distributions=[ciw.dists.Exponential(rate=0.1)],
17+
... number_of_servers=[3])
18+
19+
The following function will return the mean wait time::
20+
21+
>>> def get_mean_wait(network, seed=0, max_time=10000):
22+
... """Return the mean waiting time for a given network"""
23+
... ciw.seed(seed)
24+
... Q = ciw.Simulation(N)
25+
... Q.simulate_until_max_time(max_simulation_time=max_time)
26+
... recs = Q.get_all_records()
27+
... waits = [r.waiting_time for r in recs]
28+
... mean_wait = sum(waits) / len(waits)
29+
... return mean_wait
30+
>>> get_mean_wait(network=N)
31+
3.386690...
32+
33+
To be able to better approximate the average wait, the above function will be
34+
repeated and the average taken::
35+
36+
>>> import numpy as np
37+
>>> max_time = 500
38+
>>> repetitions = 200
39+
>>> np.mean([run_simulation(network=N, max_time=max_time, seed=seed) for seed in range(repetitions)])
40+
3.762233...
41+
42+
To obtain the above by running 2 simulations at the same time (assuming that 2
43+
cores are available), the :code:`multiprocessing` library can be used. In which
44+
case the following :download:`main.py
45+
<../_static/script_for_parallel_processing/main.py>` script gives a working
46+
example:
47+
48+
.. literalinclude:: ../_static/script_for_parallel_processing/main.py
49+
50+
It is possible to use :code:`multiprocessing.cpu_count()` to obtain the number
51+
of available cores.
52+
53+
Note that the conditional :code:`if __name__ == '__main__':` is needed to ensure
54+
that :code:`get_mean_wait` can be pickled. This is necessary to ensure that it
55+
can be used by the parallel processing pool.
56+
57+
The :code:`multiprocessing` library is part of the python standard library so no
58+
further dependencies are required. However other options are available, one
59+
example of which is `dask <https://www.dask.org>`_.
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
This directory contains a script for carrying out parallel processing with ciw.
2+
3+
This is referred to in `./docs/Guides/parallel_process.rst`.
4+
5+
This script is shown in the docs but is also tested as part of the CI.
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
import ciw
2+
import multiprocessing
3+
import numpy as np
4+
5+
N = ciw.create_network(
6+
arrival_distributions=[ciw.dists.Exponential(rate=0.2)],
7+
service_distributions=[ciw.dists.Exponential(rate=0.1)],
8+
number_of_servers=[3],
9+
)
10+
11+
max_time = 500
12+
repetitions = 200
13+
14+
15+
def get_mean_wait(network, seed=0, max_time=10000):
16+
"""Return the mean waiting time for a given network"""
17+
ciw.seed(seed)
18+
Q = ciw.Simulation(N)
19+
Q.simulate_until_max_time(max_simulation_time=max_time)
20+
recs = Q.get_all_records()
21+
waits = [r.waiting_time for r in recs]
22+
mean_wait = sum(waits) / len(waits)
23+
return mean_wait
24+
25+
26+
if __name__ == "__main__":
27+
pool = multiprocessing.Pool(processes=2)
28+
args = [(N, seed, max_time) for seed in range(repetitions)]
29+
waits = pool.starmap(get_mean_wait, args)
30+
print(np.mean(waits))

0 commit comments

Comments
 (0)