Skip to content

Commit 98b5274

Browse files
authored
Update ddp_series_theory.rst
Revised based on comments
1 parent ba40331 commit 98b5274

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

β€Žbeginner_source/ddp_series_theory.rst

+4-4
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ Authors: `Suraj Subramanian <https://github.com/suraj813>`__
2323

2424
* νŒŒμ΄ν† μΉ˜ `λΉ„λΆ„μ‚° ν•™μŠ΅ <https://tutorials.pytorch.kr/beginner/basics/quickstart_tutorial.html>`__ 에 μ΅μˆ™ν•  것
2525

26-
μ•„λž˜μ˜ μ˜μƒμ΄λ‚˜ `유투브 μ˜μƒ youtube <https://www.youtube.com/watch/Cvdhwx-OBBo>`__ 을 따라 μ§„ν–‰ν•˜μ„Έμš”.
26+
μ•„λž˜μ˜ μ˜μƒμ΄λ‚˜ `유튜브 μ˜μƒ youtube <https://www.youtube.com/watch/Cvdhwx-OBBo>`__ 을 따라 μ§„ν–‰ν•˜μ„Έμš”.
2727

2828
.. raw:: html
2929

@@ -36,7 +36,7 @@ Authors: `Suraj Subramanian <https://github.com/suraj813>`__
3636
μ—¬λŸ¬ 개의 λ””λ°”μ΄μŠ€μ—μ„œ μ—¬λŸ¬ 데이터 λ°°μΉ˜λ“€μ„ λ™μ‹œμ— μ²˜λ¦¬ν•˜λŠ” λ°©λ²•μž…λ‹ˆλ‹€.
3737
νŒŒμ΄ν† μΉ˜μ—μ„œ, `λΆ„μ‚° μƒ˜ν”ŒλŸ¬ <https://pytorch.org/docs/stable/data.html#torch.utils.data.distributed.DistributedSampler>`__ λŠ”
3838
각 λ””λ°”μ΄μŠ€κ°€ μ„œλ‘œ λ‹€λ₯Έ μž…λ ₯ 배치λ₯Ό λ°›λŠ” 것을 보μž₯ν•©λ‹ˆλ‹€.
39-
λͺ¨λΈμ€ λͺ¨λ“  λ””λ°”μ΄μŠ€μ— 볡제되며, 각 사본은 변화도λ₯Ό κ³„μ‚°ν•˜λŠ” λ™μ‹œμ— `링 올-λ¦¬λ“€μŠ€
39+
λͺ¨λΈμ€ λͺ¨λ“  λ””λ°”μ΄μŠ€μ— 볡제되며, 각 사본은 기울기λ₯Ό κ³„μ‚°ν•˜λŠ” λ™μ‹œμ— `링 올-λ¦¬λ“€μŠ€
4040
μ•Œκ³ λ¦¬μ¦˜ <https://tech.preferred.jp/en/blog/technologies-behind-distributed-deep-learning-allreduce/>`__ 을 μ‚¬μš©ν•΄ λ‹€λ₯Έ 사본과 λ™κΈ°ν™”λ©λ‹ˆλ‹€.
4141

4242
`μ˜ˆμ‹œ νŠœν† λ¦¬μ–Ό <https://tutorials.pytorch.kr/intermediate/dist_tuto.html#>`__ μ—μ„œ DDP λ©”μ»€λ‹ˆμ¦˜μ— λŒ€ν•΄ 파이썬 κ΄€μ μ—μ„œ 심도 μžˆλŠ” μ„€λͺ…을 λ³Ό 수 μžˆμŠ΅λ‹ˆλ‹€.
@@ -56,8 +56,8 @@ DP λŠ” κ°„λ‹¨ν•˜μ§€λ§Œ, (ν•œ μ€„λ§Œ μΆ”κ°€ν•˜λ©΄ 됨) μ„±λŠ₯은 훨씬 λ–¨μ–΄μ§‘
5656
- λͺ¨λΈμ΄ ν•œ 번만 볡제됨
5757
* - 단일 λ…Έλ“œ 병렬 처리만 κ°€λŠ₯
5858
- μ—¬λŸ¬ λ¨Έμ‹ μœΌλ‘œ ν™•μž₯ κ°€λŠ₯
59-
* - 느림; 단일 ν”„λ‘œμ„ΈμŠ€μ—μ„œ λ©€ν‹° μŠ€λ ˆλ”©μ„ μ‚¬μš©ν•˜κΈ° λ•Œλ¬Έμ— Global Interpreter Lock (GIL) 좩돌이 λ°œμƒ
60-
- 빠름; λ©€ν‹° ν”„λ‘œμ„Έμ‹±μ„ μ‚¬μš©ν•˜κΈ° λ•Œλ¬Έμ— GIL 좩돌 μ—†μŒ
59+
* - 느림. 단일 ν”„λ‘œμ„ΈμŠ€μ—μ„œ λ©€ν‹° μŠ€λ ˆλ”©μ„ μ‚¬μš©ν•˜κΈ° λ•Œλ¬Έμ— Global Interpreter Lock (GIL) 좩돌이 λ°œμƒ
60+
- 빠름. λ©€ν‹° ν”„λ‘œμ„Έμ‹±μ„ μ‚¬μš©ν•˜κΈ° λ•Œλ¬Έμ— GIL 좩돌 μ—†μŒ
6161

6262

6363
읽을거리

0 commit comments

Comments
Β (0)