[FSU] Inference FSU with Shared memory #2969
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Update FSU forwarding logic - FSU will handle look ahead tensor inside of pool - so we don't need to call Loadtensor for f + i **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghak PARK <[email protected]>
Add memory ptr for allocate shared mem - add mem_ptr - add unmap - array for manage unmapped ptr **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghak PARK <[email protected]>
I have changed the method from using dynamic memory allocation to using static memory allocation. In order to prevent multiple frees, I added a map to check whether the mem_address has already been processed. Previously, memory was allocated through buf, but now it is being allocated directly. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Co-authored-by: jijoong.moon <[email protected]> Signed-off-by: Donghak PARK <[email protected]>
make neuralnet can pass path to the swap_device & weight offset (file offset) it can make calculate weight file's offset **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Co-authored-by: hyeonseok <[email protected]> Signed-off-by: Donghak PARK <[email protected]>
DonghakPark
requested review from
myungjoo,
jijoongmoon,
again4you,
jaeyun-jung,
leemgs,
wooksong,
gichan-jang,
anyj0527,
lhs8928,
songgot,
jihochu,
SeoHyungjun,
baek2sm,
skykongkong8,
djeong20,
EunjuYang,
dkjung and
haehun
as code owners
This was referenced Feb 25, 2025
DonghakPark
force-pushed
the
FSU_with_shared_Mem
branch
from
3b395aa to
1629bad
Compare
Apply Shared mem & FSU - when inference mode : read from weight bin ( weight offset ) - when train mode : same logic with swap **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghak PARK <[email protected]>
DonghakPark
force-pushed
the
FSU_with_shared_Mem
branch
from
1629bad to
769c28a
Compare
Fix Unittest Fail bug at Training Case Swap - There are some issue on PutBuffer that can not free ptr **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghak PARK <[email protected]>
DonghakPark
force-pushed
the
FSU_with_shared_Mem
branch
from
ec044b7 to
d328785
Compare
Apply clang format at changed File **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghak PARK <[email protected]>
DonghakPark
force-pushed
the
FSU_with_shared_Mem
branch
from
d328785 to
995c37c
Compare
Update FSU Unitter - For now, we should set our weight & input size as pagesize * N - For later i will add Page Align Algorithm **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghak PARK <[email protected]>
DonghakPark
force-pushed
the
FSU_with_shared_Mem
branch
from
1ebd358 to
e41af8c
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[FSU] Inference FSU with Shared memory
To reduce memory usage during inference by utilizing FSU, and to minimize speed degradation by performing loading during forwarding, changed to use shared memory. and ensure the existing swap in training mode is also performed normally.
Commit 1 : [FSU] Update FSU Forwarding (Load) Logis
Commit 2 : [FSU] Update swap device & cache element
Commit 3 : [FSU] Update FSU mem allocate Logic
Commit 4 : [FSU] add FSU file offset info
Commit 5 : [FSU] Apply Shared Mem & FSU
This PR was include #2957 #2927 #2949, So i will close previous PRs