feat: add sparse rcnn #136
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Owner
long8v
commented
Jul 24, 2023
•
long8v
commented
- https://arxiv.org/abs/2011.12450
long8v
commented
Jul 24, 2023
Comment on lines
+23
to
+28
| class SetCriterion(nn.Module): | ||
| """ This class computes the loss for SparseRCNN. | ||
| The process happens in two steps: | ||
| 1) we compute hungarian assignment between ground truth boxes and the outputs of the model | ||
| 2) we supervise each pair of matched ground-truth / prediction (supervise class and box) | ||
| """ |
There was a problem hiding this comment.
loss 구하는 부분
bipartite matching -> loss 계산
There was a problem hiding this comment.
deformable DETR(focal 때문에)랑 똑같다
Comment on lines
+161
to
+162
| # In case of auxiliary losses, we repeat this process with the output of each intermediate layer. | ||
| if 'aux_outputs' in outputs: |
There was a problem hiding this comment.
얘도 aux가 있나? 그냥 복사하다가 달려온건가..
|
|
||
|
|
||
|
|
||
| class HungarianMatcher(nn.Module): |
| _DEFAULT_SCALE_CLAMP = math.log(100000.0 / 16) | ||
|
|
||
|
|
||
| class DynamicHead(nn.Module): |
There was a problem hiding this comment.
RoI Align된 feature와 무작위로 초기화되는 proposal features 부분을 합치는 부분 -> DynamicHead로 붙이겠다.
There was a problem hiding this comment.
그냥 이 클래스에서 RoIPooler까지 해주넹
| ) | ||
| return box_pooler | ||
|
|
||
| def forward(self, features, init_bboxes, init_features): |
There was a problem hiding this comment.
forward
- feature
- init_bboxes
- init_features
| pro_features: (1, N * nr_boxes, self.d_model) | ||
| roi_features: (49, N * nr_boxes, self.d_model) | ||
| ''' | ||
| features = roi_features.permute(1, 0, 2) |
There was a problem hiding this comment.
roi feature (49, N * nr_boxes, self.d_model) -> ( N * nr_boxes, 49, self.d_model)
| roi_features: (49, N * nr_boxes, self.d_model) | ||
| ''' | ||
| features = roi_features.permute(1, 0, 2) | ||
| parameters = self.dynamic_layer(pro_features).permute(1, 0, 2) |
There was a problem hiding this comment.
pro_features : (49, N * nr_boxes, self.d_model) -> (49, N * nr_boxes, self.num_dynamic * self.num_params)
Comment on lines
+287
to
+288
| param1 = parameters[:, :, :self.num_params].view(-1, self.hidden_dim, self.dim_dynamic) | ||
| param2 = parameters[:, :, self.num_params:].view(-1, self.dim_dynamic, self.hidden_dim) |
There was a problem hiding this comment.
proposal feature을 num_params 기준으로 두개로 나누고 (hidden_dim, dim_dynmaic), (dim_dynamic, hidden_dim)으로 나눔
Comment on lines
+290
to
+296
| features = torch.bmm(features, param1) | ||
| features = self.norm1(features) | ||
| features = self.activation(features) | ||
|
|
||
| features = torch.bmm(features, param2) | ||
| features = self.norm2(features) | ||
| features = self.activation(features) |
Comment on lines
+297
to
+303
|
|
||
| features = features.flatten(1) | ||
| features = self.out_layer(features) | ||
| features = self.norm3(features) | ||
| features = self.activation(features) | ||
|
|
||
| return features |
long8v
commented
Aug 2, 2023
Comment on lines
+1
to
+9
|
|
||
|
|
||
| ### Common Datasets | ||
|
|
||
| The dataset implemented here do not need to load the data into the final format. | ||
| It should provide the minimal data structure needed to use the dataset, so it can be very efficient. | ||
|
|
||
| For example, for an image dataset, just provide the file names and labels, but don't read the images. | ||
| Let the downstream decide how to read. |
There was a problem hiding this comment.
dataset 쪽
Comment on lines
+21
to
+24
| # All coco categories, together with their nice-looking visualization colors | ||
| # It's from https://github.com/cocodataset/panopticapi/blob/master/panoptic_coco_categories.json | ||
| COCO_CATEGORIES = [ | ||
| {"color": [220, 20, 60], "isthing": 1, "id": 1, "name": "person"}, |
There was a problem hiding this comment.
meta 부분. id랑 category name이랑 연결하는 부분
| @@ -0,0 +1,508 @@ | |||
| # Copyright (c) Facebook, Inc. and its affiliates. | |||
Comment on lines
+30
to
+49
| def load_coco_json(json_file, image_root, dataset_name=None, extra_annotation_keys=None): | ||
| """ | ||
| Load a json file with COCO's instances annotation format. | ||
| Currently supports instance detection, instance segmentation, | ||
| and person keypoints annotations. | ||
|
|
||
| Args: | ||
| json_file (str): full path to the json file in COCO instances annotation format. | ||
| image_root (str or path-like): the directory where the images in this json file exists. | ||
| dataset_name (str): the name of the dataset (e.g., coco_2017_train). | ||
| If provided, this function will also put "thing_classes" into | ||
| the metadata associated with this dataset. | ||
| extra_annotation_keys (list[str]): list of per-annotation keys that should also be | ||
| loaded into the dataset dict (besides "iscrowd", "bbox", "keypoints", | ||
| "category_id", "segmentation"). The values for these keys will be returned as-is. | ||
| For example, the densepose annotations are loaded in this way. | ||
|
|
||
| Returns: | ||
| list[dict]: a list of dicts in Detectron2 standard dataset dicts format. (See | ||
| `Using Custom Datasets </tutorials/datasets.html>`_ ) |
There was a problem hiding this comment.
coco json 불러와서 list of dicts 만드는 부분
Comment on lines
+1
to
+7
| # Copyright (c) Facebook, Inc. and its affiliates. | ||
| import logging | ||
| import torch | ||
| import torch.distributed as dist | ||
| import torch.multiprocessing as mp | ||
|
|
||
| from detectron2.utils import comm |
| return port | ||
|
|
||
|
|
||
| def launch(main_func, num_gpus_per_machine, num_machines=1, machine_rank=0, dist_url=None, args=()): |
Comment on lines
+40
to
+52
| world_size = num_machines * num_gpus_per_machine | ||
| if world_size > 1: | ||
| # https://github.com/pytorch/pytorch/pull/14391 | ||
| # TODO prctl in spawned processes | ||
|
|
||
| if dist_url == "auto": | ||
| assert num_machines == 1, "dist_url=auto not supported in multi-machine jobs." | ||
| port = _find_free_port() | ||
| dist_url = f"tcp://127.0.0.1:{port}" | ||
| if num_machines > 1 and dist_url.startswith("file://"): | ||
| logger = logging.getLogger(__name__) | ||
| logger.warning( | ||
| "file:// is not a reliable init_method in multi-machine jobs. Prefer tcp://" |
There was a problem hiding this comment.
launch내에 들어갈 dist_url 찾아주는 부분
Comment on lines
+55
to
+60
| mp.spawn( | ||
| _distributed_worker, | ||
| nprocs=num_gpus_per_machine, | ||
| args=(main_func, world_size, num_gpus_per_machine, machine_rank, dist_url, args), | ||
| daemon=False, | ||
| ) |
There was a problem hiding this comment.
torch.multiprocessing.spawn
프로세스를 그냥 실행 하는걸 spawn이라 부르는듯. _distributed_worker 들한테 main_func를 실행해라~고 주는 명령어
Comment on lines
+65
to
+94
| def _distributed_worker( | ||
| local_rank, main_func, world_size, num_gpus_per_machine, machine_rank, dist_url, args | ||
| ): | ||
| assert torch.cuda.is_available(), "cuda is not available. Please check your installation." | ||
| global_rank = machine_rank * num_gpus_per_machine + local_rank | ||
| try: | ||
| dist.init_process_group( | ||
| backend="NCCL", init_method=dist_url, world_size=world_size, rank=global_rank | ||
| ) | ||
| except Exception as e: | ||
| logger = logging.getLogger(__name__) | ||
| logger.error("Process group URL: {}".format(dist_url)) | ||
| raise e | ||
| # synchronize is needed here to prevent a possible timeout after calling init_process_group | ||
| # See: https://github.com/facebookresearch/maskrcnn-benchmark/issues/172 | ||
| comm.synchronize() | ||
|
|
||
| assert num_gpus_per_machine <= torch.cuda.device_count() | ||
| torch.cuda.set_device(local_rank) | ||
|
|
||
| # Setup the local process group (which contains ranks within the same machine) | ||
| assert comm._LOCAL_PROCESS_GROUP is None | ||
| num_machines = world_size // num_gpus_per_machine | ||
| for i in range(num_machines): | ||
| ranks_on_i = list(range(i * num_gpus_per_machine, (i + 1) * num_gpus_per_machine)) | ||
| pg = dist.new_group(ranks_on_i) | ||
| if i == machine_rank: | ||
| comm._LOCAL_PROCESS_GROUP = pg | ||
|
|
||
| main_func(*args) |
There was a problem hiding this comment.
_distributed_worker 구하는 부분
- init_process_group : https://pytorch.org/docs/stable/distributed.html#torch.distributed.init_process_group 그냥 distributed process 처음에 생성하는 부분
- comm.synchronize : timeout을 방지하려면 barrier라는걸 쓸 수 있나봄. How does the
synchronizefunction work? facebookresearch/maskrcnn-benchmark#172
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.