Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

image similarity-based annotation for generating bootstrap data #5

Open
keighrim opened this issue Sep 26, 2023 · 0 comments
Open

image similarity-based annotation for generating bootstrap data #5

keighrim opened this issue Sep 26, 2023 · 0 comments

Comments

@keighrim
Copy link
Member

keighrim commented Sep 26, 2023

As a subtask of #4 , we'd like to experiment with the idea of having quick-and-dirty annotation environment for creating a relatively small bootstrap annotation, based on pseudo-clustering based on image similarity. Some rough sketch on the implementation.

  1. run shot detection algorithm (e.g., https://www.scenedetect.com/, https://github.com/clamsproject/app-pyscenedetect-wrapper) on a set of video (or a fixed total duration of videos)
  2. sample N frames from each "shot"
  3. apply a feature extraction on all images
  4. taking the first image (chronologically) as the reference, sort all images based on similarity to the reference image
  5. present first N similar images to annotators and ask for a label, and membership verification
  6. sort the leftover (unlabeled) images chronologically and go back to 4, until the number of remaining images drops under the threshold
  7. go to the next batch of videos and start from 1, merging new feature vectors with the leftover
@clams-bot clams-bot added this to apps Sep 26, 2023
@github-project-automation github-project-automation bot moved this to Todo in apps Sep 26, 2023
@keighrim keighrim added this to the swt-v1 milestone Oct 8, 2023
@clams-bot clams-bot removed this from the swt-v1 milestone Jan 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Todo
Development

No branches or pull requests

2 participants