Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new routing to "add" a new file #2

Open
keighrim opened this issue Jun 20, 2023 · 1 comment
Open

Add a new routing to "add" a new file #2

keighrim opened this issue Jun 20, 2023 · 1 comment
Labels
✨N New feature or request

Comments

@keighrim
Copy link
Member

We'd like to have a routing to "obtain" a new file, given GUID and file type, within the main webapp. At the moment, we haven't figure out how to actually obtain new files from AAPB API, so for now the function can just create some dummy files under some random subdirectory.
But the newly obtained file should be indexed into the DB, and can be retrieved via mapper route.

@keighrim keighrim added the ✨N New feature or request label Jun 20, 2023
@keighrim
Copy link
Member Author

keighrim commented Jun 20, 2023

A few things to consider:

  • "obtain" function can take a fair amount of time when actually downloading a few hundreds MB (or a few GB) file. Hence we might need to consider a internal function that performs asynchronous download in the future.
  • we can change the mapper function to call the "obtain" function when the requested GUID is not found in the DB. Combining this idea with the above, handling async download can be tricky if mapper route needs to immediately response.
  • At the moment, we have <5,000 video files in our data server, but in the future, when we "obtain" too many files, we might need to delete some files to fit in the disk space. We need to decide a caching strategy for that (LRU? LFU? FIFO?) but for any kind of caching, we probably need a few additional columns in the DB scheme to record "obtain" time and "last-used" time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✨N New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant