When directly use API to write to pack, if the content is already there, the content will duplicate in the pack. It was to avoid to read and write twice for computing the hash.
> Maybe the way would be to stream anyways to sandbox first, and then copy to the pack. But then you are going to write most data twice.
For the loose to pack, it is not a problem since hash is there already. For directly write to pack, I think it will be a good feature to have a option key to support write to sandbox first. So users can choose to have a more disk space optimized solution or a more performance solution.
Maybe there is also a way that the file handler move the pointer in file after it finds the content already exist, not sure but would be interesting to check. But anyway it is another issue out the scope of this PR. I'll open an issue here and not contaminate with the discussion.
Originally posted by @unkcpz in #179 (comment)
When directly use API to write to pack, if the content is already there, the content will duplicate in the pack. It was to avoid to read and write twice for computing the hash.
For the loose to pack, it is not a problem since hash is there already. For directly write to pack, I think it will be a good feature to have a option key to support write to sandbox first. So users can choose to have a more disk space optimized solution or a more performance solution.
Maybe there is also a way that the file handler move the pointer in file after it finds the content already exist, not sure but would be interesting to check. But anyway it is another issue out the scope of this PR. I'll open an issue here and not contaminate with the discussion.
Originally posted by @unkcpz in #179 (comment)