-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: race condition in both upsert post and upsert patch #88
base: main
Are you sure you want to change the base?
Conversation
Pull Request Test Coverage Report for Build 13215384412Details
💛 - Coveralls |
@@ -598,12 +610,15 @@ func (r *AssetRepository) insert(ctx context.Context, ast *asset.Asset) (string, | |||
return id, nil | |||
} | |||
|
|||
func (r *AssetRepository) update(ctx context.Context, assetID string, newAsset, oldAsset *asset.Asset, clog diff.Changelog) error { | |||
func (r *AssetRepository) update(ctx context.Context, tx *sqlx.Tx, newAsset, oldAsset *asset.Asset, clog diff.Changelog) error { | |||
assetID := oldAsset.ID |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is there any case where the assetID changed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should be not. Its UUID anyway, so ideally we use URN instead for Upsert identifier
And actually assetID came from fetchedAsset
err = r.update(ctx, fetchedAsset.ID, ast, &fetchedAsset, changelog) |
@@ -176,6 +180,10 @@ func (r *AssetRepository) GetByURN(ctx context.Context, urn string) (asset.Asset | |||
} | |||
|
|||
func (r *AssetRepository) getWithPredicate(ctx context.Context, pred sq.Eq) (asset.Asset, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we can remove this function if not needed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
still used in GetAssetByID API here:
Line 213 in 7acbcda
ast, err = s.assetRepository.GetByID(ctx, id) |
When some jobs that has same recipe run at the same time, it makes those assets has the
version
inassets_versions
table greater thanversion
inassets
table. This is not expected becauseversion
inassets
table should be the (new) highest number, so should be equals to highest number of theversion
inassets_versions
. The expectation is with transaction sql (the race condition should be handled), it will insert new rows in assets_version with increasing minor version (exp: 0.9 → 0.10) IF ONLY there is an update on the job AND has the changelog.For example bug case:
The solution:
Using Row-level Locks (ref: https://www.postgresql.org/docs/current/explicit-locking.html#LOCKING-ROWS) with SELECT FOR UPDATE within same transaction for update