Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] [Feature] add batch delete execution_phase api #6267

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

taieeuu
Copy link
Contributor

@taieeuu taieeuu commented Feb 22, 2025

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Labels

Please add one or more of the following labels to categorize your PR:

  • added: For new features.
  • changed: For changes in existing functionality.
  • deprecated: For soon-to-be-removed features.
  • removed: For features being removed.
  • fixed: For any bug fixed.
  • security: In case of vulnerabilities

This is important to improve the readability of release notes.

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

Summary by Bito

This PR introduces a new API endpoint for batch deleting execution phases in Flyte, including new protobuf message types (ExecutionPhaseDeleteRequest/Response) and corresponding implementations across multiple languages. The changes encompass the addition of DeleteExecutionPhase functionality in the execution manager, repository layer modifications, and gRPC service definitions with HTTP gateway handlers. The implementation includes proper error handling, validation, and dependency cleanup.

Unit tests added: True

Estimated effort to review (1-5, lower is better): 5

@flyte-bot
Copy link
Collaborator

flyte-bot commented Feb 22, 2025

Code Review Agent Run #50b8fe

Actionable Suggestions - 8
  • flyteidl/gen/pb-go/flyteidl/admin/execution.pb.go - 1
  • flyteadmin/pkg/rpc/adminservice/execution.go - 1
    • Consider using specific metric for DeleteExecutionPhase · Line 160-161
  • flyteadmin/pkg/repositories/gormimpl/execution_repo.go - 1
    • Consider adding validation for execution phase · Line 175-179
  • flyteidl/clients/go/admin/mocks/AdminServiceClient.go - 1
  • flyteidl/protos/flyteidl/service/admin.proto - 1
  • flyteadmin/pkg/manager/impl/execution_manager.go - 1
  • flyteidl/gen/pb-go/gateway/flyteidl/service/admin.pb.gw.go - 1
  • flyteidl/gen/pb-go/flyteidl/service/admin_grpc.pb.go - 1
Additional Suggestions - 5
  • flyteidl/gen/pb_python/flyteidl/service/admin_pb2_grpc.py - 1
  • flyteidl/gen/pb-go/gateway/flyteidl/service/admin.pb.gw.go - 1
  • flyteidl/gen/pb-go/flyteidl/service/admin_grpc.pb.go - 2
    • Consider adding matching interface method definition · Line 79-79
    • Consider adding input parameter validation · Line 717-724
  • flyteidl/gen/pb-js/flyteidl.d.ts - 1
Review Details
  • Files reviewed - 25 · Commit Range: 48c547a..48c547a
    • flyteadmin/pkg/manager/impl/execution_manager.go
    • flyteadmin/pkg/manager/interfaces/execution.go
    • flyteadmin/pkg/repositories/gormimpl/execution_repo.go
    • flyteadmin/pkg/repositories/interfaces/execution_repo.go
    • flyteadmin/pkg/rpc/adminservice/execution.go
    • flyteidl/clients/go/admin/mocks/AdminServiceClient.go
    • flyteidl/clients/go/admin/mocks/AdminServiceServer.go
    • flyteidl/gen/pb-es/flyteidl/admin/execution_pb.ts
    • flyteidl/gen/pb-es/flyteidl/service/admin_connect.ts
    • flyteidl/gen/pb-go/flyteidl/admin/execution.pb.go
    • flyteidl/gen/pb-go/flyteidl/service/admin.pb.go
    • flyteidl/gen/pb-go/flyteidl/service/admin_grpc.pb.go
    • flyteidl/gen/pb-go/gateway/flyteidl/service/admin.pb.gw.go
    • flyteidl/gen/pb-js/flyteidl.d.ts
    • flyteidl/gen/pb-js/flyteidl.js
    • flyteidl/gen/pb_python/flyteidl/admin/execution_pb2.py
    • flyteidl/gen/pb_python/flyteidl/admin/execution_pb2.pyi
    • flyteidl/gen/pb_python/flyteidl/service/admin_pb2.py
    • flyteidl/gen/pb_python/flyteidl/service/admin_pb2_grpc.py
    • flyteidl/gen/pb_rust/flyteidl.admin.rs
    • flyteidl/gen/pb_rust/flyteidl.service.tonic.rs
    • flyteidl/protos/flyteidl/admin/execution.proto
    • flyteidl/protos/flyteidl/service/admin.proto
    • go.mod
    • go.sum
  • Files skipped - 2
    • flyteidl/clients/go/assets/admin.swagger.json - Reason: Filter setting
    • flyteidl/gen/pb-go/gateway/flyteidl/service/admin.swagger.json - Reason: Filter setting
  • Tools
    • Golangci-lint (Linter) - ✖︎ Failed
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

AI Code Review powered by Bito Logo

@taieeuu taieeuu changed the title [WPI] Feat: add batch delete execution_phase api [WIP] Feat: add batch delete execution_phase api Feb 22, 2025
@taieeuu taieeuu changed the title [WIP] Feat: add batch delete execution_phase api [WIP] [Feature] add batch delete execution_phase api Feb 22, 2025
Copy link

codecov bot commented Feb 22, 2025

Codecov Report

Attention: Patch coverage is 0.42373% with 235 lines in your changes missing coverage. Please review.

Project coverage is 33.28%. Comparing base (ee919f8) to head (48c547a).
Report is 8 commits behind head on master.

Files with missing lines Patch % Lines
.../gen/pb-go/gateway/flyteidl/service/admin.pb.gw.go 0.00% 81 Missing ⚠️
flyteidl/gen/pb-go/flyteidl/admin/execution.pb.go 1.36% 72 Missing ⚠️
...teidl/clients/go/admin/mocks/AdminServiceClient.go 0.00% 32 Missing ⚠️
...teidl/clients/go/admin/mocks/AdminServiceServer.go 0.00% 25 Missing ⚠️
...yteidl/gen/pb-go/flyteidl/service/admin_grpc.pb.go 0.00% 25 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6267      +/-   ##
==========================================
- Coverage   36.86%   33.28%   -3.59%     
==========================================
  Files        1318     1020     -298     
  Lines      134767   109390   -25377     
==========================================
- Hits        49684    36410   -13274     
+ Misses      80753    69754   -10999     
+ Partials     4330     3226    -1104     
Flag Coverage Δ
unittests-datacatalog 51.58% <ø> (ø)
unittests-flyteadmin ?
unittests-flytecopilot 30.99% <ø> (ø)
unittests-flytectl 62.29% <ø> (ø)
unittests-flyteidl 7.18% <0.42%> (-0.05%) ⬇️
unittests-flyteplugins 53.98% <ø> (-0.05%) ⬇️
unittests-flytepropeller 42.86% <ø> (+0.07%) ⬆️
unittests-flytestdlib 55.33% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@flyte-bot
Copy link
Collaborator

Changelist by Bito

This pull request implements the following key changes.

Key Change Files Impacted
New Feature - Add Execution Phase Delete API

execution.proto - Added new ExecutionPhaseDelete request/response message definitions

admin.proto - Added DeleteExecutionPhase RPC endpoint with HTTP mapping

admin_pb2.py - Updated Python protobuf definitions with DeleteExecutionPhase service method

admin_pb2_grpc.py - Added Python gRPC service code for DeleteExecutionPhase

flyteidl.admin.rs - Added Rust protobuf definitions for ExecutionPhaseDelete messages

flyteidl.service.tonic.rs - Added Rust gRPC service code for DeleteExecutionPhase

Other Improvements - Dependency Cleanup

go.mod - Removed multiple unused indirect dependencies

go.sum - Updated checksums after removing unused dependencies

sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields

ExecutionPhase string `protobuf:"bytes,1,opt,name=execution_phase,json=executionPhase,proto3" json:"execution_phase,omitempty"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding execution phase validation

Consider adding validation for the execution_phase field in ExecutionPhaseDeleteRequest. The field appears to be used for deleting execution phases but lacks input validation to ensure valid phase values are provided. This could potentially lead to runtime errors if invalid phases are specified.

Code suggestion
Check the AI-generated fix before applying
Suggested change
ExecutionPhase string `protobuf:"bytes,1,opt,name=execution_phase,json=executionPhase,proto3" json:"execution_phase,omitempty"`
// +required The phase of execution to be deleted. Must be one of the valid execution phases.
ExecutionPhase string `protobuf:"bytes,1,opt,name=execution_phase,json=executionPhase,proto3" json:"execution_phase,omitempty" validate:"required,oneof=QUEUED RUNNING SUCCEEDED FAILED ABORTED"`

Code Review Run #50b8fe


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment on lines +160 to +161
m.Metrics.executionEndpointMetrics.terminate.Time(func() {
response, err = m.ExecutionManager.DeleteExecutionPhase(ctx, request)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using specific metric for DeleteExecutionPhase

Consider using a more specific metric name for DeleteExecutionPhase. Currently using terminate metric which seems to be shared with termination functionality. This could lead to misleading metrics.

Code suggestion
Check the AI-generated fix before applying
 	terminate   util.RequestMetrics
 +	deletePhase util.RequestMetrics
 @@ -160,9 +160,9 @@
 -	m.Metrics.executionEndpointMetrics.terminate.Time(func() {
 -		response, err = m.ExecutionManager.DeleteExecutionPhase(ctx, request)
 -	})
 -	if err != nil {
 -		return nil, util.TransformAndRecordError(err, &m.Metrics.executionEndpointMetrics.terminate)
 -	}
 -	m.Metrics.executionEndpointMetrics.terminate.Success()
 +	m.Metrics.executionEndpointMetrics.deletePhase.Time(func() {
 +		response, err = m.ExecutionManager.DeleteExecutionPhase(ctx, request)
 +	})
 +	if err != nil {
 +		return nil, util.TransformAndRecordError(err, &m.Metrics.executionEndpointMetrics.deletePhase)
 +	}
 +	m.Metrics.executionEndpointMetrics.deletePhase.Success()

Code Review Run #50b8fe


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment on lines +175 to +179
result := r.db.Delete(&models.Execution{}, "phase = ?", executionPhase)
if result.Error != nil {
return result.Error
}
return nil
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding validation for execution phase

Consider adding validation to ensure executionPhase is not empty and matches valid execution phase values. Also, consider adding error handling for when no records are deleted.

Code suggestion
Check the AI-generated fix before applying
Suggested change
result := r.db.Delete(&models.Execution{}, "phase = ?", executionPhase)
if result.Error != nil {
return result.Error
}
return nil
if executionPhase == "" {
return fmt.Errorf("execution phase cannot be empty")
}
result := r.db.Delete(&models.Execution{}, "phase = ?", executionPhase)
if result.Error != nil {
return result.Error
}
if result.RowsAffected == 0 {
return fmt.Errorf("no executions found with phase %s", executionPhase)
}
return nil

Code Review Run #50b8fe


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment on lines +376 to +379
_va := make([]interface{}, len(opts))
for _i := range opts {
_va[_i] = opts[_i]
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding nil check for opts

Consider adding validation for the opts parameter in the DeleteExecutionPhase method. Currently, there's no check if opts is nil before iterating over it.

Code suggestion
Check the AI-generated fix before applying
Suggested change
_va := make([]interface{}, len(opts))
for _i := range opts {
_va[_i] = opts[_i]
}
_va := []interface{}{}
if opts != nil {
_va = make([]interface{}, len(opts))
for _i := range opts {
_va[_i] = opts[_i]
}
}

Code Review Run #50b8fe


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment on lines +671 to +672
get: "/api/v1/delete_execution_phase/{execution_phase}"
};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using DELETE instead of GET

The DeleteExecutionPhase endpoint is using GET HTTP method for a deletion operation. Consider using DELETE HTTP method instead to better align with REST conventions and make the API more intuitive.

Code suggestion
Check the AI-generated fix before applying
Suggested change
get: "/api/v1/delete_execution_phase/{execution_phase}"
};
delete: "/api/v1/delete_execution_phase/{execution_phase}"
};

Code Review Run #50b8fe


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them


err := m.db.ExecutionRepo().Delete(ctx, executionPhase)
if err != nil {
return nil, err
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding context to error return

Consider adding error wrapping with context when returning the database error. This would help with debugging by providing more context about what operation failed. Maybe something like: fmt.Errorf("failed to delete execution phase %s: %w", executionPhase, err)

Code suggestion
Check the AI-generated fix before applying
Suggested change
return nil, err
return nil, fmt.Errorf("failed to delete execution phase %s: %w", executionPhase, err)

Code Review Run #50b8fe


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment on lines +5811 to +5814
protoReq.ExecutionPhase, err = runtime.String(val)
if err != nil {
return nil, metadata, status.Errorf(codes.InvalidArgument, "type mismatch, parameter: %s, error: %v", "execution_phase", err)
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding empty string validation

Consider adding validation for empty string in execution_phase parameter. The current implementation accepts empty strings which could lead to unintended behavior when deleting execution phases.

Code suggestion
Check the AI-generated fix before applying
Suggested change
protoReq.ExecutionPhase, err = runtime.String(val)
if err != nil {
return nil, metadata, status.Errorf(codes.InvalidArgument, "type mismatch, parameter: %s, error: %v", "execution_phase", err)
}
protoReq.ExecutionPhase, err = runtime.String(val)
if err != nil {
return nil, metadata, status.Errorf(codes.InvalidArgument, "type mismatch, parameter: %s, error: %v", "execution_phase", err)
}
if protoReq.ExecutionPhase == "" {
return nil, metadata, status.Errorf(codes.InvalidArgument, "parameter %s cannot be empty", "execution_phase")
}

Code Review Run #50b8fe


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment on lines +2045 to +2047
in := new(admin.ExecutionPhaseDeleteRequest)
if err := dec(in); err != nil {
return nil, err
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding nil check for request

Consider adding error validation in the _AdminService_DeleteExecutionPhase_Handler function to ensure the input request is not nil before proceeding with the decoding operation.

Code suggestion
Check the AI-generated fix before applying
Suggested change
in := new(admin.ExecutionPhaseDeleteRequest)
if err := dec(in); err != nil {
return nil, err
if srv == nil {
return nil, status.Error(codes.InvalidArgument, "server interface is nil")
}
in := new(admin.ExecutionPhaseDeleteRequest)
if err := dec(in); err != nil {
return nil, err

Code Review Run #50b8fe


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants