Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Supplier] Implement supplier service activation #707

Merged
merged 43 commits into from
Aug 2, 2024

Conversation

red-0ne
Copy link
Contributor

@red-0ne red-0ne commented Jul 26, 2024

Summary

This PR modifies the functionality so that suppliers staking for services mid-session will become active and be included in Session.Suppliers starting from the next session.

It adds the property ServiceActivationHeight which is a map of serviceId to activationHeight to the Supplier type and supports activation for both staking and restaking.

A remaining issue, is the immediate effect of removing a service when a Supplier restakes.

~550 LOC are protobuf generated code

Issue

When a Supplier stakes or restakes mid-session, the session endpoint may include that supplier before the next session starts. This can lead to inconsistencies between session queries and clients that fetch Session data only when a new session begins.

Type of change

Select one or more:

  • New feature, functionality or library
  • Bug fix
  • Code health or cleanup
  • Documentation
  • Other (specify)

Testing

Documentation changes (only if making doc changes)

  • make docusaurus_start; only needed if you make doc changes

Local Testing (only if making code changes)

  • Unit Tests: make go_develop_and_test
  • LocalNet E2E Tests: make test_e2e
  • See quickstart guide for instructions

PR Testing (only if making code changes)

  • DevNet E2E Tests: Add the devnet-test-e2e label to the PR.
    • THIS IS VERY EXPENSIVE, so only do it after all the reviews are complete.
    • Optionally run make trigger_ci if you want to re-trigger tests without any code changes
    • If tests fail, try re-running failed tests only using the GitHub UI as shown here

Sanity Checklist

  • I have tested my changes using the available tooling
  • I have commented my code
  • I have performed a self-review of my own code; both comments & source code
  • I create and reference any new tickets, if applicable
  • I have left TODOs throughout the codebase, if applicable

Summary by CodeRabbit

  • New Features

    • Introduced a mapping for service activation heights in the Supplier structure, allowing tracking of service activation timings.
    • Added new configuration parameter for controlling resource allocation in relay services.
    • New function to retrieve the next session start height for improved session management.
  • Bug Fixes

    • Modified logic in supplier activity checks to enhance validation by incorporating service-specific conditions.
  • Tests

    • Refactored test suite for supplier management with new helper functions and additional test cases to cover new scenarios.
  • Chores

    • Enhanced mock generation directives to include session management interfaces for improved testing capabilities.

@red-0ne red-0ne added supplier Changes related to the Supplier actor session Changes related to Session management labels Jul 26, 2024
@Olshansk Olshansk added this to the Shannon Beta TestNet Launch milestone Jul 30, 2024
Copy link
Member

@Olshansk Olshansk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small NIT. Will approve after we merge the base PR in just to sanity check the changes.


return nil
}

// getNextSessionStartHeight returns the current height's next session start height.
func getNextSessionStartHeight(ctx context.Context, sharedParams *sharedtypes.Params) int64 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OPTIONAL NIT: Wdyt of beginning to have functions like this be receiver functions of sharedtypes.Params so they're easier to reuse?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather have it at x/shared/session.go and pass in the query height.

The current getNextSessionStartHeight uses sdkCtx to get the current height which makes it non-portable to non on-chain use cases.

GetNextSessionStartHeight(sharedParams *sharedtypes.Params, queryHeight int64) int64

@red-0ne red-0ne requested a review from Olshansk July 31, 2024 20:06
@@ -155,3 +155,9 @@ func GetEarliestSupplierProofCommitHeight(
//return proofWindowOpenHeight + randCreateProofHeightOffset
return proofWindowOpenHeight
}

// GetNextSessionStartHeight returns the start block height of the session
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great comment!

@red-0ne red-0ne changed the base branch from feat/unbonding-gov-param to main August 2, 2024 18:15
Copy link

github-actions bot commented Aug 2, 2024

The CI will now also run the e2e tests on devnet, which increases the time it takes to complete all CI checks.

You may need to run make trigger_ci to submit an empty commit that'll trigger the tests.

GCP workloads (requires changing the namespace to 707)
Grafana network dashboard for devnet-issue-{issue-id}

@github-actions github-actions bot added devnet push-image CI related - pushes images to ghcr.io labels Aug 2, 2024
Copy link

coderabbitai bot commented Aug 2, 2024

Walkthrough

This update introduces significant enhancements across various components, primarily focusing on the Supplier structure and its management within the API. Key changes include the addition of a ServicesActivationHeightsMap to track service activation heights, improvements in the test suite functionality, and expanded session management capabilities. These modifications enhance clarity, usability, and maintainability, ensuring better tracking of supplier states and interactions with services.

Changes

Files Change Summary
api/poktroll/shared/supplier.pulsar.go, proto/poktroll/shared/supplier.proto Added ServicesActivationHeightsMap to the Supplier struct for tracking service activation heights; updated protobuf definitions accordingly.
e2e/tests/init_test.go, e2e/tests/stake_supplier.feature Refactored test methods for clarity; added new tests for supplier staking and session validation, enhancing coverage of supplier management scenarios.
testutil/network/network.go Updated genesis state initialization to include ServicesActivationHeightsMap, improving clarity and consistency in service identifier handling.
x/session/keeper/query_get_session.go, x/session/keeper/session_hydrator.go Enhanced session querying logic for flexible block height requests and refined supplier activity checks during session hydration.
x/shared/session.go, x/shared/types/supplier.go Introduced GetNextSessionStartHeight function for session management; updated IsActive method to account for service-specific activation logic.
x/supplier/keeper/msg_server_stake_supplier.go, x/supplier/keeper/msg_server_stake_supplier_test.go Modified methods to manage service activation heights; integrated a helper function for test message preparation, enhancing test clarity and reusability.
x/supplier/types/expected_keepers.go Added SessionKeeper interface for managing session-related functionality and updated mock generation directives accordingly.
config.yml Added compute_units_per_relay parameter for anvil and ollama services, enhancing configuration options for resource management.

Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 6bafce7 and aec4956.

Files selected for processing (1)
  • config.yml (1 hunks)
Additional comments not posted (2)
config.yml (2)

163-163: LGTM!

The compute_units_per_relay parameter is correctly added under the anvil service.


171-171: LGTM!

The compute_units_per_relay parameter is correctly added under the ollama service.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 11

Outside diff range, codebase verification and nitpick comments (2)
x/session/keeper/session_hydrator.go (1)

171-171: Function hydrateSessionSuppliers is not integrated into the main application code.

The function hydrateSessionSuppliers is only referenced within its own file, session_hydrator.go, and does not appear to be utilized in the broader application logic.

  • Ensure that the function is properly called and integrated within the main application code.
Analysis chain

LGTM! Verify the function integration.

The modification improves the validation of supplier activity.

However, ensure that the function hydrateSessionSuppliers is properly integrated and executed.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the function `hydrateSessionSuppliers` is properly integrated and executed.

# Test: Search for the function usage. Expect: Proper integration in the codebase.
rg --type go -A 5 $'hydrateSessionSuppliers'

Length of output: 1194


Script:

#!/bin/bash
# Description: Verify the usage of `session_hydrator.go` or its functions in the broader codebase.

# Test: Search for references to `session_hydrator` or any functions within the file.
rg --type go -A 5 'session_hydrator' | grep -v 'session_hydrator.go'

Length of output: 1115


Script:

#!/bin/bash
# Description: Verify the usage of `hydrateSessionSuppliers` function or `session_hydrator` struct in the main application code.

# Test: Search for references to `hydrateSessionSuppliers` or `session_hydrator` in the main application code.
rg --type go -A 5 'hydrateSessionSuppliers|session_hydrator' | grep -v 'test'

Length of output: 1234

api/poktroll/shared/supplier.pulsar.go (1)

1016-1019: Document the new field ServicesActivationHeightsMap.

The new field ServicesActivationHeightsMap should be documented clearly to explain its purpose and usage.

// services_activation_heights_map is a map of serviceIds to the height at
// which the staked supplier will become active for that service.
// Activation heights are session start heights.
ServicesActivationHeightsMap map[string]uint64 `protobuf:"bytes,5,rep,name=services_activation_heights_map,json=servicesActivationHeightsMap,proto3" json:"services_activation_heights_map,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"varint,2,opt,name=value,proto3"`

Comment on lines +71 to +73
type _Supplier_5_map struct {
m *map[string]uint64
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure proper initialization of the map.

The _Supplier_5_map struct should ensure the map is properly initialized to avoid nil pointer dereferences.

type _Supplier_5_map struct {
	m *map[string]uint64
}

func new_Supplier_5_map() *_Supplier_5_map {
	m := make(map[string]uint64)
	return &_Supplier_5_map{m: &m}
}

Comment on lines +75 to +80
func (x *_Supplier_5_map) Len() int {
if x.m == nil {
return 0
}
return len(*x.m)
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check for nil map before accessing length.

The Len method should handle the case where the map is nil.

func (x *_Supplier_5_map) Len() int {
	if x.m == nil {
		return 0
	}
	return len(*x.m)
}

Comment on lines +82 to +93
func (x *_Supplier_5_map) Range(f func(protoreflect.MapKey, protoreflect.Value) bool) {
if x.m == nil {
return
}
for k, v := range *x.m {
mapKey := (protoreflect.MapKey)(protoreflect.ValueOfString(k))
mapValue := protoreflect.ValueOfUint64(v)
if !f(mapKey, mapValue) {
break
}
}
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check for nil map before ranging.

The Range method should handle the case where the map is nil.

func (x *_Supplier_5_map) Range(f func(protoreflect.MapKey, protoreflect.Value) bool) {
	if x.m == nil {
		return
	}
	for k, v := range *x.m {
		mapKey := (protoreflect.MapKey)(protoreflect.ValueOfString(k))
		mapValue := protoreflect.ValueOfUint64(v)
		if !f(mapKey, mapValue) {
			break
		}
	}
}

Comment on lines +95 to +103
func (x *_Supplier_5_map) Has(key protoreflect.MapKey) bool {
if x.m == nil {
return false
}
keyUnwrapped := key.String()
concreteValue := keyUnwrapped
_, ok := (*x.m)[concreteValue]
return ok
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check for nil map before checking key existence.

The Has method should handle the case where the map is nil.

func (x *_Supplier_5_map) Has(key protoreflect.MapKey) bool {
	if x.m == nil {
		return false
	}
	keyUnwrapped := key.String()
	concreteValue := keyUnwrapped
	_, ok := (*x.m)[concreteValue]
	return ok
}

Comment on lines +105 to +112
func (x *_Supplier_5_map) Clear(key protoreflect.MapKey) {
if x.m == nil {
return
}
keyUnwrapped := key.String()
concreteKey := keyUnwrapped
delete(*x.m, concreteKey)
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check for nil map before deleting a key.

The Clear method should handle the case where the map is nil.

func (x *_Supplier_5_map) Clear(key protoreflect.MapKey) {
	if x.m == nil {
		return
	}
	keyUnwrapped := key.String()
	concreteKey := keyUnwrapped
	delete(*x.m, concreteKey)
}

Comment on lines +127 to +136
func (x *_Supplier_5_map) Set(key protoreflect.MapKey, value protoreflect.Value) {
if !key.IsValid() || !value.IsValid() {
panic("invalid key or value provided")
}
keyUnwrapped := key.String()
concreteKey := keyUnwrapped
valueUnwrapped := value.Uint()
concreteValue := valueUnwrapped
(*x.m)[concreteKey] = concreteValue
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handle invalid key or value in Set method.

The Set method should handle invalid keys or values gracefully.

func (x *_Supplier_5_map) Set(key protoreflect.MapKey, value protoreflect.Value) {
	if !key.IsValid() || !value.IsValid() {
		panic("invalid key or value provided")
	}
	keyUnwrapped := key.String()
	concreteKey := keyUnwrapped
	valueUnwrapped := value.Uint()
	concreteValue := valueUnwrapped
	(*x.m)[concreteKey] = concreteValue
}

Comment on lines +138 to +140
func (x *_Supplier_5_map) Mutable(key protoreflect.MapKey) protoreflect.Value {
panic("should not call Mutable on protoreflect.Map whose value is not of type protoreflect.Message")
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid panic in Mutable method.

The Mutable method should avoid panicking and provide a meaningful error message.

func (x *_Supplier_5_map) Mutable(key protoreflect.MapKey) protoreflect.Value {
	panic("should not call Mutable on protoreflect.Map whose value is not of type protoreflect.Message")
}

Comment on lines +142 to +145
func (x *_Supplier_5_map) NewValue() protoreflect.Value {
v := uint64(0)
return protoreflect.ValueOfUint64(v)
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initialize new map value correctly.

The NewValue method should initialize a new map value correctly.

func (x *_Supplier_5_map) NewValue() protoreflect.Value {
	v := uint64(0)
	return protoreflect.ValueOfUint64(v)
}

Comment on lines +147 to +149
func (x *_Supplier_5_map) IsValid() bool {
return x.m != nil
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check for nil map in IsValid method.

The IsValid method should handle the case where the map is nil.

func (x *_Supplier_5_map) IsValid() bool {
	return x.m != nil
}

Comment on lines +1070 to +1075
func (x *Supplier) GetServicesActivationHeightsMap() map[string]uint64 {
if x != nil {
return x.ServicesActivationHeightsMap
}
return nil
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check for nil map in GetServicesActivationHeightsMap method.

The GetServicesActivationHeightsMap method should handle the case where the map is nil.

func (x *Supplier) GetServicesActivationHeightsMap() map[string]uint64 {
	if x != nil {
		return x.ServicesActivationHeightsMap
	}
	return nil
}

@red-0ne red-0ne merged commit 57ced24 into main Aug 2, 2024
10 checks passed
okdas pushed a commit that referenced this pull request Nov 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devnet devnet-test-e2e push-image CI related - pushes images to ghcr.io session Changes related to Session management supplier Changes related to the Supplier actor
Projects
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

2 participants