-
Notifications
You must be signed in to change notification settings - Fork 0
Disk management: Global GC activity counters
Previous solutions try to solve the problem by using locks in order to avoid the creation of new references during the analysis. This solution avoids any kind of locks by using Activity Counters.
Each object has an activity counter. An activity counter is a counter present in each node. Any ‘activity’ (send the object, ...) started must be registered. More concretely, every time an object is send, the activity counter of this object must be increased. Any finished activity must be registered also. So, for each object received, the activity counter of this object is decreased.
As you can see, one node can have +1 since it has send the object, and the other -1 since it has received the object.
In addition, each node has a Reference Counter per each object, which counts the number of references (in the current node) pointing to the object. References registered are:
- Alias: Any object having an alias have 1 reference pointing to it (only 1 or 0)
- Federation: Any federated object have 1 reference pointing to it (only 1 or 0)
- Other objects: Any reference in disk pointing to the object. During the update process of any object on disk, the counters are also updated.
- Sessions: Any reference from an active session. When the object is sent to a client, or is received from a client, this information is stored into a table Session ID → Objects. Any session ‘using’ the object is referencing it. When the session is closed, its table entry is removed and the reference also.
- Memory: The node is referencing the object if it is present in its memory (objects map). The reason for this is to prevent objects being removed ‘during’ the garbage collection. The object might not have any reference because it was not updated yet in disk.
This solution is based on the following statements:
- The reference counter of all nodes are zero.
- The sum of all activity counters of an object must be zero
Every once in a while, for each candidate object to be removed, the node that owns the object ‘asks’ all other nodes about the number of references pointing to the object to remove and the value of its activity counter. If there are some reference, the process is stopped. Otherwise, if there is no reference and the sum of all activity counters is zero, the object can be removed.
This solution has two main problems:
As you can see, the table that registers the ‘objects being referenced by sessions’ needs resources. In the worst case, it can occupy a lot of resources. The solution for this problem is to establish limitations. The table can grow horizontally and vertically. Therefore, we define two limitations:
- Horizontally: limit the number of concurrent sessions per node.
- Vertically: limit the number of objects each session can have on the client side. This limit will encourage users to register models that create and compute in a remote node and not sending and receiving all the time from the client side.
The limits are necessary, defining which are the limits must be explained by Resource Contention solutions.
The same problem appears in the table of Activity Counters. However, there is no solution for that and that’s the reason why Activity Counters is not a possible solution. However, if in the future we want to define an asynchronous model for dataClay, activity counters can be reconsidered.