Skip to content

Refactor neighbor lists and alphas to be more memory efficient #205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kuberry opened this issue Jun 8, 2020 · 2 comments
Closed

Refactor neighbor lists and alphas to be more memory efficient #205

kuberry opened this issue Jun 8, 2020 · 2 comments
Assignees
Labels

Comments

@kuberry
Copy link
Collaborator

kuberry commented Jun 8, 2020

Currently, the size of the neighbors lists 2D array is (# targets x max number of neighbors + 1). The alphas generated by GMLS have a similar, though not exactly the same, structure.

If one target site has significantly more neighbors than others, this can create quite a large amount of wasted memory. Ideally, this should be stored similar to how a compressed row storage graph is stored, with offsets into each row.

Alphas sizes are known in advance of their generation because they are directly related to neighbor lists sizes, so it is possible to switch this for alphas without creating more computational effort. However, for the neighbor lists, it is unknown at search time how many neighbors will be found, so either a.) do a search to get the number of neighbors needed, or b.) batch the neighbor search and temporarily store the batch inefficiently but then move it to a persistent location in an efficiently pack storage shape.

a.) is generally called a 'dry-run' and is already supported by the code. Currently though, it only identifies the maximum number of neighbors over all sites, which is sufficient for the current method of storage but would not be for this improved storage design.

@kuberry
Copy link
Collaborator Author

kuberry commented Jun 15, 2020

Addressed by PR #208

@kuberry
Copy link
Collaborator Author

kuberry commented Jun 15, 2020

Neighbor lists are now the same size as the sum of the number of neighbors needed by each target, and alphas similarly so. Previously they were number of targets * maximum number of neighbors over all targets.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant