Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

internal: add overlay port for MakeID library to fix download issues #2330

Merged
merged 4 commits into from
Feb 14, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/formatting.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,4 @@ jobs:
uses: jidicula/[email protected]
with:
clang-format-version: '15'
exclude-regex: '(.*skyrim-platform/src/platform_se/skyrim_platform/Concepts.h)|(.*serialization/include/concepts/.*)|(.*third_party.*)'
exclude-regex: '(.*overlay_ports.*)|(.*skyrim-platform/src/platform_se/skyrim_platform/Concepts.h)|(.*serialization/include/concepts/.*)|(.*third_party.*)'
339 changes: 339 additions & 0 deletions overlay_ports/makeid/MakeID.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,339 @@
#ifndef MAKEID_H
#define MAKEID_H

/*

Author:
Emil Persson, A.K.A. Humus.
http://www.humus.name

Version history:
1.0 - Initial release.
1.01 - Code review fixes. Code reviewed by Denis A. Gladkiy.
1.02 - Fixed an off-by-one error in DestroyRange() found by Markus
Billeter 1.03 - Added ability to reset to startup state, possible setting a new
max_id.

License:
Public Domain

This file is released in the hopes that it will be useful. Use in
whatever way you like, but no guarantees that it actually works or fits any
particular purpose. It has been unit-tested and benchmarked though, and seems
to do what it was designed to do, and seems pretty quick at it too.

Notes:
There are many applications where it is desired to generate unique IDs
at runtime for various resources, such that they can be distinguished, sorted
or otherwise processed in an efficient manner. It can in some cases replace
hashes, handles and pointers. In cases where resource pointers are used as IDs,
it offers a unique ID that requires far fewer bits, especially for 64bit apps.
The design goal of this implementation was to return the most compact
IDs as possible, limiting to a specific range if necessary.

The properties of this system are as follows:
- Creating a new ID returns the smallest possible unused ID.
- Creating a new range of IDs returns the smallest possible
continuous range of the specified size.
- Created IDs remain valid until destroyed.
- Destroying an ID returns it to the pool and may be returned
by subsequent allocations.
- The system is NOT thread-safe.

Performance properties:
- Creating an ID is O(1) and generally super-cheap.
- Destroying an ID is also cheap, but O(log(n)), where n is the
current number of distinct available ranges.
- The system merges available ranges when IDs are destroyed,
keeping said n generally very small in practice.
- After warmup, no further memory allocations should be
necessary, or be very rare.
- The system uses very little memory.
- It is possible to construct a pathological case where
fragmentation would cause n to become large. This can be done by first
allocating a very large range of IDs, then deleting every other ID, causing a
new range to be allocated for every free ID, or as many ranges as there are
free IDs. I believe nothing close to this situation happens in practical
applications. In tests, millions of random scattered creations and deletions
only resulted in a relatively short list in the worst case. This is because
freed IDs are quickly reused and ranges eagerly merged.

Where would this system be useful? It was originally thought up as a
replacement for resource pointers as part of sort-ids in rendering. Using for
instance a 64-bit sort-id packing various flags and states, putting a pointer
in there takes an awful lot of bits, especially considering the actual possible
resources range in the thousands at most. This got far worse of course with the
switch to 64bit as pointers are now twice as large and essentially eats all
bits except bottom few for alignment. Another application would be for managing
a shared pool of resources. IDs could be handed out as handles and used to
access the actual resource from an array. By always returning the lowest
possible ID or range of IDs we get very good cache behavior since all active
resources will grouped together in the bottom part of the array. Using IDs
instead of pointers for handles also allows easy resizing of the allocated
memory since IDs can remain the same even if the underlying storage changed.

*/

#include <cstdio> // For printf(). Remove if you don't need the PrintRanges() function (mostly for debugging anyway).
#include <cstdint> // uint32_t
#include <cstdlib>
#include <cstring>

class MakeID
{
private:
// Change to uint16_t here for a more compact implementation if 16bit or less
// IDs work for you.
typedef uint32_t uint;

struct Range
{
uint m_First;
uint m_Last;
};

Range* m_Ranges; // Sorted array of ranges of free IDs
uint m_Count; // Number of ranges in list
uint m_Capacity; // Total capacity of range list

MakeID& operator=(const MakeID&);
MakeID(const MakeID&);

public:
explicit MakeID(const uint max_id)
{
// Start with a single range, from 0 to max allowed ID (specified)
m_Ranges = static_cast<Range*>(::malloc(sizeof(Range)));
m_Capacity = 1;
Reset(max_id);
}

~MakeID() { ::free(m_Ranges); }

void Reset(const uint max_id)
{
m_Ranges[0].m_First = 0;
m_Ranges[0].m_Last = max_id;
m_Count = 1;
}

bool CreateID(uint& id)
{
if (m_Ranges[0].m_First <= m_Ranges[0].m_Last) {
id = m_Ranges[0].m_First;

// If current range is full and there is another one, that will become
// the new current range
if (m_Ranges[0].m_First == m_Ranges[0].m_Last && m_Count > 1) {
DestroyRange(0);
} else {
++m_Ranges[0].m_First;
}
return true;
}

// No availble ID left
return false;
}

bool CreateRangeID(uint& id, const uint count)
{
uint i = 0;
do {
const uint range_count = 1 + m_Ranges[i].m_Last - m_Ranges[i].m_First;
if (count <= range_count) {
id = m_Ranges[i].m_First;

// If current range is full and there is another one, that will become
// the new current range
if (count == range_count && i + 1 < m_Count) {
DestroyRange(i);
} else {
m_Ranges[i].m_First += count;
}
return true;
}
++i;
} while (i < m_Count);

// No range of free IDs was large enough to create the requested continuous
// ID sequence
return false;
}

bool DestroyID(const uint id) { return DestroyRangeID(id, 1); }

bool DestroyRangeID(const uint id, const uint count)
{
const uint end_id = id + count;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guard against integer overflow in the expression 'id + count'.

Suggested change
const uint end_id = id + count;
const uint end_id = (count > UINT32_MAX - id) ? UINT32_MAX : id + count;


// Binary search of the range list
uint i0 = 0;
uint i1 = m_Count - 1;

for (;;) {
const uint i = (i0 + i1) / 2;

if (id < m_Ranges[i].m_First) {
// Before current range, check if neighboring
if (end_id >= m_Ranges[i].m_First) {
if (end_id != m_Ranges[i].m_First)
return false; // Overlaps a range of free IDs, thus (at least
// partially) invalid IDs

// Neighbor id, check if neighboring previous range too
if (i > i0 && id - 1 == m_Ranges[i - 1].m_Last) {
// Merge with previous range
m_Ranges[i - 1].m_Last = m_Ranges[i].m_Last;
DestroyRange(i);
} else {
// Just grow range
m_Ranges[i].m_First = id;
}
return true;
} else {
// Non-neighbor id
if (i != i0) {
// Cull upper half of list
i1 = i - 1;
} else {
// Found our position in the list, insert the deleted range here
InsertRange(i);
m_Ranges[i].m_First = id;
m_Ranges[i].m_Last = end_id - 1;
return true;
}
}
} else if (id > m_Ranges[i].m_Last) {
// After current range, check if neighboring
if (id - 1 == m_Ranges[i].m_Last) {
// Neighbor id, check if neighboring next range too
if (i < i1 && end_id == m_Ranges[i + 1].m_First) {
// Merge with next range
m_Ranges[i].m_Last = m_Ranges[i + 1].m_Last;
DestroyRange(i + 1);
} else {
// Just grow range
m_Ranges[i].m_Last += count;
}
return true;
} else {
// Non-neighbor id
if (i != i1) {
// Cull bottom half of list
i0 = i + 1;
} else {
// Found our position in the list, insert the deleted range here
InsertRange(i + 1);
m_Ranges[i + 1].m_First = id;
m_Ranges[i + 1].m_Last = end_id - 1;
return true;
}
}
} else {
// Inside a free block, not a valid ID
return false;
}
}
}

bool IsID(const uint id) const
{
// Binary search of the range list
uint i0 = 0;
uint i1 = m_Count - 1;

for (;;) {
const uint i = (i0 + i1) / 2;

if (id < m_Ranges[i].m_First) {
if (i == i0)
return true;

// Cull upper half of list
i1 = i - 1;
} else if (id > m_Ranges[i].m_Last) {
if (i == i1)
return true;

// Cull bottom half of list
i0 = i + 1;
} else {
// Inside a free block, not a valid ID
return false;
}
}
}

uint GetAvailableIDs() const
{
uint count = m_Count;
uint i = 0;

do {
count += m_Ranges[i].m_Last - m_Ranges[i].m_First;
++i;
} while (i < m_Count);

return count;
}

uint GetLargestContinuousRange() const
{
uint max_count = 0;
uint i = 0;

do {
uint count = m_Ranges[i].m_Last - m_Ranges[i].m_First + 1;
if (count > max_count)
max_count = count;

++i;
} while (i < m_Count);

return max_count;
}

void PrintRanges() const
{
uint i = 0;
for (;;) {
if (m_Ranges[i].m_First < m_Ranges[i].m_Last)
printf("%u-%u", m_Ranges[i].m_First, m_Ranges[i].m_Last);
else if (m_Ranges[i].m_First == m_Ranges[i].m_Last)
printf("%u", m_Ranges[i].m_First);
else
printf("-");

++i;
if (i >= m_Count) {
printf("\n");
return;
}

printf(", ");
}
}

private:
void InsertRange(const uint index)
{
if (m_Count >= m_Capacity) {
m_Capacity += m_Capacity;
m_Ranges = (Range*)realloc(m_Ranges, m_Capacity * sizeof(Range));
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check realloc's return value to avoid dereferencing a null pointer on allocation failure.

Suggested change
m_Ranges = (Range*)realloc(m_Ranges, m_Capacity * sizeof(Range));
if (Range* temp = (Range*)realloc(m_Ranges, m_Capacity * sizeof(Range))) m_Ranges = temp;

}

::memmove(m_Ranges + index + 1, m_Ranges + index,
(m_Count - index) * sizeof(Range));
++m_Count;
}

void DestroyRange(const uint index)
{
--m_Count;
::memmove(m_Ranges + index, m_Ranges + index + 1,
(m_Count - index) * sizeof(Range));
}
};

#endif // MAKEID_H
24 changes: 24 additions & 0 deletions overlay_ports/makeid/portfile.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Line endings handling
if(VCPKG_TARGET_IS_WINDOWS)
set(SHA_EXPECTED b4993d53cd2b7f967982fb3a2277070a58b7f80528cffd6588280dea2e919dce710174a9a56e46a1b978786831a14bb8a9b49f93b93aa09ddcf097deb2ea73fe)
else()
set(SHA_EXPECTED cb5fffe9958915da46ada56e478c118c53e90f0ec9cfe72d992acf4b5e8353d917b44ca260f91e145eb46468357adc792bb1da48fb21d0fbec64efb27c3375b6)
endif()

vcpkg_download_distfile(ARCHIVE
URLS "file://${CMAKE_CURRENT_LIST_DIR}/MakeID.h"
FILENAME "MakeID.h-${VERSION}"
SHA512 "${SHA_EXPECTED}"
)

file(INSTALL "${ARCHIVE}" DESTINATION "${CURRENT_PACKAGES_DIR}/include" RENAME "MakeID.h")

set(license_text
"Public Domain

This file is released in the hopes that it will be useful. Use in whatever way you like, but no guarantees that it
actually works or fits any particular purpose. It has been unit-tested and benchmarked though, and seems to do
what it was designed to do, and seems pretty quick at it too."
)

file(WRITE "${CURRENT_PACKAGES_DIR}/share/makeid/copyright" "${license_text}")
7 changes: 7 additions & 0 deletions overlay_ports/makeid/vcpkg.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"name": "makeid",
"version": "1.0.3",
"description": "MakeID is a cross platform C++ library for IDs allocation/deallocation",
"homepage": "http://www.humus.name/index.php?page=3D",
"license": null
}
Loading