Implement Python DownloadProgress void* mappings

[es-fabricemarie]

This is probably not the right way to do it, or it may not even be complete. But I just want to help to move along the issue I filled and contribute back. Hopefully it will help a maintainer to get started.

Fixes #1441

[es-fabricemarie]

Tested with the following test program:

#!/usr/bin/python3
import libdnf5

class DownloadProgress(libdnf5.repo.DownloadCallbacks):
    def add_new_download(self, user_data, description, total_to_download):
        print("Just added package %s to download list" % description)
        return description

    def progress(self, user_cb_data, total_to_download, downloaded):
        print(f"progress achieved on {user_cb_data}")
        return 0

    def end(self, user_cb_data, status, msg):
        print(f"download ended for {user_cb_data}, with status={status} and msg={msg}")
        return 0


base = libdnf5.base.Base()
base.load_config()
base.setup()
sack = base.get_repo_sack()
sack.create_repos_from_system_configuration()
sack.update_and_load_enabled_repos(True)
goal = libdnf5.base.Goal(base)
goal.add_rpm_upgrade()
transaction = goal.resolve()
downloader_callbacks = DownloadProgress()
base.set_download_callbacks(
    libdnf5.repo.DownloadCallbacksUniquePtr(downloader_callbacks)
)
transaction.download()
print("Done downloading")

I get:

Just added package enforce-0:1.7-18.ev40.x86_64 to download list
Just added package entutils-0:1.3-16.ev40.x86_64 to download list
Just added package util_assets_inventory-0:1.6-22.ev40.x86_64 to download list
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on entutils-0:1.3-16.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
download ended for entutils-0:1.3-16.ev40.x86_64, with status=0 and msg=None
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on enforce-0:1.7-18.ev40.x86_64
download ended for enforce-0:1.7-18.ev40.x86_64, with status=0 and msg=None
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
progress achieved on util_assets_inventory-0:1.6-22.ev40.x86_64
download ended for util_assets_inventory-0:1.6-22.ev40.x86_64, with status=0 and msg=None
Done downloading

[jrohel]

Hi. Thank you for your work. I'm working on that too. I'm looking for a solution that would work correctly with memory and I need to solve all the languages ​​supported by the libdnf5 project - Python, Ruby, Perl.

I am considering whether there is a need to store objects in a helper container.

I looked at your code. I see you are dealing with Python and storing objects. You use std::map<void*, PyObject*> user_data_map for this. I have a question? Why are you saving the pointer twice?
key = reinterpret_cast<void *>(py_obj); -> py_obj = reinterpret_cast<PyObject *>(key);
The mapped value can be retrieved from the key using reinterpret_cast at any time. So std::set<void *> (or std::set<PyObject *>) could be enough.

[jrohel]

@es-fabricemarie
I'm closing this draft. I solved the issue some time ago with these PRs: #1849 , #1939 .

After testing, I took a different approach. Passing an integer instead of a pointer. More details are given in the PRs. In general passing pointers to void is an issue - object ownership in different languages. Your draft only tries to solve the interface for Python. It needs to work with Ruby and Perl too. And easily portable to other languages in the future.

Thanks for the draft and sorry for closing it.