Automated analysis of open source ecosystems based on Awesome Lists

[epicfaace]

Discussion for @Ly0n's proposal (https://github.com/PlaintextGroup/oss-virtual-incubator/blob/main/proposals/automated_analysis_of_open_source_ecosystems.md)

[epicfaace]

Hi @Ly0n ! Interesting proposal. How does this proposal relate to your post in https://opensustain.tech/blog/impact_and_potential_of_open_source_on_climate_technology/?

[Ly0n]

Hi @epicfaace,
We plan to make the tools we create within study itself into an independent open-source project. This way, the study can easily be repeated every year to better identify trends. It could also be used in other open-source domains. The software itself is already open source but not documented and in a Proof-of-Concept state:
https://github.com/protontypes/AwesomeCure

In principle, it works perfectly but we are missing developers supporting us. I do get support with the interviews and writing from Eirini Malliaraki, which we want to do within the community. Unfortunately, I don't have the team to turn the software it into a generic structure. But I think it is essential to use similar tools for other open-source infrastructure areas as well. This also includes the usage of open-source alternatives for tools like AirTable, which we are using at the moment.

That's why I am looking for a partnership and support to bring the tools and methods we have developed to a professional level. I'm also inviting every developer to join our study.

[LawrenceHecht]

1. The work you are doing is great.
2. "It took you a long time to prep the data before you actually did any analysis. You screened projects and added added meta data like categories and organizations. I wouldn't say this is truly automated., BUT, I do think it can mostly set up to be mostly automated.
3. Using Awesome lists is a decent way to define an ecosystem, but I would be hesitant to rely exclusively on the judgement of https://github.com/sindresorhus/awesome to determine which lists to include.
4. How did you calculate the community development distribution score.

[Ly0n]

@LawrenceHecht

"It took you a long time to prep the data before you actually did any analysis. You screened projects and added added meta data like categories and organizations. I wouldn't say this is truly automated., BUT, I do think it can mostly set up to be mostly automated.

That's right, the Awesome list was created by us according to certain guidelines. For later analysis, we took care from the beginning that projects are always entered with their Git URL. Many Awesome lists instead link directly to the project website if there is one. However, I think it should be possible to get the Git URL of the projects from most websites with a crawler. In this way, it would be possible to make the tools we have developed more generic and apply them to awesome lists.

Using Awesome lists is a decent way to define an ecosystem, but I would be hesitant to rely exclusively on the judgement of https://github.com/sindresorhus/awesome to determine which lists to include.

That's right. An alternative way would be to transfer the dependency tree of individual projects into such a list. For the LibreSelery project, we have already developed a dependency scanner that supports all package systems that are supported by https://libraries.io/. Technically this is feasible.

How did you calculate the community development distribution score.

We are still experimenting with different metrics. The value is similar to the bus factor and should make clear to what extent the project depends on a single person. The value is calculated at the moment as follows. However, we are still experimenting with numbers and formulas at this point.