manage the common code base between zos3 and zos4

[Omarabdul3ziz]

see the discussion here threefoldtech/zos4#11

[xmonader]

@iwanbk input here is appreciated

[iwanbk]

@iwanbk input here is appreciated

For now, let me answer with my knowledge from two months ago.
AFAIK, the main difference of zos4 is netlight, CMIIW.

I can see that zos already use interface tricks a lot.
For example the storage, it currently only support btrfs, but we can add bcachefs relatively easy.

Why we can't do the same with the netlight?

Some ways that came in my mind

1. current way, redo the PR back and forth between repos
   Current way is very manual, depends on the manual human labour to do it.
   We know that human error can easily happen, and i personally don't trust anyone can do it.

2. git tricks
   I heard that some company like ARM processor producer keep their own separate Linux kernel tree and sync it periodically.
   But i don't know how they do it and whether it is relevant for us.

3. Use the interface tricks, just like the storage.
   I haven't checked the details yet, but i think it could be something like this

• create two packages for current network and the new netlight

pkg/net/network
pkg/net/netlight

• in bootstrap, boot the relevant network package based on the version (i.e.: zos4 boot netlight)
• along the way, extract common code into separate package

pkg/net/internal/common
pkg/net/network
pkg/net/netlight

I believe that this last method already considered by the team, but met some difficulties because it is not as simple as i described above.
Lets share what the difficulties are and solve it here

[iwanbk]

current way, redo the PR back and forth between repos

To not blocking things, i think we can improve current way while waiting for the new way.

I pick a random PR to zos4 and compare it with a PR in zos with the same title.

Some improvement idea:

• give link to the same PR in zos
• if there is difference, mention the reasons.
  • in case of the PR i check, the number of files already different.

[ashraffouda]

I think we can start by separating the common code that is exactly the same in zos3 and zos4 in a separate repo and we call it for example zosbase, after that it will be clear how shoulf we manage the differences
this also will make changes that are related to both repos in one place
we tried to use the interfaces before when we started implementing zos4 but we faced some difficulties which I don't remember will now, but will consider rethinking about it after we do the separation

[iwanbk]

but will consider rethinking about it after we do the separation

I suggest to do it before the separation.
Because if we can do interfacing, we don't need the separation.

So, i can see at least three paths:

1. do interfacing

• we won't need separate repo, simply separate packages

2. using common packages, in separate (three) repos
   like you mentioned above

3. using common packages, in one repo
   Basically the same as number 2, but put it in the same repo

Could you tell the reason why we want it in separate repos?
Yes, the separation would be more clear, as we can see, there are duplicated works as well.

If we have zos5, will it become another repo as well?

[ashraffouda]

I want to do the separation because inside the network itself we copied some files in both dirs so it is better if we cleaned that up first so we have a minimal changes clear and clean then we do the interfacing if possible because now if we did the interfacing we will be maintaining duplicate code also but if we did the separation most of the changes will be done in one place which is the base code and even if we didn't do the interfacing small changes will be required in the networking/machine stuff

[iwanbk]

OK,
I could understand for the separation.

More questions:

1. why need to be in separate repo, instead of separate package?
2. I'm afraid that it is not a scalable way, what if we also have storage-light, container-light, etc...?