Removing subsock tok 0xae643042 pi:2 state 7 is_meta? 0

[maisammohsin]

Hi team,
I have got 4 sim card connections to aggregate , all of the sim are of same carrier , on 2 sim cards i am able to establish subflows but over the rest two sim i am not able to create subflow. below are the logs i am getting in dmesg

[ 2167.233081] addr4_event_handler created event for 100.81.194.76, code 3 prio 0 idx 6
[ 2167.745339] mptcp_address_worker updated IP 100.81.194.76 on ifidx 6 prio 0 id 4
[ 2167.753636] mptcp_add_sock: token 0xae643042 pi 2, src_addr:0.0.0.0:0 dst_addr:0.0.0.0:0
[ 2167.762310] __mptcp_init4_subsockets: token 0xae643042 pi 2 src_addr:100.106.182.185:0 dst_addr:176.205.49.200:9899 ifidx: 7
[ 2167.774281] mptcp_add_sock: token 0xae643042 pi 3, src_addr:0.0.0.0:0 dst_addr:0.0.0.0:0
[ 2167.782958] __mptcp_init4_subsockets: token 0xae643042 pi 3 src_addr:100.81.194.76:0 dst_addr:176.205.49.200:9899 ifidx: 6
[ 2167.794716] mptcp_add_sock: token 0xae643042 pi 4, src_addr:0.0.0.0:0 dst_addr:0.0.0.0:0
[ 2167.803266] __mptcp_init4_subsockets: token 0xae643042 pi 4 src_addr:100.106.182.185:0 dst_addr:92.99.8.132:9899 ifidx: 7
[ 2167.814893] mptcp_add_sock: token 0xae643042 pi 7, src_addr:0.0.0.0:0 dst_addr:0.0.0.0:0
[ 2167.823454] __mptcp_init4_subsockets: token 0xae643042 pi 7 src_addr:100.81.194.76:0 dst_addr:92.99.8.132:9899 ifidx: 6
[ 2167.838842] mptcp_del_sock: Removing subsock tok 0xae643042 pi:2 state 7 is_meta? 0
[ 2167.850645] mptcp_del_sock: Removing subsock tok 0xae643042 pi:3 state 7 is_meta? 0
[ 2167.866585] mptcp_del_sock: Removing subsock tok 0xae643042 pi:4 state 7 is_meta? 0
[ 2167.879695] mptcp_del_sock: Removing subsock tok 0xae643042 pi:7 state 7 is_meta? 0
[ 2167.895713] mptcp_del_sock: Removing subsock tok 0xae643042 pi:6 state 7 is_meta? 0

[matttbe]

Hello,

Which kernel versions are you using?
With which path manager? How many subflows are being created?
Are the routing rules OK, e.g. if you create a connection from 100.106.182.185, is it correctly routed though the right interface? http://multipath-tcp.org/pmwiki.php/Users/ConfigureRouting

Even if it is the same network operator, are you sure MPTCP is not blocked by accident? You can check that with Tracebox: https://multipath-tcp.org/pmwiki.php/Users/Tools

[maisammohsin]

My Linux Kernal Version is 5.4.128 , Path-manager is Full Mesh and there are 4 sim card being used at client end where-as at server end there are 2 ADSL lines.

I have configured my server as well as client devices as per the instructions given in http://multipath-tcp.org/pmwiki.php/Users/ConfigureRouting .

Practically it should create 8 subflows between the server and client ( 4 client sim's and 2 server ADSL ) , but the creation of subflows are completely very random. Sometimes i all subflow being created from all sims toward both ADSL ( 8 subflows ) and sometime it just from first 2 sim or last 2 sims ( 4 subflows )

All the sim's i am using are of same carrier so i dont think there's any blocking happening on carrier side , if it was to block the MPTCP then it should have not created subflows from either of the SIM. Still i check with the tracebox tool to make sure if there's any blocking happening on any SIM.

[matttbe]

Practically it should create 8 subflows between the server and client ( 4 client sim's and 2 server ADSL ) , but the creation of subflows are completely very random. Sometimes i all subflow being created from all sims toward both ADSL ( 8 subflows ) and sometime it just from first 2 sim or last 2 sims ( 4 subflows )

May you check what the Fullmesh PM module is doing?

cat /proc/net/mptcp_fullmesh

Maybe there are other IPs in the list you might want to blacklist.
https://multipath-tcp.org/pmwiki.php/Users/Tools

Also I guess the connection you are doing is long enough (with at least one byte sent) for the client to establish more than one path, right?

All the sim's i am using are of same carrier so i dont think there's any blocking happening on carrier side , if it was to block the MPTCP then it should have not created subflows from either of the SIM. Still i check with the tracebox tool to make sure if there's any blocking happening on any SIM.

Maybe useful to take a packet trace with one MPTCP connection and look at what is sent and received, especially around the TCP SYN packets:

sudo tcpdump -i any -n "tcp[tcpflags] & (tcp-syn) != 0"

[maisammohsin]

Hi Matt,
I tried doing the tests using ADSL line's as well as SIM cards. Over ADSL line i am getting all the sub-flows but yet i am not able to establish all sub-flows over SIM cards,

I was just wondering on a single device if i could set separate MPTCP policies like path manager and schedulers for each interface. For example , if i want to use "redundant scheduler" for particular traffic destined to SIM cards interface and "default scheduler" for particular traffic destined to ADSL / Wired WAN interfaces , can i do that ?

[matttbe]

Hi,

Over ADSL line i am getting all the sub-flows but yet i am not able to establish all sub-flows over SIM cards,

Do you mean that MPTCP is sometimes blocked when using the mobile networks? Some ISP accidentally blocks MPTCP on some specific ports, e.g. trying to "optimise" HTTP traffic on port 80.

I was just wondering on a single device if i could set separate MPTCP policies like path manager and schedulers for each interface. For example , if i want to use "redundant scheduler" for particular traffic destined to SIM cards interface and "default scheduler" for particular traffic destined to ADSL / Wired WAN interfaces , can i do that ?

You cannot configure a different scheduler per route like you could do with a TCP congestion control but if you control your app, you can also set a different packet scheduler per connection, e.g.

char scheduler[] = "redundant";
setsockopt(fd, SOL_TCP, MPTCP_SCHEDULER, scheduler, sizeof(scheduler));

https://multipath-tcp.org/pmwiki.php/Users/ConfigureMPTCP