[VPP-984] NAT: crash when try to create 10M sessions

[vvalderrv]

Description

Thread 1 "vpp_main" received signal SIGABRT, Aborted.

0x00007ffff67a9428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54

54 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.

(gdb) bt

#0 0x00007ffff67a9428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54

#1 0x00007ffff67ab02a in __GI_abort () at abort.c:89

#2 0x0000000000406d63 in os_panic () at /home/cisco/nat/vpp/build-data/../src/vpp/vnet/main.c:268

#3 0x00007ffff72b45fd in clib_mem_alloc_aligned_at_offset (os_out_of_memory_on_failure=1, align_offset=0, align=64, size=524288) at /home/cisco/nat/vpp/build-data/../src/vppinfra/mem.h:102

#4 clib_mem_alloc_aligned (align=64, size=524288) at /home/cisco/nat/vpp/build-data/../src/vppinfra/mem.h:119

#5 value_alloc_8_8 (log2_pages=log2_pages@entry=13, h=0x7fffb3deb200 <snat_main+864>, h=0x7fffb3deb200 <snat_main+864>) at /home/cisco/nat/vpp/build-data/../src/vppinfra/bihash_template.c:64

#6 0x00007ffff72b4634 in split_and_rehash_8_8 (h=h@entry=0x7fffb3deb200 <snat_main+864>, old_values=old_values@entry=0x7fff49020180, old_log2_pages=old_log2_pages@entry=11, new_log2_pages=new_log2_pages@entry=13)

at /home/cisco/nat/vpp/build-data/../src/vppinfra/bihash_template.c:162

#7 0x00007ffff72b6cb2 in clib_bihash_add_del_8_8 (h=h@entry=0x7fffb3deb200 <snat_main+864>, add_v=add_v@entry=0x7fffb7a02b70, is_add=is_add@entry=1) at /home/cisco/nat/vpp/build-data/../src/vppinfra/bihash_template.c:360

#8 0x00007fffb3bad826 in slow_path (sm=sm@entry=0x7fffb3deaea0 <snat_main>, rx_fib_index0=, key0=key0@entry=0x7fffb7a02de0, sessionp=0x7fffb7a02d68, next0=0, thread_index=0, node=0x7fffb5cf6b80, ip0=0x7ffefba218ce,

ip0=0x7ffefba218ce, b0=<optimized out>) at /home/cisco/nat/vpp/build-data/../src/plugins/nat/in2out.c:447

#9 0x00007fffb3bb3d5d in snat_in2out_node_fn_inline (is_output_feature=0, is_slow_path=1, frame=, node=, vm=) at /home/cisco/nat/vpp/build-data/../src/plugins/nat/in2out.c:1663

#10 snat_in2out_slow_path_fn (vm=, node=, frame=) at /home/cisco/nat/vpp/build-data/../src/plugins/nat/in2out.c:2021

#11 0x00007ffff795d3a4 in dispatch_node (last_time_stamp=17041081471058496, frame=0x7fffb70ddd80, dispatch_state=VLIB_NODE_STATE_POLLING, type=VLIB_NODE_TYPE_INTERNAL, node=0x7fffb5cf6b80, vm=0x7ffff7bb02e0 <vlib_global_main>)

at /home/cisco/nat/vpp/build-data/../src/vlib/main.c:1011

#12 dispatch_pending_node (vm=vm@entry=0x7ffff7bb02e0 <vlib_global_main>, pending_frame_index=pending_frame_index@entry=2, last_time_stamp=last_time_stamp@entry=17041081471058496)

at /home/cisco/nat/vpp/build-data/../src/vlib/main.c:1161

#13 0x00007ffff795f0c1 in vlib_main_or_worker_loop (is_main=1, vm=0x7ffff7bb02e0 <vlib_global_main>) at /home/cisco/nat/vpp/build-data/../src/vlib/main.c:1622

#14 vlib_main_loop (vm=0x7ffff7bb02e0 <vlib_global_main>) at /home/cisco/nat/vpp/build-data/../src/vlib/main.c:1641

#15 vlib_main (vm=vm@entry=0x7ffff7bb02e0 <vlib_global_main>, input=input@entry=0x7fffb7a02fa0) at /home/cisco/nat/vpp/build-data/../src/vlib/main.c:1799

#16 0x00007ffff7996df3 in thread0 (arg=140737349616352) at /home/cisco/nat/vpp/build-data/../src/vlib/unix/main.c:534

#17 0x00007ffff6f88488 in clib_calljmp () at /home/cisco/nat/vpp/build-data/../src/vppinfra/longjmp.S:110

#18 0x00007fffffffd120 in ?? ()

#19 0x00007ffff79979b0 in vlib_unix_main (argc=, argv=) at /home/cisco/nat/vpp/build-data/../src/vlib/unix/main.c:597

#20 0x000000000000005b in ?? ()

#21 0x0000000000000000 in ?? ()

Assignee

Matus Fabian

Reporter

Matus Fabian

Comments

• matfabia (Thu, 14 Sep 2017 11:36:30 +0000): Need to add total number of sessions limitation to avoid running out of memory crash.

• matfabia (Thu, 14 Sep 2017 11:35:11 +0000): for 10M sessions at least configure (otherwise crash):

  • translation hash buckets 1048576
  • translation hash memory 536870912

Original issue: https://jira.fd.io/browse/VPP-984

[vvalderrv]

Need to add total number of sessions limitation to avoid running out of memory crash.

[vvalderrv]

for 10M sessions at least configure (otherwise crash):

• translation hash buckets 1048576
• translation hash memory 536870912