Comment 7 for bug 1906280

Revision history for this message
Billy Olsen (billy-olsen) wrote : Re: Charm stuck waiting for ovsdb 'no key "ovn-remote" in Open_vSwitch record'

From the crash made available in a recreate, we get the following:

#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
        set = {__val = {0, 140725298456336, 140664669834388, 0, 140725298456672, 140664668256100, 14583431671641669699, 14583431671641719254, 0, 140664668255802, 4707197592648237900, 7161402270843880775,
            140724610755886, 11, 140725298456672, 140664668255726}}
        pid = <optimized out>
        tid = <optimized out>
        ret = <optimized out>
#1 0x00007fef0b965921 in __GI_abort () at abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0x5, sa_sigaction = 0x5}, sa_mask = {__val = {140664672163456, 10, 11, 11, 94494056588192, 140725298456896, 140664668628179, 140664672163456, 140664695139080,
              11, 140664668607515, 0, 94494055958702, 1, 140725298457136, 94494060193760}}, sa_flags = 695003688, sa_restorer = 0x55f11ce422a8}
        sigs = {__val = {32, 0 <repeats 15 times>}}
        __cnt = <optimized out>
        __set = <optimized out>
        __cnt = <optimized out>
        __set = <optimized out>
#2 0x000055f11ca3892e in ovs_abort_valist (err_no=<optimized out>, format=<optimized out>, args=args@entry=0x7ffd296ce940) at ../lib/util.c:419
No locals.
#3 0x000055f11ca389c4 in ovs_abort (err_no=<optimized out>, format=format@entry=0x55f11cad23a0 "pthread_create failed") at ../lib/util.c:411
        args = {{gp_offset = 16, fp_offset = 48, overflow_arg_area = 0x7ffd296cea20, reg_save_area = 0x7ffd296ce960}}
#4 0x000055f11ca048fe in ovs_thread_create (name=name@entry=0x55f11cab2908 "handler", start=start@entry=0x55f11c9413f0 <udpif_upcall_handler>, arg=arg@entry=0x55f11ce422a8) at ../lib/ovs-thread.c:449
        once = {done = true, mutex = {lock = pthread_mutex_t = {Type = Error check, Status = Not acquired, Robust = No, Shared = No, Protocol = None}, where = 0x55f11caaacfe "<unlocked>"}}
        aux = 0x55f11ce427e0
        thread = 8388608
        error = <optimized out>
        attr = {__size = '\000' <repeats 17 times>, "\020", '\000' <repeats 37 times>, __align = 0}
#5 0x000055f11c93e7d6 in udpif_start_threads (udpif=0x55f11ce41b50, n_handlers_=<optimized out>, n_revalidators_=<optimized out>) at ../ofproto/ofproto-dpif-upcall.c:569
        handler = 0x55f11ce422a8
        i = 3

the pthread_create is failing with EAGAIN, which:

EAGAIN A system-imposed limit on the number of threads was
              encountered. There are a number of limits that may trigger
              this error: the RLIMIT_NPROC soft resource limit (set via
              setrlimit(2)), which limits the number of processes and
              threads for a real user ID, was reached; the kernel's system-
              wide limit on the number of processes and threads,
              /proc/sys/kernel/threads-max, was reached (see proc(5)); or
              the maximum number of PIDs, /proc/sys/kernel/pid_max, was
              reached (see proc(5)).

Which means one of the system limits was reached.