Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FS#3070 - kmod-cryptodev: WARNING: possible circular locking dependency detected #7837

Open
openwrt-bot opened this issue May 8, 2020 · 3 comments
Labels

Comments

@openwrt-bot
Copy link

stintel:

OpenWrt master r13174-73fa1aba94 on octeon with kernel 5.4

[ 70.971145] [ 70.972662] ====================================================== [ 70.978843] WARNING: possible circular locking dependency detected [ 70.985029] 5.4.39 #0 Not tainted [ 70.988348] ------------------------------------------------------ [ 70.994531] unbound/2895 is trying to acquire lock: [ 70.999414] 800000041baeb868 (&pcr->fcrypt.sem){+.+.}, at: crypto_get_session_by_sid+0x40/0x12b8 [cryptodev] [ 71.009267] [ 71.009267] but task is already holding lock: [ 71.015103] 800000041b911468 (&ses_new->sem){+.+.}, at: crypto_get_session_by_sid+0xbc/0x12b8 [cryptodev] [ 71.024691] [ 71.024691] which lock already depends on the new lock. [ 71.024691] [ 71.032866] [ 71.032866] the existing dependency chain (in reverse order) is: [ 71.040347] [ 71.040347] -> #1 (&ses_new->sem){+.+.}: [ 71.045773] lock_acquire+0xe0/0x220 [ 71.049888] __mutex_lock+0x94/0x628 [ 71.054001] crypto_get_session_by_sid+0xbc/0x12b8 [cryptodev] [ 71.060365] crypto_get_session_by_sid+0x1e4/0x12b8 [cryptodev] [ 71.066806] [ 71.066806] -> #0 (&pcr->fcrypt.sem){+.+.}: [ 71.072485] check_noncircular+0x1a8/0x260 [ 71.077110] __lock_acquire+0x12f8/0x19f8 [ 71.081648] lock_acquire+0xe0/0x220 [ 71.085754] __mutex_lock+0x94/0x628 [ 71.089862] crypto_get_session_by_sid+0x40/0x12b8 [cryptodev] [ 71.096226] crypto_get_session_by_sid+0x6cc/0x12b8 [cryptodev] [ 71.102666] [ 71.102666] other info that might help us debug this: [ 71.102666] [ 71.110668] Possible unsafe locking scenario: [ 71.110668] [ 71.116588] CPU0 CPU1 [ 71.121119] ---- ---- [ 71.125650] lock(&ses_new->sem); [ 71.129058] lock(&pcr->fcrypt.sem); [ 71.135242] lock(&ses_new->sem); [ 71.141165] lock(&pcr->fcrypt.sem); [ 71.144834] [ 71.144834] *** DEADLOCK *** [ 71.144834] [ 71.150756] 1 lock held by unbound/2895: [ 71.154681] #0: 800000041b911468 (&ses_new->sem){+.+.}, at: crypto_get_session_by_sid+0xbc/0x12b8 [cryptodev] [ 71.164700] [ 71.164700] stack backtrace: [ 71.169067] CPU: 1 PID: 2895 Comm: unbound Not tainted 5.4.39 #0 [ 71.175075] Stack : ffffffff82790000 0000000000000000 0000000010108ce0 ed34220aed96997f [ 71.183091] ed34220aed96997f 0000000000000000 800000041bb976f8 ffffffff837d0000 [ 71.191106] 0000000000000000 0000000000000001 0000000000000000 ffffffff811ac4bc [ 71.199118] 6e626f756e64204e 0000000000000000 ffffffffffffffff 0000000000000010 [ 71.207132] 0000000000000000 ffffffff81b40000 fffe000000000000 ffffffff81bc0000 [ 71.215145] 0000000000000000 0000000000000000 ffffffff81b40000 0000000000000000 [ 71.223158] 800000041f2e6200 0000000000000000 ffffffff81597628 1e00000010734ac7 [ 71.231173] 0000000000000001 800000041bb94000 800000041bb976f0 47500c0a872e7996 [ 71.239186] ffffffff81865d8c 0000000000000000 800000041bb97828 ed34220aed96997f [ 71.247198] ffffffff81b40bf7 ffffffff81865c54 ffffffff8111d4c8 ffffffff81a42a50 [ 71.255213] ... [ 71.257668] Call Trace: [ 71.260136] [] show_stack+0x40/0x128 [ 71.265288] [] dump_stack+0xe4/0x150 [ 71.270444] [] check_noncircular+0x1a8/0x260 [ 71.276289] [] __lock_acquire+0x12f8/0x19f8 [ 71.282046] [] lock_acquire+0xe0/0x220 [ 71.287374] [] __mutex_lock+0x94/0x628 [ 71.292706] [] crypto_get_session_by_sid+0x40/0x12b8 [cryptodev] [ 71.300290] [] crypto_get_session_by_sid+0x6cc/0x12b8 [cryptodev]
@openwrt-bot
Copy link
Author

n8v8R:

Maybe something device specific or unbound flavour (light vs heavy)? Does not reproduce on

{"kernel":"5.4.45","hostname":"OpenWrt","system":"ARMv7 Processor rev 1 (v7l)","model":"Turris Omnia","board_name":"cznic,turris-omnia","release":{"distribution":"OpenWrt","version":"SNAPSHOT","revision":"r13552-cd09f26660","target":"mvebu/cortexa9","description":"OpenWrt SNAPSHOT r13552-cd09f26660"}}

with

  • kmod-cryptodev 5.4.45+1.10-mvebu-2
  • unbound-daemon-heavy 1.10.1-2
  • libunbound-heavy 1.10.1-2

@openwrt-bot
Copy link
Author

michael-dev:

I'm seeing the same on a custom OpenWRT build based onfae69177db52e14b7b966d73603ee269f6ef02c7

[ 45.669944] ====================================================== [ 45.676115] WARNING: possible circular locking dependency detected [ 45.682292] 5.4.99 #0 Not tainted [ 45.685598] ------------------------------------------------------ [ 45.691772] nrpe/3560 is trying to acquire lock: [ 45.696382] ef68f23c (&pcr->fcrypt.sem){+.+.}, at: crypto_get_session_by_sid+0x38/0xc4 [cryptodev] [ 45.705360] [ 45.705360] but task is already holding lock: [ 45.711184] ef68fa3c (&ses_new->sem){+.+.}, at: crypto_get_session_by_sid+0xa8/0xc4 [cryptodev] [ 45.719884] [ 45.719884] which lock already depends on the new lock. [ 45.719884] [ 45.728052] [ 45.728052] the existing dependency chain (in reverse order) is: [ 45.735526] [ 45.735526] -> #1 (&ses_new->sem){+.+.}: [ 45.740942] __mutex_lock+0x94/0x84c [ 45.745037] crypto_get_session_by_sid+0xa8/0xc4 [cryptodev] [ 45.751213] cryptodev_ioctl+0xa4/0x9c8 [cryptodev] [ 45.756609] do_vfs_ioctl+0xc4/0x948 [ 45.760698] ksys_ioctl+0x50/0xbc [ 45.764535] ret_from_syscall+0x0/0x38 [ 45.768797] [ 45.768797] -> #0 (&pcr->fcrypt.sem){+.+.}: [ 45.774467] __lock_acquire+0x1004/0x2100 [ 45.778992] lock_acquire+0xe0/0x1ec [ 45.783085] __mutex_lock+0x94/0x84c [ 45.787179] crypto_get_session_by_sid+0x38/0xc4 [cryptodev] [ 45.793356] cryptodev_ioctl+0x538/0x9c8 [cryptodev] [ 45.798835] do_vfs_ioctl+0xc4/0x948 [ 45.802925] ksys_ioctl+0x50/0xbc [ 45.806756] ret_from_syscall+0x0/0x38 [ 45.811017] [ 45.811017] other info that might help us debug this: [ 45.811017] [ 45.819012] Possible unsafe locking scenario: [ 45.819012] [ 45.824923] CPU0 CPU1 [ 45.829444] ---- ---- [ 45.833965] lock(&ses_new->sem); [ 45.837360] lock(&pcr->fcrypt.sem); [ 45.843534] lock(&ses_new->sem); [ 45.849446] lock(&pcr->fcrypt.sem); [ 45.853102] [ 45.853102] *** DEADLOCK *** [ 45.853102] [ 45.859015] 1 lock held by nrpe/3560: [ 45.862668] #0: ef68fa3c (&ses_new->sem){+.+.}, at: crypto_get_session_by_sid+0xa8/0xc4 [cryptodev] [ 45.871803] [ 45.871803] stack backtrace: [ 45.876156] CPU: 1 PID: 3560 Comm: nrpe Not tainted 5.4.99 #0 [ 45.881894] Call Trace: [ 45.884342] [ef5b7ba8] [c06844a8] dump_stack+0xb0/0x124 (unreliable) [ 45.890695] [ef5b7bc8] [c0092404] check_noncircular+0x16c/0x1f4 [ 45.896611] [ef5b7c08] [c0094b5c] __lock_acquire+0x1004/0x2100 [ 45.902439] [ef5b7cc8] [c00964ac] lock_acquire+0xe0/0x1ec [ 45.907838] [ef5b7d08] [c06a429c] __mutex_lock+0x94/0x84c [ 45.913236] [ef5b7d78] [f13d53e4] crypto_get_session_by_sid+0x38/0xc4 [cryptodev] [ 45.920716] [ef5b7d98] [f13d59a8] cryptodev_ioctl+0x538/0x9c8 [cryptodev] [ 45.927500] [ef5b7ec8] [c01c5028] do_vfs_ioctl+0xc4/0x948 [ 45.932894] [ef5b7f18] [c01c58fc] ksys_ioctl+0x50/0xbc [ 45.938030] [ef5b7f38] [c0012298] ret_from_syscall+0x0/0x38 [ 45.943599] --- interrupt: c01 at 0xfb2aa74

Though, no actual deadlock occurs.

@openwrt-bot
Copy link
Author

michael-dev:

Looking at crypto_get_session_by_sid, I also cannot see how this should ever happen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant