| Kernel debug configuration option | Description |
|---|---|
| CONFIG_DEBUG_LIST | Turns on checks for performing standard linked list manipulations with the list.h header file. If the pointers don’t match, the system prints a warning, followed by a BUG_ON crash. |
| CONFIG_PAGE_POISONING | Fills the pages with the poison pattern (PAGE_POISON 0xaa), after calling free_pages(). |
| CONFIG_DEBUG_PAGEALLOC | Verifies the patterns before calling alloc_pages(). |
| CONFIG_DEBUG_USER | Prints a message when the system kills a user-space process due to a segmentation fault (segfault) or an invalid instruction, such as user_debug=31 in the arch/arm/Kconfig.debug file. Add the Kernel boot parameter to the BoardConfig.mk file. |
| CONFIG_DEBUG_SPINLOCK | Identifies missing spinlock initialization and common spinlock errors, such as: |
- Waiting for more than one second on a spinlock
- Freeing an already freed lock
- Reinitializing a lock that was already used
| CONFIG_DEBUG_MUTEXES | Detects Mutex semantic violations. |
|---|---|
| DEBUG_LOCK_ALLOC | Detects wrong freeing of live locks. |
| CONFIG_SLUB CONFIG_SLUB_DEBUG | Performs extra checks to detect the corruption of internal kernel memory allocation structures by adding poison for use-after-free (0x6b) and buffer-overflow-padding (0xbb). |
| Kernel debug configuration options for extra debugging that can be verbose and can make the system slow. | |
| CONFIG_DEBUG_ATOMIC_SLEEP | Causes routines that may sleep to become noisy when they’re called within the atomic sections. |
| DEBUG_SPINLOCK_SLEEP | Causes routines that may sleep to become noisy when they’re called with a spinlock held. |
| CONFIG_DEBUG_VM, CONFIG_DEBUG_HIGHMEM | Provides an extra debugging support for virtual memory management corruption issues. |
| CONFIG_DEBUG_OBJECTS | Tracks the lifetime of various objects and validates the operations on those objects. |
List corruption issues
To identify the following crash signatures that indicate list corruption issues, enable theCONFIG_DEBUG_LIST option.
Sample crash signature 1
\<4\> WARNING: at kernel/lib/list\_debug.c:60 \_\_list\_del\_entry\+0xa0/0xd0()
\<6\> list\_del corruption. prev-\>next should be c6fc374c, but was c18d804c
\<6\> Modules linked in: adsprpc
\<6\> \[\<c010cc94\>\] (unwind\_backtrace\+0x0/0x138) from \[\<c018ad54\>\] (warn\_slowpath\_common\+0x4c/0x64)
\<6\> \[\<c018ad54\>\] (warn\_slowpath\_common\+0x4c/0x64) from \[\<c018ae00\>\] (warn\_slowpath\_fmt\+0x30/0x40)
\<6\> \[\<c018ae00\>\] (warn\_slowpath\_fmt\+0x30/0x40) from \[\<c03b3128\>\] (\_\_list\_del\_entry\+0xa0/0xd0)
\<6\> \[\<c03b3128\>\] (\_\_list\_del\_entry\+0xa0/0xd0) from \[\<c01bc968\>\] (account\_entity\_dequeue\+0x84/0x94)
\<6\> \[\<c01bc968\>\] (account\_entity\_dequeue\+0x84/0x94) from \[\<c01bdaa4\>\] (dequeue\_task\_fair\+0x64/0x190)
\<4\>WARNING: at kernel/lib/list\_debug.c:52 \_\_list\_del\_entry\+0x8c/0xac()
\<4\> list\_del corruption, e240c500-\>prev is LIST\_POISON2 (00200200)
\<4\> Modules linked in: wlan(PO) cfg80211 adsp\_loader
\<4\> \[\<c001498c\>\] (unwind\_backtrace\+0x0/0x11c)
from \[\<c0068c90\>\] (warn\_slowpath\_common\+0x4c/0x64)
\<4\> \[\<c0068c90\>\] (warn\_slowpath\_common\+0x4c/0x64)
from \[\<c0068d28\>\] (warn\_slowpath\_fmt\+0x2c/0x3c)
\<4\> \[\<c0068d28\>\] (warn\_slowpath\_fmt\+0x2c/0x3c)
from \[\<c023ef94\>\] (\_\_list\_del\_entry\+0x8c/0xac)
\<4\> \[\<c023ef94\>\] (\_\_list\_del\_entry\+0x8c/0xac)
from \[\<c023efc0\>\] (list\_del\+0xc/0x24)
\<4\> \[\<c023efc0\>\] (list\_del\+0xc/0x24)
from \[\<c043c78c\>\] (binder\_thread\_read\+0x488/0xb70)
\<4\> \[\<c043c78c\>\] (binder\_thread\_read\+0x488/0xb70)
from \[\<c043d094\>\] (binder\_ioctl\+0x220/0x5b8)
\<4\> WARNING: at kernel/lib/list\_debug.c:47 \_\_list\_del\_entry\+0x90/0xb0()
\<4\> list\_del corruption, d6ed3720-\>next is LIST\_POISON1 (00100100)
\<4\> Modules linked in:
\<4\> \[\] (unwind\_backtrace\+0x0/0x12c) from \[\] (warn\_slowpath\_common\+0x4c/0x64)
\<4\> \[\] (warn\_slowpath\_common\+0x4c/0x64) from \[\] (warn\_slowpath\_fmt\+0x2c/0x3c)
\<4\> \[\] (warn\_slowpath\_fmt\+0x2c/0x3c) from \[\] (\_\_list\_del\_entry\+0x90/0xb0)
\<4\> \[\] (\_\_list\_del\_entry\+0x90/0xb0) from \[\] (list\_del\+0xc/0x28)
\<4\> \[\] (list\_del\+0xc/0x28) from \[\] (bam\_mux\_write\_done\+0x34/0x11c)
Identify spinlock corruption issues
To identify crash signatures that indicate spinlock corruption issues, enable theCONFIG_DEBUG_SPINLOCK and CONFIG_DEBUG_MUTEXES kernel configuration options.
Sample crash signature 1
\<0\>BUG: spinlock lockup on CPU#1, ndroid.launcher/1071
\<0\> lock: 0xd5e8f480, .magic: dead4ead, .owner: \<none\>/-1, .owner\_cpu: -1
\<4\> \[\<c0734d84\>\] (spin\_dump\+0x74/0x84) from \[\<c028f8fc\>\]
(do\_raw\_spin\_lock\+0x144/0x188)
\<4\> \[\<c028f8fc\>\] (do\_raw\_spin\_lock\+0x144/0x188) from \[\<c0332594\>\] (kgsl\_mmu\_pt\_get\_flags\+0x18/0x44)
\<4\> \[\<c0332594\>\] (kgsl\_mmu\_pt\_get\_flags\+0x18/0x44) from \[\<c033c6c8\>\] (adreno\_ringbuffer\_submitcmd\+0x168/0x228)
\<4\> \[\<c033c6c8\>\] (adreno\_ringbuffer\_submitcmd\+0x168/0x228) from \[\<c033d9a4\>\] (sendcmd\+0x3c/0x254)
\<0\> BUG: spinlock lockup on CPU#2, kworker/2:0H/2910
\<0\> lock: kpss\_clock\_reg\_lock\+0x0/0x10, .magic: dead4ead, .owner:
kworker/3:0H/2904, .owner\_cpu: 3
\<6\> kworker/2:0H (2910): undefined instruction: pc=c0963098
kernel BUG at kernel/lib/spinlock\_debug.c:95!
\<4\> \[\<c0963098\>\] (spin\_dump\+0x7c/0x94) from \[\<c038e030\>\]
(do\_raw\_spin\_lock\+0xcc/0x164)
\<4\> \[\<c038e030\>\] (do\_raw\_spin\_lock\+0xcc/0x164) from \[\<c0972228\>\] (\_raw\_spin\_lock\_irqsave\+0x20/0x28)
\<4\> \[\<c0972228\>\] (\_raw\_spin\_lock\_irqsave\+0x20/0x28) from \[\<c011d4ac\>\] (\_\_kpss\_mux\_set\_sel\+0x14/0x80)
\<4\> \[\<c011d4ac\>\] (\_\_kpss\_mux\_set\_sel\+0x14/0x80) from \[\<c011d54c\>\] (kpss\_mux\_set\_sel\+0x18/0x20)
Identify slub poisoning issues
To identify crash signatures that indicate slub poisoning issues, enable theCONFIG_SLUB and CONFIG_SLUB_DEBUG kernel configuration options.
<3\>\[ 3438.930472\] =====================================================================
<3\>\[ 3438.937628\] BUG kmalloc-64 (Tainted: G W O): Poison overwritten
<3\>\[ 3438.944223\] ---------------------------------------------------------------------
<3\>\[ 3438.944228\]
<3\>\[ 3438.953861\] INFO: 0xce308408-0xce30840b. First byte 0x0 instead of 0x6b
<3\>\[ 3438.960470\] INFO: Allocated in kgsl\_ioctl\_drawctxt\_create\+0x2c/0x2ac age=4426 cpu=0 pid=21702
<3\>\[ 3438.968970\] \_\_slab\_alloc.isra.37.constprop.43\+0x4d4/0x534
<3\>\[ 3438.974435\] kmem\_cache\_alloc\_trace\+0x240/0x258
<3\>\[ 3438.978947\] kgsl\_ioctl\_drawctxt\_create\+0x2c/0x2ac
<3\>\[ 3438.983721\] kgsl\_ioctl\+0xfc/0x324
<3\>\[ 3438.987108\] do\_vfs\_ioctl\+0x80/0x54c
<3\>\[ 3438.990669\] sys\_ioctl\+0x38/0x5c
<3\>\[ 3438.993880\] ret\_fast\_syscall\+0x0/0x30
<3\>\[ 3438.997615\] INFO: Freed in kgsl\_release\+0xb8/0xc0 age=10 cpu=3 pid=21842
<3\>\[ 3439.004298\] \_\_slab\_free\+0x30/0x308
<3\>\[ 3439.007770\] kgsl\_release\+0xb8/0xc0
<3\>\[ 3439.011242\] fput\+0xcc/0x23c
<3\>\[ 3439.014106\] filp\_close\+0x68/0x80
<3\>\[ 3439.017407\] put\_files\_struct\+0xd8/0x110
<3\>\[ 3439.021312\] do\_exit\+0x164/0x860
\<3\>\[ 3439.024524\] do\_group\_exit\+0x3c/0xb0
\<3\>\[ 3439.028084\] get\_signal\_to\_deliver\+0x2c4/0x59c
\<3\>\[ 3439.032510\] do\_signal\+0x90/0x480
<3\>\[ 3439.035809\] do\_notify\_resume\+0x50/0x5c
<3\>\[ 3439.039629\] work\_pending\+0x24/0x28
<3\>\[ 3439.043101\] INFO: Slab 0xc0fbcd20 objects=16 used=16 fp=0x (null) flags=0x0080
<3\>\[ 3439.050401\] INFO: Object 0xce308400 @offset=1024 fp=0xce308900
<3\>\[ 3439.050405\]
<3\>\[ 3439.057694\] Bytes b4 ce3083f0: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
<3\>\[ 3439.066457\] Object ce308400: 6b 6b 6b 6b 6b 6b 6b 6b 00 00 00 00 6b 6b 6b 6b kkkkkkkk....kkkk
<3\>\[ 3439.075051\] Object ce308410: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
<3\>\[ 3439.083639\] Object ce308420: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
<3\>\[ 3439.092234\] Object ce308430: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkkkkkk.
<3\>\[ 3439.100828\] Redzone ce308440: bb bb bb bb ....
<3\>\[ 3439.108463\] Padding ce3084e8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
<3\>\[ 3439.117145\] Padding ce3084f8: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ
<4\>\[ 3439.125144\] \[\<c0014f78\>\] (unwind\_backtrace\+0x0/0x138) from \[\<c0150250\>\] (check\_bytes\_and\_report\+0xc0/0xe4)
<4\>\[ 3439.134771\] \[\<c0150250\>\] (check\_bytes\_and\_report\+0xc0/0xe4) from \[\<c015042c\>\] (check\_object\+0x1b8/0x214)
<4\>\[ 3439.144237\] \[\<c015042c\>\] (check\_object\+0x1b8/0x214) from \[\<c071b0bc\>\] (alloc\_debug\_processing\+0x7c/0x150)
<4\>\[ 3439.153782\] \[\<c071b0bc\>\] (alloc\_debug\_processing\+0x7c/0x150) from \[\<c071bcc0\>\] (\_\_slab\_alloc.isra.37.constprop.43\+0x4d4/0x534)
<4\>\[ 3439.165153\] \[\<c071bcc0\>\] (\_\_slab\_alloc.isra.37.constprop.43\+0x4d4/0x534) from \[\<c015221c\>\] (kmem\_cache\_alloc\_trace\+0x240/0x258)
<4\>\[ 3439.176611\] \[\<c015221c\>\] (kmem\_cache\_alloc\_trace\+0x240/0x258) from \[\<c014668c\>\] (\_\_get\_vm\_area\_node.isra.26\+0x84/0x174)
<4\>\[ 3439.187376\] \[\<c014668c\>\] (\_\_get\_vm\_area\_node.isra.26\+0x84/0x174) from \[\<c0147038\>\] (get\_vm\_area\_caller\+0x44/0x4c)
<4\>\[ 3439.197615\] \[\<c0147038\>\] (get\_vm\_area\_caller\+0x44/0x4c) from \[\<c014739c\>\] (vmap\+0x50/0x90)
<4\>\[ 3439.205866\] \[\<c014739c\>\] (vmap\+0x50/0x90) from \[\<c032a694\>\] (\_kgsl\_sharedmem\_page\_alloc\+0x238/0x3d4)
<4\>\[ 3439.214979\] \[\<c032a694\>\] (\_kgsl\_sharedmem\_page\_alloc\+0x238/0x3d4) from \[\<c0323e44\>\] (\_gpumem\_alloc\+0xb0/0xfc)
<4\>\[ 3439.224874\] \[\<c0323e44\>\] (\_gpumem\_alloc\+0xb0/0xfc) from \[\<c0323ed0\>\] (kgsl\_ioctl\_gpumem\_alloc\_id\+0x40/0x1a8)
<4\>\[ 3439.234684\] \[\<c0323ed0\>\] (kgsl\_ioctl\_gpumem\_alloc\_id\+0x40/0x1a8) from \[\<c0321874\>\] (kgsl\_ioctl\+0xfc/0x324)
<4\>\[ 3439.244319\] \[\<c0321874\>\] (kgsl\_ioctl\+0xfc/0x324) from \[\<c0167fe8\>\] (do\_vfs\_ioctl\+0x80/0x54c)
<4\>\[ 3439.252738\] \[\<c0167fe8\>\] (do\_vfs\_ioctl\+0x80/0x54c) from \[\<c01684ec\>\] (sys\_ioctl\+0x38/0x5c)
<4\>\[ 3439.260986\] \[\<c01684ec\>\] (sys\_ioctl\+0x38/0x5c) from \[\<c000eb00\>\] (ret\_fast\_syscall\+0x0/0x30)
<3\>\[ 3439.269400\] FIX kmalloc-64: Restoring 0xce308408-0xce30840b=0x6b
Identify page poisoning issues
To identify a crash signature that indicates page poisoning issues, enable theCONFIG_DEBUG_PAGEALLOC and CONFIG_PAGE_POISONING kernel configuration options.
\<1\> Unable to handle kernel paging request at virtual address aaaaaaae
\<1\> pgd = e98b4000
\<1\> \[aaaaaaae\] [\*](https://docs.qualcomm.com/doc/80-80021-12/topic/configure_the_kernel_for_debugging.html?product=895724676033554725&facet=Debug%20Subsystem&version=2.0-rc2#id2)pgd=00000000
\<0\> Internal error: Oops: 5 \[#1\] PREEMPT SMP ARM
\<4\> Modules linked in: adsp\_loader exfat\_fs(P) exfat\_core(P)
\<4\> CPU: 1 Tainted: P W (3.4.0-628250-eng #1)
\<4\> PC is at pid\_nr\_ns\+0xc/0x3c
\<4\> LR is at do\_task\_stat\+0x248/0x83c
\<4\> pc : \[\<c00abda8\>\] lr : \[\<c018c15c\>\] psr: a0000093