debug link

與符號(hào)表分離程序或動(dòng)態(tài)庫(kù), 如何用GDB調(diào)試

五竹

1 ? ? ? Debugging Information in Separate Files

GDB支持用戶將程序調(diào)試信息放在一個(gè)獨(dú)立的文件里衫哥，而不是和可執(zhí)行程序在一個(gè)文件中普监，GDB可以某種方式來查找和自動(dòng)加載調(diào)試信息。由于調(diào)試信息可能非常大,有時(shí)可能比可執(zhí)行代碼自身還要大,某些系統(tǒng)將其可執(zhí)行程序的調(diào)試信息以單獨(dú)的文件發(fā)布仙畦，在需要調(diào)試問題的時(shí)候，用戶可以再安裝這些文件酪刀。

GDB支持兩種設(shè)置單獨(dú)調(diào)試信息文件的方式：

可執(zhí)行程序里包含了一個(gè)調(diào)試鏈接眨猎，此鏈接指定了單獨(dú)的調(diào)試信息文件名。單獨(dú)調(diào)試文件名通常是’executable.debug’弥姻， executable是相應(yīng)的可執(zhí)行程序名南片，不帶路徑（例如，’ls.debug’是’/usr/bin/ls’的調(diào)試信息文件）庭敦。此外疼进，調(diào)試鏈接為調(diào)試文件設(shè)置了CRC32的校驗(yàn)和，GDB用此校驗(yàn)和來確毖砹可執(zhí)行文件和調(diào)試文件是同一個(gè)版本的伞广。

可執(zhí)行文件包含一個(gè)版本ID號(hào)和一個(gè)唯一的bit串拣帽，而相應(yīng)的調(diào)試信息文件也包含此bit串。（此方式只在某些系統(tǒng)上支持嚼锄，特別是那些在二進(jìn)制文件里使用ELF格式和GNU Binutils的系統(tǒng)诞外。）更多關(guān)于此功能的細(xì)節(jié)，參見’–build-id’命令行選項(xiàng)的介紹灾票，在GNU連接器的“命令行選項(xiàng)”節(jié)中峡谊。雖然版本ID號(hào)沒有執(zhí)行指出調(diào)試信息文件名，但是可以從版本ID號(hào)里計(jì)算出來刊苍，參見下面既们。

由于有兩種方法可以設(shè)置調(diào)試信息文件，GDB也用兩種不同的方式查找調(diào)試信息文件：

對(duì)于“調(diào)試鏈接”方式正什，GDB在可執(zhí)行文件的目錄里查找對(duì)應(yīng)名字的文件啥纸，接著在此目錄下的子目錄’.debug’下查找，最后在全局調(diào)試目錄下的一個(gè)子目錄里查找婴氮，此子目錄的名字和可執(zhí)行文件的絕對(duì)文件名的先導(dǎo)目錄名相同斯棒。

對(duì)于“版本ID”方式，GDB在全局調(diào)試目錄下的’.build-id’子目錄下查找名為’nn/nnnnnnnn.debug’的文件主经，這里nn是版本ID字符串的頭兩個(gè)16進(jìn)制字符荣暮，nnnnnnnn是余下的字符。（真正的版本ID字符串是32個(gè)或更多的16進(jìn)制字符罩驻，不是10個(gè)穗酥。）

舉個(gè)例子，假設(shè)你要調(diào)試’/usr/bin/ls’,此程序有個(gè)調(diào)試鏈接指定了調(diào)試文件’ls.debug’惠遏，且其版本ID是是16進(jìn)制的abcdef1234砾跃。如果全局調(diào)試目錄是’/usr/lib/debug’，那么GDB按順序查找下列調(diào)試信息文件：

–‘/usr/lib/debug/.build-id/ab/cdef1234.debug’

–‘/usr/bin/ls.debug’

–‘/usr/bin/.debug/ls.debug’

–‘/usr/lib/debug/usr/bin/ls.debug’.

你可以設(shè)置全局調(diào)試信息目錄的名稱节吮，并查看當(dāng)前GDB所使用的名稱抽高。

set debug-file-directory directory

將directory設(shè)置為GDB搜索單獨(dú)調(diào)試信息文件的目錄。

show debug-file-directory

顯示搜索單獨(dú)調(diào)試信息文件的目錄透绩。

2 ? ? ? gdb 調(diào)試與符號(hào)表分離的二進(jìn)制程序

一般情況下,如果沒有在程序的 spec 中明確指定不進(jìn)行 strip,則缺省打rpm包時(shí),都會(huì)把二進(jìn)制程序或動(dòng)態(tài)庫(kù)的符號(hào)表等 debuginfo 信息與執(zhí)行程序分離,生成一個(gè)debuginfo 的 rpm 包. 如 localagent 打 rpm 包時(shí),會(huì)生成如下 3 個(gè)rpm 包:

local_agent-0.7.1-rc_1.x86_64.rpm

local_agent-debuginfo-0.7.1-rc_1.x86_64.rpm

local_agent-devel-0.7.1-rc_1.x86_64.rpm

將 local_agent-debuginfo-0.7.1-rc_1.x86_64.rpm 解壓,可以看到相應(yīng)的 debug info 文件.

[tmp]$ rpm2cpio local_agent-debuginfo-0.7.1-rc_1.x86_64.rpm |cpio –idv

./usr/local/bin/.debug/local_agent_client.debug

./usr/local/bin/.debug/local_agent_server.debug

./usr/local/lib64/.debug/liblocal_agent.so.debug

./usr/src/debug/local_agent-0.7.1

./usr/src/debug/local_agent-0.7.1/build

./usr/src/debug/local_agent-0.7.1/build/release64

./usr/src/debug/local_agent-0.7.1/build/release64/local_agent

./usr/src/debug/local_agent-0.7.1/build/release64/local_agent/ClientParamParser.cpp

./usr/src/debug/local_agent-0.7.1/build/release64/local_agent/FileUtil.cpp

./usr/src/debug/local_agent-0.7.1/build/release64/local_agent/LocalAgent.cpp

./usr/src/debug/local_agent-0.7.1/build/release64/local_agent/LocalAgentClient.cpp

./usr/src/debug/local_agent-0.7.1/build/release64/local_agent/LocalAgentClientMain.cpp

./usr/src/debug/local_agent-0.7.1/build/release64/local_agent/LocalAgentMain.cpp

./usr/src/debug/local_agent-0.7.1/build/release64/local_agent/LocalAgentServerAdapter.cpp

./usr/src/debug/local_agent-0.7.1/build/release64/local_agent/ServerParamParser.cpp

./usr/src/debug/local_agent-0.7.1/build/release64/local_agent/SystemWrapper.cpp

./usr/src/debug/local_agent-0.7.1/build/release64/local_agent/local_agent.pb.cc

./usr/src/debug/local_agent-0.7.1/build/release64/local_agent/local_agent.pb.h

./usr/src/debug/local_agent-0.7.1/local_agent

./usr/src/debug/local_agent-0.7.1/local_agent/AgentErrorDefine.h

./usr/src/debug/local_agent-0.7.1/local_agent/ClientParamParser.h

./usr/src/debug/local_agent-0.7.1/local_agent/FileUtil.h

./usr/src/debug/local_agent-0.7.1/local_agent/LocalAgent.h

./usr/src/debug/local_agent-0.7.1/local_agent/LocalAgentClient.h

./usr/src/debug/local_agent-0.7.1/local_agent/LocalAgentServerAdapter.h

./usr/src/debug/local_agent-0.7.1/local_agent/ServerParamParser.h

./usr/src/debug/local_agent-0.7.1/local_agent/SystemWrapper.h

對(duì)于這種符號(hào)表與二進(jìn)程序分離的程序,該如何調(diào)試呢? 先看下例: ? [local]$ gdb bin/local_agent_server

GNU gdb Fedora (6.8-37.el5)

License GPLv3+: GNU GPL version 3 or later

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law. ?Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-redhat-linux-gnu"...

(no debugging symbols found)

(gdb) b main

Breakpoint 1 at 0x401a80

(gdb) r

Starting program: /home/admin/tmp/usr/local/bin/local_agent_server

[Thread debugging using libthread_db enabled]

[New Thread 0x2ad3e6935860 (LWP 27869)]

Breakpoint 1, 0x0000000000401a80 in main ()

(gdb) bt

#0 ?0x0000000000401a80 in main ()

(gdb)

上述例子可以看出,因?yàn)間db 沒有符號(hào)表,所以顯示的都是二進(jìn)制地址.

2.1 ?方法一: gdb 啟動(dòng)時(shí)通過 –s 指定

對(duì)于這種符號(hào)表與二進(jìn)程序,可以在gdb 啟動(dòng)時(shí),通過 –s 指定符號(hào)表文件來解決. 如下例,通過 –s 指定debug/local_agent_server.debug 后, gdb 調(diào)試時(shí)可以看到符號(hào)表了.

[ local]$ gdb -e bin/local_agent_server -s debug/local_agent_server.debug

GNU gdb Fedora (6.8-37.el5)

License GPLv3+: GNU GPL version 3 or later

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law. ?Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-redhat-linux-gnu"...

(gdb) b main

Breakpoint 1 at 0x401a80: file build/release64/local_agent/LocalAgentMain.cpp, line 34.

(gdb) bt

No stack.

(gdb) r

Starting program: /home/admin/tmp/usr/local/bin/local_agent_server

[Thread debugging using libthread_db enabled]

[New Thread 0x2af3399a7860 (LWP 27862)]

Breakpoint 1, main (argc=1, argv=0x7fff5728f0b8) at build/release64/local_agent/LocalAgentMain.cpp:34

34 ? ? ?build/release64/local_agent/LocalAgentMain.cpp: No such file or directory.

in build/release64/local_agent/LocalAgentMain.cpp

(gdb) quit

如果是這種符號(hào)表與二進(jìn)程序分離的程序進(jìn)行所產(chǎn)生的 core ,可以通過如下方式調(diào)試:

gdb -c core.1234 -e bin/local_agent_server -s debug/local_agent_server.debug

其中：

-c 指定core文件

-e 指定binary翘骂，用線上的binary即可

-s 指定符號(hào)表，也就是我們新生成的符號(hào)表

2.2 ?方法二: 將 .debug 目錄放到可執(zhí)行程序所在的目錄

除了通過 –s 指定 debuginfo 文件外,還可以將 .debug 目錄復(fù)制到可執(zhí)行程序所在的目錄或者將 local_agent_server.debug 復(fù)制到可執(zhí)行程序所在的目錄.

[admin@s002182.cm6 local]$ pwd ?/home/admin/tmp/usr/local

[admin@s002182.cm6 local]$ cp -r debug/ bin/.debug

[admin@s002182.cm6 local]$ ls -lha bin/

total 44K

drwxr-xr-x 3 admin admin 4.0K Mar 21 18:41 .

drwxr-xr-x 6 admin admin 4.0K Mar 21 17:47 ..

drwxr-xr-x 3 admin admin 4.0K Mar 21 18:49 .debug

-rwxr-xr-x 1 admin admin ?14K Mar 21 17:44 local_agent_client

-rwxr-xr-x 1 admin admin ?14K Mar 21 17:44 local_agent_server

此時(shí),也可以看到符號(hào)表了.

[local]$ gdb bin/local_agent_server

GNU gdb Fedora (6.8-37.el5)

License GPLv3+: GNU GPL version 3 or later

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law. ?Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-redhat-linux-gnu"...

(gdb) b main

Breakpoint 1 at 0x401a80: file build/release64/local_agent/LocalAgentMain.cpp, line 34.

(gdb) q

2.3 ?Set debug-file-directory 方式不生效

嘗試用 set debug-file-directory 方式來設(shè)置 debug 路徑,好像不生效.

[admin@s002182.cm6 local]$ gdb bin/local_agent_server

GNU gdb Fedora (6.8-37.el5)

License GPLv3+: GNU GPL version 3 or later

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law. ?Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-redhat-linux-gnu"...

(no debugging symbols found)

(gdb) b main

Breakpoint 1 at 0x401a80

(gdb) show debug-file-directory

The directory where separate debug symbols are searched for is "/usr/lib/debug".

(gdb) set debug-file-directory /usr/lib/debug:/home/admin/tmp/usr/local/debug

(gdb) show debug-file-directory

The directory where separate debug symbols are searched for is "/usr/lib/debug:/home/admin/tmp/usr/local/debug".

(gdb) shell ls -lh /home/admin/tmp/usr/local/debug

total 416K

-rwxr-xr-x 1 admin admin 203K Mar 21 17:44 local_agent_client.debug

-rwxr-xr-x 1 admin admin 201K Mar 21 17:44 local_agent_server.debug

(gdb) r

Starting program: /home/admin/tmp/usr/local/bin/local_agent_server

(no debugging symbols found)

[Thread debugging using libthread_db enabled]

[New Thread 0x2b5703177860 (LWP 29389)]

Breakpoint 1, 0x0000000000401a80 in main ()

(gdb)

3 ? ? ? gdb 調(diào)試與符號(hào)表分離的動(dòng)態(tài)庫(kù)

如果二進(jìn)制程序所依賴的動(dòng)態(tài)庫(kù)是符號(hào)表等調(diào)試信息與動(dòng)態(tài)庫(kù)二進(jìn)制分離的,那又該如何讓 gdb 加載這些動(dòng)態(tài)庫(kù)的符號(hào)表呢?

3.1.1 ? 可執(zhí)行程序運(yùn)行時(shí),依賴的動(dòng)態(tài)庫(kù)是通過 rpm 包安裝的情況

這種比較簡(jiǎn)單,只要將 debuginfo 的 rpm 包將通過 rpm 安裝即可,因?yàn)?debug 目錄缺省會(huì)安裝在動(dòng)態(tài)庫(kù)所在的目錄下. 下面的 anet 庫(kù)的rpm 包解壓后,可以清楚的看到這一點(diǎn).

[admin@s002182.cm6 tmp]$ rpm2cpio anet-1.3.2-rc_2.x86_64.rpm |cpio -idv

./usr/local/lib64/libanet.so

./usr/local/lib64/libanet.so.3

./usr/local/lib64/libanet.so.3.0.0

563 blocks

[admin@s002182.cm6 tmp]$ rpm2cpio anet-debuginfo-1.3.2-rc_2.x86_64.rpm |cpio -idv

./usr/local/lib64/.debug/libanet.so.3.0.0.debug

./usr/src/debug/anet-1.3.2

./usr/src/debug/anet-1.3.2/anet

./usr/src/debug/anet-1.3.2/anet/addrspec.h

./usr/src/debug/anet-1.3.2/anet/adminclient.h

./usr/src/debug/anet-1.3.2/anet/admincmds.h

./usr/src/debug/anet-1.3.2/anet/adminserver.h

./usr/src/debug/anet-1.3.2/anet/advancepacket.h

./usr/src/debug/anet-1.3.2/anet/advancepacketfactory.h

./usr/src/debug/anet-1.3.2/anet/anet.h

./usr/src/debug/anet-1.3.2/anet/aneterror.h

./usr/src/debug/anet-1.3.2/anet/appadapter.h

3.1.2 ? 可執(zhí)行程序運(yùn)行時(shí),依賴的動(dòng)態(tài)庫(kù)是通過 rpm2cipo 解壓到某個(gè)目錄的情況

如果可執(zhí)行程序運(yùn)行時(shí),依賴的動(dòng)態(tài)庫(kù)是用戶自已通過 rpm2cpio 解壓后復(fù)制到某個(gè)用戶自定義的目錄的,那么,拿到該動(dòng)態(tài)庫(kù)的 debuginfo rpm 后,也相應(yīng)的用 rpm2cpio 解壓,并把 .debug 目錄復(fù)制到相應(yīng)的動(dòng)態(tài)庫(kù)所在的目錄.

如下面sap_server的示例, sap_server 依賴 libarpc.so.1,但 libarpc.so.1 在打是將 rpm 包時(shí), 其debuginfo 信息分離在單獨(dú)的 debuginfo rpm 包中的. 所以導(dǎo)致 gdb core時(shí),看不到 libarpc.so.1 的符號(hào)表.

gdb bin/sap_server_d core.17131

(gdb) bt

#0 ?0x0000003959230265 in raise (sig=) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64

#1 ?0x0000003959231d10 in abort () at abort.c:88

#2 ?0x000000395d6bec44 in __gnu_cxx::__verbose_terminate_handler () at ../../../../libstdc++-v3/libsupc++/vterminate.cc:97

#3 ?0x000000395d6bcdb6 in __cxxabiv1::__terminate (handler=)

at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:43

#4 ?0x000000395d6bcde3 in std::terminate () at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:53

#5 ?0x000000395d6bd2ef in __cxa_pure_virtual () at ../../../../libstdc++-v3/libsupc++/pure.cc:55

#6 ?0x00002b8729bfc933 in arpc::ClientPacketHandler::handlePacket () from /home/admin/sap/lib/libarpc.so.1

#7 ?0x00002b872a0ddcd8 in anet::Connection::handlePacket () from /home/admin/sap/lib/libanet.so.3

#8 ?0x00002b872a0e75fb in anet::TCPConnection::readData () from /home/admin/sap/lib/libanet.so.3

#9 ?0x00002b872a0e624b in anet::TCPComponent::handleReadEvent () from /home/admin/sap/lib/libanet.so.3

#10 0x00002b872a0e8c0b in anet::Transport::eventIteration () from /home/admin/sap/lib/libanet.so.3

#11 0x00002b872a0e8cf2 in anet::Transport::eventLoop () from /home/admin/sap/lib/libanet.so.3

#12 0x00002b872a0e8d47 in anet::Transport::run () from /home/admin/sap/lib/libanet.so.3

#13 0x00002b872a0ea02d in anet::Thread::hook () from /home/admin/sap/lib/libanet.so.3

#14 0x0000003959e064a7 in start_thread (arg=) at pthread_create.c:297

#15 0x00000039592d3c2d in clone () from /lib64/libc.so.6

(gdb) thread 2

要讓 gdb 調(diào)試 core 時(shí),能顯示動(dòng)態(tài)庫(kù)的符號(hào)表,可以用如下方法:

a). 先解壓 debuginfo ?rpm 包

rpm2cpio arpc-debuginfo-0.14.1-rc_1.x86_64.rpm |cipo -idv

b). ?將 ./usr/local/lib64/.debug 復(fù)制到 ?libarpc.so.1 所在目錄:

/home/admin/sap/lib/ ?cp -r ?./usr/local/lib64/.debug ?/home/admin/sap/lib/

[admin@s007238.cm6 sap]$ gdb bin/sap_server_d core.17131

GNU gdb Fedora (6.8-37.el5)

License GPLv3+: GNU GPL version 3 or later

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law. ?Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-redhat-linux-gnu"...

Reading symbols from /home/admin/sap/lib/libhb_node.so...done.

Loaded symbols for /home/admin/sap/lib/libhb_node.so

Reading symbols from /home/admin/sap/lib/libcm_basic.so...done.

Loaded symbols for /home/admin/sap/lib/libcm_basic.so

Reading symbols from /home/admin/sap/lib/libarpc.so.1...Reading symbols from /home/admin/sap/lib/.debug/libarpc.so.1.0.0.debug...done.

... ...

#0 ?0x0000003959230265 in raise (sig=) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64

64 ? ? ? ?return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);

(gdb) bt

#0 ?0x0000003959230265 in raise (sig=) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64

#1 ?0x0000003959231d10 in abort () at abort.c:88

#2 ?0x000000395d6bec44 in __gnu_cxx::__verbose_terminate_handler () at ../../../../libstdc++-v3/libsupc++/vterminate.cc:97

#3 ?0x000000395d6bcdb6 in __cxxabiv1::__terminate (handler=)

at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:43

#4 ?0x000000395d6bcde3 in std::terminate () at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:53

#5 ?0x000000395d6bd2ef in __cxa_pure_virtual () at ../../../../libstdc++-v3/libsupc++/pure.cc:55

#6 ?0x00002b8729bfc933 in arpc::ClientPacketHandler::handlePacket (this=0x2aac6df6ee38, packet=0x2aac6df6fc30,

args=) at build/release64/arpc/ClientPacketHandler.cpp:35