一顾孽、breakpad簡介
breakpad 而是一個全平臺的C/C++程序的崩潰日志收集工具庄岖，適配了Windows/MacOX/Linux，當然也支持了Android蝴猪。
breadpad的工作原理如下圖所示:
image.png
breadpad 主要包含三個模塊：
client
編譯進入項目中，隨項目一起編譯發(fā)布蛔糯，發(fā)布出去的so是strip掉debug信息的拯腮。當在用戶手機上崩潰的時候，client就收集信息蚁飒，寫入特定格式的崩潰文件。文件最后被收集到服務(wù)端萝喘。Breakpad symbol dumper （dump_syms工具）
當你在編譯so的時候淮逻，除了編譯strip后的so，還得保留strip前的so阁簸。dump_syms 就是用來從strip前的so 提取符號信息.sym文件
如何獲得保留strip信息的so?
假如你是用gradle編譯的話爬早，strip前的so和strip后的so都是存在的，可以在特定目錄查找,如下：
└── transforms
├── mergeJniLibs
│ └── debug
│ ├── 0
│ │ └── lib
│ │ └── arm64-v8a
│ │ └── libbreakpad-core.so
│ └── __content__.json
└── stripDebugSymbol
└── debug
├── 0
│ └── lib
│ └── arm64-v8a
│ └── libbreakpad-core.so
└── __content__.json
app/build/intermediates/transforms/ 目錄下启妹，mergeJniLibs中的so是strip debug信息之前的so;stripDebugSymbol中的so是strip掉debug信息之后的so
找到未strip的so之后筛严，可以用以下命令提取出符號信息
./dump_syms libbreakpad-core.so > libbreakpad-core.so.sy
- minidump processer（minidump_stackwalk工具）
sym符號文件和.dmp minidump文件發(fā)送給Server端后，通過minidump_stackwalk指令饶米，從.sym符號文件和包含崩潰信息的.dmp文件中提取出完整的奔潰時的堆棧信息桨啃。
二、下載breakpad源碼
可以從breadpad-github檬输，然后編譯照瘾。
很人反映從官網(wǎng)下載的breadpad源碼編譯會報錯，需要補充缺失的文件丧慈。
所以也可以直接
feifei-123的github直接下載可以編譯通過的版本析命。
執(zhí)行命令:
git clone https://github.com/feifei-123/breakpad
三、編譯breadpad
breakpad 是跨平臺的，支持linux鹃愤、window和Mac os系統(tǒng)簇搅，不同平臺上的編譯配置也是不同的。
- linux 平臺編譯出來的dump_syms 僅能再linux上運行软吐，來解析linux上運行的so的符號信息
- macOS 平臺編譯出來的dump_syms 僅能再mac OS 上運行馍资，來解析mac 上運行的so的符號信息。
- window 平臺編譯出來的dump_syms关噪，僅能在Window上運行鸟蟹，并解析window上運行的dll的符號信息。
因為我最終的應(yīng)用場景時 捕捉Android上的natvie crash使兔，然后解析奔潰時的堆棧建钥，而Android 是屬于Linux系統(tǒng)的。
所以編譯breadpad 也必須在Linux系統(tǒng)上進行虐沥。否則編譯出來的dump_syms 無法解析android的minidump信息的熊经。
編譯過程:
清空上次build的臨時文件
make clean
運行配置信息
./configure
parallels@parallels-vm:~/Desktop/Parallels Shared Folders/Home/Desktop/TM/Github/orginal_breakpad/breakpad$ ./configure
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether UID '1000' is supported by ustar format... yes
checking whether GID '1000' is supported by ustar format... yes
checking how to create a ustar tar archive... gnutar
checking whether to enable maintainer-specific portions of Makefiles... no
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for ar... ar
checking the archiver (ar) interface... ar
checking dependency style of gcc... gcc3
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
checking how to run the C preprocessor... gcc -E
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking for ranlib... ranlib
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking a.out.h usability... yes
checking a.out.h presence... yes
checking for a.out.h... yes
checking sys/random.h usability... no
checking sys/random.h presence... no
checking for sys/random.h... no
checking for arc4random... no
checking for getrandom... no
checking whether g++ supports C++11 features by default... no
checking whether g++ supports C++11 features with -std=c++11... yes
checking whether C++ compiler accepts -Werror=unknown-warning-option... no
checking whether C++ compiler accepts -Wmissing-braces... yes
checking whether C++ compiler accepts -Wnon-virtual-dtor... yes
checking whether C++ compiler accepts -Woverloaded-virtual... yes
checking whether C++ compiler accepts -Wreorder... yes
checking whether C++ compiler accepts -Wsign-compare... yes
checking whether C++ compiler accepts -Wunused-local-typedefs... yes
checking whether C++ compiler accepts -Wunused-variable... yes
checking whether C++ compiler accepts -Wvla... yes
checking for O_CLOEXEC defined in fcntl.h... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating breakpad.pc
config.status: creating breakpad-client.pc
config.status: creating Makefile
config.status: creating src/config.h
config.status: src/config.h is unchanged
config.status: executing depfiles commands
注意 checking build system type... x86_64-pc-linux-gnu 表明我當前的編譯環(huán)境是linux
make 執(zhí)行編譯
make
編譯完成后
libbreakpad_client.a 位于/src/client/linux 目錄下，此文件可以編譯進android 的app,完成native crash的捕捉和生成minidump文件
dump_syms 文件位于 /src/tools/linux/dump_syms 目錄欲险，用于提取so庫的sym符號文件
parallels@parallels-vm:~/Desktop/Parallels Shared Folders/Home/Desktop/TM/Github/orginal_breakpad/breakpad/src/tools/linux/dump_syms$ ls
dump_syms dump_syms.cc src_tools_linux_dump_syms_dump_syms-dump_syms.o
- minidump_stackwalk 位于/src/processor目錄下镐依，用于將.dmp minudump文件和.sym文件合成可讀的堆棧信息
parallels@parallels-vm:~/Desktop/Parallels Shared Folders/Home/Desktop/TM/Github/orginal_breakpad/breakpad/src/processor$ ls | grep minidump_stackwalk
minidump_stackwalk
minidump_stackwalk.cc
minidump_stackwalk_machine_readable_test
minidump_stackwalk.o
minidump_stackwalk_test
三、解析minudump文件
1天试、 dump_syms 提取特定so庫的符號信息
以libbreakpad-core.so為例：
./dump_syms libbreakpad-core.so > libbreakpad-core.so.sym
2槐壳、根據(jù)1中生成的libbreakpad-core.so.sym生成特定的目錄結(jié)構(gòu)：
├── symbol
│ └── libbreakpad-core.so
│ └── 57399AA1EE2607A34686D5DED7D43C310
│ └── libbreakpad-core.so.sym
命令如下:
head -n1 libbreakpad-core.so.sym
MODULE Linux arm64 57399AA1EE2607A34686D5DED7D43C310 libbreakpad-core.so
mkdir -p ./symbol/libbreakpad-core.so/57399AA1EE2607A34686D5DED7D43C310
mv libbreakpad-core.so.sym ./symbol/libbreakpad-core.so/57399AA1EE2607A34686D5DED7D43C310/
3、調(diào)用minidump_stackwalk命令喜每，將dmp文件和sym文件合成可讀的crashinfo.txt
./minidump_stackwalk 8439c979-cecf-41a6-25eaf89c-dbf9c03b.dmp ./symbol > crashinfo.txt
crashinfo.txt 部分內(nèi)容如下:
Operating system: Android
0.0.0 Linux 4.4.83 #2 SMP PREEMPT Sun Jan 12 10:48:20 CST 2020 aarch64
CPU: arm64
4 CPUs
GPU: UNKNOWN
Crash reason: SIGSEGV /SEGV_MAPERR
Crash address: 0x0
Process uptime: not available
Thread 32 (crashed)
0 libbreakpad-core.so!testCrash1() + 0x14
x0 = 0x0000000000000027 x1 = 0x00000078e60f5f30
x2 = 0x0000000000000005 x3 = 0x0000000000000003
x4 = 0x0000000040100401 x5 = 0xa800000040404000
x6 = 0x0000000000000000 x7 = 0x7f7f7f7f7f7f7f7f
x8 = 0x0000000000000000 x9 = 0x0000000000000001
x10 = 0x00000078e60f60c0 x11 = 0x0000000000000018
x12 = 0x000000000000000b x13 = 0xffffffffffffffff
x14 = 0xff00000000000000 x15 = 0xffffffffffffffff
x16 = 0x00000078e82b7608 x17 = 0x00000078e823b454
x18 = 0x0000000000000008 x19 = 0x00000078e7e2fe00
x20 = 0x0000007902ae7f20 x21 = 0x00000078e7e2fe00
x22 = 0x00000078e60f693c x23 = 0x00000078e8f1bc31
x24 = 0x0000000000000000 x25 = 0x00000078e60f7588
x26 = 0x00000078e7e2fea0 x27 = 0x0000000000000000
x28 = 0x0000000000000000 fp = 0x00000078e60f6650
lr = 0x00000078e823b4c8 sp = 0x00000078e60f6620
pc = 0x00000078e823b468
Found by: given as instruction pointer in context
1 libbreakpad-core.so!testCoffeCacher() + 0x4c
x19 = 0x00000078e7e2fe00 x20 = 0x0000007902ae7f20
x21 = 0x00000078e7e2fe00 x22 = 0x00000078e60f693c
x23 = 0x00000078e8f1bc31 x24 = 0x0000000000000000
x25 = 0x00000078e60f7588 x26 = 0x00000078e7e2fea0
x27 = 0x0000000000000000 x28 = 0x0000000000000000
fp = 0x00000078e60f6650 sp = 0x00000078e60f6630
pc = 0x00000078e823b4c8
Found by: call frame info
2 libbreakpad-core.so!call_dangerous_function() + 0x14
x19 = 0x00000078e7e2fe00 x20 = 0x0000007902ae7f20
x21 = 0x00000078e7e2fe00 x22 = 0x00000078e60f693c
x23 = 0x00000078e8f1bc31 x24 = 0x0000000000000000
x25 = 0x00000078e60f7588 x26 = 0x00000078e7e2fea0
x27 = 0x0000000000000000 x28 = 0x0000000000000000
fp = 0x00000078e60f6670 sp = 0x00000078e60f6660
pc = 0x00000078e823b508
Found by: call frame info
3 libbreakpad-core.so!Java_com_sogou_translate_jni_BreakpadInit_go2crash + 0x14
x19 = 0x00000078e7e2fe00 x20 = 0x0000007902ae7f20
x21 = 0x00000078e7e2fe00 x22 = 0x00000078e60f693c
x23 = 0x00000078e8f1bc31 x24 = 0x0000000000000000
x25 = 0x00000078e60f7588 x26 = 0x00000078e7e2fea0
x27 = 0x0000000000000000 x28 = 0x0000000000000000
fp = 0x00000078e60f6690 sp = 0x00000078e60f6680
pc = 0x00000078e823b534
Found by: call frame info
4 base.odex + 0x111c0
x19 = 0x00000078e7e2fe00 x20 = 0x0000007902ae7f20
x21 = 0x00000078e7e2fe00 x22 = 0x00000078e60f693c
x23 = 0x00000078e8f1bc31 x24 = 0x0000000000000000
x25 = 0x00000078e60f7588 x26 = 0x00000078e7e2fea0
x27 = 0x0000000000000000 x28 = 0x0000000000000000
fp = 0x00000078e60f6768 sp = 0x00000078e60f66a0
pc = 0x00000078e8e261c4
Found by: call frame info
四务唐、一路踩的坑
因為我自己的筆記本是mac pro,最初編譯breakpad是在mac上編譯的。
執(zhí)行./configure & make 成功編譯之后
./configure & make
feifeideMacBook-Pro:src feifei$ ./configure & make
[1] 45277
make: *** No targets specified and no makefile found. Stop.
feifeideMacBook-Pro:src feifei$ checking build system type... x86_64-apple-darwin18.5.0
checking host system type... x86_64-apple-darwin18.5.0
在/src/client/linux中未生成 dump_syms文件,
feifeideMacBook-Pro:dump_syms feifei$ pwd
/Users/feifei/Desktop/tmp/breakpad/src/src/tools/linux/dump_syms
feifeideMacBook-Pro:dump_syms feifei$ ls
dump_syms.cc
進而去/src/client/mac中 存在一個dump_syms.xcodeproj文件带兜。
/Users/feifei/Desktop/tmp/breakpad/src/src/tools/mac/dump_syms
feifeideMacBook-Pro:dump_syms feifei$ ls
dump_syms.xcodeproj dump_syms_tool.cc macho_dump.cc
利用xcode 打開dump_syms.xcodeproj編譯通過
image.png
得到dump_syms,利用該dump_syms 解析Android 工程中的 libbreakpad-core.so 失敗枫笛。
feifeideMacBook-Pro:try feifei$ mac/dump_syms libbreakpad-core.so > test.txt
libbreakpad-core.so: file is neither a fat binary file nor a Mach-O object file
但是src/processor下的minidump_stackwalk 可以直接處理android 項目生成的dmp文件的.
feifeideMacBook-Pro:processor feifei$ pwd
/Users/feifei/Desktop/tmp/breakpad/src/processor
feifeideMacBook-Pro:processor feifei$ ls | grep minidump_stackwalk
minidump_stackwalk
minidump_stackwalk.cc
minidump_stackwalk.o
minidump_stackwalk_machine_readable_test
minidump_stackwalk_test
/Users/feifei/Desktop/tmp/breakpad/src/processor/minidump_stackwalk 094495a4-c61d-473e-8505f986-72daf425.dmp > crash.info
最終解決辦法:安裝了一個ubuntn 虛擬機，在虛擬機中對breadpad 進行編譯刚照，搞定~
五刑巧、breadpad 解析奔潰堆棧的另一種玩法
1、minidump_stackwalk 直接將.dmp 文件解析成可讀的信息无畔，但是缺少堆棧的解析(奔潰點的函數(shù)符號和行號)
Android studo安裝目錄自帶了一份minidump_stackwalk自帶的minidump_stackwalk可以在mac上直接運行 解析dmp文件
查找minidump_stackwalk位置
find / -name minidump_stackwalk
我mac上的目錄為
/Applications/Android Studio.app/Contents/bin/lldb/bin/minidump_stackwalk
也可以直接使用“四”步中mac 環(huán)境編譯生成的 minidump_stackwalk,位置如下:
breakpad/src/src/processor/minidump_stackwalk
minidump_stackwalk 解析minidump文件
/Users/feifei/Library/Android/sdk/lldb/3.1/bin/minidump_stackwalk 094495a4-c61d-473e-8505f986-72daf425.dmp > crash.info
crash.info 內(nèi)容:
Operating system: Android
0.0.0 Linux 4.4.83 #2 SMP PREEMPT Sun Jan 12 10:48:20 CST 2020 aarch64
CPU: arm64
4 CPUs
GPU: UNKNOWN
Crash reason: SIGSEGV /SEGV_MAPERR
Crash address: 0x0
Process uptime: not available
Thread 32 (crashed)
0 libbreakpad-core.so + 0x28468
x0 = 0x0000000000000027 x1 = 0x00000078e6b82f30
x2 = 0x0000000000000005 x3 = 0x0000000000000003
x4 = 0x0000000040100401 x5 = 0xa800000040404000
x6 = 0x0000000000000000 x7 = 0x7f7f7f7f7f7f7f7f
x8 = 0x0000000000000000 x9 = 0x0000000000000001
x10 = 0x00000078e6b830c0 x11 = 0x0000000000000018
x12 = 0x000000000000000b x13 = 0xffffffffffffffff
x14 = 0xff00000000000000 x15 = 0xffffffffffffffff
x16 = 0x00000078e86ec608 x17 = 0x00000078e8670454
x18 = 0x00000078e6b8166c x19 = 0x00000078fa1e8800
x20 = 0x0000007902ae7f20 x21 = 0x00000078fa1e8800
x22 = 0x00000078e6b8393c x23 = 0x00000078e9018c31
x24 = 0x0000000000000000 x25 = 0x00000078e6b84588
x26 = 0x00000078fa1e88a0 x27 = 0x0000000000000000
x28 = 0x0000000000000000 fp = 0x00000078e6b83650
lr = 0x00000078e86704c8 sp = 0x00000078e6b83620
pc = 0x00000078e8670468
Found by: given as instruction pointer in context
1 libbreakpad-core.so + 0x284c4
fp = 0x00000078e6b83670 lr = 0x00000078e8670508
sp = 0x00000078e6b83660 pc = 0x00000078e86704c8
Found by: previous frame's frame pointer
2 libbreakpad-core.so + 0x28504
fp = 0x00000078e6b83690 lr = 0x00000078e8670534
sp = 0x00000078e6b83680 pc = 0x00000078e8670508
Found by: previous frame's frame pointer
3 libbreakpad-core.so + 0x28530
由此可知程序 奔潰在了libbreakpad-core.so 的相對偏移位置 0x28468的地址
調(diào)用的關(guān)系如下：
libbreakpad-core.so + 0x28468
libbreakpad-core.so + 0x284c4
libbreakpad-core.so + 0x28504
libbreakpad-core.so + 0x28530
2啊楚、利用addr2line 根據(jù)發(fā)生crash的so文件，以及偏移地址檩互，得出產(chǎn)生carsh的方法特幔、行數(shù)和調(diào)用堆棧關(guān)系。
aarch64-linux-android-addr2line 工具也是在android sdk 安裝目錄下自帶的闸昨，可以自行查找蚯斯。
feifeideMacBook-Pro:~ feifei$ find / -name aarch64-linux-android-addr2line
我mac上aarch64-linux-android-addr2line的目錄為
/Users/feifei/Library/Android/sdk/ndk-bundle/toolchains/aarch64-linux-android-4.9/prebuilt/darwin-x86_64/bin/aarch64-linux-android-addr2line
針對上面得出的調(diào)用堆棧薄风，解析libbreakpad-core.so 的 0x28468、0x284c4拍嵌、0x28504遭赂、0x28530分別對應(yīng)哪個函數(shù)符號
libbreakpad-core.so + 0x28468
libbreakpad-core.so + 0x284c4
libbreakpad-core.so + 0x28504
libbreakpad-core.so + 0x28530
feifeideMacBook-Pro:try feifei$ /Users/feifei/Library/Android/sdk/ndk-bundle/toolchains/aarch64-linux-android-4.9/prebuilt/darwin-x86_64/bin/aarch64-linux-android-addr2line -f -C -e libbreakpad-core.so 0x28468 0x284c4 0x28504 0x28530
testCrash1()
??:?
testCoffeCacher()
??:?
call_dangerous_function()
??:?
Java_com_sogou_translate_jni_BreakpadInit_go2crash
??:?
可以得出實際的方法調(diào)用棧為:
testCrash1()
testCoffeCacher()
call_dangerous_function()
Java_com_sogou_translate_jni_BreakpadInit_go2crash
arm-linux-androideabi-addr2line 使用方法介紹:
arm-linux-androideabi-addr2line -C -f -e ${SOPATH} ${Address}
-C -f //打印錯誤行數(shù)所在的函數(shù)名稱
-e //打印錯誤地址的對應(yīng)路徑及行數(shù)
${SOPATH} //so庫路徑
${Address} //需要轉(zhuǎn)換的堆棧錯誤信息地址，可以添加多個横辆，但是中間要用空格隔開
