Why does every developer need Otx?
Otx is a disassembler based on Apple’s otool, it has some really funky features, but why should a normal developer need such a tool?
Long question, simple answer: crashlogs. A crashlog contains some really valuable information, but this information needs to be interpreted in the right way to be helpful.
So how does a crashlog look like? Crashlogs can be found in ~/Library/Logs/CrashReporter and open with the Console.app from the utilities folder. The basic structure of a crashlog is made of four parts. First there are some general informations about the computer, the app and the type of crash that happened. The second part is a backtrace of all the running threads and of the thread that crashed. After that is a dump of the CPU-state with the values of each register at the moment of the crash. Last but not least there’s the ‘Binary Image Description’ which lists all the bundles and frameworks that have been loaded into the app and at which offset they are located.
So lets have a closer look to each part.
General information
here’s a snippet of what the general information says about my computer when CuteClips crashed:
Host Name: Karstens-PowerBook Date/Time: 2006-11-30 16:54:38.423 +0100 OS Version: 10.4.8 (Build 8L127) Report Version: 4 Command: CuteClips Path: /Applications/CuteClips.app/Contents/MacOS/CuteClips Parent: WindowServer [72] Version: ??? (2.0.1) PID: 1093 Thread: 0 Exception: EXC_BREAKPOINT (0x0006) Code[0]: 0x00000001 Code[1]: 0x90a48ca0
So this part tells us the name of the host, when the crash happened, which operating system the host is running and which version of the report you have. The version seems to always be four on my computer, maybe the logs differ in the system-version, donno. 😉
Next is the name of the app that crashed and the path from which it was launched. Parent is the process that started the app, in that case WindowServer. The version is 2.0.2 and it was running with process id 3288. Thread 0 was obviously running and also the one that crashed.
The exception is a breakpoint that has been hit. In this case it’s a objc_trap. Seems a message was sent to the wrong object or something like that, but we’ll find more about that later.
Backtraces
That’s the most interesting part and the part that is the reason for that post 😉
This section lists all the threads of the app and also the one that was crashing.
Here’s the thread that crashed in my log:
Thread 0 Crashed: 0 libobjc.A.dylib 0x90a48ca0 _objc_trap + 0 1 libobjc.A.dylib 0x90a48c14 _objc_error + 76 2 libobjc.A.dylib 0x90a48ba4 __objc_error + 64 3 com.briksoftware.CuteClips 0x00013a00 0x1000 + 76288 4 com.briksoftware.searcher4 0x007c88c0 -[Searcher4 addObjectToIndex:withName:] + 52 5 com.briksoftware.searcher4 0x007c8a04 -[Searcher4 controller:] + 208 6 com.briksoftware.CuteClips 0x000187a8 0x1000 + 96168 7 com.briksoftware.CuteClips 0x00016148 0x1000 + 86344 8 com.briksoftware.CuteClips 0x000159b8 0x1000 + 84408 9 com.briksoftware.CuteClips 0x00003878 0x1000 + 10360 10 com.apple.Foundation 0x9296b668 -[NSSet makeObjectsPerformSelector:] + 164 11 com.apple.AppKit 0x93725a70 -[NSIBObjectData nibInstantiateWithOwner:topLevelObjects:] + 864 12 com.apple.AppKit 0x93711c9c loadNib + 240 13 com.apple.AppKit 0x937116f4 +[NSBundle(NSNibLoading) _loadNibFile:nameTable:withZone:ownerBundle:] + 716 14 com.apple.AppKit 0x93768bc4 +[NSBundle(NSNibLoading) loadNibFile:externalNameTable:withZone:] + 156 15 com.apple.AppKit 0x937f8a70 +[NSBundle(NSNibLoading) loadNibNamed:owner:] + 344 16 com.apple.AppKit 0x937f8810 NSApplicationMain + 344 17 com.briksoftware.CuteClips 0x00002d6c 0x1000 + 7532 18 com.briksoftware.CuteClips 0x00002be0 0x1000 + 7136
So there’s a lot of information here. First thing is the name of the thread: Thread 0. In this case it crashed obviously.
Then there’re running numbers on the left, some bundle-id and some offset followed by a function-name or yet another offset.
What does all that mean?
Each line is a thread frame. The top most is the function that is being executed when the thread crashed and the frame that follows is the one that should continue execution when the current frame finishes.
The bundle id shows to which bundle the code belongs. There’re functions from libobjc.A.dylib which is the objective-c runtime engine. com.briksoftware.CuteClips is code from the application itself, which can also be seen at the offset, which is pretty small (0x00013a00).
This is the point where the disassembler comes in, but i’ll just finish off with the explanation of the crashlog.
CPU-State
Well that part is pretty simple. It shows the value of each register in the CPU in the moment when the app crashed.
Not very interesting for most kind of bugs and crashes.
Loaded bundles
This section shows which code is loaded from which file and which offset and at which offset it has been relocated into the virtual memory.
0x1000 - 0xabfff com.briksoftware.CuteClips ??? (2.0.1) /Applications/CuteClips.app/Contents/MacOS/CuteClips 0xb7000 - 0xc4fff org.andymatuschak.Sparkle ??? (1.1) /Applications/CuteClips.app/Contents/Frameworks/Sparkle.framework/Versions/A/Sparkle ...
The first column is the offset in the vm, the second is the file offset, then comes the bundleid with version and at the end the path of the file.
How does Otx come in here?
So, lets get back to finding the crash.
We have the offset that caused the crash, so lets find that offset in the source-code.
First we need to disassemble the application. This can be done with the otx application for people who like a nice gui, or if you prefer the terminal for such proposes (like i do), you can disassemble it via command line and pipe it directly into a text-editor. Check out the latest beta from the otx svn-server for the command line version. As the text that comes out of otx is quite long, you should use a text-editor that can handle such long files. Textedit is quite inappropriate for this task, but the free TextWrangler or its big brother BBEdit do a fine job on this.
Now it’s time to find the place where your app crashes and find the matching place in the source-code. The thread’s backtrace shows:
0 libobjc.A.dylib 0x90a48ca0 _objc_trap + 0 1 libobjc.A.dylib 0x90a48c14 _objc_error + 76 2 libobjc.A.dylib 0x90a48ba4 __objc_error + 64 3 com.briksoftware.CuteClips 0x00013a00 0x1000 + 76288 4 com.briksoftware.searcher4 0x007c88c0 -[Searcher4 addObjectToIndex:withName:] + 52 ....
so let’s search for 00013a00 in the disassembly. It finds the offset here:
-(id)[ClipControl(PreviewStuff) stringForDict:length:] +0 000139c4 7c0802a6 mfspr r0,lr +4 000139c8 bf41ffe8 stmw r26,0xffe8(r1) +8 000139cc 3c80000b lis r4,0xb +12 000139d0 7c7e1b78 or r30,r3,r3 +16 000139d4 7cdd3378 or r29,r6,r6 +20 000139d8 7cbf2b78 or r31,r5,r5 +24 000139dc 60000000 nop +28 000139e0 90010008 stw r0,0x8(r1) +32 000139e4 9421ffa0 stwu r1,0xffa0(r1) +36 000139e8 60000000 nop +40 000139ec 60000000 nop +44 000139f0 60000000 nop +48 000139f4 60000000 nop +52 000139f8 808409e0 lwz r4,0x9e0(r4) stringForDict: +56 000139fc 4800d0f5 bl 0x20af0 [r3 stringForDict:] +60 00013a00 7c7c1b79 or. r28,r3,r3 +64 00013a04 4182002c beq 0x13a30 +68 00013a08 3c80000b lis r4,0xb
Oh, evil ppc instructions ;-)…nothing to worry about, ppc is easier to read than i386 imho. The crash happens when r3 receives the stringForDict: message. Otx is so kind to show us the message send in the normal Obj-c style, so kudos to the dev for this really neat feature! But what value does register 3 have? Obviously it’s never changed in the view instructions before that message-send, so it’s still the receiver of the current method: stringForDict:length: which was also sent to r3. Now that’s really weird, isn’t it? When r3 is valid at the beginning of the function, why is it invalid when it receives the message stringForDict:? Debugging this would reveal that this message is actually being send.
If this message is sent, where does it crash then, and why don’t we see this method in the crashlog? The answer is quite easy when we take a look at the method that is being called here.
The method contains the line:
+364 000139a8 4800d148 b 0x20af0 [r3 string]
so r3 receives the message string this time, but the instruction is different this time. This time it’s a different branch instruction: b, and in the previous case it was a branch instruction with bl. b only sets the process-counter to the new offset and continues execution there. This line can be compared with the c-statement:
return [r3 string];
How can this place be found in the source-code?
As the code above shows, there’s already the name of the method: stringForDict:length: and it’s implemented in the class ClipControl. Finding the class and the method in Xcode should not be a problem now. Looking at the source shows, that the method stringForDict: indeed has a statement like return [r3 string].
Eliminating the bug was another story, but in version 2.0.2 of CuteClips it has been fixed 🙂
I hope that ‘little’ post was interesting and helpful for other developers out there!
One additional note though: make sure to disassemble the app for the architecture of the crashlog, otherways you’ll find the wrong or no matching offset in the disassembly if your own mac has a different cpu.
Karsten
here’re the links again:
December 11th, 2006 at 22:59
Very nice! Looks like someone found the field width variables 🙂
May 9th, 2007 at 13:54
[…] Debugging Crash Logs with OTX […]
July 21st, 2010 at 20:25
[…] already written about why every developer should know his way around with otx so this is sort of like a follow up on this […]