I use OpenSolaris nearly exclusively since snv_131 or for about six weeks now. Having used Linux for years, its quite a new world for me, although its similar. After my gnome-panel crashes from time to time I looked around how to find out where exactly it crashes. Under Linux you would have used the backtrace function in gdb, but OpenSolaris uses Sun Studio compilers and is different. I stumbled upon a couple of articles how to do this.
There seems to be three standard ways.
$ pstack core core 'core' of 3853: gnome-panel --sm-config-prefix /gnome-panel-gkaWKj/ --sm-client-id 10f fc6c2d84 ???????? (80476a0, 0, 80476c8, fed1395a) fed139c0 _gdk_events_queue (80fd000) + 74 fed13b92 gdk_event_dispatch (8105ed8, 0, 0, fe754ddc) + 2e fe754fa2 g_main_context_dispatch (8105f20, 0, 85bc6d8, 17) + 262 fe755653 g_main_context_iterate (8105f20, 1, 1, 80de700) + 483 fe755c7d g_main_loop_run (81ba250, 81ba250, 8047888, fe97f466) + 1dd fe97f50f gtk_main (80478a0, feffb804, 80478d8, 8074b9d, 7, 80478e4) + b7 0807902e main (7, 80478e4, 8047904, 8074b3f) + 15a 08074b9d _start (7, 8047a14, 8047a20, 8047a33, 8047a48, 8047a57) + 7d
You can see its somewhere in _gdk_events_queue() at address fc6c2d84. The interesting thing on pstack is, that you can use this for running processes, too.
$ mdb core Loading modules: [ libuutil.so.1 ld.so.1 ] > ::status debugging core file of gnome-panel (32-bit) from stars file: /usr/bin/gnome-panel initial argv: gnome-panel --sm-config-prefix /gnome-panel-gkaWKj/ --sm-client-id 10fb8f7f3338 threading model: native threads status: process terminated by SIGSEGV (Segmentation Fault), addr=fc6c2d84 > $c1 ! head -10 0xfc6c2d84(80476a0) libgdk-x11-2.0.so.0.1800.5`_gdk_events_queue+0x74(80fd000) libgdk-x11-2.0.so.0.1800.5`gdk_event_dispatch+0x2e(8105ed8) libglib-2.0.so.0.2200.3`g_main_context_dispatch+0x262(8105f20) libglib-2.0.so.0.2200.3`g_main_context_iterate+0x483(8105f20) libglib-2.0.so.0.2200.3`g_main_loop_run+0x1dd(81ba250) libgtk-x11-2.0.so.0.1800.5`gtk_main+0xb7(80478a0) main+0x15a(7) _start+0x7d(7)
Doesn't tell you more for real, just you can see the library being called, too. Unlike pstack you are now in a real debugger. So you can investigate a bit more, although not on source level, as mdb is just an assembly level debugger.
> 80476a0::dis 0x80476a0: andl %eax,(%eax) 0x80476a2: addb %al,(%eax) 0x80476a4: leave 0x80476a5: decl %edi 0x80476a6: addb %al,(%eax) 0x80476a8: addl %eax,(%eax) 0x80476aa: addb %al,(%eax) 0x80476ac: incl %eax 0x80476ad: orl $0x20069c08,(%edi) 0x80476b3: addl %esi,0x3(%ebp) 0x80476b6: addb %al,(%eax) >So you see where it crashed on assembly level and can even get the values of registers at crash time. Sadly I'm not familiar with assembly anymore - I knew a bit 8086 assembly on MS-DOS around 15 years ago - so I can't give further information on that topic. There is an extensive guide on mdb and some blog entries as well.
Using xdb
As I stated before mdb is just an assembly level debugger. When you are used to gdb, you will miss a source level debugger. There is one for Sun Studio as well: xdb
$ dbx -c 'where;quit' - core Corefile specified executable: "/usr/bin/gnome-panel" Reading gnome-panel core file header read successfully Reading ld.so.1 Reading libc.so.1 ... Reading libpixbufloader-xpm.so t@1 (l@1) program terminated by signal SEGV (no mapping at the fault address) 0xffffffffffffffff: current thread: t@1 [1] 0xfc6c2d84(0x80f8140, 0x120069c, 0x80476a0, 0x0), at 0xfc6c2d84 [2] XFilterEvent(0x80476a0, 0x0, 0x80476c8, 0xfed1395a), at 0xfe47ad91 [3] _gdk_events_queue(0x80fd000), at 0xfed139c0 [4] gdk_event_dispatch(0x8105ed8, 0x0, 0x0, 0xfe754ddc), at 0xfed13b92 [5] g_main_context_dispatch(0x8105f20, 0x0, 0x85bc6d8, 0x17), at 0xfe754fa2 [6] g_main_context_iterate(0x8105f20, 0x1, 0x1, 0x80de700), at 0xfe755653 [7] g_main_loop_run(0x81ba250, 0x81ba250, 0x8047888, 0xfe97f466), at 0xfe755c7d [8] gtk_main(0x80478a0, 0xfeffb804, 0x80478d8, 0x8074b9d, 0x7, 0x80478e4), at 0xfe97f50f [9] main(0x7, 0x80478e4, 0x8047904, 0x8074b3f), at 0x807902eAnd as you can see, it even mapped one function deeper:
XFilterEvent().Conclusion
The tools on OpenSolaris are different, but not necessarily worse than Linux or GNU tools. This blog entry is just a starting point and a note to myself how to use them. Feel free to comment and supply more useful information.
Possibly Relevant Content:
Recent comments
1 day 22 hours ago
2 days 28 min ago
2 days 1 hour ago
2 days 1 hour ago
8 weeks 4 days ago
8 weeks 5 days ago
8 weeks 5 days ago
8 weeks 5 days ago
12 weeks 2 days ago
13 weeks 5 days ago