[Return to Library] [TOC] [PREV] SECT-- [INDEX] [Help]

A    Output from the crashdc Command

This appendix contains a sample crash-data.n file created by the crashdc command. The output is explained in the list following the example. 
#
# Crash Data Collection (Version 1.4)
#
_crash_data_collection_time: Fri Sep  2 15:01:07 EDT 1994   (1)
_current_directory: /
_crash_kernel: /var/adm/crash/vmunix.0
_crash_core: /var/adm/crash/vmcore.0
_crash_arch: alpha
_crash_os: DEC OSF/1
_host_version: DEC OSF/1 3.0  (Rev. 331); Thu Sep  1 09:24:01 EDT 1994
_crash_version: DEC OSF/1 3.0  (Rev. 331); Thu Sep  1 09:24:01 EDT 1994
_crashtime:  struct {
    tv_sec = 746996332
    tv_usec = 145424
}
_boottime:  struct {
    tv_sec = 746993148
    tv_usec = 92720
}
_config:  struct {
    sysname = "OSF1"
    nodename = "madmax.zk3.dec.com"
    release = "3.0"
    version = "331"
    machine = "alpha"
}
_cpu:  30
_system_string:  0xfffffc0000442fa8 = "DEC3000 - M500"
_avail_cpus:  1
_partial_dump:  1
_physmem(MBytes):  96
_panic_string:  0xfffffc000043cf70 = "kernel memory fault" (2)
_preserved_message_buffer_begin: (3)
struct {
    msg_magic = 0x63061
    msg_bufx = 0x56e
    msg_bufr = 0x432
    msg_bufc = "Alpha boot: available memory from 0x678000 to 0x6000000
DEC OSF/1 T2.0-1  (Rev. 114.2); Wed Sep  1 09:24:01 EDT 1993
physical memory = 94.00 megabytes.
available memory = 84.50 megabytes.
using 360 buffers containing 2.81 megabytes of memory
tc0 at nexus
scc0 at tc0 slot 7
tcds0 at tc0 slot 6
asc0 at tcds0 slot 0
rz0 at asc0 bus 0 target 0 lun 0 (DEC     RZ26     (C) DEC T384)
rz4 at asc0 bus 0 target 4 lun 0 (DEC     RRD42   (C) DEC  4.5d)
tz5 at asc0 bus 0 target 5 lun 0 (DEC     TLZ06     (C)DEC 0374)
asc1 at tcds0 slot 1
rz8 at asc1 bus 1 target 0 lun 0 (DEC     RZ57     (C) DEC 5000)
rz9 at asc1 bus 1 target 1 lun 0 (DEC     RZ56     (C) DEC 0300)
fb0 at tc0 slot 8
 1280X1024
bba0 at tc0 slot 7
ln0: DEC LANCE Module Name: PMAD-BA
ln0 at tc0 slot 7
ln0: DEC LANCE Ethernet Interface, hardware address: 08-00-2b-2c-f3-83
DEC3000 - M500 system
Firmware revision: 2.4
PALcode: OSF version 1.28
lvm0: configured.
lvm1: configured.
<3>/var: file system full
<3>/var: file system full
<3>/var: file system full
<3>/var: file system full
<3>/var: file system full

trap: invalid memory ifetch access from kernel mode

    faulting virtual address:     0x0000000000000000
    pc of faulting instruction:   0x0000000000000000
    ra contents at time of fault: 0xfffffc000028951c
    sp contents at time of fault: 0xffffffff96199a48

panic: kernel memory fault
syncing disks... done
"
}
_preserved_message_buffer_end:
_kernel_process_status_begin:  (4)
  PID	COMM
00000	kernel idle
00001	init
00002	exception hdlr
00342	xdm
00012	update
00341	Xdec
00239	nfsiod
00113	syslogd
00115	binlogd
00240	nfsiod
00241	nfsiod
00340	csh
00124	routed
00188	portmap
00197	ypbind
00237	nfsiod
00249	sendmail
00294	internet_mom
00297	snmp_pe
00291	mold
00337	xdm
00325	lpd
00310	cron
00305	inetd
00489	tar
_kernel_process_status_end:
_current_pid:  489  (5)
_current_tid:  0xffffffff863d36c0  (6)
_proc_thread_list_begin:
thread 0x863d36c0 stopped at [boot:1118,0xfffffc0000374a08] Source not available
_proc_thread_list_end:
_dump_begin: (7)
>  0 boot(reason = 0, howto = 0) ["../../../../src/kernel/arch/alpha/machdep.c":
1118, 0xfffffc0000374a08]
mp = 0xffffffff961962f8
nmp = 0xffffffff86333ab8
fsp = (nil)
rs = 5368785696
error = -1776721160
ind = 2424676
nbusy = 4643880

   1 panic(s = 0xfffffc000043cf70 = "kernel memory fault") ["../../../../src\
/kernel/bsd/subr_prf.c"\
:616, 0xfffffc000024ff60]
bootopt = 0

   2 trap() ["../../../../src/kernel/arch/alpha/trap.c":945, 0xfffffc0000381440]
t = 0xffffffff863d36c0
pcb = 0xffffffff96196000
task = 0xffffffff86306b80
p = 0xffffffff95aaf6a0
syst = struct {
    tv_sec = 0
    tv_usec = 0
}
nofault_save = 0
exc_type = 18446739675665756628
exc_code = 0
exc_subcode = 0
i = -2042898428
s = 2682484
ret = 536993792
map = 0xffffffff808fc5a0
prot = 5
cp = 0xffffffff95a607a0 =
i = 0
result = 18446744071932830456
pexcsum = 0xffffffff00000000
i = 16877
pexcsum = 0xffffffff00001000
i = 2682240
ticks = -1784281184
tv = 0xfffffffc00500068

   3 _XentMM() ["../../../../src/kernel/arch/alpha/locore.s":949, 0xfffff\
c0000372dec]

_dump_end:

warning: Files compiled -g3: parameter values probably wrong
_kernel_thread_list_begin: (8)
thread 0x8632faf0 stopped at [thread_block:1427 ,0xfffffc00002ca3a0]  Source\
 not available
thread 0x8632f8d8 stopped at [thread_block:1427 ,0xfffffc00002ca3a0]  Source\
 not available

.
.
.

thread 0x8632d328 stopped at [thread_block:1400 +0x1c,0xfffffc00002ca2f8]  \
Source not available
thread 0x8632d110 stopped at [thread_block:1400 +0x1c,0xfffffc00002ca2f8]  \
Source not available
_kernel_thread_list_end:
_savedefp:  0xffffffff96199940 (9)
_kernel_memory_fault_data_begin:  (10)
struct {
    fault_va = 0x0
    fault_pc = 0x0
    fault_ra = 0xfffffc000028951c
    fault_sp = 0xffffffff96199a48
    access = 0xffffffffffffffff
    status = 0x0
    cpunum = 0x0
    count = 0x1
    pcb = 0xffffffff96196000
    thread = 0xffffffff863d36c0
    task = 0xffffffff86306b80
    proc = 0xffffffff95aaf6a0
}
_kernel_memory_fault_data_end:
Invalid character in input
_uptime: .88 hours

_stack_trace_begin: (11)
>  0 boot(reason = 0, howto = 0) ["../../../../src/kernel/arch/alpha/machdep.c"\
:1118, 0xfffffc0000374a08]
   1 panic(s = 0xfffffc000043cf70 = "kernel memory fault") ["../../../. ./src\
/kernel/bsd/subr_prf.c":616, 0xfffffc000024ff60]
   2 trap() ["../../../../src/kernel/arch/alpha/trap.c":945, 0xfffffc0000381\
440]
   3 _XentMM() ["../../../../src/kernel/arch/alpha/locore.s":949, 0xfffffc000\
0372dec]
_stack_trace_end:
_savedefp_exception_frame_(savedefp/33X):  (12)
ffffffff96199940:  0000000000000000 fffffc000046f888
ffffffff96199950:  ffffffff863d36c0 0000000079c2c93f
ffffffff96199960:  000000000000007d 0000000000000001
ffffffff96199970:  0000000000000000 fffffc000046f4e0
ffffffff96199980:  0000000000000000 ffffffff961962f8
ffffffff96199990:  0000000140012b20 0000000000000000
ffffffff961999a0:  0000000140045690 0000000000000000
ffffffff961999b0:  00000001400075e8 0000000140026240
ffffffff961999c0:  ffffffff96199af0 ffffffff8635adc0
ffffffff961999d0:  ffffffff96199ac0 00000000000001b0
ffffffff961999e0:  fffffc00004941b8 0000000000000000
ffffffff961999f0:  0000000000000001 fffffc000028951c
ffffffff96199a00:  0000000000000000 0000000000000fff
ffffffff96199a10:  0000000140026240 0000000000000000
ffffffff96199a20:  0000000000000000 fffffc000047acd0

.
.
.

ffffffff96199a30:  0000000000901402 0000000000001001
ffffffff96199a40:  0000000000002000
_savedefp_exception_frame_ptr:  0xffffffff96199940
_savedefp_stack_pointer:  0x140026240
_savedefp_processor_status:  0x0
_savedefp_return_address:  0xfffffc000028951c
_savedefp_pc:  0x0
_savedefp_pc/i:

can't read from process (address 0x0)
_savedefp_return_address/i:
  [spec_open:997, 0xfffffc000028951c] 	bis	r0, r0, r19
_kernel_memory_fault_data.fault_pc/i:

can't read from process (address 0x0)
_kernel_memory_fault_data.fault_ra/i:
  [spec_open:997, 0xfffffc000028951c] 	bis	r0, r0, r19

_kdbx_sum_start:
Hostname : madmax.zk3.dec.com
cpu: DEC3000 - M500     avail: 1
Boot-time:      Thu Sep  2 14:05:48 1993
Time:   Thu Sep  2 14:58:52 1993
Kernel : OSF1 release T2.0 version 114.2 (alpha)
_kdbx_sum_end:
_kdbx_swap_start:  (13)
       Swap device name           Size       In Use       Free
-----------------------------  ----------  ----------  ----------
/dev/rz0b                       131072k      10560k     120512k  Dumpdev
                                 16384p       1320p      15064p
-----------------------------  ----------  ----------  ----------
Total swap partitions:  1       131072k      10560k     120512k
                                 16384p       1320p      15064p
_kdbx_swap_end:
_kdbx_proc_start:  (14)
Addr         PID  PPID  PGRP  UID  NICE  SIGCATCH P_SIG    Event  Flags
===========  ===  ====  ====  ===  ====  ======== ======== =====  =====
v0x95aaf6a0  489  340   489    0     0   00000000 00000000 NULL   in pagv ctty
v0x95aad5d0  342  337   342    0     0   00000000 00000000 NULL   in pagv ctty
v0x95aad8f0  341  337   341    0     0   00000000 00000000 NULL   in pagv

.
.
.

v0x95aad2b0    1    0     1    0     0   00000000 00000000 NULL   in omask pagv
v0x95aad120    0    0     0    0     0   00000000 00000000 NULL   in sys
kdbx_proc_end:

Audit subsystem not installed
#
_crash_data_collection_finished:

  1. The first several lines of output display the contents of system variables that give statistics about the crash, such as:

  2. The _panic_string label marks the message that indicates why the crash occurred. In this case the message is kernel memory fault, indicating that a memory operation failed in the kernel.

  3. The preserved message buffer contains status and other information about the devices connected to the system: Notice the following message: 
    trap:  invalid memory ifetch access from kernel mode
    This message describes the kernel memory fault and indicates that the kernel was unable to fetch a needed instruction.

    The preserved message buffer also contains the faulting virtual address, the pc of the instruction that failed, the contents of the return address register, and the stack pointer at the time of the memory fault.

  4. The kernel process status list shows the processes that were active at the time of the crash.

  5. The _current_pid label marks the process ID of the process that was executing at the time of the crash. In this case, it is the tar process, which is identified as process 489 in the kernel process status list.

  6. The _current_tid label marks the address of the thread that was executing at the time of the crash.

  7. The dump section shows information about the variables passed to the routines executing at the time of the crash. In this case, the dump displays variable information for the boot, panic, and trap functions.

  8. The kernel thread list shows the threads of execution in the kernel. This information can be helpful for verifying which routine called the panic function.

  9. The savedefp variable contains a pointer to the exception frame.

  10. The kernel memory fault data displays the following information, recorded at the time of the memory fault:

  11. The _stack_trace_begin line begins a trace of the current thread block's stack at the time of the crash. In this case the _XentMM function called the trap function. The trap function called the panic function, which called the boot function and the system crashed.

  12. The exception frame is a stack frame created to store the state of the process running at the time of the exception. It stores the registers and pc associated with the process. To determine where registers are stored in the exception frame, refer to the /usr/include/machine/reg.h header file.

  13. Swap information is shown to help you determine whether swap space is sufficient.

  14. The process table gives information about the processes active at the time of the crash. The information includes:

    • The process ID of each process.

    • The process ID of the parent process for each process.

    • The process group ID for each process.

    • The UID of the of the user that started each process. In this case all process are started by root.

    • The priority at which the process was running at the time of the memory fault.

    • The event the process was waiting for, if any. An event might be the completion of an input or output request, for example.

    • Any flags assigned to the process. For example, the ctty flag indicates that the process has a controlling terminal and, the sys flag indicates that the process is a swapper or pager process.

[TOC] --SECT