Working with core dump

Thierry GAYET )

GOAL

The main goal of this document is to provide some information about CORE dump.

Revolution framework have been adapt in order to be compliant with core dump.

Indeed, both the GNU/linux kernel and the busybox have been configure for that need.

Introduction

Enabling / disabling the feature :

Enabling: ulimit -c unlimited

Disabling: ulimit -S -c 0

Optionally, we can enable core dump for SUID programs:

echo 1 > /proc/sys/kernel/suid_dumpable

(not applicable on the dxi807 boards yet)

We can also ask to generate core with a PID suffix (core.PID) :

echo “1" > /proc/sys/kernel/core_uses_pid

This is possible to locate all core file in a global temp directory:

       $ mkdir /tmp/corefiles

       $ chmod 777 /tmp/corefiles

       $ echo "/tmp/corefiles/core" > /proc/sys/kernel/core_pattern

All corefiles then get tossed to /tmp/corefiles (don't change core_uses_pid if you do this).

Usage

Syntax : ulimit [-acdfHlmnpsStuv] [limit]

Options

-S Change and report the soft limit associated with a resource.

-H Change and report the hard limit associated with a resource.

-a All current limits are reported.

-c The maximum size of core files created.

-d The maximum size of a process's data segment.

-f The maximum size of files created by the shell(default option)

-l The maximum size that may be locked into memory.

-m The maximum resident set size.

-n The maximum number of open file descriptors.

-p The pipe buffer size.

-s The maximum stack size.

-t The maximum amount of cpu time in seconds.

-u The maximum number of processes available to a single user.

-v The maximum amount of virtual memory available to the process.

Ulimit detail

When a process is dumped, all anonymous memory is written to a core file as long as the size of the core file isn't limited. But sometimes we don't want to dump some memory segments, for example, huge shared memory. Conversely, sometimes we want to save file-backed memory segments into a core file, not only the individual files.

/proc/<pid>/coredump_filter allows you to customize which memory segments will be dumped when the <pid> process is dumped. coredump_filter is a bitmask of memory types. If a bit of the bitmask is set, memory segments of the corresponding memory type are dumped, otherwise they are not dumped.

The following 7 memory types are supported:

bit 0) anonymous private memory

bit 1) anonymous shared memory

bit 2) file-backed private memory

bit 3) file-backed shared memory

bit 4) ELF header pages in file-backed private memory areas (it is effective only if the bit 2 is cleared)

bit 5) hugetlb private memory

bit 6) hugetlb shared memory

Note that MMIO pages such as frame buffer are never dumped and vDSO pages are always dumped regardless of the bitmask status.

Note bit 0-4 doesn't effect any hugetlb memory. hugetlb memory are only effected by bit 5-6.

Default value of coredump_filter is 0x23; this means all anonymous memory segments and hugetlb private memory are dumped.

If you don't want to dump all shared memory segments attached to pid 1234, write 0x21 to the process's proc file.

$ echo 0x21 > /proc/1234/coredump_filter

When a new process is created, the process inherits the bitmask status from its parent. It is useful to set up coredump_filter before the program runs. For example:

$ echo 0x7 > /proc/self/coredump_filter

$ ./some_program

Dumping more information

Another example :

# cat /proc/<pid>/coredump_filter 00000003

The following 4 memory types are supported:

(bit 0) anonymous private memory

(bit 1) anonymous shared memory

(bit 2) file-backed private memory

(bit 3) file-backed shared memory

# echo 0xF > /proc/<pid>/coredump_filter

Dumping more information

The default action of certain signals is to cause a process to terminate and produce acore dump file, a disk file containing an image of the process's memory at the time of termination. A list of the signals which cause a process to dump core can be found insignal(7).A process can set its soft RLIMIT_CORE resource limit to place an upper limit on the size of the core dump file that will be produced if it receives a "core dump" signal; seegetrlimit(2) for details.

There are various circumstances in which a core dump file is not produced:

1. The process does not have permission to write the core file. (By default the core file is called core, and is created in the current working directory. See below for details on naming.) Writing the core file will fail if the directory in which it is to be created is non-writable, or if a file with the same name exists and is not writable or is not a regular file (e.g., it is a directory or a symbolic link).

2. The directory in which the core dump file is to be created does not exist.

3. RLIMIT_CORE or RLIMIT_FSIZE resource limits for a process are set to zero (see getrlimit(2)).

4. The binary being executed by the process does not have read permission enabled.

5. The process is executing a set-user-ID (set-group-ID) program that is owned by a user (group) other than the real user (group) ID of the process. (However, see the description of the prctl(2) PR_SET_DUMPABLE operation, and the description of the /proc/sys/fs/suid_dumpable file in proc(5).)

Core detail

Naming of core dump files :

By default, a core dump file is named core, but the /proc/sys/kernel/core_pattern file (new in Linux 2.5) can be set to define a template that is used to name core dump files.

The template can contain % specifiers which are substituted by the following values when a core file is created:

%% A single % character %p PID of dumped process %u real UID of dumped process %g real GID of dumped process %s number of signal causing dump %t time of dump (seconds since 0:00h, 1 Jan 1970) %h hostname (same as 'nodename' returned by uname(2)) %e executable filename

A single % at the end of the template is dropped from the core filename, as is the combination of a % followed by any character other than those listed above. All other characters in the template become a literal part of the core filename. The template may include '/' characters, which are interpreted as delimiters for directory names. The maximum size of the resulting core filename is 64 bytes. The default value in this file is "core". For backward compatibility, if /proc/sys/kernel/core_pattern does not include "%p" and /proc/sys/kernel/core_uses_pid (see below) is non-zero, then .PID will be appended to the core filename.

Linux 2.4 does not provide /proc/sys/kernel/core_pattern, but does provide a more primitive method of controlling the name of the core dump file. If the/proc/sys/kernel/core_uses_pid file contains the value 0, then a core dump file is simply named core. If this file contains a non-zero value, then the core dump file includes the process ID in a name of the form core.PID.

Notes :

the gdb gcore command can be used to obtain a core dump of a running process.

If a multithreaded process (or, more precisely, a process that shares its memory with another process by being created with the CLONE_VM flag of clone(2)) dumps core, then the process ID is always appended to the core filename, unless the process ID was already included elsewhere in the filename via a %p specification in/proc/sys/kernel/core_pattern.

Core detail

Once the core is generated this is possible to debug with :

$ gdb mybinary core

The backtrace can be displayed using the bt command:

GNU gdb 5.0rh-5 Red Hat Linux 7.1Copyright 2001 Free Software Foundation, Inc.GDB is free software, covered by the GNU General Public License, and you arewelcome to change it and/or distribute copies of it under certain conditions.Type "show copying" to see the conditions.There is absolutely no warranty for GDB. Type "show warranty" for details.This GDB was configured as "i386-redhat-linux"...Core was generated by `./mybinary.Program terminated with signal 11, Segmentation fault.Cannot access memory at address 0x40016bec#0 0x400a68d6 in ?? ()(gdb) bt#0 0x400a68d6 in ?? ()Cannot access memory at address 0xbffff8a8(gdb) where#0 0x400a68d6 in ?? ()Cannot access memory at address 0xbffff8a8(gdb) quit

Debugging

This is possible to simulate a core dump using the following command:

# kill -s SIGSEGV PID_OF_MY_BINARY

or

# kill -11 PID_OF_MY_BINARY

checks

The following method have been used for validating the core dump feature :

# vi chk_core.c

int main(void)

{

char *s = "hello world";

*s = 'H';

} /* Main */

# gcc chk_core.c -o chk_core

# ./chk_core

show_signal_msg: 20 callbacks suppressed

chk_core[2637]: segfault at 8048490 ip 080483c4 sp bfcb8118 error 7 in sf[8048000+1000]

Segmentation fault (core dumped)

# ls /tmp/corefiles

Core

# file core

core: ELF 32-bit LSB core file Intel 80386, version 1 (SYSV), SVR4-style

Checks

Globally, the ulimit command say:

 

# ulimit -a

-f: file size (blocks) unlimited

-t: cpu time (seconds) unlimited

-d: data seg size (kb) unlimited

-s: stack size (kb) 8192

-c: core file size (blocks) unlimited

-m: resident set size (kb) unlimited

-l: locked memory (kb) 64

-p: processes 12993

-n: file descriptors 1024

-v: address space (kb) unlimited

-w: locks unlimited

Checks

The GNU/linux kernel generates oop that are similare to core dump. They can be anaylyse thanks to the ksymsoops tool.

This is also possible to use /proc/kcore. Kcore is a memory image

of the kernel and cannot be used through the cat command.

GNU/Linux Kernel

More help http://en.wikipedia.org/wiki/Core_dump

http://linux.die.net/man/5/core

http://www.mjmwired.net/kernel/Documentation/filesystems/proc.txt