The ELF format - how programs look from the inside
Introduction
ELF is the file format used for object files (.o's), binaries, shared
libraries and core dumps in Linux.
It's actually pretty simple and well thought-out.
ELF has the same layout for all architectures, however endianness and word size can differ; relocation types, symbol types and the like may have platform-specific values, and of course the contained code is arch specific.
An ELF file provides 2 views on the data it contains: A linking view and an execution view. Those two views can be accessed by two headers: the section header table and the program header table.
Linking view: Section Header Table (SHT)
The SHT gives an overview on the sections contained in the ELF file. Of
particular interest are REL sections (relocations), SYMTAB/DYNSYM (symbol
tables), VERSYM/VERDEF/VERNEED sections (symbol versioning information).
greek0@iphigenie:~$ readelf -S /bin/bash
There are 26 section headers, starting at offset 0xa4e10:
Section Headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 00000 000000 00 0 0 0
[ 1] .interp PROGBITS 08048134 00134 000013 00 A 0 0 1
[ 2] .note.ABI-tag NOTE 08048148 00148 000020 00 A 0 0 4
[ 3] .hash HASH 08048168 00168 002e48 04 A 4 0 4
[ 4] .dynsym DYNSYM 0804afb0 02fb0 007890 10 A 5 1 4
[ 5] .dynstr STRTAB 08052840 0a840 0074e2 00 A 0 0 1
[ 6] .gnu.version VERSYM 08059d22 11d22 000f12 02 A 4 0 2
[ 7] .gnu.version_r VERNEED 0805ac34 12c34 000090 00 A 5 2 4
[ 8] .rel.dyn REL 0805acc4 12cc4 000040 08 A 4 0 4
[ 9] .rel.plt REL 0805ad04 12d04 0005a8 08 A 4 11 4
[10] .init PROGBITS 0805b2ac 132ac 000017 00 AX 0 0 4
[11] .plt PROGBITS 0805b2c4 132c4 000b60 04 AX 0 0 4
[12] .text PROGBITS 0805be30 13e30 077154 00 AX 0 0 16
[13] .fini PROGBITS 080d2f84 8af84 00001a 00 AX 0 0 4
[14] .rodata PROGBITS 080d2fa0 8afa0 015198 00 A 0 0 32
[15] .eh_frame_hdr PROGBITS 080e8138 a0138 00002c 00 A 0 0 4
[16] .eh_frame PROGBITS 080e8164 a0164 00009c 00 A 0 0 4
[17] .ctors PROGBITS 080e9200 a0200 000008 00 WA 0 0 4
[18] .dtors PROGBITS 080e9208 a0208 000008 00 WA 0 0 4
[19] .jcr PROGBITS 080e9210 a0210 000004 00 WA 0 0 4
[20] .dynamic DYNAMIC 080e9214 a0214 0000d8 08 WA 5 0 4
[21] .got PROGBITS 080e92ec a02ec 000004 04 WA 0 0 4
[22] .got.plt PROGBITS 080e92f0 a02f0 0002e0 04 WA 0 0 4
[23] .data PROGBITS 080e95e0 a05e0 004764 00 WA 0 0 32
[24] .bss NOBITS 080edd60 a4d44 004bc8 00 WA 0 0 32
[25] .shstrtab STRTAB 00000000 a4d44 0000cc 00 0 0 1
Execution view: Program Header Table (PHT)
The PHT contains information for the kernel on how to start the program. The
LOAD directives determinate what parts of the ELF file get mapped into memory.
The INTERP directive specifies an ELF interpreter, which is normally
/lib/ld-linux.so.2 on Linux systems.
The DYNAMIC entry points to the .dynamic section which contains information
used by the ELF interpreter to setup the binary.
greek0@iphigenie:~$ readelf -l /bin/bash
Elf file type is EXEC (Executable file)
Entry point 0x805be30
There are 8 program headers, starting at offset 52
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
PHDR 0x000034 0x08048034 0x08048034 0x00100 0x00100 R E 0x4
INTERP 0x000134 0x08048134 0x08048134 0x00013 0x00013 R 0x1
[Requesting program interpreter: /lib/ld-linux.so.2]
LOAD 0x000000 0x08048000 0x08048000 0xa0200 0xa0200 R E 0x1000
LOAD 0x0a0200 0x080e9200 0x080e9200 0x04b44 0x09728 RW 0x1000
DYNAMIC 0x0a0214 0x080e9214 0x080e9214 0x000d8 0x000d8 RW 0x4
NOTE 0x000148 0x08048148 0x08048148 0x00020 0x00020 R 0x4
GNU_EH_FRAME 0x0a0138 0x080e8138 0x080e8138 0x0002c 0x0002c R 0x4
GNU_STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RW 0x4
Section to Segment mapping:
Segment Sections...
00
01 .interp
02 .interp .note.ABI-tag .dynsym .dynstr .gnu.version .gnu.version_r .rel.dyn .rel.plt ...
03 .ctors .dtors .jcr .dynamic .got .got.plt .data .bss
04 .dynamic
05 .note.ABI-tag
06 .eh_frame_hdr
07
Putting it all together: the ELF header
Neither the STH nor the PTH have fixed positions, they can be located anywhere in an ELF file. To find them the ELF header is used, which is located at the very start of the file.
The first bytes contain the elf magic "\x7fELF", followed by the class ID (32
or 64 bit ELF file), the data format ID (little endian/big endian), the machine
type, etc.
At the end of the ELF header are then pointers to the SHT and PHT.
greek0@iphigenie:~$ readelf -h /bin/bash
ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: Intel 80386
Version: 0x1
Entry point address: 0x805be30
Start of program headers: 52 (bytes into file)
Start of section headers: 675344 (bytes into file)
Flags: 0x0
Size of this header: 52
Size of program headers: 32
Number of program headers: 8
Size of section headers: 40
Number of section headers: 26
Section header string table index: 25
The Relocation Table
The relocation table specifies where relocations are needed in order for the program to run. In programs these are normally symbol relocations, i.e. the dynamic linker has to resolve the needed symbol by its name, and then write the symbol address to the place specified in the relocation entry.
Relocation types are architecture specific and there are usually quite a lot of
them. On i386 the most important ones are the R_386_COPY type, meaning "just
copy the address of the symbol to that address", and R_386_JUMP_SLOT, which
is used for the normal PLT/GOT function call relocation mechanism.
The resolution of the symbol value itself is done by the dynamic linker
(contained within /lib/ld-linux.so.2, the ELF interpreter commonly used), and
is a pretty complex process. Basically the linker searches all loaded ELF objects
(the binary itself and the loaded libraries) and uses the first definition of
the symbol it finds.
greek0@iphigenie:~$ readelf -r /bin/bash
Relocation section '.rel.dyn' at offset 0x12cc4 contains 8 entries:
Offset Info Type Sym.Value Sym. Name
080e92ec 00078006 R_386_GLOB_DAT 00000000 __gmon_start__
080edd68 00035205 R_386_COPY 080edd68 stdout
080edd6c 00035d05 R_386_COPY 080edd6c stderr
080edd70 00046405 R_386_COPY 080edd70 PC
080edd74 00067405 R_386_COPY 080edd74 stdin
080edd78 0006e305 R_386_COPY 080edd78 UP
Relocation section '.rel.plt' at offset 0x12d04 contains 181 entries:
Offset Info Type Sym.Value Sym. Name
080e9368 00012c07 R_386_JUMP_SLOT 00000000 fileno
080e936c 00013807 R_386_JUMP_SLOT 00000000 strcmp
080e9370 00014107 R_386_JUMP_SLOT 0805b4a4 close
080e9374 00015307 R_386_JUMP_SLOT 00000000 dlsym
080e937c 00016a07 R_386_JUMP_SLOT 00000000 fprintf
080e9388 00018307 R_386_JUMP_SLOT 00000000 fflush
080e9390 00019c07 R_386_JUMP_SLOT 0805b524 unlink
080e930c 00003307 R_386_JUMP_SLOT 00000000 regexec
080e9328 00007a07 R_386_JUMP_SLOT 00000000 ferror
080e9330 00008307 R_386_JUMP_SLOT 00000000 readdir64
080e9334 00008f07 R_386_JUMP_SLOT 00000000 strchr
080e9338 0000a507 R_386_JUMP_SLOT 00000000 fdopen
080e9344 0000da07 R_386_JUMP_SLOT 00000000 getpid
080e9360 00012207 R_386_JUMP_SLOT 00000000 write
080e95cc 00078707 R_386_JUMP_SLOT 00000000 strcpy
...
...
Exported symbols
When searching for a symbol the dynamic linker looks through the dynamic symbol
table .dynsym, so all symbols present there are usable by other programs (in other
words: exported and in case of a library, part of the ABI).
Actually the process is more complicated (involving the hashes in the .hash
section), but the end result is the same as just described.
greek0@iphigenie:~$ readelf -D -s /lib/libc.so.6
Symbol table for image:
Num Buc: Value Size Type Bind Vis Ndx Name
260 0: 0011a580 4 OBJECT GLOBAL DEFAULT 29 _nl_domain_bindings
1693 1: 000b0f60 1303 FUNC GLOBAL DEFAULT 11 fts_read
601 2: 00027df0 13 FUNC WEAK DEFAULT 11 scalbln
208 3: 000698f0 141 FUNC GLOBAL DEFAULT 11 memmove
1798 4: 000b8ae0 117 FUNC GLOBAL DEFAULT 11 lsearch
348 4: 000dfd20 189 FUNC GLOBAL DEFAULT 11 xdr_u_hyper
1675 9: 0005ad10 231 FUNC GLOBAL DEFAULT 11 fputc
381 9: 000b92f0 389 FUNC WEAK DEFAULT 11 error_at_line
166 9: 000864d0 36 FUNC GLOBAL DEFAULT 11 versionsort64
119 9: 000f2950 36 FUNC GLOBAL DEFAULT 11 versionsort64
2113 16: 000ac770 58 FUNC WEAK DEFAULT 11 mkdir
516 16: 000de9c0 677 FUNC GLOBAL DEFAULT 11 svctcp_create
979 17: 000b7040 60 FUNC GLOBAL DEFAULT 11 madvise
1815 18: 000c61f0 42 FUNC GLOBAL DEFAULT 11 pthread_mutex_lock
2018 25: 00054ac0 326 FUNC WEAK DEFAULT 11 fputs
432 30: 000ebc40 33 FUNC GLOBAL DEFAULT 11 getutxid
1879 31: 000292b0 64 FUNC GLOBAL DEFAULT 11 sigdelset
1902 33: 000ba530 107 FUNC GLOBAL DEFAULT 11 gnu_dev_makedev
1385 34: 000f3bc0 153 FUNC GLOBAL DEFAULT 11 getrlimit64
895 34: 000b2ad0 153 FUNC GLOBAL DEFAULT 11 getrlimit64
1290 37: 0009f400 319 FUNC WEAK DEFAULT 11 re_comp
82 40: 000dbd70 1653 FUNC GLOBAL DEFAULT 11 clnt_broadcast
1892 41: 0008a6c0 53 FUNC WEAK DEFAULT 11 getresgid
...
...
A more detailed look at versionsort64
The observant reader may have noticed that e.g. versionsort64 is present
twice in the dynamic symbol table shown above, and the two symbols have
different values.
The reason is pretty simple; libc.so.6 uses symbol versioning, and there are
two versions of versionsort64 available. The binutils readelf unfortunately
doesn't show the symbol versions, eu-readelf from the elfutils package
however does.
greek0@iphigenie:~$ readelf -D -s /lib/libc.so.6 | grep versionsort64
166 9: 000864d0 36 FUNC GLOBAL DEFAULT 11 versionsort64
119 9: 000f2950 36 FUNC GLOBAL DEFAULT 11 versionsort64
greek0@iphigenie:~$ eu-readelf -s /lib/libc.so.6 | grep versionsort64
119: 000f2950 36 FUNC GLOBAL DEFAULT 11 versionsort64@GLIBC_2.1
166: 000864d0 36 FUNC GLOBAL DEFAULT 11 versionsort64@@GLIBC_2.2
Program loading in the kernel
ELF files themselves arent't terribly interesting. How ELF files are loaded into memory, and what has to happen before the program can execute its own code, can be.
The execution of a program starts inside the kernel, in the exec system call. There the file type is looked up and the appropriate handler is called. The binfmt-elf handler then loads the ELF header and the program header table (PHT), followed by lots of sanity checks.
The kernel then loads the parts specified in the LOAD directives in the PHT
into memory. If an INTERP entry is present, the interpreter is loaded too.
Statically linked binaries can do without an interpreter; dynamically linked
programs always need /lib/ld-linux.so as interpreter because it includes some
startup code, loads shared libraries needed by the binary, and performs
relocations.
Finally control can be transfered to the program, to the interpreter, if present, otherwise to the binary itself.
greek0@iphigenie:~$ readelf -l /bin/bash
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
PHDR 0x000034 0x08048034 0x08048034 0x00100 0x00100 R E 0x4
INTERP 0x000134 0x08048134 0x08048134 0x00013 0x00013 R 0x1
[Requesting program interpreter: /lib/ld-linux.so.2]
LOAD 0x000000 0x08048000 0x08048000 0xa0200 0xa0200 R E 0x1000
LOAD 0x0a0200 0x080e9200 0x080e9200 0x04b44 0x09728 RW 0x1000
DYNAMIC 0x0a0214 0x080e9214 0x080e9214 0x000d8 0x000d8 RW 0x4
GNU_STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RW 0x4
...
Dynamic linking and the ELF interpreter
In case of a statically linked binary that's pretty much it, however with dynamically linked binaries a lot more magic has to go on.
First the dynamic linker (contained within the interpreter) looks at the
.dynamic section, whose address is stored in the PHT.
There it finds the NEEDED entries determining which libraries have to be
loaded before the program can be run, the *REL* entries giving the address of
the relocation tables, the VER* entries which contain symbol versioning
information, etc.
So the dynamic linker loads the needed libraries and performs relocations (either directly at program startup or later, as soon as the relocated symbol is needed, depending on the relocation type).
Finally control is transferred to the address given by the symbol _start in
the binary. Normally some gcc/glibc startup code lives there, which in the end
calls main().
greek0@iphigenie:~$ readelf -d /bin/bash
Dynamic section at offset 0xa0214 contains 22 entries:
Tag Type Name/Value
0x00000001 (NEEDED) Shared library: [libncurses.so.5]
0x00000001 (NEEDED) Shared library: [libdl.so.2]
0x00000001 (NEEDED) Shared library: [libc.so.6]
0x0000000a (STRSZ) 29922 (bytes)
0x0000000b (SYMENT) 16 (bytes)
0x00000003 (PLTGOT) 0x80e92f0
0x00000002 (PLTRELSZ) 1448 (bytes)
0x00000014 (PLTREL) REL
0x00000017 (JMPREL) 0x805ad04
0x00000011 (REL) 0x805acc4
0x00000012 (RELSZ) 64 (bytes)
0x6ffffffe (VERNEED) 0x805ac34
0x6fffffff (VERNEEDNUM) 2
0x6ffffff0 (VERSYM) 0x8059d22
0x00000000 (NULL) 0x0
Symbol lookup by the dynamic linker
As mentioned before, symbol lookup is a complicated process, I'll give a simplified description.
For every loaded object RTLD (the runtime dynamic linker) keeps a list of loaded objects called the "lookup scope". Every scope contains pointers to all the loaded objects (the binary and all loaded libraries), but the order of objects can differ between different scopes. What is constant is that the binary is the first object in every scope.
When the RTLD has to resolve a symbol, it first checks for which object it needs to perform the relocation. Was the lookup caused in the binary itself or in one of the loaded libraries. Then it gets the lookup scope for that object, and iterates through every object in it.
For each object it looks for the needed symbol is in the dynamic symbol table. In case of a match it just uses that symbol value for the relocation, otherwise it continues its search looking at the next object in the scope.
Consequences of the symbol lookup rules
Libs can't just directly jump to functions they export (they of course know where their own functions are), but have to go through the described symbol lookup mechanism too.
This along with the fact that the binary is always first in every lookup scope means that symbols defined in the binary override symbols defined in libraries.
It's this way on purpose, to allow the binary to override library functions it doesn't like. If this happens nothing will use the library's function, not even calls by the library itself.
Normally that's a good thing, but it can lead to problems if the binary unintentionally defines a symbol that's also used by some loaded library (think program uses GTK, which pulls in different theme and input plugins depending on the user's system config).
E.g. if the program defines a function void print_error(int error_code, char*
str); and some plugin defines a function with the same name, but another
signature, like int print_error(char* str), that may be problematic. If the
plugin doesn't export the print_error symbol, there's no problem at all,
because the code in the plugin can just call the proper function directly,
without the need for a symbol lookup.
However if the plugin does export the symbol it has to lookup the symbol itself
(because that's required by the SystemV ABI spec and, consequently, by the
LSB). Then print_error will be interposed by the symbol in the binary, which
has an incompatible signature, which will probably lead to a crash.
Solutions for this
The right solution is to just not export symbols that you don't explicitly want others to use. That's The Right Way™ for many reasons, it speeds up the library/plugin (no symbol lookup has to be performed for internal uses of that function), you don't pollute the namespace that way, you avoid the possible problems outlined above, and finally, the symbol is not part of your ABI, which means you can change it every way you like without breaking any dependent applications.
There's a quick hack that can be used to avoid the above problem. When
-Bsymbolic is specified on the linker commandline when linking a library, the
lookup scope of that library is changed so the library itself is in the first
spot, followed by the binary.
This doesn't give you any of the other advantages though, and the program can't
interpose your symbol, even if it intentionally wants to do so. Consequently
-Bsymbolic should be avoided, unless you're really, really lazy ;-).
Further reading
- The ELF Specification
- Definitive resource on libraries: Ulrich Drepper's DSO Howto
- Josselin Mouette's "Packaging shared libraries" talk at Debconf6
- The Debian Library Packaging Guide by Junichi Uekawa