Ytdyklly

While browsing through the Kernel Makefiles, I found these terms. So I would like to know what is the difference between vmlinux, vmlinuz, vmlinux.bin, zimage & bzimage?

vmlinux

This is the Linux kernel in an statically linked executable file format. Generally, you don't have to worry about this file, it's just a intermediate step in the boot procedure.

The raw vmlinux file may be useful for debugging purposes.

vmlinux.bin

The same as vmlinux, but in a bootable raw binary file format. All symbols and relocation information is discarded. Generated from vmlinux by objcopy -O binary vmlinux vmlinux.bin.

vmlinuz

The vmlinux file usually gets compressed with zlib. Since 2.6.30 LZMA and bzip2 are also available. By adding further boot and decompression capabilities to vmlinuz, the image can be used to boot a system with the vmlinux kernel. The compression of vmlinux can occur with zImage or bzImage.

The function decompress_kernel() handles the decompression of vmlinuz at bootup, a message indicates this:

Decompressing Linux... done

Booting the kernel.

zImage (make zImage)

This is the old format for small kernels (compressed, below 512KB). At boot, this image gets loaded low in memory (the first 640KB of the RAM).

bzImage (make bzImage)

The big zImage (this has nothing to do with bzip2), was created while the kernel grew and handles bigger images (compressed, over 512KB). The image gets loaded high in memory (above 1MB RAM). As today's kernels are way over 512KB, this is usually the preferred way.

An inspection on Ubuntu 10.10 shows:

ls -lh /boot/vmlinuz-$(uname -r)

-rw-r--r-- 1 root root 4.1M 2010-11-24 12:21 /boot/vmlinuz-2.6.35-23-generic



file /boot/vmlinuz-$(uname -r)

/boot/vmlinuz-2.6.35-23-generic: Linux kernel x86 boot executable bzImage, version 2.6.35-23-generic (buildd@rosea, RO-rootFS, root_dev 0x6801, swap_dev 0x4, Normal VGA

Do a verbose kernel build and search for the files

This approach can give some insight, will never get out of date, and will help you to easily find which part of the build system is doing what.

Once you have a build configuration that generates one of the files, build with:

make V=1 |& tee f.log

Modify a comment on some C file to force a re-link (e.g. init/main.c is a good one) if you have already built previously.

Now, inspect f.log and search for the images of interest.

For example, on v4.19 we will conclude that:

init/main.c

|

| gcc -c

|

v

init/.tmp_main.o

|

| CONFIG_MODVERSIONS stuff

|

v

init/main.o

|

| ar T (thin archive)

|

v

init/built-in.a

|

| ar T (thin archive)

|

v

built-in.a

|

| ld

|

v

vmlinux (regular ELF file)

|

| objcopy

|

v

arch/x86/boot/compressed/vmlinux.bin

|

| GZIP

|

v

arch/x86/boot/compressed/vmlinux.bin.gz

|

| .incbin

|

v

arch/x86/boot/compressed/piggy.S

|

| gcc -c

|

v

arch/x86/boot/compressed/piggy.o

|

| ld

|

v

arch/x86/boot/compressed/vmlinux (regular ELF file with gzipped code)

|

| objcopy

|

v

arch/x86/boot/vmlinux.bin

|

| arch/x86/boot/tools/build.c

|

v

arch/x86/boot/bzImage

Thin archives are mentioned at: https://stackoverflow.com/questions/2157629/linking-static-libraries-to-other-static-libraries/27676016#27676016 They are archives that just point other archives / objects instead of copying them.

The kernel moved from incremental linking to thin archives in v4.9 as described at: https://stackoverflow.com/questions/29391965/what-is-partial-linking-in-gnu-linker/53959624#53959624

Full log interpretation

When we start reading the verbose build logs from the back up, first we see:

ln -fsn ../../x86/boot/bzImage ./arch/x86_64/boot/bzImage

so those two are just symlinked.

Then we search a bit further for x86/boot/bzImage and find:

arch/x86/boot/tools/build 

arch/x86/boot/setup.bin 

arch/x86/boot/vmlinux.bin 

arch/x86/boot/zoffset.h 

arch/x86/boot/bzImage

arch/x86/boot/tools/build is an executable, so we run it, see the help message:

Usage: build setup system zoffset.h image

and grep to find the source:

arch/x86/boot/tools/build.c

So this tool must be generating arch/x86/boot/bzImage from arch/x86/boot/vmlinux.bin and other files TODO what is the point of build exactly?

If we follow arch/x86/boot/vmlinux.bin we see that it is just an objcopy from arch/x86/boot/compressed/vmlinux:

objcopy 

-O binary 

-R .note 

-R .comment 

-S arch/x86/boot/compressed/vmlinux 

arch/x86/boot/vmlinux.bin

and arch/x86/boot/compressed/vmlinux is just a regular ELF file:

ld 

-m elf_x86_64 

-z noreloc-overflow 

-pie 

--no-dynamic-linker 

-T arch/x86/boot/compressed/vmlinux.lds 

arch/x86/boot/compressed/head_64.o 

arch/x86/boot/compressed/misc.o 

arch/x86/boot/compressed/string.o 

arch/x86/boot/compressed/cmdline.o 

arch/x86/boot/compressed/error.o 

arch/x86/boot/compressed/piggy.o 

arch/x86/boot/compressed/cpuflags.o 

arch/x86/boot/compressed/early_serial_console.o 

arch/x86/boot/compressed/kaslr.o 

arch/x86/boot/compressed/kaslr_64.o 

arch/x86/boot/compressed/mem_encrypt.o 

arch/x86/boot/compressed/pgtable_64.o 

-o arch/x86/boot/compressed/vmlinux

ls -hlSr says that piggy.o is by far the largest file, so we search for it, and it must come from:

gcc 

-Wp,-MD,arch/x86/boot/compressed/.piggy.o.d 

-nostdinc 

-Ilinux/arch/x86/include 

-I./arch/x86/include/generated 

-Ilinux/include 

-I./include 

-Ilinux/arch/x86/include/uapi 

-I./arch/x86/include/generated/uapi 

-Ilinux/include/uapi 

-I./include/generated/uapi 

-include linux/include/linux/kconfig.h 

-D__KERNEL__ 

-m64 

-O2 

-fno-strict-aliasing 

-fPIE 

-DDISABLE_BRANCH_PROFILING 

-mcmodel=small 

-mno-mmx 

-mno-sse 

-ffreestanding 

-fno-stack-protector 

-Wno-pointer-sign 

-D__ASSEMBLY__ 

-c 

-o arch/x86/boot/compressed/.tmp_piggy.o 

arch/x86/boot/compressed/piggy.S

.tmp_ prefix explained below.

arch/x86/boot/compressed/piggy.S contains:

.incbin "arch/x86/boot/compressed/vmlinux.bin.gz"

see also: https://stackoverflow.com/questions/4158900/embedding-resources-in-executable-using-gcc/36295692#36295692

arch/x86/boot/compressed/vmlinux.bin.gz comes from:

cat arch/x86/boot/compressed/vmlinux.bin arch/x86/boot/compressed/vmlinux.relocs | 

gzip -n -f -9 > arch/x86/boot/compressed/vmlinux.bin.gz

which comes from:

objcopy  -R .comment -S vmlinux arch/x86/boot/compressed/vmlinux.bin

which comes from:

LD      vmlinux

which does:

ld 

-m elf_x86_64 

-z max-page-size=0x200000 

--emit-relocs 

--build-id 

-o vmlinux 

-T ./arch/x86/kernel/vmlinux.lds 

--whole-archive 

built-in.a 

--no-whole-archive 

--start-group 

lib/lib.a 

arch/x86/lib/lib.a 

--end-group 

.tmp_kallsyms2.o

vmlinux is huge, but all shown objects are tiny according to ls -l, so I researched and learned about a new ar feature I didn't know about: thin archives.

At:

AR      built-in.a

the build does:

ar 

rcsTPD 

built-in.a 

arch/x86/kernel/head_64.o 

arch/x86/kernel/head64.o 

arch/x86/kernel/ebda.o 

arch/x86/kernel/platform-quirks.o 

init/built-in.a 

usr/built-in.a 

arch/x86/built-in.a 

kernel/built-in.a 

certs/built-in.a 

mm/built-in.a 

fs/built-in.a 

ipc/built-in.a 

security/built-in.a 

crypto/built-in.a 

block/built-in.a 

lib/built-in.a 

arch/x86/lib/built-in.a 

drivers/built-in.a 

sound/built-in.a 

firmware/built-in.a 

arch/x86/pci/built-in.a 

arch/x86/power/built-in.a 

arch/x86/video/built-in.a 

net/built-in.a 

virt/built-in.a

T specifies the thin archive.

We can then see that all sub archives are also thin, e.g., since I modified init/main.c, we have:

ar 

rcSTPD 

init/built-in.a 

init/main.o 

init/version.o 

init/do_mounts.o 

init/do_mounts_initrd.o 

init/initramfs.o 

init/calibrate.o 

init/init_task.o

which finally comes from the C file through a command like:

gcc 

-Wp,-MD,init/.main.o.d 

-c 

-o 

init/.tmp_main.o 

/work/linux-kernel-module-cheat/submodules/linux/init/main.c

I can't find the init/.tmp_main.o to init/main.o step on the logs which is a shame... with:

git grep '.tmp_'

we see that likely comes from scripts Makefile.build and is linked to CONFIG_MODVERSIONS which I had enabled:

ifndef CONFIG_MODVERSIONS

cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $<



else

# When module versioning is enabled the following steps are executed:

# o compile a .tmp_<file>.o from <file>.c

# o if .tmp_<file>.o doesn't contain a __ksymtab version, i.e. does

#   not export symbols, we just rename .tmp_<file>.o to <file>.o and

#   are done.

# o otherwise, we calculate symbol versions using the good old

#   genksyms on the preprocessed source and postprocess them in a way

#   that they are usable as a linker script

# o generate <file>.o from .tmp_<file>.o using the linker to

#   replace the unresolved symbols __crc_exported_symbol with

#   the actual value of the checksum generated by genksyms



cmd_cc_o_c = $(CC) $(c_flags) -c -o $(@D)/.tmp_$(@F) $<



cmd_modversions_c =                             

    if $(OBJDUMP) -h $(@D)/.tmp_$(@F) | grep -q __ksymtab; then     

        $(call cmd_gensymtypes_c,$(KBUILD_SYMTYPES),$(@:.o=.symtypes))  

            > $(@D)/.tmp_$(@F:.o=.ver);                 

                                        

        $(LD) $(KBUILD_LDFLAGS) -r -o $@ $(@D)/.tmp_$(@F)       

            -T $(@D)/.tmp_$(@F:.o=.ver);                

        rm -f $(@D)/.tmp_$(@F) $(@D)/.tmp_$(@F:.o=.ver);        

    else                                    

        mv -f $(@D)/.tmp_$(@F) $@;                  

    fi;

endif

Analysis done with this config which contains CONFIG_KERNEL_GZIP=y.

aarch64 arch/arm64/boot/Image

Just an uncompressed objcopy from vmlinux:

objcopy  -O binary -R .note -R .note.gnu.build-id -R .comment -S vmlinux arch/arm64/boot/Image

vmlinux is obtained in basically the exact same way as for x86 though the thin archives.

arch/arm/boot/zImage

Very similar to X86 with a zipped vmlinux, but no magic build.c step. Call chain summary:

objcopy -O binary -R .comment -S  arch/arm/boot/compressed/vmlinux arch/arm/boot/zImage



ld 

-EL 

--defsym _kernel_bss_size=469592 

-p 

--no-undefined 

-X 

-T arch/arm/boot/compressed/vmlinux.lds 

arch/arm/boot/compressed/head.o 

arch/arm/boot/compressed/piggy.o 

arch/arm/boot/compressed/misc.o 

arch/arm/boot/compressed/decompress.o 

arch/arm/boot/compressed/string.o 

arch/arm/boot/compressed/hyp-stub.o 

arch/arm/boot/compressed/lib1funcs.o 

arch/arm/boot/compressed/ashldi3.o 

arch/arm/boot/compressed/bswapsdi2.o 

-o arch/arm/boot/compressed/vmlinux



gcc 

-c 

-o arch/arm/boot/compressed/piggy.o 

linux/arch/arm/boot/compressed/piggy.S



.incbin "arch/arm/boot/compressed/piggy_data"



cat arch/arm/boot/compressed/../Image | gzip -n -f -9 > arch/arm/boot/compressed/piggy_data



objcopy -O binary -R .comment -S  vmlinux arch/arm/boot/Image

It's all in here:
http://en.wikipedia.org/wiki/Vmlinux

vmlinux:

A non-compressed and non-bootable Linux kernel file format, just an intermediate step to producing vmlinuz.

vmlinuz:

A compressed and bootable Linux kernel file. It is actually zImage or bzImage file.

zImage:

For old kernels, just fit 640k ram size.

bzImage:
Big zImage, no 640k ram size limit, can much larger.

Please refer this document: vmlinuz Definition.