wiki

# Custom in memory OpenBSD image

Some months ago, I was playing around FreeBSD in memory system (I don't remember the name of the project), it was a really nice thing and put some custom functions (like dynamically add package in image, modify size of the image and so on).

I like FreeBSD, but I prefer OpenBSD for lot of reason, and I was thinking: what if I do the same with OpenBSD? A lightweight pre-installed in memory image, that you can customize on demand? Is it me or it looks like MirageOS or RumpKernel? Yeah, a little but this is actually just for fun (and for test)!

Here the question I want to ask:

  • How OpenBSD releases are made under the hood?
  • How bsd.rd is built?
  • How bsd.rd works?
  • How to build a custom bsd.rd?
    • How to remove features (remove software/configuration)?
    • How to add features (add software/configuration)?
    • How to design new rc script?
  • How to generate a fully custom image?
  • How to boot it? from cdrom? from network? from disk (raw, qcow)?
  • (bonus) How to install package into a custom image?
  • (bonus) How to run this image in vmd?
  • (bonus) How to cross-build a custom image for different architecture?
  • (bonus) How to build it automatically in CD/CI pipeline?
  • (bonus) How to install/configure buildbot on it?
  • (bonus) How to install/configure ansible on it?
  • (bonus) How to make a secure access?
  • (bonus) How and where to store data?
  • (bonus) How to benchmark it?

Notes: you need to get OpenBSD sources from git or cvs. Please read the doc, everything is noted somewhere. I am using OpenBSD-current, but I think using a stable release can make things easier for both of us.

# How to make an OpenBSD release from scratch?

TL;DR:

# cd /usr/src
# make obj && make build

from https://man.openbsd.org/release ❤️

# How to build a ramdisk image automatically?

If you already have build an OpenBSD release, you already have used make tool. Like gnumake it gives you the ability to automate things. This is the main scripting language to build OpenBSD release, but you can also find some sed or awk scripts.

# How to build a ramdisk image manually? The hard way.

So, to build an OpenBSD ramdisk, we need to read these files

  • http://bxr.su/OpenBSD/sys/arch/amd64/conf/RAMDISK_CD
  • http://bxr.su/OpenBSD/sys/arch/amd64/conf/RAMDISK
  • http://bxr.su/OpenBSD/distrib/miniroot/
  • http://bxr.su/OpenBSD/distrib/landisk/ramdisk/
  • http://bxr.su/OpenBSD/distrib/amd64/ramdisk_cd/
  • http://bxr.su/OpenBSD/distrib/amd64/ramdiskA/

To fully understand how it works, we need to read some Makefile. I want to create amd64 image, so I will look around distrib/amd64/ramdisk_cd. Okay! We need to read the Makefile. Before starting anything, please ensure you are reading the same file, here the release:

#       $OpenBSD: Makefile,v 1.20 2019/06/07 14:39:56 deraadt Exp $

# crunchgen configuration generator

crunchgen(8) is used to create one binary containing many. If you have done some embedded program or play around some light version of linux, it is the same as (don't remember the name).

# line 93
# instbin.conf: ${LISTS}

# line 94
# awk -f ${UTILS}/makeconf.awk  ${LISTS} > instbin.conf
# this script will generate a configuration file
# for crunchspec from the list specification
awk -f ../../miniroot/makeconf.awk list > instbin.conf

The list file contains different instruction about the files that will be installed in the ramdisk

# Build libraries

# line 89
# instbin: instbin.mk instbin.cache instbin.c

# line 90
# ${MAKE} -f instbin.mk SRCLIBDIR=${.CURDIR}/../../../lib all
make -f instbin.k SRCLIBDIR=../../../lib all

# line 91
# strip -R .comment -R .SUNW_ctf instbin
strip -R .comment -R .SUNW_ctf instbin

# Generate a crunched binary

# line 85
# instbin.mk instbin.cache instbin.c: instbin.conf

# line 86:87
# crunchgen -E -D ${.CURDIR}/../../.. -L ${DESTDIR}/usr/lib -c instbin.c -e instbin -m instbin.mk instbin.conf
crunchgen -E -D ../../.. -L ./usr/lib -c instbin.c -e instbin -m instbin.mk instbin.conf

# Build crunched libraries

# line 89
# instbin: instbin.mk instbin.cache instbin.c

# line 90
# ${MAKE} -f instbin.mk SRCLIBDIR=${.CURDIR}/../../../lib all
make -f instbin.mk SRCLIBDIR=../../../lib all

# line 91
# strip -R .comment -R .SUNW_ctf instbin
strip -R .comment -R .SUNW_ctf instbin

# Generate local file system

# line 75
# mr.fs: instbin

# line 76
# rm -rf $@.d
rm -rf mr.fs.d

# line 77
#  install -d -o root -g wheel $@.d
install -d -o root -g wheel mr.fs.d

# line 78
# mtree -def ${MTREE} -p $@.d -u
mtree -def ../../miniroot/mtree.conf -p mr.fs.d -u

# line 79:81
# CURDIR=${.CURDIR} OBJDIR=${.OBJDIR} OSrev=${OSrev} TARGDIR=$@.d UTILS=${UTILS} RELEASEDIR=${RELEASEDIR} sh ${UTILS}/runlist.sh ${LISTS}

# line 82
# rm $@.d/instbin
rm mk.fs.d/instbin

# line 83
# makefs ${MRMAKEFSARGS} $@ $@.d
makefs -o disklabel=rdrootb,minfree=0,density=4096

# convert disk image as ramdisk

# line 67
# cp bsd bsd.rd
cp bsd bsd.rd

# line 68
# rdsetroot bsd.rd mr.fs
rdsetroot bsd.rd mr.fs

# clean kernel

# line 60
# 

# line 61
# cp bsd.rd bsd.strip
cp bsd.rd bsd.strip

# line 62
# strip bsd.strip
strip bsd.strip

# line 63
# strip -R .comment -R .SUNW_ctf bsd.strip
strip -R .comment -R .SUNW_ctf bsd.strip

# line 64
# gzip -9cn bsd.strip > bsd.gz
gzip -9cn bsd.strip > bsd.gz

# generate disk image

# line 18
# ${FS}: bsd.gz
# miniroot65.fs: bsd.gz

# line 19
# dd if=/dev/zero of=${FS} bs=512 count=${FSSIZE}
# first we will create a raw image where 
#   ${FS} is set to miniroot${OSrev}.fs
#   ${FSSIZE} is set to 9600
# it will generate an image named miniroot65.fs
# with a size of (4915200) 9600*512 
dd if=/dev/zero of=miniroot65.fs bs=512 count=9600

# line 20
# vnconfig -v ${FS} > vnd
# we create a vnode based on the created file
# and store the vnode name to vnd file (usually /dev/vndX)
vnconfig -v miniroot65.fs > vnd

# line 21
# fdisk -yi -l ${FSSIZE} -b 960 -f ${DESTDIR}/usr/mdec/mbr `cat vnd`
# we create a new mbr schema based on previous block size (9600)
# by using the MBR located in /usr/mdec/mbr to the vnode
fdisk -yi -l 9600 -b 960 -f /usr/mdec/mbr `cat vnd`

# line 22
# disklabel -wAT ${.CURDIR}/template `cat vnd`
# we create a new pack label based from the file ./template
disklabel -wAT /template `cat vnd`

# line 23
# newfs -t msdos /dev/r`cat vnd`i
# the first partition is for UEFI, and must
# be formated in fat32. the path used look
# like /dev/rvndXi
 newfs -t msdos /dev/r`cat vnd`i

# line 24
# mount /dev/`cat vnd`i ${MOUNT_POINT}
# we mount the UEFI partition to defined
# mount point (usually /mnt)
mount /dev/`cat vnd`i /mnt

# line 25
# mkdir -p ${MOUNT_POINT}/efi/boot
# we create the standard efi boot dir
mkdir -p /mnt/efi/boot

# line 26
# cp ${EFIBOOT} ${MOUNT_POINT}/efi/boot
# and we copy the EFI into the right path
cp /usr/mdec/BOOTX64.EFI /usr/mdec/BOOTIA32.EFI /mnt/efi/boot 

# line 27
# umount ${MOUNT_POINT}
# we can now umount the EFI partition
umount /mnt

# line 28 
# newfs -m 0 -o space -i 524288 -c ${FSSIZE} /dev/r`cat vnd`a
# We now format our root partition with 524288 inodes
# available with 0% free reserved space and optimized
# for space. The path look like /dev/rvdXa.
newfs -m 0 -o space -i 524288 -c 9600 /dev/r`cat vnd`a

# line 29
# mount /dev/`cat vnd`a ${MOUNT_POINT}
# we mount our vnode to /mnt
mount /dev/`cat vnd`a /mnt

# line 30
# cp ${DESTDIR}/usr/mdec/boot ${.OBJDIR}/boot
# we copy our boot file into OBJDIR directory
cp /usr/mdec/boot boot

# line 31
# strip ${.OBJDIR}/boot
# we remove symbols symbols
strip boot

# line 32
# strip -R .comment -R .SUNW_ctf ${.OBJDIR}/boot
# we remove the .comment section and the .SUNW_ctf section
strip -R .comment -R .SUNW_ctf boot

# line 33:34
# installboot -v -r ${MOUNT_POINT} `cat vnd` ${DESTDIR}/usr/mdec/biosboot ${.OBJDIR}/boot
# we can now install the boot file
install -v -r /mnt `cat vnd` /usr/mdec/biosboot /boot

# line 35
# install -c -m 555 -o root -g wheel bsd.gz ${MOUNT_POINT}/bsd
install -c -m 555 -o root -g wheel bsd.gz /mnt/bsd

# line 36
# df -i ${MOUNT_POINT}
df -i /mnt

# line 37
# umount ${MOUNT_POINT}
umount /mnt

# line 38
# vnconfig -u `cat vnd`
vnconfig -u `cat vnd`

# line 39
# rm -f vnd
rm -f vnd

# Resources

  • https://man.openbsd.org/make
  • https://man.openbsd.org/dd
  • https://man.openbsd.org/vnconfig
  • https://man.openbsd.org/fdisk
  • https://man.openbsd.org/disklabel
  • https://man.openbsd.org/newfs
  • https://man.openbsd.org/strip
  • https://man.openbsd.org/installboot
  • https://man.openbsd.org/install
  • https://man.openbsd.org/mkhybrid
  • https://man.openbsd.org/rdsetroot
  • https://man.openbsd.org/mtree
  • https://man.openbsd.org/makefs
  • https://man.openbsd.org/crunchgen
  • https://man.openbsd.org/mtree
  • http://bxr.su
  • http://cvsweb.openbsd.org/