Devices and partitions

Devices

Hardware RAID controllers have some drawback when used with ZFS, as ZFS in a raidz configuration is able to detect more errors and deal better with them than hardware RAID. It will be necessary to access each disk individually to have the full benefits of ZFS.

Usually hardware RAID:

• doesn’t allow a direct access to the disk, so it is necessary to create one RAID volume with one disk
• when the last disk is removed from a RAID volume, the volume is destroyed

This will lead to a situation where it is nearly impossible to ensure that device numbering will always stay the same across modification, addition or disk deletion, and these kinds of operation are common when failure of a drive occurs. This difficulty to logically represent disk devices, make it prone to failure for the operating system to find the right device when the computer is rebooted. This device numbering problem also exists with USB devices where numbers are assigned according to the ones previously allocated.

The solution is not to rely on the operating system automatic numbering of devices but to use explicit labels to identify the disks.

There are several way to create labels, through the GPT partition table, the filesystem (UFS, MSDOS, …) or the GEOM infrastructure:

1
2
3

gpart modify -i index -l label device     # GUID partition table (GPT)
tunefs -L label device                    # UFS filesystem
glabel label -v label device              # GEOM infrastructure

Listing all of the labels and their associated devices number can be done for all the created labels (GPT, UFS, GEOM, …) with:

1

glabel status

Partitions

Partitioning done using MBR doesn’t allow to have partition size of several TB. To avoid this limitation, GUID Partition Table (GPT) needs to be used for large disk. Disks will be systematically partitioned using GPT, with a possible exception for the disk used to boot because installation of FreeBSD 8 can be a bit more complex. From FreeBSD 9, installation procedure is able to create this type of partition.

To create a single partition (GPT style) using the whole disk on the device da1:

1
2
3

gpart create -s GPT         da1                     # Create GUID Partition Table
gpart add    -t freebsd-ufs da1                     # Add new partition for an UFS filesystem
gpart show                  da1                     # Show the disk partitions

To create a bootable disk, on the ad1 device, with a ZFS partition and a GPT partionning:

1
2
3
4
5

gpart create   -s GPT                         da1   # Creates a GUID partition table (GPT)
gpart add      -b 40   -s 984 -t freebsd-boot da1   # Creates a bootloader partition
gpart add      -b 1024 -s 5g  -t freebsd-zfs  da1   # Creates a ZFS partition
gpart bootcode -b /boot/pmbr                  da1   # Protects the MBR
gpart bootcode -p /boot/gptzfsboot -i 1       da1   # Installs the bootloader

The partition types that will be commonly used are:

Boot

Loader

The file /boot/loader.conf is automatically read and processed by the boot loader at startup, it contains instructions to load modules or pass configuration options to the kernel.

Since FreeBSD 13.0 the bootloader as the possibility to rollback ZFS checkpoint as well as to select the ZFS boot environment.

Unable to find the root partition

If the root partition (ie: /) is not found during the boot process; FreeBSD will prompt you for that information. Unfortunately, sometimes USB keyboards are not operational. The solution is to manually specify it from the boot loader (when BIOS is still handling the keyboard):

1

set vfs.root.mountfrom=????

To mount from ZFS, don’t forget you need to have the zfs module loaded by the bootloader.

System configuration (rc.conf)

System configuration is done by the /etc/rc.conf configuration file and it will override default values defined by the system (in /etc/defaults/rc.conf).

Below are some reasonable values ​​to:

• configure the keyboard and mouse
• avoid the standby screen, in order to display the information in case of kernel-panic
• check disk status
• enable crash dumps
• delete temporary directories (ie: /tmp/, …) at reboot time
• force fsck to run in unattended mode
• prevent syslog to listen on the network

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

# Keyboard / Mouse / Screen
#---------------------------
keymap="fr.iso.acc"                     # French keyboard
moused_flags="-3"                       # Emulate 3rd button
saver="NO"                              # No screen saver logo
blanktime="NO"                          # Never shutdown the screen

# Auto-diagnostic
#-----------------                 
smartd_enable="YES"                     # Check Harddisk health         
dumpdev="AUTO"                          # Save dump during kernel panic

# Misc
#------
clear_tmp_enable="YES"                  # Clear /tmp at startup
fsck_y_enable="YES"                     # Assume 'yes' for fsck
syslogd_flags="-s -s"                   # secure mode (no network)

Tuning

ZFS

• http://wiki.freebsd.org/ZFSTuningGuide
• http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide

Since FreeBSD 7.2, it is no more required to perform tuning on amd64 architecture with at least 2GB of memory to ensure a correct system behaviour.

Boot

The boot process can be customized in the /boot/loader.conf file.

To show a nice menu with beastie, the FreeBSD logo:

1
2

# Boot menu
loader_logo="beastie"

Network device

For network interface with heavy traffic load, it is advised to activate polling instead of using interrupts. This will reduce context switches overhead and chances of livelock.

Kernel needs to be compiled with the following options (the recommended value for HZ is 1000):

1
2

options         DEVICE_POLLING          # Device polling support
options         HZ=1000                 # Granularity of clock interrupts

Activating polling mode is done using ifconfig for the network card supporting it:

1

ifconfig device polling

Services

• Man pages: rc (8), rc.conf (5)

The whole set of services can be manually started or stopped if needed. This is managed by scripts available in the /etc/rc.d/ and /usr/local/etc/rc.d/ directories.

Some examples of script invocation (more actions are available: start, stop, restart, reload, status, poll):

1
2
3
4
5
6
7

# Depends on rc.conf value (requires: `rc_script`_enabled="YES")
service rc_script start        # Start service
service rc_script stop         # Stop  service

# Executes service action independently of rc.conf configuration
service rc_script onestart     # Start service
service rc_script onesstop     # Stop  service

Debug

DTrace

DTrace is a dynamic tracing framework for troubleshooting kernel and application problems on production systems in real time. This tool has been imported from the Solaris operating system and until now in FreeBSD is essentially functional for the kernel.

The kernel needs to be compiled with the following options to support the DTrace entry points and to embed the “Compact C Type Format” required:

1
2
3
4

options KDTRACE_HOOKS        # all architectures - enable general DTrace hooks
options KDTRACE_FRAME        # amd64-only
makeoptions WITH_CTF=yes     # embed Compact C Type code
options DDB_CTF              # all architectures - kernel ELF linker loads CTF data

Embeding the CTF inside the code is done by setting the WITH_CTF variable, however:

• FreeBSD 8: This variable is not taken into account in the files /etc/src.conf or /etc/make.conf, the compiling method to use it is then: make kernel KERNCONF=YOUR_KERNEL WITH_CTF=1
• FreeBSD 9: This variable can be set up in the files /etc/src.conf or /etc/make.conf, however, it can have some side effects. The method to use is the one described previously, using makeoptions WITH_CTF=yes in the kernel configuration file.