hardened Archives - Tom's Basement

Note: this note has been originally written end of 2015, but I updated it on 4 feb 2017 as some steps were not consistent anymore.

Assumptions

This tutorial tries to stick to a step-by-step approach, but your chances to reach the end with a working system will be most likely higher if you already have previous experience installing Gentoo systems. If not, I would recommend getting some by following the Gentoo Handbook on a virtual system like VMware Workstation/Player. Once you get familiar following the handbook and installing a simple system, you’ll be able to move on to the slightly more complex setup described here.

The main differences between the Gentoo Handbook and this tutorial are the emphasis on RAID and the use of Gentoo Hardened. I’ve actually put this tutorial together to remember how to install the OS on the rented server this blog is running on, or if I have to do it again to install a new server. Once you get a working system under VMware Workstation/Player, it should be rather straightforward to adapt this tutorial to any virtual or physical hardware.

For this tutorial I assume a RAID1 setup on two physical (or virtual) 16GB+ hard drives. When using VMWare Workstation/Player, create your two virtual hard drives on different physical hard drives if possible, to make it easier on your drives mechanics. If you have SSDs then it doesn’t really matter.

The kernel config linked in this tutorial should work as-is if you use VMware Workstation/Player, but it doesn’t have many features activated. You should customize it further according to your system and/or the features you need. Network setup is assumed to be based on DHCP, localization is minimal.

Note about RAID: RAID won’t protect you from silent data corruption, it’s really a basic tool to recover from the failure of a drive assuming that the data on the other drive(s) is reliable. Last generation filesystems such as ZFS or BTRFS should be preferred over software or hardware RAID when possible. At some point I should edit this post to describe a ZFS based install… oh well, if I have the time.

Let’s start. Boot on your minimal install .iso, or system rescue. Note: I’ve linked some info about OVH/SoYouStart servers, but before you try this on one of their offers, or similar, be sure you understand how to configure your firewall so that your hosting company won’t dispatch maintenance because they are not able to monitor your server.

Prepare the drives

Wipe out disk A
gdisk /dev/sda

Type x (for expert), then z (for zap), answer Y (wipe out GPT) and again Y (blank out MBR)

Then do the same for disk B
gdisk /dev/sdb

Repeat same steps (as for disk A): type x (for expert), then z (for zap), answer Y (wipe out GPT) and again Y (blank out MBR)

Partition disk A
gdisk /dev/sda

Type o (create a new empty GUID partition table), answer Y (proceed)

Type n, type enter to select the default partition number (partition 1), type enter to select the default first sector value 2048, type +2M for last sector to create a 2M BIOS partition, type ef02 for hex code to set a BIOS partition type

Type n, type enter to select the default partition number at this stage (partition 2), type enter to select the default first sector value, type +128M for last sector to create a 128M boot partition, type fd00 for hex code to set a software raid partition type

Type n, type enter to select the default partition number at this stage (partition 3), type enter to select the default first sector value, type +2G for last sector to create a 2GB swap partition, type fd00 for hex code to set a software raid partition type

Type n, type enter to select the default partition number at this stage (partition 4), type enter to select the default first sector value, type enter for last sector to create a root partition using the remaining space, type fd00 for hex code to set a software raid partition type

Type w to write your changes to disk a, answer Y (proceed)

Now clone the partition layout of disk A to disk B
sgdisk -R=/dev/sdb /dev/sda

And create a new GUID for disk B
sgdisk -G /dev/sdb

Activate your software raid partitions

Activate a software RAID1 to create the boot partition (where the “/boot” directory will be located)
mdadm --create --name=server:boot --verbose /dev/md/boot --level=1 --raid-devices=2 /dev/sda2 /dev/sdb2
answer y

Note: we are not using /dev/sda1 and /dev/sdb1 for the RAID1 boot partition because /dev/sda1 and /dev/sdb1 will be used by GRUB to host the bootloader

Activate a software RAID1 to create the swap partition
mdadm --create --name=server:swap --verbose /dev/md/swap --level=1 --raid-devices=2 /dev/sda3 /dev/sdb3
answer y

Activate a software RAID1 to create the root partition (where the “/” directory will be located)
mdadm --create --name=server:root --verbose /dev/md/root --level=1 --raid-devices=2 /dev/sda4 /dev/sdb4
answer y

Note: if you want RAID5 mode instead of RAID1, replace --level=1 --raid-devices=2 by --level=5 --raid-devices=<n> with n being at least 3, and the right partition names. You’ll also need RAID5 activated in the kernel.

The rest of this howto is now very similar to the standard Gentoo handbook, except that instead of {/dev/sda2, /dev/sda3, /dev/sda4} found in the handbook, our boot, swap, root partitions are respectively {/dev/md/boot, /dev/md/swap, /dev/md/root}

Install Gentoo “almost” as usual

Create an ext2 filesystem on the boot partition
mkfs.ext2 /dev/md/boot

Create a swap on the swap partition
mkswap /dev/md/swap

Activate the swap
swapon /dev/md/swap

Create an ext4 filesystem on the root partition
mkfs.ext4 /dev/md/root

Get the latest portage tree snapshot (use your preferred mirror that you can select from the gentoo mirrors)
wget http://gentoo.mirrors.ovh.net/gentoo-distfiles/snapshots/portage-latest.tar.xz

Get the latest hardened stage3, for example at the time of writing (you need to dig in the autobuilds directory to get the latest, DO NOT pick the “nomultilib” version unless you know what you are doing)
wget http://gentoo.mirrors.ovh.net/gentoo-distfiles/releases/amd64/autobuilds/current-stage3-amd64-hardened/stage3-amd64-hardened-20170202.tar.bz2

Create the gentoo directory in /mnt if it’s not already there
mkdir -p /mnt/gentoo

Mount the root partition in the directory you’ve just created
mount /dev/md/root /mnt/gentoo/

Create the boot directory
mkdir /mnt/gentoo/boot

Mount the boot partition in the directory you’ve just created
mount /dev/md/boot /mnt/gentoo/boot

Extract the stage3 to the root of the target filesystem. Note: if you get an error message from the tar utility complaining about missing extended attributes support, it’s not a show stopper to install a hardened system, but you will need to re-emerge your full system after completing the install guide to get the correct extended attributes on your files. More on this later.
tar xpf stage3* --xattrs -C /mnt/gentoo/

Extract the portage tree snapshot to /usr in the target filesystem
tar xf portage* -C /mnt/gentoo/usr/

Link your /etc/resolv.conf file to the target root filesystem
cp -L /etc/resolv.conf /mnt/gentoo/etc/

Mount /proc in the target root filesystem
mount -t proc none /mnt/gentoo/proc

Bind /sys in the target root filesystem
mount --rbind /sys /mnt/gentoo/sys

Bind /dev in the target root filesystem
mount --rbind /dev /mnt/gentoo/dev

Now chroot yourself in the filesystem you just prepared
chroot /mnt/gentoo /bin/bash

Update your environment variables in the chroot
source /etc/profile

Change the prompt to remind us that we are in chroot
export PS1="(chroot) $PS1"

Optional: if you have enough RAM (≥8GB) you can save compile time (and the life or your SSDs if you use them as disk A and disk B) by mounting /var/tmp/portage in RAM
mkdir /var/tmp/portage
mount -t tmpfs tmpfs -o nr_inodes=1M /var/tmp/portage

Set your timezone. My timezone here is “Europe/Berlin” but if yours is different pick one from /usr/share/zoneinfo/
echo "Europe/Berlin" > /etc/timezone

Reconfigure the sys-libs/timezone-data package
emerge --config sys-libs/timezone-data

Select the locale, uncomment the entry en_US.UTF-8 UTF-8 in /etc/locale.gen
nano /etc/locale.gen

#en_US ISO-8859-1
en_US.UTF-8 UTF-8
#ja_JP.EUC-JP EUC-JP
#ja_JP.UTF-8 UTF-8
#ja_JP EUC-JP
#en_HK ISO-8859-1
#en_PH ISO-8859-1
#de_DE ISO-8859-1
#de_DE@euro ISO-8859-15
#es_MX ISO-8859-1
#fa_IR UTF-8
#fr_FR ISO-8859-1
#fr_FR@euro ISO-8859-15
#it_IT ISO-8859-1

Generate the locale
locale-gen

Activate the locale, first look for the entry related to en_US.utf8
eselect locale list

Available targets for the LANG variable:
  [1]   C
  [2]   POSIX
  [3]   en_US.utf8
  [ ]   (free form)

Select the entry related to en_US.utf8, for me it’s the third but yours could be different
eselect locale set 3

reload your environment
env-update && source /etc/profile && export PS1="(chroot) $PS1"

Now we need to edit /etc/fstab. Recent changes in udev require the use of UUID’s. To find them, issue
lsblk -f

And look for the UUID’s of the /dev/mdXXX devices. You’ll need them for the /etc/fstab file.

Edit your fstab
nano /etc/fstab

Here is my /etc/fstab file, but you need to adjust yours according to the values of your UUID’s. Pay attention to the mount points to put the right UUIDs. Again, you can uncomment the last line if you have lots of RAM

#<fs>                                           <mountpoint>            <type>          <opts>          <dump/pass>
UUID=9ba69117-4896-41d4-bbac-c7ccb3fa0ced       /boot                   ext2            auto,noatime            1 2
UUID=45c9cc7e-3967-4692-a523-6fe4112bb827       /                       ext4            noatime                 0 1
UUID=377610a3-b921-4b4a-afa3-54b4f8cfb1bf       none                    swap            sw                      0 0
#tmpfs                                          /var/tmp/portage        tmpfs           auto,nr_inodes=1M       0 0

(optional) set your hostname, for the example we use “gentoo” here
nano /etc/conf.d/hostname

hostname="gentoo"

Configure your network with DHCP
nano /etc/conf.d/net

config_eth0="dhcp"

Alternatively, if you’re not on DHCP, set your IPv4 (and optionnally IPv6) and respective default gateways. Here in my example I have only IPv4, replace the x’s by your actual values.
nano /etc/conf.d/net

config_eth0="x.x.x.x/x"
routes_eth0="default via x.x.x.x"

Also, if you’re not on DHCP, set your DNS server(s), replace the x’s by the IP of your DNS server(s)
nano /etc/resolv.conf

nameserver x.x.x.x
nameserver x.x.x.x

If you’re not on DHCP and you have changed your hostname, update your hosts file with your IP addresse(s). Replace the x’s by your IP address set above, add your IPv6 address if you have one. If your host name has a FQDN, you should put it in.
nano /etc/hosts

x.x.x.x gentoo

If you’re on DHCP and you have changed your hostname, for gentoo for example, add your hostname to the localhost IPv4 and IPv6 aliases to avoid headaches with some software that bind to the hostname.
nano /etc/hosts

# IPv4 and IPv6 localhost aliases
127.0.0.1       localhost gentoo
::1             localhost gentoo

Now in any case (DHCP or static) set eth0 to be a standard service during startup
cd /etc/init.d/
ln -s net.lo net.eth0
cd
rc-update add net.eth0 default

Allow the test version of the gradm package
nano /etc/portage/package.keywords

sys-apps/gradm

Adjust your make.conf file according to your hardware and location. Your CPU_FLAGS_X86 variable can be found by emerging app-portage/cpuid2cpuflags and running cpuinfo2cpuflags-x86. Below is my file.
nano /etc/portage/make.conf

CFLAGS="-O2 -pipe -march=native"
CXXFLAGS="${CFLAGS}"
CHOST="x86_64-pc-linux-gnu"
USE="unicode python lzma xattr"
CPU_FLAGS_X86="aes avx mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3"
EMERGE_DEFAULT_OPTS="--jobs=4 --load-average=4.0 --keep-going --with-bdeps y --complete-graph"
MAKEOPTS="-j4"
GENTOO_MIRRORS="http://gentoo.mirrors.ovh.net/gentoo-distfiles/"
PORTDIR="/usr/portage"
DISTDIR="${PORTDIR}/distfiles"
PKGDIR="${PORTDIR}/packages"
PORTAGE_NICENESS=10
GRUB_PLATFORMS="pc"
KERNEL="linux"
PAX_MARKINGS="XT"

Create the repos.conf directory that is needed for syncing the portage tree
mkdir /etc/portage/repos.conf

Create the gentoo.conf file
nano /etc/portage/repos.conf/gentoo.conf

Mine looks like this

[DEFAULT]
main-repo = gentoo

[gentoo]
location = /usr/portage
sync-type = rsync
sync-uri = rsync://rsync2.fr.gentoo.org/gentoo-portage/
auto-sync = yes

Sync the portage tree
emerge --sync

Read the news. You’ll need to do this several times during this tutorial.
eselect news read | more

Install and update eix, it will be useful later. Remember here that this is Gentoo, everything is compiled from source, so installing packages takes time 🙂
emerge eix
eix-update

Install the kernel source and a few needed utils
emerge hardened-sources && emerge gradm mdadm genkernel

If you read the news again here, you’ll see that unfortunately grsecurity only makes the test versions of their kernel patch available to the public. That’s infortunate for us small end users, but still better than nothing.

Create the MDADM config file
mdadm -Es >> /etc/mdadm.conf

And start the MDADM service at boot
rc-update add mdadm boot

Edit some needed genkernel config tweaks, make sure the following options are uncommented / adjusted as follows. Note, in the genkernel config file I suggest to disable the CLEAN option because if you need to adjust your kernel config file and compile it again, you’ll save a lot of time.
nano /etc/genkernel.conf

CLEAN="no"
MRPROPER="no"
MDADM="yes"
MDADM_CONFIG="/etc/mdadm.conf"
COMPRESS_INITRD_TYPE="xz"

Edit your kernel config, or grab my config which should work on VMWare Workstation/Player (as long as you stay in the 4.8.x release, but you can always start from it with a “make oldconfig” on 4.9+). Note: the config file will also work on the rented server I run this blog on.
wget https://spacetux.org/tommie/.config -O /usr/src/linux/.config

Now compile and install your kernel. Go take a coffee, it usually takes a few minutes.
genkernel all

Install the bootloader
emerge grub

Edit a few grub tweaks
nano /etc/default/grub

Uncomment and change the following line

GRUB_CMDLINE_LINUX_DEFAULT="domdadm rootfstype=ext4 net.ifnames=0"

Deactivate a few unneeded grub boot options, create a grub config file, and install the bootloader on /dev/sda and /dev/sdb
cd /etc/grub.d/
chmod -x 20_linux_xen 30_os-prober 40_custom 41_custom
mkdir /boot/grub
cd
grub-mkconfig -o /boot/grub/grub.cfg
grub-install /dev/sda
grub-install /dev/sdb

Install the system logger
emerge syslog-ng

Start the system logger automatically
rc-update add syslog-ng default

(optional) start the ssh daemon. This is obviously mandatory if you’re following the guide to install a remote server, but then you should also secure your sshd config a lot more. It’s not covered here.
rc-update add sshd default

Add a user (change the user’s name if you don’t like “user”)
useradd -m -G wheel -s /bin/bash user

Change the root password
passwd

Change the user’s password
passwd user

Uncomment a few openrc settings
nano /etc/rc.conf

rc_logger="YES"
rc_sys=""

Remove the files portage left in its tmp directory
rm -rf /var/tmp/portage/*

Quit the chroot
exit

Unmount all our stuff
umount -R /mnt/gentoo

And…phew. We’re (almost) done, reboot !
shutdown -r now

If all went well the system should boot fine.

Post install

At the time of writing, init complains that udev doesn’t use the proper init script format, so let’s fix this
perl -i -pe 's/runscript/openrc-run/g' /etc/init.d/udev

(optional) create a trusted users group
groupadd -g 101 trusted

(optional) and add your user to it, or you won’t be able to run executable scripts created under this user
gpasswd -a user trusted

(optional) if the tar utility you used to extract the stage3 tarball doesn’t support extended attributes, it’s safer to recompile your whole system. This will take anywhere between half an hour and several hours depending on your hardware. At the time of writing, the tar version on the minimal Gentoo install .iso supports xattr, so this is not needed
emerge -e @world

(optional) if you recompiled your system in the step above, you can safely ignore all /etc updates (choose the -7 option when running the etc-update utility), and then clean your /usr/portage/disfiles directory as there will be a lot of source code there
rm /usr/portage/disfiles/*

(optional) when you’re done installing and happy with your kernel config, then clean the kernel source directory from all the pre-compiled objects
cd /usr/src/linux
make clean

admin December 13th, 2015

Posted In: gentoo

Tags: gentoo, hardened, howto, software raid

Tom's Basement

Random Geekery

Gentoo Hardened on software RAID howto