Poudriere Inside FreeBSD VNET Jail

I usually prefer to run Poudriere inside FreeBSD Bhyve VM – its just faster to setup – but as sometimes its desired to use the power of FreeBSD Jails instead – here is how to do this.

First we will setup the host system.

To make things faster we will install the host system using FreeBSD provided raw system image – as described – Install FreeBSD with One Command – here for example. You can also use the bsdinstall(8) installer it needed.

Here is how we expect the host system to look like.

HOST # cat /etc/rc.conf hostname="host.lab.org" ifconfig_vtnet0="inet 10.1.1.123/24 up" defaultrouter="10.1.1.1" cloned_interfaces="bridge0" ifconfig_bridge0="up addm vtnet0" gateway_enable=YES kld_list="${kld_list} linux linux64" sshd_enable="YES" zfs_enable="YES" nginx_enable="YES"

We will need to configure the devfs.rules file.

HOST # cat << EOF > /etc/devfs.rules [poudriere=120] add include $devfsrules_jail add path 'fd*' unhide EOF

… and some system settings.

HOST # cat << EOF > /etc/sysctl.conf # SECURITY security.bsd.see_jail_proc=0 security.bsd.unprivileged_proc_debug=0 # JAILS/ALLOW UPGRADES IN JAILS security.jail.chflags_allowed=1 # JAILS/ALLOW RAW SOCKETS security.jail.allow_raw_sockets=1 # JAILS/ALLOW fdescfs(5) security.jail.mount_fdescfs_allowed=1 security.jail.param.allow.mount.fdescfs=1 # JAILS/poudriere/INSIDE/JAIL security.jail.sysvipc_allowed=1 EOF

For some reason the /etc/jail.conf file must exist if you use /etc/jail.conf.d/ dir.

HOST # :> /etc/jail.conf

Now – the Poudriere Jail config – I will call it joudriere for convenience.

HOST # cat /etc/jail.conf.d/joudriere.conf joudriere { # GLOBAL exec.start = "/bin/sh /etc/rc"; exec.stop = "/bin/sh /etc/rc.shutdown"; exec.consolelog = "/var/log/jail_console_${name}.log"; exec.clean; mount.devfs; host.hostname = joudriere.lab.org; path = /jail/${name}; # PERMISSIONS allow.chflags; allow.read_msgbuf; allow.mlock; allow.chflags; allow.mount; allow.mount.devfs; allow.mount.nullfs; allow.mount.fdescfs; allow.mount.nullfs; allow.mount.tmpfs; allow.mount.zfs=true; allow.mount.procfs; allow.mount.linprocfs; allow.raw_sockets; allow.socket_af; allow.sysvipc; devfs_ruleset = 120; enforce_statfs = 1; children.max = 200; sysvmsg=new; sysvsem=new; sysvshm=new; # VNET/VIMAGE vnet; vnet.interface = "${if}b"; # NETWORKS/INTERFACES $id = "234"; $ip = "10.1.1.${id}/24"; $gw = "10.1.1.1"; $br = "vm-public"; $if = "epair${id}"; # ADD TO bridge0 INTERFACE exec.prestart += "ifconfig ${if} create up"; exec.prestart += "ifconfig ${if}a up descr jail:${name}"; exec.prestart += "ifconfig ${br} addm ${if}a up"; exec.start += "ifconfig ${if}b ${ip} up"; exec.start += "route add default ${gw}"; exec.poststop += "ifconfig ${if}a destroy"; }

We will need to extract FreeBSD base.txz there.

HOST # mkdir -p /jail/joudriere /jail/BASE HOST # fetch -o /jail/BASE/14.3-RELEASE-base.txz 'https://download.freebsd.org/releases/amd64/14.3-RELEASE/base.txz' HOST # tar --unlink -C /jail/joudriere -xf /jail/BASE/14.3-RELEASE-base.txz

Now we can start our Poudriere joudriere Jail and enter it for more setup.

HOST # service jail onestart joudriere HOST # env PS1='joudriere # ' jexec joudriere /bin/sh

Some basic stuff for a start.

joudriere # service sshd enable joudriere # service sshd start joudriere # service nginx enable joudriere # service nginx start joudriere # sed -i '' s.quarterly.latest.g /etc/pkg/FreeBSD.conf joudriere # echo nameserver 1.1.1.1 > /etc/resolv.conf joudriere # pkg install -y poudriere-devel tree screen git-lite

The Jail setup main /etc/rc.conf config file.

joudriere # cat /etc/rc.conf sshd_enable="YES" nginx_enable="YES"

Now the rest of the Poudriere setup commands.

joudriere # export SSL=/usr/local/etc/ssl joudriere # mkdir -p \ /usr/ports/distfiles \ ${SSL}/keys \ ${SSL}/certs joudriere # chmod 0600 ${SSL}/keys joudriere # openssl genrsa -out ${SSL}/keys/poudriere.key 4096 joudriere # openssl rsa \ -in ${SSL}/keys/poudriere.key -pubout \ -out ${SSL}/certs/poudriere.cert joudriere # mkdir /var/ccache joudriere # cat << EOF > /usr/local/etc/poudriere.conf ZPOOL=zroot FREEBSD_HOST=ftp://ftp.freebsd.org BASEFS=/usr/local/poudriere POUDRIERE_DATA=/usr/local/poudriere/data DISTFILES_CACHE=/usr/ports/distfiles CCACHE_DIR=/var/ccache CHECK_CHANGED_OPTIONS=verbose CHECK_CHANGED_DEPS=yes PKG_REPO_SIGNING_KEY=/usr/local/etc/ssl/keys/poudriere.key URL_BASE=http://0.0.0.0/ USE_TMPFS=no TMPFS_LIMIT=12 MAX_MEMORY=12 PARALLEL_JOBS=4 PREPARE_PARALLEL_JOBS=4 MAX_FILES=4096 KEEP_OLD_PACKAGES=yes KEEP_OLD_PACKAGES_COUNT=3 CHECK_CHANGED_OPTIONS=verbose CHECK_CHANGED_DEPS=yes RESTRICT_NETWORKING=no EOF joudriere # mkdir -p /usr/local/poudriere/data/logs/bulk joudriere # ln -s \ /usr/local/etc/ssl/certs/poudriere.cert \ /usr/local/poudriere/data/logs/bulk/poudriere.cert joudriere # cat << EOF > /usr/local/etc/poudriere.d/make.conf # general ALLOW_UNSUPPORTED_SYSTEM=yes DISABLE_LICENSES=yes # ccache(1) WITH_CCACHE_BUILD=yes # ports options FORCE_MAKE_JOBS=yes MAKE_JOBS_UNSAFE=yes MAKE_JOBS_NUMBER=8 EOF joudriere # sed -i '' -E 's|text/plain[\t\ ]*txt|text/plain txt log|g' /usr/local/etc/nginx/mime.types joudriere # cat << EOF > /usr/local/etc/nginx/nginx.conf events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; server { listen 80 default; server_name 0.0.0.0; root /usr/local/share/poudriere/html; location /data { alias /usr/local/poudriere/data/logs/bulk; autoindex on; } location /packages { root /usr/local/poudriere/data; autoindex on; } } } EOF joudriere # mkdir /root/.cache joudriere # ln -sf /var/ccache /root/.cache/ccache joudriere # cat < /var/ccache/ccache.conf max_size = 0 cache_dir = /var/ccache base_dir = /var/ccache hash_dir = false EOF

We will now fetch FreeBSD release and also the FreeBSD Ports tree.

joudriere # poudriere jail -c -j 14-3-R-amd64 -v 14.3-RELEASE joudriere # poudriere jail -l JAILNAME VERSION OSVERSION ARCH METHOD TIMESTAMP PATH 14-3-R-amd64 14.3-RELEASE 1403000 amd64 http 2025-07-01 18:15:02 /usr/local/poudriere/jails/14-3-R-amd64 joudriere # poudriere ports -c joudriere # poudriere ports -l PORTSTREE METHOD TIMESTAMP PATH default git+https 2025-07-01 18:21:08 /usr/local/poudriere/ports/default

Now a final test – the poudriere-bulk(8) job.

Without better target I have chosen to build devel/cmake package.

joudriere # poudriere bulk -j 14-3-R-amd64 -b latest devel/cmake [00:00:00] Creating the reference jail... done [00:00:02] Mounting system devices for 14-3-R-amd64-default [00:00:02] Warning: Using packages from previously failed, or uncommitted, build: /usr/local/poudriere/data/packages/14-3-R-amd64-default/.building [00:00:02] Mounting ccache from: /var/ccache [00:00:02] Mounting ports from: /usr/local/poudriere/ports/default [00:00:02] Mounting packages from: /usr/local/poudriere/data/packages/14-3-R-amd64-default [00:00:02] Mounting distfiles from: /usr/ports/distfiles [00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/make.conf /etc/resolv.conf -> /usr/local/poudriere/data/.m/14-3-R-amd64-default/ref/etc/resolv.conf [00:00:02] Starting jail 14-3-R-amd64-default Updating /var/run/os-release done. [00:00:02] Will build as root:wheel (0:0) [00:00:03] Ports supports: FLAVORS SUBPACKAGES SELECTED_OPTIONS [00:00:03] Inspecting /usr/local/poudriere/data/.m/14-3-R-amd64-default/ref//usr/ports for modifications to git checkout... no [00:00:05] Ports top-level git hash: 385ccb2a2 [00:00:05] Acquiring build logs lock for 14-3-R-amd64-default... done [00:00:05] Logs: /usr/local/poudriere/data/logs/bulk/14-3-R-amd64-default/2025-07-03_00h26m17s [00:00:05] WWW: http://0.0.0.0//build.html?mastername=14-3-R-amd64-default&build=2025-07-03_00h26m17s [00:00:05] Loading MOVED for /usr/local/poudriere/data/.m/14-3-R-amd64-default/ref/usr/ports [00:00:06] Gathering ports metadata (...)

Seems to be working properly.

EOF