bhyve as a Hypervisor on FreeBSD - Creating a Virtualization Environment
| Kategorien: | credativ® Inside |
|---|
Creating a Virtualization Environment
Everyone is nowadays talking about containerization but there are still many reasons to run virtual machines and it doesn’t always have to be Proxmox on Linux systems to create a virtualization environment with open-source tools!
Next to the well known Proxmox, there are also other open-source alternatives when it comes to virtualization. A promising solution is also given by bhyve (pronounced bee-hive) which runs on FreeBSD based systems. It was initially written for FreeBSD but now also runs on a number of illumos based distributions like OpenIndiana. bhyve offers a robust and high-performance virtualization solution where it directly operates on the bare metal, utilizing the hardware virtualization features for enhanced performance and isolation between virtual machines. Known for its performance, stability and security, bhyve is integrated into FreeBSD, benefiting from the reliability of the FreeBSD kernel. Of course it also provides the typical feature set such as snapshotting and cloning of VMs. Especially in such cases it benefits from additional features in FreeBSD like the ZFS filesystem. Unfortunately, bhyve does not offer any web-frontend for its administration. This is where bhyve-webadmin (BVCP) steps in to fill the gap. This blog post we will cover the initial setup and features of bhyve and bhyve-admin to provide a fully usable virtualization environment.
Features
bhyve-webadmin (also known as BVCP) provides an API, CLI and a secure web interface for administrating bhyve and virtual machines. bhyve-webadmin mainly features:
- Modularity
- Backend, Frontend, API and helpers are working independently including separate privileges.
- Multi Node
- Multiple server/instances can be orchestrated by a single interface.
- Security
- Native Application
- Roles
- Providing user management including multiple roles.
Requirements
Some general requirements must be fulfilled to run bhyve bundled with bhyve-webadmin:
- System with a minimal FreeBSD image installed.
- 64-bit CPU
- POPCNT (POPulation Count) processor feature
Additional configuration regarding the underlying storage and network configuration are needed but while these are individual for each setup this is not covered in detail within this guide.
Installation
This guide is based on FreeBSD 14 and bhyve-webadmin v1.9.8p9 to provide a virtualization infrastructure. bhyve-webadmin’s concept relies on working as close as possibly to the FreeBSD system and will not change its configuration. Instead, it will install and maintain everything in dedicated directories, configurations and services. As a result, the whole installation including all further dependencies and configuration can be done in minutes.
Overview
File-Structure
By default, bhyve-webadmin will use the following directories for the content:
- /var/lib/nPulse/BVCP: bhyve-webadmin content
- /var/lib/nPulse/BVCP/bvcp.conf: configuration file
- /vms
- /vms/instances
- /vms/iso-images
- /vms/db
- /vms/logs
- /vms/tmp
- /vms/bootrom
Additional helper tools can be found within these directories. For example, a forgotten password can be resetted by running:
- /var/lib/nPulse/BVCP/Backend/vmm reset_password
Services
As already mentioned before, it consists of multiple and independently working software components that can also be orchestrated.
Frontend:
service bvcp-frontend start / stop / restart
Backend:
service bvcp-backend start / stop / restart
Helper:
service bvcp-helper start / stop / restart
Software Installation
cd /tmp/ fetch https://github.com/DaVieS007/bhyve-webadmin/archive/refs/tags/v1.9.8p9.tar.gz tar xfvz v1.9.8p9.tar.gz
sha256sum /tmp/bhyve-webadmin-1.9.8p9.tar.gz $> 758f5900c75a1832c980ed23c74a893f05252aa130e7c803843dac1d2531516f /tmp/bhyve-webadmin-1.9.8p9.tar.gz
cd bhyve-webadmin-1.9.8p9/ ./install.sh
After finishing the installation, all needed directories, configuration and services have been created and the services are already running on the system. On the CLI, the login credentials for the web interface are provided and a login on the web frontend on https://<ip>:8086 is possible. By default, self-signed certificates are generated for the encryption of the web frontend and VNC sessions. The generated self-signed certificates can be later replaced by proper ones (e.g. by using Let’s Encrypt).
Adding ISO-Images
ISO-Images for virtual machines are located in /vms/iso_images. Adding images is simply done by dropping the images within that directory. This can be done on the uploading images via SCP, SFTP or on the system itself by downloading an image from a remote like in the given example:
cd /vms/iso_images/ fetch https://download.freebsd.org/releases/amd64/amd64/ISO-IMAGES/14.0/FreeBSD-14.0-RELEASE-amd64-bootonly.iso
Configuration
An initial first login on the web frontend can now be done. The web frontend can be reached on https://<ip>:8086 and uses self-signed certificates by default. The credentials for the root user login have been printed on the CLI during the setup runtime.
bhyve BVCP Login
After a successful login, the default overview page is being displayed.
After an initial installation it will guide you through the following three next steps which will be covered in detail:
- Configure Network
- Configure Storage
- Create VM
Configure Network
Networking is one of the most important and complex parts when it comes to virtualized environments. Even more complex setups including VLAN separations etc. are possible but not covered within this guide which primarily focuses on a simple bridged– & NAT– networking setup.
Bridged Network
A simple bridged network can be created by clicking on the first Option 1. Configure at least one virtual network where we can create a new network on our cluster. Creating a new network is simple where just the desired network interface on the cluster must be selected.
Within this guide we select the following options which may vary depending on your personal setup:
Network Gateway to bound: (+) [em0] (mtu: 1500) 10.10.10.77 Unique Network Name: uplink-dev Descriptive Text: Uplink network for the dev environments
bhyve BVCP – Create Network
In the given drop-down menu all available network cards will be shown with their interface name and assigned ip address(es) which makes it easy to choose the right one. After saving the configuration, the newly created network is directly usable.
NAT Network
NAT is already considered as a more advanced networking setup but often needed when working with RFC 1918 addresses. NAT is not supported by default and needs to be configured manually. Within this guide a NAT networking will be created for the interface em1 by using PF (a FreeBSD integrated firewall).
Therefore, we enable the PF service in the rc.conf file by running the following command, enabling IP forwarding and directly edit the firewall configuration file:
echo 'pf_enable="YES"' >> /etc/rc.conf echo "net.inet.ip.forwarding=1" >> /etc/sysctl.conf sysctl -f /etc/sysctl.conf vi /etc/pf.conf
The firewall configuration for NAT networking on the em1 interface will be extended by the following content:
nat on em1 from {10.10.10.1/24} to any -> (em1)
Optional: Also port forwarding can be defined in this file which would need to add:
# Forward SSH (tcp/22) to NAT destination host 192.168.1.99 rdr on em1 proto tcp from any to em1 port 22 -> 192.168.1.99 port 22
As a last step, a dummy interface must be created, that be selected in the web frontend service. As a result, this will be added to the /etc/rc.conf file by adding:
cloned_interfaces="nat0" ifconfig_nat0="inet 192.168.1.1 netmask 255.255.255.0 up"
Finally, the required services can be restarted by running:
service netif restart && service routing restart
Configure Storage
In a next step the storage must be configured. Therefore, this can simply be initiated by clicking on 2. Configure at least one data store in the web UI. By default, no one is created and configured:
bhyve BVCP – Storage Overview
By clicking on Create Storage a new storage will be created. Newly created storages must be a mount point.
Create VM
Within the last step, the first virtual machine gets created. This can be finally done by clicking on 3. Create your first Virtual Machine where the options for the VM will be defined.
bhyve BVCP – Create VM
Within this menu, a unique name, a description, the hardware resources like number of cores and memory als the underlying hardware architecture must be defined. After saving, the VM object gets created and we can perform additional tasks like defining an ISO-Image that should be booted in the VM.
bhyve BVCP – Add ISO Image to VM
As already mentioned before in the File Structure chapter, all available ISO-Images must be located in /vms/iso-images. Placed images can be directly selected from the drop-down menu.
Within the last step, the VM can be started by clicking on the play button. Switching back to the Virtual Machine overview (List of Virtual Machines) the VM will be displayed with a green icon as an indication that this virtual machine is up and running.
The VM can now be accessed by the integrated VNC server by clicking on the monitor icon in the middle of the option menu.
Statistics
Performance metrics and statistics of the system usage are an important task. BVCP offers many options to validate the machine health status including performance metrics and statistics of the memory-, CPU-, storage-, networking, etc. usage to quickly detect any negative impacts on an overloaded host node. Beside this, it can also help to evaluate the source of any performance problems in the setup or of a single virtual machine instance.
Conclusion
It doesn’t always have to be Proxmox on Linux systems to create a fast and secure virtualization environment with open-source tools. FreeBSD, bhyve and bhyve-webadmin (BVCP) offer a great bundled solution to run a dedicated virtualization infrastructure which does not need any knowledge in FreeBSD, bhyve or any CLI commands for the end user. Given by the included IAM, users are able to login to a graphical web frontend and to manage their virtual machines on their own.
As a specialists for open-source and open-source infrastructure we are also happy to assist you and your business in BSD based systems including their features like ZFS, PF, Jails, bhyve etc.. Since 1999, credativ® has been recognized for providing 24/7 open source support that rivals manufacturer support. We do not just provide technical support, we also provide all other services across the entire lifecycle of open-source landscapes completed by high flexibility.
| Kategorien: | credativ® Inside |
|---|
über den Autor
gyptazy
Senior DevOps Consultant
zur Person
In my tech repertoire, FreeBSD and federated services stand not just as a personal preference but as a symbol of a commitment to the ethos of open-source. I am a fervent believer in the symbiotic relationship between developers including DevOps, SysOps and the open source community. As I continue to lead teams in the ever-evolving landscape of technology, their dedication to open-source contributions and advocacy for decentralized and federated services echoes not only in the code they write but in the ethos they instill within the teams.