Grab Kernel
Get the one you desire from https://www.kernel.org/

wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.10.65.tar.xz
tar xvf linux-5.10.65.tar.xz
cd linux-5.10.65

Grab Packages

apt-get install bc libncurses-dev bzip2 make build-essential libelf-dev libssl-dev bison flex rsync liblz4-tool

Run localmodconfig
Hold enter until you are done

make localmodconfig

Customization of the Kernel

make menuconfig

Network
If you want wireguard or anything else

Device Drivers => Network device support => VXLAN, Wireguard

BBR increases TCP Network throughput

Network Support => Network options => Network packet filtering framework (Netfilter) (e), QoS
Network Support => Network options => TCP: advance congestion control => BBR
Network Support => Network options => QoS => Qdisc, Fair Queuing

dhcpclient dosen't work without it

Network Support => Network options => Packet Socket

Latency reduction

General setup => Timers subsystem => High Resolution Timer Support, Timer tick handling
General setup => Preemption Model
Processor type and features => Timer frequency
Power management and ACPI options => CPU Frequency scaling => governor

virtio
If your virtual server uses scsi virtio you need to enable this

Device drivers => SCSI device support => disk support
Device drivers => SCSI device support => SCSI low-level drivers => virtio-scsi support

Build the Kernel

make -j8 deb-pkg LOCALVERSION=-rt KDEB_PKGVERSION=1