Everything, Everything

2024: J F M A M J J A S O N
2023: J F M A M J J A S O N D
2022: J F M A M J J A S O N D
2021: J F M A M J J A S O N D
2020: J F M A M J J A S O N D
2019: J F M A M J J A S O N D
2018: J F M A M J J A S O N D
2017: J F M A M J J A S O N D
2016: J F M A M J J A S O N D
2015: J F M A M J J A S O N D
2014: J F M A M J J A S O N D
2013: J F M A M J J A S O N D
2012: J F M A M J J A S O N D
2011: J F M A M J J A S O N D
2010: J F M A M J J A S O N D
2009: J F M A M J J A S O N D
2008: J F M A M J J A S O N D
2007: J F M A M J J A S O N D
2006: J F M A M J J A S O N D
2005: J F M A M J J A S O N D
2004: J F M A M J J A S O N D
Hardware Transcoding With Plex
Monday 20th June, 2022 11:38 Comments: 4
Background

Plex can be a bit fiddly to run as a service under Windows. Sure, you can set it up using third party software to run it as a service, but then you can't use your NVIDIA graphics card for hardware accelerated transcoding because reasons. You can configure your Windows system to automatically log in (leaving a cleartext password stored in the registry), launch Plex under your normal user, and immediately lock the system again using a Scheduled Task that kicks in when you log in. The most sensible option is to use Linux, as it has much better support for running as a 24/7 service. In an ideal world, you'll set it up to use Plex's repository, so you can easily update Plex along with your other system packages.

Trying to get this all to work with a virtual machine can be tricky. ESXi might have an easier interface for assigning a graphics card to a virtual machine, but I've not tried it myself. If you have an ESXi host it may be worth a try. But I've traditionally used Hyper-V, which is a lot more fiddly.

Hyper-V

The main issue with Hyper-V is it only supports Discrete Device Assignment on Windows Server, and other techniques such as RemoteFX have been disabled for the last year due to security concerns.

Even if you have Windows Server and can get GPU passthrough enabled using DDA, you have to use PowerShell to identify information to disable and configure the graphics card to pass it through to the VM. It's all a bit fiddly, although there is some guidance on Reddit:
  1. Find the device id of the device you want to have mapped to your VM (Get-PnpDevice)
  2. Disable it on the host (Disable-PnpDevice)
  3. Dismount it from the host, to make it available for pass-through (Dismount-VmHostAssignableDevice)
  4. Mount it to the vm (Add-VMAssignableDevice)
To move a device back to the host:
  1. Dismount it from the VM (assuming you know its ID) (Remove-VMAssignableDevice)
  2. Mount it to the host, making it unavailable for pass-through (Mount-VMHostAssignableDevice)
  3. Enable it on the host (Enable-PnpDevice)
In hindsight this route might have been easier! But instead of messing about with VMs, I decided to re-use some old PC components.

Bare Metal

I have a very old HTPC that used to work fine (or so I thought?). It was missing a graphics card and hard disk, but the CPU and RAM were present. Admittedly it's only got 4GB of RAM, I would have liked more but I don't seem to have any DDR2 sticks hanging around. Yes, the motherboard is that old. From memory, the CPU is also only dual core with hyper-threading (it's only an Intel E8400 Core 2 Duo with 2 cores and 2 threads running at 3.0GHz). But you only really need CPU power for transcoding if you're not doing hardware accelerated encoding; streaming files using Direct Play puts little burden on the CPU. It will probably mean slower detection of intros etc. when you update your library, but I didn't want to spend money on more components for what is essentially a proof of concept.

Installing Ubuntu

Install Ubuntu 22.04 LTS Server from the latest ISO image. Despite being the latest, you'll get the chance to download an updated installer during the setup. The hardest step for me with my old hardware was getting the system to boot. The motherboard is so old it doesn't support booting off a USB pendrive, so I had to dig out an old USB DVD rewriter, burn the ISO to a decade old DVD-RW that I had to dig out of a drawer, then very slowly boot off the DVD. This is even more annoying when you go through it more than once (the SATA cable I re-used from the system appears to be f**ked - I ended up installing onto an even older 80GB SSD before working out it was the cable at fault, and then installing it again on the slightly newer 240GB SSD).
  1. Select an appropriate language for the keyboard.
  2. Select the Ubuntu Server (minimized) install, as no one will be logging into the host.
  3. I left the host using DHCP, as my router will assign a fixed IP address. I used a local mirror, and didn't need to use a proxy.
  4. I also used the entire disk, setup as an LVM group. Feel free to encrypt it with LUKS if you want, but every time you reboot the server you'll need to enter a passphrase.
  5. Create a suitable hostname and admin user (plex will create its own plex user)
  6. Install OpenSSH server so you can remotely log in (I'm using a passphrase to begin with, but feel free to import your SSH key at this point).
  7. Do not install the nvidia-driver-510-server third-party drivers when offered. I'm not 100% sure how Ubuntu will do this, but it'll possibly configure a repo and use nouveau. If you want to patch the drivers (see below) you really don't want them to be updated automatically.
  8. Do not add any popular snaps!
Once you're done (this can also take a while), you can SSH into the host for easier post-configuration (as there's a lot of copying and pasting), and run the server headless from now on (although you will need to install a desktop later on because of NVIDIA driver weirdness affecting fan control).

And because I'm massively reckless not going to screw anything up, I'm going to escalate to root using "sudo su" to run all of the commands below without having to type "sudo" all of the time.

NVIDIA Drivers

In order to compile stuff when installing the NVIDIA drivers, make sure you have some build tools. It is recommended by nvidia-patch that you install dkms.
apt-get install dkms
However, I've also read that you can get away with just these, which I used instead on my last run:
apt-get install build-essential libglvnd-dev pkg-config
One of the reasons for doing all of this was to re-use a spare NVIDIA GeForce GTX 1060, as it supports NVENC. Using a patch, you can encode multiple streams at the same time (I think this card should be limited to 1, with higher end cards like the 1080 limited to 3?). Use the latest driver (using "Driver link") and patch listed at https://github.com/keylase/nvidia-patch. For example:
mkdir /opt/nvidia && cd /opt/nvidia
wget http://international.download.nvidia.com/XFree86/Linux-x86_64/515.48.07/NVIDIA-Linux-x86_64-515.48.07.run
chmod +x ./NVIDIA-Linux-x86_64-515.48.07.run
./NVIDIA-Linux-x86_64-515.48.07.run
Ignore the Ubuntu warning and say yes to everything (e.g. Continue installation), and wait for the NVIDIA installer to complain about nouveau and let it add a blacklist (I know, I know) for you. After the installer has exited, you'll need to update-initramfs, and then reboot the server:
update-initramfs -u
reboot
Once we're back up and running, SSH in (and "sudo su" if feeling lazy) and do it all over again:
cd /opt/nvidia
./NVIDIA-Linux-x86_64-515.48.07.run
Say yes to everything (Continue installation, register as a DKMS kernel module, ignore the X library path warning, install 32-bit compatibility for good measure, let it pointless run the nvidia-xconfig utility just in case you decide to run an X Window System in the future). You should install it as a DKMS module so when you update the Linux kernel the driver will still load/work. Building the module does take a while!

Although you could download and extract the code, it doesn't hurt to have a git client on your system:
apt-get install git
Then grab the nvidia patch, and browse to the new folder (you can always change the name using your git command by adding a folder name). I'm doing all of this within /opt/nvidia:
git clone https://github.com/keylase/nvidia-patch.git
cd ./nvidia-patch
Check that your NVIDIA driver is working okay, then install both patches:
nvidia-smi
bash ./patch.sh
bash ./patch-fbc.sh
reboot
Once this is all done, and after you've rebooted, you might see the fans on your GPU stop spinning. In order to control things like the fan curve, you basically need to install a desktop (or temporarily run X using scripts to stop the fans/control the curve?), which is annoying if your GPU is a bit noisy. At 30% when idle, my fans aren't too loud with the case off, but I'd prefer to have the fans off when it's idle. There does seem to be a possible bug/issue that means cards aren't dropping below 30% for some reason. It sounds like versions before 465.31 may support 0% when idle, so you may want to use https://international.download.nvidia.com/XFree86/Linux-x86_64/465.31/NVIDIA-Linux-x86_64-465.31.run instead, but I've also been unable to install that version as a DKMS module (and I'd quite like to upgrade my kernel when necessary). The older version Ubuntu packages are "transitional" packages that appear to install 4.70 (which is too new), plus the patch may not work on those binaries (not something I've checked though). The good news is it's possible to "apt-get install ubuntu desktop" to allow you to use or set automatic fan control, which will allow 0 RPM when cool on my GPU. If you're coming from Ubuntu Server this is a somewhat time-consuming 2GB download that needs to be unpacked. It may be possible to just do "apt-get install gnome-session gdm3" instead, but you can definitely get away with "ubuntu-desktop-minimal" to avoid the full "ubuntu-desktop" installation (which includes productivity apps). With "ubuntu-desktop-minimal" installed this did appear to make the fans stop once it had started to display GNOME.
apt-get install ubuntu-desktop-minimal
One thing I noticed in the next step is that some opencl packages were recommended and installed afterwards, so you may want to do that before doing the reboot above.
apt-get update
apt-get upgrade -y
Installing Plex

Now that the hardware is hopefully all ready (including the fans not spinning when idle), we just need to install Plex in a way that will keep itself up to date. Doing it this way will cause a deprecation warning due to how we add the PGP key.
curl https://downloads.plex.tv/plex-keys/PlexSign.key | sudo apt-key add -
echo deb https://downloads.plex.tv/repo/deb public main | sudo tee /etc/apt/sources.list.d/plexmediaserver.list
apt update
apt-get install plexmediaserver
To avoid that legacy PGP key issue ("Key is stored in legacy trusted.gpg keyring"), this bash script is suggested on the Plex forums:
 # If Ubuntu 20.04+ or Debian 10+, switch from using apt-key to gpg trusted keyring
  DistroName="$(grep ^ID= /etc/os-release | sed -e 's/^.*=//' )"
  DistroVersion="$(grep ^VERSION_ID= /etc/os-release | sed -e 's/^.*=//' | tr -d '".')"
  HostArch="$(dpkg --print-architecture)"

  UseGpg=0
  [ $DistroName = "ubuntu" ] && [ $DistroVersion -gt 2000 ] && UseGpg=1
  [ $DistroName = "debian" ] && [ $DistroVersion -ge 1000 ] && UseGpg=1

  if [ $UseGpg -gt 0 ]; then
    wget -O - https://downloads.plex.tv/plex-keys/PlexSign.key | gpg --dearmor -o /usr/share/keyrings/plexmediaserver.gpg
    sed -i -e "s+deb https://downloads.plex.tv+deb [arch=$HostArch signed-by=/usr/share/keyrings/plexmediaserver.gpg] https://downloads.plex.tv+" \
        /etc/apt/sources.list.d/plexmediaserver.list
  fi
If you want updates to be installed regularly, you may want to add a cron job. Feel free to install/use another text editor. For example, to do this daily:
apt-get install vim
vim /etc/cron.daily/aptgetupgrade
And enter suitable contents such as:
#!/bin/bash
apt-get update
apt-get upgrade -y
apt-get autoclean
You might want to autoremove things too? Adding "-y" to upgrade will avoid being prompted to install updates, and as this will be executed headless you won't get prompted to press Q to close reading the notes.

Once Plex is installed, you can add your libraries. My content is stored on a separate file server. To access the files over SMB this Ubuntu install will need cifs-utils to be installed. You'll also need to edit /etc/fstab and create a credentials file in /root/ which I prefer to do over SSH using vim.
apt-get install cifs-utils
Once that's done, I created a credentials file in root's home directory containing the cleartext credentials. Not great, but it works. The share is read-only, and designed for people to access over the internal network, so I'm not too fussed about protecting these low-privileged credentials. If you're providing write access (so you can delete files via Plex's web interface) or the remote user is an administrator then consider doing something different (e.g. don't use an admin account!). Replace USERNAME and PASSWORD with valid creds. For example, you could put your SMB credentials in the file .smbcredentials:
mkdir /mnt/smb
vim /root/.smbcredentials
username=USERNAME
password=PASSWORD
You'll need to remove world/group access to keep fstab happy, otherwise it won't mount the share on boot.
chmod 600 /root/.smbcredentials
vim /etc/fstab
Add a suitable line, replacing the IP and share name with whatever you're using, such as:
//192.168.0.2/Share   /mnt/smb        cifs    credentials=/root/.smbcredentials,noexec        0       0
Before you sign in to Plex, you may want to add a redirected port on your router. Plex uses 34200 by default. Then log into the server at it's IP on port 32400. For example:

http://192.168.0.3:32400/web

Ubuntu Server doesn't install a firewall, so feel free to install something like ufw and add exceptions for TCP ports 22 (for SSH) and 32400 (for Plex) before enabling it. It'll probably look a little something like this:
apt-get install ufw
ufw allow 22/tcp
ufw allow 32400/tcp
ufw enable
After you've created your library, you'll hopefully see that you can transcode 4K videos to 1080p without maxing out the CPU. The nvidia-smi command can be used to monitor load on the GPU.

With the patch installed, I was able to play 4K HDR streams in a browser and on my phone at the same time, despite both being transcoded by the GPU.

If I decide to use this long term (the PoC system's 4GB really wasn't enough for running a Plex server as it kept hitting "out of memory"), I'll need to manually update the drivers and patch, or work out a nice way to automate it.
Avatar Robert - Wednesday 27th July, 2022 11:59
The PoC system worked well enough that I have since replaced the motherboard/CPU/RAM/SSD with new components and did a complete reinstall.
Avatar Robert - Wednesday 7th September, 2022 17:25
I manually installed an older version of the NVIDIA driver recently as the newer one seemed to die after a while (nvidia-smi wouldn't show anything). I'll try a newer version again sometime, but for now I might stick with the known-good older driver (until I'm aware of any security issues). If you have any issues after patching the driver, try an older version.
Avatar Robert - Tuesday 13th December, 2022 10:56
I think I must have installed an updated driver without enabling DKMS. You can apparently enable DKMS using the flag "--dkms" when running the installer, otherwise make sure you select it within the installer (the default is "no"). Without DKMS, upgrading the Linux kernel will break the driver.

If it still doesn't work for some reason, forum posts suggest running: sudo apt -y install linux-headers-$(uname -r)
Avatar Robert - Wednesday 31st January, 2024 23:17
I think I'm going to give up on the patched driver. NVIDIA increased the number of concurrent streams from 3 to 5 last year, and with their latest driver they've apparently increased it again to 8.

https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new

The latest Ubuntu meta package should be easier to install and maintain automatically. I imagine the new 550 driver will get added to their repository fairly soon, not that I've needed more than 3 concurrent streams to begin with. Even 5 is plenty.
© Robert Nicholls 2002-2024
The views and opinions expressed on this site do not represent the views of my employer.
HTML5 / CSS3