Plex in a VM on ProxmoxVE
In order to create a VM with Ansible, I first need to create an API user for Ansible to be able to interact with ProxmoxVE. Once I have a VM, I will install Plex Media Server on it.
ProxmoxVE users
I started by adding default (blank) lists for users and groups to the proxmox_virtual_environment role’s defaults/main.yaml:
pve_groups: []
pve_users: []
Creating the groups and users, in an idempotent way, requires querying the current list then adding the ones that do not exist. I did not remove ones that are not in Ansible’s list to allow for groups and users managed outside of its control (and I don’t have to worry about builtin root user this way):
- name: Proxmox groups exist
# Cluster wide configuration
run_once: true
block:
- name: Current groups are known
become: true
ansible.builtin.command: /usr/sbin/pveum group list --output-format=json
changed_when: false # Always read-only
register: pve_group_list_out
- name: Group exists
become: true
ansible.builtin.command: "/usr/sbin/pveum group add {{ item.name | quote }}{% if 'comment' in item %} --comment={{ item.comment | quote }}{% endif %}"
loop: '{{ pve_groups }}'
when: item.name not in pve_group_list_out.stdout | from_json | map(attribute='groupid')
- name: Proxmox users exist
# Cluster wide configuration
run_once: true
block:
- name: Current users are known
become: true
ansible.builtin.command: /usr/sbin/pveum user list --output-format=json
changed_when: false # Always read-only
register: pve_user_list_output
- name: Required users exist
become: true
ansible.builtin.command: >-
/usr/sbin/pveum
user add {{ pve_user_name | quote }}
{% if 'groups' in item %}
--groups {{ item.groups | join(',') | quote }}
{% endif %}
{% if 'password' in item %}
--password {{ item.password | quote }}
{% endif %}
loop: '{{ pve_users }}'
loop_control:
label: '{{ pve_user_name }}'
vars:
pve_user_name: '{{ item.name }}@{{ item.realm }}'
when: >-
pve_user_list_output.stdout
| from_json
| selectattr('userid', 'eq', pve_user_name)
| length
== 0
no_log: true # Don't log passwords
- name: Group memberships are correct
become: true
ansible.builtin.command: >-
/usr/sbin/pveum
user modify {{ pve_user_name | quote }}
--groups {{ item.groups | join(',') | quote }}
loop: '{{ pve_users }}'
loop_control:
label: '{{ pve_user_name }}'
vars:
pve_user_name: '{{ item.name }}@{{ item.realm }}'
when: >-
pve_user_list_output.stdout
| from_json
| selectattr('userid', 'eq', pve_user_name)
| map(attribute='groups')
| map('split', ',')
| flatten
| ansible.builtin.symmetric_difference(item.groups)
| length
!= 0
For API users, I also decided to use token access so need to create those as well (added to the user block, above):
- name: Ansible user tokens are known
become: true
ansible.builtin.command: "/usr/sbin/pveum user token list {{ pve_user_name }} --output-format=json"
changed_when: false # Always read-only
register: pve_user_token_list_output
loop: '{{ pve_users }}'
vars:
pve_user_name: '{{ item.name }}@{{ item.realm }}'
loop_control:
label: '{{ pve_user_name }}'
- name: Complete list of current user tokens is initialised (even if empty)
ansible.builtin.set_fact:
pve_current_user_tokens: []
- name: Complete list of current user tokens is known
ansible.builtin.set_fact:
pve_current_user_tokens: >-
{{
pve_current_user_tokens
+
[[item.item.name, item.item.realm]] | product(item.stdout | from_json)
}}
loop: '{{ pve_user_token_list_output.results }}'
- name: Incorrect Tokens are removed
become: true
ansible.builtin.command: '/usr/sbin/pveum user token remove {{ pve_user_name | quote }} {{ item[1].tokenid | quote }}'
loop: '{{ pve_current_user_tokens }}'
vars:
pve_user_name: '{{ item[0][0] }}@{{ item[0][1] }}'
when: >-
pve_users
| selectattr('name', 'eq', item[0][0])
| selectattr('realm', 'eq', item[0][1])
| selectattr('api_tokens', 'defined')
| map(attribute='api_tokens')
| flatten
| selectattr('name', 'eq', item[1].tokenid)
| length
== 0
- name: Complete list of user tokens required is initialised (even if empty)
ansible.builtin.set_fact:
pve_user_tokens: []
- name: Complete list of user tokens required is known
ansible.builtin.set_fact:
pve_user_tokens: >-
{{
pve_user_tokens
+
[[item.name, item.realm]] | product(item.api_tokens | default([]))
}}
loop: '{{ pve_users }}'
loop_control:
label: '{{ item.name }}@{{ item.realm }}'
- name: User tokens exist
become: true
ansible.builtin.command: /usr/sbin/pveum user token add {{ pve_user_name | quote }} {{ item[1].name }} --output-format=json
loop: '{{ pve_user_tokens }}'
vars:
pve_user_name: '{{ item[0][0] }}@{{ item[0][1] }}'
when: >-
pve_current_user_tokens
| selectattr('0', 'eq', item[0])
| selectattr('1.tokenid', 'eq', item[1].name)
| length
== 0
register: pve_user_token_add_output
- name: New token secrets are known as facts
# Based on https://stackoverflow.com/questions/38143647/set-fact-with-dynamic-key-name-in-ansible
ansible.builtin.set_fact:
"pve_user_{{ item.item[0][0] }}_{{ item.item[0][1] }}_token_{{ item.item.1.name }}": "{{ (item.stdout | from_json).value }}"
loop: "{{ pve_user_token_add_output.results | selectattr('changed', 'eq', true) }}"
Authorisation
ProxmoxVE uses role and path based authorisation. For user permissions, I used groups with roles assigned to them and use group membership to delegate those rights to individual users. I did, however, assign roles directly to tokens to limit them to just the permissions they required.
It is important to note that users must have the same permissions (which can be via group membership), when assigning permissions directly to tokens:
Additionally, privilege separated tokens can never have permissions on any given path that their associated user does not have.
For now, I stuck to the built-in roles so just need to be able to add ACLs to groups and tokens by adding entries for them. I first added ACLs for the groups (to the group create block):
- name: Complete list of group ACLs is known
ansible.builtin.set_fact:
pve_group_acls: >-
{{
pve_group_acls
+
[item.name] | product(item.acls)
}}
loop: '{{ pve_groups }}'
- name: Current ACLs are known
become: true
ansible.builtin.command: /usr/sbin/pveum acl list --output-format=json
register: pve_acl_list_output
changed_when: false # Read only command
- name: Incorrect ACLs are removed
become: true
ansible.builtin.command: '/usr/sbin/pveum acl delete {{ item.path | quote }} --role {{ item.roleid | quote }} --group {{ item.ugid | quote }}'
loop: >-
{{
pve_acl_list_output.stdout
| from_json
| selectattr('type', 'eq', 'group')
}}
when: >-
pve_groups
| selectattr('name', 'eq', item.ugid)
| map(attribute='acls')
| flatten
| selectattr('path', 'eq', item.path)
| selectattr('role', 'eq', item.roleid)
| length
== 0
- name: ACLs are set
become: true
ansible.builtin.command: /usr/sbin/pveum acl modify {{ item[1].path | quote }} -group {{ item[0] | quote }} -role {{ item[1].role | quote }}
loop: '{{ pve_group_acls }}'
when: >-
pve_acl_list_output.stdout
| from_json
| selectattr('type', 'eq', 'group')
| selectattr('ugid', 'eq', item[0])
| selectattr('path', 'eq', item[1].path)
| selectattr('roleid', 'eq', item[1].role)
| length
== 0
And then tokens (to the user creation block):
- name: Required token ACLs are initialised (even if none required)
ansible.builtin.set_fact:
pve_user_token_acls: []
- name: Required token ACLs are known
ansible.builtin.set_fact:
pve_user_token_acls: >-
{{
pve_user_token_acls
+
[item[0] + [item[1].name]] | product(item[1].acls | default([]))
}}
loop: '{{ pve_user_tokens }}'
loop_control:
label: '{{ item[0][0] }}@{{ item[0][1] }}'
# Refresh ACLs as may have been changed by deleting redundant
# tokens (which will automatically remove related ACLs).
- name: Current ACLs are known
become: true
ansible.builtin.command: /usr/sbin/pveum acl list --output-format=json
register: pve_acl_list_output
changed_when: false # Read only command
- name: Incorrect ACLs are removed
become: true
ansible.builtin.command: '/usr/sbin/pveum acl delete {{ item.path | quote }} --role {{ item.roleid | quote }} --token {{ item.ugid | quote }}'
loop: >-
{{
pve_acl_list_output.stdout
| from_json
| selectattr('type', 'eq', 'token')
}}
vars:
pve_token_tuple: "{{ item.ugid.split('@') | map('split', '!') | flatten }}"
when: >-
pve_user_token_acls
| selectattr('0', 'eq', pve_token_tuple)
| selectattr('1.path', 'eq', item.path)
| selectattr('1.role', 'eq', item.roleid)
| length
== 0
- name: ACLs are set
become: true
ansible.builtin.command: /usr/sbin/pveum acl modify {{ item[1].path | quote }} -token {{ token | quote }} -role {{ item[1].role | quote }}
loop: '{{ pve_user_token_acls }}'
vars:
token: '{{ item[0][0] }}@{{ item[0][1] }}!{{ item[0][2] }}'
when: >-
pve_acl_list_output.stdout
| from_json
| selectattr('type', 'eq', 'token')
| selectattr('ugid', 'eq', token)
| selectattr('path', 'eq', item[1].path)
| selectattr('roleid', 'eq', item[1].role)
| length
== 0
Tidying up
Finally, the additional options are added to the proxmox_virtual_environment role’s meta/argument_specs.yaml:
pve_groups:
description: List of groups to create in PVE
type: list
elements: dict
default: []
options:
name:
description: Name of the group
required: true
type: str
comments:
description: Comment for the group
required: false
type: str
acls:
description: ACLs for this group
default: []
type: list
elements: dict
options:
path:
description: Path for the ACL
type: str
required: true
role:
description: Role the group will have on this path
type: str
required: true
pve_users:
description: List of users to create in PVE
type: list
elements: dict
default: []
options:
name:
description: Name of the user
type: str
required: true
realm:
description: Name of PVE role for the user
type: str
required: true
password:
description: Initial password for the user
type: str
required: false
groups:
description: List of groups this user should be in
type: list
required: false
elements: str
api_tokens:
description: |
List of API tokens that should exist for this user.
New tokens will have the secret stored in a fact named:
pve_user_{ user name }_{ realm }_token_{ token name }
type: list
required: false
elements: dict
options:
name:
description: Name for the token (also known as token id)
type: str
required: true
acls:
description: ACLs for this token
default: []
type: list
elements: dict
options:
path:
description: Path for the ACL
type: str
required: true
role:
description: Role the token will have on this path
type: str
required: true
I added groups and then users for myself and Ansible to the proxmox_virtual_enviornment_hosts group’s group_vars file:
pve_groups:
- name: administrators
comment: "System Administrators"
acls:
- path: /
role: Administrator
- name: vm_admins
comment: "VM Administrators"
acls:
- path: /vms
role: PVEVMAdmin
# Required to use the network
- path: /sdn/zones/localnetwork/vmbr0
role: PVESDNUser
# Required to be able to allocated new disks on VM creation
- path: /storage/ceph-vm
role: PVEDatastoreAdmin
pve_users:
- name: ansible
realm: pve
groups:
- vm_admins
api_tokens:
- name: vm_creation
# Additionally, privilege separated tokens can never have
# permissions on any given path that their associated user
# does not have.
acls:
- path: /vms
role: PVEVMAdmin
- path: /sdn/zones/localnetwork/vmbr0
role: PVESDNUser
# Required to be able to allocated new disks on VM creation
- path: /storage/ceph-vm
role: PVEDatastoreAdmin
- name: my_user
realm: pve
password: >-
{{
lookup(
'community.hashi_vault.vault_read',
'kv/proxmoxve/users/my_user'
).data.password
}}
groups:
- administrators
I manually generated and stored a password for my user in the vault.
Finally, I added code to store the ansible user’s new token secret in the vault, when it is created:
- name: Ansible user's token is stored in vault
run_once: true # Only need to do this once
delegate_to: localhost
community.hashi_vault.vault_write:
path: kv/proxmoxve/users/ansible/tokens/vm_creation
data:
token: '{{ pve_user_ansible_pve_token_vm_creation }}'
when: pve_user_ansible_pve_token_vm_creation is defined
Building a Proxmox VM with Ansible
My initial thought was to use Terraform to deploy virtual machines (VMs) and Ansible to configure the deployed VM, as I did before with Azure. However, some cursory research online suggested that Terraform’s Proxmox providers are not good so Ansible is what most people use.
For Ansible to communicate with Proxmox, I had to add proxmoxer Python module. Added it to the requirements.txt and used pip to install it in my Ansible virtual environment:
# For Proxmox API (community.general.proxmox*)
proxmoxer
I added my new VM, holoship, to inventory (once the VM existed, I also added the MAC address to its interfaces variable manually looked up from Proxmox):
holoship:
proxmox_vm:
disk_size_gb: 30
cores: 2
memory_mb: 4096
vlan: 20
There is a Proxmox inventory source for Ansible, I have not yet thought about whether I want to use this directly.
I added a group for Proxmox VMs, which will be how I target which hosts are to be created as VMs in Proxmox:
proxmox_vms:
hosts:
holoship:
For the actual creation, I created a new create-vms.yaml playbook. I empirically discovered the settings required to make this match one I created interactively in the Proxmox UI. For now, I use a specific host as the API target (this will eventually be a ha-proxy front end to the entire cluster, I hope). I also hardcoded the host the VM will be created on - it can be migrated but this could be more intelligent, perhaps working the least populated host out from querying the cluster?:
- hosts: proxmox_vms
gather_facts: false # Don't need them
tasks:
- name: Create Proxmox VM
delegate_to: localhost
community.general.proxmox_kvm:
api_user: "ansible@pve"
api_token_id: vm_creation
api_token_secret: >-
{{
lookup(
'community.hashi_vault.vault_read',
'kv/proxmoxve/users/ansible/tokens/vm_creation'
).data.token
}}
# XXX Will be haproxy front-end, when I set one up
api_host: "pve01"
cpu: x86-64-v2-AES
bios: ovmf
cores: '{{ proxmox_vm.cores | default(1) }}'
efidisk0:
efitype: 4m
format: raw
# Setting this to true implicitly enables secure boot
pre_enrolled_keys: true
storage: ceph-vm
ide:
ide2: none,media=cdrom
memory: '{{ proxmox_vm.memory_mb | default(2048) }}'
name: '{{ inventory_hostname }}'
net:
net0: virtio,bridge=vmbr0,firewall=1{% if 'vlan' in proxmox_vm %},tag={{ proxmox_vm.vlan }}{% endif %}
# XXX Hardcoded to create VMs on a single host
node: "pve01"
onboot: true
ostype: l26
scsi:
scsi0: ceph-vm:{{ proxmox_vm.disk_size_gb | default(20) }},format=raw,iothread=1
scsihw: virtio-scsi-single
state: present
storage: ceph-vm # Storage pool for the disk
tpmstate0:
storage: ceph-vm
Installing Plex on the VM
Once the VM was installed, I had to disable secure boot to enable it to PXE boot from my existing network (working on finding a secure-boot compatible alternative) then selected the pre-seeded Debian install from the menu. I then ran my existing install playbook against the new VM, which set it up including remotely-unlockable encryption, creating the ansible user and securing the root account, both with machine-specific random passwords.
I created a new group called plex_media_servers and added the new VM to it:
plex_media_servers:
hosts:
holoship:
In my site.yaml, I applied a new role (which I am about to create) called plex-media-server to this group:
- name: Plex media servers are setup
hosts: plex_media_servers
roles:
- plex-media-server
Mounting the NAS storage
My media is stored on a NAS - I used to run Plex directly on the NAS but it kept running out of memory corrupting the NAS’s underlying BTRFS requiring a complete rebuild and restore from backup. Although I am moving Plex onto a different server, the media files are still on the NAS and need to be mounted for Plex to use.
I already had a task to add arbitrary custom mounts to hosts with Ansible, however it does not create the groups (specifically the store group I use for the NAS mounts) embedded in the mount options, so I added the creation of extra groups to the Linux.yaml task file in my existing common role:
- name: Additional groups exist
become: true
ansible.builtin.group:
name: '{{ item[6:] }}'
loop: "{{ q('ansible.builtin.varnames', '^group_.+') }}"
# Skip the built in special variable
when: item != 'group_names'
My common role is one of the first ones I write, and could do with some updating - for example, it was written when I was using my own account rather than a dedicated ansible account for Ansible to login and so copied my preferred .vimrc file to the home directory of whichever user Ansible logs in as, which is no longer ideal.
In my plex_media_server’s group_vars file I added an entry for the group, copied the mounts from another host and adjusted the /var volume to be larger (I discovered after building it, with the default size that Plex would run out of disk space on this filesystem while transcoding):
---
# (various mount_ variables, one for each the NAS shares)
group_store:
# Plex needs a large /var for its database and transcoding cache
filesystems_lvm_volume_groups:
- name: vg_
logical_volumes:
- name: var
size: 15G
...
plex-media-server role
Firstly, the Plex package repositories need to be available so I added them to the new role’s meta/main.yaml:
---
dependencies:
- role: apt-source
vars:
name: plexmediaserver
uri: "{{ local_mirror['plex'].uri | default('https://downloads.plex.tv/') }}repo/deb/"
gpg_key:
url: "{{ local_mirror['plex'].uri | default('https://downloads.plex.tv/') }}plex-keys/PlexSign.key"
suite: public
components:
- main
src:
no_src: yes
when: ansible_facts['os_family'] == 'Debian'
...
Installing Plex
The plex-media-servers role’s tasks/main.yaml begins with a simple package install:
- name: Handlers are flushed (so any new repos are synced)
meta: flush_handlers
- name: Plex is installed
become: true
ansible.builtin.package:
name: plexmediaserver
state: present
I then added the plex user (created by the package) to the store group (I mentioned above) - as noted in the comment, this does not belong in the role as it is site-specific but I have not yet figured out the best way to sort this sort of thing (in some cases, users will need to be in groups for Ansible to do things and others, like this, Ansible will need to do something for the group to exist):
# XXX this doesn't belong here - site specific - how to do this "right"?
- name: Plex user is in store group
become: true
ansible.builtin.user:
append: true # Adding to group, not removing existing groups
name: plex
groups:
- store
The service needs to be running:
- name: Plex server is running
become: true
ansible.builtin.service:
name: plexmediaserver
enabled: true
state: started
and the firewall configured to allow access:
# plex service come with firewalld
# XXX Want to make this better (properly zoned, not just opened to everything via default zone)
- name: firewalld is configured to allow connecting to plex
block:
- name: firewalld role is included (for handler)
ansible.builtin.include_role:
name: firewalld
- name: Plex service is permitted
become: yes
ansible.posix.firewalld:
service: plex
permanent: true
state: enabled
notify: reload firewalld
Claiming the install
Finally, the install can be registered interactively (by navigating to the service’s web UI) or with a claim code - whether the install is registered or not can be determined by checking for the presence of PlexOnlineHome property on the Preferences tag in the Preferences.xml file:
- name: Plex preference file is read
become: true
ansible.builtin.slurp:
src: /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Preferences.xml
register: plex_preferences_xml
- name: Plex preferences is converted to Ansible data structure
ansible.builtin.set_fact:
plex_preferences: >-
{{
plex_preferences_xml.content
| ansible.builtin.b64decode
| ansible.utils.from_xml
}}
#- ansible.builtin.debug: var=plex_preferences
- name: Plex server is registered
block:
- name: Claim key is set
ansible.builtin.assert:
that: plex_claim_key is defined
msg: plex_claim_key required to register system
- name: Register server
ansible.builtin.uri:
method: POST
url: http://127.0.0.1:32400/myplex/claim?token={{ plex_claim_key }}
return_content: true
register: plex_claim_output
- ansible.builtin.debug: var=plex_claim_output
when: "'@PlexOnlineHome' not in plex_preferences.Preferences"
The plex_claim_key option I added to the role’s meta/argument_specs.yaml file:
---
argument_specs:
main:
short_description: Installs and configures Plex Media Server
author: Laurence Alexander Hurst
options:
plex_claim_key:
description: Claim key to join Plex media server to account
type: str
# As they are only valid for a short time (5 minutes), will
# only want to supply one when joining system initially.
required: false
...
Post install configuration
I did consider scripting, e.g., adding the “Libraries” (media stores) to Plex, however I encountered a Catch-22 - according the Plex documentation finding a X-Plex-Token to interact with the API requires looking at a media file, which one cannot do if Plex cannot yet see any media files… So I just did it by hand.