Up until now I have been managing APT repositories individually as individual states in SaltStack (there is an example in one of my previous posts). As the number of 3rd party repositories and combinations of components (main, contrib and non-free in the Debian default repositories) has grown this has become a little unwieldy and pushes some specific configuration into the state that could be moved to pillar.

The state

To make it a bit more manageable, I combined the individual states into a single state (linux/apt/repos.sls, in my states tree) that manages each repository in a file in /etc/apt/sources.list.d, deleting any that should not be there (so repositories can be removed by removing their configuration from a server):

# file.managed turned out to be much more reliable than the obvious
# pkgrepo.managed at configuring these correctly (without duplication
# and removing incorrect entries)

{% for (repo, data) in salt['pillar.get']('apt:repos', {}).items() %}
apt-source-{{ repo }}:
  file.managed:
    - name: /etc/apt/sources.list.d/{{ repo }}.list
    - contents: |
        deb {% if data.get('options') %}[ {% for (key, value) in data.options %}{{ key }}={{ value }} {% endfor %}] {% endif %}{{ data.uri }} {{ data.suite }} {{ ' '.join(data.get('components',{}).key
s()) }}
        {% if not data.get('no-source', False) %}deb-src {% if data.get('options') %}[ {% for (key, value) in data.options %}{{ key }}={{ value }} {% endfor %}] {% endif %}{{ data.uri }} {{ data.suite
 }} {{ ' '.join(data.get('components',{}).keys()) }}{% endif %}
    - owner: root
    - group: root
    - mode: 0o444
{% endfor %}

# Remove unmanaged sources
{# [2:] strips off '.' and '..' #}
{% for file in salt['file.readdir']('/etc/apt/sources.list.d/')[2:] %}
{% if '.'.join(file.split('.')[:-1]) not in salt['pillar.get']('apt:repos', {}).keys() %}
remove-apt-source-{{ file }}:
  file.absent:
    - name: /etc/apt/sources.list.d/{{ file }}
{% endif %}
{% endfor %}

{# As a safety net, only remove core repos file if there are repos to push out #}
{% if salt['pillar.get']('apt:repos') %}
# Remove now redundant core file
apt-default-sources:
  file.absent:
    - name: /etc/apt/sources.list
{% endif %}

The pillar

To generate lines in this form (from the source.list man page) in /etc/apt/sources.list.d/<name>.list:

deb [ option1=value1 option2=value2 ] uri suite [component1] [componenet2] [...]

The state above expects repos that look like this from pillar:

apt:
  repos:
    <name>:
      options:
        <key>: <value>
        ...
      uri: <uri>
      suite: <suite>
      no-source: False
      components:
        <component1>: True
        <component2>: True
        ...

no-source (defaults to False), options and components are optional. Like for my roles, components need to be a dict (instead of a list) to merge correctly when stacked. When done this way, however, contrib and non-free can be enabled (for example) via a pillar file that stacks with the base configuration, without duplicating the rest of the data (uri etc.).

Example pillar files

Debian

apt:
  repos:
    debian-main:
      uri: http://ftp.uk.debian.org/debian/
      suite: 
      components:
        main: True
    debian-security:
      uri: http://security.debian.org/debian-security
      suite: -security
      components:
        main: True
    debian-updates:
      uri: http://ftp.uk.debian.org/debian/
      suite: -updates
      components:
        main: True

Debian non-free

This one stacks with the base Debian, which either needs to be applied (in my pillar it is applied to all systems with the grain os:Debian) or included. Duplicating the main component is unnecessary, as it will be merged with the base components which includes main, but I think duplicating it here reduces the risk of confusion:

apt:
  repos:
    debian-main:
      components:
        main: True
        contrib: True
        non-free: True
    debian-security:
      components:
        main: True
        contrib: True
        non-free: True
    debian-updates:
      components:
        main: True
        contrib: True
        non-free: True

Debian backports

apt:
  repos:
    debian-backports:
      uri: http://ftp.uk.debian.org/debian/
      suite: -backports
      components:
        main: True

Debian backports non-free

Again, this stacks with the base backports pillar.

apt:
  repos:
    debian-backports:
      components:
        main: True
        contrib: True
        non-free: True

Puppet 7

apt:
  repos:
    puppet7:
      uri: http://apt.puppetlabs.com
      suite: 
      components:
        puppet7: True
      no-source: True

Sublime Text

apt:
  repos:
    sublimetext:
      uri: https://download.sublimetext.com/
      suite: apt/stable/