System¶

The system section exposes some configuration settings of the operating system and the VPP dataplane that are relevant for the operation of the appliance.

Operating system¶

The following fields expose operating system settings:

dns.servers

A list of DNS server IP addresses that the appliance uses for domain name resolution.

ntp.servers

A list of NTP time server IP addresses that the appliance uses for time synchronization.

kernel

A group of Linux kernel settings that are exposed.

orphan:

Warning

Be advised that you should only change these values if you understand the implications. In case you have any questions, reach out to customer-support@anapaya.net.

hugepage_size

The size of hugepages the kernel should allocate at boot time.

hugepages

The number of hugepages the kernel should allocate at boot time.

iommu_enabled

A flag configuring whether the IOMMU subsystem in the Linux kernel is enabled. To improve performance, this should be set to true on all systems that support it.

Hint

Use the following command to find out if the appliance host supports IOMMU:

find /sys | grep dmar

It should report a non-empty list of devices.

Note

After changing this option the appliance needs to be rebooted.

system

The following example configures enables IOMMU and configures custom DNS and NTP servers.

"system": {
  "dns": {
    "servers": [
      {
        "address": "2.2.2.2"
      }
    ]
  },
  "kernel": {
    "iommu_enabled": true
  },
  "ntp": {
    "servers": [
      {
        "address": "time3.example.ch"
      }
    ]
  }
}

Vector Packet Processor (VPP)¶

orphan:

Warning

Be advised that you should only change these values if you understand the implications. In case you have any questions, reach out to customer-support@anapaya.net.

The appliance router and IP-in-SCION tunneling processes use a VPP dataplane to provide fast packet processing.

The following fields configure relevant settings of and related to the VPP dataplane:

buffers.data_size

The default size of the buffer VPP allocates for a packet, in bytes (VPP Reference).

buffers.num_buffers

The number of buffers VPP allocates per NUMA node (VPP Reference).

connection

Configures the appliance’s connection to the VPP dataplane. The VPP control service is the part of the appliance that configures the underlying network devices for the VPP dataplane and regularly performs health checks to make sure it works as expected.

health_check.probe_interval
The interval in which a health check probe is sent to the VPP dataplane.
health_check.reply_timeout
The time in which the VPP control service expects a reply from the VPP dataplane.
health_check.threshold
The number of health checks, from VPP control service to the VPP dataplane, that is allowed to time out before the connection is considered dead.
reconnect_attempts
The number of connection attempts from VPP control service to the VPP dataplane after starting up.
reconnect_interval
The interval at which a connection is attempted from VPP control service to the VPP dataplane after starting up.

cpu.main_core

The logical CPU core the main VPP dataplane thread runs on (VPP Reference).

cpu.workers

The number of workers that are created for the VPP dataplane. The workers are automatically pinned to consecutive CPU cores. (VPP Reference)

cpu.corelist_workers`

The list of CPU cores to be used by the workers. The cores are pinned to the workers in the order they are listed. The format for the list is A,B1-Bn,C1-Cn. This setting is mutually exclusive with workers and the list must not contain the main_core.

Note

Each configured worker is pinned to a separate CPU core. These workers will consume 100% of the core they are pinned to, because the worker is constantly polling for packets.

statseg.size: The size of the VPP statseg memory. The statseg memory is used to store the active flow when doing flow exporting. Also check the flow exporting configuration in the management section.

Warning

Changing the statseg size will trigger a restart of the dataplane and thus will interrupt traffic.

VPP creates a TUN device to accept packets from and route packets to Linux. The device gets created automatically and can be configured with the following fields:

tun.mtu: The MTU (Maximum Transmission Unit) on the TUN device that the VPP dataplane creates in Linux.
tun.prefixes: A list of prefixes that should not be forwarded by VPP but routed to Linux.

system.vpp

The following example configures the VPP dataplane to use a larger than default buffer size and only one CPU.

"system": {
  "vpp": {
    "buffers": {
      "data_size": 9000,
      "num_buffers": 16834
    },
    "cpu": {
      "main_core": 1,
      "workers": 1
    }
  }
}

Managing system resources¶

The system.resources.service_limits section allows you to configure CPU and memory resource limits for the services running on the appliance.

Note

The default system resource limits configured in the appliance are suitable for most deployments. Only change these settings if you know what you are doing.

service_limits

A list of per service resource limits. Each entry has the following fields:

name
The name of the service to which the resource limits apply. This must be one of the following values:

CONTROL

ROUTER

DISPATCHER

DAEMON

GATEWAY

CA_FRONTEND

DATAPLANE_CONTROL

MOLE

CRON

TELEMETRY

FRR

FRR_EXPORTER

NODE_EXPORTER

PROMTAIL

cpu
The CPU limit in number of CPU cores that can be used by the service. This can be a floating point number to specify a fraction of a core. Setting this to 0 disables the CPU limit.

memory
The memory limit in Megabytes that can be used by the service. Setting this to 0 disables the memory limit.

system.resources

The following example configures the CPU and memory limits for the ROUTER service.

"system": {
  "resources": {
    "service_limits": [
      {
        "name": "ROUTER",
        "cpu": 1.5,
        "memory": 2048
      }
    ]
  }
}