Appliance Release v0.34¶

This page contains the release notes for the v0.34 Anapaya appliance software release. The appliance software release is applicable for the following Anapaya products:

Anapaya CORE
Anapaya EDGE
Anapaya GATE

We recommend always upgrading to the latest available patch release. Please refer to Upgrade Notes (if any) of each release if there are any special steps to be taken when upgrading. For general information on how to upgrade your appliance, please refer to Updating an Appliance.

Known Issues

Changing the log level in the appliance-controller configuration will also change the log level in all managed services, thus leading to a restart of all services.
Removing a VLAN interface on top of an interface with LINUX driver will not get removed from the system. It needs to be removed manually. This has been fixed in release v0.34.1.
Adding a VLAN on a VF is not working correctly in all instances. This has been fixed in release v0.34.1.

Upgrade Notes¶

Warning

Release v0.34.0 and newer require Ubuntu 22.04 as the system and at least the anapaya-system package with version 2.8.0.

To check the current Ubuntu version use:

lsb_release -a

If that does not show Ubuntu 22.04, please install the base image with Ubuntu 22.04. Following the docs: Installing the Appliance Base Image, use version sys_v2.8.0-scion_v0.34.0-1.

To check the current version of the anapaya-system package either use the appliance-cli:

appliance-cli get software/system/installed

or curl (replace the $APPLIANCE_API with the correct address of the appliance):

curl -k https://$APPLIANCE_API/api/v1/software/system/installed

To upgrade an appliance that already is running Ubuntu 22.04 follow the following steps:

Install the anapaya-system package version v2.8.0 or newer if needed. Follow the the guide: Update the anapaya-system package
Restart the appliance-installer by running systemctl restart appliance-installer on the appliance.
Install the anapaya-scion package v0.34.0 or newer by following the guide: Updating an Appliance.
By default the appliance will not configure the new firewall rules. Enabling the firewall management through the appliance, will delete existing iptables rules. To enable the firewall in auto-mode, simply drop the firewall section from the appliance configuration (appliance-cli edit config -i), or follow the Firewall documentation to create a custom configuration.

v0.34.2 (2023-11-07)¶

Fixes¶

The appliance now correctly validates configuration options for bond interfaces. This includes rejecting virtual functions as members for bond interfaces. Furthermore, VLAN interfaces are now correctly configured on top of bond interfaces.
The appliance now removes the default DHCP configuration from the base image, when the appliance is first configured. Previously, the DHCP configuration was not removed, which could lead to unexpected behavior in certain circumstances.
A bug in the IP-in-SCION tunneling component was fixed which would lead it to restart in certain circumstances when multiple local ISD-ASes were configured.

Improvements¶

The appliance now supports configuring a telemetry address that is not available at API gateway start time. Previously, the appliance would fail to start if the telemetry address was not available at start time. Now, the appliance will retry to connect to the telemetry address.

v0.34.1 (2023-10-20)¶

Fixes¶

The appliance now correctly uses VPP_DPDK as a default driver for an empty driver field in the interfaces section.
VLANs configured on virtual functions that were running on interfaces with the igbvf driver are now correctly configured, however, on such interfaces only a single VLAN can be configured. The appliance configuration validator will reject a config that adds multiple VLANs on such a virtual function.
The appliance now correctly deconfigures Linux VLAN interfaces when they are removed from the configuration.
The appliance now correctly deconfigures existing virtual functions when they are removed from the configuration.
The appliance now correctly unbinds a network interface from vfio in all circumstances when the driver is change from VPP_DPDK to LINUX.
The appliance now correctly sets MTU on devices that use ice or iavf drivers.
The appliance management API now correctly handles requests to the /api/v1/network/* endpoints again. In v0.34.0, they were returning 502.

Improvements¶

The appliance now supports interfaces using the ice driver, i.e., Intel E810 Ethernet network adapters.
The version and checksum of the installed anapaya-system package is now included in the output of appliance-cli info.
appliance-cli inspect scion-tunneling is now an alias for appliance-cli inspect tunneling.

v0.34.0 (2023-09-28)¶

Features¶

SCION RSS¶

Starting with release v0.34, the Anapaya appliance supports SCION RSS. This feature significantly enhances the throughput on multi-core systems by enabling receive side scaling (RSS) for SCION traffic. SCION RSS operates by leveraging source port entropy on the UDP underlay and can only be enabled if both sides of a link support it. The appliance automatically detects if a sibling appliance supports SCION RSS via topology synchronization, and uses the feature accordingly. For non-sibling appliances, SCION RSS must be manually enabled in the appliance configuration for each neighbor link individually. By default, SCION RSS is not used for IP-in-SCION tunneling traffic. To benefit from increased performance of IP-in-SCION tunneling, enable SCION RSS by setting scion_tunneling.endpoint.enable_scion_rss = true. For more information, please refer to the documentation here.

Firewall management¶

The appliance now supports firewall management via the appliance management API. By default the appliance configures a sensible set of firewall rules to protect from undesired access to the appliance. The firewall is implemented using nftables. For further details, see the documentation available: Firewall.

New Appliance API endpoints¶

The HTTP POST endpoint /debug/notifications triggers a system reconfiguration with the latest stored version of the configuration. The endpoint can be used to manually trigger a reconfiguration even when the notifications are turned off.
The HTTP POST endpoint /config/validate can be used to validate an appliance configuration without applying it.
The HTTP POST endpoint POST /cppki/certificates/request enables the manual request of a SCION CPPKI AS certificate chain for a given CSR using the regular certificate renewal mechanism. The endpoint expects a CSR and uses that to request a certificate renewal. The certificate renewal request is signed by an active key/certificate of the appliance such that the CA will be able to authenticate the renewal request and issue the certificate. This is useful if one appliance has been disconnected from the SCION network for several days and thus has no valid AS certificate anymore that could be used for certificate renewal. In such a case, one can generate a new CSR on the appliance that was offline and use this endpoint on an appliance that still has a valid AS certificate to request a new certificate on behalf of the sibling. The returned certificate can then be deployed to the offline appliance using the regular POST /cppki/certificates endpoint.

Improvements¶

Merged appliance-host & appliance-controller¶

The appliance-host and appliance-controller services have been merged into the appliance-controller. As part of this change, the appliance-controller now runs as a systemd service and no longer as a docker container. As part of the upgrade the old services are automatically stopped and removed.

Improved configuration validation¶

There are several new configuration validation checks that are performed when pushing an appliance configuration. This should further minimize the risk of misconfiguration.

Interface pinning by default¶

Interface pinning of gateway endpoints (i.e. configuration of allowed_interfaces) is now enabled by default. When the allowed_interfaces property is not set, the local gateway endpoint will automatically use the locally configured SCION interfaces as allowed_interfaces. This feature is enabled by default, but can be disabled by setting the new API property disable_auto_allowed_interfaces to true. This feature is also disabled, when the cluster.synchronization.address property is set and therefore topology synchronization is enabled.

The allowed_interfaces are also automatically filled in for gateway endpoints on peer appliances, when they are listed in the cluster.peers section. This can also be disabled for peer appliances by setting disable_auto_allowed_interfaces on the peer tunneling endpoint to true.

Serving the appliance API via SCION¶

The appliance API can now be served via SCION. This is useful to manage the appliance via the SCION network.

Example SCION listener address: [1-ff00:0:110,127.0.0.1]:443

Various other improvements¶

A set of BGP and FRR metrics are now available on the appliance metrics endpoint. The new metrics are prefixed with frr_. The documentation for the metrics can be found under Telemetry.

Two alerts related to the FRR daemon state and the BGP peer state are added to all products. The relevant runbooks can be found under Troubleshooting & Runbooks.
Add two new alerts related to the SCION dispatcher (SCIONApplicationReadError and SCIONApplicationRegistrationError). The alert applies to all appliance products. Check the corresponding runbooks under Troubleshooting & Runbooks.
The appliance now exposes the RootDistanceMaxSec configuration for NTP. The configuration option can be found on system.ntp.root_distance_max.

For more information refer to: systemd-timesyncd docs
The appliance now strictly parses the configuration file, and a PUT request will fail if the configuration file is invalid. This behavior can be disabled by setting the disable_strict_parsing query parameter.
The driver for a ethernet network interface can now be changed from VPP_DPDK and VPP_VMXNET3 to LINUX without having to reboot the appliance.
VLAN filters can now be configured on a virtual function (VF), meaning the NIC adds and removes the VLAN tag for outgoing and incoming traffic respectively.

This configuration is required when the VF configures a single VLAN which is also configured on the PF.
The TLS configuration of an appliance-cli context can now be updated after the context has been created.
It is now possible to configure static neighbor entries for interfaces in the appliance configuration. A static neighbor defines a mapping of IP to MAC address. The configuration can be set via the interface.ethernets.neighbors property. For more details see the full configuration specification.

Fixes¶

The appliance cron daemon now properly handles updates to the configuration of individual cron tasks. Under certain circumstances, an updated configuration was not properly picked up by the daemon.
The appliance now correctly applies a change of the telemetry address. Previously, the appliance ignored the change of the telemetry address if it was already configured.
The appliance will now properly ensure that all services are started on boot even if the periodic configuration notifications are disabled.
The appliance configures a fixed value of 25 seconds as persistent keepalive value for all wireguard tunnels (this value is claimed to be working for most setups according to WireGuard docs). Previously, it was disabled, due to a bug.
On reboot, the appliance ensures that network interfaces are bound to VFIO before handing control to the VPP dataplane.
The appliance now properly configures wireguard interfaces and brings them up. Previously, the appliance might fail to bring the wireguard interfaces up if there was no change in the configuration.
The appliance sets context deadline on notification updates. This ensures that the managers eventually finish handling a notification and do not keep hanging.

The private appliance_controller_notifications_queued_total metric was added to track how many notifications are queued and/or dropped.
The prefix discovery of the remote gateway now checks multiple paths to the remote AS. By checking one path for each interface in the remote AS, prefix discovery is successful even if two appliances in the remote AS are not internally connected.
The following metrics were missing the remote_address label which meant that multiple gateways in an remote AS were overwriting each others values:
- gateway_prefixes_advertised
- gateway_prefixes_fetched
- gateway_prefix_fetch_errors_total
The router now properly cleans up the internal interface UDP socket, in case it is interrupted while configuring the dataplane. Previously, cleanup after the router received an interrupt could fail, because the UDP socket still existed when deleting the mirror and scion interface.
The dataplane-control now properly handles all netlink messages. Previously, it could miss netlink messages if there were many of them in a short period of time.
The appliance-cli now supports context names containing a ..

Breaking Changes¶

In the appliance configuration, the following names can no longer contain the pipes (|) character.
- domain names
- traffic matcher names
- path policy names
When migrating to the new version the appliance will automatically replace pipes in these names by slashes.

The appliance-cli inspect tunneling commands and their output were restructured.

appliance-cli inspect tunneling paths:

FINGERPRINT  STATE LATENCY JITTER DROPS EXPIRY   HOPS
2d1dfa220f6b alive 4.81ms  0.54ms 0.00% 3h14m29s 1-ff00:0:110 2>2 1-ff00:0:111
2dfc890de2ca alive 2.77ms  0.50ms 0.00% 3h14m29s 1-ff00:0:110 3>3 1-ff00:0:112
80fc02efd428 alive 4.77ms  0.90ms 0.00% 3h14m29s 1-ff00:0:110 1>1 1-ff00:0:111
b950e977ed92 alive 4.73ms  0.61ms 0.00% 3h14m29s 1-ff00:0:110 4>4 1-ff00:0:113

appliance-cli inspect tunneling summary:

DOMAIN: donkeycorp
  PREFIXES: 10.0.1.0/24
    TRAFFIC MATCHER: normal
      PATH FILTER: default
        REMOTE: 1-ff00:0:111,172.20.2.3:30856
              STATE LATENCY JITTER DROPS EXPIRY   PATH
          --> alive 8.29ms  3.12ms 0.00% 3h14m29s 1-ff00:0:110 1>1 1-ff00:0:111
        REMOTE: 1-ff00:0:111,172.20.2.4:30856
              STATE LATENCY JITTER DROPS EXPIRY   PATH
            > alive 8.29ms  3.12ms 0.00% 3h14m29s 1-ff00:0:110 1>1 1-ff00:0:111

--> Indicates the active path for a traffic matcher within its domain.
> Indicates the candidate path for a currently unused remote.

Multiple command line options were introduced to customize the output of the summary command.