Appliance Release v0.35¶

This page contains the release notes for the v0.35 Anapaya appliance software release. The appliance software release is applicable for the following Anapaya products:

Anapaya CORE
Anapaya EDGE
Anapaya GATE

We recommend always upgrading to the latest available patch release. Please refer to Upgrade Notes (if any) of each release if there are any special steps to be taken when upgrading. For general information on how to upgrade your appliance, please refer to Appliance Update Guide.

Known Issues

There is a bug in version v0.35.0 that prevents the appliance from configuring the system kernel settings. This can result in the dataplane process not starting up properly. This is fixed in v0.35.1.
In version v0.35.0 metrics of the IP-in-SCION tunneling service were sometimes incomplete. This is fixed in v0.35.1.
Starting with version v0.35.0, for Mellanox Connectx-5/6 interfaces (including VFs), custom interface names are not supported anymore. If you have custom interface names for Mellanox Connectx-5/6 interfaces, you need to rename them to the default interface names before upgrading to v0.35.0.

Often the name of a Mellanox Connectx-5/6 interface is ensXnp0 where X is a number. The default name for a VF is ensXvY where X is the number of the physical interface and Y is the number of the VF.

You can check the list of interfaces with their current names and description using lshw -class network -businfo.
Appliance configurations that contain IP prefixes in non-canonical form can lead to IP-in-SCION tunneling service crashes in releases prior to v0.35.9 or on releases from v0.36.0 to v0.36.3.

On releases v0.36.8 and older, the following accept filter could lead to a crash of a software component and loss of IP-in-SCION tunneling connectivity:
```
"accept_filter": [
    {
      "action": "ACCEPT",
      "prefixes": [
        "198.51.100.210/28"
      ],
      "sequence_id": 0
    }
  ],
]
```
Instead, the accept filter should be expressed in its canonical form:
```
"accept_filter": [
    {
      "action": "ACCEPT",
      "prefixes": [
        "198.51.100.208/28"
      ],
      "sequence_id": 0
    }
  ],
]
```

Upgrade Notes¶

Warning

Release v0.34.0 and newer require Ubuntu 22.04 as the system and at least the anapaya-system package with version 2.8.0. Please refer to Upgrade Notes for more information.

Warning

In release v0.35.9, the appliance controller has been extended with additional validation rules. If you are upgrading from a previous verison and your appliance configuration contains IP prefixes in non-canonical form, the appliance controller will start and wait for the configuration to be fixed.

To ensure that your appliance is healthy, check the appliance controller logs after the update:

journalctl -u appliance-controller.service -e

Verify that the logs do not contain any log events with the following message:

"validating latest configuration","cause":{"msg":"invalid configuration"...

v0.35.9 (2024-07-19)¶

Fixes¶

The appliance controller now requires all IP prefixes to be provided in canonical form. In prior releases, a non-canonical IP prefix in the accept filter could lead to crashing of the IP-in-SCION tunneling service.

v0.35.8 (2024-06-19)¶

Fix a bug in the IP-in-SCION tunneling path monitoring that could lead to incorrect path statistics and only a subset of desirable paths being available

v0.35.7 (2024-06-14)¶

Improvements¶

The IP-in-SCION tunneling compmonent now monitors maximum 100 (configurable) paths per remote AS. This is a measure to prevent high load on topologies with very high path diversity.

Fixes¶

Fix an issue when monitoring a high number of paths in the IP-in-SCION tunneling component. Previously, path probes could be dropped due to an internal queue not having enough space.

v0.35.6 (2024-05-16)¶

Improvements¶

Reduce resource consumption of the IP-in-SCION tunneling gateway.

v0.35.5 (2024-05-10)¶

Fixes¶

Improve failover of IP-in-SCION tunneling under certain setups.

v0.35.4 (2024-03-08)¶

Fixes¶

Fix setup of explicit MAC addresses on VFs on top of i40e NICs.
Fix race condition in setup of Mellanox 5 NICs.
Fix a rare race condition in the appliance controller if interfaces have multiple addresses configured.

v0.35.3 (2024-02-19)¶

Fixes¶

The gateway uses less SIG streams for the IP-in-SCION tunneling service. This fixes a compatibility issue with releases older than v0.34, that could lead to traffic being dropped under certain circumstances.

v0.35.2 (2024-02-14)¶

Improvements¶

VRRP can now be configured to run in unicast mode. In the unicast mode, the VRRP packets are sent to all physical routers in the VRRP group using the unicast addresses. By default, VRRP uses the multicast mode where the VRRP packets are broadcasted to all routers using the multicast address. The unicast mode is useful when something in the network does not support the standard VRRP multicast mode.

The unicast mode can be enabled by providing the unicast addresses of the routers in the VRRP group. The unicast addresses can be provided using the peers option in the VRRP configuration.

The following example shows how to configure unicast VRRP on an ethernet interface:

{
  "interfaces": {
    "ethernets": [
      {
        "name": "eno7",
        "vrrp": [
          {
            "vrid": 1,
            "addresses": [
              "10.100.1.1"
            ],
            "priority": 100,
            "peers": [
              "10.2.1.2",
              "10.2.1.3"
            ]
          }
        ]
      },
    ]
  }
}

Fixes¶

Static peer appliance SCION interfaces are now correctly generated in auto-allowed mode. Previously, this was not always the case and this may have led to subtle problems in certain cases. In particular, the discovery service announced the gateways without allowed interfaces, which lead to allowed interfaces not being respected. Instead, all SCION interfaces were treated as allowed.
The appliance now correctly configures the telemetry listen endpoint if no IP is specified.
The dataplane is now always stopped before the appliance removes virtual functions (VFs). This prevents the appliance getting stuck in a state where it cannot remove VFs because the dataplane is still using them.

v0.35.1 (2024-01-09)¶

Fixes¶

Fix a bug that was introduced in v0.35.0 that caused kernel configurations to not be applied.
Fix inconsistent metrics in the IP-in-SCION tunneling service.
Fix the IP-in-SCION tunneling service not properly updating remote gateway monitors when the remote control address or the allowed interfaces change.
Omit unnecessary reconfigurations of the prefix server in the IP-in-SCION tunneling service.

v0.35.0 (2023-12-21)¶

Features¶

New IP-in-SCION Tunneling Gateway Service¶

The IP-in-SCION tunneling gateway service has been redesigned and rewritten from the ground up. Instead of wiping and recreating the entire gateway control and dataplane state on every change of inputs (configuration, remote tunneling endpoints, IP prefixes, etc.), the new gateway now calculates the minimal set of changes required to go from the current state to the desired state. This minimizes disruption to the dataplane and solves a longstanding issue where the gateway would not be able to process configuration and state changes fast enough under highly dynamic network conditions. While this rewrite does not change any user facing behavior, it ensures that the gateway service is now much more scalable and can handle much more dynamic network conditions.

Dataplane Performance Optimizations¶

The VPP dataplane has been upgraded to the latest stable release 23.10. This brings with it a number of improvements to reliability and forwarding performance. Furthermore, we optimized the AES computation needed for SCION packet forwarding to improve the forwarding performance of CORE/EDGE/GATE appliances up to 2.5x! Finally, the appliance now uses an improved hugepage and RX/TX queue allocation strategy depending on the available memory of the system and the port speed of the NICs in use.

Improvements¶

Improved support for Mellanox Connectx-5/6 NICs¶

The appliance has now improved support for Mellanox Connectx-5/6 NICs by using the VPP native RDMA driver. This improves forwarding performance for these NICs and allows users to configure Mellanox Connectx-5/6 devices directly as physical interfaces. Previously, Mellanox Connectx-5/6 devices had to be configured as virtual functions (VFs) and were only supported through the VPP DPDK driver.

Appliance API improvements¶

The appliance API includes several new endpoints:

GET /debug/vpp/bond to list information about VPP bond interfaces.
GET /debug/vpp/lacp to list VPP LACP information.
GET /debug/cluster/status to provide information about the cluster synchronization status of various peers.

Additionally, the appliance API now validates that all tunneling endpoints that are configured on an appliance, either as local endpoints or as peer endpoints, are unique.

Finally, the API documentation now lists for every API the earliest release in which it is available.

Support for EBGP multihop¶

The appliance now supports EBGP multihop. This allows to configure EBGP sessions between appliances that are not directly connected. The number of hops can be configured via the bgp.neighbor.ebgp_multihop property in the appliance configuration.

Various other improvements¶

If the appliance is part of a an AS acting as a CA and the CA service_type is set, the appliance validates that the address and necessary credentials are set.
The appliance-cli info command now contains information about the overall appliance health and SCION interface states.
The appliance configuration now includes an experiments section. This section can be used to configure experimental features that are not yet officially supported. Possible experiments will be documented in the relevant documentation sections.
The routing daemon (FRR) and telemetry exporters (OpenTelemetry, Node Exporter, PromTail) have been updated to their latest minor versions.

Fixes¶

The appliance now properly configures the IPv4 and IPv6 gateway addresses defined on the network interface. Previously, gateway addresses were only set for Linux interfaces.
The appliance now properly propagates changes to the MAC address done on a VLAN’s parent interface to the VLAN interface itself.
The appliance now properly configures bond interfaces even if the configuration was interrupted through any means.
Interactions between controllers and the dataplane cannot fail anymore with a “all channel IDs are in use” error. This sometimes caused a disconnection between controllers and dataplane, which could lead to a brief loss of connectivity in rare cases.