Appliance Release v0.33¶

This page contains the release notes for the v0.33 Anapaya appliance software release. The appliance software release is applicable for the following Anapaya products:

Anapaya CORE
Anapaya EDGE
Anapaya GATE

We recommend always upgrading to the latest available patch release. Please refer to Upgrade Notes (if any) of each release if there are any special steps to be taken when upgrading. For general information on how to upgrade your appliance, please refer to Updating an Appliance.

Important Upgrade Notes

When an interface has virtual functions defined on it that are owned by VPP, it is necessary to adapt the appliance configuration and change the driver of the parent interface to LINUX.

Known Issues

Certain DPDK drivers with missing features are not supported in v0.33.0 and v0.33.1. Support is added in v0.33.2.
The algorithm for incremental changes to the network stack which was introduced in v0.33.0 has some issues. They are addressed in v0.33.2.
Multiple entries in system/vpp/tun/prefixes are not correctly handled in versions v0.33.0 to v0.33.2. Starting from v0.33.3 all of the prefixes are correctly installed again.
Configuring static BGP announcements in bgp/global/networks does not work in release v0.33.3. If you rely on this feature, upgrade to v0.33.4.
Changing the management/telemetry/address is not working in all current releases. As a workaround, remove the address field and add it again with the new value in two separate configuration updates.
Changing the log level in the appliance-controller configuration will also change the log level in all managed services, thus leading to a restart of all services.

v0.33.0 (2023-04-06)¶

Features¶

Alerts, Runbooks, and Monitoring Dashboards¶

Starting with release v0.33, we publish alerts, troubleshooting runbooks, and a set of prebuilt Grafana dashboards that can be used to monitor Anapaya EDGE, CORE, and GATE appliances. Check out the runbooks and the documentation on how to set up a monitoring stack and use the dashboards.

Appliance CLI¶

The appliance CLI is a new command line interface to interact with the appliance management API and replaces the previous reliance on the curl command. The appliance CLI can be used to configure the appliance, extract information, or upgrade the appliance software. It can be used on the appliance host itself or on a remote machine. For further details, refer to the appliance CLI documentation.

New Anapaya Appliance Base Image Installer¶

The Anapaya base image installer streamlines the installation of the Anapaya appliance on bare-metal devices. The installation can either be guided interactively or preseeded using a configuration file. Please refer to the base image installation guide for more details. Furthermore, the new base images are now based on Ubuntu 22.04 LTS.

Improvements¶

Improved Path Selection Algorithm¶

Anapaya EDGE and GATE appliances use an improved path selection algorithm. The algorithm now also properly takes droprate of the paths into account when selecting the best path. Furthermore, the heuristic for scoring paths based on latency and jitter has been improved. Finally, the algorithm generally prefers paths with fewer hops when other performance metrics are similar. The new path selection algorithm now also reports the reasoning for path switches in the log.

Appliance Management API Improvements¶

The newly introduced /management/api/listeners configuration option allows specifying multiple listening addresses for the appliance management API. This is useful if the appliance management API should be reachable from multiple networks.

Furthermore, the appliance management API exposes new endpoints to provide additional insights:

A new /debug/service-health endpoint provides the health status of all services running on the appliance. /debug/services/{service_name}/health can be used to fetch the health status of a single service.
A new /debug/bgp/neighbors endpoint shows the state of BGP sessions to individual neighbors

Various Other Improvements¶

When network configuration changes dataplane-control does incremental changes to the network stack instead of wiping everything and recreating it from scratch. This enables most network configuration changes without service disruption.
The bgpd daemon that is running as part of the FRR module now always configures soft-reconfiguration inbound for all BGP neighbors. This allows BGP sessions to be reconfigured without having to reset the BGP session.
The number of hugepages can now be configured without the need for a reboot.
The appliance configuration validator has been extended to validate that the /scion/ases/forwarding-key is properly base64 encoded.
The appliance API now supports configuration of strict CSR validation. For this, the /scion/ases/ca_service/anapaya_vault/validation/subject property has been added to the API. It can be set to one of the following values:
- MATCHING_ISD_AS (default): The Anapaya Vault backend validates that the ISD-AS field of the CSR subject is the same as in the certificate that was used to sign the request.
- EXACT_MATCH: The Anapaya Vault backend validates that the complete CSR subject is exactly the same as in the certificate that was used to sign the request.
This setting is only relevant if the appliance is part of a CA AS that uses the Anapaya SCION CA service.

Fixes¶

The appliance management API now stays responsive even after prolonged (around two weeks) inactivity. Previously, the appliance API would hang because it failed to issue the self-signed certificates due to an internal mismatch between the cache and the actual files on disk.
Routing updates on the Linux routing table are now always properly mirrored to the VPP-based forwarding plane. In rare circumstances, the VPP-based forwarding plane was missing some routing updates leading to a loss of connectivity.
Network interfaces based on Mellanox Connectx-5 NICs are now properly detected and configured.
Appliance configuration changes are now atomically written to the file system. Previously, aborting the process (e.g., by pulling the plug) could lead to corrupted files.
The gateway_session_is_healthy metric is now updated correctly. Previously, it would only be updated when the session state changed, leaving the metric uninitialized for a long time in certain cases.

Breaking Changes¶

The /management/api/address has been removed in favor of the newly introduced /management/api/listeners list, which allows specifying multiple listening addresses at the same time. The appliance configuration needs to be adopted accordingly.

E.g., if your previous api configuration looked like this:
```
"address": "127.0.0.1:443"
```
It must be migrated to the following:
```
"listeners": [
    {
        "address": "127.0.0.1:443"
    }
]
```
The appliance will automatically migrate the configuration on startup.
The /cluster/synchronization/legacy_address configuration option has been removed. This option was used to configure cluster synchronization with releases prior to v0.30. This release does not support cluster synchronization with appliances running releases prior to v0.30 anymore.

v0.33.1 (2023-04-12)¶

Fixes¶

Fixes a bug in v0.33.0, where the appliance was unable to configure VLANs.

v0.33.2 (2023-05-08)¶

Improvements¶

The initial password for the default anapaya user on the appliance API is changed to anapaya. This simplifies initial setups of the appliance.

Note

This only affects new installations. Existing installations are not affected by this change.

Warning

Make sure to change the default password before exposing the appliance API to the network.

Fixes¶

Ensure that localhost is always configured on an appliance. For clean installations from the base image, it was missing in some cases and would lead to the API gateway not being able to reach the internal processes.
Fallback to a legacy method for DPDK drivers that do not support a specific multicast ethernet address setup operation. In prior releases, adding a multicast MAC address would fail for such drivers.
Properly send neighbor probes. Previously, these probes were not sent and neighbors would be consistently considered not reachable. This resulted in short interruptions of reachability, and some dropped packets depending on the load.
Changes to the MTU of a VLAN interface and its parent interface are now applied in the correct order. Previously, MTU changes to VLAN interfaces could fail.
Correctly handle multiple IP routes to the same prefix.
If no MAC address is defined for a bond interface, a new random MAC address is used instead of the MAC address of the first added interface. This prevents both interfaces from using the same MAC address in case the first added interface is removed from the bond.
The IPv6 link local address is no longer set on an interface in DOWN state. Because Linux never applies the address on a interface in DOWN state, setting the address would previously fail.

v0.33.3 (2023-06-08)¶

Improvements¶

Add support for the VPP native driver for VMXNet3 interfaces. This driver works better than the DPDK driver for VMXNet3 and fixes an issues with multicast MAC setup on interfaces with VMXNet3 driver.
The appliance now only redistributes routes to the BGP peers that were learned over SGRP. Routes that are learned via BGP are not redistributed to peers.
The appliance API now validates the OAuth metadata and JWKS URLs when OAuth is enabled. If the appliance cannot reach the URLs, the configuration is rejected.
Use NAT Endpoint Dependent (ED) mode instead of Endpoint Independent to track TCP connection state and improve NAT session recycling for TCP connections.
Improve IP-in-SCION tunneling flow reporting. Flows are now always exported even in case of no client traffic, to ensure that the flow exporting mechanism is properly running. Old flows are correctly deleted.

Fixes¶

The IP-in-SCION tunneling component no longer produces gaps in the frame number. This fixes an issues where the receiving side would report frame drops, although, in fact, no data was lost.
Fix setting up multiple prefixes in the system/vpp/tun/prefixes list. This was broken with the state synchronization introduced in release 0.33.0.
Properly support disabling OAuth. Previously disabling OAuth could result in a wrongly configured API Gateway.
Ensure that the API Gateway starts after a reboot. Previously it sometimes failed to start because the network interfaces were not ready.

v0.33.4 (2023-06-14)¶

Fixes¶

The appliance correctly announces statically defined prefixes defined in bgp/global/networks to its BGP peers after it was broken in release v0.33.3.
The gateway service now properly serializes configuration and state updates. In rare cases, having multiple concurrent configuration updates and state changes would lead to a crash of the gateway service.

v0.33.5 (2023-07-07)¶

Fixes¶

The IP-in-SCION tunneling module can no longer end up in a deadlocked state when configuring the forwarding plane. Certain (rare) interleavings of reconfiguration events could lead to a state where the module that configured the forwarding plane could not make progress anymore.

v0.33.6 (2023-08-30)¶

Fixes¶

Fetching the topology from other appliances now correctly works even if one appliance is not reachable. Previously, the fetch could fail even on appliances that are alive if one appliance was not reachable.
Fix a race condition between the certificate cleanup job and the CSR to certificate promotion. In rare cases, the cleanup job would delete the key of a newly posted certificate, this will now no longer happen. So far we only saw this in testing environments.
Fix an issue in determining the difference between two IP-in-SCION tunneling configurations. If the length of the failover_sequence was decreased it could lead to a crash of the IP-in-SCION tunneling component and if it was increased it could be that it wouldn’t be applied. The difference is now correctly determined.
IPv6 link-local addresses are now reliably configured. In rare instances, the configuration of a IPv6 link-local was not completed in time, which could lead the network dataplane to be incorrectly configured.

Improvements¶

The appliance API now also accepts appliance.reader and appliance:reader in addition to appliance/reader and appliance.writer and appliance:writer in addition to appliance/writer for the roles parameter of an OAuth2 token.
The appliance-cli now properly formats PEM encoded content, such as certificates, keys, and CSRs.
Logs of all services are now sent to journald. This simplifies ingestion to centralized log management systems, such as Loki.