Anapaya EDGE¶
The Anapaya EDGE can be deployed in various deployment setups. Depending on the requirements different deployment architectures should be used. In general, more instances increase redundancy and thus provide more reliability. In the following, we present the most common deployment architectures.
Single EDGE setup¶
The single EDGE setup consists of a single Appliance configured as an EDGE. Note that with this setup, there will be downtime when the Appliance is being upgraded or fails.
The diagram shows an EDGE appliance with three network interfaces. The WAN interface has the IP
address 169.254.10.2/30 and directly connects to the ISP which provides it with SCION
connectivity. The SCION link to the ISP has the interface identifier 1. On the LAN side
the EDGE has an interface with the IP address 10.10.0.2/24, additionally it has a default
gateway on the LAN interface towards 10.10.0.1. The LAN interface should connect to the
internal network of the organization which wants to use a SCION connection. The services
which should be reachable by other participants of the SCION Internet can be placed anywhere
inside the LAN, but must be reachable from the EDGE appliance. Lastly, the EDGE appliance
is configured with a management interface with the IP address 192.168.1.2/24.
In order to send traffic from the LAN into the SCION network, the LAN needs to be configured
with the LAN interface IP (10.10.0.2) of the EDGE appliance as a next hop.
Full Configuration
{ "config": { "interfaces": { "ethernets": [ { "addresses": [ "10.10.0.2/24" ], "name": "lan", "gateway": { "ipv4_gateway": "10.10.0.1" } }, { "addresses": [ "169.254.10.2/30" ], "name": "wan" }, { "addresses": [ "192.168.1.2/24" ], "name": "mgmt", "driver": "LINUX" } ] }, "management": { "api": { "address": "192.168.1.2:42000" }, "hostname": "host.site.org", "telemetry": { "address": "192.168.1.2:42001" } }, "scion": { "ases": [ { "control": { "address": "10.10.0.2:40001", "enabled": true }, "cppki": { "issuers": [ { "isd_as": "1-ff00:0:24", "priority": 0 } ] }, "details": { "name": "Example AS Name" }, "forwarding_key": "PLACEHOLDER_FW_KEY", "isd_as": "1-ff00:0:14", "router": { "enabled": true, "internal_interface": "10.10.0.2:30100" }, "scion_mtu": 1472, "neighbors": [ { "neighbor_isd_as": "1-ff00:0:24", "relationship": "PARENT", "interfaces": [ { "address": "169.254.10.2:30100", "administrative_state": "UP", "interface_id": 1, "remote": { "address": "169.254.10.1:30100", "interface_id": 10 }, "scion_mtu": 1472 } ] } ] } ] }, "scion_tunneling": { "endpoint": { "control_port": 40201, "data_port": 40200, "enabled": true, "ip": "10.10.0.2", "probe_port": 40202 }, "domains": [ { "local_isd_ases": [ "1-ff00:0:14" ], "name": "default-domain", "prefixes": { "accept_filter": [ { "action": "ACCEPT", "prefixes": [ "10.20.0.0/24" ], "sequence_id": 0 } ], "announce_filter": [ { "action": "ACCEPT", "prefixes": [ "10.10.0.15/32" ], "sequence_id": 0 } ] }, "remote_isd_ases": [ { "action": "ACCEPT", "isd_as": "1-ff00:0:25", "sequence_id": 0 } ], "traffic_policies": [ { "path_policy": "allow-all", "sequence_id": 0, "traffic_matcher": "default" } ] } ], "path_policies": [ { "name": "allow-all", "path_filters": [ { "acl": [ "+" ], "sequence_id": 0 } ] } ], "remotes": [ { "isd_as": "1-ff00:0:25" } ], "static_announcements": [ { "prefixes": [ "10.10.0.15/24" ], "sequence_id": 0 } ], "traffic_matchers": [ { "condition": "BOOL=true", "name": "default" } ] }, "system": { "ntp": { "servers": [ { "address": "time.example.org" } ] } } } }
Redundant EDGE Appliances¶
Two or more appliances provide more resilience against failures and allow for downtime-free upgrades of the appliances. This is the recommended architecture for all EDGE deployments. Within the redundant appliance setups, there are multiple viable setups depending on the organization requirements.
We recommend using at least two connected appliances with either static or dynamic redundancy. If a connected setup is not possible, i.e. it is not possible or desired for the EDGE appliances to communicate, it is recommended to use multiple Single EDGE setup setups, each being its own ISD-AS. This applies for example if the two EDGE appliances are placed in different physical sites.
The advantage of connected appliances is that it allows both appliances to take advantage of the provider uplinks of both appliances. In this setup, the appliances automatically share path information between each other and thus enable provider uplink redundancy.
Static redundancy (VRRP)¶
Static redundancy is the easiest way to achieve appliance and provider uplink redundancy. In this setup the appliances form a cluster using the Virtual Router Redundancy Protocol (VRRP) and are configured with a virtual IP (VIP) which is shared between the appliances. The VIP is configured as the next hop on the LAN side for all traffic that should be sent via the SCION network.
Static redundancy is recommended when no dynamic routing protocols are available in the part of the organization’s network where the EDGE appliances are placed. It is required that the EDGE appliances are placed within a layer 2 network on the LAN side, such that they can form a VRRP cluster.
Note
In the static redundancy setup, only one EDGE appliance will be used for outgoing traffic, the second EDGE appliance acts as a backup. Incoming traffic from the SCION network may arrive on both EDGE appliances and is not tied to the state of the VIP.
The diagram shows two EDGE appliances that are configured very similarly to the Single EDGE setup setup. On the LAN side, the two EDGE appliances are connected using a Layer 2 network, such that they can form a VRRP cluster.
In order to send traffic from the LAN into the SCION network, the LAN needs to be configured
with the VIP (10.10.0.5) as a next hop.
LAN Ethernet Interface
"ethernets": [ { "addresses": [ "10.10.0.2/24" ], "name": "lan", "gateway": { "ipv4_gateway": "10.10.0.1" }, "vrrp": [ { "addresses": [ "10.10.0.5" ], "vrid": 1 } ] } ]
Full Configuration
{ "config": { "cluster": { "peers": [ { "name": "host2.site.org", "scion": { "ases": [ { "control": { "address": "10.10.1.1:40001" }, "isd_as": "1-ff00:0:14", "shard_id": 2, "neighbors": [ { "neighbor_isd_as": "1-ff00:0:33", "interfaces": [ { "interface_id": 2, "next_hop": "10.10.1.1:31000" } ] } ] } ] }, "scion_tunneling": { "endpoint": { "control_port": 40201, "data_port": 40200, "ip": "10.10.1.1", "probe_port": 40202 } } } ] }, "interfaces": { "ethernets": [ { "addresses": [ "10.10.0.2/24" ], "name": "lan", "gateway": { "ipv4_gateway": "10.10.0.1" }, "vrrp": [ { "addresses": [ "10.10.0.5" ], "vrid": 1 } ] }, { "addresses": [ "169.254.10.2/30" ], "name": "wan" }, { "addresses": [ "192.168.1.2/24" ], "name": "mgmt", "driver": "LINUX" } ] }, "management": { "api": { "address": "192.168.1.2:42000" }, "hostname": "host1.site.org", "telemetry": { "address": "192.168.1.2:42001" } }, "scion": { "ases": [ { "control": { "address": "10.10.0.2:40001", "enabled": true }, "cppki": { "issuers": [ { "isd_as": "1-ff00:0:24", "priority": 0 } ] }, "details": { "name": "Example AS Name" }, "forwarding_key": "PLACEHOLDER_FW_KEY", "isd_as": "1-ff00:0:14", "router": { "enabled": true, "internal_interface": "10.10.0.2:30100" }, "scion_mtu": 1472, "shard_id": 1, "neighbors": [ { "neighbor_isd_as": "1-ff00:0:24", "relationship": "PARENT", "interfaces": [ { "address": "169.254.10.2:30100", "administrative_state": "UP", "interface_id": 1, "remote": { "address": "169.254.10.1:30100", "interface_id": 10 }, "scion_mtu": 1472 } ] } ] } ] }, "scion_tunneling": { "endpoint": { "control_port": 40201, "data_port": 40200, "enabled": true, "ip": "10.10.0.2", "probe_port": 40202 }, "domains": [ { "local_isd_ases": [ "1-ff00:0:14" ], "name": "default-domain", "prefixes": { "accept_filter": [ { "action": "ACCEPT", "prefixes": [ "10.20.0.0/24" ], "sequence_id": 0 } ], "announce_filter": [ { "action": "ACCEPT", "prefixes": [ "10.10.0.15/32" ], "sequence_id": 0 } ] }, "remote_isd_ases": [ { "action": "ACCEPT", "isd_as": "1-ff00:0:25", "sequence_id": 0 } ], "traffic_policies": [ { "path_policy": "allow-all", "sequence_id": 0, "traffic_matcher": "default" } ] } ], "path_policies": [ { "name": "allow-all", "path_filters": [ { "acl": [ "+" ], "sequence_id": 0 } ] } ], "remotes": [ { "isd_as": "1-ff00:0:25" } ], "static_announcements": [ { "prefixes": [ "10.10.0.15/24" ], "sequence_id": 0 } ], "traffic_matchers": [ { "condition": "BOOL=true", "name": "default" } ] }, "system": { "ntp": { "servers": [ { "address": "time.example.org" } ] } } } }
Dynamic Redundancy (BGP)¶
Dynamic redundancy is slightly more complicated to set up than static redundancy but also offers more flexibility for the user. In this setup, each EDGE appliance is configured with a BGP session towards a router on the LAN side. IP prefixes that are learned from the SCION network are advertised to the LAN via BGP, conversely, IP prefixes that are reachable in the LAN need to be advertised to the EDGE appliances via BGP as well. When an IP prefix is no longer reachable via the SCION network, it is retracted from BGP and the LAN will no longer use that EDGE appliance to reach the SCION network.
Note
In the dynamic redundancy setup both EDGE appliances can be used for outgoing traffic simultaneously, this can be influenced by the operator of the LAN side BGP router.
The diagram shows two EDGE appliances that are configured very similarly to the
Single EDGE setup setup. In this setup, instead of directly connecting to the LAN, the
EDGE appliances are configured to establish a BGP session with their counterparts on the LAN
side (10.10.0.2 and 10.10.1.2). The BGP sessions are configured using eBGP and can use
private ASNs.
The EDGE appliances advertise the IP prefixes which they learn via the SCION network to their BGP peer towards the LAN. Like this, it’s the duty of the BGP peers to route traffic from the LAN towards the EDGE appliances for sending it via the SCION network.
BGP
"bgp": { "global": { "as": 64545, "router_id": "10.10.0.1" }, "neighbors": [ { "auth_password": "PLACEHOLDER_AUTH_PASS", "enabled": true, "local_as": 64545, "neighbor_address": "10.10.0.2", "peer_as": 64546 } ] }
Full Configuration
{ "config": { "bgp": { "global": { "as": 64545, "router_id": "10.10.0.1" }, "neighbors": [ { "auth_password": "PLACEHOLDER_AUTH_PASS", "enabled": true, "local_as": 64545, "neighbor_address": "10.10.0.2", "peer_as": 64546 } ] }, "cluster": { "peers": [ { "name": "host2.site.org", "scion": { "ases": [ { "control": { "address": "10.10.1.1:40001" }, "isd_as": "1-ff00:0:14", "shard_id": 2, "neighbors": [ { "neighbor_isd_as": "1-ff00:0:33", "interfaces": [ { "interface_id": 2, "next_hop": "10.10.1.1:31000" } ] } ] } ] }, "scion_tunneling": { "endpoint": { "control_port": 40201, "data_port": 40200, "ip": "10.10.1.1", "probe_port": 40202 } } } ] }, "interfaces": { "ethernets": [ { "addresses": [ "10.10.0.2/24" ], "name": "lan", "gateway": { "ipv4_gateway": "10.10.0.1" } }, { "addresses": [ "169.254.10.2/30" ], "name": "wan" }, { "addresses": [ "192.168.1.2/24" ], "name": "mgmt", "driver": "LINUX" } ] }, "management": { "api": { "address": "192.168.1.2:42000" }, "hostname": "host1.site.org", "telemetry": { "address": "192.168.1.2:42001" } }, "scion": { "ases": [ { "control": { "address": "10.10.0.2:40001", "enabled": true }, "cppki": { "issuers": [ { "isd_as": "1-ff00:0:24", "priority": 0 } ] }, "details": { "name": "Example AS Name" }, "forwarding_key": "PLACEHOLDER_FW_KEY", "isd_as": "1-ff00:0:14", "router": { "enabled": true, "internal_interface": "10.10.0.2:30100" }, "scion_mtu": 1472, "shard_id": 1, "neighbors": [ { "neighbor_isd_as": "1-ff00:0:24", "relationship": "PARENT", "interfaces": [ { "address": "169.254.10.2:30100", "administrative_state": "UP", "interface_id": 1, "remote": { "address": "169.254.10.1:30100", "interface_id": 10 }, "scion_mtu": 1472 } ] } ] } ] }, "scion_tunneling": { "endpoint": { "control_port": 40201, "data_port": 40200, "enabled": true, "ip": "10.10.0.2", "probe_port": 40202 }, "domains": [ { "local_isd_ases": [ "1-ff00:0:14" ], "name": "default-domain", "prefixes": { "accept_filter": [ { "action": "ACCEPT", "prefixes": [ "10.20.0.0/24" ], "sequence_id": 0 } ], "announce_filter": [ { "action": "ACCEPT", "prefixes": [ "10.10.0.15/32" ], "sequence_id": 0 } ] }, "remote_isd_ases": [ { "action": "ACCEPT", "isd_as": "1-ff00:0:25", "sequence_id": 0 } ], "traffic_policies": [ { "path_policy": "allow-all", "sequence_id": 0, "traffic_matcher": "default" } ] } ], "path_policies": [ { "name": "allow-all", "path_filters": [ { "acl": [ "+" ], "sequence_id": 0 } ] } ], "remotes": [ { "isd_as": "1-ff00:0:25" } ], "static_announcements": [ { "prefixes": [ "10.10.0.15/24" ], "sequence_id": 0 } ], "traffic_matchers": [ { "condition": "BOOL=true", "name": "default" } ] }, "system": { "ntp": { "servers": [ { "address": "time.example.org" } ] } } } }