Deployment Examples

In the following, we describe the common deployment scenarios for the Anapaya EDGE. The simple EDGE setup is the most basic setup and is recommended for small deployments without the need for redundancy. For larger and more critical deployments, we recommend using a redundant setup with two or more EDGE appliances that are configured as clusters.

Simple EDGE setup

The simple EDGE setup consists of a single EDGE appliance with one SCION link to an ISP. If the ISP provides multiple links, the EDGE appliance can be configured to use multiple SCION links for additional redundancy.

Note that with this setup, there will be downtime when the appliance is being upgraded or fails.

Topology

Our target topology contains the following elements:

• One EDGE appliance with one SCION link to an ISP. The ISP provides the customer with the necessary IP and SCION addressing information:

  • IP address of the WAN interface of the EDGE: 169.254.2.2/30

  • IP address of the ISP’s SCION CORE router: 169.254.2.1

  • SCION interface ID of the ISP’s SCION CORE router: 20

  • Underlay port of the ISP’s SCION CORE router: 31020

• The EDGE appliance is connected to the customer network via a LAN interface with the IP address 10.10.0.12/24. This information is provided by the customer.

• The EDGE appliance has a loopback interface with the IP address 10.20.0.1/32. This IP address is used to configure the SCION services.

We will now configure the EDGE appliance step-by-step to implement the above scenario.

Network Interface Configuration

First, we will look at the configuration for the network interfaces. In the setup, there are two physical network interfaces - one for the customer network lan and one for the SCION link to the ISP wan. Additionally, we configure a loopback interface loop1 that is used to configure the SCION services.

In this example, we also configure the lan interface to be the default route.

Network Interface Configuration

"interfaces": {
  "ethernets": [
    {
      "addresses": ["10.10.0.12/24"],
      "name": "lan",
      "gateway": {
        "ipv4_gateway": "10.10.0.1"
      }
    },
    {
      "addresses": ["169.254.2.2/30"],
      "name": "wan"
    }
  ],
  "loopbacks": [
    {
      "addresses": ["10.20.0.1/32"],
      "name": "loop1"
    }
  ]
},

Please refer to Network Interfaces for the full documentation on network interface configuration and to Network for troubleshooting network configuration issues.

Management Configuration

Next, we will look at the management configuration. The management configuration contains the configuration of the hostname, management API and telemetry access.

In this example, we configure the following fields:

hostname

The hostname is used to identify the host in the telemetry data. In this example, we set the hostname to simple-edge.

api

The management API is used to configure the appliance. In this example, we configure the management API to listen on the lan interface and on localhost. This allows us to access the management API from the LAN and from the appliance itself.

telemetry

The telemetry configuration is used to configure where telemetry data is exposed. In this example, we configure the telemetry to listen on the lan interface on port 42001.

Management Configuration

"management": {
  "api": {
    "listeners": [
      {
        "address": "127.0.0.1:443",
        "description": "localhost"
      },
      {
        "address": "10.10.0.12:443",
        "description": "Management API from LAN"
      }
    ]
  },
  "hostname": "simple-edge",
  "telemetry": {
    "address": "10.10.0.12:42001"
  }
},

Please refer to Management for the full documentation on management configuration.

SCION Configuration

Next, we will look at the SCION configuration. The SCION section contains the configuration of the SCION protocol and AS and can be grouped into the general AS, the control plane, and the data plane configuration.

General AS Configuration

Each SCION AS has several general AS configuration options such as the ISD-AS identifier, the AS forwarding secret key, or a human-readable description of the AS. For the full list of the general AS configuration options, please refer to General AS Configuration.

In this example, we configure the following fields:

isd_as

The ISD-AS identifier of the AS. In this example, the ISD-AS identifier is 1-ff00:0:10.

forwarding_key

The base64 encoded secret key used to verify the integrity and authenticity of the SCION packet header during forwarding on the data plane.

Note

In the example, we use a dummy key. In production, the secret key should be a strong password that is at least 12 characters long and contain uppercase letters, lowercase letters, numbers, and special characters.

You can create a new forwarding key with the following command:

appliance-cli crypto forwarding-key

scion_mtu

The SCION MTU is the maximum size of a SCION packet supported in the internal network. Since we are using an IPv4 underlay network, we set this to 1472 bytes. This assumes an Ethernet frame size of 1500 bytes, minus 20 bytes for the IPv4 header and 8 bytes for the UDP header that is used to transport SCION packets.

General AS Configuration

"scion": {
  "ases": [
    {
      "isd_as": "1-ff00:0:10",
      "forwarding_key": "PLACEHOLDER_FW_KEY",
      "scion_mtu": 1472,

Control Plane Configuration

The SCION control plane configuration consists of the control and cppki sections that configure the network endpoint of the control service and the AS certificate renewal. The full documentation on the SCION control plane configuration can be found in Control Plane Configuration.

The network endpoint of the control service is typically configured to listen on the loop1 interface. The port can be set to 0 to let the appliance automatically allocate a free port.

Note

The control service can also be bound to any other interface with a VPP driver. This can be useful in cases the control service needs to be accessible to other appliances in the network.

The cppki section configures the SCION AS certificate issuers. These are required to periodically renew the AS certificate on the EDGE appliance. In this example, we assume that there are two certificate issuers for ISD 1 at 1-ff00:0:1 and 1-ff00:0:200. We want to configure 1-ff00:0:1 as the primary issuer and 1-ff00:0:200 as the fallback in case the primary issuer is not reachable.

SCION Control Plane Configuration

"control": {
  "address": "10.20.0.1:40001",
  "enabled": true
},
"cppki": {
  "issuers": [
    {
      "isd_as": "1-ff00:0:1",
      "priority": 0
    },
    {
      "isd_as": "1-ff00:0:200",
      "priority": 1
    }
  ]
},

SCION Data Plane Configuration

The SCION data plane configuration consists of the router and the neighbors sections. The router section contains the configuration of the internal SCION interface internal_interface, while the neighbors section defines all the SCION links to other SCION ASes, i.e., the external SCION interfaces. The full documentation on the SCION data plane configuration can be found in Data Plane Configuration.

The internal SCION interface is where the appliance receives SCION packets from AS internal hosts for forwarding. It is defined as a UDP/IP network endpoint on which the SCION packets are received from the internal network. The endpoint needs to be specified as a string in the format <ip>:<port>. If the port is set to 0 it will be automatically assigned.

In our case, we configure the internal SCION interface to listen on the loop1 interface, since only services on the appliance itself need to send SCION packets to the internal interface.

Note

Similar to the control service, the internal SCION interface can also be bound to any other interface with a VPP driver.

Each entry in the neighbors section contains a remote SCION AS to which one or multiple links are established. The EDGE appliance in this example has one neighbor, the ISP that provides SCION connectivity. For the neighbor, we need to specify the local and remote interface network address endpoints, the link type (always PARENT for EDGEs), the SCION interface ID, and the SCION MTU supported on this link.

The SCION interface ID must be unique for each SCION interface in an AS. Since for the simple EDGE setup there is only one SCION link, we set the interface ID to 1. This SCION link is configured on the WAN interface and on the IP address provided by the ISP. The port can be chosen freely and we set it to 31000 following the recommendations in Default Port Allocations. The SCION MTU is set to 1472 bytes as indicated by the ISP. For the remote interface we also used the values provided by the ISP: the IP address of the ISP’s SCION CORE router and the underlay port 31020 as remote address and the SCION interface ID 20.

SCION Data Plane Configuration

"router": {
  "enabled": true,
  "internal_interface": "10.20.0.1:30100"
},
"neighbors": [
  {
    "neighbor_isd_as": "1-ff00:0:1",
    "relationship": "PARENT",
    "interfaces": [
      {
        "address": "169.254.2.2:31000",
        "administrative_state": "UP",
        "interface_id": 1,
        "remote": {
          "address": "169.254.10.1:31020",
          "interface_id": 20
        },
        "scion_mtu": 1472
      }
    ]
  }
]

Please refer to SCION for the full documentation on the SCION configuration and to SCION / CPPKI for troubleshooting SCION configuration issues.

SCION Tunneling Configuration

The SCION tunneling configuration enables the IP-in-SCION tunneling module of the appliance and can be used to configure IP tunnels towards other EDGE or GATE appliances.

The full documentation on the IP-in-SCION tunneling configuration can be found in IP-in-SCION Tunneling.

In this example, we configure a routing domain towards our communication partner with the SCION AS 1-ff00:0:25. We want to make our local network 10.10.0.0/24 reachable for end hosts in the remote SCION AS 1-ff00:0:25 in the 10.20.0.0/24 network.

The tunneling configuration is configured with the following:

endpoint

We bind the IP-in-SCION tunneling endpoint to our loop1 interface and use the default port values.

remotes

We configure a single remote SCION AS 1-ff00:0:25.

path_filters

We configure a simple allow-all paths filter using the acl property.

traffic_matchers

We configure a default traffic matcher that matches all IP traffic.

static_announcements

We configure a static announcement for the 10.10.0.0/24 IP prefix that we want to be reachable for the remote SCION AS. We also enable next-hop tracking with the target 10.10.0.1.

domains

We configure a single routing domain that only allows communication with the remote SCION AS 1-ff00:0:25. The domain is configured with the following fields:

local_isd_ases

Our local SCION AS 1-ff00:0:10.

prefixes.accept_filter

The IP prefixes that are accepted from the remote SCION AS. In this example, we accept the 10.20.0.0/24 prefix.

prefixes.announce_filter

The IP prefixes that are announced to the remote SCION AS. In this example, we announce the 10.10.0.0/24 prefix.

remote_isd_ases

We accept the remote SCION AS 1-ff00:0:25 with an ACCEPT filter.

traffic_policies

The traffic policies combine a traffic matcher with a set of path filters. In this case we use the default traffic matcher and the allow-all path filter.

SCION Tunneling Configuration

"scion_tunneling": {
  "endpoint": {
    "control_port": 40201,
    "data_port": 40200,
    "enabled": true,
    "ip": "10.20.0.1",
    "probe_port": 40202
  },
  "domains": [
    {
      "local_isd_ases": ["1-ff00:0:10"],
      "name": "default-domain",
      "prefixes": {
        "accept_filter": [
          {
            "action": "ACCEPT",
            "prefixes": ["10.20.0.0/24"],
            "sequence_id": 0
          }
        ],
        "announce_filter": [
          {
            "action": "ACCEPT",
            "prefixes": ["10.10.0.0/24"],
            "sequence_id": 0
          }
        ]
      },
      "remote_isd_ases": [
        {
          "action": "ACCEPT",
          "isd_as": "1-ff00:0:25",
          "sequence_id": 0
        }
      ],
      "traffic_policies": [
        {
          "sequence_id": 0,
          "traffic_matcher": "default",
          "failover_sequence": [
            {
              "path_filter": "allow-all-paths",
              "sequence_id": 0
            }
          ]
        }
      ]
    }
  ],
  "path_filters": [
    {
      "acl": ["+"],
      "name": "allow-all-paths"
    }
  ],
  "remotes": [
    {
      "isd_as": "1-ff00:0:25"
    }
  ],
  "static_announcements": [
    {
      "prefixes": ["10.10.0.0/24"],
      "sequence_id": 0,
      "next_hop_tracking": {
        "target": "10.10.0.1"
      }
    }
  ],
  "traffic_matchers": [
    {
      "condition": "BOOL=true",
      "name": "default"
    }
  ]
}

Complete Configuration

For completeness, we provide the full configuration example of the EDGE appliance:

Complete Configuration

{
  "config": {
    "interfaces": {
      "ethernets": [
        {
          "addresses": ["10.10.0.12/24"],
          "name": "lan",
          "gateway": {
            "ipv4_gateway": "10.10.0.1"
          }
        },
        {
          "addresses": ["169.254.2.2/30"],
          "name": "wan"
        }
      ],
      "loopbacks": [
        {
          "addresses": ["10.20.0.1/32"],
          "name": "loop1"
        }
      ]
    },
    "management": {
      "api": {
        "listeners": [
          {
            "address": "127.0.0.1:443",
            "description": "localhost"
          },
          {
            "address": "10.10.0.12:443",
            "description": "Management API from LAN"
          }
        ]
      },
      "hostname": "simple-edge",
      "telemetry": {
        "address": "10.10.0.12:42001"
      }
    },
    "scion": {
      "ases": [
        {
          "isd_as": "1-ff00:0:10",
          "forwarding_key": "PLACEHOLDER_FW_KEY",
          "scion_mtu": 1472,
          "control": {
            "address": "10.20.0.1:40001",
            "enabled": true
          },
          "cppki": {
            "issuers": [
              {
                "isd_as": "1-ff00:0:1",
                "priority": 0
              },
              {
                "isd_as": "1-ff00:0:200",
                "priority": 1
              }
            ]
          },
          "router": {
            "enabled": true,
            "internal_interface": "10.20.0.1:30100"
          },
          "neighbors": [
            {
              "neighbor_isd_as": "1-ff00:0:1",
              "relationship": "PARENT",
              "interfaces": [
                {
                  "address": "169.254.2.2:31000",
                  "administrative_state": "UP",
                  "interface_id": 1,
                  "remote": {
                    "address": "169.254.10.1:31020",
                    "interface_id": 20
                  },
                  "scion_mtu": 1472
                }
              ]
            }
          ]
        }
      ]
    },
    "scion_tunneling": {
      "endpoint": {
        "control_port": 40201,
        "data_port": 40200,
        "enabled": true,
        "ip": "10.20.0.1",
        "probe_port": 40202
      },
      "domains": [
        {
          "local_isd_ases": ["1-ff00:0:10"],
          "name": "default-domain",
          "prefixes": {
            "accept_filter": [
              {
                "action": "ACCEPT",
                "prefixes": ["10.20.0.0/24"],
                "sequence_id": 0
              }
            ],
            "announce_filter": [
              {
                "action": "ACCEPT",
                "prefixes": ["10.10.0.0/24"],
                "sequence_id": 0
              }
            ]
          },
          "remote_isd_ases": [
            {
              "action": "ACCEPT",
              "isd_as": "1-ff00:0:25",
              "sequence_id": 0
            }
          ],
          "traffic_policies": [
            {
              "sequence_id": 0,
              "traffic_matcher": "default",
              "failover_sequence": [
                {
                  "path_filter": "allow-all-paths",
                  "sequence_id": 0
                }
              ]
            }
          ]
        }
      ],
      "path_filters": [
        {
          "acl": ["+"],
          "name": "allow-all-paths"
        }
      ],
      "remotes": [
        {
          "isd_as": "1-ff00:0:25"
        }
      ],
      "static_announcements": [
        {
          "prefixes": ["10.10.0.0/24"],
          "sequence_id": 0,
          "next_hop_tracking": {
            "target": "10.10.0.1"
          }
        }
      ],
      "traffic_matchers": [
        {
          "condition": "BOOL=true",
          "name": "default"
        }
      ]
    }
  }
}

Redundant EDGE setup

Two or more appliances provide more resilience against failures and allow for downtime-free upgrades of the appliances. This is the recommended architecture for all EDGE deployments. Within the redundant appliance setups, there are multiple viable setups depending on the organization requirements.

We recommend using at least two connected appliances with either static or dynamic redundancy. If a connected setup is not possible, i.e. it is not possible or desired for the EDGE appliances to communicate, it is recommended to use multiple Simple EDGE setup setups, each being its own SCION ISD-AS. This applies for example if the two EDGE appliances are placed in different physical sites which don’t have connectivity.

Note

For an even higher level of redundancy, we recommend a setup with four EDGE appliances that are operated as two clusters in different physical sites. This setup is not covered in this documentation. Please contact the Anapaya customer success team for more information.

The advantage of connected appliances is that it allows all appliances to take advantage of the provider uplinks of all appliances. In this setup, the appliances automatically share path information between each other and thus enable provider uplink redundancy.

Topology

Our target topology contains the following elements:

• Two EDGE appliances in different physical locations s01.chzrh1.cust and s01.chgtg1.cust that are connected via the LAN in 10.10.0.0/24.

• The EDGE appliance s01.chzrh1.cust has two SCION links to ISP 1 (1-ff00:1:1). The ISP provides the customer with the necessary IP and SCION addressing information:

  • IP address of the WAN interface of the EDGE: 169.254.2.1/29

  • Details of the ISP’s first CORE router

  • IP address: 169.254.2.2

  • SCION interface ID: 20

  • Underlay port: 31020

  • Second ISP CORE router

  • IP address: 169.254.2.4

  • SCION interface ID: 40

  • Underlay port: 31040

• The EDGE appliance s01.chgtg1.cust has two SCION links to ISP 2 (1-ff00:1:2). The ISP provides the customer with the necessary IP and SCION addressing information:

  • IP address of the WAN interface of the EDGE: 10.120.12.6/29

  • Details of the ISP’s first CORE router

  • IP address: 10.120.12.1

  • SCION interface ID: 121

  • Underlay port: 31121

  • Second ISP CORE router

  • IP address: 10.120.12.2

  • SCION interface ID: 122

  • Underlay port: 31122

• The EDGE appliance s01.chzrh1.cust is connected to the customer network

  via a LAN interface with the IP address 10.10.0.10. This information is provided by the customer.

• The EDGE appliance s01.chgtg1.cust is connected to the customer network

  via a LAN interface with the IP address 10.10.0.12. This information is provided by the customer.

We will now configure the EDGE appliance step-by-step to implement the above scenario.

Network Interface Configuration

First, we will look at the configuration for the network interfaces. In the setup, there are two physical network interfaces - one for the customer network lan and one for the SCION link to the ISP wan.

In this example, we also configure the lan interface to be the default route.

Network Interface Configuration

s01.chzrh1.cust

"interfaces": {
  "ethernets": [
    {
      "addresses": ["10.10.0.10/24"],
      "name": "lan",
      "gateway": {
        "ipv4_gateway": "10.10.0.11"
      }
    },
    {
      "addresses": ["169.254.2.1/29"],
      "name": "wan"
    }
  ]
},

s01.chgtg1.cust

"interfaces": {
  "ethernets": [
    {
      "addresses": ["10.10.0.12/24"],
      "name": "lan",
      "gateway": {
        "ipv4_gateway": "10.10.0.13"
      }
    },
    {
      "addresses": ["10.120.12.6/29"],
      "name": "wan"
    }
  ]
},

Please refer to Network Interfaces for the full documentation on network interface configuration and to Network for troubleshooting network configuration issues.

Management Configuration

Next, we will look at the management configuration. The management configuration contains the configuration of the hostname, management API and telemetry access.

In this example, we configure the following fields:

hostname

The hostname is used to identify the host in the telemetry data. In this example, we set the hostname to s01-chzrh1-cust and s01-chgtg1-cust.

api

The management API is used to configure the appliance. In this example, we configure the management API to listen on the lan interface and on localhost. This allows us to access the management API from the LAN and from the appliance itself.

telemetry

The telemetry configuration is used to configure where telemetry data is exposed. In this example, we configure the telemetry to listen on the lan interface on port 42001.

Management Configuration

s01.chzrh1.cust

"management": {
  "api": {
    "listeners": [
      {
        "address": "127.0.0.1:443",
        "description": "localhost"
      },
      {
        "address": "10.10.0.10:443",
        "description": "Management API from LAN"
      }
    ]
  },
  "hostname": "s01-chzrh1-cust",
  "telemetry": {
    "address": "10.10.0.10:42001"
  }
},

s01.chgtg1.cust

"management": {
  "api": {
    "listeners": [
      {
        "address": "127.0.0.1:443",
        "description": "localhost"
      },
      {
        "address": "10.10.0.12:443",
        "description": "Management API from LAN"
      }
    ]
  },
  "hostname": "s01-chgtg1-cust",
  "telemetry": {
    "address": "10.10.0.12:42001"
  }
},

Please refer to Management for the full documentation on management configuration.

SCION Configuration

Next, we will look at the SCION configuration. The SCION section contains the configuration of the SCION protocol and AS and can be grouped into the general AS, the control plane, and the data plane configuration.

General AS Configuration

Each SCION AS has several general AS configuration options such as the ISD-AS identifier, the AS forwarding secret key, or a human-readable description of the AS. For the full list of the general AS configuration options, please refer to General AS Configuration.

In this example, we configure the following fields:

isd_as

The ISD-AS identifier of the AS. In this example, the ISD-AS identifier is 1-ff00:0:10.

forwarding_key

The base64 encoded secret key used to verify the integrity and authenticity of the SCION packet header during forwarding on the data plane.

Note

In the example, we use a dummy key. In production, the secret key should be a strong password that is at least 12 characters long and contain uppercase letters, lowercase letters, numbers, and special characters.

You can create a new forwarding key with the following command:

appliance-cli crypto forwarding-key

scion_mtu

The SCION MTU is the maximum size of a SCION packet supported in the internal network. Since we are using an IPv4 underlay network, we set this to 1472 bytes. This assumes an Ethernet frame size of 1500 bytes, minus 20 bytes for the IPv4 header and 8 bytes for the UDP header that is used to transport SCION packets.

General AS Configuration

s01.chzrh1.cust

"scion": {
  "ases": [
    {
      "isd_as": "1-ff00:0:10",
      "forwarding_key": "PLACEHOLDER_FW_KEY",
      "scion_mtu": 1472,

s01.chgtg1.cust

"scion": {
  "ases": [
    {
      "isd_as": "1-ff00:0:10",
      "forwarding_key": "PLACEHOLDER_FW_KEY",
      "scion_mtu": 1472,

Control Plane Configuration

The SCION control plane configuration consists of the control and cppki sections that configure the network endpoint of the control service and the AS certificate renewal. The full documentation on the SCION control plane configuration can be found in Control Plane Configuration.

The network endpoint of the control service is configured to listen on the lan interface. The port can be set to 0 to let the appliance automatically allocate a free port.

The cppki section configures the SCION AS certificate issuers. These are required to periodically renew the AS certificate on the EDGE appliance. In this example, we assume that there are two certificate issuers for ISD 1 at 1-ff00:0:1 and 1-ff00:0:200. We want to configure 1-ff00:0:1 as the primary issuer and 1-ff00:0:200 as the fallback in case the primary issuer is not reachable.

SCION Control Plane Configuration

s01.chzrh1.cust

"control": {
  "address": "10.10.0.10:40001",
  "enabled": true
},
"cppki": {
  "issuers": [
    {
      "isd_as": "1-ff00:0:1",
      "priority": 0
    },
    {
      "isd_as": "1-ff00:0:200",
      "priority": 1
    }
  ]
},

s01.chgtg1.cust

"control": {
  "address": "10.10.0.12:40001",
  "enabled": true
},
"cppki": {
  "issuers": [
    {
      "isd_as": "1-ff00:0:1",
      "priority": 0
    },
    {
      "isd_as": "1-ff00:0:200",
      "priority": 1
    }
  ]
},

SCION Data Plane Configuration

The SCION data plane configuration consists of the router and the neighbors sections. The router section contains the configuration of the internal SCION interface internal_interface, while the neighbors section defines all the SCION links to other SCION ASes, i.e., the external SCION interfaces. The full documentation on the SCION data plane configuration can be found in Data Plane Configuration.

The internal SCION interface is where the appliance receives SCION packets from AS internal hosts for forwarding. It is defined as a UDP/IP network endpoint on which the SCION packets are received from the internal network. The endpoint needs to be specified as a string in the format <ip>:<port>. If the port is set to 0 it will be automatically assigned.

In our case, we configure the internal SCION interface to listen on the lan interface. Since the appliances are connected via the LAN, this allows the appliances to send packets via the paths provided by another appliance.

Each entry in the neighbors section contains a remote SCION AS to which one or multiple links are established. The EDGE appliance in this example has one neighbor, the ISP that provides SCION connectivity. For the neighbor, we need to specify the local and remote interface network address endpoints, the link type (always PARENT for EDGEs), the SCION interface ID, and the SCION MTU supported on this link.

The SCION interface ID must be unique for each SCION interface in an AS. Since for the redundant EDGE setup there are two appliances with two SCION links, we use the interface IDs 1 to 4. The SCION links are configured on the WAN interfaces and on the IP addresses provided by the ISP. The port can be chosen freely and we set it to 31001 to 31004 following the recommendations in Default Port Allocations. The SCION MTU is set to 1472 bytes as indicated by the ISP. For the remote interfaces we also use the values provided by the ISP: the IP address of the ISP’s SCION CORE routers and the respective underlay ports together with the provided SCION interface IDs.

SCION Data Plane Configuration

s01.chzrh1.cust

"router": {
  "enabled": true,
  "internal_interface": "10.10.0.10:30100"
},
"neighbors": [
  {
    "neighbor_isd_as": "1-ff00:0:1",
    "relationship": "PARENT",
    "interfaces": [
      {
        "address": "169.254.2.1:31001",
        "administrative_state": "UP",
        "interface_id": 1,
        "remote": {
          "address": "169.254.2.2:31020",
          "interface_id": 20
        },
        "scion_mtu": 1472
      },
      {
        "address": "169.254.2.1:31002",
        "administrative_state": "UP",
        "interface_id": 2,
        "remote": {
          "address": "169.254.2.4:31040",
          "interface_id": 40
        },
        "scion_mtu": 1472
      }
    ]
  }
]

s01.chgtg1.cust

"router": {
  "enabled": true,
  "internal_interface": "10.10.0.12:30100"
},
"neighbors": [
  {
    "neighbor_isd_as": "1-ff00:0:1",
    "relationship": "PARENT",
    "interfaces": [
      {
        "address": "10.120.12.6:31003",
        "administrative_state": "UP",
        "interface_id": 3,
        "remote": {
          "address": "10.120.12.1:31121",
          "interface_id": 121
        },
        "scion_mtu": 1472
      },
      {
        "address": "10.120.12.6:31004",
        "administrative_state": "UP",
        "interface_id": 4,
        "remote": {
          "address": "10.120.12.2:31122",
          "interface_id": 122
        },
        "scion_mtu": 1472
      }
    ]
  }
]

Please refer to SCION for the full documentation on the SCION configuration and to SCION / CPPKI for troubleshooting SCION configuration issues.

SCION Tunneling Configuration

The SCION tunneling configuration enables the IP-in-SCION tunneling module of the appliance and can be used to configure IP tunnels towards other EDGE or GATE appliances.

The full documentation on the IP-in-SCION tunneling configuration can be found in IP-in-SCION Tunneling.

In this example, we configure a routing domain towards our communication partner with the SCION AS 1-ff00:0:25. We want to make our local network 10.10.0.0/24 reachable for end hosts in the remote SCION AS 1-ff00:0:25 in the 10.20.0.0/24 network.

The tunneling configuration is configured with the following:

endpoint

We bind the IP-in-SCION tunneling endpoint to our lan interface and use the default port values.

remotes

We configure a single remote SCION AS 1-ff00:0:25.

path_filters

We configure a simple allow-all paths filter using the acl property.

traffic_matchers

We configure a default traffic matcher that matches all IP traffic.

static_announcements

We configure a static announcement for the 10.10.0.0/24 IP prefix that we want to be reachable for the remote SCION AS, and enable next hop tracking.

domains

We configure a single routing domain that only allows communication with the remote SCION AS 1-ff00:0:25. The domain is configured with the following fields:

local_isd_ases

Our local SCION AS 1-ff00:0:10.

prefixes.accept_filter

The IP prefixes that are accepted from the remote SCION AS. In this example, we accept the 10.20.0.0/24 prefix.

prefixes.announce_filter

The IP prefixes that are announced to the remote SCION AS. In this example, we announce the 10.10.0.0/24 prefix.

remote_isd_ases

We accept the remote SCION AS 1-ff00:0:25 with an ACCEPT filter.

traffic_policies

The traffic policies combine a traffic matcher with a set of path filters. In this case we use the default traffic matcher and the allow-all path filter.

SCION Tunneling Configuration

s01.chzrh1.cust

"scion_tunneling": {
  "endpoint": {
    "control_port": 40201,
    "data_port": 40200,
    "enabled": true,
    "ip": "10.10.0.10",
    "probe_port": 40202
  },
  "domains": [
    {
      "local_isd_ases": ["1-ff00:0:10"],
      "name": "default-domain",
      "prefixes": {
        "accept_filter": [
          {
            "action": "ACCEPT",
            "prefixes": ["10.20.0.0/24"],
            "sequence_id": 0
          }
        ],
        "announce_filter": [
          {
            "action": "ACCEPT",
            "prefixes": ["10.10.0.0/24"],
            "sequence_id": 0
          }
        ]
      },
      "remote_isd_ases": [
        {
          "action": "ACCEPT",
          "isd_as": "1-ff00:0:25",
          "sequence_id": 0
        }
      ],
      "traffic_policies": [
        {
          "sequence_id": 0,
          "traffic_matcher": "default",
          "failover_sequence": [
            {
              "path_filter": "allow-all-paths",
              "sequence_id": 0
            }
          ]
        }
      ]
    }
  ],
  "path_filters": [
    {
      "acl": ["+"],
      "name": "allow-all-paths"
    }
  ],
  "remotes": [
    {
      "isd_as": "1-ff00:0:25"
    }
  ],
  "static_announcements": [
    {
      "prefixes": ["10.10.0.0/24"],
      "sequence_id": 0,
      "next_hop_tracking": {
        "target": "10.10.0.11"
      }
    }
  ],
  "traffic_matchers": [
    {
      "condition": "BOOL=true",
      "name": "default"
    }
  ]
}

s01.chgtg1.cust

"scion_tunneling": {
  "endpoint": {
    "control_port": 40201,
    "data_port": 40200,
    "enabled": true,
    "ip": "10.10.0.12",
    "probe_port": 40202
  },
  "domains": [
    {
      "local_isd_ases": ["1-ff00:0:10"],
      "name": "default-domain",
      "prefixes": {
        "accept_filter": [
          {
            "action": "ACCEPT",
            "prefixes": ["10.20.0.0/24"],
            "sequence_id": 0
          }
        ],
        "announce_filter": [
          {
            "action": "ACCEPT",
            "prefixes": ["10.10.0.0/24"],
            "sequence_id": 0
          }
        ]
      },
      "remote_isd_ases": [
        {
          "action": "ACCEPT",
          "isd_as": "1-ff00:0:25",
          "sequence_id": 0
        }
      ],
      "traffic_policies": [
        {
          "sequence_id": 0,
          "traffic_matcher": "default",
          "failover_sequence": [
            {
              "path_filter": "allow-all-paths",
              "sequence_id": 0
            }
          ]
        }
      ]
    }
  ],
  "path_filters": [
    {
      "acl": ["+"],
      "name": "allow-all-paths"
    }
  ],
  "remotes": [
    {
      "isd_as": "1-ff00:0:25"
    }
  ],
  "static_announcements": [
    {
      "prefixes": ["10.10.0.0/24"],
      "sequence_id": 0,
      "next_hop_tracking": {
        "target": "10.10.0.13"
      }
    }
  ],
  "traffic_matchers": [
    {
      "condition": "BOOL=true",
      "name": "default"
    }
  ]
}

Cluster Configuration

The cluster configuration is used to configure the clustering of EDGE appliances. This allows the EDGE appliances to share path information between each other and thus enable provider uplink redundancy. For EDGE appliances, the static cluster configuration is used. The full documentation on the cluster configuration can be found in Cluster.

In this example, we configure a cluster with the following fields:

peers

The cluster consists of two EDGE appliances, therefore in the peers section we configure the other EDGE appliance.

name

The name of the peer. In this example, we set the name to s01-chgtg1-cust and s01-chzrh1-cust.

scion.ases

The list of SCION ASes that exist on the peer. In this example, we configure the following a single SCION AS with the following fields:

isd_as

The ISD-AS identifier of the peer. In this example, the ISD-AS identifier is 1-ff00:0:10.

shard_id

The shard ID of the peer. In this example, we set the shard ID to 1 for s01-chzrh1-cust and 2 for s01-chgtg1-cust.

neighbors

The list of SCION neighbors of the peer. In this example, both EDGEs have a single neighbor 1-ff00:0:1 or 1-ff00:0:2 respectively.

interfaces

The list of SCION interfaces of the peer. In this example, both EDGEs have two interfaces to their neighbor. The crucial part is the next_hop field, which corresponds to the value of the internal_interface of the peer appliance.

scion_tunneling.endpoint

The IP-in-SCION tunneling endpoint of the peer. These are the same values as are configured in the scion_tunneling.endpoint section of the peer appliance.

Cluster Configuration

s01.chzrh1.cust

"cluster": {
  "peers": [
    {
      "name": "s01.chgtg1.cust",
      "scion": {
        "ases": [
          {
            "control": {
              "address": "10.10.0.12:30252"
            },
            "isd_as": "1-ff00:1:10",
            "shard_id": 2,
            "neighbors": [
              {
                "neighbor_isd_as": "1-ff00:1:2",
                "relationship": "PARENT",
                "interfaces": [
                  {
                    "interface_id": 3,
                    "next_hop": "10.10.0.12:30100",
                    "scion_mtu": 1472
                  },
                  {
                    "interface_id": 4,
                    "next_hop": "10.10.0.12:30100",
                    "scion_mtu": 1472
                  }
                ]
              }
            ]
          }
        ]
      },
      "scion_tunneling": {
        "endpoint": {
          "control_port": 30256,
          "data_port": 30056,
          "ip": "10.10.0.12",
          "probe_port": 30856
        }
      }
    }
  ]
},

s01.chgtg1.cust

"cluster": {
  "peers": [
    {
      "name": "s01.chzrh1.cust",
      "scion": {
        "ases": [
          {
            "control": {
              "address": "10.10.0.10:30252"
            },
            "isd_as": "1-ff00:1:10",
            "shard_id": 1,
            "neighbors": [
              {
                "neighbor_isd_as": "1-ff00:1:1",
                "relationship": "PARENT",
                "interfaces": [
                  {
                    "interface_id": 1,
                    "next_hop": "10.10.0.10:30100",
                    "scion_mtu": 1472
                  },
                  {
                    "interface_id": 2,
                    "next_hop": "10.10.0.10:30100",
                    "scion_mtu": 1472
                  }
                ]
              }
            ]
          }
        ]
      },
      "scion_tunneling": {
        "endpoint": {
          "control_port": 30256,
          "data_port": 30056,
          "ip": "10.10.0.10",
          "probe_port": 30856
        }
      }
    }
  ]
},

Complete Configuration

For completeness, we provide the full configuration example of the EDGE appliance:

Complete Configuration

s01.chzrh1.cust

{
  "config": {
    "cluster": {
      "peers": [
        {
          "name": "s01.chgtg1.cust",
          "scion": {
            "ases": [
              {
                "control": {
                  "address": "10.10.0.12:30252"
                },
                "isd_as": "1-ff00:1:10",
                "shard_id": 2,
                "neighbors": [
                  {
                    "neighbor_isd_as": "1-ff00:1:2",
                    "relationship": "PARENT",
                    "interfaces": [
                      {
                        "interface_id": 3,
                        "next_hop": "10.10.0.12:30100",
                        "scion_mtu": 1472
                      },
                      {
                        "interface_id": 4,
                        "next_hop": "10.10.0.12:30100",
                        "scion_mtu": 1472
                      }
                    ]
                  }
                ]
              }
            ]
          },
          "scion_tunneling": {
            "endpoint": {
              "control_port": 30256,
              "data_port": 30056,
              "ip": "10.10.0.12",
              "probe_port": 30856
            }
          }
        }
      ]
    },
    "interfaces": {
      "ethernets": [
        {
          "addresses": ["10.10.0.10/24"],
          "name": "lan",
          "gateway": {
            "ipv4_gateway": "10.10.0.11"
          }
        },
        {
          "addresses": ["169.254.2.1/29"],
          "name": "wan"
        }
      ]
    },
    "management": {
      "api": {
        "listeners": [
          {
            "address": "127.0.0.1:443",
            "description": "localhost"
          },
          {
            "address": "10.10.0.10:443",
            "description": "Management API from LAN"
          }
        ]
      },
      "hostname": "s01-chzrh1-cust",
      "telemetry": {
        "address": "10.10.0.10:42001"
      }
    },
    "scion": {
      "ases": [
        {
          "isd_as": "1-ff00:0:10",
          "forwarding_key": "PLACEHOLDER_FW_KEY",
          "scion_mtu": 1472,
          "control": {
            "address": "10.10.0.10:40001",
            "enabled": true
          },
          "cppki": {
            "issuers": [
              {
                "isd_as": "1-ff00:0:1",
                "priority": 0
              },
              {
                "isd_as": "1-ff00:0:200",
                "priority": 1
              }
            ]
          },
          "router": {
            "enabled": true,
            "internal_interface": "10.10.0.10:30100"
          },
          "neighbors": [
            {
              "neighbor_isd_as": "1-ff00:0:1",
              "relationship": "PARENT",
              "interfaces": [
                {
                  "address": "169.254.2.1:31001",
                  "administrative_state": "UP",
                  "interface_id": 1,
                  "remote": {
                    "address": "169.254.2.2:31020",
                    "interface_id": 20
                  },
                  "scion_mtu": 1472
                },
                {
                  "address": "169.254.2.1:31002",
                  "administrative_state": "UP",
                  "interface_id": 2,
                  "remote": {
                    "address": "169.254.2.4:31040",
                    "interface_id": 40
                  },
                  "scion_mtu": 1472
                }
              ]
            }
          ]
        }
      ]
    },
    "scion_tunneling": {
      "endpoint": {
        "control_port": 40201,
        "data_port": 40200,
        "enabled": true,
        "ip": "10.10.0.10",
        "probe_port": 40202
      },
      "domains": [
        {
          "local_isd_ases": ["1-ff00:0:10"],
          "name": "default-domain",
          "prefixes": {
            "accept_filter": [
              {
                "action": "ACCEPT",
                "prefixes": ["10.20.0.0/24"],
                "sequence_id": 0
              }
            ],
            "announce_filter": [
              {
                "action": "ACCEPT",
                "prefixes": ["10.10.0.0/24"],
                "sequence_id": 0
              }
            ]
          },
          "remote_isd_ases": [
            {
              "action": "ACCEPT",
              "isd_as": "1-ff00:0:25",
              "sequence_id": 0
            }
          ],
          "traffic_policies": [
            {
              "sequence_id": 0,
              "traffic_matcher": "default",
              "failover_sequence": [
                {
                  "path_filter": "allow-all-paths",
                  "sequence_id": 0
                }
              ]
            }
          ]
        }
      ],
      "path_filters": [
        {
          "acl": ["+"],
          "name": "allow-all-paths"
        }
      ],
      "remotes": [
        {
          "isd_as": "1-ff00:0:25"
        }
      ],
      "static_announcements": [
        {
          "prefixes": ["10.10.0.0/24"],
          "sequence_id": 0,
          "next_hop_tracking": {
            "target": "10.10.0.11"
          }
        }
      ],
      "traffic_matchers": [
        {
          "condition": "BOOL=true",
          "name": "default"
        }
      ]
    }
  }
}

s01.chgtg1.cust

{
  "config": {
    "cluster": {
      "peers": [
        {
          "name": "s01.chzrh1.cust",
          "scion": {
            "ases": [
              {
                "control": {
                  "address": "10.10.0.10:30252"
                },
                "isd_as": "1-ff00:1:10",
                "shard_id": 1,
                "neighbors": [
                  {
                    "neighbor_isd_as": "1-ff00:1:1",
                    "relationship": "PARENT",
                    "interfaces": [
                      {
                        "interface_id": 1,
                        "next_hop": "10.10.0.10:30100",
                        "scion_mtu": 1472
                      },
                      {
                        "interface_id": 2,
                        "next_hop": "10.10.0.10:30100",
                        "scion_mtu": 1472
                      }
                    ]
                  }
                ]
              }
            ]
          },
          "scion_tunneling": {
            "endpoint": {
              "control_port": 30256,
              "data_port": 30056,
              "ip": "10.10.0.10",
              "probe_port": 30856
            }
          }
        }
      ]
    },
    "interfaces": {
      "ethernets": [
        {
          "addresses": ["10.10.0.12/24"],
          "name": "lan",
          "gateway": {
            "ipv4_gateway": "10.10.0.13"
          }
        },
        {
          "addresses": ["10.120.12.6/29"],
          "name": "wan"
        }
      ]
    },
    "management": {
      "api": {
        "listeners": [
          {
            "address": "127.0.0.1:443",
            "description": "localhost"
          },
          {
            "address": "10.10.0.12:443",
            "description": "Management API from LAN"
          }
        ]
      },
      "hostname": "s01-chgtg1-cust",
      "telemetry": {
        "address": "10.10.0.12:42001"
      }
    },
    "scion": {
      "ases": [
        {
          "isd_as": "1-ff00:0:10",
          "forwarding_key": "PLACEHOLDER_FW_KEY",
          "scion_mtu": 1472,
          "control": {
            "address": "10.10.0.12:40001",
            "enabled": true
          },
          "cppki": {
            "issuers": [
              {
                "isd_as": "1-ff00:0:1",
                "priority": 0
              },
              {
                "isd_as": "1-ff00:0:200",
                "priority": 1
              }
            ]
          },
          "router": {
            "enabled": true,
            "internal_interface": "10.10.0.12:30100"
          },
          "neighbors": [
            {
              "neighbor_isd_as": "1-ff00:0:1",
              "relationship": "PARENT",
              "interfaces": [
                {
                  "address": "10.120.12.6:31003",
                  "administrative_state": "UP",
                  "interface_id": 3,
                  "remote": {
                    "address": "10.120.12.1:31121",
                    "interface_id": 121
                  },
                  "scion_mtu": 1472
                },
                {
                  "address": "10.120.12.6:31004",
                  "administrative_state": "UP",
                  "interface_id": 4,
                  "remote": {
                    "address": "10.120.12.2:31122",
                    "interface_id": 122
                  },
                  "scion_mtu": 1472
                }
              ]
            }
          ]
        }
      ]
    },
    "scion_tunneling": {
      "endpoint": {
        "control_port": 40201,
        "data_port": 40200,
        "enabled": true,
        "ip": "10.10.0.12",
        "probe_port": 40202
      },
      "domains": [
        {
          "local_isd_ases": ["1-ff00:0:10"],
          "name": "default-domain",
          "prefixes": {
            "accept_filter": [
              {
                "action": "ACCEPT",
                "prefixes": ["10.20.0.0/24"],
                "sequence_id": 0
              }
            ],
            "announce_filter": [
              {
                "action": "ACCEPT",
                "prefixes": ["10.10.0.0/24"],
                "sequence_id": 0
              }
            ]
          },
          "remote_isd_ases": [
            {
              "action": "ACCEPT",
              "isd_as": "1-ff00:0:25",
              "sequence_id": 0
            }
          ],
          "traffic_policies": [
            {
              "sequence_id": 0,
              "traffic_matcher": "default",
              "failover_sequence": [
                {
                  "path_filter": "allow-all-paths",
                  "sequence_id": 0
                }
              ]
            }
          ]
        }
      ],
      "path_filters": [
        {
          "acl": ["+"],
          "name": "allow-all-paths"
        }
      ],
      "remotes": [
        {
          "isd_as": "1-ff00:0:25"
        }
      ],
      "static_announcements": [
        {
          "prefixes": ["10.10.0.0/24"],
          "sequence_id": 0,
          "next_hop_tracking": {
            "target": "10.10.0.13"
          }
        }
      ],
      "traffic_matchers": [
        {
          "condition": "BOOL=true",
          "name": "default"
        }
      ]
    }
  }
}

EDGE - LAN Integrations

The EDGE appliance can be integrated with the LAN in two different ways. The first way is to use the Virtual Router Redundancy Protocol (VRRP) to form a cluster of EDGE appliances. The second way is to use the Border Gateway Protocol (BGP) to establish a BGP session between the EDGE appliances and the LAN. Both ways have their advantages and disadvantages and the choice depends on the requirements of your organization. The following sections describe the two integration methods in more detail.

Static redundancy (VRRP)

Static redundancy is the easiest way to achieve appliance and provider uplink redundancy. In this setup the appliances form a cluster using the Virtual Router Redundancy Protocol (VRRP) and are configured with a virtual IP (VIP) which is shared between the appliances. The VIP is configured as the next hop on the LAN side for all traffic that should be sent via the SCION network.

Static redundancy is recommended when no dynamic routing protocols are available in the part of the organization’s network where the EDGE appliances are placed. It is required that the EDGE appliances are placed within a layer 2 network on the LAN side, such that they can form a VRRP cluster.

Note

In the static redundancy setup, only one EDGE appliance will be used for outgoing traffic, the second EDGE appliance acts as a backup. Incoming traffic from the SCION network may arrive on both EDGE appliances and is not tied to the state of the VIP.

Topology

Our target topology builds on top of the Redundant EDGE Topology and is extended with the following elements:

• The EDGE appliances share the VIP 10.10.0.254 on the LAN side.

Network Interface Configuration

The network interface configuration is extended with the VRRP configuration on the lan interface. In this example we use the virtual router ID 1 and configure s01.chzrh1.cust as the primary and s01.chgtg1.cust as the backup.

Network Interface Configuration

s01.chzrh1.cust

"interfaces": {
  "ethernets": [
    {
      "addresses": ["10.10.0.10/24"],
      "name": "lan",
      "gateway": {
        "ipv4_gateway": "10.10.0.11"
      },
      "vrrp": [
        {
          "addresses": ["10.10.0.254"],
          "vrid": 1,
          "priority": 100
        }
      ]
    },
    {
      "addresses": ["169.254.2.1/29"],
      "name": "wan"
    }
  ]
},

s01.chgtg1.cust

"interfaces": {
  "ethernets": [
    {
      "addresses": ["10.10.0.12/24"],
      "name": "lan",
      "gateway": {
        "ipv4_gateway": "10.10.0.13"
      },
      "vrrp": [
        {
          "addresses": ["10.10.0.254"],
          "vrid": 1,
          "priority": 90
        }
      ]
    },
    {
      "addresses": ["10.120.12.6/29"],
      "name": "wan"
    }
  ]
},

Complete Configuration

For completeness, we provide the full configuration example of the EDGE appliance:

Complete Configuration

s01.chzrh1.cust

{
  "config": {
    "cluster": {
      "peers": [
        {
          "name": "s01.chgtg1.cust",
          "scion": {
            "ases": [
              {
                "control": {
                  "address": "10.10.0.12:30252"
                },
                "isd_as": "1-ff00:1:10",
                "shard_id": 2,
                "neighbors": [
                  {
                    "neighbor_isd_as": "1-ff00:1:2",
                    "relationship": "PARENT",
                    "interfaces": [
                      {
                        "interface_id": 3,
                        "next_hop": "10.10.0.12:30100",
                        "scion_mtu": 1472
                      },
                      {
                        "interface_id": 4,
                        "next_hop": "10.10.0.12:30100",
                        "scion_mtu": 1472
                      }
                    ]
                  }
                ]
              }
            ]
          },
          "scion_tunneling": {
            "endpoint": {
              "control_port": 30256,
              "data_port": 30056,
              "ip": "10.10.0.12",
              "probe_port": 30856
            }
          }
        }
      ]
    },
    "interfaces": {
      "ethernets": [
        {
          "addresses": ["10.10.0.10/24"],
          "name": "lan",
          "gateway": {
            "ipv4_gateway": "10.10.0.11"
          },
          "vrrp": [
            {
              "addresses": ["10.10.0.254"],
              "vrid": 1,
              "priority": 100
            }
          ]
        },
        {
          "addresses": ["169.254.2.1/29"],
          "name": "wan"
        }
      ]
    },
    "management": {
      "api": {
        "listeners": [
          {
            "address": "127.0.0.1:443",
            "description": "localhost"
          },
          {
            "address": "10.10.0.10:443",
            "description": "Management API from LAN"
          }
        ]
      },
      "hostname": "s01-chzrh1-cust",
      "telemetry": {
        "address": "10.10.0.10:42001"
      }
    },
    "scion": {
      "ases": [
        {
          "isd_as": "1-ff00:0:10",
          "forwarding_key": "PLACEHOLDER_FW_KEY",
          "scion_mtu": 1472,
          "control": {
            "address": "10.10.0.10:40001",
            "enabled": true
          },
          "cppki": {
            "issuers": [
              {
                "isd_as": "1-ff00:0:1",
                "priority": 0
              },
              {
                "isd_as": "1-ff00:0:200",
                "priority": 1
              }
            ]
          },
          "router": {
            "enabled": true,
            "internal_interface": "10.10.0.10:30100"
          },
          "neighbors": [
            {
              "neighbor_isd_as": "1-ff00:0:1",
              "relationship": "PARENT",
              "interfaces": [
                {
                  "address": "169.254.2.1:31001",
                  "administrative_state": "UP",
                  "interface_id": 1,
                  "remote": {
                    "address": "169.254.2.2:31020",
                    "interface_id": 20
                  },
                  "scion_mtu": 1472
                },
                {
                  "address": "169.254.2.1:31002",
                  "administrative_state": "UP",
                  "interface_id": 2,
                  "remote": {
                    "address": "169.254.2.4:31040",
                    "interface_id": 40
                  },
                  "scion_mtu": 1472
                }
              ]
            }
          ]
        }
      ]
    },
    "scion_tunneling": {
      "endpoint": {
        "control_port": 40201,
        "data_port": 40200,
        "enabled": true,
        "ip": "10.10.0.10",
        "probe_port": 40202
      },
      "domains": [
        {
          "local_isd_ases": ["1-ff00:0:10"],
          "name": "default-domain",
          "prefixes": {
            "accept_filter": [
              {
                "action": "ACCEPT",
                "prefixes": ["10.20.0.0/24"],
                "sequence_id": 0
              }
            ],
            "announce_filter": [
              {
                "action": "ACCEPT",
                "prefixes": ["10.10.0.0/24"],
                "sequence_id": 0
              }
            ]
          },
          "remote_isd_ases": [
            {
              "action": "ACCEPT",
              "isd_as": "1-ff00:0:25",
              "sequence_id": 0
            }
          ],
          "traffic_policies": [
            {
              "sequence_id": 0,
              "traffic_matcher": "default",
              "failover_sequence": [
                {
                  "path_filter": "allow-all-paths",
                  "sequence_id": 0
                }
              ]
            }
          ]
        }
      ],
      "path_filters": [
        {
          "acl": ["+"],
          "name": "allow-all-paths"
        }
      ],
      "remotes": [
        {
          "isd_as": "1-ff00:0:25"
        }
      ],
      "static_announcements": [
        {
          "prefixes": ["10.10.0.0/24"],
          "sequence_id": 0,
          "next_hop_tracking": {
            "target": "10.10.0.11"
          }
        }
      ],
      "traffic_matchers": [
        {
          "condition": "BOOL=true",
          "name": "default"
        }
      ]
    }
  }
}

s01.chgtg1.cust

{
  "config": {
    "cluster": {
      "peers": [
        {
          "name": "s01.chzrh1.cust",
          "scion": {
            "ases": [
              {
                "control": {
                  "address": "10.10.0.10:30252"
                },
                "isd_as": "1-ff00:1:10",
                "shard_id": 1,
                "neighbors": [
                  {
                    "neighbor_isd_as": "1-ff00:1:1",
                    "relationship": "PARENT",
                    "interfaces": [
                      {
                        "interface_id": 1,
                        "next_hop": "10.10.0.10:30100",
                        "scion_mtu": 1472
                      },
                      {
                        "interface_id": 2,
                        "next_hop": "10.10.0.10:30100",
                        "scion_mtu": 1472
                      }
                    ]
                  }
                ]
              }
            ]
          },
          "scion_tunneling": {
            "endpoint": {
              "control_port": 30256,
              "data_port": 30056,
              "ip": "10.10.0.10",
              "probe_port": 30856
            }
          }
        }
      ]
    },
    "interfaces": {
      "ethernets": [
        {
          "addresses": ["10.10.0.12/24"],
          "name": "lan",
          "gateway": {
            "ipv4_gateway": "10.10.0.13"
          },
          "vrrp": [
            {
              "addresses": ["10.10.0.254"],
              "vrid": 1,
              "priority": 90
            }
          ]
        },
        {
          "addresses": ["10.120.12.6/29"],
          "name": "wan"
        }
      ]
    },
    "management": {
      "api": {
        "listeners": [
          {
            "address": "127.0.0.1:443",
            "description": "localhost"
          },
          {
            "address": "10.10.0.12:443",
            "description": "Management API from LAN"
          }
        ]
      },
      "hostname": "s01-chgtg1-cust",
      "telemetry": {
        "address": "10.10.0.12:42001"
      }
    },
    "scion": {
      "ases": [
        {
          "isd_as": "1-ff00:0:10",
          "forwarding_key": "PLACEHOLDER_FW_KEY",
          "scion_mtu": 1472,
          "control": {
            "address": "10.10.0.12:40001",
            "enabled": true
          },
          "cppki": {
            "issuers": [
              {
                "isd_as": "1-ff00:0:1",
                "priority": 0
              },
              {
                "isd_as": "1-ff00:0:200",
                "priority": 1
              }
            ]
          },
          "router": {
            "enabled": true,
            "internal_interface": "10.10.0.12:30100"
          },
          "neighbors": [
            {
              "neighbor_isd_as": "1-ff00:0:1",
              "relationship": "PARENT",
              "interfaces": [
                {
                  "address": "10.120.12.6:31003",
                  "administrative_state": "UP",
                  "interface_id": 3,
                  "remote": {
                    "address": "10.120.12.1:31121",
                    "interface_id": 121
                  },
                  "scion_mtu": 1472
                },
                {
                  "address": "10.120.12.6:31004",
                  "administrative_state": "UP",
                  "interface_id": 4,
                  "remote": {
                    "address": "10.120.12.2:31122",
                    "interface_id": 122
                  },
                  "scion_mtu": 1472
                }
              ]
            }
          ]
        }
      ]
    },
    "scion_tunneling": {
      "endpoint": {
        "control_port": 40201,
        "data_port": 40200,
        "enabled": true,
        "ip": "10.10.0.12",
        "probe_port": 40202
      },
      "domains": [
        {
          "local_isd_ases": ["1-ff00:0:10"],
          "name": "default-domain",
          "prefixes": {
            "accept_filter": [
              {
                "action": "ACCEPT",
                "prefixes": ["10.20.0.0/24"],
                "sequence_id": 0
              }
            ],
            "announce_filter": [
              {
                "action": "ACCEPT",
                "prefixes": ["10.10.0.0/24"],
                "sequence_id": 0
              }
            ]
          },
          "remote_isd_ases": [
            {
              "action": "ACCEPT",
              "isd_as": "1-ff00:0:25",
              "sequence_id": 0
            }
          ],
          "traffic_policies": [
            {
              "sequence_id": 0,
              "traffic_matcher": "default",
              "failover_sequence": [
                {
                  "path_filter": "allow-all-paths",
                  "sequence_id": 0
                }
              ]
            }
          ]
        }
      ],
      "path_filters": [
        {
          "acl": ["+"],
          "name": "allow-all-paths"
        }
      ],
      "remotes": [
        {
          "isd_as": "1-ff00:0:25"
        }
      ],
      "static_announcements": [
        {
          "prefixes": ["10.10.0.0/24"],
          "sequence_id": 0,
          "next_hop_tracking": {
            "target": "10.10.0.13"
          }
        }
      ],
      "traffic_matchers": [
        {
          "condition": "BOOL=true",
          "name": "default"
        }
      ]
    }
  }
}

Dynamic Redundancy (BGP)

Dynamic redundancy is slightly more complicated to set up than static redundancy but also offers more flexibility for the user. In this setup, each EDGE appliance is configured with a BGP session towards a router on the LAN side. IP prefixes that are learned from the SCION network are advertised to the LAN via BGP, conversely, IP prefixes that are reachable in the LAN need to be advertised to the EDGE appliances via BGP as well. When an IP prefix is no longer reachable via the SCION network, it is retracted from BGP and the LAN will no longer use that EDGE appliance to reach the SCION network.

Note

In the dynamic redundancy setup both EDGE appliances can be used for outgoing traffic simultaneously, this can be influenced by the operator of the LAN side BGP router.

Topology

Our target topology builds on top of the Redundant EDGE Topology and is extended with the following elements:

• Both EDGE appliances are configured with an eBGP session towards the LAN side routers with IPs 10.10.0.11 and 10.10.0.13.

• The eBGP sessions are configured with private ASNs 65001 on both EDGEs and 65002

  on the LAN side routers.

BGP Configuration

The BGP configuration section is used to configure the BGP sessions of the EDGE appliances. In this example we configure the following fields:

global.as

The local AS number of the EDGE appliance. Here, we set the AS number to 65001 for both EDGE appliances.

global.router_id

The router ID of the EDGE appliance. We set the router ID to 10.10.0.10 and 10.10.0.12 which corresponds to the IP address of the lan interface.

neighbors

The list of BGP neighbors. We configure a single neighbor with the following fields:

neighbor_address

The IP address of the BGP neighbor which is 10.10.0.11 for s01.chzrh1.cust and 10.10.0.13 for s01.chgtg1.cust.

remote_as

The BGP AS number of the BGP neighbor. We set the remote AS number to 65002 for both EDGE appliances.

The full documentation on the BGP configuration can be found in Border Gateway Protocol (BGP).

BGP Configuration

s01.chzrh1.cust

"bgp": {
  "global": {
    "as": 65001,
    "router_id": "10.10.0.10"
  },
  "neighbors": [
    {
      "enabled": true,
      "neighbor_address": "10.10.0.11",
      "peer_as": 65002
    }
  ]
},

s01.chgtg1.cust

"bgp": {
  "global": {
    "as": 65001,
    "router_id": "10.10.0.12"
  },
  "neighbors": [
    {
      "enabled": true,
      "neighbor_address": "10.10.0.13",
      "peer_as": 65002
    }
  ]
},

SCION Tunneling Configuration

In the scion_tunneling section, we can now remove the static_announcements list. These are no longer needed, since the EDGE appliances advertise the IP prefixes they receive via BGP from the LAN side routers to the SCION network.

Note

Only BGP prefixes which match the announce filters in domains.prefixes.announce_filter are advertised to the SCION network.

On the other hand, all prefixes that the EDGE appliances learn from the SCION network, and that match the domains.prefixes.accept_filter are now advertised to the LAN side routers via BGP.

Complete Configuration

For completeness, we provide the full configuration example of the EDGE appliance:

Complete Configuration

s01.chzrh1.cust

{
  "config": {
    "bgp": {
      "global": {
        "as": 65001,
        "router_id": "10.10.0.10"
      },
      "neighbors": [
        {
          "enabled": true,
          "neighbor_address": "10.10.0.11",
          "peer_as": 65002
        }
      ]
    },
    "cluster": {
      "peers": [
        {
          "name": "s01.chgtg1.cust",
          "scion": {
            "ases": [
              {
                "control": {
                  "address": "10.10.0.12:30252"
                },
                "isd_as": "1-ff00:1:10",
                "shard_id": 2,
                "neighbors": [
                  {
                    "neighbor_isd_as": "1-ff00:1:2",
                    "relationship": "PARENT",
                    "interfaces": [
                      {
                        "interface_id": 3,
                        "next_hop": "10.10.0.12:30100",
                        "scion_mtu": 1472
                      },
                      {
                        "interface_id": 4,
                        "next_hop": "10.10.0.12:30100",
                        "scion_mtu": 1472
                      }
                    ]
                  }
                ]
              }
            ]
          },
          "scion_tunneling": {
            "endpoint": {
              "control_port": 30256,
              "data_port": 30056,
              "ip": "10.10.0.12",
              "probe_port": 30856
            }
          }
        }
      ]
    },
    "interfaces": {
      "ethernets": [
        {
          "addresses": ["10.10.0.10/24"],
          "name": "lan",
          "gateway": {
            "ipv4_gateway": "10.10.0.11"
          }
        },
        {
          "addresses": ["169.254.2.1/29"],
          "name": "wan"
        }
      ]
    },
    "management": {
      "api": {
        "listeners": [
          {
            "address": "127.0.0.1:443",
            "description": "localhost"
          },
          {
            "address": "10.10.0.10:443",
            "description": "Management API from LAN"
          }
        ]
      },
      "hostname": "s01-chzrh1-cust",
      "telemetry": {
        "address": "10.10.0.10:42001"
      }
    },
    "scion": {
      "ases": [
        {
          "isd_as": "1-ff00:0:10",
          "forwarding_key": "PLACEHOLDER_FW_KEY",
          "scion_mtu": 1472,
          "control": {
            "address": "10.10.0.10:40001",
            "enabled": true
          },
          "cppki": {
            "issuers": [
              {
                "isd_as": "1-ff00:0:1",
                "priority": 0
              },
              {
                "isd_as": "1-ff00:0:200",
                "priority": 1
              }
            ]
          },
          "router": {
            "enabled": true,
            "internal_interface": "10.10.0.10:30100"
          },
          "neighbors": [
            {
              "neighbor_isd_as": "1-ff00:0:1",
              "relationship": "PARENT",
              "interfaces": [
                {
                  "address": "169.254.2.1:31001",
                  "administrative_state": "UP",
                  "interface_id": 1,
                  "remote": {
                    "address": "169.254.2.2:31020",
                    "interface_id": 20
                  },
                  "scion_mtu": 1472
                },
                {
                  "address": "169.254.2.1:31002",
                  "administrative_state": "UP",
                  "interface_id": 2,
                  "remote": {
                    "address": "169.254.2.4:31040",
                    "interface_id": 40
                  },
                  "scion_mtu": 1472
                }
              ]
            }
          ]
        }
      ]
    },
    "scion_tunneling": {
      "endpoint": {
        "control_port": 40201,
        "data_port": 40200,
        "enabled": true,
        "ip": "10.10.0.10",
        "probe_port": 40202
      },
      "domains": [
        {
          "local_isd_ases": ["1-ff00:0:10"],
          "name": "default-domain",
          "prefixes": {
            "accept_filter": [
              {
                "action": "ACCEPT",
                "prefixes": ["10.20.0.0/24"],
                "sequence_id": 0
              }
            ],
            "announce_filter": [
              {
                "action": "ACCEPT",
                "prefixes": ["10.10.0.0/24"],
                "sequence_id": 0
              }
            ]
          },
          "remote_isd_ases": [
            {
              "action": "ACCEPT",
              "isd_as": "1-ff00:0:25",
              "sequence_id": 0
            }
          ],
          "traffic_policies": [
            {
              "sequence_id": 0,
              "traffic_matcher": "default",
              "failover_sequence": [
                {
                  "path_filter": "allow-all-paths",
                  "sequence_id": 0
                }
              ]
            }
          ]
        }
      ],
      "path_filters": [
        {
          "acl": ["+"],
          "name": "allow-all-paths"
        }
      ],
      "remotes": [
        {
          "isd_as": "1-ff00:0:25"
        }
      ],
      "traffic_matchers": [
        {
          "condition": "BOOL=true",
          "name": "default"
        }
      ]
    }
  }
}

s01.chgtg1.cust

{
  "config": {
    "bgp": {
      "global": {
        "as": 65001,
        "router_id": "10.10.0.12"
      },
      "neighbors": [
        {
          "enabled": true,
          "neighbor_address": "10.10.0.13",
          "peer_as": 65002
        }
      ]
    },
    "cluster": {
      "peers": [
        {
          "name": "s01.chzrh1.cust",
          "scion": {
            "ases": [
              {
                "control": {
                  "address": "10.10.0.10:30252"
                },
                "isd_as": "1-ff00:1:10",
                "shard_id": 1,
                "neighbors": [
                  {
                    "neighbor_isd_as": "1-ff00:1:1",
                    "relationship": "PARENT",
                    "interfaces": [
                      {
                        "interface_id": 1,
                        "next_hop": "10.10.0.10:30100",
                        "scion_mtu": 1472
                      },
                      {
                        "interface_id": 2,
                        "next_hop": "10.10.0.10:30100",
                        "scion_mtu": 1472
                      }
                    ]
                  }
                ]
              }
            ]
          },
          "scion_tunneling": {
            "endpoint": {
              "control_port": 30256,
              "data_port": 30056,
              "ip": "10.10.0.10",
              "probe_port": 30856
            }
          }
        }
      ]
    },
    "interfaces": {
      "ethernets": [
        {
          "addresses": ["10.10.0.12/24"],
          "name": "lan",
          "gateway": {
            "ipv4_gateway": "10.10.0.13"
          }
        },
        {
          "addresses": ["10.120.12.6/29"],
          "name": "wan"
        }
      ]
    },
    "management": {
      "api": {
        "listeners": [
          {
            "address": "127.0.0.1:443",
            "description": "localhost"
          },
          {
            "address": "10.10.0.12:443",
            "description": "Management API from LAN"
          }
        ]
      },
      "hostname": "s01-chgtg1-cust",
      "telemetry": {
        "address": "10.10.0.12:42001"
      }
    },
    "scion": {
      "ases": [
        {
          "isd_as": "1-ff00:0:10",
          "forwarding_key": "PLACEHOLDER_FW_KEY",
          "scion_mtu": 1472,
          "control": {
            "address": "10.10.0.12:40001",
            "enabled": true
          },
          "cppki": {
            "issuers": [
              {
                "isd_as": "1-ff00:0:1",
                "priority": 0
              },
              {
                "isd_as": "1-ff00:0:200",
                "priority": 1
              }
            ]
          },
          "router": {
            "enabled": true,
            "internal_interface": "10.10.0.12:30100"
          },
          "neighbors": [
            {
              "neighbor_isd_as": "1-ff00:0:1",
              "relationship": "PARENT",
              "interfaces": [
                {
                  "address": "10.120.12.6:31003",
                  "administrative_state": "UP",
                  "interface_id": 3,
                  "remote": {
                    "address": "10.120.12.1:31121",
                    "interface_id": 121
                  },
                  "scion_mtu": 1472
                },
                {
                  "address": "10.120.12.6:31004",
                  "administrative_state": "UP",
                  "interface_id": 4,
                  "remote": {
                    "address": "10.120.12.2:31122",
                    "interface_id": 122
                  },
                  "scion_mtu": 1472
                }
              ]
            }
          ]
        }
      ]
    },
    "scion_tunneling": {
      "endpoint": {
        "control_port": 40201,
        "data_port": 40200,
        "enabled": true,
        "ip": "10.10.0.12",
        "probe_port": 40202
      },
      "domains": [
        {
          "local_isd_ases": ["1-ff00:0:10"],
          "name": "default-domain",
          "prefixes": {
            "accept_filter": [
              {
                "action": "ACCEPT",
                "prefixes": ["10.20.0.0/24"],
                "sequence_id": 0
              }
            ],
            "announce_filter": [
              {
                "action": "ACCEPT",
                "prefixes": ["10.10.0.0/24"],
                "sequence_id": 0
              }
            ]
          },
          "remote_isd_ases": [
            {
              "action": "ACCEPT",
              "isd_as": "1-ff00:0:25",
              "sequence_id": 0
            }
          ],
          "traffic_policies": [
            {
              "sequence_id": 0,
              "traffic_matcher": "default",
              "failover_sequence": [
                {
                  "path_filter": "allow-all-paths",
                  "sequence_id": 0
                }
              ]
            }
          ]
        }
      ],
      "path_filters": [
        {
          "acl": ["+"],
          "name": "allow-all-paths"
        }
      ],
      "remotes": [
        {
          "isd_as": "1-ff00:0:25"
        }
      ],
      "traffic_matchers": [
        {
          "condition": "BOOL=true",
          "name": "default"
        }
      ]
    }
  }
}