IP-in-SCION Tunneling¶
Overview¶
IP-in-SCION tunneling is a mechanism that allows to tunnel IP packets between two IP-in-SCION tunneling endpoints, i.e., Anapaya EDGE appliances. This mechanism enables any type of application and communication to take advantage of SCION-based networking without the need to update any application, client, or server.
On a high-level, IP-in-SCION tunneling involves the following steps:
A sender sends an IP packet towards an IP destination.
The IP packet reaches an IP-in-SCION tunneling endpoint, such as an Anapaya EDGE appliance, in the sender’s network via standard local IP routing.
Based on the destination IP address, the Anapaya EDGE appliance determines the destination SCION AS and possible IP-in-SCION tunneling endpoints within the destination SCION AS. To achieve this, the Anapaya EDGE appliance uses the SCION Gateway Routing Protocol (SGRP).
Next, the Anapaya EDGE appliance determines the optimal SCION path to reach the destination SCION AS. The choice of the path depends on the Routing Domain Configuration and the real-time network performance characteristics, such as latency, jitter, drop rate, etc.
Then, the original IP packet is encapsulated within a SCION packet by the Anapaya EDGE appliance and sent to the remote IP-in-SCION tunneling endpoint in the destination SCION AS. The SCION path used is the one chosen in the previous step.
The remote IP-in-SCION tunneling endpoint receives the SCION packet and decapsulates the original IP packet. It then forwards the packet to the final IP destination using standard local IP routing.
Routing Domains¶
Routing Domains are the primary mechanism to configure the traffic policies used when communicating with remote IP destinations. Conceptually, a routing domain is defined by a set of IP prefixes to which a common set of traffic policies should be applied as shown in the following figure.
These IP prefixes could be announced by one or multiple SCION ASes. The Anapaya EDGE appliance can optimize path selection across all possible destination SCION ASes (while respecting the configured traffic policy). This flexibility enables operators to capture a wide range of routing use-cases - from simple site-to-site tunnels, to complex setups involving remotes that are part of multiple SCION isolation domains, or destinations that are reachable via many SCION ASes.
A routing domain can be further specified by a set of remote SCION ASes. This is useful for example when a domain should only include remote ASes of a specific ISD or a specific set of remote ASes, e.g., the sites of the company’s wide area network. Furthermore, it is also possible to limit the routing domain to a set of source ASes. This is used by Anapaya EDGE appliances that are part of multiple ISDs and need to separate the traffic and use different policies per ISD.
Finally, a domain also includes the ability to limit the IP prefixes that are announced by a local Anapaya EDGE appliance to remote IP-in-SCION tunneling endpoints.
Traffic Policies¶
Traffic policies allow users to influence and control the path selection process of the Anapaya EDGE appliance. For example, an operator can define policies to keep traffic within Swiss jurisdiction or avoid certain network providers. A traffic policy uses path filters to specify the set of SCION paths that should be considered for the path selection. Each path filter defines a subset of all available SCION paths.
Path filters are combined into a failover sequence to define the order in which the path filters are considered. The path filters in a failover sequence are applied separately to the set of possible paths until one of them returns a non-empty path set.
Finally, the traffic policy includes a traffic matcher that defines the set of IP packets for which the failover sequence should be applied.
SCION Gateway Routing Protocol (SGRP)¶
The SCION Gateway Routing Protocol (SGRP) is a routing protocol that enables IP-in-SCION tunneling endpoints, i.e., Anapaya EDGE appliances, to map IP prefixes to SCION ASes.
At a high level, a tunneling endpoint participating in SGRP between two SCION ASes does the following steps:
It discovers the tunneling endpoints in the remote SCION AS. To that end, it periodically sends a discovery message to the control plane of the remote AS which replies with a list of local IP-in-SCION tunneling endpoints.
It periodically queries each discovered endpoint in the remote AS to learn the IP prefixes it announces. From that, the local endpoint builds a mapping from IP prefix to remote tunneling endpoints.
When queried by a remote tunneling endpoint about its prefixes, the local endpoint replies with the set of IP prefixes it wants to announce to the remote.
For the Anapaya EDGE appliance, the set of announced IP prefixes can either be statically configured or is dynamically learned via BGP. Please refer to the BGP section for information on how to configure a BGP session to a peering BGP router.
Domain Configuration¶
As explained in Routing Domains, domains define a common set of traffic policies towards a distinct range of IP destinations. Thus, there are two main aspects to configuring a domain:
What entities are part of the domain?
How is traffic routed within the domain?
The what includes the IP prefixes that are accepted and announced by the local endpoint, the remote SCION ASes, and possibly the local SCION AS identities (in case the Anapaya EDGE appliance is part of multiple SCION ISDs).
The how includes the traffic policies that are applied to outgoing traffic towards any remote destination that is part of the domain.
Domain Entities¶
This sections explains how to define what entities are part of a domain.
IP Prefixes¶
The prefixes section of the domain configuration defines the set of IP
prefixes that are accepted and announced by the local endpoint.
The accept_filter subsection defines the set of IP prefixes that are accepted
from remote tunneling endpoints. IP packets towards an address in these prefixes
are tunneled to a remote tunneling endpoint that matches the domain’s remote matchers.
There cannot be more than one routing domain that accepts a specific IP prefix.
Thus, the accept_filter matchers of different domains must not overlap.
The prefixes matched by the announce_filter subsection of the domain define the
maximum set of IP prefixes that will be announced to remote tunneling endpoints.
Each filter in either of these sections is a list of IP prefix matchers combined
with an action. The action is either ACCEPT or REJECT to implement
allow- and block lists.
Note
The accept and announce filters are what the name implies - filters. This means that not everything they contain is actually accepted or announced. The set of announced IP prefixes are all the IP prefixes configured in Static Announcements or received from BGP peers configured in Border Gateway Protocol (BGP) and filtered by the announce filter. Similarly, the set of accepted IP prefixes is the set of all IP prefixes received from remote IP-in-SCION tunneling endpoints filtered by the accept filter.
Remote ASes¶
The remote_isd_ases section contains a list of SCION AS matchers that -
together with an action (ACCEPT or REJECT) - define the set of SCION
ASes that are part of the domain. This has the effect that IP prefix
announcements will be accepted from these remote ASes and all IP traffic will be
tunneled over paths that end in one of these remote ASes. A SCION AS matcher has
the form <isd>-<as> where <isd> is the ISD ID and <as> is the AS ID.
Both <isd> and <as> can be 0 to match all ISDs and all ASes
respectively. E.g., the matcher 1-ff00:0:1 matches exactly this single AS,
1-0 matches all ASes that are part of ISD 1, and 0-ff00:0:1 matches the
specific AS in all ISDs. A single 0 (or 0-0) matches all ASes in all
ISDs.
Local AS Identities¶
The local_isd_ases section defines a list of local SCION AS identities that
are part of the domain. Setting this field is only necessary if the Anapaya EDGE
appliance is part of multiple SCION ISDs and an operator wants to restrict the
domain to a subset of the local SCION ISDs. This has the effect that traffic
towards remote tunneling endpoints in this domain only uses paths that start in
one of the listed local ASes. Note that if the Anapaya EDGE appliance is only
part of a single ISD or if all available local AS identities are part of the
domain, then local_isd_ases does not need to be set.
Default Domain¶
There can be a single default domain. The default domain is used when no
other domain matches a packet’s destination, i.e., the default domain implicitly
accepts the entire IP space that is not covered by other domains. For that
reason, the default domain can not include any accept_filter. Specifying a
default domain is optional.
Examples¶
Allow-all domain
This example shows a domain that accepts any IP prefix from any remote AS and announces any learned or configured IP prefixes to any remote AS.
{
"remote_isd_ases": [
{
"isd_as": "0-0",
"action": "ACCEPT",
"description": "Include all remote ASes",
"sequence_id": 0
},
],
"prefixes": {
"accept_filter": [
{
"prefixes": [ "0.0.0.0/0, ::/0" ],
"action": "ACCEPT",
"sequence_id": 0
},
],
"announce_filter": [
{
"prefixes": [ "0.0.0.0/0, ::/0" ],
"action": "ACCEPT",
"sequence_id": 0
},
],
},
}
ISD-specific domain
This example shows a domain that only accepts 1.0.0.0/8 and 2.0.0.0/8
from ASes in ISD 1 and announces only prefixes in 1.100.0.0/16.
Furthermore, the domain only includes the local AS identity that is part of
ISD 1.
{
"remote_isd_ases": [
{
"isd_as": "1-0"
"action": "ACCEPT"
"description": "Only ASes in ISD 1 are part of the domain"
"sequence_id": 0
},
],
"local_isd_ases": [ "1-ff00:0:1" ],
"prefixes": {
"accept_filter": [
{
"prefixes": [ "1.0.0.0/8", "2.0.0.0/8" ],
"action": "ACCEPT"
"sequence_id": 0
}
],
"announce_filter": [
{
"prefixes": [ "1.100.0.0/16" ],
"action": "ACCEPT",
"sequence_id": 0
}
]
}
}
ISD-specific default Domain
This example shows the definition of a default domain. The default domain can be thought of as the default route in classical IP routing and is entirely optional. While the default domain cannot limit the set of accepted IP prefixes, it is perfectly possible to limit the set of announced IP prefixes, as well as the set of remote ASes.
{
"default": true,
"remote_isd_ases": [
{
"isd_as": "1-0"
"action": "ACCEPT"
"description": "Only ASes in ISD 1 are part of the domain"
"sequence_id": 0
},
],
"prefixes": {
"announce_filter": [
{
"prefixes": [ "1.100.0.0/16" ],
"action": "ACCEPT",
"sequence_id": 0
}
]
}
}
Traffic Policies¶
Traffic policies let an operator influence how traffic is routed within a domain. Each domain can have one or multiple traffic policies. A traffic policy contains a traffic matcher that defines the set of packets to which the policy applies and a set of path filters (grouped into a failover sequence) that defines what paths are allowed to be used by the matched packets.
A path filter defines a set of rules that a path must satisfy to be accepted by the filter. To select a path for the tunnel, the following steps are taken:
The path filters are traversed in order of their sequence ID within the failover sequence.
Each path filter is applied to the candidate set of paths.
The best path out of the acceptable set is chosen according to path health and performance metrics. If no such path exists, i.e., no candidate path is allowed by the filter or no path is healthy, selection continues to the next path filter in the failover sequence.
If all filters have been considered and no healthy path was selected, traffic will be dropped.
Failover Sequence
To illustrate the failover sequence consider the example below. First, the
path filter provider_1_outgoing is applied to the set of all available
paths leading to an acceptable set of paths. If one or more paths of the
acceptable set are healthy, the best path is chosen according to its
performance metrics. If no path of the acceptable set is healthy, the path
filter allow_all is applied to the set of all available paths. This leads
to a (potentially) different acceptable set. Again, the best healthy path is
chosen and if no such path exists, the traffic is dropped.
{
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "provider_1_outgoing"
},
{
"sequence_id": 1,
"path_filter": "allow_all"
},
]
}
Traffic policies are applied in the order of their sequence ID (ascending). For each IP packet, only a single traffic policy will be applied even if it matches multiple policies. A traffic policy matches if the traffic matcher defined in the policy matches the incoming IP packet.
While only a single traffic policy applies to an incoming IP packet, within the traffic policy the failover sequence can be used to configure “failover” behavior. If a path filter in the failover sequence has no healthy path to the remote AS, the next filter will be tried in the order defined by the failover sequence.
Traffic matchers and path filters are referenced by name. This allows to easily reuse these objects in different traffic policies. The traffic matchers are defined in Traffic Matchers and the path filters are defined in Path Filters.
Examples¶
Traffic policy limited to a single ISD
This example shows a configuration with a single traffic policy that configures all traffic to only use paths within the Swiss isolation domain. Refer to the path filters and traffic matchers examples to see how the corresponding filters and matchers are defined.
{
"traffic_policies": [
{
"sequence_id": 0,
"name": "CH ISD only",
"description": "No traffic leaves the Swiss ISD.",
"traffic_matcher": "all",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "ch_isd_only"
}
]
}
]
}
Traffic engineering for priority traffic
This example shows a configuration with two traffic policies. The first policy chooses paths via provider 1 for priority traffic (defined as traffic tagged with the EF DSCP flag) and falls back to paths via provider 2 if no paths via provider 1 are available. The second policy chooses paths via provider 2 for all non-priority traffic. If no paths are available via provider 2 no fallback happens and traffic is dropped. Refer to the path filters and traffic matchers examples to see how the corresponding filters and matchers are defined.
{
"traffic_policies": [
{
"sequence_id": 0,
"name": "Priority traffic"
"description": "Priority traffic is sent preferred via provider 1",
"traffic_matcher": "ef_tagged",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "provider_1_outgoing"
},
{
"sequence_id": 1,
"path_filter": "allow_all"
},
]
},
{
"sequence_id": 1,
"name": "Non-priority traffic"
"description": "Non-priority traffic is sent preferred via provider 2",
"traffic_matcher": "all",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "provider_2_outgoing"
},
]
},
]
}
Path Filters¶
A path filter defines a set of paths by applying the filter to all available paths. The filter is identified by a unique name and can contain an Access Control List (ACL) and a hop pattern both of which operate on hop predicates.
A hop predicate defines a hop of a SCION path and has the form
<isd>-<as>#<interface>, where <isd> is the SCION ISD identifier,
<as> is the SCION AS identifier, and <interface> is the interface
identifier of a SCION path hop, i.e., the SCION link in the given SCION AS. The
hop predicate can be fully or partially qualified, i.e., all entries of the hop
predicate are optional or can include wildcards (0). A single 0 matches
every SCION path hop.
An ACL consists of a list of ACL entries. Each ACL entry has the form
<action> <hop-predicate>, where the action can either be accept (+) or
deny (-). The hop predicate is optional. If no hop predicate is specified,
the action matches every hop, i.e., a single + is the default accept action
and a single - is the default deny action. The ACL is applied by
sequentially applying all ACL entries to paths. If the ACL is empty, it defaults
to accepting all paths.
A hop pattern defines a sequence of hop predicates that a path has to match to
be accepted. It can be thought of as a regular expression on SCION paths. Each
hop predicate can optionally be extended with a modifier * or +. The
* modifier means 0 or more occurrences. The + means one or more
occurrences.
Path Filters
{
"path_filters": [
{
"name": "allow_all",
"description": "Allow all paths",
"acl": [
"+ 0"
]
},
{
"name": "ch_isd_only",
"description": "Only allow paths within the Swiss ISD (64)",
"acl": [
"+ 64-0",
"-"
]
},
{
"name": "provider_1_outgoing",
"description": "Paths that leave the local AS through SCION interface 1",
"hop_pattern": "1-ff00:1:111#1 0*"
},
{
"name": "provider_1_incoming",
"description": "Paths that enter the local AS through SCION interface 1",
"hop_pattern": "0* 1-ff00:1:111#1"
},
{
"name": "provider_2_outgoing",
"description": "Paths that leave the local AS through SCION interface 2",
"hop_pattern": "1-ff00:1:111#2 0*"
}
{
"name": "provider_2_incoming",
"description": "Paths that enter the local AS through SCION interface 2",
"hop_pattern": "0* 1-ff00:1:111#2"
}
]
}
Traffic Matchers¶
A traffic matcher is used to classify traffic in traffic policies. Each packet is classified based on configured traffic matchers and put in a traffic class. A traffic class is used in a traffic policy to map a failover sequence to a traffic class.
A traffic matcher consists of a unique name (name) and a condition
(condition). The condition defines the matching criteria for the traffic
matcher. It is expressed as a boolean expression that then evaluates to either
true - the IP packet matches the traffic matcher - or false - the IP packet does
not match the traffic matcher. The expression consists of atoms and combinators.
The following atoms are supported:
BOOL=<true|false>Evaluates to the specified boolean value. This can be used to define a traffic matcher that always matches:
BOOL=truematches all packets. On the other hand,BOOL=falsematches no packets.SRC=<IP-prefix>Evaluates to true if
IP-prefixcontains the source IP address of the packet, otherwise false.DST=<IP-prefix>Evaluates to true if
IP-prefixcontains the destination IP address of the packet, otherwise false.SRCPORT=<port-range>Evaluates to true if the source port of the packet is in the range
port-range, otherwise false. This applies to TCP and UDP packets.DSTPORT=<port-range>Evaluates to true if the destination port of the packet is in the range
port-range, otherwise false. This applies to TCP and UDP packets.PROTOCOL=<protocol>Evaluates to true if the L4 protocol of the packet is
protocol, otherwise false. Valid strings forprotocolaretcp,udp, andicmp.DSCP=<dscp-value>Evaluates to true if the DSCP value of the packet is
dscp-value, otherwise false. The DSCP value must be specified as a hexadecimal number of the form0xYY.
The following combinators are supported:
ANY(cond1, cond2, ...)Evaluates to true if any of the
condconditions evaluates to true, otherwise false.ALL(cond1, cond2, ...)Evaluates to true if all of the
condconditions evaluate to true, otherwise false.NOT(cond)Evaluates to true if
condevaluates to false and vice versa.
Traffic Matchers
{
"traffic_matchers": [
{
"name": "all",
"description": "All traffic",
"condition": "BOOL=true"
},
{
"name": "ef_tagged",
"description": "Traffic tagged as Expedited Forwarding",
"condition": "DSCP=0x2e"
},
{
"name": "site1_to_site2",
"description": "Traffic from Site 1 to Site 2.",
"condition": "ALL(SRC=10.1.0.0/24, DST=10.2.0.0/24)"
}
]
}
Local Endpoint¶
The local endpoint configuration defines the local IP-in-SCION tunneling
endpoint. The local endpoint configuration contains the local network endpoint
on which IP packets are received and encapsulated. In addition to the local
ip (host) address, a set of ports for data, control, and probing
purposes can be configured. However, these ports do not need to be set
explicitly as the Anapaya EDGE appliance will automatically assign them if left
empty. Both, IPv4 and IPv6 addresses can be configured. Furthermore, SCION RSS
can be activated by setting enable_scion_rss if the local routers support
it. With SCION RSS activated, the gateway utilizes UDP source port entropy on
the underlay to enable RSS for SCION traffic, which can drastically increase
forwarding performance.
Additionally, the local endpoint configuration contains a set of
allowed_interfaces for each ISD-AS the appliance is part of, to be used by
the IP-in-SCION tunnels involving this appliance. This can be used to control
incoming traffic.
The appliance is announced only with the locally configured SCION interfaces.
Local Endpoint Configuration
Local endpoint configuration including the host IP address to bind to.
{
"endpoint": {
"enabled": true,
"ip": "10.8.0.1",
"enable_scion_rss": true
}
}
If you want to manually define the interfaces through which the Anapaya EDGE should be reachable, you can explicitly specify the set of interfaces that should be used. For example, if an Anapaya EDGE appliance has two SCION interfaces configured, with IDs 1 and 2, but it should be reachable only through interface 1, the configuration would look like this:
Local Endpoint Configuration With Allowed Interfaces
Local endpoint configuration including the host IP address to bind to and limiting the set of interfaces that can be used for the incoming IP-in-SCION tunnels.
{
"endpoint": {
"enabled": true,
"ip": "10.8.0.1",
"allowed_interfaces": [
{
"interfaces": [
1
],
"isd_as": "1-ff00:0:1"
}
],
"enable_scion_rss": true
}
}
Note
Please note that if you specify a set of allowed interfaces for an appliance, you need to also specify the same set of allowed interfaces in the tunneling section of the Cluster section of the peer appliances’ configuration.
Remote Endpoints¶
Remote IP-in-SCION tunneling endpoints are automatically discovered within their
respective SCION AS. However, the set of remote ASes which should be considered
for remote endpoint discovery needs to be statically provided. The remotes
section contains a list of all remote SCION ASes towards which the local
endpoint should establish IP-in-SCION tunnels.
Note
Routing Domains also include remote_isd_ases,
however, that section is used to limit the set of remote ASes to which a
domain applies and can include wildcards as well as allow and block lists.
If a remote SCION AS is not listed as part of the remotes section, it
will not be considered during remote endpoint discovery.
Remotes
Consider ASes 1-ff00:0:1 and 2-ff00:0:2 during remote endpoint
discovery.
{
"remotes": [
{
"isd_as": "1-ff00:0:1",
"description": "Branch office in Zurich"
},
{
"isd_as": "2-ff00:0:2",
"description": "Branch office in London"
}
]
}
Static Announcements¶
The IP prefixes that will be statically announced are defined in the
static_announcements section. Each entry defines a list of prefixes to
announce.
Note
The set of prefixes that are announced to any given remote tunnel endpoint is
not necessarily the statically defined prefixes list. Configured
announce-filters in each domain can limit the set of announced prefixes in a
given domain.
With static announcements, we need to ensure that the prefixes are announced
only if the appliance can actually reach the internal network of the customer.
For this reason, we need to enable next-hop tracking. Then, the prefixes are
only distributed if the specified target responds to ICMP ECHO requests.
This can be used to implement dynamically retractable routes without having to
resort to a dynamic routing protocol. It is not recommended to disable next-hop
tracking.
Static IP Prefixes
Statically announce 10.8.0.2/32 and 10.8.0.5/32 if 10.8.0.2
is reachable and unconditionally announce 10.8.0.1/32.
{
"static_announcements": [
{
"next_hop_tracking": {
"target": "10.8.0.2"
},
"prefixes": [
"10.8.0.2/32",
"10.8.0.5/32"
],
"sequence_id": 0
},
{
"prefixes": [
"10.8.0.1/32"
],
"sequence_id": 1
},
]
}
Use Cases¶
This section contains common IP-in-SCION tunneling use cases and how they can be implemented using the Anapaya EDGE appliance’s domain-based routing configuration. There are of course many more use cases and implementation options. The examples below are intended to illustrate common use cases and should serve the reader as a basis for implementation and adaptation.
Company WAN and Public Cloud SaaS Access¶
In this use case, the operator has two Anapaya EDGE appliances (edge1 and edge2) in two local data center sites, each with one SCION access link to an upstream SCION ISP. This is a typical deployment for a regional or global headquarter site (HQ Zurich). The operator has to consider the company’s remote sites (Branch Paris and Branch London) that are part of the company’s wide area network (WAN). Furthermore, the operator wants to configure access to two Software-as-a-Service (SaaS) applications that run in a public cloud. As the public cloud is not directly SCION-enabled, the company makes use of two service providers that provide SCION-enabled access to the public cloud (Cloud Access Provider). Please refer to the figure when working through the examples.
This use case can be naturally implemented with two domains: one domain for the company’s WAN and a separate domain to configure public cloud access.
Domains Configuration Skeleton
"domains": [
{
"name": "company_wan",
"description": "The company's wide area network.",
},
{
"name": "public_cloud",
"description": "Access to SaaS applications in a public cloud.",
}
]
Domain Entities¶
The first step is to define what entities are part of the domains, i.e., announced and accepted IP prefixes and remote SCION ASes.
Let us first focus on the company_wan domain and assume the following
requirements:
Possible IP ranges for the company’s WAN:
10.0.0.0/8and192.168.0.0/16.The local sites should only announce prefixes in
10.1.0.0/16and10.2.0.0/16Only the remote sites
2-ff00:0:2(Branch Paris) and2-ff00:0:3(Branch London) should be part of thecompany_wandomain.IP-in-SCION tunnels should be established to both these branches from HQ Zurich.
Putting all this together, we get the following configuration for the
company_wan domain:
Company WAN Domain with Configured Remotes
"scion_tunneling": {
"domains": [
{
"name": "company_wan",
"description": "The company's wide area network.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["10.0.0.0/8", "192.168.0.0/16"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"prefixes": ["10.1.0.0/16", "10.2.0.0/16"],
"action": "ACCEPT",
"sequence_id": 0
}
]
},
"remote_isd_ases": [
{
"isd_as": "2-ff00:0:2",
"action": "ACCEPT",
"sequence_id": 0
},
{
"isd_as": "2-ff00:0:3",
"action": "ACCEPT",
"sequence_id": 1
}
]
}
],
"remotes": [
{
"isd_as": "2-ff00:0:2",
"description": "Branch office in Paris"
},
{
"isd_as": "2-ff00:0:3",
"description": "Branch office in London"
}
]
}
Next, the operator needs to define the what entities that are part of the
public_cloud domain. The following requirements are assumed:
The public cloud is accessible via the Cloud Access Providers (CAP)
2-ff00:0:100,1-ff00:0:200, and2-ff00:0:200. Note that the latter two ASes are distinct on the SCION level, but obviously they belong to the same CAP.The IP prefix assigned app1 is
1.0.1.0/24and the one assigned to app2 is1.0.2.0/24, therefore these prefixes must be accepted.All internal hosts are NATed to an IP in
2.0.0.0/24, i.e., this is the IP prefix that needs to be announced in thepublic_clouddomain, such that the return traffic can be correctly routed from the public cloud to the local site.
Putting all this together, we get the following configuration for the
public_cloud domain:
Public Cloud Domain Entities
"scion_tunneling": {
"domains": [
{
"name": "public_cloud",
"description": "Access to SaaS applications in a public cloud.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["1.0.1.0/24", "1.0.2.0/24"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"prefixes": ["2.0.0.0/24"],
"action": "ACCEPT",
"sequence_id": 0
}
]
},
"remote_isd_ases": [
{
"isd_as": "1-ff00:0:100",
"action": "ACCEPT",
"sequence_id": 0
},
{
"isd_as": "0-ff00:0:200",
"action": "ACCEPT",
"sequence_id": 1
}
]
}
],
"remotes": [
{
"isd_as": "1-ff00:0:100",
"description": "Access provider 1 to public cloud"
},
{
"isd_as": "1-ff00:0:200",
"description": "Access provider 2 to public cloud (in ISD 1)"
},
{
"isd_as": "2-ff00:0:200",
"description": "Access provider 2 to public cloud (in ISD 2)"
}
]
}
Note the use of the ISD wildcard 0-ff00:0:200 in the remote_isd_ases
section. This matches the AS identifier ff00:0:200 in every ISD, however,
since the operator only explicitly adds the ASes 1-ff00:0:200 and
2-ff00:0:200 in the remotes section, only those will be used to
establish tunnels with.
Announced Prefixes¶
The prefixes defined in the announce_filter of the domains are not the
prefixes that are necessarily announced. Instead, these are filters applied on
statically configured and dynamically learned prefixes (via BGP). In this
example, both Anapaya EDGE appliances are connected internally to a BGP peer
that announces prefixes to and accepts prefixes from the Anapaya EDGE
appliances. To learn how to configure BGP peers on an Anapaya EDGE appliance,
please refer to the Border Gateway Protocol (BGP) section.
Traffic Policies¶
Besides defining the entities that are part of the domains, the operator also needs to configure at least one traffic policy per domain.
Note
Often, a permissive traffic policy, i.e., a traffic policy that does not limit the set of acceptable SCION paths too much, is advantageous. The Anapaya EDGE appliance has a powerful path optimization engine that can ensure optimal path selection amongst the set of acceptable paths.
The requirements are as follows:
Generally, reliability and performance should be maximized.
app1 must only be accessible via paths within ISD 1 as it deals with highly sensitive and regulated data.
There are no specific requirements for app2.
Note
The operator has little influence over the traffic policies used by the remote endpoints. To ensure that the traffic for app1 also only uses paths within ISD 1 on the return path, the operator needs to coordinate that with the remote endpoint. If the operator also controls the remote endpoint then this is not an issue.
Given these requirements, the operator first needs to define three traffic matchers: one that matches traffic belonging to app1, one that matches traffic belonging to app2, and a third that matches any traffic.
Traffic Matchers
"traffic_matchers": [
{
"name": "app1",
"description": "Traffic belonging to app1",
"condition": "DST=1.0.1.0/24"
},
{
"name": "app2",
"description": "Traffic belonging to app2",
"condition": "DST=1.0.2.0/24"
},
{
"name": "all",
"description": "All traffic",
"condition": "BOOL=true"
}
]
Furthermore, two path filters need to be defined - one that only allows paths within ISD 1 and one that allows all paths.
Path Filters
"path_filters": [
{
"name": "isd_1_only",
"description": "Only allow paths within ISD 1",
"acl": ["+ 1-0", "-"]
},
{
"name": "allow_all",
"description": "Allow all paths",
"acl": ["+ 0"]
}
]
Having defined the traffic matchers and path filters, the operator can now assemble the traffic policies for the domains.
Traffic Policies
"domains": [
{
"name": "company_wan",
"traffic_policies": [
{
"sequence_id": 0,
"description": "Allow all traffic to use all available paths.",
"traffic_matcher": "all",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "allow_all"
}
]
}
]
},
{
"name": "public_cloud",
"traffic_policies": [
{
"sequence_id": 0,
"description": "Traffic belonging to app1 can only use paths within ISD 1.",
"traffic_matcher": "app1",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "isd_1_only"
}
]
},
{
"sequence_id": 1,
"description": "Traffic belonging to app2 can use all paths",
"traffic_matcher": "app2",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "allow_all"
}
]
}
]
}
]
Local Endpoint¶
To finalize the IP-in-SCION tunneling configuration, the operator needs to
configure the local tunneling endpoint for each Anapaya EDGE appliance. This is
the part of the configuration that will be different for each appliance. Besides
the host address to which the tunneling endpoint is bound, the operator also
wants to control incoming traffic such that traffic coming via SCION interface 1
is always handled by edge1, and traffic coming via SCION interface 2 is
always handled by edge2. To that end, the operator needs to specify
allowed_interfaces accordingly.
Local Endpoint Configuration
edge1:
"endpoint": {
"ip": "10.1.0.1",
"enabled": true
},
edge2:
"endpoint": {
"ip": "10.1.0.2",
"allowed_interfaces": [
{
"interfaces": [
2
],
"isd_as": "1-ff00:1:1"
}
],
"enabled": true
}
Complete Configuration¶
For completeness the entire scion_tunneling configurations for
edge1 is shown below. The one for edge2 only differs in the
endpoint section.
IP-in-SCION tunneling configuration for edge1
{ "scion_tunneling": { "endpoint": { "ip": "10.1.0.1", "enabled": true }, "domains": [ { "name": "company_wan", "description": "The company's wide area network.", "prefixes": { "accept_filter": [ { "prefixes": ["10.0.0.0/8", "192.168.0.0/16"], "action": "ACCEPT", "sequence_id": 0 } ], "announce_filter": [ { "prefixes": ["10.1.0.0/16", "10.2.0.0/16"], "action": "ACCEPT", "sequence_id": 0 } ] }, "remote_isd_ases": [ { "isd_as": "2-ff00:0:2", "action": "ACCEPT", "sequence_id": 0 }, { "isd_as": "2-ff00:0:3", "action": "ACCEPT", "sequence_id": 1 } ], "traffic_policies": [ { "sequence_id": 0, "description": "Allow all traffic to use all available paths.", "traffic_matcher": "all", "failover_sequence": [ { "sequence_id": 0, "path_filter": "allow_all" } ] } ] }, { "name": "public_cloud", "description": "Access to SaaS applications in a public cloud.", "prefixes": { "accept_filter": [ { "prefixes": ["1.0.1.0/24", "1.0.2.0/24"], "action": "ACCEPT", "sequence_id": 0 } ], "announce_filter": [ { "prefixes": ["2.0.0.0/24"], "action": "ACCEPT", "sequence_id": 0 } ] }, "remote_isd_ases": [ { "isd_as": "1-ff00:0:100", "action": "ACCEPT", "sequence_id": 0 }, { "isd_as": "0-ff00:0:200", "action": "ACCEPT", "sequence_id": 1 } ], "traffic_policies": [ { "sequence_id": 0, "description": "Traffic belonging to app1 can only use paths within ISD 1.", "traffic_matcher": "app1", "failover_sequence": [ { "sequence_id": 0, "path_filter": "isd_1_only" } ] }, { "sequence_id": 1, "description": "Traffic belonging to app2 can use all paths", "traffic_matcher": "app2", "failover_sequence": [ { "sequence_id": 0, "path_filter": "allow_all" } ] } ] } ], "remotes": [ { "isd_as": "2-ff00:0:2", "description": "Branch office in Paris" }, { "isd_as": "2-ff00:0:3", "description": "Branch office in London" }, { "isd_as": "1-ff00:0:100", "description": "Access provider 1 to public cloud" }, { "isd_as": "1-ff00:0:200", "description": "Access provider 2 to public cloud (in ISD 1)" }, { "isd_as": "2-ff00:0:200", "description": "Access provider 2 to public cloud (in ISD 2)" } ], "traffic_matchers": [ { "name": "app1", "description": "Traffic belonging to app1", "condition": "DST=1.0.1.0/24" }, { "name": "app2", "description": "Traffic belonging to app2", "condition": "DST=1.0.2.0/24" }, { "name": "all", "description": "All traffic", "condition": "BOOL=true" } ], "path_filters": [ { "name": "isd_1_only", "description": "Only allow paths within ISD 1", "acl": ["+ 1-0", "-"] }, { "name": "allow_all", "description": "Allow all paths", "acl": ["+ 0"] } ] } }
Multi-ISD Anapaya EDGE Setup¶
This example shows a single Anapaya EDGE appliance (edge) that is part of two ISDs. One use case for this setup is having an appliance that is part of an ISD that is completely disconnected from the rest of the public SCION Internet, but needs to be remotely managed. In this case, a second ISD that is connected to the public SCION Internet needs to be configured on the appliance, such that it can be remotely managed. Another use case is if the Anapaya EDGE appliance has two upstream SCION ISPs that are in different ISDs themselves. This is only possible if the Anapaya EDGE appliance is also part of both those ISDs.
The following example shows an Anapaya EDGE appliance that is part of two separate and isolated ISDs (ISD 1 and ISD 2). The requirements are as follows:
Only IP prefixes in
1.0.0.0/8are announced and accepted to and from ISD 1.Only IP prefixes in
2.0.0.0/8are announced and accepted to and from ISD 2.Only remote ASes that are part of the corresponding ISD can announce these prefixes to the local Anapaya EDGE appliance.
The Anapaya EDGE appliance should use its local AS identity that corresponds the ISD when communicating with other IP-in-SCION tunneling endpoints in the same ISD.
IP prefix
1.0.1.0/24should be announced to ISD 1 and2.0.1.0/24to ISD 2 if (and only if) internally the firewall at192.168.2.100is reachable.
Given these requirements, the operator defines two routing domains: one for ISD 1 and one for ISD 2.
Domains Configuration Skeleton
"domains": [
{
"name": "isd_1",
"description": "Domain for communication in ISD 1.",
},
{
"name": "isd_2",
"description": "Domain for communication in ISD 2.",
]
}
Domain Entities¶
The first step is to define what entities are part of the domains, i.e., announced and accepted IP prefixes and remote SCION ASes.
Let us first focus on the isd_1 domain. As described above, the operator
wants to ensure that only IP prefixes in 1.0.0.0/8 are announced to and
accepted from remote IP-in-SCION tunneling endpoints in ISD 1. Furthermore, the
operator wants to limit the set of remote ASes to those that are part of ISD 1
and use the local AS identity in ISD 1 for communication in this domain.
Finally, the remote site Remote 1 (1-ff00:0:2) needs to be explicitly
added to the remotes section such that the Anapaya EDGE appliance will establish
IP-in-SCION tunnels to endpoints in Remote 1.
ISD1 Domain with Configured Remotes
"scion_tunneling": {
"domains": [
{
"name": "isd_1",
"description": "Domain for communication in ISD 1.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["1.0.0.0/8"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"prefixes": ["1.0.0.0/8"],
"action": "ACCEPT",
"sequence_id": 0
}
]
},
"local_isd_ases": ["1-ff00:0:1"],
"remote_isd_ases": [
{
"isd_as": "1-0",
"action": "ACCEPT",
"sequence_id": 0
}
]
}
],
"remotes": [
{
"isd_as": "1-ff00:0:2",
"description": "Remote AS in ISD 1"
]
}
}
The isd_2 domain is configured similarly, with the straightforward adaption
of the configuration to the requirements of ISD 2.
ISD2 Domain with Configured Remotes
"scion_tunneling": {
"domains": [
{
"name": "isd_2",
"description": "Domain for communication in ISD 2.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["2.0.0.0/8"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"prefixes": ["2.0.0.0/8"],
"action": "ACCEPT",
"sequence_id": 0
}
]
},
"local_isd_ases": ["2-ff00:0:1"],
"remote_isd_ases": [
{
"isd_as": "2-0",
"action": "ACCEPT",
"sequence_id": 0
}
]
}
],
"remotes": [
{
"isd_as": "2-ff00:0:3",
"description": "Remote AS in ISD 2"
}
}
}
Announced Prefixes¶
As described in the introduction, the Anapaya EDGE appliance should announce the
local prefixes 1.0.1.0/24 in ISD 1 and 2.0.1.0/24 in ISD 2 if internally
the firewall at 192.168.2.100 is reachable. This can be accomplished by
configuring the static_announcements section with next_hop_tracking
activated. Note that the static_anouncements section contains both IP
prefixes. The announce_filter in the corresponding domain configurations
ensure that 1.0.1.0/24 is announce in ISD 1 and 2.0.1.0/24 in ISD 2.
Static Announcements
"static_announcements": [
{
"next_hop_tracking": {
"target": "192.168.2.100"
},
"prefixes": ["1.0.1.0/24", "2.0.1.0/24"],
"sequence_id": 0
}
]
}
Traffic Policies¶
Besides defining the entities that are part of the domains, the operator also needs to configure at least one traffic policy per domain.
Note
Often, a permissive traffic policy, i.e., a traffic policy that does not limit the set of acceptable SCION paths too much, is advantageous. The Anapaya EDGE appliance has a powerful path optimization engine that can ensure optimal path selection amongst the set of acceptable paths.
In this example, the operator wants to ensure that traffic for destinations in ISD 1 only uses paths in ISD 1 and similarly for destinations in ISD 2. Besides that, the operator does not want to further limit the set of acceptable paths to give the Anapaya EDGE appliance the biggest possible path selection scope.
Note
If the two ISDs are truly isolated from the rest of the SCION Internet, limiting the set of acceptable paths to those in the same ISD is not required as no other paths exist anyway. Nevertheless, it is good practice to be specific about the set of acceptable paths.
To implement these requirements, the operator first needs to define one traffic matcher to match all traffic and two path filters to match paths in ISD 1 and ISD 2 respectively.
Traffic Matchers and Path Filters
"traffic_matchers": [
{
"name": "all",
"description": "All traffic",
"condition": "BOOL=true"
}
],
"path_filters": [
{
"name": "isd_1_only",
"description": "Only allow paths within ISD 1",
"acl": ["+ 1-0", "-"]
},
{
"name": "isd_2_only",
"description": "Only allow paths within ISD 2",
"acl": ["+ 2-0", "-"]
}
}
Using the defined traffic matcher and path filters, the operator can define the traffic policies for each domain:
Traffic Policies
"domains": [
{
"name": "isd_1",
"traffic_policies": [
{
"sequence_id": 0,
"description": "All traffic must only use paths within ISD 1.",
"traffic_matcher": "all",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "isd_1_only"
}
]
}
]
},
{
"name": "isd_2",
"traffic_policies": [
{
"sequence_id": 0,
"description": "All traffic must only use paths within ISD 2.",
"traffic_matcher": "all",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "isd_2_only"
}
]
}
]
}
}
Local Endpoint¶
To finalize the IP-in-SCION tunneling configuration, the operator needs to configure the local tunneling endpoint for the Anapaya EDGE appliance. Since the operator does not need to control incoming traffic, the only required configuration is to specify the host address to bind to and to enable the IP-in-SCION tunneling endpoint.
Local Endpoint Configuration
"endpoint": {
"ip": "192.168.2.1",
"enabled": true
},
Complete Configuration¶
For completeness the entire scion_tunneling configurations for
edge is shown below.
Complete IP-in-SCION tunneling configuration
{ "scion_tunneling": { "endpoint": { "ip": "192.168.2.1", "enabled": true }, "domains": [ { "name": "isd_1", "description": "Domain for communication in ISD 1.", "prefixes": { "accept_filter": [ { "prefixes": ["1.0.0.0/8"], "action": "ACCEPT", "sequence_id": 0 } ], "announce_filter": [ { "prefixes": ["1.0.0.0/8"], "action": "ACCEPT", "sequence_id": 0 } ] }, "local_isd_ases": ["1-ff00:0:1"], "remote_isd_ases": [ { "isd_as": "1-0", "action": "ACCEPT", "sequence_id": 0 } ], "traffic_policies": [ { "sequence_id": 0, "description": "All traffic must only use paths within ISD 1.", "traffic_matcher": "all", "failover_sequence": [ { "sequence_id": 0, "path_filter": "isd_1_only" } ] } ] }, { "name": "isd_2", "description": "Domain for communication in ISD 2.", "prefixes": { "accept_filter": [ { "prefixes": ["2.0.0.0/8"], "action": "ACCEPT", "sequence_id": 0 } ], "announce_filter": [ { "prefixes": ["2.0.0.0/8"], "action": "ACCEPT", "sequence_id": 0 } ] }, "local_isd_ases": ["2-ff00:0:1"], "remote_isd_ases": [ { "isd_as": "2-0", "action": "ACCEPT", "sequence_id": 0 } ], "traffic_policies": [ { "sequence_id": 0, "description": "All traffic must only use paths within ISD 2.", "traffic_matcher": "all", "failover_sequence": [ { "sequence_id": 0, "path_filter": "isd_2_only" } ] } ] } ], "remotes": [ { "isd_as": "1-ff00:0:2", "description": "Remote AS in ISD 1" }, { "isd_as": "2-ff00:0:3", "description": "Remote AS in ISD 2" } ], "traffic_matchers": [ { "name": "all", "description": "All traffic", "condition": "BOOL=true" } ], "path_filters": [ { "name": "isd_1_only", "description": "Only allow paths within ISD 1", "acl": ["+ 1-0", "-"] }, { "name": "isd_2_only", "description": "Only allow paths within ISD 2", "acl": ["+ 2-0", "-"] } ], "static_announcements": [ { "next_hop_tracking": { "target": "192.168.2.100" }, "prefixes": ["1.0.1.0/24", "2.0.1.0/24"], "sequence_id": 0 } ] } }
Configuring Egress Source NAT if only few public IP addresses are available¶
In some cases, the available IP addresses that an appliance can announce to remote IP-in-SCION tunneling endpoints are limited, e.g., if a customer only has a single public IP address assigned by its ISP. To allow multiple internal hosts to share a single IP address, the appliance can be configured to perform source NATing on the outgoing traffic. This way, the appliance can use a single IP address to represent multiple internal hosts.
Note
It is possible and also used in practice to announce private IP ranges to a remote IP-in-SCION tunneling endpoint. However, sometimes it is simpler to limit the set of announced prefixes to public IP ranges because then there is no coordination needed with the remote network to ensure that the announced IP ranges do not clash with the remote network’s internal IP ranges.
Let us assume that the operator has only a single public IP address available
203.0.113.50/32 but wants to connect multiple internal hosts to the remote
while only announcing the single public IP address. Let us further assume that
the internal hosts are in the range 192.168.1.0/24. The operator can use the
following configuration to achieve this:
Egress NAT Configuration
"interfaces": {
"ethernets": [
{
"name": "lan0",
"addresses": ["192.168.1.1/24"],
"driver": "VPP_DPDK"
},
{
"name": "wan0",
"addresses": ["169.254.0.2/31"],
"driver": "VPP_DPDK"
}
}
],
"loopbacks": [
{
"name": "loop0",
"addresses": ["203.0.113.50/32"]
}
]
},
"nat": {
"snat": {
"address_pool": ["203.0.113.50/32"],
"exclude": [],
"interfaces": ["scion-gateway"]
}
}
The lan0 interface is configured with an address of the internal private IP
range. Note that the public IP address is added on a loopback interface and NOT
on the wan0 interface. This is because on the wan0 interface we have the
IP underlay for the SCION access link (usually provided by the upstream SCION
ISP). The public IP address is only used for IP-in-SCION tunneling.
The NAT configuration specifies the public IP address that should be used for
source NATing and the interface on which the NAT should be applied to. In this
case the address pool to use is the single public IP address 203.0.113.50/32
and the NAT should be applied to the special scion-gateway interface to
apply it to all IP-in-SCION tunneling traffic.
Warning
When using egress NAT it is not possible by default to create IP connections from a host in a remote AS to a host in the local AS that would go through the IP-in-SCION tunnel. Using NAT means that only outgoing connections are possible. If incoming connections are needed, the appliance needs to be configured to exclude the IP addresses that should not be NATed.
Configuring Ingress NAT to collect Users from the Anapaya GATE¶
In this use case, the operator wants to ensure that traffic streams coming from an IP-in-SCION tunnel from a remote Anapaya GATE are returned via the local Anapaya EDGE appliance, even though the remote IP prefixes are also reachable via the legacy Internet.
To address this use case, the EDGE appliance can do NATing on the incoming traffic. Source addresses in packets arriving from the tunnel are rewritten by one of the addresses from the NAT’s address pool.
The operator is responsible for setting up the static routes so that replies to the addresses in the NAT’s address pool are routed back to the appliance. This way the operator can ensure that replies to the packets arriving via the tunnel are routed back to the tunnel, even if the remote IP prefixes are also reachable via the legacy Internet.
If, on the other hand, a packet arrives through the Internet it keeps its original source address and the reply is routed back through the legacy Internet. Again, it is up to the operator to set up the routes in the local network accordingly.
Ingress NAT Configuration
"nat": {
"snat": {
"address_pool": ["192.168.0.1/32"],
"exclude": [],
"interfaces": ["eth0"]
}
}
Note
Ingress NAT does not work with BGP enabled in the appliance.
Warning
When using ingress NAT it is not possible by default to create connections from the local AS to the remote AS that would go through the IP-in-SCION tunnel. Using NAT means that there are no fixed IP addresses to connect to. If this is needed, a specific IP address, or a range of addresses, can be excluded from the NAT. The packet will then arrive from the tunnel with its original source address unmodified. In this case it’s up to the operator to route the packets destined to that address to the SCION tunnel rather than routing them through the legacy Internet.