What Is ExpressRoute in Azure and How Does It Work?

ExpressRoute is a direct, dedicated connection between your organization's premises and Microsoft's global network, allowing for fast and reliable access to Azure services.

This connection is made possible through a partnership with a variety of network service providers, such as Equinix and Verizon.

ExpressRoute enables you to bypass the public internet, reducing latency and increasing security.

Your organization can establish multiple connections to Azure, each with its own dedicated bandwidth and latency characteristics.

Broaden your view: Azure Front Door Latency

ExpressRoute is a fast and secure way to connect your on-premises network to Microsoft Azure. It's a dedicated, private connection that provides low latency and high throughput.

ExpressRoute is ideal for organizations that need to transfer large amounts of data between their on-premises network and Azure. With ExpressRoute, you can access Azure resources quickly and securely, without having to go over the public internet.

ExpressRoute is a way to create a dedicated, high-bandwidth connection between your organization's network and Microsoft's cloud services. This connection is typically faster and more reliable than the internet.

ExpressRoute offers a range of benefits, including lower latency and higher security. You can use it to connect to Microsoft Azure, Office 365, and other cloud services.

ExpressRoute is available in multiple regions around the world, but it's essential to understand the geopolitical regions where your circuits are located. This will help you determine if you need ExpressRoute Premium for connectivity between circuits.

ExpressRoute Premium is a feature that enables connectivity between ExpressRoute circuits in different geopolitical regions. If your circuits are in the same region, you don't need Premium for connectivity.

You might enjoy: Latency between Azure Regions

You can have ExpressRoute circuits with multiple service providers. Each circuit is associated with one service provider only.

You can have multiple ExpressRoute circuits with the same or different service providers. This is particularly useful if you have multiple peering locations in a metro.

If the circuits are created at different peering locations, you can link them to the same virtual network. This allows for greater flexibility and scalability.

If the circuits are created at the same peering location, you can link up to four circuits to the same virtual network. This can be a big help if you're working with multiple service providers.

Redundancy is built into every ExpressRoute peering location for higher reliability. Each circuit consists of two connections to two Microsoft Enterprise edge routers (MSEEs) at an ExpressRoute Location from the connectivity provider or your network edge.

Connectivity providers use redundant devices to ensure that your connections are handed off to Microsoft in a redundant manner. You can choose not to deploy redundant devices/Ethernet circuits at your end, but this is not recommended.

Multiple ExpressRoute circuits from different peering locations or up to four connections from the same peering location can be connected to the same virtual network to provide high-availability in the case a single circuit becomes unavailable. You can then assign higher weights to one of the local connections to prefer a specific circuit.

Here are some key redundancy and resiliency facts:

In the event of a single circuit failure, a redundant connection is available to support the load of your network and provide high availability of your ExpressRoute circuit.

ExpressRoute connections offer several key benefits that make them an attractive option for businesses. One of the most significant advantages is built-in redundancy in every peering location, ensuring higher reliability.

ExpressRoute connections provide Layer 3 connectivity between your on-premises network and the Microsoft Cloud through a connectivity provider, which can be from an any-to-any (IPVPN) network, a point-to-point Ethernet connection, or through a virtual cross-connection via an Ethernet exchange.

ExpressRoute connections offer dynamic routing between your network and Microsoft via BGP, ensuring that your data is transmitted efficiently. This is especially important for businesses that rely on real-time communication.

ExpressRoute connections also provide a connection uptime SLA, giving you peace of mind knowing that your connection is reliable.

Here are some of the key benefits of ExpressRoute connections:

ExpressRoute connections also offer QoS support for Skype for Business, ensuring that your communication is clear and uninterrupted.

To implement redundancy for private peering, you can connect multiple ExpressRoute circuits from different peering locations or up to four connections from the same peering location to the same virtual network. This provides high-availability in case a single circuit becomes unavailable.

Discover more: Azure Vnet Peering

You can then assign higher weights to one of the local connections to prefer a specific circuit. It's recommended that your setup has at least two ExpressRoute circuits to avoid single points of failure.

Multiple ExpressRoute circuits can be connected to the same virtual network to provide high-availability. This setup can be achieved by connecting circuits from different peering locations or from the same peering location with up to four connections.

This setup allows for circuit-level resilience and can be used to avoid single points of failure.

Here are the ways to implement redundancy for private peering:

By implementing private peering redundancy, you can ensure high-availability and avoid single points of failure in your ExpressRoute setup.

ExpressRoute allows you to connect to Microsoft cloud services from your own network, providing a fast and secure way to access Azure services.

You can connect to Microsoft from one of our peering locations and access regions within the same geopolitical region. For example, connecting to Microsoft in Amsterdam through ExpressRoute gives you access to all of Microsoft cloud services hosted in North and West Europe.

ExpressRoute connectivity enables access to the following services: Microsoft Azure services and Microsoft 365 services. Microsoft 365 was created to be accessed securely and reliably via the Internet, but ExpressRoute is recommended for specific scenarios.

ExpressRoute premium offers increased routing table limits, allowing for up to 10,000 routes for private peering, and increased numbers of VNets and ExpressRoute Global Reach connections.

Here's a breakdown of the geopolitical regions and their associated Microsoft cloud regions:

This allows you to connect to Microsoft cloud services across geopolitical boundaries with ExpressRoute Premium.

Connecting your on-premises location to Azure requires a few key components. You'll need two routers, ideally in two different locations, supporting BGP sessions. This is because ExpressRoute relies on Border Gateway Protocol (BGP) to establish and manage connections.

To establish the peering, you'll need 2 IPv4 /30 peering and an AS number for your network. The AS number can be either 16 or 32-bit, public or private. You'll also need a VLAN ID to establish the peering and one or more prefixes to announce to the Azure side.

The bandwidth for your ExpressRoute circuit can range from 50 Mbps to 10 Gbps, depending on your needs. For ExpressRoute Direct, you'll need to set up a Point of Presence (PoP) in the same location as Azure, and use two 10 Gbps or two 100 Gbps ports to connect to the Microsoft backbone.

Here are the technical requirements for connecting your on-premises location to Azure:

These requirements apply to both ExpressRoute and ExpressRoute Direct, and are essential for establishing a reliable and high-performance connection between your on-premises location and Azure.

Local connectivity is a game-changer for businesses that need to transfer data cost-effectively. You can enable the Local SKU to bring your data to an ExpressRoute location near the Azure region you want.

With Local SKU, data transfer is included in the ExpressRoute port charge. This means you won't have to worry about extra costs for data transfer.

ExpressRoute Local is a SKU of ExpressRoute circuit, in addition to Standard and Premium SKUs. It gives you access to one or two Azure regions in or near the same metro.

You can only advertise routes over Microsoft and private peering from the corresponding local region of the ExpressRoute circuit with a Local SKU. This means you won't receive routes for other regions different than the defined local region.

ExpressRoute Local might not be available for an ExpressRoute Location. For peering location and supported Azure local region, check out the locations and connectivity providers.

To connect your on-premises location to Azure, you'll need to meet the technical requirements listed on the ExpressRoute prerequisites page. This includes having two routers supporting BGP sessions, 2 IPv4 /30 peering for the peering, and an AS number for your network.

ExpressRoute provides two main options for connecting on-premises to Azure: ExpressRoute and ExpressRoute Direct. ExpressRoute allows you to delegate connectivity to an operator, such as Equinix or BT, while ExpressRoute Direct enables direct connection to Microsoft's global network.

You'll need to choose the right ExpressRoute SKU for your needs, with options including Standard, Premium, and Local. The Local SKU provides access to only one or two Azure regions in or near the same metro, making it a cost-effective option for data transfer.

To establish a connection, you'll need to configure BGP peerings and link your virtual network to the ExpressRoute circuit. You can also connect multiple virtual networks to a single ExpressRoute circuit.

Here are the technical requirements for connecting your on-premises location to Azure:

Note that ExpressRoute Direct requires a more significant investment, with dual 100-Gbps connectivity and a fixed billing model. However, it provides massive data ingestion into services like Azure Storage and Azure Cosmos DB, as well as physical isolation for regulated industries.

ExpressRoute offers flexible billing models to suit your needs. You can choose from unlimited data, metered data, or ExpressRoute premium add-on, each with its own pricing structure.

Unlimited data billing includes all inbound and outbound data transfer in the monthly fee, while metered data charges only for outbound data transfer. Data transfer rates vary by region.

ExpressRoute premium add-on provides additional capabilities, but you'll pay for it on top of standard ExpressRoute charges and connectivity provider charges.

Consider reading: Add Azure

ExpressRoute Direct is billed for the port pair at a fixed amount, with standard circuits included at no extra charge and premium circuits having a slight add-on charge. Egress is billed on a per circuit basis based on the zone of the peering location.

ExpressRoute Global Reach is billed separately from the existing ExpressRoute service, with an add-on fee for enabling this feature on each ExpressRoute circuit. Traffic between your on-premises networks gets billed for an egress rate at the source and an ingress rate at the destination, based on the zone at which the circuits are located.

Here's a summary of the ExpressRoute billing models:

There's no limit on the amount of data you can transfer using ExpressRoute, so you don't have to worry about hitting a cap.

To learn more about pricing and bandwidth rates, you can refer to the pricing details.

The public IP address used for internal management doesn't pose a security risk to your virtual network.

Virtual networks connected to an ExpressRoute circuit can be advertised to on-premises, but the ExpressRoute gateway only advertises the virtual network address space, not individual subnets.

ExpressRoute has a maximum of 1000 IPv4 prefixes advertised on a single connection, or through virtual network peering using gateway transit. This limit includes prefixes from peered virtual networks.

You can increase this limit to 10,000 routes for private peering when using ExpressRoute premium.

For dual-stack circuits, there's a maximum of 100 IPv6 prefixes on a single ExpressRoute connection, or through virtual network peering using gateway transit.

To access Azure public services like Azure Storage or Microsoft 365, you can use Microsoft peering, which provides access to public IP address ranges used in Azure over the circuit.

Microsoft peering supports services like Microsoft 365, Power BI, and Azure Public IP addresses for IaaS, among others.

To implement redundancy for Microsoft peering, it's recommended to have multiple circuits in different peering locations to avoid single points of failure.

On a similar theme: Azure Ip Ranges

You can either advertise the same prefix on both circuits and use AS PATH prepending or advertise different prefixes to determine path from on-premises.

To manage the risk of exceeding prefix limits when advertising prefixes received over Microsoft Peering to Private Peering, you can configure Service Health to review prefix updates, implement monitoring to detect changes, and consider upgrading the SKU or summarizing routes.

You might like: Azure Networking Learning Path

ExpressRoute circuit bandwidth is duplex, meaning it's allocated for both ingress and egress traffic.

This means you're getting 200 Mbps for ingress traffic and 200 Mbps for egress traffic if you purchase a 200-Mbps ExpressRoute circuit.

You can think of it like a two-way street, where data flows in and out at the same speed.

This separate allocation ensures that your data can move in and out quickly and efficiently, without any bottlenecks.

This is especially important for businesses that rely on fast and reliable data transfer, like online retailers or cloud service providers.

For more insights, see: Traffic Manager in Azure

If you exceed the prefix limit on an ExpressRoute connection, the connection between the ExpressRoute circuit and the gateway will disconnect, affecting peered virtual networks using gateway transit. This means you'll lose connectivity until the prefix limit is no longer exceeded.

ExpressRoute supports up to 11,000 routes, covering virtual network address spaces, on-premises networks, and virtual network peering connections. To avoid this issue, you can update the prefixes to be within the allowed range.

To make this change, navigate to the Advisor resource, select the "Performance" pillar, and click on the recommendation for "Max prefix reached for ExpressRoute Gateway." Then, select the Gateway with this recommendation and reduce the advertised address space to within the limit.

Here's a summary of the prefix limits for ExpressRoute connections:

By understanding these prefix limits, you can avoid connectivity issues and ensure a smooth ExpressRoute experience.

Peering is a crucial aspect of ExpressRoute, allowing you to connect your on-premises network to Azure and access Azure public services. You can advertise your virtual network address space on ExpressRoute private peering, but you can't include or exclude subnets.

The maximum number of IPv4 prefixes you can advertise on a single ExpressRoute connection is 1000. If you have multiple virtual networks connected to the same ExpressRoute circuit, the total number of prefixes advertised will be the sum of each virtual network's prefixes.

For Microsoft peering, the maximum number of IPv4 prefixes you can advertise is 200. However, if you're using ExpressRoute premium, you can increase the limit to 10,000 prefixes for private peering.

To design for high availability, it's recommended to have at least two ExpressRoute circuits in different peering locations. This will ensure that if one circuit becomes unavailable, your network will still have connectivity to Azure.

Here's a summary of the maximum number of prefixes you can advertise on ExpressRoute:

Note that these limits apply to the total number of prefixes advertised, not the number of virtual networks or subnets.

ExpressRoute supports a maximum TCP and UDP packet size of 1,400 bytes. Packet sizes larger than 1,400 bytes will get fragmented.

For another approach, see: Azure 400 Certification

ExpressRoute can handle a maximum of 1,000,000 packets per second, with some gateway SKUs supporting up to 1,000,000 packets per second.

The number of supported VMs in a virtual network varies depending on the gateway SKU, with some supporting up to 11,000 VMs.

Here is a summary of ExpressRoute's performance and capacity limits:

ExpressRoute can facilitate up to 11,000 routes that span virtual network address spaces, on-premises networks, and any relevant virtual network peering connections.

ExpressRoute Traffic Collector can handle up to 300,000 flows a minute. This is a significant capacity, but it's essential to remember that excess flows are dropped when this limit is reached.

This means you should carefully plan and monitor your network traffic to avoid hitting this limit. A good rule of thumb is to regularly check the count of flows metric on your circuit.

ExpressRoute Traffic Collector can handle up to 300,000 flows a minute.

Here's a quick reference to help you plan your network traffic:

The maximum MTU supported is 1,400 bytes. This is a key consideration for optimizing network performance.

ExpressRoute supports this high MTU, making it an ideal choice for demanding applications.

For Azure VMs, you can tune the MTU by following the guidance in TCP/IP performance tuning for Azure VMs.

To set up ExpressRoute, you'll need to create a circuit, which is a dedicated connection between your network and Microsoft's network. This is done through the Azure portal.

A circuit requires a peering location, which is the physical location where your network meets Microsoft's network. There are several peering locations available, including New York, Los Angeles, and Dallas.

Once you've created a circuit and selected a peering location, you'll need to specify the bandwidth and SKU (Service Level Agreement) for your circuit. The bandwidth determines how much data can be transmitted over the circuit, and the SKU determines the level of service and support you'll receive.

The sampling rate is a crucial aspect of ExpressRoute Traffic Collector. It uses a sampling rate of 1:4096, which means 1 out of 4096 packets are captured.

This sampling rate is designed to balance network traffic monitoring with performance. It allows for efficient data collection without overwhelming the system.

ExpressRoute Traffic Collector's sampling rate is a trade-off between data accuracy and system resources. By capturing only a fraction of packets, it reduces the load on the system.

Creating the foundation for a well-configured system involves several key steps.

First, you need to define the system's purpose and requirements. This will help you determine the necessary hardware and software components.

A clear understanding of the system's purpose will also guide your decisions on scalability and maintenance.

For example, a system designed for high-traffic websites may require more powerful servers and a content delivery network.

You should also establish a standard for naming conventions and organization to ensure that the system is easy to navigate and manage.

ExpressRoute offers high availability through multiple connectivity options. You can connect up to 4 ExpressRoute circuits in the same peering location to your virtual network, or up to 16 circuits in different peering locations.

To ensure high availability, it's essential to implement the Local Preference attribute on your routers, so the path from on-premises to Azure is always preferred on your ExpressRoute circuits. This helps ensure that your traffic is always routed through the most optimal path.

ExpressRoute Traffic Collector supports availability zones by default in the regions where it's available. This means you can take advantage of high availability and redundancy in your ExpressRoute setup.

Here's a summary of the high availability options for ExpressRoute:

ExpressRoute also offers a Service Level Agreement (SLA) for its services, including ExpressRoute Global Reach and ExpressRoute Direct. The SLA provides the same availability as the regular ExpressRoute service.

ExpressRoute Traffic Collector setup does not cause any ExpressRoute Circuit downtime. This means you can configure it without worrying about any disruptions to your service.

Setting up ExpressRoute Traffic Collector is a seamless process that won't affect your ExpressRoute Circuit's availability.

Reach SLA is a crucial aspect of ExpressRoute, and it's essential to understand what's covered. ExpressRoute Global Reach provides the same availability SLA as the regular ExpressRoute service.

You can connect up to 16 ExpressRoute circuits in different peering locations to your virtual network, but be aware that you can use Equal-Cost Multipath (ECMP) across a maximum of four circuits to load-balance outgoing traffic from your virtual network.

If you're planning to implement ExpressRoute, be sure to check the SLA for ExpressRoute Global Reach, which is the same as the regular ExpressRoute service. This ensures you have a clear understanding of what's covered in case of any issues.

Here's a quick rundown of the SLA for ExpressRoute:

To ensure high availability, consider connecting up to 4 ExpressRoute circuits in the same peering location to your virtual network. This way, if one circuit disconnects, connectivity fails over to another ExpressRoute circuit.

In Azure, virtual networks connected to the same ExpressRoute circuit aren't isolated from each other from a routing perspective. This means that if you need route isolation, you need to create a separate ExpressRoute circuit.

You can coexist ExpressRoute with site-to-site VPNs, making it a flexible option for your network architecture.

To block internet traffic for virtual networks connected to ExpressRoute circuits, you need to withdraw the advertised route of 0.0.0.0/0 and place a Network Security Group on all subnets with an Outbound Deny rule for internet traffic.

Expand your knowledge: Site to Site Vpn to Azure

If the advertised route of 0.0.0.0/0 is withdrawn from the routes advertised due to an outage or a misconfiguration, Azure will provide a system route to resources on the connected Virtual Network to provide connectivity to the internet.

To block egress traffic to the internet, you'll need to place a Network Security Group on all subnets with an Outbound Deny rule for internet traffic. This is a crucial step to ensure that internet access is truly blocked.

In the event of a default route or Internet route prefix not being advertised through the BGP session, virtual networks connected to ExpressRoute circuits still have internet connectivity. This means you can connect to the internet from a virtual network linked to an ExpressRoute circuit.

Expand your knowledge: Trace Route in Azure

Virtual networks connected to the same circuit are not isolated from each other. From a routing perspective, they're part of the same routing domain.

To achieve route isolation, you need to create a separate ExpressRoute circuit. This ensures that each virtual network has its own isolated routing domain.

ExpressRoute can coexist with site-to-site VPNs. This means you can have both ExpressRoute and site-to-site VPNs connected to the same network without conflicts.

ExpressRoute offers a range of bandwidth options to suit your needs. You can purchase circuits with bandwidths of 50 Mbps, 100 Mbps, 200 Mbps, 500 Mbps, 1 Gbps, 2 Gbps, 5 Gbps, or 10 Gbps.

The bandwidth options are as follows:

Having a secondary connection can be beneficial, as it allows you to use the available bandwidth for more traffic if necessary. However, it's not guaranteed and shouldn't be used for extra traffic for a sustained period of time.

You can link up to 10 virtual networks to a standard ExpressRoute circuit, and up to 100 on a premium ExpressRoute circuit.

ExpressRoute authorizations can span subscription, tenant, and enrollment boundaries with no extra configuration required.

You can link up to 10 virtual networks in the same subscription as the circuit or different subscriptions using a single ExpressRoute circuit.

Connectivity and bandwidth charges for the dedicated circuit get applied to the ExpressRoute circuit owner and all virtual networks share the same bandwidth.

The limit of 10 virtual networks per standard ExpressRoute circuit can be increased by enabling the ExpressRoute premium feature.

ExpressRoute premium feature allows for up to 100 virtual network connections on a premium ExpressRoute circuit.

Suggestion: Azure Monitor Private Link Scope

Your existing ExpressRoute circuit can be configured to support connectivity to Microsoft 365 services, but make sure you have sufficient capacity and have enabled the premium add-on.

ExpressRoute Direct provides customers with direct 100 Gbps or 10-Gbps port pairs into the Microsoft global backbone, which is especially beneficial for massive data ingestion.

For customers in regulated markets, ExpressRoute Direct offers physical isolation, which is a must-have for compliance.

With ExpressRoute Direct, you can also get dedicated capacity for burst scenarios, such as rendering, which requires a lot of bandwidth.

Discover more: Azure Expressroute Direct Locations

Dynamics 365 is supported on ExpressRoute, and its environments are hosted on Azure, which means customers can benefit from ExpressRoute support for Azure resources.

To connect to Dynamics 365 service endpoints, your router filter must include the Azure regions where your Dynamics 365 environments are hosted.

You don't need ExpressRoute Premium for Dynamics 365 connectivity via Azure ExpressRoute if your ExpressRoute circuit is deployed within the same geopolitical region.

Here's a quick rundown of the requirements:

In summary, ExpressRoute provides a reliable and secure connection to Dynamics 365 environments, and you can easily set up connectivity with the right configuration.

Does My Existing ExpressRoute Support Microsoft 365?

Your existing ExpressRoute circuit can be configured to support connectivity to Microsoft 365 services.

If you have sufficient capacity and have enabled the premium add-on, you're good to go.

For Microsoft 365 connectivity, you'll need to ensure your ExpressRoute circuit has sufficient capacity and that the premium add-on is enabled.