Calix EXA - E7 Configuration Notes

This page seeks to document various aspects of the Calix E7 EXA platform and recommended configurations of the platform for various services.

Initial Access

Initial access to the E7 can be done through the front Ethernet management port on the fantray. The E7 will hand a laptop a DHCP address and is reachable at 192.168.1.1 on either https or SSH.

default username: e7 default password: admin

VLAN Configuration

VLANs on the E7 are created and edited under the VLAN section of the E7 navigation tree. Here are some notes on the various E7 VLAN settings and when to use them.

IGMP Mode: Defaults to flood, snoop and proxy also supported. This would only be used for an IPTV Multicast VLAN
IGMP Profile: Corresponds to above. The IGMP profile will set a proxy IP if in proxy mode and other settings
DHCP Snoop: Whether to snoop on this VLAN. Recommended for DHCP Internet, SIP Voice, IPTV Unicast, AE ONT Management. Leave disabled for Static Internet.
MAC Forced-Fwd (PON/DSL ONLY): Enable this for Internet services only, recommended for both Static and DHCP
IP Source Verify (PON/DSL ONLY): Only allows traffic from defined IP. Recommended for DHCP and Static Internet. Note that for Static Internet you will need to set the IP on the actual ONT or DSL port for traffic to get through
MAC Learning: You can disable MAC learning for E-Line services if you need to, but we usually don't so that we can monitor that two way traffic is flowing
AE Discovery Event: Turn on for VLAN 85 or whatever the AE ONT Mgmt VLAN is. Causes the E7 to send traps to CMS when an AE ONT LLDP message is seen
TLAN (PON/DSL ONLY): Essentially disables multicast filtering on the VLAN and allows for things like OSPF or other mcast based protocols to pass through
PON Hairpin (PON ONLY): Allows two ONTs to switch traffic to each other. Really only needed if an Ethernet service has multiple endpoints on the same PON. Recommended for TLS services.

Ethernet Port Settings

There are two screens related to GE and 10GE ports on the E7. The Port->Provisioning screen corresponds to the physical port, while the Associated Interface->Provisioning screen corresponds to logical settings.

Physical Port settings

Most of the settings are self explanatory, but watch out for the following

Broadcast, Unknown Mcast and DLF Max Rates: We learned at WTRTC that 0 pps is not the same as off and can lead to CPU overload and dropped packets. Unless you have a good reason make sure these are set to off rather than 0
LACP Priority: Under the old way of doing cross card LAG which was active standby, the port you wanted to be active needed to be set with a lower priority. Furthermore, in active-standby cross card LAG the two ports must have different priorities or the configuration will be rejected

Ethernet Interface Settings

Role: Can be set to Edge or Trunk. These do not correspond to the Cisco meaning of trunk port. There are two primary things to know. First, with split horizon forwarding enabled on an edge port, it can only forward to a trunk port. This is similar to port isolation on some switches. Second, edge ports can have tag actions and such while trunk ports cannot. The general idea was for Trunk ports to the uplinks and edge ports downlinks. In practice this doesn't matter much.
RSTP: RSTP can be disabled on a per port basis. It first has to be enabled at the system level for this to have an effect
Ingress Policy Map: We use the policy map to perform two useful tasks. First, you can use a policy MAP to re-mark traffic on ingress, such as setting SIP to p-bit 5 or 6. Second, it can apply rate limiters to traffic as it enters.
MTU: Can be as high as 9600 on most E7 cards
Split Horizon Forwarding: disabling this allows edge port traffic to flow to another edge port. We have to disable this all the time to support TLS services or if a static Internet VLAN doesn't use MAC Forced Forwarding
Trusted: Relates to DHCP snooping. Disable trusted on most ports such as AE ONT ports or subscriber facing ports. Should be left on for ring ports, uplinks, etc. If the trusted is left enabled on the AE ONT facing port, the ONT will not appear under the port.

Ethernet Bandwidth Profiles

A COS profile is needed to configure bandwidth shaping on an active GE port. The port has to be in "access" mode to apply bandwidth restrictions. The default COS profile is full line rate.

The configure a COS profile, go to:

Profiles - > COS - > Ethernet

Provide bandwidth limit in Mbs.

To apply profile, go to the GE port at PORT - > Provisioning and change the COS Config to the new profile.

ONT Speed Profiles

For whatever reason AE ONT and GPON speed profiles are different. Here are the rules, note that IPTV for GPON is different. You will find that I usually put AE or GPON in the name to keep them straight. It's also helpful to make sure and check AE/E7 Y or N when creating the profile so someone doesn't accidentally assign the wrong profile.

Profiles should generally be created at the CMS PROFILE->E3-48C/E5/48/E7/E3-8G/E3-48R2/ONT->SERVICE->ETHERNET BANDWIDTH level unless you have a good reason to make it local. Global profiles are prefixed with the @ symbol. Profiles can have all properties changed except their actual name, so choose a good name when you make it.

GPON ONT Speed Profiles for Internet

UP-CIR: should be 0
UP-PIR: should be the desired speed
DOWN-PIR: should be desired speed
For E7: should be Y
For AE ONT: should be N

AE ONT Speed Profiles for Internet

UP-CIR: should be desired speed
UP-PIR: should be the desired speed
DOWN-PIR: should be desired speed
For E7: should be N
For AE: ONT should be Y

Assigning an AE profile to a GPON service will yield an error, assigning a GPON profile to an AE ONT will yield an unlimited (1G) upload speed.

Limiting DHCP leases on an ONT port

The default security profile on the E7 has a 8 lease limit. This profile is found in CMS under PROFILE->E3-48C/E5/48/E7/E3-8G/E3-48R2/ONT->SECURITY->ETHERNET and is called system-default. You can either change the system-default profile to a more sane number like 2 or 3, or create a new profile that is assigned to ONT Ethernet ports as needed. These profiles are the same for AE and GPON.

Enable Option 82

You can enable Option 82 under the DHCP->PROVISIONING screen on the E7. These settings apply to GPON, DSL, and untrusted GE ports.

Regarding AE ONT

In order for the AE ONT to appear under the port, make sure that the trusted setting is disabled on the E7 interface. In addition, make sure that AE Discover is enabled on the VLAN. If both those settings are correct, Inventory Snapshot will need to be run on that node or entire region (CMS>System>Scheduled Tasks>Inventory Snapshot) for the ONT to appear under the port. If it is not initiated manually, Inventory Snapshot, by default, runs every night, so the ONT should appear the next day.

When building data service for static IPs, you have to create STATIC IP/SUBNET entry for that data service. This is not needed for DHCP services, however DHCP snooping, IP Source Verify, MCC FF and Multicast Filtering will need to be turned off. Which one of these exactly enables DHCP to work, I am not sure. Please update this entry if you figure out. If the service will need both DHCP and static, static entry will need to be done on the IPs or blocks that are static and all settings mentioned above need to be turned off.

E7 CLI Commands

The E7 does have a CLI that can be useful in some cases compared to the GUI. Here are some useful commands for the CLI.

Show MAC Table

show mac

Show MACs on a specific VLAN

show mac on-vlan <ID>

Common TLS or Ethernet Circuit Issues

OSPF or something like that doesn't work

Make sure the Ethernet Security profile allows multicast
Make sure the L2CP filter is set to all-tunnel

Endpoints Can't Reach Each Other

If GPON make sure that PON Hairpin and TLS are enabled on the VLAN
IF ONTs are on different PON ports make sure that split horizon is disabled
If AE Make sure that E7 eth ports have split horizon disabled

E7 Software Upgrades

THIS SECTION HAS BEEN MOVED TO THE KB

The easiest way to upgrade an E7 network is to use CMS to push the files to the shelves, then manually rebooting the shelves during a maintenance window. Note that this procedure can be done locally an E7 to an FTP server running on a laptop if needed as well. Upgrade commands are at the E7 SYSTEM->PROVISIONING level under the ACTION->Upgrade menu. The basic process is Upgrade -> Reset -> Commit.

First, obtain the desired software release and ensure that the release can be upgraded to from the existing version based on the release notes
Upload the release ZIP file to the FTP server, usually CMS
Unzip the release file and delete everything except the rel.zip file (The other files are for if you are using a windows laptop and want to use the bundled FTP server)
Unzip the rel.zip file, this should create a folder called something like E7_R03.01.081.0001
Move the created folder to a directory under the cmsftp user, for example, at Mid-Plains E7 software is saved in /home/cmsftp/e7software
Obtain the official release version by running the command
```
head manifest.xml
```
and noting the information in the version release tag such as
```
<Version release="3.1.81.1" customerrelease="0"></Version>
```
Open CMS and go to SYSTEM->SCHEDULED TASKS->SOFTWARE UPGRADE
Click Create and choose E7-2 or E7-20 Upgrade as appropriate
Select the network, network group, or nodes that you wish to upgrade using the tree on the left. Green means that the item has been selected. Right click on an item or group and choose Select to select it.
Fill in the following information
- Upgrade Option: File Distribution Only
- ONT Mode: Retain Existing
- Source Ftp Server: CMS Server IP Address
- Source Ftp User: usually cmsftp
- Source Ftp password: usually cmsftp or ftpcms
- Source Ftp confirm password: same as above
- Source Directory Path: relative path of release files from FTP root, for example at Mid-Plains /e7software/E7_R03.01.081.0001
- Version: The exact version noted in the manifest.xml file
- Forced: N
Click OK to start the task
Wait for each E7 to download the new release files. CMS does about 4 at a time so for a large network this might take awhile.
Navigate to an E7 that you wish to reboot into the new software
Click ACTION->Upgrade->Reset System
Choose the new version from the provided drop down and click OK
Wait for both cards in the shelf to reboot into the new version
After software initialization has completed, go to ACTION->Upgrade->Commit System and choose the new version and click OK
Repeat as necessary until all nodes have been upgraded

E7 Basic Node Turn-up Script using CLI[edit]

This is the script provided by Aaron to Tommy to configure the nodes at Big Bend Six Shooter exchange. The ring is configured as G.8032. This is a raw script - to be formatted later

!!! Big Bend Six Shooter E7 Turn Up Script 
!!! 
!!! How to use this script: 
!!! Gather the information listed below from the network design spreadsheet 
!!! Run the commands by replacing the <CONFIG INFO> items with the correct 
!!! information. 
!!! 
!!! The following information is required to use this script 
! - <SYSTEM NAME> 
! - <SYSTEM IP> 
! - <G8032 NAME> 
! - <G8032 ID> 
! - <G8032 CONTROL VLAN> 
! - <G8032 PORT 1> 
! - <G8032 PORT 2> 
! - Is this node the RPL owner? 
!
!!! Connect to the front management port of the E7 with a cat5 cable and your computer set to DHCP 
!!! use putty or secureCRT to SSH to 192.168.1.1 and login with the following information: 
! - user: e7 
! - password: admin 
!
!!! Skip or ignore the configuration wizard by choosing no (n) when logging in the first time, then proceed with 
!!! the following configuration 
!
!!! First Set the system name appropriately 
!
set system name <SYSTEM NAME> 
!
!!! Next create all service VLANS 
!
create vlan 2702 name "E7 Management" 
create vlan 2704 name "DHCP Internet" 
create vlan 3679 name "SIP" 
!!! Note: Missing PWE3 VLAN assignment 
!
!!! Now configure inband management 
!
set mgmt-cfg vlan 2702 ip <SYSTEM IP> netmask 255.255.240.0 admin-state enabled 
set system mgmt-gw 10.63.112.254 
!
!!! CREATE G8032 Ring 
!
set eth-port <G8032 PORT 1> duplex full flow-ctrl none admin-state disabled 
set eth-port <G8032 PORT 2> duplex full flow-ctrl none admin-state disabled 
!
create g8032-ring <G8032 NAME> ring-id <G8032 ID> interface-1 <G8032 PORT 1> interface-2 <G8032 PORT 2> ctrl-vlan <G8032 CONTROL VLAN> 
!
!!! IF THIS NODE IS THE RPL OWNER (note that the /1 refers to ring port 1, and not the physical port specified in the ring configuration) 
!
set g8032-ring-interface <G8032 NAME>/1 rpl-mode owner 
!
!!! Enable the Ring Ports 
!
set eth-port <G8032 PORT 1> admin-state enabled 
set eth-port <G8032 PORT 2> admin-state enabled 
!
!!! add VLANS to ring 
!
add g8032-ring <G8032 NAME> to-vlan 2-3679 
!
!!! Set NTP 
!
set ntp server-1 10.34.0.10 
set ntp admin-state enabled 
!
!!! Commands to Verify operation 
!
!!! show alarm output show should port down or module missing for both ring interfaces, should also show alarm for g8032 ring being down 
show alarm 
!
show

Adding VLANs to an interface using CLI[edit]

Use the following commands to add the VLANs to an interface. It is easier and quicker to do this instead of clicking on each VLAN to add the interface/ERPS in CMS. Note that this command allows you to add only a single VLAN or a VLAN range (VLAN 102-110).

add interface <interface> to-vlan <VLAN ID>  /  Example: add interface 2/g6 to-vlan 102

Use can also use the same approach to add VLANs to the ERPS domain

add erps-domain <ERPS name> to-vlan <VLAN ID>  /  Example: add erps-domain DCLYERPS1 to-vlan 102