Last week I came across a very strange error. I needed to create a new VLAN on our core switch. no problem to create a layer2 VLAN, but impossible to create a layer3…
here is the error that pop up
********(config)#in vlan 299
%Cannot create a VLAN interface
Maximum allowed interfaces reached
^
% Invalid input detected at '^' marker.
We don’t have thousands and thousands of VLAN so it was obviously something else. Is there a much smaller limit on layer 3 VLAN than on layer 2 ? Yes, but the smaller cisco devices can support 4000 interfaces (including vlans and other virtual interfaces) . And our switch has got a limit of 12000…
Goggling this error won’t give you much information and I have to say that the error point you easily in the wrong direction. This is only when you have a look at the logs that you can see something a bit more interesting.
entry number 215 : INTERFACE_API-1-NOMORESWIDB
No more SWIDB can be allocated, maximum allowed 12000
Here is some theory to get a better understanding of what an IDB is:
SWIDB stand for “Software for Interface Descriptor Block”. you have two type of IDBs, SW (software) and HW (hardware). Cisco website describe an IDB as following:
An Interface Descriptor Block (IDB) is a special control structure internal to the Cisco IOS software that contains information such as the IP address, interface state, and packet statistics. Cisco IOS software maintains one IDB for each interface present on a platform and one IDB for each subinterface.
A HWIDB represents a physical interface, which includes physical ports and channelized interface definitions. A SWIDB represents a logical sub-interface (Permanent Virtual Circuit (PVC) or virtual LAN (VLAN)), or a Layer 2 encapsulation (Point-to-Point Protocol (PPP), High-Level Data Link Control (HDLC), and so forth).
Each physical interface on the router consumes a minimum of two IDBs:
- One HWIDB for the physical port
- One SWIDB for the Layer 2 encapsulation
A channelized port consumes N+1 HWIDBs, where N is the number of channels within the physical port, plus a minimum of N SWIDBs (Level 2 encapsulation per channel). Any sub-interfaces that you define each add another SWIDB.
Each tunnel interface definition, such as Universal Transport Interface (UTI), Generic Routing Encapsulation (GRE), Multiprotocol Label Switching Traffic Engineering (MPLS TE), or Any Transport over MPLS (AToM) consumes an HWIDB plus one SWIDB per tunnel, plus an additional SWIDB for each additional sub-interface, for example, a Frame Relay PVC, that is tunneled. The tunnel IDBs are in addition to the original interface(s) that are tunneled.
You can have a look at your IDB used with the command:
sh idb
So my problem is that I actually reached 12000 interfaces? Impossible! Unfortunately it is!
when I saw the output of that command I directly understood what’s happened…
A couple of week before, we installed a new blade chassis. But a error on the configuration created some kind of loop and as a consequence, new interfaces keep registering themselves on the core switch until the IDB limit is reached. something you don’t realize until you need to configure a new interface…
What is the fix then?
unfortunately, there is no easy fix or I should say there is only one and very quick fix, reloading the IOS… just crazy on a prod switch, even more knowing its the core switch… but unfortunately, even a fail-over of the modules does not resolve this issue! (according to Cisco)
this issue just amaze me because you can clear all sort of things on the cisco IOS, but not a table of IDBs… I suppose it doesn’t get full every day but still, I don’t understand why there is not another way to clean this table.