POP SP4 Unavailability
Incident Report for Huge Networks
Resolved
BGP import policies with match action drop (or md-cli reject) and bgp-leak, can cause memory utilization to increase whenever a route matches the policy entry's criteria. Any changes to policy, or removing, re-adding an import policy with such an entry, which matches a route or set of routes will also cause memory utilization to increase. In some cases where there are a lot of routes and/ or lot of changes, the memory utilization may increase to a point where the CPM runs out of memory and cause system wide instability.

In a redundant system, a dual CPM SWO can help clear the memory utilization, but the same set of actions will start increasing utilization again. [453453-MI]
Posted May 08, 2024 - 23:38 GMT-03:00
Update
We are continuing to work on a fix for this issue with Nokia. Currently our environment is stable and the problem will not occur again, but, we already informed Nokia regards this issue. In the next 24 hours, we'll issue an RFO.
Posted Mar 25, 2024 - 01:59 GMT-03:00
Identified
The issue has been identified and a fix is being implemented.
Posted Mar 25, 2024 - 01:55 GMT-03:00
Investigating
We are currently investigating this issue.
Posted Mar 25, 2024 - 00:12 GMT-03:00
This incident affected: South America (São Paulo).