Switch Port xx is down (AutomaticBadSpeedOrWidth)

How to enable an Infiniband SUN DCS 36p switch port when it 's auto-disabled (Doc ID 1605955.1) Applies to: Sun Network QDR InfiniBand Gateway Switch - Version All Versions and later Exadata Database Machine X2-2 Hardware - Version All Versions and later Exadata X3-2 Hardware - Version All Versions and later All Platforms autodisable feature has been introduced only in firmware 2.1 and does not apply to firmware 1.3. Correct way to account for this is to check and ensure whether any auto-disabled ports exist and if present then re-enable using enableswitchport --automatic 'before' up/downgrading fw to a different version. This will ensure compatible settings when moving between different fw. autodisable feature is in the 2.x series and should be in the /conf/disabledport.conf if autodisable is set for this port. This should not be here for the 1.x series and looks like a artifact from the downgrade Our internal team is looking at how to handle this in the future Note: Using ibportstate to administer a port will not be preserved through subsequent reboots, hence should not be relied on. You should always use en(dis)ableswitchport. since /conf/disabledports.conf does not apply to 1.3, you can safely delete it for now and issue a switch reset command 'ibdevreset Switch' or simply reboot the switch. for switch with version 2.1.3-4 run enableswitchport --automatic command and port number Symptoms   listlinkup show that port 29 is autodisabled. # listlinkup Connector  4B Present <-> Switch Port 27 is up (Enabled) Connector  5B Present <-> Switch Port 29 is down (AutomaticBadSpeedOrWidth) <--------------- Connector  6B Present <-> Switch Port 36 is up (Enabled)   Changes To enable the port , you first need to determine the firmware version that 's running on the switch using the "version" command. I included below the output for version: 2.1.3-4 and version: 1.3.3-2 which is supported as per the latest release notes.   version SUN DCS 36p version: 2.1.5-1 version SUN DCS 36p version: 2.1.3-4 version SUN DCS 36p version: 1.3.3-2    Cause The issue can be caused by a bad partner link or misconfiguration  Solution  For version 1.3.3-2 ,   2.1.3-4 and above , make sure the file /conf/disabledports.conf doesn't exist . # version SUN DCS 36p version: 1.3.3-2 Build time: Apr  4 2011 11:15:19 SP board info: Manufacturing Date: 2011.12.25 Serial Number: "NCD7K0152" Hardware Revision: 0x0006 Firmware Revision: 0x0000 BIOS version: SUN0R100 BIOS date: 06/22/2010 # getportstatus 29 Port status for connector 3B Switch Port 29 Adminstate:......................Disabled (AutomaticBadSpeedOrWidth) LinkWidthEnabled:................1X or 4X LinkWidthSupported:..............1X or 4X LinkWidthActive:.................4X LinkSpeedSupported:..............2.5 Gbps or 5.0 Gbps or 10.0 Gbps LinkState:.......................Down PhysLinkState:...................Disabled LinkSpeedActive:.................2.5 Gbps LinkSpeedEnabled:................2.5 Gbps or 5.0 Gbps or 10.0 Gbps   This below command works on version: 2.1.3-4 but not on version: 1.3.3-2 # enableswitchport --reason=AutomaticBadSpeedOrWidth 29 Invalid reason AutomaticBadSpeedOrWidth Usage: enableswitchport [--reason=reason] connector | [ibdevicename] port Values for ibdevicdename: Switch Values for port: 1-36 Values for connector: 0A-17A, 0B-17B Values for reason: Blacklist, Partition   Somehow /conf/disabledports.conf had these settings.  Only possibility I can think of is this system had a fw with the autodisable feature with 2 ports that had this condition and then the system was downgraded and this file did not get updated.   the file should be renamed or removed completely. cat /conf/disabledports.conf # List of Disabled ports # Format: # ibdev port Adminstate #Switch 34 AutomaticBadSpeedOrWidth #Switch 29 AutomaticBadSpeedOrWidth after removing or renaming the file /conf/disabledports.conf, enable the port as follow enableswitchport Switch 29 getportstatus 29 Port status for connector 5B Switch Port 29 Adminstate:......................Enabled LinkWidthEnabled:................1X or 4X LinkWidthSupported:..............1X or 4X LinkWidthActive:.................1X LinkSpeedSupported:..............2.5 Gbps or 5.0 Gbps or 10.0 Gbps LinkState:.......................Active PhysLinkState:...................LinkUp LinkSpeedActive:.................10.0 Gbps LinkSpeedEnabled:................2.5 Gbps or 5.0 Gbps or 10.0 Gbps ibportstate 29 query PortInfo: # Port info: Lid 29 port 0 LinkState:.......................Active PhysLinkState:...................LinkUp LinkWidthSupported:..............1X or 4X LinkWidthEnabled:................1X or 4X LinkWidthActive:.................4X LinkSpeedSupported:..............2.5 Gbps or 5.0 Gbps or 10.0 Gbps LinkSpeedEnabled:................2.5 Gbps or 5.0 Gbps or 10.0 Gbps LinkSpeedActive:.................10.0 Gbps #listlinkup Connector  4B Present <-> Switch Port 27 is up (Enabled) Connector  5B Present <-> Switch Port 29 is up (Enabled) Connector  6B Present <-> Switch Port 36 is up (Enabled)   to disable the port again you can use disableswitchport switch 29 Now for version SUN DCS 36p version: 2.1.3-4 and above # listlinkup Connector  5B Present <-> Switch Port 29 is down (AutomaticBadSpeedOrWidth) <---------- # enableswitchport  --automatic Switch 29 Enable connector 5B Switch port 29 Adminstate:......................Enabled LinkWidthEnabled:................1X or 4X LinkWidthSupported:..............1X or 4X LinkWidthActive:.................4X LinkSpeedSupported:..............2.5 Gbps or 5.0 Gbps or 10.0 Gbps LinkState:.......................Down PhysLinkState:...................PortConfigurationTraining LinkSpeedActive:.................2.5 Gbps LinkSpeedEnabled:................2.5 Gbps or 5.0 Gbps or 10.0 Gbps NeighborMTU:.....................2048 OperVLs:.........................VL0-7 #  ibportstate 3 29 PortInfo: # Port info: Lid 3 port 29 LinkState:.......................Down PhysLinkState:...................Polling LinkWidthSupported:..............1X or 4X LinkWidthEnabled:................1X or 4X LinkWidthActive:.................4X LinkSpeedSupported:..............2.5 Gbps or 5.0 Gbps or 10.0 Gbps LinkSpeedEnabled:................2.5 Gbps or 5.0 Gbps or 10.0 Gbps LinkSpeedActive:.................10.0 Gbps #  ibportstate 3 29 PortInfo: # Port info: Lid 3 port 29 LinkState:.......................Down PhysLinkState:...................Polling LinkWidthSupported:..............1X or 4X LinkWidthEnabled:................1X or 4X LinkWidthActive:.................4X LinkSpeedSupported:..............2.5 Gbps or 5.0 Gbps or 10.0 Gbps LinkSpeedEnabled:................2.5 Gbps or 5.0 Gbps or 10.0 Gbps LinkSpeedActive:.................10.0 Gbps # #  ibportstate 3 29 PortInfo: # Port info: Lid 3 port 29 LinkState:.......................Active PhysLinkState:...................LinkUp LinkWidthSupported:..............1X or 4X LinkWidthEnabled:................1X or 4X LinkWidthActive:.................4X LinkSpeedSupported:..............2.5 Gbps or 5.0 Gbps or 10.0 Gbps LinkSpeedEnabled:................2.5 Gbps or 5.0 Gbps or 10.0 Gbps LinkSpeedActive:.................2.5 Gbps Peer PortInfo: # Port info: Lid 3 DR path slid 65535; dlid 65535; 0,29 port 2 LinkState:.......................Active PhysLinkState:...................LinkUp LinkWidthSupported:..............1X or 4X LinkWidthEnabled:................1X or 4X LinkWidthActive:.................4X LinkSpeedSupported:..............2.5 Gbps or 5.0 Gbps or 10.0 Gbps LinkSpeedEnabled:................2.5 Gbps or 5.0 Gbps or 10.0 Gbps LinkSpeedActive:.................2.5 Gbps ibwarn: [2687] validate_speed: Peer ports operating at active speed 1 rather than  4 (10.0 Gbps) [root@scam07sw-ibb0 IBdata]# ibportstate 3 29 PortInfo: # Port info: Lid 3 port 29 LinkState:.......................Down PhysLinkState:...................Disabled LinkWidthSupported:..............1X or 4X LinkWidthEnabled:................1X or 4X LinkWidthActive:.................4X LinkSpeedSupported:..............2.5 Gbps or 5.0 Gbps or 10.0 Gbps LinkSpeedEnabled:................2.5 Gbps or 5.0 Gbps or 10.0 Gbps LinkSpeedActive:.................2.5 Gbps [root@scam07sw-ibb0 IBdata]# This might explain why the port got disabled in the first place. You have an SDR link speed on peer port instead of QDR. It could be either a cable or HCA problem.  try switching cables?