_ FCO 63XMX-F003 Rev C, Secondary proc fails RDB test, parity errors ______________________________________________________________________________ | DIGITAL FCO CATEGORY PAGE 1 | | [F] OF 8 | |______________________________________________________________________________| | FIELD CHANGE ORDER NUMBER: 63XMX-F003 | |______________________________________________________________________________| | APPLICABILITY: Upgrade all T2011-YA/-YC spares and any CPU modules in | | systems which exhibit the problems described below. This FCO incorporates | | ECO's T2011-YA-MK001, T2011-YA-MK002, T2011-YA-TW003 and T2011-YA-TW004. | | NON-CONTRACT/NON-WARRANTY CUSTOMERS: It is recommended that T2011-YA spares | | be upgraded to Rev. "AE01" for the T2011-YA and "AD01" for T2011-YC. | | || This supersedes previous revision, dated 18 February 1991. | |______________________________________________________________________________| | PROBLEM & SYMPTOM:1) Secondary processor intermittently fails RBD test 7 and/| | or ELKMP; 2) Duplicate Tag Parity Errors occur during periods of heavy XMI | | activity; 3) Cache Tag Parity errors and/or related bugchecks in which the | | exception PC equals MMG$PAGEFAULT+10E,MMG$PAGEFAULT+91. | |______________________________________________________________________________| | SOLUTION: Replace failing T2011-YA/-YC module with revision AE01 for the | | T2011-YA or revision AD01 for the T2011-YC. | |______________________________________________________________________________| | QUICK CHECK: Check ROM at loc. E16 (Console Rom). Should be 23-188E9-00 for | | the -YA and 23-192E9-00 for the -YC. | |______________________________________________________________________________| | PRE/CO-REQUISITE FCO: | MTTI HRS | | N/A | 1 Hr. | |___________________________________________________________________|__________| | TOOL/TEST EQUIPMENT: Field Service Tool Kit. | |______________________________________________________________________________| | FCO PARTS INFORMATION | |______________________________________________________________________________| | FCO KIT NO. | DESCRIPTION OF CONTENTS | EQ KIT VARIATION | |______________|____________________________________________| APPLICABILITY | |EQ-01596-XX | See Page 2. | | |FA-04928-01 | FCO Document | N/A | |______________|____________________________________________|__________________| | FCO CHARGING INFORMATION | |______________________________________________________________________________| | WARRANTY/CONTRACT || NONWARRANTY/NONCONTRACT | |___________________________||_________________________________________________| | ON-SITE | OFF-SITE || ON-SITE | OFF-SITE | MATERIAL ONLY | |_____________|_____________||_____________|_____________|_____________________| |TRAVEL/| EQ | | EQ ||TRAVEL/| EQ | | EQ |ORDER-ADMIN,HANDLING | |INSTALL| KIT |INSTALL| KIT ||INSTALL| KIT |INSTALL| KIT |PKG,SHIPPING & EQ KIT| |_______|_____|_______|_____||_______|_____|_______|_____|_____________________| | DEC | DEC | DEC | DEC || CUS | CUS | CUS | CUS | CUS | |_______|_____|_______|_____||_______|_____|_______|_____|_____________________| | APPROVALS | |______________________________________________________________________________| | CSSE | FSHQ LOGISTICS | FS PRODUCT SAFETY | | Ken Jackson | Len Pellerin | Robert Brister | |___________________|____________________________|_____________________________| | CSSE MANAGER |This Document is published | FCO RELEASE DATE | | Ric Grogan |on multiple media including | 18-MAR-1991 | |___________________|Hardcopy, Customer Services |_____________________________| | MICROMEDIA |Microfiche Libraries, | FCO REVISION | | Diane MacDonald |Customer Services CD-ROM and| C | |___________________|MDS Microfiche Libraries. |_____________________________| | POPULATION | | PARTS AVAILABILITY | | 1,450 | | January, 1991 | |___________________|____________________________|_____________________________| _ _ _ _ _ _ _ | FCO 63XMX-F003 | | | | | | | | | |d|i|g|i|t|a|l| | PAGE 2 OF 8 |_|_|_|_|_|_|_| | | _______________________________|_________________________________________ FCO Parts Information: (Continued from Page 1) FCO KIT NO. CONTENTS ---------- -------- EQ-01596-01 T2011-YA FA-04928-01 FCO Document EQ-01596-02 T2011-YC FA-04928-01 FCO Document EQ-01596-03 TK50 (Cache Exerciser Program P/N FA-04928-02 FIELD INSTALLATION AND TEST PROCEDURE FOR 63XMX-F003 ---------------------------------------------------- ******************************************************************** * NOTE 1 * * Upon a scheduled site visit to remove and replace a failing * * T2011-YA/-YC experiencing "Cache Tag Parity Errors" and/or * * related "Fatal Bugchecks" in which the exception PC equals * * MMG$PAGEFAULT+10E,MMG$PAGEFAULT+91, it is recommended that the * * Cache Exerciser Program, EQ-01596-03, be invoked to test the * * remaining T2011-YA modules remaining in the system to verify the* * cache integrity. Only those modules which exhibit the problem * * should be replaced. There is no need to replace all CPU modules * * in a system. There are no compatibility problems between * * revisions. * * NOTE 2 * * It is recommended that an additional module be brought to the * * customer site in the event the Cache Exerciser identifies an * * additional failing CPU module. * ******************************************************************** 1. Shut down the operating system using the approved methods, such as @SYS$SYSTEM:SHUTDOWN for VMS. If using SHUTDOWN, answer all prompts accordingly. Here is a sample of a shutdown session: $ @sys$system:shutdown SHUTDOWN -- Perform an Orderly System Shutdown on node XXX _ _ _ _ _ _ _ | FCO 63XMX-F003 | | | | | | | | | |d|i|g|i|t|a|l| | PAGE 3 OF 8 |_|_|_|_|_|_|_| | | _______________________________|_________________________________________ How many minutes until final shutdown [0]: Reason for shutdown [Standalone]: Do you want to spin down the disk volumes [NO]? Do you want to invoke the site-specific shutdown procedure [YES]? Should an automatic system reboot be performed [NO]? When will the system be rebooted [later]: Shutdown options (enter as a comma-separated list): REMOVE_NODE Remaining nodes in the cluster should adjust quorum CLUSTER_SHUTDOWN Entire cluster is shutting down REBOOT_CHECK Check existence of basic system files SAVE_FEEDBACK Save AUTOGEN feedback information from this boot Shutdown options [NONE]: reboot, remove VMS will issue several messages indicating it is shutting down. VMS will issue: SYSTEM SHUTDOWN COMPLETE - USE CONSOLE TO HALT SYSTEM 2. At this point type a Control-P to halt the primary processor. ******************************************************************* * NOTE * * ---- * * INFORMATION IS TO BE RECORDED IN STEPS 3,5,7, AND 8 THAT WILL BE* * USED LATER ON IN THESE INSTRUCTIONS. * ******************************************************************* 3. Move the lower key switch to the HALT position and record the original position. 4. Enter INITIALIZE at the >>> prompt. This will reset the whole system and force all processors into console mode. 5. You should examine the console map to determine the location of each processor MODULE in your system. Record the location of each processor. The map denotes a processor by printing an uppercase letter P on the TYP line. Note which processors have been disabled from becoming the Boot Processor. The BPD line gives this information: an E indicates that the processor may be a Boot Processor, a D indicates that it may not. 6. Look at the console map and determine which nodes contain processors then connect the terminal to the processor at the lowest node or to a processor specifically designated as the primary by entering - >>> SET CPU n where n is the node number _ _ _ _ _ _ _ | FCO 63XMX-F003 | | | | | | | | | |d|i|g|i|t|a|l| | PAGE 4 OF 8 |_|_|_|_|_|_|_| | | _______________________________|_________________________________________ 7. Enter the SHOW BOOT command, and record the saved boot specific- actions. Here is a sample of the command output: >>> SHOW BOOT DEFAULT /XMI:E /BI:4 DU3D R54A /R5:00000001/XMI:E/BI:4 DU4A DIAG /R5:00000010/XMI:E/BI:4 DU15 R5 /R5:00000001/XMI:E/BI:4 DUD If the SHOW BOOT command prints no information, that is okay. It means there was no stored boot specification. 8. Enter the SHOW SYSTEM SERIAL command, and record the system serial number. Here is a sample of the command output: >>> $^?SHOW SYSTEM SERIAL System serial number: AG83701988 9. If this is a multiprocessing system, invoke the Cache exerciser program to verify the cache tag ram integrity on the remaining CPU modules. Load the TK50 tape contained in EQ-01596-03 into the TK50/TK70 unit. At the console prompt >>> type B CSA1. Example of a three processor system running the cache program follows: FCO TAG VERIFICATION PROGRAM VERSION V1.3...... COPYRIGHT 1990, DIGITAL EQUIPMENT CORPORATION, ALL RIGHTS RESERVED STARTING SECONDARY PROCESSOR @ NODE ID 03 STARTING SECONDARY PROCESSOR @ NODE ID 04 END OF PASS 000/ CPU 02 TPE SUMMARY: THIS PASS 0000 TOTAL = 00000 CPU 03 TPE SUMMARY: THIS PASS 0000 TOTAL = 00000 CPU 04 TPE SUMMARY: THIS PASS 0000 TOTAL = 00000 ^ ^ \ ERRORS / TYPE ^P (TO STOP THE PROGRAM) ?02 external halt PC = ??? Failure. (reflects the ^P) >>> _ _ _ _ _ _ _ | FCO 63XMX-F003 | | | | | | | | | |d|i|g|i|t|a|l| | PAGE 5 OF 8 |_|_|_|_|_|_|_| | | _______________________________|_________________________________________ 10. After running the Tag Verification Program, make note of all CPU modules which exhibited cache tag parity errors including the original failing module. 11. Before powering down the system, set the console terminal speed to 1200. Do not worry if the console writes strange characters after issuing the command. This means your terminal is set to some baud rate other than 1200 baud. 12. Press the SETUP key on your terminal and set the baud rate of your terminal to 1200 baud. SAVE this setting. This allows you to issue console commands once the new T2011-YA/YC(s) are installed. 13. Power down the system by turning the upper key switch on the front control panel to the OFF position. Pull the circuit breaker on the AC power controller (H405) to the OFF position and unplug the system from the source. 14. Open the front cabinet door. 15. Remove the clear plastic cover in front of the XMI cage. **************************************************************** * C A U T I O N * * * * All VAX modules contain electrostatic discharge * * sensitive devices (ESDS). The use of an antistatic * * wrist strap attached to the cabinet is essential when * * when handling modules. * **************************************************************** 16. Remove the failing T2011-YA/-YC module(s) from the cardcage and install the new T2011-YA/-YC(s) from EQ-01596-01 or -02 into the slot(s) from which the original module(s) were removed. 17. Re-install the XMI cardcage cover assembly and tighten retaining screw. _ _ _ _ _ _ _ | FCO 63XMX-F003 | | | | | | | | | |d|i|g|i|t|a|l| | PAGE 6 OF 8 |_|_|_|_|_|_|_| | | _______________________________|_________________________________________ ****************************************************************** * **NOTE 1** * * THE NEW XMI BASED SYSTEMS HAVE NECESSITATED CHANGES TO THE * * PROCESSOR CONSOLE ROM FOR SUPPORT. WHEN THESE PROCESSOR * * MODULES ARE CYCLED INTO THE FIELD AS ADD-ON OPTIONS, SPARES, * * OR DUE TO AN FCO, IT MAY RESULT IN VAX 6000 SYSTEMS HAVING * * PROCESSOR MODULES WITH CONSOLE ROMS AT DIFFERENT REVISIONS IN * * THE SAME SYSTEM. * * - * * YOU MAY SEE CONSOLE ROM MISMATCH MESSAGES PRINTED DURING * * SYSTEM INITIALIZATION. THESE DO NOT IDENTIFY A PROBLEM. THESE * * SHOULD BE CONSIDERED INFORMATIONAL FOR LISTING ROM REVISIONS. * * * * **NOTE 2** * * DO NOT USE "SAVE/RESTORE EEPROM" AND/OR "UPDATE ALL" COMMANDS * * WITH MULTIPLE CPU SYSTEMS WHICH MAY CONTAIN MIXED VERSIONS OF * * CONSOLE/DIAGNOSTIC ROMS. USE INSTEAD, "PATCH UPDATE UTILITY", * * EVUCA. * ****************************************************************** 18. Power on the system by setting the upper front panel keyswitch to ENABLE. Ensure Self Test is completed successfully on all T2011-YA/-YC modules. If some modules fail selftest, you may have to ensure the processor modules are seated correctly in the backplane. It is not uncommon to have to reseat the boards once or twice. 19. Move the lower key switch to the Update position and restore the system serial number and boot specifications recorded in Steps 7 and 8, to the T2011-YA/-YC(s) just installed. Do the following: >>>SET CPU n where n is the node number. 20. Enter the SET SYSTEM SERIAL command, using the serial number you recorded in Step 8. Following is a sample output from the command: >>> $^?SET SYSTEM SERIAL System Serial Number>>> AG83701988 Serial number read as: AG83701988 Update EEPROM? (Y or N) >>> Y ?73 System serial number updated. _ _ _ _ _ _ _ | FCO 63XMX-F003 | | | | | | | | | |d|i|g|i|t|a|l| | PAGE 7 OF 8 |_|_|_|_|_|_|_| | | _______________________________|_________________________________________ 21. Now, enter the boot specifications you saved in Step 7, using the SET BOOT command. Here is sample output from the command: >>> SET BOOT DEFAULT /XMI:E/BI:4 DU3D It may be helpful to check the boot specification you just entered. Enter the SHOW BOOT command to check the boot specification or specifications. If your system contains more than one processor, entering the SET BOOT command causes the boot specification to be copied to all processors, so this command does not need to be repeated on each processor. 22. In Step 5 you recorded which of the CPUs were prevented from becoming primaries. You need to set that condition again. >>> SET CPU [n] /NOPRIMARY where n equals the node number. 23. Press RESET on the control panel or enter the INITIALIZE command and ensure the console prints no error messages. 24. Return the lower front panel keyswitch to the position you recorded in Step 3. 25. Run the Diagnostic Supervisor (VAX/DS) ELSAA. 26. Run EVSBA - VAX Diagnostics Autosizer program. 27. If multiprocessors are present run EVUCA to patch ROMS. ****************************************************************** * NOTE 1 * * EVUCA REQUIRES VAX DIAGNOSTIC SUPERVISOR (VAX/DS) VERSION 14.0 * * OR LATER AND BINARY FILE ELUCB.BIN BE PRESENT FOR VAX6000-3XX. * * * * NOTE 2 * * DO NOT USE "SAVE/RESTORE EEPROM" AND/OR "UPDATE ALL" COMMANDS * * WITH MIXED VERSIONS OF CONSOLE/DIAGNOSTIC ROMS. USE INSTEAD, * * "PATCH UPDATE UTILITY", EVUCA * ****************************************************************** _ _ _ _ _ _ _ | FCO 63XMX-F003 | | | | | | | | | |d|i|g|i|t|a|l| | PAGE 8 OF 8 |_|_|_|_|_|_|_| | | _______________________________|_________________________________________ 28. Load and run the following diagnostics: ELKAX Functionality ELKMP Multiprocessor Exerciser EVKAQ Basic Instructions Exerciser Part I EVKAR Basic Instructions Exerciser Part II EVKAS Floating Point Instructions Exerciser Part I EVKAT Floating Point Instructions Exerciser Part II EVKAU Privileged Architecture Exerciser Part I EVKAV Privileged Architecture Exerciser Part II 29. Upon successful completion of the diagnostics, exit the VAX/DS. 30. Bring up the operating system. 31. Update Site Management Guide to reflect this FCO. 32. Return the old Rev. T2011-YA(s)/-YC(s) in the ESD container for upgrading. 33. Report this FCO activity on the LARS form in the "Fail Area/ Module/FCO/Comments" column as follows: FCO 63XMX-F003. LARS USA GIA EUROPE Activity - (a)Contract and Warranty W U Y (b)IN-DEC Contract K Hardware Segment Code 111 Non Contract/Non Warranty F F F (c)RTD/Off-site Agreement F Product Line 01 DEC Option 63XX 63XX 63XX Type of Call M M M Action Taken D D I Fail Area-Module-FCO-Comments 63XMX-F003 63XMX-F003 63XMX-F003 Material Used EQ-01596-01/ EQ-01596-01/ EQ-01596-01/ EQ-01596-02/ EQ-01596-02/ EQ-01596-02/ EQ-01596-03 EQ-01596-03 EQ-01596-03 (a) Warranty Optimum, Warranty Standard and Warranty Basic (on-site) Agreements. (b) Applies to INDEC AREA ONLY (c) RTD=Return to Digital or Off-site Agreements; If Field Engineer On-site, use Activity Code "F". \^ 63XMX \\63XMX \\FCO_DOCS \\MAR \\1991 \\T2011-YA \\T2011-YC \\JACKSON