VAX 7000/10000 KA7AA CPU Technical Manual Order Number EK·KA7AA·TM.001 The KA7AA is a VAX CPU module designed for the LSB platform. It is based on the NVAX+ microprocessor and is used in the VAX 7000 and VAX 10000 computer systems. It supports up to seven MS7AA memory modules in a uniprocessor configuration and one IOP module per system. Used in a single­processor system, the KA7AA module achieves a minimum scalar performance equivalent to that of more than 20 VAX 11/780 systems. A multiprocessor system supports up to six KA7AA CPU modules. First Printing, December 1992 The information in this document is subject to change without notice and should not be construed as a com­ mitment by Digital Equipment Corporation. Digital Equipment Corporation assumes no responsibility for any errors that may appear in this document. The software, if any, described in this document is furnished under a license and may be used or copied only in accordance with the terms of such license. No responsibility is assumed for the use or reliability of soft­ ware or equipment that is not supplied by Digital Equipment Corporation or its affiliated companies. Copyright © 1992 by Digital Equipment Corporation. All Rights Reserved. Printed in U.S.A. The following are trademarks of Digital Equipment Corporation: Alpha AXP DECUS VAXBI AXP DWMVA VAXELN DEC OpenVMS VMScluster DECchip ULTRIX XMI DEC LANcontroller UNIBUS The AXP logo DECnet VAX OSF/1 is a registered trademark of the Open Software Foundation, Inc. FCC NOTICE: The equipment described in this manual generates, uses, and may emit radio frequency en­ ergy. The equipment has been type tested and found to comply with the limits for a Class A computing de­ vice pursuant to Subpart J of Part 15 of FCC Rules, which are designed to provide reasonable protection against such radio frequency interference when operated in a commercial environment. Operation of this equipment in a residential area may cause interference, in which case the user at his own expense may be required to take measures to correct the interference. iii Contents Preface ............................................................................................................................................. xi Chapter 1 CPU Module Overview 1.1 NVAX+ CPU Chip ......................................................................................................... 1­3 1.2 Backup Cache (B­Cache) .............................................................................................. 1­4 1.3 LSB Interface (LEVI) ................................................................................................... 1­4 Chapter 2 CPU Chip 2.1 Data Types ..................................................................................................................... 2­2 2.2 Instruction Set .............................................................................................................. 2­2 2.3 ........................................................................................................................................ 2­2 2.4 Address Space ............................................................................................................... 2­3 2.4.1 Virtual Address Space ............................................................................................ 2­3 2.4.2 Physical Address Space .......................................................................................... 2­3 2.5 Memory Management ................................................................................................... 2­4 2.5.1 System Space Address Translation ....................................................................... 2­4 2.5.2 Process Space Address Translation ....................................................................... 2­5 2.5.2.1 P0 Region Address Translation ....................................................................... 2­5 2.5.2.2 P1 Region Address Translation ....................................................................... 2­5 2.5.3 Page Table Entry Format ....................................................................................... 2­6 2.5.4 Translation Buffer .................................................................................................. 2­7 2.5.5 Memory Management Control ............................................................................... 2­7 2.6 Exceptions and Interrupts ............................................................................................ 2­9 2.6.1 Exceptions ............................................................................................................. 2­10 2.6.1.1 Arithmetic Exceptions ................................................................................... 2­11 2.6.1.2 Memory Management Exceptions ................................................................. 2­12 2.6.1.3 Emulated Instruction Exceptions ................................................................. 2­13 2.6.1.4 System Failure Exceptions ............................................................................ 2­14 2.6.2 Interrupts .............................................................................................................. 2­15 2.6.2.1 External Interrupt Requests ........................................................................ 2­16 2.6.2.2 Internal Interrupt Requests .......................................................................... 2­16 2.7 System Control Block .................................................................................................. 2­18 2.8 Process Structure ........................................................................................................ 2­22 2.9 Functional Partitions .................................................................................................. 2­25 2.9.1 Ibox ........................................................................................................................ 2­26 2.9.2 Ebox and Microsequencer .................................................................................... 2­27 2.9.3 Fbox ....................................................................................................................... 2­28 2.9.4 Mbox ...................................................................................................................... 2­28 2.9.5 Cbox ....................................................................................................................... 2­29 2.10 General Purpose Registers ......................................................................................... 2­30 iv 2.11 Internal Processor Registers ...................................................................................... 2­31 2.11.1 Identification Registers ........................................................................................ 2­35 CPUID·CPU Identification Register ................................................................. 2­36 SID·System Identification Register .................................................................. 2­37 2.11.2 Ibox Registers ....................................................................................................... 2­38 VMAR·VIC Memory Address Register .............................................................. 2­39 VTAG·VIC Tag Register .................................................................................... 2­40 VDATA·VIC Data Register ................................................................................ 2­41 ICSR·Ibox Control and Status Register ............................................................ 2­42 BPCR·Branch Prediction Control Register ...................................................... 2­43 2.11.3 Ebox Registers ...................................................................................................... 2­45 PCSCR·Patchable Control Store Control Register ........................................... 2­46 ECR·Ebox Control Register ............................................................................... 2­48 2.11.4 Mbox Registers ..................................................................................................... 2­50 MP0BR·Mbox P0 Base Register ........................................................................ 2­51 MP0LR·Mbox P0 Length Register .................................................................... 2­52 MP1BR·Mbox P1 Base Register ........................................................................ 2­53 MP1LR·Mbox P1 Length Register .................................................................... 2­54 MSBR·Mbox System Base Register .................................................................. 2­55 MSLR·Mbox System Length Register ............................................................... 2­56 MMAPEN·Mbox Map Enable Register ............................................................. 2­57 PAMODE·Physical Address Mode Register ..................................................... 2­58 MMEADR·MME Address Register ................................................................... 2­59 MMEPTE·MME PTE Address Register ............................................................ 2­60 MMESTS·MME Status Register ....................................................................... 2­61 TBADR·Translation Buffer Parity Address Register ....................................... 2­63 TBSTS·Translation Buffer Parity Status Register .......................................... 2­64 PCADR·P­Cache Parity Error Address Register .............................................. 2­66 PCSTS·P­Cache Parity Status Register ........................................................... 2­67 PCCTL·P­Cache Control Register ..................................................................... 2­68 PCTAG·P­Cache Tag Registers ......................................................................... 2­70 PCDAP·P­Cache Data Parity Registers ............................................................ 2­71 2.11.5 Cbox Registers ...................................................................................................... 2­72 ICCS·Interval Count Control and Status Register .......................................... 2­73 NICR·Next Interval Count Register ................................................................. 2­75 ICR·Interval Count Register ............................................................................. 2­76 TODR·Time­of­Day Register ............................................................................. 2­77 BIU_CTL·BIU Control Register ........................................................................ 2­78 DIAG_CTL·Diagnostic Control Register ........................................................... 2­82 BC_TAG·B­Cache Error Tag Register .............................................................. 2­85 BIU_STAT·BIU Status Register ....................................................................... 2­87 BIU_ADDR·BIU Address Register .................................................................... 2­91 FILL_SYND·Fill Syndrome Register ................................................................ 2­92 FILL_ADDR·Fill Address Register ................................................................... 2­94 BEDECC·Software ECC Register ..................................................................... 2­95 CHALT·Console Halt Register .......................................................................... 2­96 Chapter 3 Cache Subsystem 3.1 Virtual Instruction Cache ............................................................................................. 3­3 3.2 Primary Cache ............................................................................................................... 3­4 3.3 Backup Cache ................................................................................................................ 3­5 3.3.1 B­Cache States ........................................................................................................ 3­5 3.3.2 B­Cache State Changes .......................................................................................... 3­6 3.4 Cache Backmaps ........................................................................................................... 3­8 v 3.4.1 P­Map ...................................................................................................................... 3­9 3.4.2 B­Map ...................................................................................................................... 3­9 3.5 Victim Buffer ................................................................................................................. 3­9 3.6 Write Policy ................................................................................................................. 3­10 3.7 B­Cache Operating Modes .......................................................................................... 3­11 3.8 Cache Initialization .................................................................................................... 3­11 Chapter 4 LSB Bus Interface 4.1 LEVI Address Path ....................................................................................................... 4­2 4.2 LEVI Data Path ............................................................................................................ 4­3 4.3 LEVI Controllers ........................................................................................................... 4­4 4.3.1 LEVI Processor Controller ..................................................................................... 4­4 4.3.2 LEVI Data Controller ............................................................................................. 4­4 4.3.3 LSB Controller ........................................................................................................ 4­5 4.4 Interfacing Rules ........................................................................................................... 4­6 4.4.1 Dual­Ported Access Synchronization ..................................................................... 4­6 4.4.2 LSB Arbitration ...................................................................................................... 4­6 4.5 Address Space Mapping ................................................................................................ 4­7 4.6 LEVI Transactions ........................................................................................................ 4­7 4.6.1 Processor­Initiated Transactions ........................................................................... 4­7 4.6.2 LSB­Initiated Transactions ................................................................................... 4­8 4.6.3 Transaction Ordering ............................................................................................. 4­9 Chapter 5 Console Overview 5.1 CPU Console Hardware ................................................................................................ 5­1 5.1.1 Serial ROM ............................................................................................................. 5­2 5.1.2 Serial Port ............................................................................................................... 5­3 5.1.3 FEPROMs ............................................................................................................... 5­3 5.1.4 EEPROM ................................................................................................................. 5­3 5.1.5 UARTs ..................................................................................................................... 5­3 5.1.5.1 Ctrl/P Character Detection and Halt Protection ............................................ 5­4 5.1.5.2 UART Register Addressing ............................................................................. 5­4 5.1.6 Watch Chip ............................................................................................................. 5­4 5.2 Console Program Invocation ......................................................................................... 5­5 5.3 Console Registers .......................................................................................................... 5­5 Gbus$WHAMI ........................................................................................................ 5­8 Gbus$LEDs ........................................................................................................... 5­10 Gbus$PMask ........................................................................................................ 5­11 Gbus$Intr .............................................................................................................. 5­13 Gbus$Halt ............................................................................................................. 5­15 Gbus$LSBRST ...................................................................................................... 5­17 Gbus$Misc ............................................................................................................. 5­18 Gbus$RMode ......................................................................................................... 5­20 Gbus$LTagRW ...................................................................................................... 5­21 Chapter 6 I/O Operations 6.1 Mailbox Data Structure ................................................................................................ 6­1 6.2 Mailbox Operation ........................................................................................................ 6­3 6.3 Device Interrupt Handling ........................................................................................... 6­4 6.4 I/O Operation Registers ................................................................................................ 6­4 vi LMBOX·LSB Mailbox Register ........................................................................... 6­5 Chapter 7 CPU Module Registers 7.1 Register Mapping .......................................................................................................... 7­2 7.2 Register Descriptions ................................................................................................... 7­4 LDEV·Device Register ........................................................................................ 7­5 LBER·Bus Error Register .................................................................................... 7­6 LCNR·Configuration Register ............................................................................. 7­9 LMMR07·Memory Mapping Registers ........................................................... 7­10 LBESR0­3·Bus Error Syndrome Registers ....................................................... 7­12 LBECR0,1·Bus Error Command Registers ...................................................... 7­14 LIOINTR·I/O Interrupt Register ...................................................................... 7­16 LIPINTR·Interprocessor Interrupt Register ................................................... 7­18 LMODE·Mode Register ..................................................................................... 7­20 LMERR·Module Error Register ....................................................................... 7­23 LLOCK·Lock Address Register ......................................................................... 7­25 LDIAG·LSB Diagnostic Control Register ......................................................... 7­26 LTAGA·Tag Address Register .......................................................................... 7­29 LTAGW·Tag Write Data Register .................................................................... 7­30 LCON0,1·Console Communication Registers .................................................. 7­32 LPERF·Performance Counter Control Register .............................................. 7­33 LCNTR0,1·Performance Counter Registers ..................................................... 7­37 LMISSADDR·Last Miss Address Register ...................................................... 7­38 Chapter 8 Initialization 8.1 Initialization Overview ................................................................................................. 8­1 8.2 Self­Test ......................................................................................................................... 8­2 8.2.1 SROM Operation .................................................................................................... 8­2 8.2.2 CPU Module Self­Test ............................................................................................ 8­2 8.2.3 Additional Power­Up Testing ................................................................................ 8­2 8.3 Console Entry ............................................................................................................... 8­3 8.3.1 Boot Processor Arbitration ..................................................................................... 8­3 8.3.2 Boot Processor System Setup ................................................................................. 8­3 8.3.3 Operating System Startup ..................................................................................... 8­3 Chapter 9 Error Handling 9.1 Software Error Handling .............................................................................................. 9­1 9.1.1 Error State Collection ............................................................................................ 9­2 9.1.2 Error Analysis ......................................................................................................... 9­4 9.1.3 Error Recovery ........................................................................................................ 9­4 9.1.3.1 Cache Coherence in Error Handling ............................................................... 9­5 9.1.3.2 Cache Enable, Disable, and Flush Procedures ............................................... 9­6 9.1.3.3 Extracting Data from the B­Cache ................................................................. 9­6 9.1.3.4 Cache and TB Test Procedures ....................................................................... 9­7 9.1.4 Error Retry .............................................................................................................. 9­7 9.2 Error Reports ................................................................................................................ 9­9 9.3 Console Halt and Halt Interrupt ................................................................................ 9­10 9.4 Machine Checks .......................................................................................................... 9­13 9.4.1 MCHK_UNKNOWN_MSTATUS ......................................................................... 9­23 9.4.2 MCHK_INT.ID_VALUE ....................................................................................... 9­24 vii 9.4.3 MCHK_CANT_GET_HERE ................................................................................. 9­24 9.4.4 MCHK_MOVC_STATUS ..................................................................................... 9­24 9.4.5 MCHK_ASYNC_ERROR ...................................................................................... 9­24 9.4.6 MCHK_SYNC_ERROR ........................................................................................ 9­25 9.4.7 Inconsistent Status in Machine Checks .............................................................. 9­31 9.5 Hard Error Interrupts ................................................................................................ 9­32 9.6 Soft Error Interrupts .................................................................................................. 9­40 9.7 Kernel Stack Not Valid Exception ............................................................................. 9­43 9.8 System Environment Errors ...................................................................................... 9­44 9.8.1 Error Categories ................................................................................................... 9­44 9.8.1.1 Synchronous Errors ...................................................................................... 9­44 9.8.1.2 Asynchronous Errors ..................................................................................... 9­44 9.8.2 Environment Error Sources ................................................................................. 9­45 9.8.2.1 LEVI Errors .................................................................................................... 9­45 9.8.2.2 LSB Errors ..................................................................................................... 9­46 Examples 2­1 BIU_CTL Read ............................................................................................................ 2­81 2­2 BIU_CTL Write ........................................................................................................... 2­81 2­3 DIAG_CTL Read ......................................................................................................... 2­84 2­4 DIAG_CTL Write ........................................................................................................ 2­84 9­1 Memory Storage Allocation to the Error State ............................................................ 9­3 9­2 Collection Error State .................................................................................................. 9­3 Figures Figure 1­1 Block Diagram of a VAX 7000 or VAX 10000 System ............................................. 1­1 Figure 1­2 KA7AA CPU Module Block Diagram ....................................................................... 1­2 Figure 2­1 Virtual Address Space Layout .................................................................................. 2­3 Figure 2­2 Physical Address Space Layouts .............................................................................. 2­4 Figure 2­3 SBR and SLR Registers ............................................................................................ 2­5 Figure 2­4 P0BR and P0LR Registers ........................................................................................ 2­5 Figure 2­5 P1BR and P1LR Registers ........................................................................................ 2­6 Figure 2­6 PTE Format (21­Bit PFN) ......................................................................................... 2­6 Figure 2­7 PTE Format (25­Bit PFN) ......................................................................................... 2­7 Figure 2­8 Memory Management Control Registers ................................................................. 2­8 Figure 2­9 Minimum Stack Frame ............................................................................................ 2­9 Figure 2­10 Expanded Stack Frame ......................................................................................... 2­10 Figure 2­11 Arithmetic Exception Stack Frame ...................................................................... 2­11 Figure 2­12 Memory Management Exception Stack Frame ................................................... 2­12 Figure 2­13 Emulated Instruction Trap Stack Frame ........................................................... 2­13 Figure 2­14 Emulated Instruction Fault Stack Frame ........................................................... 2­14 Figure 2­15 Interrupt Control Registers .................................................................................. 2­15 Figure 2­16 System Control Block Base Register .................................................................... 2­18 Figure 2­17 System Control Block Vector ................................................................................ 2­18 Figure 2­18 Process Control Block ............................................................................................ 2­23 Figure 2­19 Processor Status Longword ................................................................................... 2­24 Figure 2­20 NVAX+ Logic Boxes .............................................................................................. 2­25 Figure 2­21 IPR Address Formats ............................................................................................ 2­31 Figure 2­22 Console Dispatch Data Structure ......................................................................... 2­97 Figure 2­23 SYS_TYPE Parameters ......................................................................................... 2­97 Figure 3­1 KA7AA CPU Module Cache Organization. .............................................................. 3­2 Figure 4­1 LEVI Block Diagram ................................................................................................. 4­2 Figure 6­1 Mailbox Data Structure ............................................................................................ 6­2 viii Figure 6­2 Mailbox Pointer Structure ........................................................................................ 6­3 Figure 9­1 Console Saved PC .................................................................................................... 9­11 Figure 9­2 Console Saved PSL .................................................................................................. 9­11 Figure 9­3 Machine Check Exception Stack Frame ................................................................ 9­14 Figure 9­4 Machine Check Exception Parse Tree .................................................................... 9­17 Figure 9­5 Hard Error Interrupt Parse Tree ........................................................................... 9­32 Figure 9­6 Soft Error Interrupt Parse Tree ............................................................................. 9­41 Tables 1 DEC 7000/10000 and VAX 7000/10000 Documentation ............................................. xiii 1­1 NVAX+ CPU Chip Functional Units ............................................................................ 1­3 2­1 KA7AA Module Exceptions ....................................................................................... 2­11 2­2 Arithmetic Exception Codes ....................................................................................... 2­12 2­3 Memory Management Exceptions .............................................................................. 2­12 2­4 Emulated Instruction Trap Stack Frame Parameters .............................................. 2­14 2­5 External Interrupt Requests ...................................................................................... 2­16 2­6 Internal Interrupt Requests ....................................................................................... 2­17 2­7 Software Interrupt Requests ...................................................................................... 2­17 2­8 System Control Block Vector Bit Functions .............................................................. 2­19 2­9 System Control Block Layout ..................................................................................... 2­20 2­10 Processor Status Longword Bits ................................................................................ 2­24 2­11 General Purpose Register Usage ................................................................................ 2­30 2­12 IPR Address Space Decoding ...................................................................................... 2­32 2­13 KA7AA Internal Processor Registers ......................................................................... 2­33 2­14 Identification Registers ............................................................................................... 2­35 2­15 CPUID Register Bit Definitions ................................................................................. 2­36 2­16 SID Register Bit Definitions ....................................................................................... 2­37 2­17 Ibox Registers .............................................................................................................. 2­38 2­18 VMAR Register Bit Definitions .................................................................................. 2­39 2­19 VTAG Register Bit Definitions ................................................................................... 2­40 2­20 VDATA Register Bit Definitions ............................................................................... 2­41 2­21 ICSR Register Bit Definitions .................................................................................... 2­42 2­22 BPCR Register Bit Definitions ................................................................................... 2­43 2­23 BPCR Write Actions .................................................................................................... 2­44 2­24 Ebox Registers ............................................................................................................. 2­45 2­25 PCSCR Register Bit Definitions ................................................................................. 2­46 2­26 Ebox Control Register Bit Definitions ....................................................................... 2­48 2­27 Mbox Registers ............................................................................................................ 2­50 2­28 MP0BR Register Bit Definitions ................................................................................ 2­51 2­29 MP0LR Register Bit Definitions ................................................................................ 2­52 2­30 MP1BR Register Bit Definitions ................................................................................ 2­53 2­31 MP1LR Register Bit Definitions ................................................................................ 2­54 2­32 MSBR Register Bit Definitions ................................................................................. 2­55 2­33 MSLR Register Bit Definitions .................................................................................. 2­56 2­34 MMAPEN Register Bit Definition ............................................................................. 2­57 2­35 PAMODE Register Bit Definition .............................................................................. 2­58 2­36 MMEADR Register Bit Definitions ............................................................................ 2­59 2­37 MMEPTE Address Register Bit Definitions .............................................................. 2­60 2­38 MMESTS Register Bit Definitions ............................................................................ 2­61 2­39 TBADR Register Bit Definitions ............................................................................... 2­63 2­40 TBSTS Register Bit Definitions ................................................................................. 2­64 2­41 PCADR Register Bit Definitions ............................................................................... 2­66 2­42 PCSTS Register Bit Definitions ................................................................................. 2­67 2­43 PCCTL Register Bit Definitions ................................................................................. 2­68 ix 2­44 PCTAG Register Bit Definitions ................................................................................ 2­70 2­45 PCDAP Register Bit Definitions ................................................................................ 2­71 2­46 Cbox Registers ............................................................................................................. 2­72 2­47 ICCS Register Bit Definitions .................................................................................... 2­73 2­48 NICR Register Bit Definitions .................................................................................... 2­75 2­49 ICR Register Bit Definitions ...................................................................................... 2­76 2­50 TODR Register Bit Definitions ................................................................................... 2­77 2­51 BIU_CTL Register Bit Definitions ............................................................................. 2­78 2­52 DIAG_CTL Register Bit Definitions .......................................................................... 2­82 2­53 BC_TAG Register Bit Definitions .............................................................................. 2­85 2­54 BIU_STAT Register Bit Definitions ........................................................................... 2­87 2­55 BIU_ADDR Register Bit Definitions .......................................................................... 2­91 2­56 FILL_SYND Register Bit Definitions ........................................................................ 2­92 2­57 Syndromes for Single­Bit Errors ................................................................................ 2­93 2­58 FILL_ADDR Register Bit Definitions ........................................................................ 2­94 2­59 BEDECC Register Bit Definitions ............................................................................. 2­95 2­60 CHALT Register Bit Definitions ................................................................................ 2­96 2­61 SYS_TYPE Parameter Definitions ............................................................................. 2­98 3­1 Virtual Instruction Cache Attributes ........................................................................... 3­3 3­2 Primary Cache Attributes ............................................................................................ 3­4 3­3 B­Cache States .............................................................................................................. 3­6 3­4 Effect of Processor Action on B­Cache Line ................................................................. 3­7 3­5 Effect of LSB Bus Action on B­Cache Line .................................................................. 3­8 3­6 KA7AA CPU Module Response to Incoming Addresses .............................................. 3­8 3­7 Selection of the B­Cache Operating Mode ................................................................. 3­11 4­1 LSB Command Field Encodings ................................................................................... 4­7 4­2 Processor­LEVI Actions During Transactions ............................................................. 4­9 5­1 Gbus Components ......................................................................................................... 5­2 5­2 Console Registers ......................................................................................................... 5­5 5­3 Gbus$WHAMI Register Bit Definitions ..................................................................... 5­8 5­4 Gbus$LEDs Register Bit Definitions ........................................................................ 5­10 5­5 Gbus$PMask Register Bit Definitions ...................................................................... 5­11 5­6 Gbus$Intr Register Bit Definitions ........................................................................... 5­13 5­7 Gbus$Halt Register Bit Definitions .......................................................................... 5­15 5­8 Gbus$Misc Register Bit Definitions .......................................................................... 5­18 6­1 Mailbox Data Structure ................................................................................................ 6­2 6­2 Mailbox Pointer Structure ............................................................................................ 6­3 6­3 KA7AA CPU Interrupts ................................................................................................ 6­4 6­4 LMBOX Register Bit Definitions ................................................................................. 6­5 7­1 LSB Node Space Base Addresses ................................................................................. 7­2 7­2 CPU Module Registers .................................................................................................. 7­3 7­3 LDEV Register Bit Definitions ..................................................................................... 7­5 7­4 LBER Register Bit Definitions ..................................................................................... 7­7 7­5 LCNR Register Bit Definitions ..................................................................................... 7­9 7­6 LMMR Register Bit Definitions ................................................................................. 7­10 7­7 LBESR Register Bit Definitions ................................................................................. 7­12 7­8 Syndromes for Single­Bit Errors ................................................................................ 7­13 7­9 LBECR Register Bit Definitions ................................................................................ 7­14 7­10 LIOINTR Register Bit Definitions ............................................................................. 7­16 7­11 LSB Interrupt Mapping .............................................................................................. 7­17 7­12 LIPINTR Register Bit Definitions ............................................................................. 7­18 7­13 LMODE Register Bit Definitions ............................................................................... 7­20 7­14 LMERR Register Bit Definitions ................................................................................ 7­23 7­15 LLock Register Bit Definitions ................................................................................... 7­25 7­16 LDIAG Register Bit Definitions ................................................................................. 7­26 7­17 LTAGA Register Bit Definitions ................................................................................ 7­29 x 7­18 LTAGW Register Bit Definitions ............................................................................... 7­30 7­19 LCON Register Bit Definitions .................................................................................. 7­32 7­20 LPERF Register Bit Definitions ................................................................................. 7­33 7­21 LCNTR Register Bit Definitions ................................................................................ 7­37 7­22 LMISSADDR Register Bit Definitions ....................................................................... 7­38 9­1 Error Categories by SCB Entry Points ...................................................................... 9­10 9­2 Console Halt Codes ..................................................................................................... 9­12 9­3 CPU State Initialized on Console Halt ...................................................................... 9­13 9­4 Machine Check Stack Frame Fields .......................................................................... 9­15 9­5 Machine Check Codes in the Stack Frame ................................................................ 9­16 xi Preface Intended Audience This manual is written for developers of system and application software based on the LSB platform of VAX computer systems. It assumes ma­ chine level programming knowledge and familiarity with the OpenVMS VAX operating system. Document Structure The material is presented in nine chapters. · Chapter 1, CPU Module Overview, presents an overall introduction to the KA7AA CPU module. · Chapter 2, CPU Chip, describes the functions and operations of the central processor of the KA7AA CPU module. It discusses such topics as addressing, memory management, exceptions and interrupts, func­ tional partitions of the CPU chip, and internal processor registers. · Chapter 3, Cache Subsystem, describes the elements and operations of the cache hierarchy, which includes the virtual instruction cache, the primary cache, and the backup cache. · Chapter 4, LSB Bus Interface, describes the functions and operations of the LEVI gate arrays that provide the CPU module interface to the LSB bus. It discusses processor­initiated and LSB bus­initiated trans­ actions, LEVI address and data paths, and the LEVI controllers. · Chapter 5, Console Overview, gives a brief description of the various elements that comprise the console. It also describes the Gbus regis­ ters, which perform console control, diagnostic, and interrupt­related functions. · Chapter 6, I/O Operations, describes the mailbox data structure, the operation of the mailbox, interrupt handling, and the I/O registers. · Chapter 7, CPU Module Registers, lists the LSB required and CPU­ specific registers, and provides bit­level functional descriptions of each register. · Chapter 8, Initialization, gives an overview of the CPU module in­ itialization, describes the methods and process of initialization, system configuration, and bootstrapping of the operating system. · Chapter 9, Error Handling, describes how the KA7AA module han­ dles various types of errors. It discusses error analysis and recovery, machine check exceptions, and sources of error interrupts. Parse trees included in this chapter help isolate errors to particular causes. xii Conventions Used in This Document Results and Operations Results of operations termed UNPREDICTABLE must not be used by soft­ ware. Operations termed UNDEFINED do not cause the processor to hang, that is, reach a state from which there is no transition to a normal state of in­ struction execution. Nonprivileged software cannot invoke UNDEFINED operations. Register and Bit Designations Certain conventions are followed in register descriptions and in references to bits and bit fields: · Registers are referred to with their mnemonics, such as VMAR regis­ ter. The full name of a register (for example, VIC Memory Address Register) is spelled out only at the top of the register description page, or when the register is first introduced. · Acronyms are used in register description tables to indicate the access type of the bit(s). · Bits and fields are enclosed in angle brackets. For example, bit <31>; bits <31:16>. For clarity of reference, bits are usually specified by their numbers or names enclosed in angle brackets adjacent to the reg­ ister mnemonic, such as VMAR<31:11> or VMAR, which are equivalent designations. Acronym Access Type RC RO R/W R0 WO W1C W1S Read to clear. The value is written by hardware and remains unchanged until read by software or microcode. Read only. May be read by software, microcode, or hardware. Written by hardware. Software or microcode writes are ignored. Read/write. May be read and written by software, microcode, or hardware. Reads as zero. Read only. Writes are ignored. Write only. May be written by software or microcode. It is read by hardware. Reads by software or microcode return an unpredictable value. Write 1 to clear. The value may be read by software or microcode. Software or microcode writes of 1 to the position cause hardware to clear the bit. Software or microcode writes of 0 do not modify the state of the bit. Write 1 to set. May be read and written by software, microcode, or hardware. Set by software or microcode with a write of 1. xiii · When the value of a bit position is given explicitly in a register dia­ gram, the information conveyed is as follows: · Fields (in registers or data structures) noted as must be zero (MBZ) must never be filled by software with a nonzero value. If the processor encounters a nonzero value in an MBZ field, a Reserved Operand Ex­ ception occurs. · Fields (in registers or data structures) noted as should be zero (SBZ) should be filled by software with a zero value. A nonzero value in an SBZ field produces UNPREDICTABLE results. · The entry in the Type column of a register description table may in­ clude the initialization values of the bits. For example, entry R/W, 0 indicates a read/write bit that is initialized to zero. Documentation Titles Table 1 lists the books in the DEC 7000/10000 and VAX 7000/10000 docu­ mentation sets. Table 1 DEC 7000/10000 and VAX 7000/10000 Documentation Bit Value Meaning 0 1 X Reads as zero; ignored on writes. Reads as one; ignored on writes. Does not exist in hardware. The value of the bit is UNPREDICTABLE on reads and ignored on writes. Title 7000 Systems Order Number 10000 Systems Order Number Installation Kit EK7000BDK EK1000BDK Site Preparation Guide EK7000BSP EK1000BSP Installation Guide EK700EBIN EK100EBIN Hardware User Information Kit EK7001BDK EK1001BDK Operations Manual EK7000BOP EK1000BOP Basic Troubleshooting EK7000BTS EK1000BTS Service Information Kit·VAX 7000 EK7002ADK EK1002ADK Platform Service Manual EK7000ASV EK1000ASV System Service Manual EK7002BSV EK1002ASV Pocket Service Guide EK7000APG EK1000APG Advanced Troubleshooting EK7001ATS EK1001ATS xiv Table 1 DEC 7000/10000 and VAX 7000/10000 Documentation (Continued) Title 7000 Systems Order Number 10000 Systems Order Number Reference Manuals Console Reference Manual EK70C0BTM KA7AA CPU Technical Manual EKKA7AATM KN7AA CPU Technical Manual EKKN7AATM MS7AA Memory Technical Manual EKMS7AATM I/O System Technical Manual EK70I0ATM Platform Technical Manual EK7000ATM Upgrade Manuals KA7AA CPU Installation Card EKKA7AAIN KN7AA CPU Installation Card EKKN7AAIN MS7AA Memory Installation Card EKMS7AAIN KZMSA Adapter Installation Guide EKKXMSXIN DWLMA XMI PIU Installation Guide EKDWLMAIN DWMBB VAXBI PIU Installation Guide EKDWMBBIN H7237 Battery PIU Installation Guide EKH7237IN H7263 Power Regulator Installation Card EKH7263IN Futurebus+ PIU Installation Guide EKDWLAAIN BA654 DSSI Disk PIU Installation Guide EKBA654IN BA655 SCSI Disk and Tape PIU Installation Guide EKBA655IN Removable Media Installation Guide EKTFRRDIN CPU Module Overview 1­1 Chapter 1 CPU Module Overview The KA7AA CPU module is designed around the NVAX+ microprocessor chip (P/N DC262) and communicates with main memory and I/O subsys­ tems by way of the LSB bus. Figure 1­1 shows how the KA7AA CPU module fits in a VAX computer system based on the LSB bus. Figure 1­1 Block Diagram of a VAX 7000 or VAX 10000 System The CPU module supports the VAX base instruction group of 242 instruc­ tions and associated data types, an address space of 4 gigabytes, and pro­ vides full memory management. A floating­point accelerator and a three­level hierarchical instruction and data cache structure allow the CPU module to achieve a minimum scalar single­CPU performance of more than 20 times that of a VAX 11/780 on an average workload. The CPU module includes VAX­compatible macrocoded console firm­ ware. Microcode and on­board ROMs permit booting from supported de­ vices and provide self­test diagnostics on power­up. A multiprocessor VAX 7000 or VAX 10000 system can be configured with up to six CPU modules. All backplane slots except slot 8, which is dedi­ cated to the IOP module, can be used interchangeably with memory mod­ ules, allowing an array of configurations tailored to specific application requirements. It is strongly recommended, however, that the first CPU module be placed in slot 0. 1­2 CPU Module Overview The KA7AA CPU module is comprised of three major sections: · CPU chip (NVAX+) · Backup cache (B­cache) · LSB interface (LEVI) Figure 1­2 shows the major sections of the CPU module. Figure 1­2 KA7AA CPU Module Block Diagram CPU Module Overview 1­3 1.1 NVAX+ CPU Chip The NVAX+ processor is a single­chip macropipelined implementation of the VAX architecture and the VAX base instruction group. It is parti­ tioned into five functional units, or boxes, as listed in Table 1­1. Table 1­1 NVAX+ CPU Chip Functional Units The NVAX+ CPU chip includes: · Translation buffer: 96 entries, fully associative · Virtual instruction cache (VIC): Two Kbytes, direct­mapped, virtually addressed, I­stream only · Primary cache (P­cache): Eight Kbytes, 2­way set associative, physi­ cally addressed, write through, mixed I­stream and D­stream, 32­byte block and fill size · Control store: 1600 61­bit microwords · EDAL interface: Cache and memory subsystem interface The macroinstruction pipelined design of the NVAX+ CPU chip allows sig­ nificant parallel processing. NVAX+ pipelines macroinstruction decode and operand fetch with macroinstruction execution. Pipeline efficiency is increased by queuing up instruction information and operand values for later use by the execution unit. Thus, when the macropipeline is operating smoothly, the instruction unit (Ibox), which parses instructions and fetches operands, is running several macroinstructions ahead of the execution unit (Ebox). Branch predictions allow quicker execution of loops in soft­ ware. Outstanding writes to registers or memory locations are kept in a scoreboard to ensure that data is not read before it has been written. NVAX+ Unit Functional Description Ibox Ebox Fbox Mbox Cbox Instruction box. Prefetches, decodes, parses, and queues VAX instructions for execution. Execution box. Executes, together with the microsequencer, instructions received from the Ibox. Floating­point accelerator box. Executes floating­point and integer multiply VAX instructions received from the Ebox. Memory management box. Performs all virtual to physical address translations, for the Ibox and Ebox. The Mbox also handles Cbox requests for cache fills and invalidates for the primary cache. Cache control box. Initiates access to the backup cache (B­cache) and issues memory requests. 1­4 CPU Module Overview 1.2 Backup Cache (B­Cache) The external backup cache (B­cache) is a 4­Mbyte superset of the P­cache. It is a physically addressed, direct mapped, write back, mixed I­stream and D­stream cache with a block and fill size of 64 bytes. It consists of three sets of RAMs: B­data B­tag B­stat Each block of data (B­data) has a tag (B­tag) and three status bits (B­stat) associated with it. The status bits are Valid, Dirty, and Shared. The B­cache has controllable data RAM access time. The CPU module sup­ ports cache access times of 2, 3, and 4 times the NVAX+ chip cycle time. 1.3 LSB Interface (LEVI) The interface to the LSB bus is called LEVI, which consists of two chips: LEVI­A and LEVI­B. LEVI­A contains most LSB required registers, im­ plements all command execution, LSB arbitration, and B­cache manipula­ tion functions. It also contains a P­cache backmap (P­map) to allow the CPU to do invalidate filtering and to make intelligent update vs. invalidate decisions in response to LSB write traffic. LEVI­A uses an external RAM structure to implement a backmap of the B­cache to filter bus traffic from the B­cache while still maintaining cache coherence. LEVI­B completes the 128­bit data path between the NVAX+ and the LSB bus. An 8­bit communication bus between LEVI­A and LEVI­B provides a path that allows LEVI­B to perform look­aside ECC checking on incoming memory traffic. CPU Chip 2­1 Chapter 2 CPU Chip The NVAX+ CPU chip is the central processor of the KA7AA CPU mod­ ule. It executes the VAX base instruction group and provides full VAX memory management. This chapter provides a summary of the architec­ tural features and functional elements of the NVAX+ CPU chip. Sections include: · Data types · Instruction set · Address space · Memory management · Exceptions and interrupts · System control block · Process structure · Functional partitions · General purpose registers · Internal processor registers For more information on some of these topics, consult the VAX Architec­ ture Reference Manual. 2­2 CPU Chip 2.1 Data Types The NVAX+ CPU chip supports the following subset of VAX data types: Byte Word Longword Quadword Variable­length bit field Character string Absolute queues Self­relative queues D_floating F_floating G_floating The remaining VAX data types (octaword, H_floating, trailing numeric string, leading separate numeric string, and packed decimal string) are supported by macrocode emulation. 2.2 Instruction Set The NVAX+ hardware supports the following instruction classes: Integer arithmetic and logical Address Variable­length bit field Control Procedure call Miscellaneous Queue Character string Operating system support D_floating F_floating G_floating A subset of the VAX instruction group is emulated by NVAX+ microcode. See the VAX Architecture Reference Manual for detailed descriptions of the VAX instruction set. 2.3 CPU Chip 2­3 2.4 Address Space The memory space that a programmer uses is a single logical structure of contiguous pages. It is called virtual memory and is referenced in virtual address space. The memory space that the operating system allocates to a process is a structure of noncontiguous pages called physical memory and is referenced in the physical address space. The operating system uses the page mapping scheme because in a multiuser and multitasking envi­ ronment no available block in physical memory may be large enough to ac­ commodate a block of contiguous pages generated in virtual memory. 2.4.1 Virtual Address Space Software can reference a 32­bit virtual address. This address width allows access to a virtual address space of 4 gigabytes (2**32 locations). The vir­ tual address space is divided into two sections, process space and system space. The process space is further partitioned into two regions, P0 and P1. All the virtual address space except a reserved region in the system space is accessible to software. Figure 2­1 shows the virtual address space layout. NOTE: NVAX+ implements the VAX extended system region address space. The ex­ tension allows the chip to address 222­1 pages of system space. Figure 2­1 Virtual Address Space Layout 2.4.2 Physical Address Space The NVAX+ CPU generates 32­bit physical addresses. This address width can access 4 gigabytes of physical address space. Memory space occupies the first seven­eighths (3.5 Gbytes) of the physical address space. I/O space occupies the last one­eighth (512 Mbytes) of the physical address space and can be distinguished from memory space by the fact that bits <31:29> of the physical address are all ones. The NVAX+ CPU chip can also run in a 30­bit address mode if selected by the programmer. In this mode the NVAX+ CPU chip can reference only 2­4 CPU Chip one gigabyte of total address space, evenly divided into memory space and I/O space. Figure 2­2 shows the physical address space layouts. Figure 2­2 Physical Address Space Layouts During power­up, microcode configures the NVAX+ CPU chip to generate 30­bit physical addresses. Operating system initialization code can reconfigure the CPU module to generate either 30­bit or 32­bit physical ad­ dresses by writing to the Mode bit (<0>) in the PAMODE register (IPR231). 2.5 Memory Management Memory management is the translation of virtual addresses (contiguous page addresses used by the programmer) into physical addresses (noncon­ tiguous page addresses in physical memory space). The central processor extracts the virtual page number (VPN) from the virtual address (bits <29:9>) and generates a page frame number (PFN), which makes up bits <31:9> of the physical address. The physical address is then formed by ap­ pending bits <8:0> of the virtual address (address of the byte within the page) to the PFN. 2.5.1 System Space Address Translation A virtual address in system space is mapped by the system page table (SPT), which is defined by the System Base Register (SBR) and the System Length Register (SLR). The SBR register contains the page­aligned physi­ cal address of the SPT. The SLR contains the size of the SPT in longwords, that is, the number of page table entries (PTE). The PTE addressed by the SBR register maps the first page of the system virtual address space, that is, virtual byte address 8000 0000 (hex). Figure 2­3 shows the SBR and SLR registers. NOTE: When the CPU module is configured to generate 30­bit physical addresses, SBR<31:30> are ignored. CPU Chip 2­5 Figure 2­3 SBR and SLR Registers 2.5.2 Process Space Address Translation A virtual address with bit <31> = 0 is an address in the process virtual ad­ dress space. Process space is divided into two equal sized, separately mapped regions. If virtual address bit <30> = 0, the address is in region P0. If virtual address bit <30> = 1, the address is in region P1. 2.5.2.1 P0 Region Address Translation The P0 region of the address space is mapped by the P0 page table (P0PT), which is defined by the P0 Base Register (P0BR) and the P0 Length Reg­ ister (P0LR). The P0BR register contains the system page­aligned virtual address of the P0 page table. The P0LR register contains the size of the P0PT in longwords, that is, the number of PTEs. The PTE addressed by the P0BR register maps the first page of the P0 region of the virtual ad­ dress space, that is, the virtual byte address 0. Figure 2­4 shows the P0BR and P0LR registers. Figure 2­4 P0BR and P0LR Registers 2.5.2.2 P1 Region Address Translation The P1 region of the address space is mapped by the P1 page table (P1PT), which is defined by the P1 Base Register (P1BR) and the P1 Length Regis­ ter (P1LR). Because P1 space grows toward smaller addresses, and be­ cause a consistent hardware interpretation of the base and length register­ 2­6 CPU Chip sis desirable, the P1BR register and the P1LR register describe the portion of P1 space that is not accessible. Note that the P1LR register contains the number of nonexistent PTEs. The P1BR contains the page­aligned virtual address of what would be the PTE for the first page of P1, that is, virtual byte address 4000 0000 (hex). The address in the P1BR register is not nec­ essarily an address in system space, but all the addresses of PTEs are in system space. Figure 2­5 shows the P1BR and P1LR registers. Figure 2­5 P1BR and P1LR Registers 2.5.3 Page Table Entry Format When the CPU module is configured to generate 30­bit physical addresses, it interprets PTEs in the 21­bit PFN format shown in Figure 2­6. Figure 2­6 PTE Format (21­Bit PFN) If the CPU module is configured to generate 32­bit physical addresses, it interprets PTEs in the 25­bit PFN format shown in Figure 2­7. CPU Chip 2­7 Figure 2­7 PTE Format (25­Bit PFN) Note that bits <24:23> of the 25­bit PFN format are ignored by the NVAX+ CPU chip. 2.5.4 Translation Buffer The NVAX + CPU chip implements a 96­entry, fully associative, fast access translation buffer to minimize memory accesses when repeatedly referenc­ ing the same memory pages. Each entry stores a PTE for translating vir­ tual addresses to physical addresses in either process space or system space. The translation buffer caches the PTEs of recent virtual address translations. It is accessed in parallel with each memory reference. Each translation buffer entry is divided into two parts: a 26­bit tag register and a 29­bit PTE register. The tag register is used to store the VPN of the virtual page that the corresponding PTE register maps. The tag regis­ ter also contains a valid bit (TBV) that indicates a valid VPN in the tag. During virtual­to­physical address translation, the contents of the 96 tag registers are compared with the VPN field (bits <31:9>) of the virtual ad­ dress of the reference. If there is a match with one of the tag registers and the TBV bit indicates a valid entry, the translation buffer sends the correct physical address contained in the corresponding PTE register to the cache. If no match occurs, the PTE that maps the page is fetched from memory through the normal memory access stages (P­cache, B­cache, and then main memory) and the translation buffer is updated by replacing the entry at the location indicated by the replacement pointer. Translation buffer entries are replaced using a not­last­used (NLU) algo­ rithm. This algorithm guarantees that the replacement pointer is not pointing at the last translation buffer entry to be used. This is accom­ plished by rotating the replacement pointer to the next sequential transla­ tion buffer entry. Both D­stream and I­stream references can cause the NLU algorithm to execute. 2.5.5 Memory Management Control Three processor registers control memory management: · Memory Management Enable Register (MAPEN), IPR56 · Translation Buffer Invalidate Single Register (TBIS), IPR58 · Translation Buffer Invalidate All Register (TBIA), IPR57 2­8 CPU Chip Bit <0> of the MAPEN register enables memory management if set to a one and disables memory management if written with a zero. The MAPEN register is written to with a Move To Processor Register (MTPR) instruction. It is read with a Move From Processor Register (MFPR) in­ struction. The TBIS register controls single­entry invalidation in the translation buffer. Entries that map a particular virtual address are invalidated by writing the virtual address to the TBIS register using the MTPR instruc­ tion. The TBIA register controls total translation buffer invalidation. Writing a zero into the TBIA register invalidates the entire translation buffer. Fig­ ure 2­8 shows the three memory control registers. CAUTION: All affected process pages must be invalidated in the translation buffer whenever software changes one of the following: · A valid PTE for the system or the current process region · A system PTE that maps any part of the current process page table Otherwise, address translations point to wrong locations in memory. Figure 2­8 Memory Management Control Registers The base and length of the P0, P1, and S0 page tables are changed by writ­ ing the appropriate address or length to the P0BR, P0LR, P1BR, P1LR, SBR, or SLR register. The entire translation buffer is flushed when a change is made to any of these six registers. When a process context is loaded with the Load Process Context (LDPCTX) instruction, all translation buffer entries that map process space pages are automatically invalidated. System­space mappings are preserved. To determine if the translation buffer contains a valid translation for a particular virtual page, write a virtual address within that page to TBCHK using an MTPR instruction. If the translation buffer contains a valid translation for the page, the condition code V bit (PSL<1>) would be set. CPU Chip 2­9 A full invalidation of the translation buffer, whether performed with an ex­ plicit write to the TBIA register, or as an implied clear due to writes to the MAPEN register or any base/length register, resets the NLU pointer to the first location in the translation buffer. NOTE: The contents of the translation buffer are UNPREDICTABLE whenever memory management is disabled. The entire translation buffer contents should be flushed before memory management is enabled. The console firm­ ware performs this function as the system is booted. 2.6 Exceptions and Interrupts At certain times during the operation of a system, events may occur that break the explicit flow of instructions in the current process and require execution of external software routines. Two types of events can cause such a break: exceptions and interrupts. An exception is an event related to the currently executing process and in­ vokes a software routine in the context of the current process. Exception handlers are often system routines, not process routines. An interrupt is an event caused by some activity outside the current proc­ ess and invokes a software routine outside the context of the current proc­ ess. The CPU chip reports exceptions and interrupts by constructing a frame on the stack and then dispatching to the service routine through an event­ specific vector in the system control block (SCB, Section 2.7). The mini­ mum stack frame for any interrupt and exception is a program counter/ processor status longword (PC/PSL) pair, as shown in Figure 2­9. Figure 2­9 Minimum Stack Frame The minimum stack frame is used for all interrupts. Certain exceptions expand the stack frame by pushing additional parameters on the stack above the PC/PSL pair, as shown in Figure 2­10. Parameters that are pushed on the stack above the PC/PSL pair depend on the type of exception. 2­10 CPU Chip Figure 2­10 Expanded Stack Frame 2.6.1 Exceptions There are three types of exceptions: · Traps · Faults · Aborts A trap occurs at the end of an instruction. Therefore, the PC saved on the stack is the address of the next instruction that would normally have been executed had the trap not occurred. A fault occurs during the execution of an instruction and leaves the regis­ ters and memory in a consistent state, so that eliminating the fault condi­ tion and restarting the instruction gives correct results. The PC saved on the stack points to the instruction that faulted. An abort occurs during the execution of an instruction and leaves the value of the registers and memory UNPREDICTABLE, so that the instruc­ tion cannot be restarted, completed, simulated, or undone. In most cases the NVAX+ microcode attempts to convert an abort into a fault by restor­ ing the state preceding the start of the instruction that caused the abort. This section lists VAX standard exceptions and discusses their implemen­ tation on the KA7AA CPU module. The standard exceptions are described in the VAX Architecture Reference Manual. The VAX architecture recognizes six categories of exceptions. Table 2­1 lists the types of exceptions in each category. Specific characteristics are found in the following exceptions implemented on the NVAX+ chip: · Arithmetic · Memory management · Emulated instruction · System failure CPU Chip 2­11 Table 2­1 KA7AA Module Exceptions 2.6.1.1 Arithmetic Exceptions Arithmetic exceptions are detected during the execution of integer or floating­point arithmetic instructions. Figure 2­11 shows the arithmetic exception stack frame. Figure 2­11 Arithmetic Exception Stack Frame The exception is reported as either a trap or a fault, depending on the spe­ cific event. Table 2­2 shows the encoding of arithmetic exceptions. Exception Category Type Arithmetic traps/faults Memory management exceptions Operand reference exceptions Instruction execution exceptions Tracing exceptions System failure exceptions Integer overflow trap Integer divide­by­zero trap Subscript range trap Floating overflow fault Floating divide by zero fault Floating underflow fault Access control violation fault (ACV) Translation not valid fault (TNV) Modify fault (M) Reserved addressing mode fault Reserved operand fault or abort Emulated instruction fault Extended function (XFC) fault Change­mode trap Breakpoint fault Trace fault Kernel stack not valid abort Interrupt stack not valid fault Console error halt Machine check abort 2­12 CPU Chip Table 2­2 Arithmetic Exception Codes 2.6.1.2 Memory Management Exceptions Memory management exceptions are detected during a memory reference and are always reported as faults. The memory management exceptions and the associated SCB vectors are listed in Table 2­3. Table 2­3 Memory Management Exceptions All memory management exceptions push the same frame on the stack, as shown in Figure 2­12. Figure 2­12 Memory Management Exception Stack Frame The M, P, and L bits of the parameter pointed to by the stack pointer re­ flect bits <2:0> of the MMESTS register, which are described in Table 2­38. Code (Hex) Type Exception 1 2 7 8 9 A Trap Trap Trap Fault Fault Fault Integer overflow Integer divide­by­zero Subscript range Floating overflow Floating divide­by­zero Floating underflow Exception SCB Vector (Hex) Access control violation (ACV) Translation not valid (TNV) Modify fault (M) 20 24 3C CPU Chip 2­13 2.6.1.3 Emulated Instruction Exceptions The NVAX+ chip implements the VAX base instruction group in hardware and provides microcode support for macrocode emulation of certain other instructions. Two types of emulation exceptions depend on the state of PSL. If FPD is zero at the beginning of the instruction, then the exception is reported through SCB vector C8 (hex) as a trap, with the stack frame shown in Figure 2­13. Figure 2­13 Emulated Instruction Trap Stack Frame Table 2­4 describes the emulated instruction stack frame parameters. 2­14 CPU Chip Table 2­4 Emulated Instruction Trap Stack Frame Parameters If PSL (see Figure 2­19) is one at the beginning of the instruction, then the exception is reported through SCB vector CC (hex) as a fault, with the stack frame as shown in Figure 2­14. In this case, PC is the opcode of the emulated instruction. Figure 2­14 Emulated Instruction Fault Stack Frame 2.6.1.4 System Failure Exceptions A machine check exception is reported through SCB vector 04 (hex) when the NVAX+ chip detects an error condition. The frame pushed on the stack for a machine check indicates the type of error and provides internal state information that may help identify the cause of the error. Machine check exceptions are discussed at length in Chapter 9 (Error Handling). In certain microcode flows, the NVAX+ microcode may detect an inconsis­ tency in internal state, a kernel­mode HALT, or a system reset. In these instances, the microcode initiates a hardware restart sequence which passes control to the console program. This process is called console halt. When a console halt occurs, the NVAX+ microcode saves the current CPU state, partially initializes the CPU, and passes control to the console pro­ gram at the physical address contained in the CHALT register. During a hardware restart sequence, the stack pointer is saved in the ap­ propriate stack pointer (0 through 4), the current PC is saved in IPR42 (SAVPC register), and the current PSL, halt code, and validity flag are Parameter Function Opcode Old PC Specifiers New PC PSL Zero­extended opcode of the emulated instruction. Program counter of the opcode of the emulated instruction. Address of the specified operand for specifiers of either access type write (.wx) or address (.ax). Operand value for specifiers of access type read (.rx). For read­type operands whose size is smaller than a longword, the remain­ ing bits are UNPREDICTABLE. For those instructions that do not have eight specifiers, the remaining specifier longwords contain UNPREDICTABLE values. Program counter of the instruction following the emulated instruction. PSL saved at the time of the trap. CPU Chip 2­15 saved in IPR43 (SAVPSL register). Console halts are discussed at length in Chapter 9 (Error Handling). 2.6.2 Interrupts The interrupt section receives interrupt requests from both internal and external sources. When an interrupt request is generated, the NVAX+ chip compares the request with the current IPL of the CPU. If the new re­ quest is of higher priority, an internal request is generated. At the comple­ tion of the current instruction, or at selected points during the execution of interruptable instructions, a microcode interrupt handler is invoked to process the request. The microcode handler determines the highest prior­ ity interrupt, updates the IPL, pushes a PC/PSL pair on the stack, and dis­ patches to a macrocode interrupt handler through the appropriate location in the SCB. The interrupt system is controlled by three IPRs: · Interrupt Priority Level Register (IPL), IPR18 · Software Interrupt Request Register (SIRR), IPR20 · Software Interrupt Summary Register (SISR), IPR21 The IPL register is used for loading the interrupt priority level field (IPL<4:0>) into PSL<20:16>. The SIRR register is used for creating soft­ ware interrupt requests. The SISR register records pending software inter­ rupt requests at levels 1 through 15. Figure 2­15 shows the three inter­ rupt control registers. Figure 2­15 Interrupt Control Registers 2­16 CPU Chip 2.6.2.1 External Interrupt Requests NVAX+ inputs six external interrupt signals as IRQ_H<3:0>, HALT_H, and ERR_H. These signals request general­purpose interrupts at the fol­ lowing IPLs: · HALT_H: The assertion of HALT_H causes the CPU to enter the con­ sole at IPL 1F (hex) at the next macroinstruction boundary. The inter­ rupt is not gated by the current IPL, and always results in console en­ try, even if the IPL is already 1F (hex). Note that unlike normal interrupts, which cause a PC/PSL pair to be pushed on the interrupt stack, HALT_H interrupts store the current PC/PSL and halt code in the SAVPC and SAVPSL processor registers. Console halts are dis­ cussed at length in Chapter 9 (Error Handling). · ERR_H: The assertion of ERR_H indicates that an error has been de­ tected in the system environment. This results in the dispatch of the interrupt to the operating system at IPL 1D (hex) through SCB vector 60 (hex). · IRQ<3:0>: Device interrupts resulting in dispatch of the interrupt to the operating system at IPL 1417 (hex) through SCB vector D0, D4, D8, or DC (hex). Table 2­5 shows the external interrupt request levels and the associated SCB vectors. Interrupt routines at the specified SCB acknowledge the in­ terrupt. Table 2­5 External Interrupt Requests 2.6.2.2 Internal Interrupt Requests The Cbox, Ibox, and Mbox report error conditions by asserting internal in­ terrupt request signals. Table 2­6 gives the interrupt requests caused by internal hard and soft errors. Interrupt Request Request IPL (Hex) (Decimal) Interrupt Condition SCB Vector (Hex) HALT_H ERR_H IRQ<3> IRQ<2> IRQ<1> IRQ<0> 1F 31 1D 29 17 23 16 22 15 21 14 20 Ctrl/P typed at the console, LCNR set, node reset, or system reset Hardware error Device interrupt Device interrupt Device interrupt Device interrupt Console 60 DC D8 D4 D0 CPU Chip 2­17 Table 2­6 Internal Interrupt Requests The performance monitoring facility requests an interrupt at IPL 1B (hex) when the performance counters become half full. This request is serviced entirely by microcode. It is cleared by writing to the appropriate interrupt service routine. When the interval timer period expires (ICCS<6> is set), the interrupt is dispatched to the operating system at IPL 16 (hex) through SCB vector C0 (hex). An architecturally defined software interrupt request causes an associated bit to be set in the SISR register. The request is dispatched to the operat­ ing system. The IPL of the SCB vector is implied by the interrupt request. Table 2­7 shows the association between an SISR register bit, request IPL, and SCB vector. Table 2­7 Software Interrupt Requests Interrupt Request Request IPL (Hex) (Decimal) SCB Vector (Hex) H_ERR S_ERR 1D 29 1A 26 60 54 SISR Bit Set Request IPL (Hex) (Decimal) SCB Vector (Hex) <15> <14> <13> <12> <11> <10> <9> <8> <7> <6> <5> <4> <3> <2> <1> 0 15 0E 14 0D 13 0C 12 0B 11 0A 10 09 09 08 08 07 07 06 06 05 05 04 04 03 03 02 02 01 01 BC B8 B4 B0 AC A8 A4 A0 9C 98 94 90 8C 88 84 2­18 CPU Chip 2.7 System Control Block The system control block (SCB) is a page containing vectors for servicing interrupts and exceptions. The SCB is pointed to by the 32­bit System Block Base Register (SCBB). Since the SCBB contains a 32­bit physical address, the SCB can reside anywhere in memory space. For optimum performance, the SCBB should contain a page­aligned address. Microcode forces longword alignment by clearing bits <1:0> of the new value before loading it into the SCBB register. Figure 2­16 shows the format of the SCBB register. Figure 2­16 System Control Block Base Register NOTE: When When the CPU is in 30­bit physical address mode, SCBB<31:30> are ignored. An SCB vector is an aligned longword in the SCB. The NVAX+ chip microcode dispatches interrupts and exceptions through the SCB vector, shown in Figure 2­17. Figure 2­17 System Control Block Vector Table 2­8 describes the bit functions of the SCB vector. CPU Chip 2­19 Table 2­8 System Control Block Vector Bit Functions Table 2­9 shows the SCB layout. Bit(s) Function <31:2> <1:0> Virtual address of the service routine for the interrupt or exception. The routine is longword aligned, as the microcode forces the lower two bits of the address to 00 (hex). These two bits are decoded as follows: SCB<1:0> Function 00 01 10 11 The event is to be serviced on the kernel stack unless the CPU is already on the interrupt stack, in which case the event is to be serviced on the interrupt stack. The event is to be serviced on the interrupt stack. If the event is an exception, the IPL is raised to 1F (hex). Unimplemented; results in a console error halt. Unimplemented; results in a console error halt. 2­20 CPU Chip Table 2­9 System Control Block Layout Vector (Hex) Name Type1 No. of Parameters Notes 00 04 08 0C 10 14 18 1C 20 24 28 2C 30 34 383C 40 44 48 4C Unused Machine check Kernel stack not valid Unused Reserved/privileged instruction Customer reserved instruction Reserved operand Reserved addressing mode Access control violation/ vector alignment fault Translation not valid Trace pending Breakpoint instruction Unused Arithmetic Unused CHMK CHME CHMS CHMU A A F F A/F F F F F F F/T T T T T 6 0 0 0 0 0 2 2 0 0 1 1 1 1 1 Parameters reflect ma­ chine state; must be serv­ iced on interrupt stack Must be serviced on inter­ rupt stack XFC instruction Not always recoverable Parameters are virtual ad­ dress, status code Parameters are virtual ad­ dress, status code Compatibility mode in other VAX systems Parameter is type code Parameter is sign­extended operand word Parameter is sign­extended operand word Parameter is sign­extended operand word Parameter is sign­extended operand word 1  A = Abort; F = Fault; T = Trap; I = Interrupt. CPU Chip 2­21 Table 2­9 System Control Block Layout (Continued) Vector (Hex) Name Type1 No. of Parameters Notes 50 54 58 595C 60 64­80 84 88 8C 90BC C0 C4 C8 CC D0 D4 D8 DC E0FFFC Unused Soft error notification Performance monitoring counter overflow Unused Hard error notification Unused Software level 1 Software level 2 Software level 3 Software levels 415 Interval timer Unused Emulation start Emulation continue Device vector Device vector Device vector Device vector Unused I I I I I I I I F F I I I I 0 0 0 0 0 0 0 10 0 0 0 0 0 IPL is 1A (hex) IPL is 1D (hex) Ordinarily used for AST delivery Ordinarily used for process scheduling IPL is 16 (hex) Same mode exception, FPD=0; parameters are opcode, PC, specifiers Same mode exception, FPD=1; no parameters IPL is 14 (hex) IPL is 15 (hex), includes console interrupts IPL is 16 (hex), includes interprocessor interrupts IPL is 17 (hex) 1  A = Abort; F = Fault; T = Trap; I = Interrupt. 2­22 CPU Chip 2.8 Process Structure A process is a single thread of execution. The context of the current proc­ ess is contained in the process control block (PCB), which can be located anywhere in memory space. The PCB is pointed to by the address con­ tained in the Process Control Block Base (PCBB) register, IPR16. The physical address of the current PCB is changed by writing to the PCBB register. The LDPCTX instruction loads a process context from the PCB as described in the VAX Architecture Reference Manual. LDPCTX flushes only the process space entries from the translation buffer; system space en­ tries are preserved. Other process structure functions are implemented as described in the VAX Architecture Reference Manual. NOTE: When the CPU is in 30­bit addressing mode, PCBB<31:30> are ignored. Figure 2­18 shows the process control block layout. CPU Chip 2­23 Figure 2­18 Process Control Block The processor status longword (PSL) is a 32­bit register that contains in­ formation about the state of the processor. Figure 2­19 shows the PSL. 2­24 CPU Chip Figure 2­19 Processor Status Longword Table 2­10 identifies the bits in the PSL. Table 2­10 Processor Status Longword Bits Mnemonic Bit(s) Name CM TP VM FPD IS CUR_MOD PRV_MOD IPL DV FU IV T N Z V C <31> <30> <29> <27> <26> <25:24> <23:22> <20:16> <7> <6> <5> <4> <3> <2> <1> <0> Compatibility mode Trace pending Virtual machine mode1 First part done Interrupt stack Current mode Previous mode Interrupt priority level Decimal overflow trap enable Floating underflow fault enable Integer overflow trap enable Trace trap enable Negative condition code Zero condition code Overflow condition code Carry condition code 1 MBZ unless virtual machine option is implemented. CPU Chip 2­25 2.9 Functional Partitions The macropipelined VAX design of the NVAX+ CPU chip is implemented through a group of tightly coupled logic units called boxes. The VAX func­ tions are partitioned among five boxes: · Ibox · Ebox and Microsequencer · Fbox · Mbox · Cbox The NVAX+ chip achieves high performance in large part through the con­ current, and relatively independent, operation of these logic boxes. Figure 2­20 shows the logic box level block diagram of the CPU module. Figure 2­20 NVAX+ Logic Boxes 2­26 CPU Chip 2.9.1 Ibox The Ibox (instruction box) decodes VAX instructions and parses operand specifiers. Instruction control, such as the control store dispatch address, is then placed in the instruction queue for later use by the microsequencer and Ebox (see Section 2.9.2). The Ibox processes the operand specifiers at a rate of one specifier per cycle and, as necessary, initiates specifier mem­ ory read operations. All the information needed to access the specifiers is placed in the source queue and destination queue in the Ebox. The Ibox prefetches instruction stream data from a dedicated instruction cache, called the virtual instruction cache (VIC), and places it into the 16­ byte prefetch queue (PFQ). The VIC is a 2­Kbyte, direct­mapped cache, with a block and fill size of 32 bytes. The Ibox has both read and write ports to the general purpose register (GPR) and memory data (MD) portions of the Ebox register file. These ports are used to process the operand specifiers. The Ibox maintains a scoreboard to ensure that reads and writes to the register file are always performed in synchronization with the Ebox. The Ibox stops processing in­ structions and operands upon issuing certain complex instructions like CALL, RET, or character string instructions, so that proper read/write ordering is maintained while the Ebox alters large amounts of VAX state data. Since the Ibox is often parsing several macroinstructions ahead of the Ebox, the correct value for the PSL condition codes is not known at the time the Ibox executes a conditional branch instruction. Rather than emptying the pipe, the Ibox predicts which direction the branch will take and passes this information on to the Ebox via the branch queue. The Ebox later signals if there was a misprediction, and the hardware backs out of the path. The branch prediction algorithm uses a 512­entry RAM, which caches four bits of branch history per entry. The instruction fetching logic prefetches several instructions ahead of the instruction parsing logic, identifying and pre­processing each instruction's components. The parsing logic decodes part of the instruction and passes the instruction opcodes and associated information directly into the Ebox instruction queue. Operand specifier information is passed on to the oper­ and specifier processing logic. Instruction prefetching also provides a buffer, four bytes wide by four ele­ ments deep, that isolates the instruction parser from the bursts of data coming in from cache and memory. The result is that the instruction fetch­ ing and instruction parsing can be done in parallel. The operand specifier processing logic locates the operands in registers, in memory, or in the instruction stream. This logic places operand informa­ tion in the Ebox source and destination queues and makes the required op­ erand memory requests. The Ibox's branch prediction unit (BPU) monitors each instruction opcode as it is parsed, looking for a branch opcode. Upon identification of a branch opcode, the BPU predicts whether the branch will be taken. If the prediction is that the branch will be taken, the BPU adds the sign­ extended branch displacement to the current PC and broadcasts the result­ ing new PC to the rest of the Ibox. The BPU is controlled by the microcode. Since branch direction relies on Ebox condition codes, the Ibox has no prior knowledge of branch direction. Branch prediction logic makes a prediction CPU Chip 2­27 on which way the branch will go and forces the Ibox to take that path. The PC pointing to the alternate branch path is saved should the prediction prove wrong. If the prediction is wrong, the Ibox is redirected to the cor­ rect path. 2.9.2 Ebox and Microsequencer The Ebox and microsequencer work together to perform the actual work of the VAX instruction execution. Together they implement a four­stage micropipelined unit that can both stall and microtrap. The Ebox and microsequencer dequeue instruction and operand information provided by the Ibox through the instruction queue, the source queue, and the destina­ tion queue. For literal type operands, the source queue contains the ac­ tual operand value. In the case of register, memory, and immediate type operands, the source queue holds a pointer to the data in the Ebox register file. The contents of memory operands are provided by the Mbox (Section 2.9.4) based on earlier requests from the Ibox. GPR results are written di­ rectly back to the register file. Memory results are sent to the Mbox, where the data is matched with the appropriate specifier address previ­ ously sent by the Ibox. At times, the Ebox initiates its own memory reads and writes. The Ebox contains a five­port register file, which holds the VAX GPRs, six memory data registers, six microcode working registers, and ten mis­ cellaneous CPU state registers. It also contains an ALU, a shifter, and the VAX processor status longword (PSL). The Ebox uses the retire queue to order the completion of Ebox and Fbox (Section 2.9.3) instructions. Since the Ebox and the Fbox are distinct hardware resources, some execu­ tion overlap is allowed between the two units. The Ebox implements two IPRs: the Patchable Control Store Control Reg­ ister (PCSCR), used to patch the NVAX+ microcode and select certain Ebox functions, and the Ebox Control Register (ECR), also used to select certain Ebox functions. The microsequencer determines the next microword to be fetched from the control store. It then provides cycle­by­cycle control of the Ebox. The control store is an on­chip ROM that contains the microcode used to execute microinstructions and microtraps. It is made up of 1600 micro­ words. These are arranged as 200 entries, each entry consisting of 8 microwords. Each microword is 61 bits long, with bits <14:0> being used to control the microsequencer. The remainder of the microword, bits <60:15>, is used by the Ebox to control S3 through S5 of the pipeline. The control store is addressed using the output of the current address latch (CAL). The patchable control store is an on­chip SRAM that contains microcode patches. It consists of up to 20 microwords. It operates in parallel with the control store. The microaddress from the CAL is the input to its con­ tent addressable memory (CAM). If the address hits in the CAM, the out­ put of the patchable control store is selected as the new microword, rather than the output of the regular control store. 2­28 CPU Chip 2.9.3 Fbox The Fbox is the floating­point unit in the NVAX+ CPU chip. It is a four­ stage pipelined floating­point processor, with an additional stage dedicated to assisting division. The Fbox interacts with three segments of the main CPU pipeline: the microsequencer in stage 2, and the Ebox in stages 3 and 4. The Fbox supports the following operations: · Floating­Point Instructions and Data Types The Fbox provides instruction and data support for VAX floating­point instructions. VAX F_, D_, and G_floating­point data types are sup­ ported. · VAX Integer Instructions The Fbox implements longword integer multiply instructions. · Pipelined Operation Except for all the divide instructions, DIV{F,D,G}, the Fbox can start a new single­precision floating­point instruction every cycle and a double­precision floating­point or an integer multiply instruction every two cycles. The Ebox can supply two 32­bit operands or one 64­bit oper­ and to the Fbox every cycle on two 32­bit input operand buses. The Fbox drives the result operand to the Ebox on a 32­bit result bus. · Conditional Mini­Round Operation Result latency is conditionally reduced by one cycle for the most fre­ quently used instructions. Stage 3 can perform a mini­round opera­ tion on the least significant bits of the fraction for all ADD, SUB, and MUL floating instructions. If the mini­round operation does not fail, then stage 3 drives the result directly to the output, bypassing stage 4 and saving a cycle of latency. · Fault and Exception Handling The Ebox coordinates the fault and exception handling with the Fbox. Any fault or exception condition received from the Ebox is retried in the proper order. If the Fbox receives or generates any fault or excep­ tion condition, it does not change the flow of instructions in progress within the Fbox pipe. 2.9.4 Mbox The Mbox performs three primary functions: · VAX Memory Management The Mbox, in conjunction with the operating system memory manage­ ment software, manages the allocation and use of physical memory. It performs translations of virtual addresses to physical addresses, access violation checks on all memory references, and initiates the invocation of software memory management code when necessary. The Mbox uses the translation buffer, which contains 96 fully associative entries, to map virtual to physical addresses. In the case of a TB miss, the memory management hardware in the Mbox reads the PTE from the cache or main memory and fills the TB. The Mbox performs all access checks, TNV checks, M­bit checks, and quadword unaligned data processing. · Reference Processing Due to the macropipeline structure of the NVAX+ chip, and the cou­ pling between NVAX+ and its memory subsystem, the Mbox can re­ CPU Chip 2­29 ceive memory references from the Ibox, Ebox, and Cbox simultane­ ously. The Mbox receives read requests from the Ibox (both in instruc­ tion stream and data stream) and from the Ebox (data stream only). It receives write/store requests from the Ebox. Also, the Cbox sends the Mbox fill data and invalidates for the primary cache (P­cache). The Mbox arbitrates between these requesters and queues requests that cannot currently be handled. Once a request is started, the Mbox per­ forms address translation and cache lookup in two cycles, assuming there are no misses or other delays. The two­cycle Mbox operation is pipelined. · Primary Cache Control The Mbox contains the primary cache (P­cache), which is an 8­Kbyte, 2­way set associative, write­through cache with a block and fill size of 32 bytes. The Mbox maintains the P­cache state as a subset of the backup cache (B­cache). The Mbox ensures that Ibox specifier reads are ordered correctly with Ebox specifier stores. To prevent the Ibox from using data that the Ebox should have first written, the Mbox scoreboards the memory. Scoreboarding is done using the physical address queue, a small list of physical addresses that have a pending Ebox store. The Mbox has four error registers. Two record TB parity errors, and two record P­cache parity errors. They are as follows: · TBADR (TB Parity Address Register) Holds the virtual address associated with a translation buffer parity error. · TBSTS (TB Status Register) Holds the error status associated with errors occurring in the Mbox. · PCADR (P­Cache Parity Address Register) Holds the physical address associated with a P­cache parity error. · PCSTS (P­Cache Status Register) Holds the error status of errors that occur in the P­cache. When the operating system error handler routine is invoked from a microtrap or interrupt, the handler can read the state of all the error regis­ ters through IPR read operations to determine what errors were present when the error handler was invoked. IPR read operations are performed with MFPR instructions. 2.9.5 Cbox The Cbox provides the interface to the EDAL bus, which is the communica­ tion channel between the NVAX+ and the other sections of the KA7AA module (B­cache and the LEVI interface). The Cbox is tightly coupled to the Mbox. The Mbox sends read requests and writes to the Cbox; the Cbox sends fills and invalidates to the Mbox. The Cbox ensures that the P­cache is a subset of the B­cache through in­ validates. The Cbox communicates with the LEVI interface, and thus with the LSB bus, through the EDAL bus. The Cbox generates reads and receives fills; it receives cache coherence transactions from the EDAL, to which it re­ sponds with invalidates and writebacks, as appropriate. 2­30 CPU Chip The Cbox implements a programmable interval clock that provides an in­ terrupt at IPL 16 (hex) at programmed intervals. The counter is incre­ mented at 1 microsecond intervals, with at least .01% accuracy. The inter­ val clock consists of three internal processor registers (IPRs) that reside in the privileged register space: · Interval Count Control and Status Register (ICCS), IPR24 · Next Interval Count Register (NICR), IPR25 · Interval Count Register (ICR), IPR26 To program the interval clock, the negative (2's complement) of the de­ sired interval is loaded into the NICR register. Then, writing 51 (hex) to the ICCS enables interrupts, loads the next interval into the ICR, and sets ICCS. Setting this bit causes an interrupt to occur every interval count microseconds. The interrupt routine should write C1 (hex) to the ICCS to clear the interrupt. If the Interrupt bit has not been cleared (the interrupt has not been handled) by the time of the next ICR overflow, ICCS is set. If the NICR is written while the clock is running, the clock may lose or add a few ticks. If the interval clock interrupt is enabled, this may cause the loss of an interrupt. 2.10 General Purpose Registers General purpose registers are provided to be used by the operating system in carrying out its normal functions. The NVAX+ processor contains six­ teen 32­bit general purpose registers. Some of these registers are assigned to specific functions. The others can be used by the operating system for any required purpose. Table 2­11 shows the usage of the GPRs. Table 2­11 General Purpose Register Usage GPR Acronym Use R0R11 R12 R13 R14 R15 AP FP SP PC General purpose Argument pointer Frame pointer Stack pointer Program counter CPU Chip 2­31 2.11 Internal Processor Registers Internal processor registers implement functions assigned by hardware. They are used to store data, control, and status information. The internal processor registers are logically divided into three groups: · Normal Address individual registers in the NVAX+ chip or system environ­ ment. · P­Cache Tag The read/write block of IPRs that allow direct access to the P­cache tags. · P­Cache Data Parity The read/write block of IPRs that allow direct access to the P­cache data parity bits. Each group of IPRs is distinguished by a particular pattern of bits in the IPR address, as shown in Figure 2­21. Figure 2­21 IPR Address Formats Table 2­12 shows the numeric range of each group of IPRs. 2­32 CPU Chip Table 2­12 IPR Address Space Decoding Because of the sparse addressing space used for IPRs in groups other than the normal group, valid IPR addresses in those groups are separated not by 1, as are those of the normal group, but by either 8 or 20 (hex). For example, the IPR address for the first subblock of P­cache data parity is 01C0 0000 (hex), and the IPR address for the second subblock of P­cache data parity is 01C0 0008 (hex). Table 2­13 lists the normal IPRs implemented by the NVAX+ chip. Software­accessible registers are described individually in this section. CAUTION: Many of the IPRs listed in Table 2­13 are used internally by the microcode during normal operation of the CPU and are not intended to be referenced by software except during test or diagnosis of the system. These registers are flagged with the notation Testability and diagnostic use only; not for software use in normal operation. References by software to these registers during normal operation can cause undefined behavior of the CPU. IPR Group Mnemonic1 Address Range (hex) Contents Normal P­cache tag P­cache data parity PCTAG PCDAT 0000 0000 to 0000 00FF2 0180 0000 to 0180 1FE02 01C0 0000 to 01C0 1FF82 256 individual IPRs. 256 P­cache tag IPRs, 128 per P­ cache set. Each IPR is separated from the previous one by 20 (hex). 1024 P­cache data parity IPRs, 512 per P­cache set. Each IPR is sepa­ rated from the previous one by 8 (hex). 1 The mnemonic is for the first IPR in the group. 2 Unused fields for the IPR addresses for these groups should be zero. Neither hardware nor microcode detects and faults on an address in which these bits are nonzero. Although noncontiguous address ranges are shown for these groups, the entire IPR address space maps into one of these groups. If these fields are nonzero, the operation of the CPU is undefined. CPU Chip 2­33 Table 2­13 KA7AA Internal Processor Registers Name Mnemonic Type Address Dec (Hex) Kernel Stack Pointer Executive Stack Pointer Supervisor Stack Pointer User Stack Pointer Interrupt Stack Pointer P0 Base Register P0 Length Register P1 Base Register P1 Length Register System Base Register System Length Register CPU Identification Register1 Process Control Block Base Register System Control Block Base Register Interrupt Priority Level Register1 AST Level Register Software Interrupt Request Register Software Interrupt Summary Register1 Interval Count Control/Status Register1 Next Interval Count Register Interval Count Register Time­of­Day Register Machine Check Error Register Console Saved PC Register Console Saved PSL Register Memory Management Enable Register1 Translation Buffer Invalidate All Register Translation Buffer Invalidate Single Register Performance Monitor Enable Register1 System Identification Register Translation Buffer Check Register Interrupt System Status Register2 Mailbox Register Performance Monitoring Facility Count Patchable Control Store Control Register2 Ebox Control Register Mbox TB Tag Fill Register2 Mbox TB PTE Fill Register2 BIU Control Register KSP ESP SSP USP ISP P0BR P0LR P1BR P1LR SBR SLR CPUID PCBB SCBB IPL ASTLVL SIRR SISR ICCS NICR ICR TODR MCESR SAVPC SAVPSL MAPEN TBIA TBIS PME SID TBCHK INTSYS LMBOX PMFCNT PCSCR ECR MTBTAG MTBPTE BIU_CTL R/W R/W R/W R/W R/W R/W R/W R/W R/W R/W R/W R/W R/W R/W R/W R/W WO R/W R/W WO RO R/W WO RO RO R/W WO WO R/W RO WO R/W R/W R/W WO R/W WO WO R/W 0 (0) 1 (1) 2 (2) 3 (3) 4 (4) 8 (8) 9 (9) 10 (A) 11 (B) 12 (C) 13 (D) 14 (E) 16 (10) 17 (11) 18 (12) 19 (13) 20 (14) 21 (15) 24 (18) 25 (19) 26 (1A) 27 (1B) 38 (26) 42 (2A) 43 (2B) 56 (38) 57 (39) 58 (3A) 61 (3D) 62 (3E) 63 (3F) 121 (79) 122 (7A) 123 (7B) 124 (7C) 125 (7D) 126 (7E) 127 (7F) 160 (A0) 1 Initialized on reset. 2 Testability and diagnostic use only; not for software use in normal operation. 2­34 CPU Chip Table 2­18 KA7AA Internal Processor Registers (Continued) Name Mnemonic Type Address Dec (Hex) Diagnostic Control Register B­Cache Error Tag Register BIU Status Register BIU Address Register Fill Syndrome Register Fill Address Register STxC Pass Fail/CEFSTS Register Software ECC Register Console Halt Register Serial I/O Register SROM_OE_Serial I.E. Register Pack to QW Register Clear I/O Pack Register VIC Memory Address Register VIC Tag Register VIC Data Register Ibox Control and Status Register Ibox Branch Prediction Control Register Ibox Backup PC Register Ibox Backup PC with RLOG Unwind Register3 Mbox P0 Base Register2 Mbox P0 Length Register2 Mbox P1 Base Register2 Mbox P1 Length Register2 Mbox System Base Register2 Mbox System Length Register2 Mbox Map Enable Register2 Physical Address Mode Register MME Address Register MME PTE Address Register MME Status Register TB Parity Address Register TB Parity Status Register P­Cache Parity Address Register P­Cache Parity Status Register P­Cache Control Register P­Cache Tag Registers P­Cache Data Parity Registers DIAG_CTL BC_TAG BIU_STAT BIU_ADDR FILL_SYND FILL_ADDR IPR_STR_COND BEDECC CHALT SIO SOE­IE QW_PACK CLR_IO_PACK VMAR VTAG VDATA ICSR BPCR BPC BPCUNW MP0BR MP0LR MP1BR MP1LR MSBR MSLR MMAPEN PAMODE MMEADR MMEPTE MMESTS TBADR TBSTS PCADR PCSTS PCCTL PCTAG PCDAP WO RO W1C RO RO RO RO WO R/W R/W WO WO WO R/W R/W R/W R/W R/W RO RO R/W R/W R/W R/W R/W R/W R/W R/W RO RO RO RO R/W RO R/W R/W R/W R/W 161 (A1) 162 (A2) 164 (A4) 166 (A6) 168 (A8) 170 (AA) 172 (AC) 174 (AE) 176 (B0) 178 (B2) 180 (B4) 184 (B8) 185 (B9) 208 (D0) 209 (D1) 210 (D2) 211 (D3) 212 (D4) 214 (D6) 215 (D7) 224 (E0) 225 (E1) 226 (E2) 227 (E3) 228 (E4) 229 (E5) 230 (E6) 231 (E7) 232 (E8) 233 (E9) 234 (EA) 236 (EC) 237 (ED) 242 (F2) 244 (F4) 248 (F8) (0180 0000 to 0180 1FE0) 01C0 0000 to 01C0 1FF8) 2 Testability and diagnostic use only; not for software use in normal operation. 3 Chip test use only; not for software use. CPU Chip 2­35 User­visible normal internal processor registers discussed in the following sections include: · Identification registers · Ibox registers · Ebox registers · Mbox registers · Cbox registers 2.11.1 Identification Registers Table 2­14 lists the two registers used to identify the operating CPU and the system. Functional descriptions of individual identification registers follow. Table 2­14 Identification Registers Name Mnemonic Type IPR Address (Hex) CPU Identification Register System Identification Register CUPID SID RO RO 0E 3E 2­36 CPU Chip CPUID·CPU Identification Register Table 2­15 CPUID Register Bit Definitions Address Access 000E RO The CPUID register allows software to determine the currently executing CPU in a multiprocessor system. Name Bit(s) Type Function CPU_ID <7:0> RO CPU Identification. Indicates the currently executing CPU in a multiprocessor system. The KA7AA module de­ fines bits <5:0> only, corresponding to CPU5 to CPU0. This field is loaded by the console firmware at power­up and is readable by software at any time. CPU Chip 2­37 SID·System Identification Register Table 2­16 SID Register Bit Definitions Address Access 003E RO The SID register contains information that identifies the processor type and the microcode revision level. Name Bit(s) Type Function CPU_TYPE MIC_PATCHES MIC_REV <31:24> <15:8> <7:0> RO RO RO CPU Type. Indicates the type of CPU used in the system. The value of this field for the NVAX+ CPU chip is 17 (hex). Microcode Patches. Indicates the microcode patches if any, included in the CPU chip. Normally, this field has a value of zero. Microcode Revision. Indicates the microcode revi­ sion of the NVAX+ CPU chip. The value of this field is 2 for the first revision of the CPU module or higher for later revisions. 2­38 CPU Chip 2.11.2 Ibox Registers Table 2­17 lists the Ibox registers. Functional descriptions of individual Ibox registers follow. Table 2­17 Ibox Registers Name Mnemonic Type IPR Address (Hex) VIC Memory Address Register VIC Tag Register VIC Data Register Ibox Control and Status Register Branch Prediction Control Register VMAR VTAG VDATA ICSR BPCR R/W R/W R/W R/W R/W D0 D1 D2 D3 D4 CPU Chip 2­39 VMAR·VIC Memory Address Register Table 2­18 VMAR Register Bit Definitions Address Access 00D0 R/W The VMAR register provides access to the virtual instruction cache (VIC) arrays. When the VIC is disabled, the VMAR register can be used as an index for direct IPR access to the cache arrays. This register also latches and holds the virtual address on VIC array parity errors. Name Bit(s) Type Function ADDR ROW_INDEX SUBBLOCK LW <31:11> <10:5> <4:3> <2> RO R/W R/W WO Error Address. Latches tag portion of the virtual ad­ dress on VIC parity errors. Row Index. Contains row index for read and write access to cache array; also latches the virtual address. Subblock Select. Selects data subblock for access to cache array. Also latches bits <4:3> of the virtual ad­ dress on VIC parity errors. Longword Select. Selects longword of subblock for access to cache array. 2­40 CPU Chip VTAG·VIC Tag Register Table 2­19 VTAG Register Bit Definitions Address Access 00D1 R/W The VTAG register provides diagnostic access to the cache tag ar­ ray. Name Bit(s) Type Function TAG TAG_P DATA_P DATA_V <31:11> <8> <7:4> <3:0> R/W R/W R/W R/W Tag. Supplies tag on tag array read/writes. Tag Parity. Supplies tag parity on tag array read/writes. Data Parity. Supplies four parity bits for four quadwords of data on array read/writes. Each data parity bit has a corresponding data valid bit (VTAG<3:0>). VTAG<4> and VTAG<0> are associated with the quadword of data addressed when VMAR<4:3> = 00; VTAG<5> and VTAG<1> corre­ spond to the quadword of data addressed when VMAR<4:3> = 01, and so on. Data Valid. Supplies four valid bits for four quadwords of data on array read/writes. Each data valid bit has a corresponding data parity bit (VTAG<7:4>). See description of VTAG<7:4> for quadwords associated with each data valid bit. CPU Chip 2­41 VDATA·VIC Data Register Table 2­20 VDATA Register Bit Definitions VDATA Read/Write When the VDATA register is written, the cache data array entry indexed by the VMAR register is written with the IPR data. Since the IPR data is a longword, two accesses to the VDATA register are required to read or write a quadword cache subblock. Writes to the VDATA register with VMAR<2> = 0 accumulate the IPR data destined for the low longword of a subblock in FILL_DATA<31:0>. A subsequent write to the VDATA register with VMAR<2> = 1 directs the IPR data to FILL_DATA<63:32> and triggers a cache write sequence to the subblock indexed by the VMAR register. Reads to the VDATA register with VMAR<2> = 0 trigger a cache read se­ quence to the subblock indexed by the VMAR register. The low longword of a subblock is returned as IPR read data. A read of the VDATA register with VMAR<2> = 1 returns the high longword of the subblock as IPR data. Address Access 00D2 R/W The VDATA register provides diagnostic access to the cache data array. Name Bit(s) Type Function VIC_DATA <31:0> R/W VIC Data. Data for array reads and writes. 2­42 CPU Chip ICSR·Ibox Control and Status Register Table 2­21 ICSR Register Bit Definitions Address Access 00D3 R/W The ICSR register provides control and status functions for the Ibox. Name Bit(s) Type Function TPERR DPERR LOCK VIC_ACC_ENB <4> <3> <2> <0> RO RO WC R/W,0 Tag Parity Error. Sets when a VIC tag parity error occurs in the tag array. Data Parity Error. Sets when a VIC data parity er­ ror occurs in the data array. Lock. Sets when either ICSR or ICSR is set. The set state of this bit vali­ dates the status of those two error bits and prevents further modification of their status and the error ad­ dress in the VMAR register. When clear, indicates that no VIC parity error has been recorded and allows up­ dates of the ICSR register and the VMAR register. VIC Access Enable. When set, allows access to the VIC. CPU Chip 2­43 BPCR·Branch Prediction Control Register Table 2­22 BPCR Register Bit Definitions Address Access 00D4 R/W The BPCR register provides control for the branch prediction unit and read/write access to the history array. Name Bit(s) Type Function BPU_ALGORITHM LOAD_HISTORY FLUSH_CTR FLUSH_BHT MISPREDICT HISTORY <31:16> <8> <7> <6> <5> <3:0> R/W WO WO WO WO R/W Branch Prediction Unit Algorithm. Con­ trols direction of branch for given history. Load History. Write of a one loads the history array into the location addressed by the BPCR address counter. Flush Center. Write of a one resets the BPCR address counter to zero. Cleared by hardware. Flush Branch History. Write of a one resets all history table entries to a neutral value. Cleared by hardware. Mispredict. Sets if last conditional branch mispredicted. History. Branch history table entry bits. 2­44 CPU Chip BPCR Read/Write The write­only BPCR causes a BPU branch history table flush. The flush is identical to the context switch flush, which resets all branch table entries to a neutral value: HISTORY = 0100. The write­only BPCR causes the Branch Table Counter bits <8:0> in the Ibox to be cleared. The Branch Table Counter provides an address into the branch table for IPR read and write accesses. Each IPR read from the BPCR or write to the BPCR with BPCR = 1 incre­ ments the counter. This allows IPR branch table reads and writes to step through the branch table array. LOAD_HISTORY enables writes to the branch history table. A write to the HISTORY field with BPCR<8> = 1 causes a BPU branch history table write. The history bits for the entry in­ dexed by the counter are written with the IPR data. The BPCR register reads supply the history bits in BPCR<3:0> for the entry indexed by the counter. BPCR returns a 1 if the last conditional branch mispredicted. BPCR<31:16> contains the branch prediction algorithm. Any IPR write to the BPCR register updates the algorithm. An IPR read returns the value of the current algorithm. For example, a 0 in BPCR<16> means that the next branch encountered will not be taken if the history is 0000. A 1 in BPCR<21> means that the next branch encountered when the prior history is 0101 will be taken. BPCR bits <8>, <7>, and <6> are defined in Table 2­23 for IPR writes to the BPCR register. NOTE: The prediction algorithm is updated on every IPR write to the BPCR. Table 2­23 BPCR Write Actions BPCR <8:6> Write Action 000 001 010 011 100 101 110 111 Do nothing except update algorithm. Flush branch table; history not written. Address counter reset to zero; history not written. Flush branch table; reset address counter; history not written. Write history to table; counter automatically increments. UNDEFINED. Branch table flushed; new history written; counter incremented. UNDEFINED. Write history to old counter value; counter reset to zero. UNDEFINED. Branch table flushed; write history to old counter value; counter reset to zero. CPU Chip 2­45 2.11.3 Ebox Registers Table 2­24 lists the Ebox registers. Functional descriptions of individual Ebox registers follow. Table 2­24 Ebox Registers Name Mnemonic Type IPR Address (Hex) Patchable Control Store Control Register Ebox Control Register PCSCR ECR WO R/W 7C 7D 2­46 CPU Chip PCSCR·Patchable Control Store Control Register Table 2­25 PCSCR Register Bit Definitions Address Access 007C WO The PCSCR register is used to load control store patches. This reg­ ister is not used in normal operation. Name Bit(s) Type Function PATCH_REV NONSTAN­ DARD_PATCH PCS_DATA SHIFT <28:24> <23> <12> <11> R/W WO WO, 0 Patch Revision. Updated by software after load­ ing a microcode patch. Indicates the revision of the standard microcode patch that has been loaded. Nonstandard Patch. When set, indicates a non­ standard microcode patch has been loaded. Patchable Control Store Data. Loaded with the data bit to be shifted into PCS read/write chain for writing to PCS RAM or CAM. The shift occurs whenever PCSCR<11> is written with one. By re­ peatedly loading this bit with one or zero, and set­ ting PCSCR<11> after each load, any data pattern can be written into the PCS read/write chain. Shift. A write of one loads the PCS_DATA bit into the PCS R/W chain and shifts the PCS read/write chain by one. CPU Chip 2­47 Table 2­25 PCSCR Register Bit Definitions (Continued) Name Bit(s) Type Function PCS_WRITE PCS_ENB PAR_PORT_DIS <10> <9> <8> WO, 0 WO, 0 WO, 0 Patchable Control Store Write. A write of one loads the contents of the PCS read/write chain to the PCS RAM and CAM. Patchable Control Store Enable. A write of one enables PCS outputs, so that patches supersede the control store ROM. Parallel Port Disable. A write of one disables con­ trol of load/shift operations to the PCS read/write chain. 2­48 CPU Chip ECR·Ebox Control Register Table 2­26 Ebox Control Register Bit Definitions Address Access 007D R/W The ECR register is used to configure certain Ebox functions. Name Bit(s) Type Function PMF_CLEAR PMF_LFSR PMF_EMUX <31> <22> <21:19> WO, 0 R/W, 0 R/W, 0 Performance Monitor Facility Clear. Writ­ ing one to this position clears the performance monitor facility counters. This function is han­ dled by microcode. PMF Linear Feedback Shift Register. When set, enables the LFSR accumulator. This is a testability feature. PMF EMUX. Selects the Ebox events counted by the performance monitor facility, when the PMF is configured to count Ebox events. CPU Chip 2­49 Table 2­26 Ebox Control Register Bit Definitions (Continued) Name Bit(s) Type Function PMF_PMUX PMF_ENB FBOX_TEST_ENB TO_CLOCK TO_TEST TO_OCCURRED FBOX_ST4_ BYPASS_ENB MBZ FBOX_ENB <18:17> <16> <13> <6> <5> <4> <3> <2> <1> R/W, 0 R/W, 0 R/W, 0 R/W, 0 R/W, 0 W1C, 0 R/W, 0 R/W, 0 R/W, 0 PMF PMUX. Selects the source of events counted by the performance monitor facility to be the Ibox, Ebox, Mbox, or the Cbox. PMF Enable. This bit is the internal implemen­ tation of the PME processor register. Fbox Testability Enable. When set, places the Fbox in test mode. in which data is passed from stage to stage unaltered. Timeout Clock. This is the most significant bit of the timeout base counter. When timeout is functioning, this bit should be one half of the time and zero the other half of the time. Timeout Test. When set, the S3 stall timeout time is roughly 50 microseconds instead of roughly 3 seconds. Timeout Occurred. Set when a S3 stall timeout occurs. Cleared by a write of one. Fbox Stage 4 Bypass Enable. Set by configura­ tion code to enable Fbox stage 4 bypass. Must Be Zero. Must always be written as zero. Writing one disables S3 timeouts. Fbox Enable. Set by configuration code to en­ able the Fbox. 2­50 CPU Chip 2.11.4 Mbox Registers Table 2­27 lists the Mbox registers. Functional descriptions of individual Mbox registers follow. Table 2­27 Mbox Registers Name Mnemonic Type IPR Address (Hex) Mbox P0 Base Register 1 Mbox P0 Length Register1 Mbox P1 Base Register1 Mbox P1 Length Register1 Mbox System Base Register1 Mbox System Length Register1 Mbox Map Enable Register1 Address Mode Register MME2 Address Register1 MME PTE Address Register1 MME Status Register1 TB Parity Address Register TB Parity Status Register P­Cache Parity Address Register P­Cache Parity Status Register P­Cache Control Register P­Cache Tag Registers P­Cache Data Parity Registers MP0BR MP0LR MP1BR MP1LR MSBR MSLR MMAPEN PAMODE MMEADR MMEPTE MMESTS TBADR TBSTS PCADR PCSTS PCCTL PCTAG PCDAP R/W R/W R/W R/W R/W R/W R/W R/W RO RO RO RO R/W RO R/W R/W R/W R/W E0 E1 E2 E3 E4 E5 E6 E7 E8 E9 EA EC ED F2 F4 F8 0180 0000 to 0180 1FE0 01C0 0000 to 01C0 1FF8 1 Testability and diagnostic use only; not for software use in normal operation. 2 Memory management exception. CPU Chip 2­51 MP0BR·Mbox P0 Base Register Table 2­28 MP0BR Register Bit Definitions Address Access 00E0 R/W The MP0BR register contains the base address of the P0 region of the process space. Name Bit(s) Type Function SYS_VA_P0 <29:9> R/W System Virtual Address of P0. Contains the system virtual address of the P0 page table. 2­52 CPU Chip MP0LR·Mbox P0 Length Register Table 2­29 MP0LR Register Bit Definitions Address Access 00E1 R/W The MP0LR register contains the size of the P0 region of the proc­ ess space. Name Bit(s) Type Function P0_LENGTH_LW <21:0> R/W P0 Length Longwords. Contains the length of the P0 page table in longwords. CPU Chip 2­53 MP1BR·Mbox P1 Base Register Table 2­30 MP1BR Register Bit Definitions Address Access 00E2 R/W The MP1BR register contains the base address of the P1 region of the process space. Name Bit(s) Type Function SYS_VA_P1 <29:9> R/W System Virtual Address of P1. Contains the system virtual address of the P1 page table. 2­54 CPU Chip MP1LR·Mbox P1 Length Register Table 2­31 MP1LR Register Bit Definitions Address Access 00E3 R/W The MP1LR register contains the size of the P1 region of the proc­ ess space. Name Bit(s) Type Function P1_LENGTH_LW <21:0> R/W P1 Length Longwords. Contains the length of the P1 page table in longwords. CPU Chip 2­55 MSBR·Mbox System Base Register Table 2­32 MSBR Register Bit Definitions Address Access 00E4 R/W The MSBR register contains the physical address of the system page table. Name Bit(s) Type Function SYS_PT_PA <31:9> R/W System Page Table Physical Address. Con­ tains the physical address of the system page ta­ ble in longwords. 2­56 CPU Chip MSLR·Mbox System Length Register Table 2­33 MSLR Register Bit Definitions Address Access 00E5 R/W The MSLR register contains the length of the system page table in longwords. Name Bit(s) Type Function SYS_PT_LENGTH_LW <21:0> R/W System Length Longwords. Contains the length of the system page table in longwords. CPU Chip 2­57 MMAPEN·Mbox Map Enable Register Table 2­34 MMAPEN Register Bit Definition Address Access 00E6 R/W The MMAPEN register contains the bit that enables/disables mem­ ory management. Name Bit Type Function MEM_MNG_ENB <0> R/W Memory Management Enable. When set, en­ ables Mbox memory management. When clear, dis­ ables memory management. 2­58 CPU Chip PAMODE·Physical Address Mode Register Table 2­35 PAMODE Register Bit Definition Address Access 00E7 R/W The PAMODE register selects the size of the physical address space to be used in mapping virtual addresses to physical ad­ dresses. Name Bit Type Function PHYS_ADDR_MODE <0> R/W Physical Address Mode. When set, NVAX+ maps addresses from a 32­bit physical address space. When clear, NVAX+ maps addresses from a 30­bit physical address space. CPU Chip 2­59 MMEADR·MME Address Register Table 2­36 MMEADR Register Bit Definitions Address Access 00E8 RO The MMEADR register contains the address associated with the MME fault. Name Bit(s) Type Function MME_FAULT_ADDR <31:0> RO MME Fault Address. Contains the address as­ sociated with the MME fault. 2­60 CPU Chip MMEPTE·MME PTE Address Register Table 2­37 MMEPTE Address Register Bit Definitions Address Access 00E9 RO The MMEPTE register contains the PTE address associated with an address corresponding to a modify fault. Name Bit(s) Type Function MOD_FAULT_PTE_ ADDR <31:0> RO Modify Fault PTE Address. Contains the PTE address associated with an ad­ dress corresponding to a modify fault. CPU Chip 2­61 MMESTS·MME Status Register Table 2­38 MMESTS Register Bit Definitions Address Access 00EA R/W The MMESTS register reports faults or violations in memory man­ agement operations. Name Bit(s) Type Function LOCK SRC <31:29> <28:26> RO RO Lock. Indicates the lock status of the MMESTS register. This field is shadowed by MMESTS bits. Source. Complemented shadow copy of LOCK bits. However, the SRC bits do not get cleared when the LOCK bits are cleared. MMESTS <31:29> Definition 000 001 011 111 MMESTS, MMEADR, MMEPTE are unlocked. Valid IREAD error is stored. Valid Ibox specifier error is stored. Valid Ebox reference error is stored. 2­62 CPU Chip Table 2­38 MMESTS Register Bit Definitions (Continued) Name Bit(s) Type Function FAULT M PTE_REF LV <15:14> <2> <1> <0> RO RO RO RO, 0 Fault. Indicates type of memory management fault. Modify. Indicates corresponding reference had write or modify intent. Page Table Entry Reference. Indicates ACV/TNV fault occurrence on PTE reference corre­ sponding to the MMEADR register. Length Violation. Indicates ACV fault occur­ rence due to length violation. MMESTS <15:14> Fault 01 10 11 ACV fault. Highest priority fault in the presence of multiple simultane­ ous faults. TNV fault. Next priority fault to ACV. M=0 fault. Lowest priority fault. CPU Chip 2­63 TBADR·Translation Buffer Parity Address Register Table 2­39 TBADR Register Bit Definitions Address Access 00EC R/W The TBADR register contains the address of the reference that caused the TB parity error. Name Bit(s) Type Function VA_TB_PE <31:0> RO Virtual Address TB Parity Error. Con­ tains the virtual address of the reference that caused the translation buffer parity er­ ror. 2­64 CPU Chip TBSTS·Translation Buffer Parity Status Register Table 2­40 TBSTS Register Bit Definitions Address Access 00ED R/W The TBSTS register reports the status of the TB parity error. Name Bit(s) Type Function SRC CMD <31:29> <8:4> RO RO Source. Indicates the original source of the reference causing the TB parity error. Command. S5 command corresponding to TB parity error. TBSTS <31:29> Definition 111 110 100 000 Valid Mbox reference error is stored. Valid IREAD error is stored. Valid Ibox specifier error is stored. Valid Ebox reference error is stored. CPU Chip 2­65 Table 2­40 TBSTS Register Bit Definitions (Continued) Name Bit(s) Type Function EM_VAL TPERR DPERR LOCK <3> <2> <1> <0> RO RO RO W1C,0 EM Latch Valid. Indicates if EM_LATCH was valid at the time of the TB parity error detection. This helps the software error handler determine if a write operation may have been lost due to the TB parity er­ ror. Tag Parity Error. Set when a TB tag parity error occurs. Data Parity Error. Set when a TB data parity error occurs. Lock. When set, validates TBSTS contents and pre­ vents any other field from further modification. When clear, indicates that no TB parity error or PTE error has been recorded, and allows updates of the TBSTS and TBADR registers. 2­66 CPU Chip PCADR·P­Cache Parity Error Address Register Table 2­41 PCADR Register Bit Definitions Address Access 00F2 RO The PCADR register contains the quadword physical address asso­ ciated with the recorded P­cache parity error. Name Bit(s) Type Function PC_PE_PA <31:0> RO P­Cache Parity Error Physical Address. Con­ tains the quadword physical address associated with the recorded P­cache parity error. CPU Chip 2­67 PCSTS·P­Cache Parity Status Register Table 2­42 PCSTS Register Bit Definitions Address Access 00F4 R/W The PCSTS register reports information about the P­cache parity error. Name Bit(s) Type Function PTE_ER PTE_ER_WR CMD LEFT_BANK RIGHT_BANK DPERR LOCK <10> <9> <8:4> <3> <2> <1> <0> W1C, 0 W1C, 0 RO RO RO RO W1C, 0 PTE Error. Set when a hard error occurs on a PTE DREAD. PTE Error Write. Set when a hard error occurs on a PTE DREAD which resulted from a TB miss on a Write or Write_Unlock. Command. S6 command corresponding to P­cache parity error. Left Bank Tag Error. Set when a P­cache tag parity error occurs on the left bank. Right Bank Tag Error. Set when a P­cache tag parity error occurs on the right bank. Data Parity Error. Set when a P­cache data parity error occurs. Lock. When set, validates PCSTS<8:1> contents and prevents modification of these fields. When clear, invalidates PCSTS<8:1> and allows these fields and PCADR to be updated. 2­68 CPU Chip PCCTL·P­Cache Control Register Table 2­43 PCCTL Register Bit Definitions Address Access 00F8 R/W The PCCTL register controls the state of P­cache operations. Name Bit(s) Type Function RED_ENB ELEC_DISABLE PMM <9> <8> <7:5> RO R/W, 0 R/W,0 Redundancy Enable. When set, indicates that one or more P­cache redundancy elements are en­ abled. Electrically Disable. When set, the P­cache is disabled electrically to reduce power dissipation. This bit should only be set when the P­cache is functionally turned off by clearing of both I_STR_ENB and D_STR_ENB. UNPREDICT­ ABLE operation results if this bit is set when I_STR_ENB or D_STR_ENB is also set. Note that P­cache tag parity IPRs do not function properly when ELEC_DISABLE is unconditionally set. Performance Monitor Mode. Specifies Mbox performance monitor mode. This field does not con­ trol or affect the operation of the P­cache in any way. CPU Chip 2­69 Table 2­43 PCCTL Register Bit Definitions (Continued) Name Bit(s) Type Function PC_PE_ENB BANK_SEL FORCE_HIT I_STR_ENB D_STR_ENB <4> <3> <2> <1> <0> R/W,0 R/W,0 R/W,0 R/W,0 R/W,0 P­Cache Parity Error Enable. When set, enables detection of P­cache tag and data parity errors. When clear, disables P­cache parity error detection. Bank Select. When set with PCCTL<2> = 1, selects the right bank (Set 1) of the addressed P­cache in­ dex. When clear with PCCTL<2> = 1, selects the left bank (Set 0) of the addressed P­cache index. This bit is a don't care when PCCTL<2> = 0. Note that BANK_SEL does not affect bank selection during IPR reads and IPR writes to the P­cache tags or P­cache data parity bits. Bank selection for these operations is determined by the specified IPR address. Force P­Cache Hit. If set, forces a P­cache hit on all reads and writes when P­cache is enabled for I­ or D­ stream operation. I­Stream Enable. When set, enables P­cache proc­ essing of INVAL, IREAD, and I_CF commands. When clear, forces a P­cache miss on IREAD operations and prevents state modification due to an I_CF operation. D­Stream Enable. When set, enables P­cache for all INVAL operations and for all D­stream read/write/fill operations, qualified by other control bits. When clear, forces a P­cache miss on all P­cache D­stream read/write/fill operations. Note, however, that an ACV/TNV/M = 0 condition overrides a deasserted D_STR_ENB by forcing a P­cache hit condition with D_STR_ENB = 0. 2­70 CPU Chip PCTAG·P­Cache Tag Registers Table 2­44 PCTAG Register Bit Definitions Address Access 0180 0000 to 0180 1FE0 R/W The PCTAG registers provide diagnostic access to the tag and the associated state bits of the P­cache. Name Bit(s) Type Function TAG PARITY VALID ALLOC <31:12> <5> <4:1> <0> R/W R/W R/W R/W Tag. Tag data. Parity. Even tag parity. Valid. Valid bits corresponding to the four data subblocks. PCTAG<4> corresponds to uppermost quadword in the block; PCTAG<1> corresponds to lower­ most quadword in the block. Allocation. Allocation bit corresponding to index of this block. CPU Chip 2­71 PCDAP·P­Cache Data Parity Registers Table 2­45 PCDAP Register Bit Definitions Address Access 01C0 0000 to 01C0 1FF8 R/W The PCDAP registers provide diagnostic access to the even byte parity corresponding to the addressed quadword of data. Name Bit(s) Type Function DATA_PARITY <7:0> R/W Data Parity. Even byte parity corresponding to ad­ dressed quadword of data. 2­72 CPU Chip 2.11.5 Cbox Registers The internal processor registers (IPRs) in the system as well as in the Cbox are accessed through IPR read commands and IPR write commands from the Mbox to the Cbox. All IPR reads and writes to the Cbox flush the write queue before they are completed. Any IPR read sets conflict bits in all valid entries in the write queue so that all preceding writes of any kind complete before the IPR read. An IPR write is placed in the write queue after the preceding writes so that the ordering takes place naturally. If a read arrives before an IPR write has been processed, the write queue conflict bits are set so that the write queue takes priority over the read. If the IPR read addresses one of the Cbox registers, the Cbox returns just one quadword or less of data, rather than the usual four quadwords. If a write­only Cbox register is read, the Cbox returns UNPREDICTABLE data. Reading an unimplemented Cbox register also returns UNPRE­ DICTABLE data. If a write is performed to an unimplemented register, the write is discarded by the Cbox and normal operation continues. If the Cbox receives an IPR write to an address that is not within its block of IPR addresses, the Cbox discards the write and normal operation contin­ ues. If the Cbox receives an IPR read to an address that is not within its block of IPR addresses, the Cbox returns UNPREDICTABLE data. Table 2­46 lists the Cbox registers. Functional descriptions of individual Cbox registers follow. Table 2­46 Cbox Registers Register Mnemonic Type IPR Address (Hex) Interval Count Control/Status Register Next Interval Count Register Interval Count Register Time­of­Day Register BIU Control Register Diagnostic Control Register B­Cache Error Tag Register BIU Status Register BIU Address Register Fill Syndrome Register Fill Address Register Software ECC Register Console Halt Register ICCS NICR ICR TODR BIU_CTL DIAG_CTL BC_TAG BIU_STAT BIU_ADDR FILL_SYND FILL_ADDR BEDECC CHALT R/W WO RO R/W R/W WO RO W1C RO RO RO WO R/W 18 19 1A 1B A0 A1 A2 A4 A6 A8 AA AE B0 CPU Chip 2­73 ICCS·Interval Count Control and Status Register Table 2­47 ICCS Register Bit Definitions Address Access 0018 R/W The ICCS register contains control and status information on the interval clock. Name Bit(s) Type Function ERR MBZ INTR IE <31> <30:8 <7> <6> W1C, 0 R/W W1C, 0 R/W, 0 Error. Set whenever the Interval Count Register overflows and INTR (ICCS<7>) is already set. Thus, ERR indicates a missed overflow. Must be zero. Must always be written as zero. Interrupt. Set whenever the Interval Count Register overflows. If IE (ICCS<6>) is set when INTR is set, an interrupt is posted. Whenever the Interval Count Reg­ ister overflows and INTR is already set, ERR (ICCS<31>) is set. Reset clears ICCS<6> and ICCS<0>, but leaves the rest of ICCS UNPREDICTABLE. Interrupt Enable. When set, an interrupt request is generated on ICR overflows, that is, every time INTR (ICCS<7>) is set. When clear, no interrupt is re­ quested. Similarly, if INTR is already set and the soft­ ware sets IE, an interrupt is generated. This bit is cleared by reset. 2­74 CPU Chip Table 2­47 ICCS Register Bit Definitions (Continued) Name Bit(s) Type Function SGL XFR MBZ RUN <5> <4> <3:1> <0> WO WO R/W R/W Single Step. When RUN (ICCS<0>) is clear, writing one to SGL generates a pulse that causes the Interval Count Register to be incremented by one. When RUN is set, or if SGL and XFR (ICCS<4>) are written at the same time, the write to SGL is ignored. Multiple SGLs produce multiple increments. SGL does not require clearing. It always reads as zero. Transfer. Writing one to XFR generates a pulse that causes the NICR register to be copied to the ICR regis­ ter. Multiple XFRs produce multiple transfers. XFR does not require clearing. It always reads as zero. Must be zero. Must always be written as zero. Run. When set, the ICR register is incremented once per microsecond. When clear, the ICR register does not increment automatically. RUN is cleared during reset. CPU Chip 2­75 NICR·Next Interval Count Register Table 2­48 NICR Register Bit Definitions Address Access 0019 R/W The NICR register contains the value to be loaded into the ICR reg­ ister on count overflow or in response to a write of one to XFR (ICCS<4>). Name Bit(s) Type Function NINT_COUNT <31:0> R/W Next Interval Count. Contains the count that is loaded into the ICR register on overflow of ICR<31>. The value of NINT_COUNT is retained after the ICR register is loaded. NINT_COUNT is cleared by reset. 2­76 CPU Chip ICR·Interval Count Register Table 2­49 ICR Register Bit Definitions Address Access 001A RO The ICR register contains the interval count. Name Bit(s) Type Function INT_COUNT <31:0> R/W Interval Count. Contains the interval count. This field is incremented by a write of one to SGL (ICCS<5>) when RUN (ICCS<0>) is clear. When RUN is set, INT_COUNT is incremented by one every microsecond. Upon a carry­out (overflow) from ICR<31>, INT_COUNT is automatically loaded from the NICR register and continues incrementing. That is, the value of the ICR register on successive micro­ seconds will be FFFF FFFD, FFFF FFFE, FFFF FFFF, value of NICR<31:0>. The counter overflow sets INTR (ICCS<7>), which generates an interrupt, if the interrupt is enabled (that is, ICCS<6> is set). INT_COUNT can specify a maximum delay of ap­ proximately 1.2 hours. It is cleared by reset. CPU Chip 2­77 TODR·Time­of­Day Register Table 2­50 TODR Register Bit Definitions Address Access 001B R/W The TODR register forms an unsigned 32­bit binary counter that is driven from a 100 Hz oscillator, so that the least significant bit of the clock represents a resolution of 10 milliseconds. The TODR register counts only when it contains a non­zero value. Name Bit(s) Type Function TOD <31:0> RO Time­of­Day. Unsigned counter that indicates the time of day with a resolution of 10 ms. 2­78 CPU Chip BIU_CTL·BIU Control Register Table 2­51 BIU_CTL Register Bit Definitions Address Access 00A0 R/W The BIU_CTL register controls certain operations and parameters related to the P­cache, B­cache, and I/O mapping. This register reads the complement of its contents. Name Bit(s) Type Function WS_IO <31> R/W Workstation I/O. When set, indicates that I/O space is mapped for workstations. CPU Chip 2­79 Table 2­51 BIU_CTL Register Bit Definitions (Continued) Name Bit(s) Type Function BC_SIZE IO_MAP PV QW_IO_RD PCACHE_MODE <30:28> <14:13> <10> <9> <8> R/W R/W R/W R/W R/W B­Cache Size. Specifies the size of the B­cache as follows: BC_SIZE is not initialized on reset. Therefore, it must be explicitly written before the B­cache is en­ abled. I/O Map. Used on I/O references to allow selection of the range for I/O mapping by different systems. PV System Mode. Affects write probes to the B­ cache. When set, writes execute as if the B­cache were disabled; when clear, enables B­cache probes on writes. Read probes are not affected by the state of this bit. PV is set by hardware at initiali­ zation or on reset. Quadword I/O Read. When set, IO_SPACE DREADs that are not quadword­aligned return data from an internal register containing bits <63:32> of the previous quadword­aligned read. P­Cache Mode. Controls P­cache mapping mode. When set, P­cache functions as 2­way set associa­ tive; when clear, P­cache functions as direct­ mapped. BIU_CTL <30:28> B­Cache Size 000 001 010 011 100 101 110 128 Kbytes 256 Kbytes 512 Kbytes 1 Mbyte 2 Mbytes 4 Mbytes 8 Mbytes 2­80 CPU Chip Table 2­51 BIU_CTL Register Bit Definitions (Continued) BIU_CTL Read/Write A read of the BIU_CTL register provides the complement of its contents (inverted values of all bits). Bit positions not used for BIU_CTL functions reflect inverted values of the DIAG_CTL bits of the corresponding posi­ tions. Since the DIAG_CTL register is write only, its state can be deter­ mined by reading the BIU_CTL register. The following examples show how to read and write to the BIU_CTL regis­ ter through the console and how to decode BIU_CTL reads. Name Bit(s) Type Function BC_SPD BC_FHIT OE ECC BC_ENB <5:4> <3> <2> <1> <0> R/W, 0 R/W R/W R/W R/W B­Cache Speed. Indicates to the BIU the read and write access time, in CPU cycles, of the RAMs used to implement the B­cache. NVAX+ uses the BC_SPD field to program the read and write cache access time. BC_SPD is initialized on reset to 2 x CPU cycles (00 bin). B­Cache Force Hit. When this bit is set, and BC_ENB is also set, all READ_BLOCK and WRITE_BLOCK transactions are forced to hit in the B­cache. Tag and tag control parity are ignored when the BIU operates in this mode. BC_ENB takes prece­ dence over BC_FHIT. When BC_ENB is clear and BC_FHIT is set, no tag probes occur. Output Enable. When set, NVAX+ does not assert its chip enable lines during RAM write cycles, thus al­ lowing these lines to be connected to the output enable lines of the cache RAMs. Error Correction and Control. When set, NVAX+ generates/expects ECC report. When clear, NVAX+ generates/expects parity report. B­Cache Enable. When clear, disables the B­cache. When the B­cache is disabled, the BIU does not probe the B­cache tag store for read and write references; it launches a cycle request immediately. BIU_CTL <5:4> B­Cache Speed 00 01 10 2 x CPU cycle 3 x CPU cycle 4 x CPU cycle CPU Chip 2­81 Example 2­1 shows how to read the BIU_CTL register and decode its con­ tents. The read provides bit states for both the BIU_CTL register and the DIAG_CTL register. Only BIU_CTL register bits are commented on. Example 2­1 BIU_CTL Read >>> e biu_ctl ipr: 000000A0 (BIU_CTL) AFE09FF8 >>> ! >>> ! The complement of BIU_CTL is 501F 6007 >>> ! This decodes to the following state: >>> ! >>> ! BC_ENB = B­cache enabled >>> ! ECC = ECC mode >>> ! OE = OE mode >>> ! BC_FHIT = Not B­cache force hit >>> ! BC_SPD = 2x CPU cycle B­cache speed >>> ! PCACHE_MODE = P­cache in 2­way set associative mode >>> ! QW_IO_RD = Not enabled >>> ! IO_MAP = I/O map is 11 >>> ! BC_SIZE = B­cache size is 4 Mbytes >>> ! WS_IO = Not enabled Example 2­2 shows how to change B­cache speed from 2x CPU cycle time to 4x CPU cycle time. Example 2­2 BIU_CTL Write >>> e biu_ctl ipr: 000000A0 (BIU_CTL) AFE09FF8 >>> ! >>> ! The complement of BIU_CTL is 501F 6007 >>> ! Change BC_SPD from 00 (bin) to 10 (bin) gives 501F 6027 >>> ! Write this value to BIU_CTL >>> ! >>> dep biu_ctl 501F6027 >>> ! >>> e biu_ctl ipr: 000000A0 (BIU_CTL) AFE0 9FD8 >>> ! See the description of the DIAG_CTL register for examples on how to read and write to the DIAG_CTL register. 2­82 CPU Chip DIAG_CTL·Diagnostic Control Register Table 2­52 DIAG_CTL Register Bit Definitions Address Access 00A1 WO The DIAG_CTL register is used for diagnostics. Name Bit(s) Type Function SW_ECC PM_ACCS_TYPE <27> <26:24> WO, 0 WO Software ECC. When set, enables the use of ECC check bits from the BEDECC internal processor reg­ ister as given by software for write data. If SW_ECC is set when parity mode is selected (BIU_CTL<1> = 0), then BEDECC<0> is the parity bit generated for data <31:0> and BEDECC<7> is the parity bit for data <64:32>. Performance Monitor Access Type. Selects B­cache tag compare type for performance monitor access. CPU Chip 2­83 Table 2­52 DIAG_CTL Register Bit Definition (Continued) DIAG_CTL Read/Write Since the DIAG_CTL register is write only, it cannot be read directly. However, the contents of the DIAG_CTL register can be obtained indi­ rectly by reading the BIU_CTL register. Note that the state of a DIAG_CTL bit appears inverted when the BIU_CTL register is read (see description of the BIU_CTL register). The following examples show how to read and write to the DIAG_CTL reg­ ister through the console, and how to decode DIAG_CTL reads (through BIU_CTL reads). Name Bit(s) Type Function PM_HIT_TYPE DIS_ECC_ERR MBZ PACK_DISABLE TODR_INC TODR_TEST <23:21> <15> <12> <11> <7> <6> WO WO, 1 WO, 0 WO WO WO Performance Monitor Hit Type. Selects B­cache tag compare type for performance monitor access. Disable ECC Error. When set, disables ECC/data parity error reporting. Must Be Zero. Must always be written as zero. Pack Disable. When set, disables write packing, except for quadword packing, which is directed by microcode. Time­of­Day Register Increment. When set, the contents of the TODR register are incremented for testing, provided DIAG_CTL is also set. Time­of­Day Register Test. When set, places the TODR register in test mode. 2­84 CPU Chip Example 2­3 shows how to read the contents of the DIAG_CTL register. The read provides bit states for both the DIAG_CTL register and the BIU_CTL register. Only DIAG_CTL register bits are commented on. Example 2­3 DIAG_CTL Read >>> e biu_ctl ipr: 000000A0 (BIU_CTL) AFE09FF8 >>> ! >>> ! The complement of BIU_CTL is 501F 6007 >>> ! TODR_TEST = Not enabled >>> ! TODR_INC = Not enabled >>> ! PCACHE_MODE = P­cache in 2­way set associative mode >>> ! QW_IO_RD = Not enabled >>> ! PACK_DISABLE = Write packing enabled >>> ! MBZ = Written as zero >>> ! DIS_ECC_ERR = ECC reporting enabled >>> ! PM_HIT_TYPE = PMCNTR1 enabled for B­cache hit counting >>> ! PM_ACCS_TYPE = PMCNTR0 enabled for B­cache access counting >>> ! SW_ECC = Not enabled Example 2­4 shows how to change reporting of ECC errors from enabled to disabled. Example 2­4 DIAG_CTL Write >>> e biu_ctl ipr: 000000A0 (BIU_CTL) AFE0 9FF8 >>> ! >>> ! The complement of BIU_CTL is 501F 6007 >>> ! Set DIS_ECC_ERR gives 501F E007 >>> ! Write this value to DIAG_CTL >>> ! >>> dep diag_ctl 501FE007 >>> ! >>> e biu_ctl ipr: 000000A0 (BIU_CTL) AFE0 1FF8 >>> ! See the description of the BIU_CTL register for examples on how to read and write to the BIU_CTL register. CPU Chip 2­85 BC_TAG·B­Cache Error Tag Register Table 2­53 BC_TAG Register Bit Definitions 9 Address Access 00A2 RO The BC_TAG register is loaded with the results of every B­cache tag probe. When a tag or tag control parity error or primary fill data error (parity or ECC) occurs, this register is locked against further updates. The BC_TAG register is unlocked when the BIU_STAT<3:2> field is cleared. Software can read this register by using the MFPR instruction. 2­86 CPU Chip Name Bit(s) Type Function TAG TAGADR_P TAGCTL_P TAGCTL_S TAGCTL_D TAGCTL_V TAG_MATCH <31:17> <16> <15> <14> <13> <12> <11> RO RO RO RO RO RO RO Tag. Contains the tag that is being probed cur­ rently. Tag Address Parity. Reflects the state of the TAGP_H signal of the NVAX+ chip when a tag, tag control, or data parity error occurs. Tag Control Parity. Reflects the state of the TAGCTLP_H signal of the NVAX+ chip when a tag, tag control, or data parity error occurs. Tag Control Shared. Reflects the state of the TAGCTLS_H signal of the NVAX+ chip when a tag, tag control, or data parity error occurs. Tag Control Dirty. Reflects the state of the TAGCTLD_H signal of the NVAX+ chip when a tag, tag control, or data parity error occurs. Tag Control Valid. Reflects the state of the TAGCTLD_H signal of the NVAX+ chip when a tag, tag control, or data parity error occurs. Tag Match. When set, indicates that there was a tag match when a tag, tag control, or data par­ ity error occurred. CPU Chip 2­87 BIU_STAT·BIU Status Register Table 2­54 BIU_STAT Register Bit Definitions Address Access 00A4 W1C The BIU_STAT register reports error conditions that occur at the bus interface unit. This register is not unlocked or cleared by re­ set. It must be cleared explicitly by microcode. Name Bit(s) Type Function FILL_ADDR<33:32> BIU_ADDR<33:32> LOST_WRITE <31:30> <29:28> <20> RO RO W1C, 0 Fill Address <33:32>. Bits <33:32> of the FILL_ADDR register. Should be set only for I/O space address. This field is locked against fur­ ther updates when FILL_ADDR<31:5> is locked. BIU Address <33:32>. Bits <33:32> of the BIU_ADDR register. Should be set only for I/O space address. The field is locked against further updates when BIU_ADDR<31:5> is locked. Lost Write. A second error, and command is write. 2­88 CPU Chip Table 2­54 BIU_STAT Register Bit Definitions (Continued) Name Bit(s) Type Function FILL_DSP_CMD FILL_SEO FILL_QW <19:16> <14> <13:12> RO W1C, 0 RO Fill Dispatch Command. This field latches the DSP_CMD (dispatch command) that resulted in the BIU error and locks till BIU_STAT<14, 10:8> are cleared. Fill SEO. When set, indicates that a P­cache fill op­ eration resulted in either an uncorrectable ECC er­ ror or in a parity error while FILL_ECC or FILL_DPERR was already set. Fill Quadword. Identifies the quadword within the hexword P­cache fill block that caused the error. It can be used together with FILL_ADDR<33:5> to get the complete physical address of the quadword in error. This field is decoded as follows: FILL_QW is only meaningful when either FILL_ECC or FILL_DPERR is set. BIU_STAT <19:16> Dispatch Command 100X 1010 1100 1101 0010 0011 0111 0110 0101 0001 DREAD DREAD_IO DREAD_LOCK DREAD_LOCK_IO IREAD IREAD_IO WRITE_UNLOCK WRITE IO_WRITE WRITE_UNLOCK_IO BIU_STAT <13:12> Quadword in Error 00 01 10 11 1 2 3 4 CPU Chip 2­89 Table 2­54 BIU_STAT Register Bit Definitions (Continued) Name Bit(s) Type Function FILL_IRD FILL_DPERR FILL_CRD FILL_ECC BIU_SEO <11> <10> <9> <8> <7> RO W1C, 0 W1C, 0 W1C, 0 W1C, 0 Fill I­Cache Read. When set, indicates that the er­ ror that caused FILL_ECC or FILL_DPERR to set oc­ curred during an I­cache fill. When clear, it indicates that the error occurred during a D­cache fill. FILL_IRD is only relevant when either FILL_ECC or FILL_DPERR is set. BIU Parity Error. When set, indicates that the BIU received data with a parity error from outside the CPU chip while performing either a D­cache or an I­cache fill. FILL_DPERR is only relevant when the CPU chip is in parity mode, as opposed to ECC mode. Corrected Read. When set, indicates that the ECC error that caused FILL_ECC to set was correctable. When clear, it indicates that the error was not correctable. FILL_CRD is only relevant when FILL_ECC is also set. ECC Error. When set, indicates that the P­cache fill data received from outside the CPU chip contained an ECC error. BIU SEO. When set, indicates that an external cycle was terminated with a hard error or that a B­cache probe encountered bad parity in the tag address or tag control RAMs while BIU_HERR, BIU_SERR, BC_TPERR, or BC_TCPERR was already set. 2­90 CPU Chip Table 2­54 BIU_STAT Register Bit Definitions (Continued) BIU_STAT Updates When any of bits BIU_HERR, BC_SERR, BC_TPERR, or BC_TCPERR is set, BIU_STAT<6:0> are locked against further updates. The address as­ sociated with the error is latched and locked in the BIU_ADDR register. BIU_STAT<7:0> and the BIU_ADDR register are unlocked when BIU_STAT<7, 3:0> are written with ones. When FILL_ECC or FILL_DPERR is set, BIU_STAT<13:8> are locked against further updates. The address associated with the error is latched and locked in the FILL_ADDR register. BIU_STAT<14:0> and the FILL_ADDR register are unlocked when BIU_STAT<14, 11:8> are written with ones. Name Bit(s) Type Function BIU_DIS_CMD BC_TCPERR BC_TPERR BIU_SERR BIU_HERR <6:4> <3> <2> <1> <0> RO W1C, 0 W1C, 0 W1C, 0 W1C, 0 BIU Dispatch Command. This field latches dis­ patch command bits <3:1>, inverting bit <1> if the command is WRITE_UNLOCK, when a BIU_HERR, BIU_SERR, BC_TPERR, or BC_TCPERR error oc­ curs, and locks till BIU_STAT<7, 3:0> are cleared. B­Cache Tag Control Parity Error. When set, indicates that a B­cache tag probe encountered bad parity in the tag control RAM. B­Cache Tag Parity Error. When set, indicates that a B­cache tag probe encountered bad parity in the tag address RAM. BIU Soft Error. When set, indicates that an exter­ nal cycle was terminated with a soft (recoverable) er­ ror. BIU Hard Error. When set, indicates that an ex­ ternal cycle was terminated with a hard (nonre­ coverable) error. BIU_STAT <6:4> Dispatch Command 100 101 110 110 001 001 011 010 010 000 DREAD DREAD_IO DREAD_LOCK DREAD_LOCK_IO IREAD IREAD_IO WRITE_UNLOCK WRITE IO_WRITE WRITE_UNLOCK_IO CPU Chip 2­91 BIU_ADDR·BIU Address Register Table 2­55 BIU_ADDR Register Bit Definitions Address Access 00A6 RO The BIU_ADDR register contains the physical address associated with any errors reported in the BIU_STAT register. This register is locked against further updates upon detection of an error and is unlocked when it is read by the processor. Name Bit(s) Type Function BIU_ERR_PA <31:5> RO BIU Error Physical Address. Contains the physi­ cal address of the reference that caused the error re­ ported in the BIU_STAT register. BIU_ERR_PA is locked against further updates upon detection of an er­ ror and is unlocked when it is read by the processor. 2­92 CPU Chip FILL_SYND·Fill Syndrome Register Table 2­56 FILL_SYND Register Bit Definitions A syndrome of 0 indicates no ECC error for the associated longword. Table 2­57 outlines syndromes for all single­bit errors. Any non­zero syndrome not listed in the table indicates a double­bit error. Address Access 00A8 RO The FILL_SYND register contains the syndrome bits associated with the quadword in error if the NVAX+ chip is in ECC mode and an ECC error is recognized during a P­cache fill operation. The contents of the FILL_SYND register remain locked with the quadword in error until the processor reads the FILL_ADDR regis­ ter. The FILL_SYND register is 14 bits wide. Name Bit(s) Type Function HI LO <13:7> <6:0> RO RO High. Latches the ECC syndrome bits for the high longword. Low. Latches the ECC syndrome bits for the low longword. CPU Chip 2­93 Table 2­57 Syndromes for Single­Bit Errors Bit Syndrome (Hex) 1 Bit Syndrome (Hex) Data<0> Data<1> Data<2> Data<3> Data<4> Data<5> Data<6> Data<7> Data<8> Data<9> Data<10> Data<11> Data<12> Data<13> Data<14> Data<15> Data<16> Data<17> Data<18> Data<19> 4F 4A 52 54 57 58 5B 5D 23 25 26 29 2A 2C 31 34 0E 0B 13 15 Data<20> Data<21> Data<22> Data<23> Data<24> Data<25> Data<26> Data<27> Data<28> Data<29> Data<30> Data<31> ECC<0> ECC<1> ECC<2> ECC<3> ECC<4> ECC<5> ECC<6> 16 19 1A 1C 62 64 67 68 6B 6D 70 75 01 02 04 08 10 20 40 1 Any non­zero syndrome of 0 not listed in this table indicates a double­bit error. 2­94 CPU Chip FILL_ADDR·Fill Address Register Table 2­58 FILL_ADDR Register Bit Definitions Address Access 00AA RO The FILL_ADDR register contains the physical address associated with any errors reported in the BIU_STAT<14:8>. This register is locked against further updates upon detection of an error and is unlocked when it is read by the processor. Name Bit(s) Type Function FILL_ERR_PA <31:5> RO Fill Error Physical Address. Contains the physi­ cal address of the reference that caused the error re­ ported in the BIU_STAT<14:8>. FILL_ERR_PA is locked against further updates upon detection of an error and is unlocked when it is read by the proces­ sor. CPU Chip 2­95 BEDECC·Software ECC Register Table 2­59 BEDECC Register Bit Definitions Address Access 00AE WO The BEDECC register is a 14­bit write­only register. If BIU_CTL = 1, the check bits for write data are sourced from the BEDECC register instead of the normal check bit genera­ tion logic. Name Bit(s) Type Function HI LO <13:7> <6:0> WO WO High. Check bits of data<63:32>. Low. Check bits of data<31:0>. 2­96 CPU Chip CHALT·Console Halt Register Table 2­60 CHALT Register Bit Definitions Figure 2­22 shows the console dispatch data structure. Address Access 00B0 R/W The CHALT register contains the base address of the console pro­ gram. Microcode uses this register to determine where to start executing the console program whenever a console halt occurs as a result of an error condition or operator intervention (Ctrl/P typed at the terminal keyboard). Name Bit(s) Type Function CON_BASE_ADDR <31:0> R/W Console Base Address. Contains the console base address that is used by microcode to determine the starting address of the console program on a console halt. CON_BASE_ADDR is loaded at power­up by the console code. CPU Chip 2­97 Figure 2­22 Console Dispatch Data Structure The SYS_TYPE line contains information that distinguishes different NVAX+ based module implementations. Figure 2­23 shows the parame­ ters of the SYS_TYPE line. Figure 2­23 SYS_TYPE Parameters Table 2­61 gives the encoding of the SYS_TYPE parameters in the console dispatch data structure. 2­98 CPU Chip Table 2­61 SYS_TYPE Parameter Definitions Name Bit(s) Type Function SYS_TYPE REV_LEVEL SYS_VAR ARCH_ID <31:24> <23:16> <15:8> <7:0> RO RO RO RO System Type. Indicates the type of system: 0 for VAX 7000 and 1 for VAX 10000. Revision Level. Contains the revision number of the CPU module's console firmware. For the KA7AA CPU module, this field contains 01 (hex). System Variant. Distinguishes variants of similar systems. For the KA7AA CPU module, this field con­ tains 01 (hex). Architectural ID. Licensing bits that distinguish timesharing systems from workstations. For the KA7AA CPU module, this field contains 01 (hex). Cache Subsystem 3­1 Chapter 3 Cache Subsystem The cache subsystem of the KA7AA CPU module is organized as a three­ level instruction cache and a two­level data cache. It consists of the fol­ lowing structures: · Virtual Instruction Cache (VIC) On­chip, 2 Kbytes, direct mapped, virtually addressed, instruction stream only; has 32­byte block size and a valid bit per quadword subblock (fill) size. · Primary Cache (P­cache) On­chip, 8 Kbytes, 2­way set associative, physically addressed, read allocate, no­write allocate, write through, mixed instruction and data; has 32­byte block size and a valid bit per quadword subblock (fill) size. · Backup Cache (B­cache) Off­chip, 4 Mbytes, direct mapped, physically addressed, write back, mixed instruction and data; has 64­byte block size to match the LSB bus block size, and 64­byte fill size. The KA7AA CPU caches are not strictly hierarchical. Since the VIC is virtually addressed and holds I­stream data only, it is not necessarily a subset of the P­cache, which is always a subset of the B­cache. Maintain­ ing coherency in this type of cache organization requires strict adherence to the proper sequence of turning caches on and off. Although the instruction caching is not necessarily hierarchical, fetching instructions operates as if it were. If an instruction is not found in the VIC, it is looked for first in the P­cache, then in the B­cache, and finally in memory. Data is sought first from the P­cache, then the B­cache, and finally from memory. Figure 3­1 shows the KA7AA CPU module cache organization. 3­2 Cache Subsystem Figure 3­1 KA7AA CPU Module Cache Organization. Cache Subsystem 3­3 3.1 Virtual Instruction Cache The virtual instruction cache (VIC) acts as the primary source of instruc­ tion stream for the Ibox. It is direct­mapped with a 2­Kbyte cache size. Table 3­1 summarizes the VIC attributes. Table 3­1 Virtual Instruction Cache Attributes The VIC contains four internal processor registers (IPRs) that provide con­ trol and read/write access to the cache arrays. They are: · VIC Memory Address Register (VMAR), IPR208 · VIC Tag Register (VTAG), IPR209 · VIC Data Register (VDATA), IPR210 · Ibox Control and Status Register (ICSR), IPR211 The VIC registers are described in Section 2.11.2. Attribute Implementation Cache size Access type Block size Subblock size Valid bits Data parity bits Number of tags Tag parity bit Fill algorithm Access size Bus size Prefetching Data stored Virtual/physical 2 Kbytes Direct­mapped 32 bytes 8 bytes 4 per block; 1 per subblock 4 even parity bits per cache block; 1 per subblock 64 1 even parity bit per tag Fill forward, random cycle allocate if no tag hit or data subblock valid 8 bytes 8 bytes None I­stream only Virtual 3­4 Cache Subsystem 3.2 Primary Cache The primary cache (P­cache) represents the first level of D­stream memory hierarchy and the second level of I­stream memory hierarchy. The P­cache is 2­way set associative. It stores 8 Kbytes of data and 256 tags corre­ sponding to 256 hexword blocks (1 hexword = 32 bytes). Each tag is 20 bits wide corresponding to bits <31:12> of the physical address. There are four quadword subblocks per block with a valid bit associated with each subblock. The access size for P­cache reads and writes is one quadword. Parity is maintained for each byte of data (32 bits per block). One bit of parity is maintained for every tag. The P­cache has a one cycle access and a one cycle repetition rate for both reads and writes. Memory updates affect the P­cache as follows. If the block is in the P­ cache, then external logic invalidates the P­cache entry and updates the B­ cache. If the block is in the B­cache but NOT in the P­cache, external logic invalidates the B­cache entry and does nothing to the P­cache. Table 3­2 summarizes the P­cache attributes. Table 3­2 Primary Cache Attributes Five different internal processor registers (IPRs) control the P­cache and provide read/write access to the cache arrays. They are: · P­Cache Tag Registers (PCTAG) ·256 IPRs · P­Cache Data Parity Registers (PCDAP) ·1024 IPRs, 512 for each P­cache set · P­Cache Parity Error Address Register (PCADR), IPR242 · P­Cache Control Register (PCCTL), IPR248 · P­Cache Parity Error Status Register (PCSTS), IPR244 The P­cache registers are described in Section 2.11.4. Attribute Implementation Cache size Access type Block size Subblock size Valid bits Data parity bits Number of tags Tag parity bit Fill algorithm Access size Bus size Prefetching Data stored Virtual/physical 8 Kbytes 2­way set associative 32 bytes 8 bytes 4 per block; 1 per subblock 4 even parity bits per cache block; 1 per subblock 256 1 even parity bit per tag Fill forward, random cycle allocate if no tag hit or data subblock valid 8 bytes 8 bytes None Both I­ and D­stream Physical Cache Subsystem 3­5 3.3 Backup Cache The backup cache (B­cache) is implemented in RAMs and represents the second level of D­stream memory hierarchy and the third level of I­stream memory hierarchy. It consists of three RAM structures: B­tag, B­data, and B­stat, located between the NVAX+ chip and the LEVI interface, and is controlled by the Cbox and the LEVI interface. The B­cache stores 4 Mbytes of data. It is organized as direct­mapped, with a block (line) size of 64 bytes to match the LSB bus. For each block, the following information is stored: · Tag: Consists of bits <33:22> of the physical address · Tag parity: Reflects even parity over the tag field · Valid bit (V): Indicates whether the line can be considered · Shared bit (S): Indicates whether this line might be resident in an­ other cache in the system. · Dirty bit (D): Indicates whether the line has been written to by this CPU and has more recent data than memory. · Status parity bit: Reflects even parity over the V, S, and D bits. The B­cache organization groups the status bits in a single 64K X 4 RAM (B­stat) and allows these bits to be updated without changing the value of the tag. This in turn allows the CPU to set the Dirty bit on write hits to nonshared blocks. In general, the tag field is only loaded by the LEVI in­ terface, and the status and data stores are loaded by both the processor and the LEVI interface. 3.3.1 B­Cache States The B­cache state is defined by the three status bits: Valid, Shared, and Dirty. Table 3­3 shows the legal combinations of the status bits. From the perspective of the NVAX+ chip, a tag probe for a read is success­ ful if the tag matches the address and the V bit is set. A tag probe for a write is successful if the tag matches the address, the V bit is set, and the S bit is clear. 3­6 Cache Subsystem Table 3­3 B­Cache States 3.3.2 B­Cache State Changes The state of any given cache line in the B­cache is affected by either proc­ essor actions or actions of other nodes on the LSB bus. Table 3­4 shows what effect processor actions have on the state of a given B­cache line and the resulting/required bus traffic. Table 3­5 shows what effect bus actions have on the state of a given B­cache line, and the result­ ing/required module action. LSB writes always clean (make non­dirty) the cache line in both the initi­ ating node and all nodes that choose to take the update. They also update the appropriate location in main memory. The KA7AA CPU module decides whether to take an update or not as a function of the state of the P­cache backmap (P­map, Section 3.4.1). If the LEVI interface determines that the block referenced by an LSB write com­ mand is resident in the P­cache, the relevant block is updated in the B­ cache with the LSB write data and also invalidated in the P­cache, since the P­cache must be a subset of the B­cache at all times. If the LEVI inter­ face determines that the block referenced by an LSB write command is not resident in the P­cache, but is resident in the B­cache, it invalidates the relevant block in the B­cache. The KA7AA CPU module also compares incoming LSB addresses to those found in the LLOCK register, LVICT register, and LWPEND register (see Chapter 7). The behavior of the KA7AA CPU module in these cases is shown in Table 3­6. B­Stat V S D State of Cache Line Assuming Tag Match 0 X X 1 0 0 1 0 1 1 1 0 1 1 1 Not valid miss. Valid for read or write. This cache line contains the only cached copy of the block. The copy in memory is identical to this block. Valid for read or write. This cache line contains the only cached copy of the block. The contents of the block have been modified more recently than the copy in memory. Valid for read or write but writes must be broadcast on the bus. This cache line may also be present in the cache of another CPU. The copy in memory is identical to this block. Valid for read or write but writes must be broadcast on the bus. This cache line may also be present in the cache of another CPU. The contents of the block have been modified more recently than the copy in memory. Cache Subsystem 3­7 Table 3­4 Effect of Processor Action on B­Cache Line Processor Request Tag Probe Result1 Action on LSB LSB Response Next Cache State Read Read Write Write Read Read Write Write Read Read Write Write Read Write Write Write Invalid Invalid Invalid Invalid _____ _____ Match AND Dirty _____ _____ Match AND Dirty _____ _____ Match AND Dirty _____ _____ Match AND Dirty _____ Match AND Dirty _____ Match AND Dirty _____ Match AND Dirty _____ Match AND Dirty Match ______ Match AND Shared Match AND Shared Match AND Shared Read Read Read Read, Write Read Read Read Read, Write Read, Wr­Victim Read, Wr­Victim Read, Wr­Victim Read, Write, Wr­Victim None None Write Write ______ Shared Shared ______ Shared Shared ______ Shared Shared ______ Shared Shared ______ Shared Shared ______ Shared Shared None None ______ Shared Shared ______ _____ Shared, Dirty _____ Shared, Dirty ______ Shared, Dirty _____ Shared, Dirty ______ _____ Shared, Dirty _____ Shared, Dirty ______ Shared, Dirty _____ Shared, Dirty ______ _____ Shared, Dirty _____ Shared, Dirty ______ Shared, Dirty _____ Shared, Dirty No change ______ Shared, Dirty ______ _____ Shared, Dirty _____ Shared, Dirty ______ 1 An overscore on a cache block status bit indicates the complement of the state. For example, Shared = Not Shared. 3­8 Cache Subsystem Table 3­5 Effect of LSB Bus Action on B­Cache Line Table 3­6 KA7AA CPU Module Response to Incoming Addresses 3.4 Cache Backmaps The KA7AA CPU module implements two backmaps (or duplicate tag stores) that keep track of the contents of the P­cache and the B­cache. They are referred to as P­map and B­map. The backmaps are cycled with every bus transaction to allow the KA7AA CPU module to properly re­ spond to a given bus command/address. LSB Operation Tag Probe Result1 Module Response Next Cache State Comment Read Write Read Read Write Write _____ Match OR Invalid _____ Match OR Invalid _____ Match AND Dirty Match AND Dirty Match and line is interesting Match and line is uninteresting ______ _____ Shared, Dirty ______ _____ Shared, Dirty _____ Shared, Dirty Shared, Dirty _____ Shared, Dirty ______ _____ Shared, Dirty No change No change _____ Shared, Dirty Shared, Dirty _____ Shared, Dirty Invalid This module must supply the data. This module takes the update. This module takes the invalidate. ______ 1 An overscore on a cache block status bit indicates the complement of the state. For example, Shared = Not Shared. LSB Operation Address Register Matched Module Response Action Read Write Read Write Read Write LLOCK register LLOCK register LVICT register LVICT register LWPEND register LWPEND register Shared _____ Dirty Shared, Dirty ______ _____ Shared, Dirty Shared _____ Shared, Dirty No action Clear LLOCK<31> Supply data from victim buffer Invalidate victim buffer; remove bus request No action Accept update to B­cache 1 An overscore on a cache block status bit indicates negation. Cache Subsystem 3­9 NOTE: No backmap is necessary for the VIC, because the VIC is virtually ad­ dressed and is flushed on REI (Return from Exception or Interrupt) instruc­ tions. 3.4.1 P­Map The P­map is located in the LEVI gate arrays and consists of two struc­ tures, each 128 entries deep. Each entry contains a value that is equal to the difference between the B­cache tag (address bits <33:22>) and the P­ cache tag (address bits <31:12>), valid bit, and parity. Thus, the P­map is 12 bits wide: Address bits <21:12>, V, and P. The P­map is loaded by the NVAX+ chip during B­cache D­stream read hits and by LEVI during B­ cache D­stream read misses. LEVI control can read the P­map whenever an LSB write hits in the B­map. The KA7AA CPU module enforces inclusion, which ensures that the valid contents of the P­cache are always a subset of the valid contents of the B­ cache. Therefore, the KA7AA CPU module must invalidate P­cache lines whenever the given block becomes invalid in the B­cache. This occurs on refills (either a dirty victim or a nonshared victim) and on updates. When an update occurs on the LSB bus, and the given address yields a tag match and the entry is valid in the P­map, the B­cache takes the update and the CPU module invalidates the corresponding entry in the P­cache. 3.4.2 B­Map The B­map is located on the module and is a structure 64K entries deep. Each entry consists of the B­cache tag (address bits <33:22>), valid bit, and parity. The B­map is written by LEVI at the same time that the B­cache tag is written (within the context of B­cache manipulation, due to either processor action or bus action). The B­map is read on every LSB bus com­ mand/address cycle. The contents of the B­map inform the KA7AA CPU control logic when to request the B­cache to form an appropriate bus re­ sponse. 3.5 Victim Buffer The KA7AA CPU module implements a victim buffer to hold the contents of a victimized block in the B­cache. A victim block is a B­cache line that is valid and dirty but has a tag mismatch for a processor request. The proces­ sor tag probe yields a miss and the appropriate block is fetched from mem­ ory. However, the block in the B­cache at this index must be written back to memory since it is dirty. The KA7AA CPU module posts the miss refill to the bus before actually performing the victim write. A single victim block and victim address pair is stored in the LEVI chips for later transmission on the bus. While the victim buffer contains a valid victim, the KA7AA CPU module treats this block like a second set in the B­cache, compares all bus addresses to the victim address, and responds to bus reads and writes as required by the bus protocol (see Table 3­6). The KA7AA CPU module implements only a single victim buffer. It there­ fore does not process a second B­cache miss before writing the victim block to memory. 3­10 Cache Subsystem 3.6 Write Policy The NVAX+ chip contains an 8­byte write packer and an 8­entry write queue where each entry is a quadword of data. The function of the write packer is to accumulate memory­space writes that arrive sequentially to the same quadword, so that only one write needs to be done to the cache. Only normal write commands to the same quadword are packed together. All other writes (that is, Write_Unlock) pass immediately from the write packer into the write queue. Data in the write queue is assembled into an octaword (when possible) and written to the B­cache when the Cbox arbiter does not see a request of higher priority, or when one of the following con­ ditions exists: · A DREAD_LOCK or WRITE_UNLOCK command. · An IPR_READ or IPR_WRITE command (including Clear Write Buffer). · An I/O space read or I/O space write. · A DREAD or IREAD conflict with data in the write packer or write queue. (Addresses are checked to hexword granularity on address bits <31:5>.) Depending on how it is able to pack and assemble data in the write queue, the Cbox is able to write between one and four longwords of data at a time to the B­cache. For the case of a byte/word write, the Cbox does a Read­ Modify­Write operation, which is implemented in hardware. When the Cbox detects a byte/word write condition that incurred a B­cache miss, it does the following: 1. Issues an LDxL command and drives the EDAL address lines with the write address. 2. Receives a quadword of data, checks ECC, and merges the read data with the byte/word write data. 3. Issues an STxC command along with the correct LW_MASK<7:0> and write address. If the STxC command fails (LLOCK<31> not set), microcode loops back to 1 (first element in the current list) and issues an LDxL. Otherwise, the Cbox does a B­cache read hit, merge, and B­cache write hit. The KA7AA CPU module performs LSB bus write operations as follows: · Victims If a given cache line is valid and dirty and the tag does not match the address for the given processor request, the line must be written back to memory. To enhance performance, this victim is written back to memory after the refill. The victim data must be removed from the B­ cache data store and held in a victim buffer for later transmission on the LSB bus. While a block is in a victim buffer, the KA7AA CPU must respond to all reads and writes that reference the block (see Ta­ ble 3­6). · Shared Blocks If the response to a tag probe for a processor write is shared, the write must be broadcast on the LSB bus. Cache Subsystem 3­11 3.7 B­Cache Operating Modes The backup cache has two modes of operation: · B­cache on · B­cache force hit The operating modes are controlled by two bits in the BIU_CTL register: BC_ENB (bit <0>) and BC_FHIT (bit <3>). Table 3­7 shows how the operating mode of the B­cache is selected. Table 3­7 Selection of the B­Cache Operating Mode The On state is the normal operating mode of the B­cache. It is selected by setting BIU_CTL<0> and clearing BIU_CTL<3>. The B­cache force hit mode is selected by setting FORCE_HIT (BIU_CTL<3>) when the B­cache is enabled. When FORCE_HIT is set, all memory space reads and writes to the B­cache, both I­stream and D­ stream, are forced to hit. The tag store state is not changed. The data RAMs are accessed as if the tag store access produced a dirty­valid hit. In a multiprocessor environment, the B­cache must be flushed of all dirty blocks before force hit mode is selected. Force hit mode is intended to be used only for testing and initialization. Tag store parity and data RAM ECC errors are detected in this mode. 3.8 Cache Initialization On power­up or following a reset, the processor microcode and the console firmware initialize the P­cache and the B­cache. In the initialized state, the P­cache is enabled for I­stream and D­stream operations, and the B­ cache is on. CAUTION: The cache subsystem is initialized to a determined state. Software must never turn the B­cache off once the system is up and running. Turning the B­cache off during normal operation places the system in an UNDETER­ MINED state. BIU_CTL<3> BIU_CTL<0> Operating Mode 0 1 1 1 B­cache on Force hit LSB Bus Interface 4­1 Chapter 4 LSB Bus Interface The CPU module connects to the LSB bus through LEVI, the LSB inter­ face, which is implemented in two gate array chips, LEVI­A and LEVI­B. LEVI controls all the tags, maps, and data RAMs on the CPU module. It contains the P­map, which maps the processor P­cache. LEVI performs the following major tasks: · Translates CPU memory and I/O space references to the appropriate LSB transactions. · Supports control of writebacks to memory and cache fills from mem­ ory in reponse to processor actions. · Supports control of cache invalidates, cache updates, and cache block transfers to the LSB bus in response to LSB actions. · Initiates reads and writes to the CPU node private address space (the Gbus on the CPU module). · Supports LSB required interrupt logic. · Implements all LSB required registers. This chapter discusses the role of LEVI in transactions between the CPU module and other modules on the LSB bus. Sections include: · LEVI Address Path · LEVI Data Path · LEVI Controllers · Interfacing Rules · Address Space Mapping · LEVI Transactions Figure 4­1 shows a block diagram of the LEVI chips. 4­2 LSB Bus Interface Figure 4­1 LEVI Block Diagram 4.1 LEVI Address Path The LEVI address path (see Figure 4­1) is implemented in the LEVI­A chip. It consists of the following major elements: · P­Map The P­map consists of four 64 X 16 dual­ported RAMs maintained ex­ clusively by LEVI. Each entry in the P­map represents a P­cache block in the processor. LEVI writes to the P­map during processor read hits to the B­cache. One use of the P­map is deciding whether to update or invalidate a B­cache block during LSB writes from another node. If the LSB write hits in the P­map, the update is taken; otherwise LEVI in­ validates the B­cache. Another use is to invalidate P­cache blocks that are being displaced by B­cache fills. · LVICT Register LEVI keeps the address of the last victimized B­cache block and a valid bit in the LVICT register. Once the LSB Victim Write takes place, the Valid bit is cleared. Should another LSB (non­Victim) write match the address in the LVICT register, LEVI invalidates its own LVICT regis­ ter (see Table 3­6). · LLOCK Register The address is latched and LLOCK<31> is set when the processor is­ sues an LDxL instruction. LLOCK<31> is cleared after a successful LSB Bus Interface 4­3 STxC instruction. LSB reads that hit in the LLOCK register cause LEVI to respond Shared so that all subsequent writes to the address are visible on the LSB bus. LSB writes that hit in the LLOCK register cause LEVI to clear the LLOCK<31> (see Table 3­6). · LWPEND Register This register contains the address of the pending write and a valid bit. If an LSB Write hits the LWPEND register, LEVI­A takes the update even if the write missed in the P­map to assure that the write about to be issued has the latest data (see Table 3­6). 4.2 LEVI Data Path The LEVI data path, like the LSB bus and CPU module data paths, is 156 bits wide: 128 bits of data and 28 bits of ECC (7 bits for each longword). Note that LEVI treats the data and ECC bits identically, since there is no ECC correction between the B­cache and the LSB bus. An array of buffers in the LEVI data path serve to store data and synchro­ nize data movement within LEVI. The buffers are implemented in both LEVI chips, as shown in Figure 4­1. The main buffer elements on the LEVI data path are the following: · Fill Buffer The fill buffer works with the LEVI buffer on the module to receive, and possibly hold, four octawords of LSB data headed for the B­cache. The data pipeline shifts from the gate array time domain to the clock­ forwarded module time domain in the fill buffer. The fill buffer also merges write buffer data with LSB data following B­cache write misses. · Get Buffer The get buffer also works with the LEVI buffer; it captures B­cache blocks headed for the LSB bus. The data pipeline shifts back from the module to the gate array time domain in the get buffer. · Write Buffer The write buffer captures two octawords of write data from the proces­ sor in three situations: B­cache write misses B­cache write hits to shared blocks CSR writes The write buffer also receives a write data mask (LEVI­A gets four of eight bits; LEVI­B gets all eight bits) and ADDR<5> from the proces­ sor. The mask and address bits indicate which longwords are to be merged with B­cache data. · Stall Buffer The stall buffer holds four octawords of B­cache data for broadcast onto the LSB bus. It also merges write buffer data with B­cache data dur­ ing writes to shared blocks and processor CSR writes. · Victim Buffer The victim buffer holds B­cache blocks victimized by cache fills. The buffer holds only a single cache block so transactions that cause other victims are held off until the current victim reaches the LSB bus. 4­4 LSB Bus Interface 4.3 LEVI Controllers The control functions on the LEVI transactions are implemented in three controllers in the LEVI chips: · LEVI processor controller (LPC) · LEVI data controller (LDC) · LSB controller (LC) 4.3.1 LEVI Processor Controller The LPC provides the control interface between the processor and LEVI. It fields requests from the processor and initiates LEVI responses. Major functions include: · LoadLock/StoreCond The LPC first probes the cache; misses generate requests to the LSB controller for LSB transactions. · Gbus Read The LPC asks the LSB controller for a CSR read and does extra hand­ shaking on the Gbus. It appears on the LSB as a private command (not a CSR read). · Gbus Write The LPC controls the write to the LEVI write buffer, then handshakes with the Gbus and acknowledges the processor. No LSB transaction is requested. · Processor Read Fill After a processor read miss and after LEVI has received the missed data from the LSB bus, the LPC loads the processor with the two octawords it has waited for. The LEVI data controller (LDC) briefly in­ terrupts the B­cache fill after the first octaword write to allow the proc­ essor to load two octawords from the LEVI buffer. The LDC then com­ pletes the final three octaword writes. · Processor Write Data The LPC controls the processor writes to the write buffer on LEVI when the processor cannot write directly to the B­cache. · P­Cache Invalidates Whenever LEVI invalidates P­map entries, the LPC invalidates the corresponding P­cache entries in the processor. The LPC runs in the processor time domain. 4.3.2 LEVI Data Controller The LDC directs data traffic moving between the LSB and the B­cache based on requests from the LC. Each transaction described below moves one B­cache block. The LDC is involved in the following transactions: · GetRAM GetRAM moves one B­cache block to the LSB by way of the stall buffer. The LSB controller (LC) requests this transfer when another node has issued a read to a block and the local (and only valid) copy is dirty. LSB Bus Interface 4­5 · GetWBRAM GetWBRAM is used on processor writes to shared B­cache blocks and processor CSR writes. The fetched B­cache block is conditionally merged with the contents of the write buffer (based on the values of the write data mask and ADDR<5>) before being driven onto the LSB by way of the stall buffer. · GetVic GetVic is used to route B­cache blocks that are victimized by B­cache fills to the victim buffer. Note that blocks in the victim buffer must await an LSB slot; blocks in the stall buffer have already had their LSB slots allocated (by the LC). · FillRAM FillRAM moves one block directly from the LSB to the B­cache. The LC requests this to take an update to a shared block or to complete the first read of a processor write miss to a shared B­cache block. · FillProcRAM FillProcRAM is requested following processor read misses. The LDC moves data from the LSB to the fill buffer and the LEVI buffer on the module. The data pipeline is frozen briefly after the first octaword write to the B­cache to allow the LPC to load the first two octawords into the processor (in its own time domain) by way of the LEVI buffer. The LPC then releases the processor but retains control of the B­cache so that the LDC can write the remaining three octawords to the B­cache. · FillProc FillProc services processor CSR reads. This transaction is identical to FillProcRAM discussed above except that writes to the B­cache are suppressed. (CSR data is not cached.) · FillWBRAM FillWBRAM merges processor write data in the Write buffer with the incoming LSB data and writes the result into the B­cache. Merging is based on the values of the write data mask and ADDR<5>. The LC re­ quests this transaction following processor write misses to blocks that are not shared. The LDC uses clock forwarding on the CPU module for data transfers be­ tween LEVI and the B­cache. 4.3.3 LSB Controller The LC is the central controller of the LEVI chipset. It receives requests from the LPC and issues requests to the LPC, LDC, and the LSB arbiter. The LC responds to both processor­initiated and LSB­initiated transac­ tions. Specifically, the LC performs the following functions: · Controls the address path and LEVI access to the B­map, B­stat, and B­tag RAMs on the CPU module. · Schedules all LEVI and CPU module operations except B­cache hits and Gbus transactions. Requests from the LC to the LPC and LDC move data around the module, the gate arrays, and the LSB bus. · Asserts the LSB address and control signals (CNF, ERR) according to LSB protocol. LSB SHARED and DIRTY are asserted based on the re­ 4­6 LSB Bus Interface sults of B­tag and B­stat lookup. LSB STALL is asserted when re­ quired by internal conflicts. · The LC also controls B­cache access from the processor or LEVI with the LSYNC signal (Section 4.4.1). The LC schedules transfers of data between the LEVI and the CPU module during dedicated LSB cycles. 4.4 Interfacing Rules Logic on the CPU module synchronizes dual­ported accesses to the B­cache and the P­map, since these components are accessed by both the processor and LEVI. On the other hand, arbitration rules govern node accesses to the LSB bus. All cache data is longword ECC protected (seven bits per longword). LEVI does look­aside ECC error detection but no ECC error correction. The LEVI chips calculate ECC for each longword and compare it against the received ECC. Any difference between calculated and received ECC in­ dicates an error, which is signaled to the system. The ECC for longword 0 and a partial ECC syndrome for longword 1 are passed each cycle from LEVI­B to LEVI­A. 4.4.1 Dual­Ported Access Synchronization Dual­ported B­cache and P­map accesses are synchronized with the LSYNC semaphore. LSYNC is also used to synchronize access to the CPU module data path during Gbus references. Whenever LSYNC is deasserted (default state), the processor can read or write the B­stat, B­tag, and B­data RAMs directly. NOTE: The B­map RAMs are never accessed by the processor. During LoadLock and StoreCond requests, whenever LSYNC is deasserted, the LPC can read or write the B­stat, B­tag, and B­data RAMs directly. During RBlock and WBlock requests to Gbus addresses, whenever LSYNC is deasserted, the LPC can transfer data between the processor and the Gbus buffer on the CPU module. LEVI has priority to assert LSYNC and access the B­cache to service LSB transactions, since the LSB is non­ pended. LEVI also accesses the B­cache to complete processor transactions that miss the B­cache. When the LC asserts LSYNC, the processor and the LPC suspend B­stat/B­tag references (tag probes), P­map updates (PMapWE), and B­data references within a fixed number of LSB cycles. The LEVI is then free to access any resource within the CPU module until it deasserts LSYNC at the completion of the LSB­related access. 4.4.2 LSB Arbitration LEVI watches all LSB traffic to adhere to the arbitration rules. Specifi­ cally, read, write, or victim transactions from any node that reference a common memory bank cannot occur more frequently than once every three transactions (or once every 15 LSB cycles). CSR transactions are also lim­ ited in the same manner. LSB Bus Interface 4­7 4.5 Address Space Mapping The LEVI chips define which portion of the address space is cacheable or noncacheable. Cacheable address space is memory space and noncacheable address space is I/O space. The LEVI chips further separate I/O space into LSB bus CSR space and local Gbus space. The LEVI interface ensures that processor references to memory result in an LSB bus read or write command, while references to I/O space result in an LSB read CSR, write CSR command, or private command. Table 4­1 gives the encodings of commands that LEVI can send to the LSB bus. Table 4­1 LSB Command Field Encodings 4.6 LEVI Transactions As the CPU module's interface to the LSB bus, LEVI responds to transac­ tions initiated from two sources: · Processor (CPU chip) · LSB bus (other nodes) These transactions require that both the processor and LEVI have access to the B­cache on the CPU module and the P­map in LEVI. The dual­ ported accesses to these components are synchronized with the LSYNC semaphore (Section 4.4.1). The two LEVI chips operate in both the proces­ sor and the LSB bus time domains. 4.6.1 Processor­Initiated Transactions LEVI responds to the following processor requests: · Read/Write Hit During a D­stream read hit, LEVI updates its P­map. It takes no other action. · Block Read/Write LEVI captures B­stat and B­tag data, arbitrates for the LSB bus, is­ sues the read/write command code on the bus, receives/drives data on the bus, and updates all tags, maps, and B­stat bits. LSB D<37:35> Command 000 001 010 011 100 101 110 111 Read Write Reserved Write Victim Read CSR Write CSR Reserved Private 4­8 LSB Bus Interface · LoadLock/StoreCond LEVI waits for LSYNC to deassert, if necessary, then probes the B­tag. On an LDxL (LoadLock) command that hits in the B­cache, LEVI com­ pletes the read request and sets LLOCK<31>. If the LDxL is a B­ cache miss, LEVI issues an LSB bus read command and sets LLOCK<31>. On an STxC (StoreCond) request from the processor, LEVI checks LLOCK<31>. If this bit is set, (success) and the B­cache tag lookup results in a hit, LEVI immediately completes the write and clears LLOCK<31>. If the tag probe results in a miss, and LLOCK<31> is set, LEVI issues an LSB bus write command. On an STxC, if LLOCK<31> is clear, LEVI returns failed status to the proces­ sor. · Gbus Read/Write LEVI waits for LSYNC to deassert, if necessary. For Gbus reads, LEVI­A arbitrates for the LSB bus, issues a private command, for­ wards data from the Gbus to the processor by way of the LSB bus. Gbus writes slip through LEVI to the Gbus without an LSB transac­ tion. The processor can be engaged in only one external operation at a time. This means that once the processor makes a transaction request to LEVI, it remains idle until released by LEVI. 4.6.2 LSB­Initiated Transactions LEVI responds to transactions initiated by other nodes on the LSB. These transactions include: · Read LEVI checks each read address against the B­map. If there is a match, LEVI then checks the B­stat RAMs. It returns B­cache data if the Dirty bit is set. LEVI returns a victimized block if the block's address matches the read address. · Write When the write address matches that of a valid block in the B­map, LEVI reacts as follows. If the address also hits in the P­map, LEVI takes the update and invalidates the P­cache block in the processor. Otherwise, the B­cache block is simply invalidated. Note that this be­ havior can be altered with the LMODE register. · Victim Write LEVI ignores victim writes from other nodes. Victimized blocks do not hit in the local B­cache. · CSR Read/Write Only registers in the LSB node space can be read or written from the LSB. Gbus registers cannot be accessed from the LSB. Note that LEVI can also respond to its own processor­generated CSR transac­ tions on the bus. · Private Private transactions are used to return Gbus data to the processor, to allow access to the B­tag, B­stat, B­map, and P­map structures directly by the processor, and to resolve STxC boundary conditions. LEVI does not respond to private commands from other modules. LEVI is pipelined to track up to three interleaved LSB transactions. LSB Bus Interface 4­9 4.6.3 Transaction Ordering The processor controller (LEVI PC, Section 4.3.1) and the LSB controller (LEVI LC, Section 4.3.3) work together to guarantee strict ordering of transactions issued on the LSB. Processor and LEVI actions proceed in stages as shown in Table 4­2. Table 4­2 Processor­LEVI Actions During Transactions Processor Action LEVI Action P1. The processor issues a request with address A1. P2. The processor can issue a new request with address A2 any time after L1 completes. In response to P1, LEVI performs the following actions: L1. LEVI initiates an LSB trans­ action with address A1. L2. If P1 was a WBlock and L1 was an LSB Read that received a shared response, LEVI issues an LSB Write with address A1. L3. If L1 was an LSB Read and the B­cache block being displaced had the Dirty bit set, LEVI issues an LSB Write Victim command. In response to P2, LEVI performs the following action: L4. LEVI initiates an LSB trans­ action with address A2. Console Overview 5­1 Chapter 5 Console Overview The KA7AA CPU module supports the LSB system console with com­ bined hardware/software elements that control the system at power­up, on reset, or on CPU halts. This chapter describes the console hardware that resides on the CPU module. Sections include: · CPU Console Hardware · Console Program Invocation · Console Registers The console user interface and commands are discussed in the Console Reference Manual. 5.1 CPU Console Hardware The KA7AA CPU module provides hardware to support the console func­ tions. This hardware includes: · A serial ROM (read­only memory) for first­level console program stor­ age · A set of FEPROMs (flash programmable ROMs) for second­level con­ sole program storage · An EEPROM (electrically erasable/programmable ROM) for miscella­ neous parameter/log storage · A set of UARTs (universal asynchronous receivers/transmitters) that allow the console program to communicate serially with one console terminal and the system power supplies · A watch chip that provides a battery­backed­up time­of­year (TOY) clock for use by operating system software · A set of parallel I/O ports for functions such as LED status indicators and node identification · A serial I/O port for manufacturing diagnostic use The CPU module provides access to ROM, EEPROM, console UARTs, the watch chip, and other functions through the 8­bit Gbus. All Gbus component registers and memory stores are located in node pri­ vate space, which means that their addresses are constant and are inde­ pendent of slot identification. Table 5­1 gives the address ranges allo­ cated to the Gbus components. 5­2 Console Overview Every Gbus memory store byte or register byte is located on a 64­byte, naturally aligned boundary. For example, the first byte of FEPROM stor­ age is located at byte address F000 0000; the second byte is at F000 0040. Also note that a single 128­Kbyte FEPROM consumes 8 Mbytes of address space. This addressing restriction implies that processor code cannot be executed from this address space. Table 5­1 Gbus Components 5.1.1 Serial ROM After power­up, node reset, or system reset, but before any instructions are executed, the NVAX+ chip automatically loads its internal P­cache through the serial I/O port from an external, 8­Kbyte serial ROM. The serial ROM contains the first level of console/diagnostic/bootstrap code (serial ROM code). This code initializes all programmable features of the NVAX+ chip, diagnosing any faults detected along the bootstrap path and bootstrapping code execution out to the second level of console/diagnostic/ bootstrap code (the main console program). The first level bootstrap copies the main console program code from FEPROM storage to the P­cache and transfers control flow to the P­cache. Once the serial ROM is loaded into the P­cache, the same serial I/O port becomes available for use by software as a diagnostic interface. Component Address Console ROM Console EEPROM UART registers WATCH registers Gbus$WHAMI Gbus$LEDs Gbus$PMask Gbus$Intr Gbus$Halt Gbus$LSBRST Gbus$Misc Gbus$RMode Gbus$LTagRW F000 0000 to 0000 F380 0000 to FFFF F400 0000 to FFFF F600 0000 to 0FC0 F700 0000 F700 0040 F700 0080 F700 00C0 F700 0100 F700 0140 F700 0180 F780 0000 F780 0100 Console Overview 5­3 5.1.2 Serial Port The NVAX+ chip provides an initialization and diagnostic interface in the form of a serial I/O port. The serial I/O port is a full duplex connection be­ tween the CPU chip and a module connector. The port is accessed and con­ trolled through internal processor registers. The serial I/O port drives a LED indicator, which may flash as data is transmitted over the serial port, but is otherwise available to diagnostic code as a status indicator. 5.1.3 FEPROMs The console program is stored in a set of 128K X 8 FEPROM chips. This code does not appear in a structure of contiguous locations in the proces­ sor's address space. Specifically, each byte of FEPROM storage appears on a 64­byte naturally aligned boundary. This implies that the console pro­ gram cannot execute directly out of FEPROM, but instead, must be copied into a more compact contiguous space in cacheable memory and executed from there. This process of copying the code store and transferring control flow is known as the first­level bootstrap and is performed by the serial ROM code, as explained in Section 5.1.1. The FEPROMs can be programmed online without assistance from an ex­ ternal programming device. The FEPROMs cannot be patched; they can only be erased and programmed as a whole. 5.1.4 EEPROM A single 8K X 8 EEPROM is used for miscellaneous parameter and log storage. This store does not appear in a contiguous address space. Specifi­ cally, each byte of EEPROM storage appears on a 64­byte boundary. The EEPROM can be written to byte­by­byte online, without assistance from an external programming device. 5.1.5 UARTs The CPU module has six serial communication lines but uses only three. The communication lines are named and assigned as follows: · UART0A is connected to the LSB local console terminal line LOC_RX/ LOC_TX (computer room terminal for field service). · UART1B is connected to the LSB power supply status lines PS_RX and PS_TX. · UART2A is dedicated to Ctrl/P character detection. Its receive line can tap receive characters off LOC_RX, OP_RX, or RD_RX as selected by the Gbus$PMask register. Its transmit line is unused. · UART0B, UART1A, and UART2B are unused. The LSB console serial lines are connected to all CPU slots. After power­ up or system initialization, the CPU modules arbitrate for use of the com­ mon console lines; the winner is allowed to drive them. The default con­ figuration of the serial lines at power­up is as follows: 5­4 Console Overview Baud rate set to 9600 No parity One stop bit 8­bit characters One physical component (DUART) implements two UARTs, hence the naming of the UARTs as UART0A, UART0B, and so on, where the number indicates the physical component and the letter indicates the individual UART within the component. Control of these UARTs is accomplished through a set of registers in each UART. These registers are listed in Ta­ ble 5­2. 5.1.5.1 Ctrl/P Character Detection and Halt Protection UART2A is dedicated to detecting Ctrl/P characters received from the con­ sole terminal. UART2A intercepts a copy of all UART receive characters from the console terminal line and compares for Ctrl/P. Characters other than Ctrl/P re­ sult in an IPL15 interrupt posted to the processor (reflected in the Gbus$Intr register). Ctrl/P characters result in an IPL1F interrupt (halt) posted to the processor (reflected in the Gbus$Halt register). Note that the IPL1F interrupt is in addition to the IPL15 interrupt. If no serial lines are selected for console operation (the processor is halt­protected), then all re­ ceive characters result in an IPL15 interrupt. For UART2A to detect Ctrl/P characters, all control settings must be programmed to match the console terminal UART. 5.1.5.2 UART Register Addressing Each UART in a DUART component is controlled independently through its own set of registers (some registers are shared between two UARTs within a DUART). All UART registers are either read only (for status and data receive) or write only (for control and data transmit). Read registers and write registers share common addresses, that is, reading and writing a single address accesses two separate registers. For each UART there are two read registers and two write registers that are directly accessible in the processor's address space: RR0, WR0, RR8, and WR8. RR0 and WR0 are the main status and control registers for the UART. RR8 and WR8 are the data receive and transmit registers. For each UART there are a number of other control and status registers that are indirectly accessible through RR0 and WR0. These registers are accessed by writing the correct index value into WR0 and then reading RR0 or writing WR0. After the second read/write operation occurs, the in­ dex value is automatically reset back to zero. 5.1.6 Watch Chip A watch chip resides on the Gbus and provides a battery­backed­up time­ of­year clock and 50 bytes of battery­backed­up RAM. The chip contains a built­in crystal oscillator and a 10­year lithium battery. Console Overview 5­5 5.2 Console Program Invocation The NVAX+ chip operates in console mode when the CPU module encoun­ ters one of the following conditions: · System reset through power­up, control panel reset, or reset through the Gbus$LSBRST register · Module reset performed by setting NRST (LCNR<30>) · Module halted by setting NHALT (LCNR<29>) · Ctrl/P character received from the console terminal When the NVAX+ chip enters console mode, it transfers control to the lo­ cation addressed by the contents of the CHALT register. 5.3 Console Registers Table 5­2 lists the console registers with their addresses and indicates the components in which they are implemented. Table 5­2 Console Registers Register Address Implementation UARTxx$WR01 UARTxx$WR1 UARTxx$WR2 UARTxx$WR3 UARTxx$WR4 UARTxx$WR5 UARTxx$WR6 UARTxx$WR7 UARTxx_BASE1 Index 0001 Index 0010 Index 0011 Index 0100 Index 0101 Index 0110 Index 0111 DUART chip DUART chip DUART chip DUART chip DUART chip DUART chip DUART chip DUART chip 1 UART Base Addresses: xx = 0B; BASE = F400 0000 xx = 0A; BASE = F400 0080 xx = 1B; BASE = F480 0000 xx = 1A; BASE = F480 0080 xx = 2B; BASE = F500 0000 xx = 2A; BASE = F500 0080 5­6 Console Overview Table 5­2 Console Registers (Continued) A number of console/diagnostic/interrupt related registers listed in Table 5­2 are referred to with a prefix of Gbus$. These registers provide the fol­ lowing control and status functions: · Node identification · LED status indicators Register Address Implementation UARTxx$WR8 UARTxx$WR9 UARTxx$WR10 UARTxx$WR11 UARTxx$WR12 UARTxx$WR13 UARTxx$WR14 UARTxx$WR15 UARTxx$RR0 UARTxx$RR1 UARTxx$RR2 UARTxx$RR3 UARTxx$RR8 UARTxx$RR10 UARTxx$RR13 UARTxx$RR15 Watch$Seconds Watch$Minutes Watch$Hours Watch$Day_of_Month Watch$Month Watch$Year Watch$CSRA Watch$CSRB Watch$CSRC Watch$CSRD BBackup RAM (50 bytes) Gbus$WHAMI Gbus$LEDs Gbus$PMask Gbus$Intr Gbus$Halt Gbus$LSBRST Gbus$Misc Gbus$RMode Gbus$LTagRW UARTxx_BASE+40H Index 1001 Index 1010 Index 1011 Index 1100 Index 1101 Index 1110 Index 1111 UARTxx_BASE Index 0001 Index 0010 Index 0011 UARTxx_BASE+40H Index 1010 Index 1101 Index 1111 F600 0000 F600 0080 F600 0100 F600 01C0 F600 0200 F600 0240 F600 0280 F600 02C0 F600 0300 F600 0340 F600 0380 to F600 FC00 F700 0000 F700 0040 F700 0080 F700 00C0 F700 0100 F700 0140 F700 0180 F780 0000 F780 0100 DUART chip DUART chip DUART chip DUART chip DUART chip DUART chip DUART chip DUART chip DUART chip DUART chip DUART chip DUART chip DUART chip DUART chip DUART chip DUART chip Watch chip Watch chip Watch chip Watch chip Watch chip Watch chip Watch chip Watch chip Watch chip Watch chip Watch chip CPU module CPU module CPU module CPU module CPU module CPU module CPU module CPU module LEVI 1 UART Base Addresses: xx = 0B; BASE = F400 0000 xx = 0A; BASE = F400 0080 xx = 1B; BASE = F480 0000 xx = 1A; BASE = F480 0080 xx = 2B; BASE = F500 0000 xx = 2A; BASE = F500 0080 Console Overview 5­7 · Interrupt status summaries · Console terminal selection · Halt protection · System reset This section provides descriptions of individual Gbus registers. The re­ maining console registers are listed in Table 5­2 for reference only. All Gbus registers are eight bits wide. 5­8 Console Overview Gbus$WHAMI Table 5­3 Gbus$WHAMI Register Bit Definitions Address Access BB + 0000 0000 RO The Gbus$WHAMI register provides information on system con­ figuration and reflects the status of certain backplane signals. Name Bit(s) Type Function REQ_MODE <7:6> RO Request Mode. Indicates the maximum number of CPU modules that this CPU module supports in a sys­ tem. Gbus$WHAMI <7:6> CPUs Allowed in LSB Slots 00 01 10 11 1 to 6 Reserved Reserved Reserved Console Overview 5­9 Table 5­3 Gbus$WHAMI Register Bit Definitions (Continued) Name Bit(s) Type Function LSB_CONWIN LSB_BAD MFG NID <5> <4> <3> <2:0> RO RO RO RO LSB CONWIN. Reflects the inverted state of the LSB_CONWIN L backplane signal. When set, indi­ cates that Gbus$LEDs<1> is clear (asserted) in one or more CPU modules. LSB Bad. Reflects the inverted state of the LSB_BAD L backplane signal. When set, indicates that LSB_BAD L is driven by one or more CPU mod­ ules. Manufacturing Status. Used by manufacturing. Node ID. Identifies the CPU module by the slot (0­7) where it resides. 5­10 Console Overview Gbus$LEDs Table 5­4 Gbus$LEDs Register Bit Definitions Address Access F700 0000 R/W The Gbus$LEDs register is used for lighting a series of LEDs on the module to aid in debug and to indicate self­test status. Writing a zero to a bit in this register lights the corresponding LED. Name Bit(s) Type Function LEDs_L RUN_L CONWIN_L STP_L <7:3> <2> <1> <0> R/W R/W R/W R/W LEDs Low. When a bit in this field is set, the associ­ ated LED signal is asserted low. RUN Low. When set, the associated LED signal is as­ serted low. The state of this bit also indicates whether the currently running software is the operating system (and not the diagnostic/console program). CONWIN Low. When set, the associated LED signal is asserted low. Also drives the backplane signal LSB_CONWIN L. The state of this signal can be read through the Gbus$WHAMI register. Self­Test Passed Low. When set, the associated LED signal is asserted low. Console Overview 5­11 Gbus$PMask Table 5­5 Gbus$PMask Register Bit Definitions Address Access F700 0040 R/W The Gbus$PMask register controls halts to the processor. Name Bit(s) Type Function RSVD PHALT_EN <7:4> <3> R/W, 1 R/W, 1 Reserved. Initialized to ones. Ctrl/P Halt Enable. When set, enables Ctrl/P characters received by the UART selected in the Se­ lect Console Terminal field of this register to halt the processor. The Halt Enable bit of this register must also be set for a Ctrl/P character to generate a halt. 5­12 Console Overview Table 5­5 Gbus$PMask Register Bit Definitions (Continued) Name Bit(s) Type Function SEL_CONS_TERM HALT_EN <2:1> <0> R/W, 1 R/W, 1 Select Console Terminal. Selects one of three console terminals for Ctrl/P character detection. Halt Enable. When set, enables halts to the processor, including halts generated by LCNR or by detection of a Ctrl/P char­ acter received by a UART selected in the Select Console Terminal field of this register. When clear, all halts to the processor are disabled. PHALT_EN must also be set for Ctrl/P characters to generate a halt. Gbus$PMask <2:1> Console Terminal Selected 00 01 10 11 UART0A (local terminal) UART0B (Reserved) UART1A (remote diagnostic control) UART2A placed into module­ level loopback mode. In this mode, the UART2A receive line is driven by the UART2A transmit line. PHALT_EN (bit <3> of this register) must be zero (Ctrl/P halts disabled) while modifying SEL_CONS_TERM to avoid erroneous halts. Console Overview 5­13 Gbus$Intr Table 5­6 Gbus$Intr Register Bit Definitions Address Access F700 0080 R/W The Gbus$Intr register stores interrupt summary information. Specifically, it provides a means to determine the source of IPL14, IPL15, and IPL16 interrupts to the processor. Name Bit(s) Type Function INTIM IP LSB2 RSVD <7> <6> <5> <4> RO, 0 W1C, 0 RO, 0 R0 Interval Timer. When set, indicates that the watch chip is asserting its interval timer output. Interprocessor. When set, indicates that the LEVI­ A chip has detected a write to the LIPINTR register with data selecting this node. LSB 2. When set, indicates that the LEVI­A chip has an LSB level 2 interrupt pending. Reserved. Reads as zero. 5­14 Console Overview Table 5­6 Gbus$Intr Register Bit Definitions (Continued) Name Bit(s) Type Function LSB1 LSB0 DUART1_INT DUART0_INT <3> <2> <1> <0> RO, 0 RO, 0 RO, 0 RO, 0 LSB 1. When set, indicates that the LEVI­A chip has an LSB level 1 interrupt pending. LSB 0. When set, indicates that the LEVI­A chip has an LSB level 0 interrupt pending. DUART1 Interrupt. When set, indicates that either UART1A or UART1B is requesting an inter­ rupt for the processor. This bit is cleared when all possible DUART1 interrupt sources are cleared. DUART0 Interrupt. When set, indicates that either UART0A or UART0B is requesting an inter­ rupt for the processor. This bit is cleared when all possible DUART0 interrupt sources are cleared. Console Overview 5­15 Gbus$Halt Table 5­7 Gbus$Halt Register Bit Definitions Address Access F700 0100 R/W The Gbus$Halt register summarizes halt and power conditions. Name Bit(s) Type Function RSVD Ctrl/P_HALT PWR_MODB_OK PWR_MODA_OK <7> <6> <5> <4> R0 W1C, 0 RO RO Reserved. Reads as zero. Ctrl/P Halt. Set when a Ctrl/P character is received by the UART selected in the Gbus$PMask register. Power Module B Okay. Set when Power Module B of the I/O PIUs (plug­in unit) is working properly. Cleared when Module B fails. Power Module A Okay. Set when Power Module A of the I/O PIUs (plug­in unit) is working properly. Cleared when Module A fails. 5­16 Console Overview Table 5­7 Gbus$Halt Register Bit Definitions (Continued) Name Bit(s) Type Function LDC_PWR_OK LSB_SEC NHALT RSVD <3> <2> <1> <0> RO RO RO R0 LDC Power Okay. Is set when all local disk con­ verters (LDC) in the platform are working properly. Cleared when no LDCs are installed or when one or more of the LDCs fails. LSB Secure. Reflects the inverted state of the backplane signal LSB_SECURE L. When set, indi­ cates that the control panel keyswitch is in the Se­ cure position and that Ctrl/P halts to the processor are disabled by hardware. Node Halt. Reflects the state of LCNR. Reserved. Reads as zero. Console Overview 5­17 Gbus$LSBRST Address Access F700 0140 R/W The Gbus$LSBRST register is used for initiating a system reset se­ quence. When the CPU chip writes any value to this register, the LSB RESET signal is asserted for 512 LSB cycles. 5­18 Console Overview Gbus$Misc Table 5­8 Gbus$Misc Register Bit Definitions Address Access F700 0180 R/W The Gbus$Misc register controls various system functions. Name Bit(s) Type Function RSVD BAD <7:3> <2> RO, 1 R/W, 1 Reserved. Initialized to ones. Bad. When set, causes the module to drive LSB BAD which, in turn, lights the control panel fault LED. The state of this bit does not affect the Self­Test­Passed LED on the module or the STP bits in the Gbus$LEDs and LCNR registers. This bit allows software to assert LSB BAD on behalf of another sys­ tem component. To determine if any module is driving LSB BAD, software should read Gbus$WHAMI, not Gbus$Misc. Console Overview 5­19 Table 5­8 Gbus$Misc Register Bit Definitions (Continued) Name Bit(s) Type Function EXPSEL <1:0> R/W, 1 Expander Select. Selects which cabinet the power supply UART lines are logically connected to, and therefore, which of three 48V regulators are connected to the power supply lines. Gbus$Misc <1:0> Power Supply Connection 00 01 10 11 PS lines logically connected to main CPU cabinet. PS lines logically connected to right ex­ pander cabinet. PS lines logically connected to left ex­ pander cabinet. PS transmit line is looped back to PS re­ ceive line. 5­20 Console Overview Gbus$RMode Address Access F780 0000 R/W The Gbus$RMode register is a write­only register. A write to it sets LDIAG and logically disconnects the CPU module from the LSB bus. This register is intended for use as a backup system should there be a problem with the LSB interface and writes to the LDIAG register be unsuccessful (writes to the LDIAG register require a successful LSB transaction while writes to Gbus space are completed without any LSB access). Note that software should write to the LDIAG register as a first choice and use the Gbus$RMode register only if the write to the LDIAG register fails. Console Overview 5­21 Gbus$LTagRW Address Access F780 0100 R/W The Gbus$LTagRW register, when used with LTAGA, LTAGW, and LDIAG registers, allows software to read and write the B­cache, B­ map, and P­map tags. See descriptions of the LTAGA, LTAGW, and LDIAG registers in Chapter 7. I/O Operations 6­1 Chapter 6 I/O Operations I/O operations handled by the KA7AA CPU module include I/O reads, I/O writes, and device interrupts. The NVAX+ CPU chip uses four hardcoded SCB vectors for all device interrupts. Interrupt service routines at the four SCB vectors are required to determine the source of the interrupt and invoke the appropriate service routine. From the perspective of I/O operations, registers are divided into two groups: local registers and remote registers. Registers that reside on the KA7AA CPU module and the LSB bus are local registers. Those that re­ side on I/O buses are remote registers. Local registers are directly acces­ sible to software; remote registers are not. Access to remote registers is achieved by means of the mailbox protocol. The LMBOX register is pro­ vided to assist software in the mailbox protocol. 6.1 Mailbox Data Structure Remote control and status registers (CSRs) are accessed through 64­byte naturally aligned mailbox data structures located in main memory. Read requests are posted in mailboxes. Data is returned in memory with status in the following quadword. Mailboxes are allocated and managed by the operating system software. Figure 6­1 shows a mailbox data struc­ ture. 6­2 I/O Operations Figure 6­1 Mailbox Data Structure Table 6­1 describes the mailbox data structure. Refer to the DEC 7000 AXP System/VAX 7000 I/O System Technical Manual for a detailed de­ scription of the mailbox protocol. Table 6­1 Mailbox Data Structure Field Bit(s) Type Quad­ word Function HOSE MASK CMD RBADR WDATA RDATA STATUS ERR DON <63:48> <39:32> <29:0> <63:0> <63:0> <31:0> <63:2> <1> <0> R/W R/W R/W R/W R/W R/W R/W R/W R/W 0 0 0 1 2 4 5 5 5 Hose. Used to determine which remote bus the command is meant for. Mask. Contains the byte mask. The I/O mod­ ule does not use this field. Command. Contains the command. A value of one is a read; a value of seven is a write. Remote Broadcast Address. Contains the ad­ dress to be broadcast on the remote bus. Write Data. Contains the write data to be broadcast on the remote bus. Read Data. Contains read data returned from the remote bus. Status. Contains status information provided by the remote bus. Error. When set, indicates that a mailbox op­ eration failed. Done. Status bit set by the I/O module when a mailbox operation is complete. I/O Operations 6­3 6.2 Mailbox Operation The I/O module services mailbox requests by means of four mailbox pointer CSRs (LMBPR registers; see Section 6.4) located in the I/O module's node space. There is one LMBPR register for each CPU node. Software sees only one LMBPR register address, but the CPU module replaces the least significant two bits of the address (that is, D<2:1>) with the least signifi­ cant two bits of the node ID (that is, NIOD<1:0>). If a given LMBPR regis­ ter is in use when it is written to, the I/O module does not acknowledge it and CNF is not asserted. Processors use the lack of CNF assertion on writes to the LMBPR register to indicate a busy status. The write is retried later under software control. To perform a write to the LMBPR register, microcode must know the ad­ dress of the LMBPR register and the address of the mailbox data structure to be loaded into the LMBPR register. Another memory structure needs to be created to pass this information to microcode. This structure is called the Mailbox Pointer and consists of two longwords. Figure 6­2 shows the mailbox pointer structure. Table 6­2 gives the bit definitions of the mailbox pointer structure. Figure 6­2 Mailbox Pointer Structure Table 6­2 Mailbox Pointer Structure When software has created the mailbox data structure and the mailbox pointer structure, it can start the I/O operation. An MTPR to the LMBOX register (Section 6.4) initiates the I/O operation. Microcode reads the MB_ADDR field out of the mailbox pointer structure and then writes the value to the LMBPR register using the address provided in the mailbox pointer structure. An EDAL store conditional command is used to per­ form the write. Microcode then checks the Zero Condition Code bit (PSL<2>) in the BIU_STAT register to determine if the write passed or failed. If the write passed, PSL<2> is set; otherwise, PSL<2> is cleared. Name Bit(s) Type Function MB_ADDR LMBPR_ADDR <31:6> <31:0> WO WO Mailbox Address. Contains the physical ad­ dress of the mailbox data structure. Since this structure is aligned on a 64­byte boundary, bits <5:0> of the address must be zero. LMBPR Address. Contains the virtual address of the LMBPR register. 6­4 I/O Operations Software can loop on the MTPR to the LMBOX register until the write passes. After the I/O module has accepted the write to LMBPR, it performs the I/O operation. Software can now poll the status bit in the mailbox data struc­ ture until the I/O operation is complete. When the I/O operation has com­ pleted, DON in the mailbox data structure (see Table 6­1) is set. If an er­ ror occurred during the transaction, LBER (see Chapter 7) is also set. If the operation was an I/O write, no further action is required. If the op­ eration was an I/O read, software can now fetch the returned data from the RDATA field in the mailbox data structure. 6.3 Device Interrupt Handling The KA7AA CPU module uses the device interrupts as shown in Table 6­3. Interrupts from the LSB and the UARTs (device interrupts) are han­ dled by both hardware and software. After an interrupt has been posted to the CPU chip through one of the four IRQ lines, the CPU chip passes con­ trol to the operating system through four dedicated SCB entry points. Ta­ ble 6­3 shows the device interrupt sources and their matching SCB entry points. Table 6­3 KA7AA CPU Interrupts For IPL16 and IPL17 interrupts, software reads the Gbus$Intr register to determine if the interrupt is posted by an LSB I/O device, another proces­ sor in the system, or a UART. If an interprocessor or a UART interrupt has been received, software can directly pass control to the appropriate service routine. For LSB I/O interrupts, software must get the device in­ terrupt vector from the I/O module. 6.4 I/O Operation Registers Two registers are used for I/O operations: · Mailbox Pointer CSR (LMBPR) · Mailbox Register (LMBOX) The LMBPR register resides on the IOP module and is described in the DEC 7000 AXP System/VAX 7000 I/O System Technical Manual. The de­ scription of the LMBOX register follows. Interrupt Level (Hex) Interrupt Condition NVAX+ IRQ Pin SCB Vector 17 16 16 15 15 14 LSB level 3 interrupts Interprocessor interrupt LSB level 2 interrupts Console UARTs LSB level 1 interrupts LSB level 0 interrupts 3 2 2 1 1 0 DC D8 D8 D4 D4 D0 I/O Operations 6­5 LMBOX·LSB Mailbox Register Table 6­4 LMBOX Register Bit Definitions Address Access BB + 00 R/W The LMBOX register contains the physical address of the mailbox pointer structure. Name Bit(s) Type Function MBXREG <31:0> WO Mailbox Register. Contains the physical address of the mailbox pointer structure. CPU Module Registers 7­1 Chapter 7 CPU Module Registers The KA7AA CPU module, like the memory and I/O modules on the LSB bus, contains two groups of registers: · LSB required registers · CPU­specific registers LSB required registers are used for internode communication over the LSB bus. CPU­specific registers are used to perform functions specific to the CPU module. 7­2 CPU Module Registers 7.1 Register Mapping All CPU module registers reside in node space. The only exceptions to this rule are the two interrupt registers, LIOINTR and LIPINTR, which reside in LSB broadcast space. CPU module registers are mapped to the node space as offsets to a base address (BB). The base address is implemented in hardware and depends on the node ID, which is determined by the LSB backplane slot occupied by the module. Table 7­1 gives the physical base addresses of nodes on the LSB bus. Table 7­1 LSB Node Space Base Addresses Table 7­2 lists the CPU module registers and gives the address of each reg­ ister as an offset from a selected node space base address. NOTE: Two CPU registers listed in Table 7­2, LIOINTR and LIPINTR, are located in LSB broadcast space, the base address of which is FE00 0000. Node ID Module Physical Base Address (BB) (Byte) 0 1 2 3 4 5 6 7 8 CPU/Memory CPU/Memory CPU/Memory CPU/Memory CPU/Memory CPU/Memory CPU/Memory CPU/Memory I/O F800 0000 F840 0000 F880 0000 F8C0 0000 F900 0000 F940 0000 F980 0000 F9C0 0000 FA00 0000 CPU Module Registers 7­3 Table 7­2 CPU Module Registers Register Name Mnemonic Address (Byte Offset) LSB Required Device Register Bus Error Register Configuration Register Memory Mapping Register 0 Memory Mapping Register 1 Memory Mapping Register 2 Memory Mapping Register 3 Memory Mapping Register 4 Memory Mapping Register 5 Memory Mapping Register 6 Memory Mapping Register 7 Bus Error Syndrome Register 0 Bus Error Syndrome Register 1 Bus Error Syndrome Register 2 Bus Error Syndrome Register 3 Bus Error Command Register 0 Bus Error Command Register 1 I/O Interrupt Register Interprocessor Interrupt Register CPU­Specific Mode Register Module Error Register Lock Address Register LSB Diagnostic Control Register Tag Address Register Tag Write Data Register Console Communication Register 0 Console Communication Register 1 Performance Counter Control Register Performance Counter 0 Register Performance Counter 1 Register Last Miss Address Register LDEV LBER LCNR LMMR0 LMMR1 LMMR2 LMMR3 LMMR4 LMMR5 LMMR6 LMMR7 LBESR0 LBESR1 LBESR2 LBESR3 LBECR0 LBECR1 LIOINTR LIPINTR LMODE LMERR LLOCK LDIAG LTAGA LTAGW LCON0 LCON1 LPERF LCNTR0 LCNTR1 LMISSADDR BB1 + 0000 BB + 0040 BB + 0080 BB + 0200 BB + 0240 BB + 0280 BB + 02C0 BB + 0300 BB + 0340 BB + 0380 BB + 03C0 BB + 0600 BB + 0640 BB + 0680 BB + 06C0 BB + 0700 BB + 0740 BSB2 + 0000 BSB + 0040 BB + 0C00 BB + 0C40 BB + 0C80 BB + 0D00 BB + 0D40 BB + 0D80 BB + 0E00 BB + 0E40 BB + 0F00 BB +0F40 BB + 0F80 BB + 0FC0 1 BB is the node space base address of the CPU module in hex. 2 BSB is the broadcast space base address, which is FE00 0000. 7­4 CPU Module Registers 7.2 Register Descriptions LSB required registers have the following characteristics: · All writes are 32 bits wide. Byte or word operations are not supported. · Writes directed to a read­only register may be accepted and acknowl­ edged, but no action is taken, and the content of the register is not af­ fected. CPU­specific registers appear in the LSB CSR space. CPU Module Registers 7­5 LDEV·Device Register Table 7­3 LDEV Register Bit Definitions Address Access BB + 0000 R/W The LDEV register is loaded during initialization with information that identifies a node. Name Bit(s) Type Function DREV <31:16> R/W, 0 Device Revision. Identifies the revision level of an LSB node. For the KA7AA CPU module, the value of this field is zero. DTYPE <15:0> R/W, 0 Device Type. Identifies the type of node. For the KA7AA CPU module, the value of this field is set to 8002 hex. 7­6 CPU Module Registers LBER·Bus Error Register Address Access BB + 0040 R/W The LBER register stores the error bits that are flagged when an LSB node detects errors in the LSB operating environment and logs the failing commander ID. The status of this register remains locked until software resets the error bit(s). CPU Module Registers 7­7 Table 7­4 LBER Register Bit Definitions Name Bit(s) Type Function RSVD NSES CTCE DTCE DIE SHE CAE NXAE CNFE STE TDE <31:19> <18> <17> <16> <15> <14> <13> <12> <11> <10> <9> R0 RO, 0 W1C, 0 W1C, 0 W1C, 0 W1C, 0 W1C, 0 W1C, 0 W1C, 0 W1C, 0 W1C, 0 Reserved. Read as zero. Node­Specific Error Summary. Set when an error condition is reported in the LMERR register. Control Transmit Check Error. Set when an LSB con­ trol line is driven incorrectly by the CPU module. When CTCE is set, ERR is asserted by the CPU module. Data Transmit Check Error. Set when the CPU mod­ ule detects an error while driving the the D<127> and ECC<27:0> lines during a data or command cycle. When DTCE is set, ERR is asserted by the CPU module. Dirty Error. Set if the CPU module receives an asserted DIRTY signal during a cycle when DIRTY signals are not allowed. When DIE is set, ERR is asserted by the CPU module. Shared Error. Set if the CPU module receives an as­ serted SHARED signal during a cycle when SHARED sig­ nals are not allowed. When SHE is set, ERR is asserted by the CPU module. Command/Address Error. Set if the CPU module re­ ceives an asserted CA signal during a cycle when CA sig­ nals are not allowed. When CAE is set, ERR is asserted by the CPU module. Nonexistent Address Error. Set when the CPU mod­ ule does not receive confirmation for a command it sent on the LSB. When NXAE is set, ERR is asserted by the CPU module. CNF Error. Set if the CPU module receives a confirma­ tion signal during a cycle that does not permit confirma­ tion. When CNFE is set, ERR is asserted by the CPU module. STALL Error. Set when the CPU module receives a STALL signal during a cycle that does not permit stalls. When STE is set, ERR is asserted by the CPU module. Transmitter During Error. Set if a CE, UCE, CPE, or CDPE error occurs during a cycle when the CPU module was driving D<127:0>. When TDE is set, ERR is as­ serted by the CPU module. 7­8 CPU Module Registers Table 7­4 LBER Register Bit Definitions (Continued) Name Bit(s) Type Function CDPE2 CDPE CPE2 CPE CE2 CE UCE2 UCE E <8> <7 <6> <5> <4> <3> <2> <1> <0> W1C, 0 W1C, 0 W1C, 0 W1C, 0 W1C, 0 W1C, 0 W1C, 0 W1C, 0 W1C, 0 Second CSR Data Parity Error. Set when a second parity error occurs while CDPE is set on a CSR data cycle. CSR Data Parity Error. If a parity error occurs during a CSR data cycle, the CPU module sets CDPE, asserts ERR, and locks D<38:0> in the LBERCR registers. Second Command Parity Error. Set when a second parity error occurs on a command cycle while CPE is set. Command Parity Error. If a parity error occurs on a command cycle, the CPU module sets CPE, asserts ERR, and locks D<38:0> in the LBECR registers. Second Correctable Data Error. Set when a second correctable ECC error occurs on a data cycle while CE is set. Correctable Data Error. If the CPU module detects an ECC error on the LSB, it sets CE, asserts ERR, and locks D<38:0> of the command cycle in the LBECR registers. Second Uncorrectable Data Error. Set when the CPU module detects a second uncorrectable data error while UCE is set. Uncorrectable Data Error. If the CPU module detects an uncorrectable ECC error on the LSB during a data cy­ cle, it sets UCE, asserts ERR, and locks D<38:0> of the command cycle in the LBECR registers. Error. Set whenever the CPU module detects assertion of ERR on the LSB. CPU Module Registers 7­9 LCNR·Configuration Register Table 7­5 LCNR Register Bit Definitions Address Access BB + 0080 R/W The LCNR register contains LSB configuration setup and status in­ formation. Name Bit(s) Type Function STF NRST NHALT RSTSTAT MBZ CEEN <31> <30> <29> <28> <27:1> <0> W1C, 1 WO, 0 R/W, 0 W1C, 0 R/W, 0 R/W, 0 Self­Test Fail. When set, indicates that this node has not yet completed self­test. Node Reset. When set, the node enters console mode and undergoes a reset sequence. Node Halt. When set, a CPU node enters console mode. Reset Status. When set, provides an indication to con­ sole software that a given CPU node should not attempt to become the boot processor, but should rather join an already running system. This bit is set when NRST (LCNR<30>) is set. It is cleared with a write of one, at system power­up, or with an LSB RESET command. This bit is not cleared in a reset sequence caused by set­ ting NRST. Must Be Zero. Must always be written as zero. Correctable Error Detection Enable. When set, en­ ables detection of correctable errors. 7­10 CPU Module Registers LMMR07·Memory Mapping Registers Table 7­6 LMMR Register Bit Definitions Address Access BB + 0200 to BB + 03C0 R/W Eight LMMR registers define the memory configuration for all memory modules installed in the system. They are copies of the equivalent AMR registers in memory modules installed in the sys­ tem. Each LMMR register is associated with the LSB module whose node ID matches the three lower bits of the LMMR address. Thus, LMMR0 is associated with node 0, LMMR1 is associated with node 1, and so on. LMMR registers are loaded during system in­ itialization when the memory modules are initialized and config­ ured. Name Bit(s) Type Function MODULE_ADDR RSVD <31:17> <16:11> R/W R0 Module Address. Specifies the most significant bits of the base address of the memory region spanned by the memory module associated with this register (LMMR0LMMR7). These bits cor­ respond to bits <39:25> of the byte address or D<34:20> of the command cycle. Reserved. Read as zero. Writes ignored. CPU Module Registers 7­11 Table 7­6 LMMR Register Bit Definitions (Continued) Name Bit(s) Type Function NBANKS AW IA INT EN <10:9> <8:5> <4:3> <2:1> <0> R/W R/W R/W R/W R/W Number of Banks. Specifies the number of in­ dividual memory banks (1, 2, 4, or 8) contained on the memory module associated with this reg­ ister (LMMR07). The value of this field deter­ mines how many bits of the memory address (0, 1, 2, or 3) are inserted into the bank number. Address Width. Specifies the number of valid bits in MODULE_ADDR (LMMR<31:17>), start­ ing from the MSB. The remaining bits of MOD­ ULE_ADDR are ignored. Interleave Address. Specifies which inter­ leave, within a group of interleaved modules, is served by the module associated with this regis­ ter (LMMR07). Interleave. Specifies the number of memory modules interleaved with this module (1, 2, or 4). This value determines the number of bits in the INT field (0, 1, or 2, starting from the LSB) that are compared to the LSBs of the memory ad­ dress. Enable. When set, indicates that the module associated with this register (LMMR07) is in­ stalled, and it is a memory module. LMMR <10:9> Banks per Module Bits in Bank Number 00 01 10 11 1 2 4 8 0 1 2 3 LMMR <2:1> Modules Interleaved Address Bits Compared 00 01 10 11 1 2 4 Reserved 0 1 2 Reserved 7­12 CPU Module Registers LBESR0­3·Bus Error Syndrome Registers Table 7­7 LBESR Register Bit Definitions Address Access BB + 0600 06C0 RO The LBESR registers contain the syndrome computed from the LSB Data and ECC fields received during the cycle when an error was detected. The syndrome is the bit­by­bit difference between the ECC check code generated from the received data and the ECC field received over the bus. The LBESR registers lock only on the first occurrence of an ECC error (LBER or LBER). Subsequent ECC errors set LBER or LBER until software clears those error bits. Name Bit(s) Type Function RSVD SYND_0 SYND_1 SYND_2 SYND_3 <31:7> <6:0> <6:0> <6:0> <6:0> R0 RO RO RO RO Reserved. Read as zero. Syndrome 0. Syndrome computed from D<31:0> and ECC<6:0> during error cycle. Syndrome 1. Syndrome computed from D<63:32> and ECC<13:7> during error cycle. Syndrome 2. Syndrome computed from D<95:33> and ECC<20:14> during error cycle. Syndrome 3. Syndrome computed from D<127:96> and ECC<27:21> during error cycle. CPU Module Registers 7­13 Syndrome Values A syndrome of zero indicates no ECC error for the given longword. Table 7­8 gives the syndromes for all single­bit errors. Any non­zero syndrome not listed in Table 7­8 indicates a double­bit error. Table 7­8 Syndromes for Single­Bit Errors Bit Syndrome (Hex) Bit Syndrome (Hex) Data<0> Data<1> Data<2> Data<3> Data<4> Data<5> Data<6> Data<7> Data<8> Data<9> Data<10> Data<11> Data<12> Data<13> Data<14> Data<15> Data<16> Data<17> Data<18> Data<19> 4F 4A 52 54 57 58 5D 23 25 26 29 2A 2C 31 34 0E 0B 13 15 Data<20> Data<21> Data<22> Data<23> Data<24> Data<25> Data<26> Data<27> Data<28> Data<29> Data<30> Data<31> ECC<0> ECC<1> ECC<2> ECC<3> ECC<4> ECC<5> ECC<6> 16 19 1A 1C 62 64 67 68 6B 6D 70 75 01 02 04 08 10 20 40 7­14 CPU Module Registers LBECR0,1·Bus Error Command Registers Table 7­9 LBECR Register Bit Definitions Address Access BB + 0700 and BB + 0740 RO The LBECR registers save the contents of the LSB command and address fields during the command cycle when an error is de­ tected. The following errors detected by the CPU module lock the LBECR registers: LSB uncorrectable ECC error (LBER<1>) LSB correctable ECC error (LBER<3>) LSB command parity error (LBER<5>) LSB CSR data parity error (LBER<7>) LSB nonexistent address error (LBER<12>) LSB arbitration drop error (LMERR<10> LEVI P­map parity error (LMERR<3:0>) LEVI B­cache tag parity error (LMERR<4>) LEVI B­cache status parity error (LMERR<5>) LEVI B­map parity error (LMERR<6>) Name Bit(s) Type Function CA RSVD <31:0> <31:20> RO R0 Command/Address. Contents of D<31:0> during the command cycle. Reserved. Read as zero. CPU Module Registers 7­15 Table 7­9 LBECR Register Bit Definitions (Continued) Name Bit(s) Type Function DCYCLE DIRTY SHARED CNF CID CID3 P CMD CA <19:18> <17> <16> <15> <14:11> <10:7> <6> <5:3> <2:0> RO RO RO RO RO RO RO RO RO Data Cycle. Indicates which data cycle had data error. Dirty. Set when DIRTY is asserted for the current com­ mand. Shared. Set when SHARED is asserted for the current command. Confirmation. Set when CNF is asserted for the cur­ rent command. Commander ID. Contents of REQ<3:0> during com­ mand cycle. Commander ID 3. This field is the duplicate of CID (bits <14:11>). It reads the same as CID. In some early versions of the KA7AA module, CID3 reads as zero. Parity. Contents of D<38> during command cycle. Command. Contents of D<37:35> during command cy­ cle. CMD is decoded as follows: Command/Address. Contents of D<34:32> during com­ mand cycle. LBECR <19:18> Data Cycle in Error 00 01 10 11 0 1 2 3 Command Function DAS Value1 000 001 010 011 100 101 110 111 Read Write Reserved Write Victim Read CSR Write CSR Reserved Private 00 08 18 20 28 38 1 The hex value commonly found in the low byte of this register when less than 8 Gbytes of memory are present in the system. 7­16 CPU Module Registers LIOINTR·I/O Interrupt Register Table 7­10 LIOINTR Register Bit Definitions Interrupt Mapping Each interrupt target is assigned four bits of interrupt in the LIOINTR register corresponding to the four VAX I/O interrupt levels. A given CPU only looks at the four bits that correspond to its target assignment. This allows interrupts to be targeted to a single CPU or up to six CPUs, depend­ ing on the data supplied in the bus CSR write transaction from the I/O module. Address Access BSB + 0000 R/W The LIOINTR register is used by the LSB I/O module to signal in­ terrupts from the LSB I/O system to processors. Name Bit(s) Type Function MBZ CPU5 CPU4 CPU3 CPU2 CPU1 CPU0 <31:16> <23:20> <19:16> <15:12> <11:8> <7:4> <3:0> R/W, 0 W1S W1S W1S W1S W1S W1S Must Be Zero. Must always be written as zero. CPU5 I/O Interrupt. When a bit is set in this field, an interrupt is posted to CPU5. CPU4 I/O Interrupt. When a bit is set in this field, an interrupt is posted to CPU3. CPU3 I/O Interrupt. When a bit is set in this field, an interrupt is posted to CPU3. CPU2 I/O Interrupt. When a bit is set in this field, an interrupt is posted to CPU2. CPU1 I/O Interrupt. When a bit is set in this field, an interrupt is posted to CPU1. CPU0 I/O Interrupt. When a bit is set in this field, an interrupt is posted to CPU0. CPU Module Registers 7­17 This register appears in LSB broadcast space. Writes that address this lo­ cation are accepted without regard to node ID. Thus, all CPUs accept writes to the register. The register bits are write one to set (W1S). Multi­ ple writes with a value of one to a given bit in this register post an equal number of interrupts to the targeted CPU. Reads to this location are unde­ fined. Any given CPU implements only four bits of this register. Table 7­11 shows the mapping of LSB interrupt levels to NVAX+ interrupt levels. Table 7­11 LSB Interrupt Mapping When any of the four interrupt­pending bits is set, the LEVI gate array correspondingly asserts the IOINTR<3:0> signals. The CPU module then uses these signals to assert the appropriate interrupt request to the NVAX+ chip. The LEVI­A gate array also watches for LSB CSR reads to the LILID03 registers in the IOP module. When an LSB CSR read for LILID0 is asserted on the LSB bus, the LEVI­A gate array correspondingly deasserts IOINTR<0>. The LEVI­A gate array performs the same function on LILID3, LILID2, and LILID1. LSB Interrupt Level NVAX+ IPL (Dec) 3 2 1 0 IPL 23 IPL 22 IPL 21 IPL 20 7­18 CPU Module Registers LIPINTR·Interprocessor Interrupt Register Table 7­12 LIPINTR Register Bit Definitions Address Access BSB + 0040 WO The LIPINTR register is used by the CPU modules to signal interprocessor interrupts. Name Bit(s) Type Function MBZ MASK <31:16> <15:0> R/W, 0 W1S, 0 Must Be Zero. Must always be written as zero. Interprocessor Interrupt Mask. When a given bit is set, an interprocessor interrupt is posted to a specific processor. Bits are mapped to specific CPUs within a multiprocessor system as follows: LIPINTR Bits NVAX+ CPU <15:6> <5> <4> <3> <2> <1> <0> Not used. CPU5 CPU4 CPU3 CPU2 CPU1 CPU0 CPU Module Registers 7­19 Interprocessor Interrupt When a processor wishes to post an interrupt to another processor, it sim­ ply writes to the LIPINTR register to set the relevant bit. The bits in LIPINTR<3:0> are write one to set (W1S). This register appears in LSB broadcast space. Writes that address this lo­ cation are accepted without regard to node ID. Thus, all CPUs accept writes to the register. Reads to this location are undefined. The contents of LIPINTR<3:0> are qualified by the node ID. If a given CPU node is selected, the LEVI­A gate array asserts the IPINTR signal for one processor external clock. The CPU module ORs this signal and issues the appropriate interrupt request to the NVAX+ chip. 7­20 CPU Module Registers LMODE·Mode Register Table 7­13 LMODE Register Bit Definitions Address Access BB + 0C00 R/W The LMODE register contains mode setup for an operational CPU module (as opposed to the LDIAG register which provides mode setup for a CPU module while running diagnostics). Name Bit(s) Type Function MBZ LEVI_REV MBZ CLR_LOCK <31:17> <16> <15:11> <10> R/W, 0 RO, X R/W, 0 WO, 0 Must Be Zero. Must always be written as zero. LEVI Revision. When clear, indicates pass 1 LEVI­ A. When set, indicates pass 2 LEVI­A. Must Be Zero. Must always be written as zero. Clear Lock. When set, forces LEVI to deassert LSB_LOCKOUT and clear any relevant saved state irrespective of the state of LOCK_TIME, and so on. CPU Module Registers 7­21 Table 7­13 LMODE Register Bit Definitions (Continued) Name Bit(s) Type Function STCOND_TO LOCK_MODE PMODE <9:8> <7:6> <5:4> R/W, 0 R/W, 0 R/W, 0 Store Conditional Timeout. Specifies the timeout period for lockout assertions. This field is only relevant when LOCK_MODE is non­zero. Lock Mode. Specifies how LEVI will drive and respond to LSB_LOCKOUT. P­Cache Mode. Allows LEVI to work with CPU chips with varying internal cache organizations. The value of this field for the NVAX+ CPU chip is 10 (bin), which denotes an 8K P­cache and 2K vir­ tual instruction cache. LMODE <9:8> StoreCond Lockout Time 00 01 10 11 Count 1 STx_C failure before assert­ ing LSB_LOCKOUT Count 8 STx_C failures before as­ serting LSB_LOCKOUT Count 16 STx_C failures before as­ serting LSB_LOCKOUT Count 32 STx_C failures before as­ serting LSB_LOCKOUT LMODE <7:6> LEVI Lockout Behavior 00 01 10 11 LOCKOUT is off All writes prevented with LOCK­ OUT asserted Only write due to STx_C prevented with LOCKOUT asserted LEVI behavior undefined 7­22 CPU Module Registers Table 7­13 LMODE Register Bit Definitions (Continued) Name Bit(s) Type Function WMODE BSIZE <3:2> <1:0> R/W, 0 R/W, 0 Write Mode. Selects the behavior of LEVI in response to LSB writes. B­Cache Size. Tells LEVI about the size of the B­cache. LMODE <3:2> LEVI Behavior 00 01 10 11 Use results of P­map lookup to determine invalidate/update. Invalidate the B­cache. Update the B­cache. LEVI behavior undefined. LMODE <1:0> B­Cache Size 00 01 10 11 4 Mbytes 1 Mbyte 16 Mbytes LEVI behavior undefined. CPU Module Registers 7­23 LMERR·Module Error Register Table 7­14 LMERR Register Bit Definitions Address Access BB + 0C40 R/W The LMERR register provides module­specific error information. If any bits are set in this register, NSES (LBER<18>) is also set. Name Bit(s) Type Function MBZ ARBDROP ARBCOL <31:11> <10> <9> R/W, 0 W1C, 0 W1C, 0 Must Be Zero. Must always be written as zero. Arbitration Drop. Set when the LEVI arbitration logic detects an LSB cycle in which a node has failed to assert a command after having gained ac­ cess to the LSB bus. When ARBDROP is set, the LSB command and address are latched in the LBECR register. Arbitration Collision. Set when the LEVI arbi­ tration logic detects an attempt to arbitrate for the LSB bus in an illegal time slot. 7­24 CPU Module Registers Table 7­14 LMERR Register Bit Definitions (Continued) Name Bit(s) Type Function BDATADBE BDATASBE BMAPPE BSTATPE BTAGPE PMAPPE <8> <7> <6> <5> <4> <3:0> W1C, 0 W1C, 0 W1C, 0 W1C, 0 W1C, 0 W1C, 0 B­Cache Data Double­Bit Error. When set, indicates that the LEVI chips have detected a double­bit ECC error when unloading the B­ cache RAMs. This bit is set when data being transmitted on the LSB bus incurs a double­bit error. The address and the associated ECC syn­ drome are latched in the LBECR and LBESR registers, respectively. B­Cache Data Single­Bit Error. When set, indicates that the LEVI chips have detected a single­bit ECC error when unloading the B­data RAMs. This bit is set when data being trans­ mitted on the LSB bus incurs a single­bit error. The address and the associated ECC syndrome are latched in the LBECR and LBESR registers, respectively. B­Map Parity Error. When set, indicates that the LEVI­A chip has detected bad parity when reading the B­map RAMs. The associated ad­ dress is latched in the LBECR register. B­Cache Status Store Parity Error. When set, indicates that the LEVI­A chip has detected bad parity when reading the B­stat RAM. The associated address is latched in the LBECR reg­ ister for LSB probes. For LEVI probes due to processor misses (LDx_L and so on), BIU_STAT<0> is set and the address is latched in the FILL_ADDR register. B­Cache Tag Store Parity Error. When set, indicates that the LEVI chips have detected bad parity when reading the B­tag RAMs. The asso­ ciated address is latched in the LBECR register for LSB probes. For LEVI probes due to proces­ sor misses (LDx_L and so on), BIU_STAT<0> is set and the address is latched in the FILL_ADDR register. P­Map Parity Error. A bit is set in this field if the LEVI­A chip detects bad parity when read­ ing one of the four internal P­map RAM struc­ tures. The associated address is latched in the LBECR register. CPU Module Registers 7­25 LLOCK·Lock Address Register Table 7­15 LLock Register Bit Definitions Address Access BB + 0C80 RO The LLOCK register contains the physical address and lock bit of the most recently executed LDxL instruction that referenced mem­ ory space. Name Bit(s) Type Function LOCK MBZ LADR MBZ <31> <30:29> <28:1> <0> W1C, 0 R/W, 0 RO, 0 R/W, 0 Lock. When set, indicates that the LLOCK register contains a valid address used in the most recent memory space LDx_L instruction executed by the processor and that no LSB writes that reference the same 64­byte LSB block have occurred. This bit is used to determine the response to a subsequent STx_C instruction. Software can clear this bit ex­ plicitly with an LSB write to the 64­byte block refer­ enced in LLOCK<28:1>. Must Be Zero. Must always be written as zero. Lock Address. Lock address bits <33:6>. Must Be Zero. Must always be written as zero. 7­26 CPU Module Registers LDIAG·LSB Diagnostic Control Register Table 7­16 LDIAG Register Bit Definitions Address Access BB + 0D00 R/W The LDIAG register allows a diagnostic program to manipulate various sections of the CPU module for complete testing. Name Bit(s) Type Function MBZ TAG_SEL <31:11> <10:8> R/W, 0 R/W, 0 Must Be Zero. Must always be written as zero. Tag Select. Specifies which tag store is to be read/written when Gbus$LtagRW is accessed. When LDIAG is being used to read a tag, only one bit in TAG_SEL is allowed to be set. If more than one bit is set in TAG_SEL when Gbus$LtagRW is written, all specified tags are written. LDIAG<10:8> Tag Store Selected 100 010 001 B­cache B­map P­map CPU Module Registers 7­27 Table 7­16 LDIAG Register Bit Definitions (Continued) Name Bit(s) Type Function FRIGN FBDP FBCP FDBE FSBE FSHARE FDIRTY SPARE <7> <6> <5> <4> <3> <2> <1> <0> R/W, 1 R/W, 0 R/W, 0 R/W, 0 R/W, 0 R/W, 0 R/W, 0 R/W, 0 Force LSB Ignore. When set, forces the LEVI gate arrays to ignore all LSB bus traffic except transactions initiated by this node. When FRIGN transitions from set to clear, the LEVI gate array sets LSB ERR to allow all LSB arbitration to resync. Force Bad Data Parity. When set, forces the LEVI­A gate array to assert bad parity on the LSB during CSR data cycles. Force Bad Command Parity. When set, forces the LEVI­A gate array to assert bad par­ ity on the LSB during CSR command cycles. Force Double­Bit Error. Allows a diagnostic program to force the LEVI gate arrays to load data into the B­cache with double­bit ECC er­ rors. When set, LEVI inverts every ECC bit for each longword loaded into the B­cache from the LSB bus. This bit is only relevant when the LEVI gate arrays are loading the B­cache data store (fills). Force Single­Bit Error. Allows a diagnostic program to force the LEVI gate arrays to load data into the B­cache with single­bit ECC er­ rors. When set, LEVI inverts every ECC bit for each longword loaded into the B­cache from the LSB bus. This bit is only relevant when the LEVI gate arrays are loading the B­cache data store (fills). Force Share. When set, the LEVI­A chip re­ sponds to all LSB memory transactions with assertion of SHARED. Force Dirty. When set, the LEVI­A chip re­ sponds to all LSB memory space read transac­ tions with assertion of DIRTY and supplies the data from the B­cache to the LSB. Spare. Has no effect on the CPU module. 7­28 CPU Module Registers Diagnostic Notes The following notes offer additional information for performing diagnostics on the CPU module. · How to Make the B­Cache Emulate Main Memory The CPU module can be made to present its cache as main memory to the LSB environment by setting BIU_CTL, LDIAG, and LMODE=10 (bin). The selection of this mode is possi­ ble under the following two conditions: (1) Only a single CPU module is placed in this mode; (2) No memory module is present in the system. · How to Read/Write Tags The combinations of LDIAG, LTAGA, LTAGW, and Gbus$LtagRW reg­ isters allow diagnostic programs or error recovery programs to read or write any tag store on the CPU module. LDIAG selects the tag store of interest; LTAGA selects the location in the tag store; LTAGW supplies the value to be written into the tag; and Gbus$LtagRW provides the mechanism. The use of the Gbus register allows LEVI to perform the tag access without the need for any special setup (that is, FRIGN). Even though the Gbus registers are specified to be a byte in length, reads from Gbus$LtagRW return a full longword of data, since no physical Gbus location is actually being read. Gbus address space is used for convenience only. Writing Tags 1. Write LDIAG to select the tag store. 2. Write LTAGA to select the location. 3. Write LTAGW to specify the value to be written. 4. Write Gbus$LtagRW with any random data. This action trig­ gers LEVI to perform the tag write as set up. Reading Tags 1. Write LDIAG to select the tag store. 2. Write LTAGA to select the location. 3. Read Gbus$LtagRW. This action triggers LEVI to perform the tag read as set up. The data returned is the value from the se­ lected tag location in the format specified by the LTAGW register. CPU Module Registers 7­29 LTAGA·Tag Address Register Table 7­17 LTAGA Register Bit Definitions Address Access BB + 0D40 R/W The LTAGA register provides a means by which a diagnostic pro­ gram can specify the location to be accessed in the CPU cache data and tag RAM structures. Name Bit(s) Type Function MBZ TAG_ADDR <31:19> <18:0> R/W, 0 R/W, 0 Must Be Zero. Must always be written as zero. Tag Address. Specifies the location (tag address bits <23:5>) to be accessed in the tag store se­ lected by LDIAG. 7­30 CPU Module Registers LTAGW·Tag Write Data Register Table 7­18 LTAGW Register Bit Definitions Address Access BB + 0D80 R/W The LTAGW register provides a means by which a diagnostic pro­ gram can specify the value to be loaded into the CPU caches and tag RAM structures. Name Bit(s) Type Function MBZ DIRTY SHARED VALID PMAPP <31> <30> <29> <28> <27> R/W, 0 R/W, 0 R/W, 0 R/W, 0 R/W, 0 Must Be Zero. Must always be written as zero. Dirty. Loaded into the Dirty field (if any) of the B­stat store specified in LDIAG when WTAG (LDIAG<4>) is set. Shared. Loaded into the Shared field (if any) of the B­ stat store specified in LDIAG when WTAG (LDIAG<4>) is set. Valid. Loaded into the Valid field (if any) of the B­stat store specified in LDIAG when WTAG (LDIAG<4>) is set. P­Map Parity. Specifies the value to be loaded in the B­map parity location when the Gbus$LtagRW register is written. PMAPP covers tag data bits <23:10> and the valid bit (even parity). CPU Module Registers 7­31 Table 7­18 LTAGW Register Bit Definitions (Continued) Name Bit(s) Type Function BMAPP BSTATP BTAGP TAG_DATA <26> <25> <24> <23:0> R/W, 0 R/W, 0 R/W, 0 R/W, 0 B­Map Parity. Specifies the value to be loaded in the B­map parity location when the Gbus$LtagRW register is written. BMAPP covers tag data bits <33:22> and the valid bit (even parity). B­Stat Parity. Specifies the value to be loaded in the B­ stat parity location when the Gbus$LtagRW register is written. BSTATP covers the Shared, Dirty, and Valid bits. B­Tag Parity. Specifies the value to be loaded in the B­ tag parity location when the Gbus$LtagRW register is written. BTAGP covers tag data bits <33:22> (even par­ ity). Tag Data. Loaded into the tag store specified in LDIAG when the Gbus$LtagRW register is written. Mapping is performed as follows: LTAGW<23:0> Tag RAM Structure <23:12> <23:12> <13:0> <33:22> <33:22> <23:10> B­cache tag B­map tag P­map tag 7­32 CPU Module Registers LCON0,1·Console Communication Registers Table 7­19 LCON Register Bit Definitions Address Access BB + 0E00 and BB + 0E40 R/W The LCON register provides a nonmemory communication location for the KA7AA console firmware. The value contained in this regis­ ter has no direct effect on any CPU module hardware. Name Bit(s) Type Function CON_COM_DATA0 CON_COM_DATA1 <31:0> <31:0> R/W, 0 R/W, 0 Console Communication Data 0. Data stored in the LCON0 register. Console Communication Data 1. Data stored in the LCON1 register. CPU Module Registers 7­33 LPERF·Performance Counter Control Register Table 7­20 LPERF Register Bit Definitions Address Access BB + 0F00 R/W The LPERF register defines how the LEVI performance registers (LCNTR0, LCNTR1, and LMISSADDR) behave. Each counter reg­ ister has an event select field, control bits, and an overflow bit. Some of the events to be counted are subject to the node ID mask. The LMISSADDR register loads miss addresses based on the value of the Miss Address Frequency field. Name Bit(s) Type Function N_MASK LC1_HLT LC1_RUN MBZ <31:24> <23> <22> <21> R/W, 0 WO, 0 WO, 0 R/W, 0 Node Mask. When a bit is set in this field, the LSB reads, LSB writes, or victim writes are counted for the associated node. The bits are in one­to­one correspondence, so that bit <31> is associated with node 7, bit <30> with node 6, and so on, except for bit <24>, which is associated with node 0 and the IOP. More than one bit may be set, allowing trans­ actions from multiple nodes to be counted. This field ap­ plies to both LCNTR registers. LCNTR1 Halt. Write one to disable LCNTR1 counting. LCNTR1 Run. Write one to enable LCNTR1 counting. Must Be Zero. Must always be written as zero. 7­34 CPU Module Registers Table 7­20 LPERF Register Bit Definitions (Continued) Name Bit(s) Type Function LC1_SEL LC0_HLT LC0_RUN MBZ <20:16> <15> <14> <13> R/W, 0 WO, 0 WO, 0 R/W, 0 LCNTR1 Select. Selects event for the LCNTR1 register. LCNTR0 Halt. Writing one disables LCNTR0 counting. LCNTR0 Run. Writing one enables LCNTR0 counting. Must Be Zero. Must always be written as zero. LPERF <20:16>1 Event to Count in LCNTR1 00000 00001 00010 00011 00100 00101 00110 00111 01000 01001 01010 01011 01100 01101 01110 01111 10000 10001 10010 10011 10100 Other Misses due to reads Misses due to writes. Misses due to shared blocks LDL_X instructions STC_X failures LSB B­map hits Bank conflict delays ·LSB cycles Arbitration losses Victim buffer hits ·wrapped only CSR reads CSR writes LMBPR writes LSB interrupts Gbus reads Gbus writes LSB reads ·subject to node mask LSB writes ·subject to node mask Victim writes ·subject to node mask Stall cycles Total memory latency Carry out from LCNTR0 Reserved 1 When counting victim buffer hits, LC1_SEL = 01000 counts only hits for which LEVI has to swap (or wrap) the first and second hexwords to satisfy the read request in the proper order (see LVICT register in this section). LC0_SEL = 01000 counts all (wrapped or unwrapped) hits. CPU Module Registers 7­35 Table 7­20 LPERF Register Bit Definitions (Continued) Name Bit(s) Type Function LC0_SEL MBZ <12:8> <7:4> R/W, 0 R/W, 0 LCNTR0 Select. Selects event for the LCNTR0 register. Must Be Zero. Must always be written as zero. LPERF <12:8>1 Event to Count in LCNTR0 00000 00001 00010 00011 00100 00101 00110 00111 01000 01001 01010 01011 01100 01101 01110 01111 10000 10001 10010 10011 10100 Other Misses due to reads Misses due to writes. Misses due to shared blocks LDL_X instructions STC_X failures LSB B­map hits Bank conflict delays ·LSB cycles Arbitration losses Victim buffer hits ·wrapped or unwrapped CSR reads CSR writes LMBPR writes LSB interrupts Gbus reads Gbus writes LSB reads ·subject to node mask LSB writes ·subject to node mask Victim writes ·subject to node mask Stall cycles Total memory latency Carry out from LCNTR1 Reserved 1 When counting victim buffer hits, LC1_SEL = 01000 counts only hits for which LEVI has to swap (or wrap) the first and second hexwords to satisfy the read request in the proper order (see LVICT register in this section). LC0_SEL = 01000 counts all (wrapped or unwrapped) hits. 7­36 CPU Module Registers Table 7­20 LPERF Register Bit Definitions (Continued) Name Bit(s) Type Function MA_FREQ LC1_OVFL LC0_OVFL <3:2> <1> <0> R/W, 0 RO, 0 RO, 0 Miss Address Frequency. Determines how often the LMISSADDR register loads the last miss address. LCNTR1 Overflow. Records overflow from the LCNTR1 register. Clears when the LCNTR1 register is reset. LCNTR0 Overflow. Records overflow from the LCNTR0 register. Clears when the LCNTR0 register is reset. LPERF <3:2> Miss Frequency 00 01 10 11 Load every 32nd miss address Load every 64th miss address Load every 128th miss address Load every 256th miss address CPU Module Registers 7­37 LCNTR0,1·Performance Counter Registers Table 7­21 LCNTR Register Bit Definitions Address Access BB + 0F40 and BB + 0F80 R/W The LCNTR registers count events selected by the Select (LC0_SEL and LC1_SEL) and N_MASK fields of the LPERF register. Each register is a 32­bit counter with an associated overflow bit (LPERF<0> for LCNTR0 and LPERF<1> for LCNTR1). With the correct values of the Select fields, the LCNTR registers can be cas­ caded to form a single 64­bit register. The two counters are en­ abled and disabled independently through their associated LPERF control bits. The LCNTR registers can be read while the counters are stopped or running. The registers can also be stopped and re­ started without resetting to zero. A write to an LCNTR register re­ sets the counter and the associated overflow bit in the LPERF reg­ ister, and sets the counter to the stopped state. Writes to these reg­ isters are ignored. Name Bit(s) Type Function EV_COUNT0 EV_COUNT1 <31:0> <31:0> R/W, 0 R/W, 0 Event Count 0. Number of events (selected through LPERF) that occurred while LCNTR0 was enabled. Writes ignored. Event Count 1. Number of events (selected through LPERF) that occurred while LCNTR1 was enabled. Writes ignored. 7­38 CPU Module Registers LMISSADDR·Last Miss Address Register Table 7­22 LMISSADDR Register Bit Definitions Address Access BB + 0FC0 RO The LMISSADDR register captures every nth B­cache miss address (determined by LPERF. The miss may be due to a read, a write, or a shared block. Name Bit(s) Type Function RSVD MISS_ADDR <31:29> <28:0> R0 R/W, 0 Reserved. Read as zero. Missed Address. Block address of one of the last n B­cache misses. LPERF determines the value of n. Initialization 8­1 Chapter 8 Initialization The KA7AA CPU module can be initialized in three ways: · Power­Up Sequence. When the LSB system is powered up, the CPU module generates a local reset signal. · System Reset. Whenever the LSB RESET signal is asserted, the CPU module is initialized. LSB RESET can be asserted by any node or from the control panel keyswitch through CCL_RESET. · Node Reset. A single CPU module can be reset by setting LCNR. The processor chip (NVAX+) can be reset independently of the other com­ ponents on the CPU module through the serial I/O port (see Section 5.1.2). 8.1 Initialization Overview A CPU reset causes the console code to first invoke the on­board self­test sequence. Self­test begins by testing a very small portion of the CPU logic and gradually expands the scope of testing until all hardware func­ tions of the module have been verified. Action subsequent to the completion of the CPU module self­test depends on the state of LCNR. If the state of this bit indicates a power­up or system reset, CPU module self­test is followed by CPU­ based testing of other system components. Once all appropriate testing has been completed, the KA7AA console pro­ gram determines a primary processor. The primary processor is then re­ sponsible for displaying the results of all testing, configuring the LSB sys­ tem (memory, registers, and so on), and creating the software data structures necessary to communicate between processors and the operat­ ing system. The console program then enters its input loop. If the CPU module reset was caused by a node reset, no additional system components are tested. CPU registers are set to their firmware­ initialized state, but no other LSB system configuration is performed. There is no change in the primary processor, and the console program en­ ters its input loop. 8­2 Initialization 8.2 Self­Test CPU self­test is a layered process that is called as the first part of the con­ sole entry sequence. It starts with a simple load of code from the SROM into the P­cache and augments itself through additional ROM­resident code that is copied to the B­cache and runs from the B­cache. The process completes by returning a GO/NOGO status to the console entry sequence. The following subsections summarize the various stages in the CPU self­ test. A complete description and flowchart of self­test sequences can be found in the Advanced Troubleshooting manuals. 8.2.1 SROM Operation Following the deassertion of reset to the NVAX+ chip, the contents of the SROM are loaded into the internal cache (see Section 5.1.2 the PC is pointed to location zero and instruction execution is started. This code per­ forms the following: · A quick internal test of the processor chip · Tests the external B­cache tag, status, and data store RAMs and asso­ ciated control · Determines that access to the Gbus resources is operational · Copies the balance of CPU self­test and the CPU console program from the Gbus ROMs to the B­cache and, following a checksum verification, transfers control to it. At any point in this process, the SROM code can signal failures through the Gbus$LEDs register. 8.2.2 CPU Module Self­Test Following the transfer of control from the SROM code to the main body of CPU self­test in the B­cache, the CPU module is tested thoroughly. High­ lights include: · Test of all Gbus resources including the UARTs and the watch chip · LEVI tests including LSB transfers · Tests of all RAM structures 8.2.3 Additional Power­Up Testing If the CPU module self­test completes successfully, additional power­up testing is next performed to verify untested system components. Addi­ tional testing performed by all processors includes: · Tests of the processor/memory LSB interface · Tests of CPU multiprocessor logic · Tests of the LSB I/O port module The boot processor then performs tests on interfaces to the I/O port. Initialization 8­3 8.3 Console Entry When the power­up test sequence is complete, the console code entry se­ quence continues. This section briefly describes the system­level initializa­ tion functions that are required to start the operating system, performed by the console code following self­test. 8.3.1 Boot Processor Arbitration The console program determines the boot processor. A boot processor must also be determined on an interim basis, between phases of power­up test sequence, for the purpose of printing out test results. The boot processor is selected dynamically. Any processor in a multi­ processor system can become the boot processor. By default, the CPU with the lowest LSB node number that has passed all of its power­up tests thus far, and is eligible, is selected as the boot processor. If all processors fail self­test, or if all processors have been disabled through console commands from becoming boot processors,  then no boot processor is assigned. In this case, a unique code is placed on the LEDs, and all processors monitor the console terminal lines waiting for a se­ quence to be typed by the operator, which would force one of the processors to become the boot processor. The console set cpu command can be used to change the boot processor once the console is running. Refer to the Console Reference Manual for fur­ ther information on the set cpu command. 8.3.2 Boot Processor System Setup Following configuration of memory, the console creates data structures in memory that are required to communicate between processors and with the operating system. These data structures include: · Hardware restart parameter block (HWRPB) · A physical memory descriptor · A bitmap of good and bad pages of physical memory · Console routines block (CRB) · Console terminal block (CTB) 8.3.3 Operating System Startup The KA7AA console program's primary role in operating system startup is to load and transfer control to the primary bootstrap program. The method the console generally uses to load the primary bootstrap program is called bootblock booting. To begin the boot, the console reads the first logical block (LBN 0) on a disk. This is the bootblock, which contains infor­ mation that points to the location of the primary bootstrap program on the disk. Using the same routines that loaded the bootblock, the console then uses this information to load the primary bootstrap program. When booting from a network, the console must request the bootstrap im­ age from an external server. 8­4 Initialization Once control is passed to the primary bootstrap, the console program's only remaining role is to allow access to console terminal routines and I/O rou­ tines. The console terminal routines allow the operating system software to send/receive characters to/from the console terminal. The I/O routines allow the primary bootstrap program to utilize the console boot drivers to load the secondary bootstrap or operating system software. Error Handling 9­1 Chapter 9 Error Handling Errors detected by or reported to the KA7AA processor can occur anywhere in the system. If errors occur during data movement within the NVAX+ chip or between the NVAX+ and its environment, they are detected by the NVAX+ chip. Errors that occur during data movement within KA7AA hardware external to the NVAX+ (B­cache and LEVI) or during interac­ tions with the LSB bus (other nodes) are detected by the LEVI interface and reported to the NVAX+ chip. This chapter does not offer an exhaustive discussion of error handling on the KA7AA CPU module. Although it provides detailed parse trees for er­ ror isolation, it limits specific error discussions to the necessary. The parse trees and accompanying texts should provide enough information for the experienced system programmer to determine the source of an error on the CPU module and apply a recommended recovery method. Topics discussed in this chapter include: · Software Error Handling · Error Reports · Console Halt and Halt Interrupt · Machine Checks · Hard Error Interrupts · Soft Error Interrupts · Kernel Stack Not Valid Exception · System Environment Errors 9.1 Software Error Handling Software error handling (by operating system routines) can be logically di­ vided into the following steps: · State collection · Analysis · Recovery · Retry The following terminology is used in subsequent discussions: 9­2 Error Handling Fill · Any quadword or data returned to the NVAX+ chip in response to read­type operations. The quadword containing the requested data is a fill. Flush · Causes victim writebacks to memory of all dirty blocks in the B­ cache. 9.1.1 Error State Collection All relevant state must be collected before error analysis can begin. The stack frame provides the PC/PSL pair for all exceptions and interrupts. For machine checks, the stack frame also provides details about the error. Besides the stack frame, machine checks and hard and soft error inter­ rupts usually require analysis of other registers. The state of some regis­ ters should be saved prior to analysis so that analysis is not complicated by changes in state in the registers as the analysis progresses. Errors in­ curred during analysis and recovery can be processed within that context. The state of the following registers should be read and saved: Ibox ICSR: Ibox Control and Status Register VMAR: VIC Memory Address Register Ebox ECR: Ebox Control Register Mbox TBSTS: TB Parity Status Register TBADR: TB Parity Address Register PCSTS: P­Cache Status Register PCADR: P­Cache Parity Address Register Cbox BIU_STAT: Bus or Fill error status BC_TAG: Contains tag of tag parity, control parity, or fill error BIU_ADDR: Address associated with cache probe or bus error (BIU_HERR, BIU_SERR, BC_TPERR, BC_TCPERR) FILL_ADDR: Address associated with fill error, FILL_ECC, or FILL_DPERR FILL_SYNDROME: Syndrome bits associated with FILL_ADDR NOTE: In discussions and examples in this chapter, it is assumed that each of these register states is saved in a variable whose name is constructed by prepending S_ to the register name. For example, the ICSR register would be saved in the variable S_ICSR. Example 9­1 shows allocation of memory storage for the error state. Error Handling 9­3 Example 9­1 Memory Storage Allocation to the Error State ;ERROR STATE COLLECTION DATA STORAGE ;IBOX S_ICSR .LONG 0 ;IBOX VIC CONTROL AND STATUS REGISTER S_VMAR .LONG 0 ;IBOX VIC ERROR ADDRESS REGISTER ;EBOX S_ESR .LONG 0 ;EBOX CONTROL AND STATUS REGISTER ;MBOX S_TBSTS .LONG 0 ;TB STATUS REGISTER S_TBADR .LONG 0 ;TB ADDRESS REGISTER S_PCSTS .LONG 0 ;P­CACHE STATUS REGISTER S_PCADR .LONG 0 ;P­CACHE ERROR ADDRESS REGISTER ;CBOX S_BIU_STAT .LONG 0 ;Bus or fill error status S_BCTAG .LONG 0 ;Contains tag of tag_parity, control_parity, ;or fill error S_BIU_ADDR .LONG 0 ;Address associated with BIU_HERR, BIU_SERR, ;BC_TPERR, BC_TCPERR S_FILL_ADDR .LONG 0 ;Address associated with fill error, FILL_ECC, ;or FILL_DPERR S_FILL_SYNDROME .LONG 0 ;Syndrome bits associated with FILL_ADDR Example 9­2 shows collection error state that would normally be done in the error handling routine. If a second bus or fill error is detected, the SEO second error bit is set, but the error address and status are lost. Example 9­2 Collection Error State SAVE_STATE: ;Save all error state upon entry ;to error routine ;CBOX MFPR #PR19S_BIU_STAT,S_BIU_STAT MFPR #PR19S_BIU_ADDR,S_BIU_ADDR MFPR #PR19S_FILL_ADDR,S_FILL_ADDR MFPR #PR19S_FILL_SYNDROME,S_FILL_SYNDROME MFPR #PR19S_BC_TAG,S_BC_TAG ;IBOX MFPR #PR19S_ICSR,S_ICSR MFPR #PR19S_VMAR,S_VMAR ;EBOX MFPR #PR19S_ECR,S_ECR ;MBOX MFPR #PR19S_TBSTS,S_TBSTS MFPR #PR19S_TBADR,S_TBADR MFPR #PR19S_PCSTS,S_PCSTS MFPR #PR19S_PCADR,S_PCADR ;System environment ;Collection of system environment error registers goes here 9­4 Error Handling Additional state collection is recommended while/after flushing the B­ cache because certain errors can occur as a result of the flush operation. 9.1.2 Error Analysis The error condition is analyzed with the error state obtained during the collection process. The purpose is to determine, if possible, what error event caused the error notification, and what other errors may also have occurred. Analysis of machine checks and hard and soft error interrupts should be guided by the parse trees given in the appropriate sections. Errors detected in or by one of the caches usually result in the cache being automatically disabled. To minimize the possibility of nested errors, analysis and recovery for memory or cache­related errors should be per­ formed with both the P­cache and B­cache disabled (BIU_CTL<0>=0). In some cases a notification for a single error occurs in two ways. For ex­ ample, an uncorrectable error in the B­cache data RAMs will cause a soft error interrupt and may also cause a machine check. Software needs to handle cases where a machine check handler clears error bits and then the soft error handler is entered with no error bits set. In general, an error reporting register can report events that lead to ma­ chine checks, soft errors, or hard errors. A given error can result in either a machine check or a soft error interrupt, or both. Events that lead to hard error interrupts generally cannot also cause a machine check or soft error interrupt. However, if a hard error occurs from a write operation, a subse­ quent read error can result in a machine check with SEO set. Each category of error·machine check exception, soft error interrupt, and hard error interrupt·is analyzed by tracing an error cause determination parse tree. The IPLs assigned to these errors establish a hierarchy going in descending order from the machine check exception to the soft error in­ terrupt. This hierarchy is important because knowledge of which notifica­ tion event occurred is used to distinguish between certain error events. For example, an error on the initial fill quadword for a read­lock is distin­ guished from a fill error on a subsequent quadword by a machine check no­ tification issued on the former error. 9.1.3 Error Recovery Error recovery consists of clearing any latched error state and restoring the system to normal operation. Analysis and recovery from cache and memory errors require special care and are discussed separately. Multiple simultaneous errors may make useful recovery impossible. How­ ever, in cases where no conflict exists in the reporting of multiple errors, and recovery from each error is possible, then recovery from the set of er­ rors is accomplished by recovering from all. For example, recovery from a P­cache tag parity error and a B­cache correctable data error which are re­ ported together, is possible by following the recovery procedures for each error in sequence. All recovery procedures in this section assume that only one error is pre­ sent. None of the procedures are valid in multiple error cases without fur­ ther analysis. In some instances, it may be desirable to stop using the hardware that is the source of a large number of errors. For example, if a cache reports an Error Handling 9­5 excessive number of errors, it may be preferable to bypass (disable) it. Software should maintain error counts that can be compared against error thresholds on every error report. If the count (per unit time) exceeds the threshold, the hardware should be disabled. Cache and memory error recovery requires special considerations: · Cache and memory error recovery should always be done with the P­ cache and VIC off. · B­cache flush should always be done one block at a time, recapturing the relevant error registers after each block flush. · Cache coherence requires a specific procedure for reenabling the caches. · Error recovery should be performed starting with the most distant component and working toward the CPU and Ebox. This rule requires that system environment memory errors be processed first, then B­tag and B­data, P­cache, TB, and, finally, VIC errors. · BIU and FILL errors are cleared by writing the W1C (write 1 to clear) bits in the BIU_STAT register. · P­cache tag and data errors are cleared by writing the W1C bits in the PCSTS register. The suggested way to do this is to write a one to the specific error bit. P­cache flush is necessary after P­tag parity errors. · Translation buffer errors are cleared by writing the W1C bits in the TBSTS register. The suggested way to do this is to write a one to the specific error bit. · PTE read errors are cleared by writing the W1C bits in the PCSTS reg­ ister. The suggested way to do this is to write a one to the specific er­ ror bit. · VIC errors are cleared by writing the W1C bits in the ICSR register. The suggested way to do this is to write a one to the specific error bit. VIC flush and reenable is necessary after VIC tag store parity errors. 9.1.3.1 Cache Coherence in Error Handling Certain procedures must be followed to maintain cache coherence while enabling NVAX+ caches. Since many errors cause caches to be disabled, and since cache and memory error recovery is normally done with the P­ cache and VIC off, the complete cache enable procedure is done as part of recovery from all cache and memory errors. The VIC is not automatically kept coherent with memory. It is flushed as a side effect of the REI instruction (as required by the VAX architecture). Normally in error recovery, there is no need to flush the VIC. For consis­ tency and for the sake of beginning error retry in a known state, flushing the VIC during error recovery is recommended. However, in the event of VIC tag parity errors, the complete VIC flush procedure must be done. The translation buffer is not automatically kept coherent with memory. Software uses the TBIS and TBIA functions to maintain coherence, and the LDPCTX instruction clears the process PTEs in the TB. Normally in error recovery, there is no need to flush the TB. For consistency and for the sake of beginning error retry in a known state, flushing the TB during error recovery is recommended. When a TB parity error occurs, The 9­6 Error Handling Mbox hardware flushes the TB by itself (through an internally generated TBIA), but it would be appropriate for software to test the TB after a par­ ity error. 9.1.3.2 Cache Enable, Disable, and Flush Procedures Before NVAX+ caches are enabled, they must be flushed. The caches then must be enabled in a specific order. This is necessary for coherence be­ tween the B­cache, P­cache, and memory. For simplicity, one procedure is given here for enabling the NVAX+ caches, even though variations on the procedure could also produce correct results. NOTE: In error handling, the VIC and the P­cache are disabled. To disable NVAX+ caches for error handling, proceed as follows: 1. Disable the VIC (MTPR to ICSR) 2. Disable the P­cache (MTPR to PCCTL) 3. Disable the B­cache (MTPR to BIU_CTL) The procedure to enable the NVAX+ caches after an error is the same as the one used to initialize the caches after power­up. This procedure en­ sures that error retry/restart occurs with the caches in a known state. Pro­ ceed as follows: 1. Disable all NVAX+ caches and the B­cache. 2. Flush the B­cache. 3. Enable the B­cache (MTPR to BIU_CTL). 4. Flush the P­cache (Loop on MTPR to PCTAG IPRs). 5. Enable the P­cache (MTPR to PCCTL). 6. Flush the TB. MTPR #0, #PR19S_TBIA 7. Flush the VIC (loop on MTPRs to VMAR and VTAG, writing different initial values into the left and right banks). 8. Enable the VIC (MTPR to ICSR). 9.1.3.3 Extracting Data from the B­Cache To extract data from the B­cache, place the B­cache in force hit mode with an MTPR to BIU_CTL. After the B­cache is flushed, set the B­cache in force hit mode and extract the data. Note that the code that executes this procedure and its local data must be in I/O space. The TB entries (PTEs) that map this code and local data must be fixed in the TB. This is most easily done by flushing the TB through an MTPR to TBIA and then accessing all the relevant pages in se­ quence. With the B­cache in force hit mode, a read in memory space of any address whose index portion matches the index of the cache data will return the data (provided there is no uncorrectable data RAM error). This is most easily accomplished by reading from the true address of the data. Error Handling 9­7 NOTE: In force hit mode, fill ECC errors are detected. Software should prepare for an ECC error (BIU_STAT). 9.1.3.4 Cache and TB Test Procedures Testing is generally done using the force hit mode of a cache. The code and data of the test procedure must reside in I/O space. Assuming memory management is enabled during the test procedure, the needed PTEs must be in the TB before entering force hit mode in the P­cache or B­cache. For the B­cache, testing should be done with errors disabled (DIAG_CTL=1). The ECC logic should be tested thoroughly on one location by forcing various check bit patterns and exam­ ining the syndrome latched on the read (FILL_SYNDROME is loaded on every read in B­cache disable errors mode). Presently FILL_SYNDROME is valid if an error occurs and the syndrome bits for the last fill cannot be recovered with an IPR_RD of this register otherwise. P­cache and VIC parity checking should be tested by writing bad parity into the arrays. TB testing can be accomplished by writing to MTBTAG and MTBPTE (being careful not to change any TB entry necessary for the test code and data, and not to cause two TB entries to exist for one address). Probe read and probe write (setting PSL) are then used to verify the protec­ tion bits. Testing the Modify bit in the memory management stack frame would be difficult, though approaches exist. 9.1.4 Error Retry Error retry is a function of the error notification (machine check or error interrupt), error type, and error state. The sections below specify the con­ ditions under which the instruction stream may be restarted. If retry is to be attempted, the stack must be trimmed of all parameters except the PC/PSL pair. An REI will then restart the instruction stream and retry the error. Some form of software loop control should be provided to limit the possibility of an error loop. Note that pending error interrupts may be taken before the retry occurs, depending on the IPL of the inter­ rupted or machine checked code. Strictly speaking, an REI from a hard or soft error interrupt handler is not a retry since these interrupts are recognized between macroinstructions. A machine check exception is an instruction abort, and an REI from the handler will cause the failing instruction to be retried. What these cases all have in common is that the interrupted instruction stream is restarted. This is only done when the result of error analysis and recovery is such that all damaged state has been repaired and there is no reason to suspect that incorrect results will be produced if the image is restarted and an­ other error does not occur. If complete recovery from one or more errors is not possible, software must determine if the error is fatal to the current process, to the processor, or to the entire system, and take the appropriate action. It is expected that software handles machine checks, soft error interrupts, and hard error interrupts independently. For example, after handling a machine check from which retry is to occur, software does not check for er­ rors that might cause a pending hard or soft error interrupt. NOTE: In the case of HARD_ERROR interrupt, the machine check code must not clear the BIU_STAT register if the interrupt is to be taken. 9­8 Error Handling The machine check handler is exited by REI (after trimming the machine check information off the stack). If the IPL of the machine checked in­ struction stream is low enough, any pending hard or soft error interrupt is taken before the retry occurs. However, if the interrupted instruction stream was running at a high IPL, the system will continue without deal­ ing with the remaining errors. Multiple errors can be reported at the same time. In some cases the NVAX+ pipeline will contain multiple operand prefetches to the same memory block. This can cause multiple errors from a single nontransient failure. Two separate errors could also occur at nearly the same time and be reported simultaneously. Multiple error scenarios can be grouped into the following classes: · Class 1 errors are multiple distinct errors for which no error report in­ terferes with the analysis of any other (that is, no lost error bit is set). · Class 2 errors are multiple errors that could have been caused by the NVAX+ pipeline issuing more than one reference to a given block be­ fore the error interrupt or the machine check forced a pipeline flush. · Class 3 errors are multiple errors for which analysis is complicated be­ cause the reports interfere with each other. This chapter treats class 1 errors as separate errors, each with its own re­ covery. Retry or restart evaluation is based on the cumulative result of the recovery and repair procedures for each error. The chapter identifies and deals with specific cases of class 2 errors in which lost errors are tolerated. These cases are selected because the NVAX+ pipeline can easily cause them (given one error), and because suffi­ cient safeguards exist to ensure that correct operation is maintained. Class 3 errors are generally not considered recoverable and the system is crashed. Lost correctable errors are not considered serious problems, since hard­ ware recovers from these errors automatically. Error Handling 9­9 9.2 Error Reports The KA7AA processor reports errors to the operating system software through machine checks and interrupts. Error notification occurs through one of the following events listed in order of decreasing severity: · Console error halt A halt to console mode is caused by one of several errors such as inter­ rupt stack not valid. For certain halt conditions, the console prompts for a command and waits for operator input. For other halt conditions, the console may attempt a system restart or system bootstrap. · Machine check A hardware error occurred synchronous to the execution of instruc­ tions. Instruction­level recovery and retry may be possible. · Kernel stack not valid During exception processing, a memory management exception oc­ curred while trying to push information on the kernel stack. · Hard error interrupts A hardware error occurred asynchronous to the execution of instruc­ tions. Usually, data is lost or state is corrupted, and instruction­level recovery may not be possible. · Soft error interrupts A hardware error occurred asynchronous to the execution of instruc­ tions. The error is not fatal to the execution of instructions, and instruction­level recovery is usually possible. All errors (except those leading to a console halt) go through SCB (system control block) vector entry points and are handled by service routines pro­ vided by the operating system. A console halt, on the other hand, transfers control to a hardware­prescribed I/O space address. Software­driven re­ covery or retry is not recommended for errors resulting in console halt. Table 9­1 gives a summary of errors and notification entry points for the various error categories. For each SCB entry point, discussions provide the following information: · Parameters pushed on the stack · Defined failure codes · What additional information exists and should be collected for analysis · How to restore the state of the machine and what level of recovery is possible Error categories are discussed in the next sections. 9­10 Error Handling Table 9­1 Error Categories by SCB Entry Points 9.3 Console Halt and Halt Interrupt A console halt is a transfer of control by the NVAX+ CPU microcode di­ rectly into console macrocode at the address stored in the CHALT register. Console halts are initiated: · At power­up · When the microcode detects certain double­error conditions, specifi­ cally when a second error occurs during error processing · When LBER is set · When Ctrl/P is typed on the console terminal · When the system is reset · When a kernel­mode HALT instruction is executed · When the external signal HALT_H is asserted No exception stack frame is associated with a console halt. Instead, the state is saved in the SAVPC (IPR42) and SAVPSL (IPR43) processor regis­ ters. Figure 9­1 shows the SAVPC IPR. SCB Index (Hex) Error Category Summary of Errors None 04 08 54 60 Console halt1 Machine check Kernel stack not valid Soft error interrupt Hard error interrupt Interrupt stack not valid Kernel mode halt Double error halt Illegal SCB vector Memory management Interrupt Microcode­detected CPU errors CPU stall timeout TB parity errors VIC tag or data parity errors Uncorrectable data read errors CACK_H error on reads Error during exception processing VIC tag or data parity errors P­cache tag or data parity errors B­cache tag parity error on reads Uncorrectable data read errors Correctable data read errors Uncorrectable data errors on write operations B­cache tag parity error on writes CACK_H error on writes 1 Does not go through an SCB entry point. Error Handling 9­11 Figure 9­1 Console Saved PC The PSL halt code, MAPEN<0>, and a validity bit are saved in SAVPSL. Figure 9­2 shows the SAVPSL IPR. Figure 9­2 Console Saved PSL Table 9­2 lists the possible halt codes that can appear in SAVPSL<13:8>. 9­12 Error Handling Table 9­2 Console Halt Codes At the time of the halt, the current stack pointer is saved in the appropri­ ate IPR (0 to 4), and SAVPSL<31:16,7:0> are loaded from PSL<31:16,7:0>. SAVPSL<15> is set to MAPEN<0>. SAVPSL<14> is clear if the PSL is valid, and set if it is not (SAVPSL<14> is undefined after a halt due to a system reset). SAVPSL<13:8> is set to the console halt code. To complete the hardware restart sequence and thereby pass control to the console macrocode, the CPU is initialized as shown in Table 9­3. Code (Hex) Mnemonic Description: 02 03 04 05 06 07 08 0A 10 11 12 13 19 1A 1B 1D 1E 1F 3F ERR_HLTPIN ERR_PWRUP ERR_INTSTK ERR_DOUBLE ERR_HLTINS ERR_ILLVEC ERR_WCSVEC ERR_CHMFI ERR_IE0 ERR_IE1 ERR_IE2 ERR_IE3 ERR_IE_PSL_26_24_101 ERR_IE_PSL_26_24_110 ERR_IE_PSL_26_24_111 ERR_REI_PSL_26_24_101 ERR_REI_PSL_26_24_110 ERR_REI_PSL_26_24_111 ERR_SELFTEST_FAILED HALT_H asserted Initial power­up Interrupt stack not valid Machine check during exception processing HALT instruction in kernel mode Illegal SCB vector (bits <1:0> = 11) WCS SCB vector (bits <1:0> = 10) CHMx on interrupt stack ACV/TNV during machine check processing ACV/TNV during kernel­stack­not­valid processing Machine check during machine check processing Machine check during kernel­stack­not­valid processing PSL<26:24> = 101 during interrupt or exception PSL<26:24> = 110 during interrupt or exception PSL<26:24> = 111 during interrupt or exception PSL<26:24> = 101 during REI PSL<26:24> = 110 during REI PSL<26:24> = 111 during REI Microcoded power­up self­test failed Error Handling 9­13 Table 9­3 CPU State Initialized on Console Halt 9.4 Machine Checks A machine check exception indicates a serious system error. Under certain conditions, the error may be recoverable by restarting the instruction. The recoverability is a function of the following parameters: · Machine check code · VAX Restart bit (VR) in the machine check stack frame · Opcode · State of PSL · State of certain second error bits in internal error registers · External error state A machine check results from an internally detected consistency error (for instance, the microcode reaches an impossible state), or hardware­ detected error (for instance, an uncorrectable FILL_ECC error on a data read). Technically, a machine check is an aborted macro instruction. The NVAX+ microcode attempts to convert the condition to a fault by unwinding the current instruction, with no guarantee that the instruction can be properly restarted. As much information as possible is pushed on the machine check stack frame and provided in other error registers. The rest of the error parsing is left to the operating system. When the software machine check handler routine receives control, it must explicitly acknowledge receipt of the machine check early in the routine to clear the internal machine­check­in­progress flag with the following in­ struction: MTPR #0, #PR19S_MCESR CPU State Initialized Value SP PSL PC MAPEN ICCS SISR ASTLVL PAMODE BPCR<31:16> CPUID All else IPR 4 (IS) 041F 0000 from CHALT register 0 0 (after reset, code = 3, only) 0 (after reset, code = 3, only) 4 (after reset, code = 3, only) 0 (after reset, code = 3, only) FECA (after reset, code = 3, only) 0 (after reset, code = 3, only) Undefined 9­14 Error Handling Figure 9­3 shows the machine check stack frame. Figure 9­3 Machine Check Exception Stack Frame Table 9­4 describes the contents of the fields in the machine check stack frame. Fields not explicitly defined in Figure 9­3 are UNDEFINED. Error Handling 9­15 Table 9­4 Machine Check Stack Frame Fields Table 9­5 describes the machine check codes. Longword Bits Contents SP+0 <31:0> SP+4 <31:29> <23:16> <7:0> SP+8 <31:0> SP+12 <31:0> SP+16 <31:0> SP+20 <31:0> SP+24<31:28> <25:24> <23:16> <7> SP+28 <31:0> SP+32 <31:0> Byte count. The size of the stack frame in bytes, not including the PC, PSL, or the byte count longword. Stack frame PC and PSL values should always be referenced using this count as an offset from the stack pointer. AST LVL. The current value of the register. Machine check code. The reason for the machine check, as listed in Ta­ ble 9­5. CPUID. Contains the current value of the CPUID register. INT.SYS register. Contains the value of the INT.SYS register. This value is read onto the A­bus by the microcode. SAVEPC register. The SAVEPC register which is loaded by microcode with the PC value in certain circumstances. It is used in error handling for PTE read errors with PSL set in this stack frame. VA register. The contents of the Ebox VA register, which may be loaded from the output of the ALU. Q register. The contents of the Ebox Q register, which may be loaded from the output of the shifter. Rn. The value of the Rn register, which is used to obtain the register num­ ber for the CVTPL and EDIV instructions. In general, the value of this field is UNPREDICTABLE. Mode. A copy of the current mode field, PSL. Opcode. Bits <7:0> of the instruction opcode. The FD bit is not included. VR. The VAX Restart bit, which is used to communicate restart informa­ tion between the microcode and the operating system. When set, this bit indicates that no architectural state has been changed by the instruction that was executing when the error was detected. When clear, it indicates that architectural state was modified by the instruction. PC. The value of the program counter at the time of the fault. PSL. The value of the processor status longword at the time of the fault. 9­16 Error Handling Table 9­5 Machine Check Codes in the Stack Frame Figure 9­4 is a parse tree that can be used to analyze the cause of a ma­ chine check exception. The parse trees include error isolation at the proc­ essor level and module level. Discussions that follow provide detailed de­ scriptions of the errors, give the recovery procedures and, when appropriate, indicate the conditions for retry. NOTE: In the error descriptions it is assumed that the state being analyzed is the saved state. Otherwise the state could change during the analysis proce­ dure, possibly leading to incorrect conclusions. Code (Hex) Mnemonic Description: 01 02 03 04 05 06 MCHK_UNKNOWN_MSTATUS MCHK_INT.ID_VALUE MCHK_CANT_GET_HERE MCHK_MOVC.STATUS MCHK_ASYNC_ERROR MCHK_SYNC_ERROR Unknown memory management status error Illegal interrupt ID error Illegal microcode dispatch occurred Illegal combination of state bits detected dur­ ing string instruction Asynchronous hardware error occurred Synchronous hardware error occurred Error Handling 9­17 Figure 9­4 Machine Check Exception Parse Tree 9­18 Error Handling Figure 9­5 Machine Check Exception Parse Tree (Continued) Error Handling 9­19 Figure 9­5 Machine Check Exception Parse Tree (Continued) 9­20 Error Handling Figure 9­5 Machine Check Exception Parse Tree (Continued) Error Handling 9­21 Figure 9­5 Machine Check Exception Parse Tree (Continued) 9­22 Error Handling Figure 9­5 Machine Check Exception Parse Tree (Continued) Error Handling 9­23 Figure 9­5 Machine Check Exception Parse Tree (Continued) 9.4.1 MCHK_UNKNOWN_MSTATUS Description: An unknown memory management status was returned from the Mbox in response to a microcode memory management probe. This er­ ror is probably caused by an internal error in the Mbox, Ebox, or microsequencer. Recovery procedure: No explicit error recovery is required. Retry condition: This error only happens in microcode processing of mem­ ory management faults for a virtual memory reference. Retry if: (VR = 1) OR (PSL = 1) 9­24 Error Handling 9.4.2 MCHK_INT.ID_VALUE Description: An illegal interrupt ID was returned in INT.SYS during in­ terrupt processing in microcode. This error is probably caused by an inter­ nal error in the interrupt hardware, Ebox, or microsequencer. Recovery procedure: No explicit error recovery is required. Retry condition: This error only happens in microcode processing of inter­ rupts that occur between instructions or the middle of interruptible in­ structions. Retry if: (VR = 1) OR (PSL = 1) 9.4.3 MCHK_CANT_GET_HERE Description: Microcode execution reached a presumably impossible ad­ dress. This error is probably caused by an internal error in the Ebox or microsequencer. Recovery procedure: No explicit error recovery is required. Retry condition: Retry if: (VR = 1) OR (PSL = 1) 9.4.4 MCHK_MOVC_STATUS Description: During execution of MOVCx, the two state bits that encode the state of the move (forward, backward, fill) were set to the fourth (ille­ gal) combination. This condition is probably caused by an internal error in the Ebox or microsequencer. Recovery procedure: No explicit error recovery is required. Retry condition: Because the state bits encode the operation, the instruc­ tion cannot be restarted in the middle of the MOVCx. If software can de­ termine that no specifiers have been overwritten (MOVCx destroys R0R5 and memory due to string writes), the instruction can be restarted from the beginning by clearing PSL. Retry should be done only if the source and destination strings do not overlap and if: PSL = 1 9.4.5 MCHK_ASYNC_ERROR This machine check code reports serious errors that interrupt the microcode at an arbitrary point. Many internal machine states (that is, bits in PSL, PC, and SP) are questionable and recovery is typically not pos­ sible. TB Parity Error Description: Parity errors in tags and PTE data in the TB that hit cause an asynchronous machine check by directly forcing a microtrap in the microsequencer. Since the Ibox is nearly always able to issue instruction prefetches, TB  parity errors could occur at any time, making it impossible to determine what machine state is incorrect. There is no guarantee that all writes with Error Handling 9­25 a different PSL completed successfully. Therefore, even the stack frame PSL cannot be used to determine whether sys­ tem data is uncorrupted. Recovery procedure: Clear TBSTS. Retry condition: Retry is not possible. Crash the system. Ebox Stage 3 Stall Timeout Error Description: Stage 3 stall timeout errors occur when the Ebox microcode is stalled waiting for some result or action that will probably never happen. S4 stalls in the Ebox cause S3 stalls and, therefore, can lead to an S3 stall timeout. Additionally, field queue stall and instruction queue stall can cause this timeout. The timeout can occur in any microflow for a number of reasons. Machine state may be corrupted because of an internal error in NVAX+ such that one box would be waiting for another to do something which the second box will not do. An example would be the Ebox micro­ code expecting one more source specifier than the Ibox delivered and timeout occurs while the Ebox is waiting. S3 timeout errors can be caused by failures of various pipeline control circuits in the Ebox. Also, this error can be caused by a deadlock within a box or across multiple boxes. Recovery procedure: Clear ECR. Retry condition: It is not possible to determine what machine state is in­ correct. Retry is not possible. Crash the system. 9.4.6 MCHK_SYNC_ERROR This machine check code reports errors that occur in memory or I/O space instruction fetches or data reads. Except in the case of PTE read errors, NVAX+ state should be consistent since microcode has to explicitly access an operand or instruction in order to incur this error. Microcode does not access memory results or dispatch for a new instruction execution with the NVAX+ in an inconsistent state. PTE read errors on write transactions can cause a microtrap at an arbi­ trary time, and so the NVAX+ state may be inconsistent. Many of the error events described below for synchronous machine checks are possible causes. If more than one is present, there is no way to deter­ mine which actually caused the machine check. If only one cause is discov­ ered, then the machine check can be attributed to that cause. The reason multiple causes may be present is that the NVAX+ CPU prefetches in­ structions and data. If the CPU branches or takes an exception before us­ ing data it has requested, then the pending machine check is taken as a soft error interrupt (although it might not be recoverable). If multiple errors occur, recovery and retry may be possible. It is recom­ mended that retry from multiple errors be done only if one error report does not interfere with analysis of, and recovery from, another error. If two errors are entirely separate, neither interfering with the analysis and recovery of the other, then it is acceptable to retry from these errors provided all the error analysis and recovery procedures result in a retry in­ dication. In several cases, lost errors are tolerated. The strong tendency to prefetch data exhibited by the NVAX+ pipeline makes the particular lost error likely, given that one error of that kind occurred. Also, in each case, if data 9­26 Error Handling is lost in the lost error, a hard error interrupt is posted. So these errors are tolerated as long as they do not cause a hard error interrupt. BIU_STAT is maintained to report unrecorded errors on write operations. If BIU_STAT is set, the H_ERR inter­ rupt is asserted. Errors in opcode or operand specifier fetching are always detected before architecturally visible state within the CPU is modified. This means the VR bit from the machine check stack frame should be one. This error han­ dling analysis attempts to recover from multiple errors, so the retry condi­ tion for each error is made as general as possible. If the machine check handler finds only errors of the kind listed here, then VR should be one and it is an inconsistent report if it is not. · VIC parity errors · B­data uncorrectable ECC FILL errors in I­stream reads · CACK_H error in I­stream reads VIC Parity Errors Description: A parity error was detected in the VIC tag or data store in the Ibox. · VIC data parity error A parity error occurred in the data portion of the VIC. · VIC tag parity error A parity error occurred in the tag portion of the VIC. In all cases, the quadword virtual address of the error is in VMAR. NOTE: ICSR must be set for ICSR or ICSR to be valid. The Lock bit also latched the information in the VMAR register. If this bit is clear, then no parity errors have occurred and the registers can be updated. Pending interrupts: A soft error interrupt should be pending. Recovery procedure: Disable and flush the VIC by rewriting all the tags. Clear ICSR. Retry condition: Retry if: (VR = 1) OR (PSL = 1) FILL Uncorrectable ECC Errors Description: (uncorrectable ECC errors): An uncorrectable data error was detected by the Cbox in an I­stream or D­stream read fill. Fill uncorrect­ able data errors can result from reads of the B­cache, another CPU cache, memory, or LSB (double error). S_FILL_ADDR contains the address of the error, and S_FILL_SYNDROME contains the syndrome calculated by the ECC logic. Pending interrupts: A soft or hard error interrupt should be pending. Recovery procedure: The machine check handling software should defer re­ covery until the expected hard or soft interrupt occurs. If one of these in­ terrupts is taken, the B­cache should be flushed and BIU_STAT should be cleared. Error Handling 9­27 Retry condition: If no writeback error occurs in the B­cache flush, retry if: (VR = 1) OR (PSL= 1) If a writeback error occurs in the B­cache flush, then the data is presumed to be unrecoverable. Given that the address is available (no error in the tag store), software should determine if the error is fatal to one process or the whole system and take appropriate action. Otherwise, crash the sys­ tem. Lost Fill Error Description: Some fill errors were not latched because a previous fill error was reported in the BIU_STAT register. If the reported error is not a read, a fill error while merging write data from a write has been logged. The logged error is not the cause of the machine check, but FILL_SEO (BIU_STAT<14>) might be. H_ERR should be pending if the reported er­ ror was not correctable. If the reported error is a read or a correctable fill error and LOST_WRITE (BIU_STAT<20>) is not set, the error causing FILL_SEO to set may be the cause of the machine check. The operation can be retried unless the aborted instruction has altered essential state. If S_PCSTS is set, refer to PTE errors, discussed below in this section. Lost fill errors may be caused by more than one operand prefetch to the same cache block. Recovery from lost fill errors depends on whether the pending interrupt is hard or soft. The machine check error handling software should defer re­ covery until the expected hard or soft error interrupt occurs. Once the in­ terrupt is taken, the error recovery and restart instructions found in the hard error interrupt and soft error interrupt sections should be referenced (Sections 9.5 and 9.6, respectively). Software should employ some mechanism to record that an interrupt for a lost fill error is pending. This mechanism should allow detection of a case in which an expected interrupt does not occur (once the IPL is lowered). If the expected interrupt does not occur when the IPL is lowered, then a seri­ ous inconsistency exists and the system should be crashed. Pending interrupts: A hard or soft error interrupt, possibly both, should be pending. Recovery procedure: No specific recovery action is required. BIU_STAT should be cleared by the hard or soft error inter­ rupt handler. Retry condition: Retry only if: (VR = 1) OR (PSL = 1) BIU_HERR Description: An I­stream or D­stream read returned CACK_H error or did not complete because of a tag (BIU_STAT=1) or tag control par­ ity error (BIU_STAT=1) . I­stream errors cause a machine check when the Ebox microcode requests dispatch to a new instruction execution microflow or attempts to access an operand within an instruction execution microflow. 9­28 Error Handling D­stream errors cause a machine check when the Ebox microcode accesses prefetched operand data or when the Mbox returns data tagged with an er­ ror indication to the Ebox register file. D­stream ownership read errors cause a machine check when the Ebox microcode accesses prefetched operand data. If the address is in I/O space, the status is inconsistent. Pending interrupts: A soft error interrupt should be pending. Recovery procedure: Clear BIU_STAT. Retry condition: Retry if: (VR = 1) OR (PSL = 1) Lost BIU Error Description: Some fill errors were not latched because a previous BIU er­ ror was reported in the BIU_STAT register. If the reported error is not a read, a fill error while merging write data from a write has been logged. The logged error is not the cause of the machine check, but BIU_SEO (BIU_STAT<7>) might be. H_ERR should be pending. If the reported er­ ror is a read or a correctable fill error and LOST_WRITE (BIU_STAT<20>) is not set, the error causing BIU_SEO to set may be the cause of the machine check, and the operation can be retried unless the aborted instruction has altered essential state. If S_PCSTS is set, refer to PTE errors, discussed below in this section. Lost BIU errors may be caused by more than one operand prefetch to the same cache block. Recovery from lost BIU errors depends on whether the pending interrupt is hard or soft. The machine check error handling software should defer recovery until the expected hard or soft error interrupt occurs. Once the interrupt is taken, the error recovery and restart instructions found in the hard error interrupt and soft error interrupt sections should be referenced (Sections 9.5 and 9.6, respectively). Software should employ some mechanism to record that an interrupt for a lost fill error is pending. This mechanism should allow detection of a case in which an expected interrupt does not occur (once the IPL is lowered). If the expected interrupt does not occur when the IPL is lowered, then a seri­ ous inconsistency exists and the system should be crashed. Pending interrupts: A hard or soft error interrupt, possibly both, should be pending. Recovery procedure: No specific recovery action is required. BIU_STAT should be cleared by the hard or soft error inter­ rupt handler. Retry condition: Retry only if: (VR = 1) OR (PSL = 1) PTE Read Errors PTE read errors happen in reads issued by the Mbox in handling a TB miss. Handling of these errors differs from handling the same underlying error (BIU_HERR, BC_TPERR, BC_TCPERR, FILL_ECC) when PTE read is not the cause. Error Handling 9­29 If S_PCSTS is set, then a PTE read issued by the Mbox in proc­ essing a TB miss had an unrecoverable error. The TB miss sequence was aborted because of the error. The original reference can be any I­stream or D­stream read or write. If the original reference was issued by the Ebox, then the PTE read that incurred the error will have been retried once be­ cause of a special hardware/microcode mechanism for handling PTE read errors on Ebox references. PTE read errors are difficult to analyze, partly because the read error re­ port in the Cbox does not directly indicate that the failing read was a PTE read. Because of this and because PTE read errors should be rare (a very small percentage of the reads issued by the Mbox are PTE reads), multiple errors that interfere with the analysis of the PTE error are not considered recoverable. The mechanism for reporting PTE read errors on Ebox references involves the Mbox forcing the Ebox into the microcode routine which normally han­ dles memory management faults. This routine probes the address of the original reference, effectively retrying the failing PTE read. Assuming the error is not transient, the probe by microcode will cause a machine check. If the error does not occur on the probe, microcode restarts the current in­ struction stream. So machine checks caused by PTE read errors can easily occur with the particular PTE read error having occurred twice (with a lost error bit set in the relevant Cbox error register). The analysis here tolerates these particular multiple error reports and al­ lows retry in those cases, provided the remainder of the error analysis indi­ cates retry is appropriate. If the reference that incurs the PTE read error is a write, S_PCSTS will be set. In this case the original write is lost. No retry is possible partly because the instruction that took the machine check may be subsequent to the one that issued the failing write. Also, PTE read errors on write transactions can cause a machine check at an arbitrary time in a microcode flow, and core machine state may not be consistent. PTE Read Errors in Interruptible Instructions Another special case associated with PTE read errors exists for interrupt­ ible instructions (specifically CMPC3, CMPC5, LOCC, MOVC3, MOVC5, SCANC, SKPC, and SPANC). For these instructions, if the PTE read error occurred for an Ebox reference, the PC in the machine check stack frame points to the instruction following the interrupted instruction. In this case, the SAVEPC element in the machine check stack frame is the PC of the interrupted instruction. However, in all other cases SAVEPC is UNPREDICTABLE. This case is not considered recoverable because analysis of the error information can­ not unambiguously conclude that this case is present. To see if this case is present, the error handler examines PSL in the machine check stack frame. If FPD is set in the stack frame in the case of a PTE read error, then one of the following is true: · One of the interruptible instructions listed above incurred the PTE read error. In this case, SAVEPC from the machine check stack frame points to the interrupted instruction, and the PC in the stack frame points to the next instruction. 9­30 Error Handling · An REI instruction loaded a PSL with FPD set and a certain PC. The Ibox incurred the PTE read error in fetching the opcode pointed to by that PC. In this case, the PC in the stack frame points to the instruc­ tion that was the target of the REI, and SAVEPC from the stack frame is UNPREDICTABLE. It is not possible to determine which of the two above cases is the cause of a machine check with S_PCSTS set and stack frame PSL set. Retry is not possible since software cannot tell which PC to restart with. Uncorrectable ECC Fill Errors and PTE Errors on Reads Description (uncorrectable ECC errors): A fill uncorrectable data error was detected by the Cbox in a PTE read. Uncorrectable data errors are the re­ sult of a multiple­bit error in the data read from the B­cache on a fill from the system during a READ_BLOCK or LDxL. Description (all cases): S_FILL_ADDR contains the cache address of the error, and FILL_SYNDROME contains the syndrome calculated by the ECC logic. If the address is in I/O space, this is an inconsistent status. S_BIU_STAT may be set. This error is probably caused by the same PTE error occurring more than once. This is an acceptable as­ sumption unless a hard error interrupt occurs after handling this error. Pending interrupts: A soft error interrupt should be pending. Recovery procedure (uncorrectable ECC errors): Clear BIU_STAT. Recovery procedure (both cases): Flush the B­cache. Clear PCSTS. Retry condition: If no writeback error occurs in the B­cache flush, retry if: (VR = 1) AND (PSL = 0) AND (S_PCSTS = 0) Crash the system if: (PSL = 1) OR (S_PCSTS = 1) If a writeback error occurs in the B­cache flush, then the data is presumed to be unrecoverable. Software must determine if the error is fatal to one process or the whole system and take appropriate action. CACK_H Error on PTE Read Description: A PTE read returned CACK_H error. S_BIU_STAT may be set. This error is probably caused by the same PTE error occurring more than once. This is an acceptable assump­ tion unless a hard error interrupt occurs after handling the error. Pending interrupt: A soft error interrupt should be pending. Recovery procedure: Clear BIU_STAT and PCSTS. Retry condition: Retry if: (VR = 1) AND (PSL = 0) AND (S_PCSTS = 0) Error Handling 9­31 Otherwise, crash the system. Post retry recovery: If the same fill error occurs on retry, then the block is probably lost. In this case the more general sense of lost is implied. Software must determine if the error is fatal to one process or the whole system and take appropriate action. NOTE: In a block lost case, it would be appropriate to first cause each CPU in the system to flush its B­cache, then retry once more. 9.4.7 Inconsistent Status in Machine Checks Description: A presumed impossible error report was found in the error registers. This could be due to a hardware failure. Pending interrupts: Either a hard or soft error interrupt, or both should be pending. Recovery procedure: No specific recovery action is called for. Retry condition: No retry is possible. Crash the system. 9­32 Error Handling 9.5 Hard Error Interrupts A hard error interrupt reports an error that was detected asynchronous with respect to the instruction execution. A hard error interrupt results in an interrupt at IPL 1D (hex) being dis­ patched through SCB vector 60 (hex). Typically, these errors indicate that machine state has been corrupted and that a retry is not possible. The stack frame for a hard error interrupt consists of two parameters, PC and PSL. Figure 9­5 contains the hard error interrupt parse tree, which isolates er­ rors at the processor level and module level. The sections following the parse tree provide a description of the hard error, the procedure to recover, and the conditions for restarting the operation. Figure 9­5 Hard Error Interrupt Parse Tree Error Handling 9­33 Figure 9­6 Hard Error Interrupt Parse Tree (Continued) 9­34 Error Handling Figure 9­6 Hard Error Interrupt Parse Tree (Continued) Error Handling 9­35 Figure 9­6 Hard Error Interrupt Parse Tree (Continued) 9­36 Error Handling Figure 9­6 Hard Error Interrupt Parse Tree (Continued) Error Handling 9­37 Figure 9­6 Hard Error Interrupt Parse Tree (Continued) 9­38 Error Handling Figure 9­6 Hard Error Interrupt Parse Tree (Continued) Error Handling 9­39 Figure 9­6 Hard Error Interrupt Parse Tree (Continued) Uncorrectable Errors During Write or Write Unlock Processing Description: In processing a write or write unlock, the Cbox detected a CACK_H error from the system, a tag parity error, a control parity error, or an uncorrectable ECC error on the data read, which is to be merged data from the write, is lost. Uncorrectable ECC errors indicate that two or more bits of the stored data quadword have changed and the error correcting code cannot correct the data. The write merge sequence is aborted. Recovery procedure: The data in this block is lost. Restart condition: If the address of the data is available and no unex­ pected writeback errors occurred during the B­cache flush, software must determine if the lost data is fatal to one process or the whole system, and take appropriate action. System Environment Hard Error Interrupts See Section 9.8. 9­40 Error Handling Inconsistent Status in Hard Error Interrupt Cause Analysis Description: A presumed impossible error report was found in the error registers. This could be due to a hardware failure or bug. Recovery procedure: No specific error action is called for. Restart condition: No retry is possible. Crash the system. 9.6 Soft Error Interrupts A soft error interrupt is requested to report an error that was detected but which did not affect instruction execution. This results in an interrupt at IPL 1A (hex) to be dispatched through SCB vector 54 (hex). The stack frame for a soft error interrupt consists of two parameters, PC and PSL. Many errors that cause a soft error interrupt may also lead to a machine check exception. For this reason, a soft error interrupt with no apparent cause is not an inconsistent state unless the CPU has executed an instruc­ tion while IPL was lower that 1A (hex) since the most recent machine check exception. When a soft error interrupt is the only notification for a memory read error which could cause a machine check, the error did not cause a machine check for one of the following reasons: · The error did not occur on the quadword the Ebox or the Ibox re­ quested. (P­cache fill error). · The Ebox took an interrupt before accessing an instruction or operand which was prefetched by the Ibox. (It could be this software interrupt.) · A prefetched instruction or operand belonged to an instruction follow­ ing the mispredicted branch, so the Ebox never executed the instruc­ tion and it was flushed from the pipeline when the branch mispredict was recognized. · The Ebox took an exception for a different reason before attempting to use an instruction execution dispatch or access operand prefetched by the Ibox. (The pipeline was flushed because of the exception.) Figure 9­6 contains the soft error interrupt parse tree. The sections follow­ ing the parse tree provide a description of the soft error, the procedure to recover, and the conditions for restarting the operation. Error Handling 9­41 Figure 9­6 Soft Error Interrupt Parse Tree VIC Parity Errors Description: A parity error was detected in the VIC tag or data store in the Ibox. The data parity error may occur in bank 0 (DPERR0) or bank 1 (DPERR1) of the VIC. The tag parity error may occur in bank 0 (TPERR0) or bank 1 (TPERR1) of the VIC. The quadword virtual address of the error is in S_VMAR. Recovery procedure: Disable and flush the VIC by rewriting all the tags and clear ICSR. 9­42 Error Handling P­Cache Parity Errors Description: A parity error was detected in the P­cache. Either a tag par­ ity error or a data parity error is reported, although tag parity errors in both the left and right banks may be reported simultaneously. The refer­ ence, whether it was a read or write, was passed to the Cbox as if the P­ cache had missed. No data is lost. The P­cache is disabled because PCSTS is set. S_PCADR contains the physical address of the operation incurring the er­ ror. The address should not be in I/O space. If it is, it is an inconsistent status. Recovery procedure: Clear PCSTS. Flush the P­cache and initial­ ize the P­cache tag store. FILL Uncorrectable ECC Errors on I­Stream or D­Stream Reads Description: An uncorrectable ECC error was detected by the Cbox in an I­ stream or D­stream read. Uncorrectable data errors are the result of mul­ tiple bit errors in the data read. S_FILL_ADDRESS contains the address of the error, and S_FILL_SYNDROME contains the syndrome calculated by the ECC logic. If the physical address is found to be in I/O space, it is an inconsistent status. Recovery procedure: Clear BIU_STAT. Flush the B­cache. (BC_TAG can be used to determine if the fill is from the B­cache.) If the data is dirty in the B­cache and the error repeats itself (it is not transient), then a writeback error will result from the flush procedure. Restart conditions: If a writeback error occurs in the B­cache flush, then the data is presumed to be unrecoverable. Software must determine if the error is fatal to one process or the whole system and take appropriate ac­ tion. If the address of the error in the flush is not the same as that of the origi­ nal error, this is a multiple error case in the data RAMs and is a serious failure. Crash the system. Multiple Errors Associated with PTE Errors PTE errors are difficult to analyze, partly because the read error report in the Cbox does not directly indicate the failing read was a PTE read. Be­ cause of this and because PTE read errors should be rare (a very small per­ centage of the reads issued by the Mbox are PTE reads), multiple errors that interfere with the analysis of the PTE error are considered recover­ able. If the reference that incurs the PTE read error is a write, S_PCSTS will be set. In this case the original write is lost. No retry is possible partly because the instruction that took the machine check may be subsequent to the one that issued the failing write. Also, PTE read errors on write transactions can cause a machine check at an ar­ bitrary time in a microcode flow, and core machine state may not be consis­ tent. Restart condition: If no writeback error occurs in the B­cache flush, restart if: S_PCSTS = 0 Error Handling 9­43 If: S_PCSTS = 1 crash the system. If a writeback error occurs in the B­cache flush, then the data is presumed to be unrecoverable (software must determine if the error is fatal to one process or the whole system and take appropriate action). Clear PCSTS. Restart condition: Restart if: S_PCSTS = 0 Otherwise, crash the system. Multiple Errors That Interfere with Analysis of PTE Read Errors Because analysis of PTE read errors leads to several unusual causes, re­ start is not recommended in the event that other errors cloud the analysis of the PTE error. Pending interrupts: A hard or software interrupt should be pending, or possibly both. Recovery procedure: No specific recovery action is called for. Restart condition: Not possible. Crash the system. 9.7 Kernel Stack Not Valid Exception A kernel stack not valid exception occurs when a memory management ex­ ception is detected while attempting to push information on the kernel stack during microcode processing of another exception. Note that a con­ sole halt with an error code of ERR_INTSTK is taken if a memory manage­ ment exception is encountered while attempting to push data on the inter­ rupt stack. The kernel stack not valid exception is dispatched through SCB vector 08 (hex). The kernel stack not valid stack frame consists of two parameters, PC and PSL. 9­44 Error Handling 9.8 System Environment Errors System environment errors are detected by the LEVI gate arrays, which signal them to the NVAX+ processor by two mechanisms corresponding to the two sets of pins in the processor used for error reporting: CACK_H sig­ nals and IRQ_H signals. Parse trees given in Figures 9­4 and 9­5 isolate errors to the system envi­ ronment as well as to the processor. 9.8.1 Error Categories Module and system level errors can be divided into two groups: synchro­ nous errors and asynchronous errors (not to be confused with NVAX+ syn­ chronous and asynchronous errors, which are internal to the processor). 9.8.1.1 Synchronous Errors Synchronous errors are those detected on the CPU module or in the LSB system while the processor was waiting (idle) for service from LEVI. These errors imply that something drastic went wrong while LEVI was trying to complete the processor's request, and that LEVI is unable to complete the processor's transaction. Specifically, only the following errors induce syn­ chronous errors if the processor is waiting for an acknowledgment from LEVI: · B­cache tag parity error · B­cache status parity error · LSB arbitration drop · LSB command parity error · LSB nonexistent address error Other errors detected while the processor is waiting for an acknowledg­ ment are signaled as asynchronous errors because they do not prevent completion of the processor's request. LEVI reports synchronous errors to the processor by returning H_ERR status instead of the normal acknowledgment. (The CACK_H signals are driven with H_ERR instead of a normal cycle acknowledge. This in turn sets BIU_STAT.) 9.8.1.2 Asynchronous Errors Asynchronous errors include all errors detected on the CPU module or in the LSB system while the processor was not waiting for service from LEVI. Module and LSB errors not included in the synchronous error list above are categorized as asynchronous errors even if the processor is waiting for LEVI. The distinction is that only certain errors prevent LEVI from com­ pleting the processor's request. LEVI reports asynchronous errors to the processor through the interrupt signals (IRQ_H). LEVI asserts a module signal (LEVI_ERROR) which is dispatched to the appropriate IRQ_H pin on the processor. Error Handling 9­45 9.8.2 Environment Error Sources The environment errors originate either from the LEVI operations within the CPU module or from interactions of the LEVI with other nodes through the LSB bus. 9.8.2.1 LEVI Errors LEVI B­Cache Status Parity Error Description: LEVI detected a B­cache status parity error while examining the status bits on behalf of the processor or the LSB. If LEVI detects the error while evaluating status in response to a processor request, it returns an H_ERR acknowledgment. If LEVI detects the error while evaluating status to determine a response to an LSB request, it signals the error to the processor even if the proces­ sor is waiting for an acknowledgment. Processor response: If H_ERR is returned in response to a read request, NVAX+ takes a machine check at IPL 31 through SCB offset 04 (hex). If H_ERR is returned in response to a write, or IRQ_H<4> is asserted, NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). In either case, LEVI asserts ERROR on the LSB. State captured: LMERR and LBER are set. Address and LSB command are captured in the LBECR register. Recovery: None possible. Crash the system. The CPU module responds by asserting ERROR on the LSB, while SHARED and DIRTY remain unas­ serted. LEVI B­Cache Tag Parity Error Description: LEVI detected a B­cache tag parity error while examining the tag bits on behalf of the processor. (Note that LEVI never examines the B­ tag RAMs on behalf of the LSB. It uses the B­map to perform this func­ tion.) It returns H_ERR acknowledgment. Processor response: If H_ERR is returned in response to a read request, NVAX+ takes a machine check through SCB offset 04 (hex). If H_ERR is returned in response to a write, NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). In either case, LEVI asserts ERROR on the LSB. State captured: LMERR and LBER are set. The ad­ dress and LSB command are captured in the LBECR register. Recovery: None possible. LEVI P­Map Parity Error Description: LEVI detected a parity error while examining its internal P­ map to determine how to respond (invalidate or update) to an LSB write. (Note that this is only done if the write address hits in the B­map.) It noti­ fies the processor by asserting the IRQ_H <5> signal. Processor response: NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). LEVI asserts ERROR on the LSB. 9­46 Error Handling State captured: LMERR and LBER are set. The ad­ dress and LSB command are captured in the LBECR register. Recovery: Flush the relevant internal processor cache and invalidate the index in the map RAM structure. LEVI responds by asserting DIRTY from the results of the B­map only and assumes a P­map hit (asserts SHARED). LEVI B­Map Parity Error Description: LEVI detected a parity error while examining the B­map to check for an LSB hit in this cache. LEVI asserts the IRQ_H<5> signal to notify the processor. Processor response: NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). LEVI asserts ERROR on the LSB. State captured: LMERR and LBER are set. The ad­ dress and LSB command are captured in the LBECR register. Recovery: No recovery is possible since this error is detected in the LSB time domain in response to an LSB command. Crash the system. The CPU module responds by asserting ERROR on the LSB, while SHARED and DIRTY remain unasserted. 9.8.2.2 LSB Errors LSB Arbitration Drop Description: LEVI signals an arbitration drop error when a node, having requested and been granted the LSB, fails to initiate a command/address cycle (by asserting CA on the LSB). If LEVI is the commander that failed to initiate a command in response to a processor request, it acknowledges the processor with a H_ERR status. If LEVI caused the error while the processor was not waiting for a response (while writing a victim for exam­ ple) or if any other node caused the error, LEVI notifies the processor by asserting the IRQ_H<4> signal. Processor response: If H_ERR is returned in response to a read request, NVAX+ takes a machine check through SCB offset 04 (hex). If H_ERR is returned in response to a write, or IRQ_H<4> is asserted, NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). In either case, LEVI asserts ERROR on the LSB. State captured: LMERR and LBER are set. The address and LSB command are captured in the LBECR register. Recovery: All nodes resynchronize the arbitration when ERROR is as­ serted on the LSB. Crash the system. LSB Arbitration Collision Description: LEVI signals an arbitration collision error by asserting the IRQ_H<5> signal when a node tries to request the LSB (that is, drives the REQ lines) during an inappropriate cycle. Processor response: NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). LEVI asserts ERROR on the LSB. State captured: LMERR and LBER are set. Error Handling 9­47 Recovery: All nodes resynchronize the arbitration when ERROR is as­ serted on the LSB. Crash the system. LSB Transmit Check Error Description: LEVI signals a transmit check error when it detects a dis­ crepancy between the command (or data) it drives onto the LSB and the command (or data) it receives from the LSB. It notifies the processor by asserting the IRQ_H<5> signal. Processor response: NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). LEVI asserts ERROR on the LSB. State captured: LBER or LBER is set. Recovery: All nodes will resynchronize the arbitration when ERROR is as­ serted on the LSB. Software should probably crash. LSB Dirty Error Description: LEVI signals a Dirty error when LSB DIRTY is asserted dur­ ing an inappropriate cycle. It notifies the processor by asserting the IRQ_H<5> signal. Processor response: NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). LEVI asserts ERROR on the LSB. State captured: LBER is set. Recovery: LSB ERROR is asserted and all nodes resynchronize the arbi­ tration. Crash the system. LSB Shared Error Description: LEVI signals a Shared error when LSB SHARED is asserted during an inappropriate cycle. It notifies the processor by asserting the IRQ_H<5> signal. Processor response: NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). LEVI asserts ERROR on the LSB. State captured: LBER is set. Recovery: LSB ERROR is asserted and all nodes resynchronize the arbi­ tration. Crash the system. LSB CA Error Description: LEVI signals a CA error when LSB CA is asserted during an inappropriate cycle. It notifies the processor by asserting the IRQ_H<5> signal. Processor response: NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). LEVI asserts ERROR on the LSB. State captured: LBER is set. Recovery: LSB ERROR is asserted and all nodes will resynchronize the ar­ bitration. Software should probably crash. 9­48 Error Handling LSB Nonexistent Address Error Description: LEVI signals a nonexistent address error when there is no CNF response to a command/address cycle on the LSB. If the missing CNF was needed to complete a processor request, LEVI returns an H_ERR acknowledgment. If the error occurred while the processor was not wait­ ing for a response or if the error was in response to a request from another node, LEVI signals the processor by asserting the IRQ_H<4> signal. Processor response: If H_ERR is returned in response to a read request, NVAX+ takes a machine check through SCB offset 04 (hex). If H_ERR is returned in response to a write, or IRQ_H<4> is asserted, NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). In either case, LEVI asserts ERROR on the LSB. State captured: LBER is set. The address and LSB command are captured in the LBECR register. Note that although there is no CNF sig­ nal on the LSB, the transaction continues and data parity or ECC errors will likely follow when the requesting node reads an undriven LSB during the corresponding data cycles. Any of the following LBER bits may be set along with NXAE: UCE, UCE2, CE, CE2, CDPE, CDPE2. Recovery: LSB ERROR is asserted and all nodes resynchronize the arbi­ tration. Software should probably crash if this was not expected (like not during INIADAPT). LSB CNF Error Description: LEVI signals a CNF error when CNF is asserted during an inappropriate cycle. It notifies the processor by asserting the IRQ_H<5> signal. Processor response: NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). LEVI asserts ERROR on the LSB. State captured: LBER is set. Recovery: LSB ERR is asserted and all nodes will resynchronize the arbi­ tration. Software should probably crash. LSB Stall Error Description: LEVI signals a Stall error when LSB STALL is asserted dur­ ing an inappropriate cycle. It notifies the processor by asserting the IRQ_H<5> signal. Processor response: NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). LEVI asserts ERROR on the LSB. State captured: LBER is set. Recovery: LSB ERR is asserted and all nodes will resynchronize the arbi­ tration. Software should probably crash. Error Handling 9­49 LSB Transmitter During Error Description: This bit is set in the LBER of the node driving the LSB when an error is detected. Only one node should have this bit set for any given error. This bit is set only in response to LSB ERROR; it does not cause LSB ERROR to be asserted. LEVI notifies the processor when it detects the original error. Processor response: None. State captured: LBER is set. Recovery: All nodes resynchronize the arbitration once ERROR is asserted. Software should probably crash. LSB CSR Data Parity Error Description: LEVI detected a parity error on the LSB during a CSR data cycle. It notifies the processor by asserting the IRQ_H<5> signal regard­ less of whether the processor is waiting for this data. Processor response: NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). LEVI asserts ERROR on the LSB. State captured: LBER or LBER is set. Address and LSB command are captured in the LBECR register (only on the first CDPE; not on CDPE2). Recovery: LSB ERROR is asserted and all nodes resynchronize the arbi­ tration. LSB Command Parity Error Description: LEVI detected bad parity on an LSB command cycle. If the command was issued to satisfy a processor request LEVI returns an H_ERR acknowledgment. If LEVI caused the error while the processor was not waiting for a response (while writing a victim for example) or if any other node caused the error, LEVI notifies the processor by asserting one of the IRQ_H signals. Processor response: If H_ERR is returned in response to a read request, NVAX+ takes a machine check through SCB offset 04 (hex). If H_ERR is returned in response to a write, or IRQ_H<4> is asserted, NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). In either case, LEVI asserts ERROR on the LSB. State captured: LBER or LBER is set. The address and LSB command are captured in the LBECR register (only on the first com­ mand parity error; not on CPE2). Recovery: LSB ERROR is asserted and all nodes resynchronize the arbi­ tration. LSB Correctable ECC Error Description: LEVI detected a correctable data error on the LSB. It notifies the processor by asserting one of the IRQ_H signals only if LCNR is set. Processor response: NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). LEVI asserts ERROR on the LSB. 9­50 Error Handling State captured: If LCNR is clear, nothing is captured. Otherwise, LBER or CE2 is set, the LSB command and address are captured in the LBECR register, and the ECC syndromes are captured in the LBESR register (only on the first correctable ECC error; not on CE2). If the data was driven by LEVI onto the LSB, LMERR is also set. Recovery: If LCNR is set, LSB ERROR is asserted and all nodes will resynchronize the arbitration. The eventual consumer of this data (if any) will also report a single­bit error and correct it. The LEVI gate arrays will not correct data; only set­aside checking is performed for fault isola­ tion purposes. LSB Uncorrectable ECC Error Description: LEVI detected an uncorrectable data error on the LSB. It no­ tifies the processor by asserting the IRQ_H<5> signal. Processor response: If the IRQ_H<5> pin is asserted, NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). LEVI asserts ER­ ROR on the LSB. State captured: LBER or UCE2 is set. The address and LSB com­ mand are captured in the LBECR register and ECC syndromes are cap­ tured in LBESR (only on the first Uncorrectable ECC error, not on UCE2). If the data was driven by LEVI onto the LSB, LMERR is  also set. Recovery: LSB ERROR is asserted and all nodes resynchronize the arbi­ tration. The eventual consumer of this data (if any) also reports a double­ bit error and takes appropriate action. LEVI does not correct data; only set­aside checking is performed for fault isolation purposes. LSB Error Line Asserted Description: Some node on the LSB asserted ERROR. (All nodes set the LBER bit whenever LSB ERROR is asserted.) LEVI notifies the processor through the IRQ_H<5> signal. Processor response: If the IRQ_H<5> signal is asserted, NVAX+ takes a hard error interrupt at IPL 29 through SCB offset 60 (hex). LEVI asserts ERROR on the LSB. State captured: LBER is set. Recovery: LSB ERROR is asserted and all nodes resynchronize the arbi­ tration. Some other node on the LSB has noticed an error condition that was not noticed by this node. Software should query the LBER registers of other nodes and take appropriate action. Index­1 Index A Aborts, 2­10 Access synchronization, 4­6 ADDR, 2­39 Address path, LEVI, 4­2 Address space, 2­3 physical, 2­3 virtual, 2­3 Address space decoding, IPR, 2­32 Address space mapping, 4­7 Address translation process space, 2­5 system space, 2­4 Address Width bits, 7­11 ALLOC, 2­70 Allocation bit, 2­70 ARBCOL, 7­23 ARBDROP, 7­23 Arbitration collision, 9­46 Arbitration Collision bit, 7­23 Arbitration Drop bit, 7­23 Arbitration, boot processor, 8­3 Arbitration, LSB, 4­6 Architectural ID bits, 2­98 ARCH_ID, 2­98 Arithmetic exceptions, 2­11 Arithmetic exception codes, 2­12 Arithmetic exception stack frame, 2­11 Asynchronous errors, 9­44 AW, 7­11 B Backmaps, cache, 3­8 Backup cache description, 3­1, 3­5 BAD, 5­18 Bad bit, 5­18 Bank Select bit, 2­69 BANK_SEL, 2­69 Base addresses, 7­2 BC_ENB, 2­80 BC_FHIT, 2­80 BC_SIZE, 2­79 BC_SPD, 2­80 BC_TAG, 2­85 BC_TCPERR, 2­90 BC_TPERR, 2­90 BDATADBE, 7­24 BDATASBE, 7­24 BEDECC, 2­95 BIU Address Register, 2­91 BIU Address <33:32> bits, 2­87 BIU Control Register, 2­78 BIU Dispatch Command bits, 2­90 BIU Error Physical Address bits, 2­91 BIU Hard Error bit, 2­90 BIU Parity Error bit, 2­89 BIU SEO bit, 2­89 BIU Soft Error bit, 2­90 BIU Status Register, 2­87 BIU_ADDR, 2­87, 2­91 BIU_CTL, 2­78 BIU_CTL read/write, 2­80 BIU_DIS_CMD, 2­90 BIU_ERR_PA, 2­91 BIU_HERR, 2­90, 9­27 BIU_SEO, 2­89 BIU_SERR, 2­90 BIU_STAT, 2­87 BIU_STAT updates, 2­90 Block diagram CPU module, 1­2 system, 1­1 BMAPP, 7­31 BMAPPE, 7­24 Boot processor arbitration, 8­3 Boot processor system setup, 8­3 BPCR, 2­43 BPCR read/write, 2­44 BPU_ALGORITHM, 2­43 Branch Prediction Control Register, 2­43 Branch Prediction Unit Algorithm bits, 2­43 BSIZE, 7­22 BSTATP, 7­31 BSTATPE, 7­24 BTAGP, 7­31 BTAGPE, 7­24 Bus Error Command Register, 7­14 Bus Error Register, 7­6 Bus Error Syndrome Register, 7­12 Index­2 B­Cache Data Double­Bit Error bit, 7­24 B­Cache Data Single­Bit Error bit, 7­24 B­cache description, 3­1, 3­5 B­Cache emulation of main memory, 7­28 B­Cache Enable bit, 2­80 B­Cache Error Tag Register, 2­85 B­Cache Force Hit bit, 2­80 B­cache force hit mode, 3­11 B­cache operating modes, 3­11 B­Cache Size bits, 2­79, 7­22 B­Cache Speed bit, 2­80 B­cache states, 3­5 B­cache state changes, 3­6 B­cache status parity error, 9­45 B­Cache Status Store Parity Error bit, 7­24 B­Cache Tag Control Parity Error bit, 2­90 B­cache tag parity error, 9­45 B­Cache Tag Parity Error bit, 2­90 B­Cache Tag Store Parity Error bit, 7­24 B­cache, introduction, 1­4 B­map, 3­9 B­Map Parity bit, 7­31 B­Map Parity Error bit, 7­24 B­map parity parity error, 9­46 B­Stat Parity bit, 7­31 B­Tag Parity bit, 7­31 C CA, 7­14, 7­15 Cache backmaps, 3­8 Cache coherence, error handling, 9­5 Cache disable procedure, 9­6 Cache enable procedure, 9­6 Cache flush procedure, 9­6 Cache initialization, 3­11 Cache organization, 3­2 Cache test procedures, 9­7 CACK_H error on PTE read, 9­30 CAE, 7­7 CA error, 9­47 Cbox, 2­29 Cbox Registers, 2­72 CDPE2, 7­8 CE, 7­8 CEEN, 7­9 CE2, 7­8 CHALT, 2­96 CID, 7­15 CID3, 7­15 Clear Lock bit, 7­20 CLR_LOCK, 7­20 CMD, 2­64, 2­67, 7­15 CNF, 7­15 CNFE, 7­7 CNF error, 9­48 CNF Error bit, 7­7 Commander ID bits, 7­15 Commander ID 3 bits, 7­15 Command bits, 2­64, 2­67, 7­15 Command parity error, 9­49 Command Parity Error bit, 7­8 Command/Address bits, 7­14, 7­15 Command/Address Error bit, 7­7 Configuration Register, 7­9 Confirmation bit, 7­15 Console Base Address bits, 2­96 Console Communication Data 0 bits, 7­32 Console Communication Data 1 bits, 7­32 Console Communication Register, 7­32 Console dispatch data structure, 2­97 Console entry, 8­3 Console halt, 9­10 Console halt codes, 9­12 Console Halt Register, 2­96 Console halt, CPU state, 9­13 Console hardware, 5­1 Console program invocation, 5­5 Console registers, 5­5 Console saved PBC, 9­11 Console saved PSL, 9­11 Controller, 4­5 Controller, LSB, 4­5 Control Transmit Check Error bit, 7­7 CONWIN Low bit, 5­10 CONWIN_L, 5­10 CON_BASE_ADDR, 2­96 CON_COM_DATA0, 7­32 CON_COM_DATA1, 7­32 Correctable Data Error bit, 7­8 Correctable ECC error, 9­49 Correctable Error Detection Enable bit, 7­9 Corrected Read bit, 2­89 CPE2, 7­8 CPUID, 2­36 CPU chip functional units, 1­3 CPU Identification bits, 2­36 CPU Identification Register, 2­36 CPU interrupt levels, 6­4 CPU module block diagram, 1­2 register list, 7­3 self­test, 8­2 CPU registers Bus Error, 7­6 Bus Error Command, 7­14 Bus Error Syndrome, 7­12 Configuration, 7­9 Console Communication, 7­32 Device, 7­5 Interprocessor Interrupt, 7­18 I/O Interrupt, 7­16 Index­3 Last Miss Address, 7­38 Lock Address, 7­25 LSB Diagnostic Control, 7­26 Memory Mapping, 7­10 Mode, 7­20 Module Error, 7­23 Performance Counter, 7­37 Performance Counter Control, 7­33 Tag Address, 7­29 Tag Write Data, 7­30 CPU self­test, 8­2 CPU state at console halt, 9­13 CPU Type bits, 2­37 CPU0, 7­16 CPU0 I/O Interrupt bits, 7­16 CPU1 I/O Interrupt bits, 7­16 CPU2, 7­16 CPU2 I/O Interrupt bits, 7­16 CPU3, 7­16 CPU3 I/O Interrupt bits, 7­16 CPU4, 7­16 CPU4 I/O Interrupt bits, 7­16 CPU5, 7­16 CPU5 I/O Interrupt bits, 7­16 CPU_ID, 2­36 CPU_TYPE, 2­37 CSR data parity error, 9­49 CSR Data Parity Error bit, 7­8 CTCE, 7­7 Ctrl/P character detection, 5­4 Ctrl/P Halt bit, 5­15 Ctrl/P Halt Enable bit, 5­11 Ctrl/P_HALT, 5­15 D Data Cycle bits, 7­15 Data Parity bits, 2­71 Data Parity Error bit, 2­42, 2­65, 2­67 Data path, LEVI, 4­3 Data structure, mailbox, 6­1 Data Transmit Check Error bit, 7­7 Data types, 2­2 Data Valid bit, 2­40 DATA_P, 2­40 DATA_PARITY, 2­71 DATA_V, 2­40 DCYCLE, 7­15 Device interrupt handling, 6­4 Device Register, 7­5 Device Revision bits, 7­5 Device Type bits, 7­5 Diagnostic Control Register, 2­82 Diagnostic notes, 7­28 DIAG_CTL, 2­82 DIAG_CTL read/write, 2­83 DIE, 7­7 DIRTY, 7­15, 7­30 Dirty bit, 7­15, 7­30 Dirty error, 9­47 Dirty Error bit, 7­7 Disable ECC Error bit, 2­83 DIS_ECC_ERR, 2­83 DPERR, 2­42, 2­65, 2­67 DREV, 7­5 DTCE, 7­7 DTYPE, 7­5 Dual­ported access synchronization, 4­6 DUART0 Interrupt bit, 5­14 DUART0_INT, 5­14 DUART1 Interrupt bit, 5­14 DUART1_INT, 5­14 D­Stream Enable bit, 2­69 D_STR_ENB, 2­69 E E, 7­8 Ebox and microsequencer, 2­27 Ebox Control Register, 2­48 Ebox registers, 2­45 Ebox stage 3 stall timeout error, 9­25 ECC, 2­80 ECC Error bit, 2­89 ECR, 2­48 ECR­Ebox, 2­48 EEPROM, 5­3 Electrically Disable bit, 2­68 ELEC_DISABLE, 2­68 Emulated instruction exceptions, 2­13 EM Latch Valid bit, 2­65 EM_VAL, 2­65 EN, 7­11 Enable bit, 7­11 Environment error sources, 9­45 ERR, 2­73 Error Address bits, 2­39 Error analysis, 9­4 Error bit, 2­73, 7­8 Error categories, 9­44 Error categories, SBC entry points, 9­10 Error Correction and Control bit, 2­80 Error handling cache coherence, 9­5 software, 9­1 Error line asserted, 9­50 Error recovery, 9­4 Error reports, 9­9 Error retry, 9­7 Error sources, environment, 9­45 Error state collection, 9­2 Event Count 0 bits, 7­37 Index­4 Event Count 1 bits, 7­37 EV_COUNT0, 7­37 EV_COUNT1, 7­37 Exceptions, 2­10 arithmetic, 2­11 system failure, 2­14 Exceptions and interrupts, 2­9 Exception, kernel stack not valid, 9­43 Expanded stack frame, 2­10 Expander Select bits, 5­19 EXPSEL, 5­19 External interrupt requests, 2­16 Extracting data, B­cache, 9­6 F FAULT, 2­62 Faults, 2­10 Fault bits, 2­62 FBCP, 7­27 FBDP, 7­27 Fbox, 2­28 Fbox Enable bit, 2­49 Fbox Stage 4 Bypass Enable bit, 2­49 FBOX_ENB, 2­49 FBOX_ST4_, 2­49 FBOX_TEST_ENB, 2­49 FDBE, 7­27 FDIRTY, 7­27 FEPROMs, 5­3 Fill Address Register, 2­94 Fill Address <33:32> bits, 2­87 Fill buffer, 4­3 Fill Dispatch Command bits, 2­88 Fill Error Physical Address bits, 2­94 Fill I­Cache Read bit, 2­89 Fill Quadword bit, 2­88 Fill SEO bit, 2­88 Fill Syndrome Register, 2­92 FILL uncontrollable ECC errors, 9­26 FILL uncontrollable ECC errors, P­cache, 9­42 FILL_ADDR, 2­94 FILL_ADDR<33:32>, 2­87 FILL_CRD, 2­89 FILL_DPERR, 2­89 FILL_DSP_CMD, 2­88 FILL_ECC, 2­89 FILL_ERR_PA, 2­94 FILL_IRD, 2­89 FILL_QW, 2­88 FILL_SEO, 2­88 FILL_SYND, 2­92 Flush Branch History bit, 2­43 Flush Center bit, 2­43 FLUSH_BHT, 2­43 FLUSH_CTR, 2­43 Flux Testability Enable bit, 2­49 Force Bad Data Parity bit, 7­27 Force Dirty bit, 7­27 Force Double­Bit Error bit, 7­27 Force hit mode, 9­6 Force hit mode, 3­11 Force LSB Ignore bit, 7­27 Force P­Cache Hit bit, 2­69 Force Share bit, 7­27 Force Single­Bit Error bit, 7­27 FORCE_HIT, 2­69 FRIGN, 7­27 FSBE, 7­27 FSHARE, 7­27 Functional partitions, 2­25 G Gbus components, 5­2 Gbus$Halt, 5­15 Gbus$Intr, 5­13 Gbus$LEDs, 5­10 Gbus$LSBRST, 5­17 Gbus$LTagRW, 5­21 Gbus$Misc, 5­18 Gbus$PMask, 5­11 Gbus$RMode, 5­20 Gbus$WHAMI, 5­8 General purpose registers, 2­30 Get buffer, 4­3 H Halt Enable bit, 5­12 Halt interrupt, 9­10 Halt protection, 5­4 HALT_EN, 5­12 Hard error interrupts, 9­32 Hard error interrupt parse tree, 9­32 HI, 2­92, 2­95 High bits, 2­92, 2­95 HISTORY, 2­43 History bit, 2­43 I IA, 7­11 Ibox, 2­26 Ibox Control and Status Register, 2­42 Ibox registers, 2­38 ICCS, 2­73 ICR, 2­76 ICSR, 2­42 Identification registers, 2­35 IE, 2­73 Inconsistent status, hard error interrupts, 9­40 Inconsistent status, machine checks, 9­31 Index­5 Initialization cache, 3­11 overview, 8­1 Instruction set, 2­2 INT, 7­11 Interfacing rules, 4­6 Interleave Address bits, 7­11 Interleave bits, 7­11 Internal interrupt requests, 2­16 Internal processor registers discussion, 2­31 list, 2­33 Interprocessor bit, 5­13 Interprocessor interrupt, 7­19 Interprocessor Interrupt Mask bits, 7­18 Interprocessor Interrupt Register, 7­18 Interrupts, 2­15 hard error, 9­32 soft error, 9­40 Interrupt bit, 2­73 Interrupt conditions, 6­4 Interrupt control registers, 2­15 Interrupt Enable bit, 2­73 Interrupt handling, device, 6­4 Interrupt levels, 6­4 Interrupt mapping, 7­16 Interrupt requests external, 2­16 internal, 2­16 software, 2­17 Interrupt, interprocessor, 7­19 Interval Count bits, 2­76 Interval Count Control and Status Register, 2­73 Interval Count Register, 2­76 Interval Timer bit, 5­13 INTIM, 5­13 INTR, 2­73 INT_COUNT, 2­76 IO_MAP, 2­79 IP, 5­13 IPL, 7­17 IPR address space decoding, 2­32 I­Stream Enable bit, 2­69 I/O Interrupt Register, 7­16 I/O Map bits, 2­79 I/O operation registers, 6­4 I_STR_ENB, 2­69 K KA7AA block diagram, 1­2 KA7AA internal processor registers, 2­33 Kernel stack not valid exception, 9­43 L LADR, 7­25 Last Miss Address Register, 7­38 LBECR, 7­14 LBER, 7­6 LBESR, 7­12 LCNR, 7­9 LCNTR, 7­37 LCNTR0 Halt bit, 7­34 LCNTR0 Overflow bit, 7­36 LCNTR0 Run bit, 7­34 LCNTR0 Select bits, 7­35 LCNTR1 Halt bit, 7­33 LCNTR1 Overflow bit, 7­36 LCNTR1 Run bit, 7­33 LCNTR1 Select bits, 7­34 LCON, 7­32 LC0_HLT, 7­34 LC0_OVFL, 7­36 LC0_RUN, 7­34 LC0_SEL, 7­35 LC1_HLT, 7­33 LC1_OVFL, 7­36 LC1_RUN, 7­33 LC1_SEL, 7­34 LDC Power Okay bit, 5­16 LDC_PWR_OK, 5­16 LDEV, 7­5 LDIAG, 7­26 LEDs Low bits, 5­10 LEDs_L, 5­10 Left Bank Tag Error bit, 2­67 LEFT_BANK, 2­67 Length Violation bit, 2­62 LEVI, 4­4 LEVI address path, 4­2 LEVI block diagram, 4­2 LEVI B­cache status parity error, 9­45 LEVI B­cache tag parity error, 9­45 LEVI B­map parity parity error, 9­46 LEVI controllers, 4­4 LEVI data controller, 4­4 LEVI data path, 4­3 LEVI errors, 9­45 LEVI processor controller, 4­4 LEVI P­map parity parity error, 9­45 LEVI Revision bit, 7­20 LEVI transactions, 4­7 LEVI_REV, 7­20 LIOINTR, 7­16 LIPINTR, 7­18 LLOCK, 7­25 LMBOX, 6­5 LMBPR, 6­4 LMBPR Address bits, 6­3 Index­6 LMBPR_ADDR, 6­3 LMERR, 7­23 LMISSADDR, 7­38 LMMR, 7­10 LMODE, 7­20 LO, 2­92, 2­95 Load History bit, 2­43 LOAD_HISTORY, 2­43 LOCK, 2­42, 2­61, 2­65, 2­67, 7­25 Lock Address bit, 7­25 Lock Address Register, 7­25 Lock bit, 2­42, 2­65, 2­67, 7­25 Lock bits, 2­61 Lock Mode bits, 7­21 LOCK_MODE, 7­21 Logic boxes, 2­25 Longword Select bit, 2­39 Lost BIU error, 9­28 Lost fill error, 9­27 Lost Write bit, 2­87 LOST_WRITE, 2­87 Low bits, 2­92, 2­95 LPERF, 7­33 LSB, 4­5 LSB arbitration, 4­6 LSB arbitration collision, 9­46 LSB arbitration drop, 9­46 LSB Bad bit, 5­9 LSB CA error, 9­47 LSB CNF error, 9­48 LSB command field encodings, 4­7 LSB command parity error, 9­49 LSB controller, 4­5 LSB CONWIN bit, 5­9 LSB correctable ECC error, 9­49 LSB CSR data parity error, 9­49 LSB Diagnostic Control Register, 7­26 LSB Dirty error, 9­47 LSB errors, 9­46 LSB error line asserted, 9­50 LSB interface, introduction, 1­4 LSB interrupt level, 7­17 LSB Mailbox Register, 6­5 LSB node space base addresses, 7­2 LSB Secure bit, 5­16 LSB Shared error, 9­47 LSB stall error, 9­48 LSB transmitter durring error, 9­49 LSB transmit check error, 9­47 LSB uncorrectable ECC error, 9­50 LSB 0 bit, 5­14 LSB 1 bit, 5­14 LSB 2 bit, 5­13 LSB­initiated transactions, 4­8 LSB1, 5­14 LSB2, 5­13 LSB_BAD, 5­9 LSB_CONWIN, 5­9 LSB_SEC, 5­16 LTAGA, 7­29 LTAGW, 7­30 LV, 2­62 LW, 2­39 M M, 2­62 Machine check codes, 9­16 exception stack frame, 9­14 inconsistent status, 9­31 parse tree, 9­17 Machine checks, 9­13 Mailbox data structure, 6­1 operation, 6­3 pointer CSR, 6­4 pointer structure, 6­3 Mailbox Address bits, 6­3 Mailbox Register bits, 6­5 Manufacturing status bit, 5­9 Mapping address space, 4­7 interrupt, 7­16 MASK, 7­18 MA_FREQ, 7­36 Mbox, 2­28 Mbox Map Enable Register, 2­57 Mbox P0 Base Register, 2­51 Mbox P0 Length Register, 2­52 Mbox P1 Base Register, 2­53 Mbox P1 Length Register, 2­54 Mbox registers, 2­50 Mbox System Base Register, 2­55 Mbox System Length Register, 2­56 MBXREG, 6­5 MB_ADDR, 6­3 MCHK_ASYNC_ERROR, 9­24 MCHK_CANT_GET_HERE, 9­24 MCHK_INT.ID_VALUE, 9­24 MCHK_MOVC_STATUS, 9­24 MCHK_SYNC_ERROR, 9­25 MCHK_UNKNOWN_MSTATUS, 9­23 Memory emulation, B­cache, 7­28 Memory management control, 2­7 discussion, 2­4 Memory management control registers, 2­8 Memory Management Enable bits, 2­57 Memory Mapping Register, 7­10 MEM_MNG_ENB, 2­57 MFG, 5­9 Index­7 Microcode Patches bits, 2­37 Microcode Revision bits, 2­37 Microsequencer, 2­27 MIC_PATCHES, 2­37 MIC_REV, 2­37 Minimum stack frame, 2­9 MISPREDICT, 2­43 Mispredict bit, 2­43 Missed Address bits, 7­38 Miss Address Frequency bits, 7­36 MISS_ADDR, 7­38 MMAPEN, 2­57 MMEADR, 2­59 MMEPTE, 2­60 MMESTS, 2­61 MME Faulting Address bits, 2­59 MME Faulting Address Register, 2­59 MME PTE Address Register, 2­60 MME Status Register, 2­61 MME_FAULT_ADDR, 2­59 Mode Register, 7­20 Modify bit, 2­62 Modify Fault PTE Address bits, 2­60 Module Address bits, 7­10 Module Error Register, 7­23 MODULE_ADDR, 7­10 MOD_FAULT_PTE_, 2­60 MP0BR, 2­51 MP0LR, 2­52 MP1BR, 2­53 MP1LR, 2­54 MSBR, 2­55 MSLR, 2­56 N NBANKS, 7­11 Next Interval count bits, 2­75 Next Interval Count Register, 2­75 NHALT, 5­16, 7­9 NICR, 2­75 NID, 5­9 NINT_COUNT, 2­75 Node Bank bits, 7­33 Node Halt bit, 5­16, 7­9 Node ID bit, 5­9 Node Reset bit, 7­9 Node space base addresses, 7­2 Node­Specific Error Summary bit, 7­7 Nonexistent Address Error bit, 7­7 Nonstandard Patch bit, 2­46 NONSTANDARD_PATCH, 2­46 NRST, 7­9 NSES, 7­7 Number of Banks bits, 7­11 NVAX+ functional units, 1­3 NXAE, 7­7 N_MASK, 7­33 O OE, 2­80 Operating system startup, 8­3 Output Enable bit, 2­80 Overview, CPU module, 1­1 P P, 7­15 Pack Disable bit, 2­83 PACK_DISABLE, 2­83 Page table entry format, 2­6 Page Table Entry Reference bit, 2­62 PAMODE, 2­58 Parallel Port Disable bit, 2­47 PARITY, 2­70 Parity bit, 7­15 Parity bit (1), 2­70 Parse tree hard error interrupt, 9­32 machine check exception, 9­17 soft error interrupt, 9­41 PAR_PORT_DIS, 2­47 Patchable Control Store Control Register, 2­46 Patchable Control Store Data bit, 2­46 Patchable Control Store Enable bit, 2­47 Patchable Control Store Write bit, 2­47 Patch Revision bits, 2­46 PATCH_REV, 2­46 PCACHE_MODE, 2­79 PCADR, 2­66 PCCTL, 2­68 PCDAP, 2­71 PCSCR, 2­46 PCSTS, 2­67 PCS_DATA, 2­46 PCS_ENB, 2­47 PCS_WRITE, 2­47 PCTAG, 2­70 PC_PE_ENB, 2­69 PC_PE_PA, 2­66 Performance Counter Control Register, 7­33 Performance Counter Register, 7­37 Performance Monitor Access Type bits, 2­82 Performance Monitor Facility Clear bit, 2­48 Performance Monitor Hit Type bits, 2­83 Performance Monitor Mode bit, 2­68 PHALT_EN, 5­11 Physical Address Mode bits, 2­58 Physical Address Mode Register, 2­58 Physical address space, 2­3 Physical base addresses, 7­2 PHYS_ADDR_MODE, 2­58 Index­8 PMAPP, 7­30 PMAPPE, 7­24 PMF EMUX bits, 2­48 PMF Enable bit, 2­49 PMF FMUX bits, 2­49 PMF Linear Feedback Shift Register bit, 2­48 PMF_CLEAR, 2­48 PMF_EMUX, 2­48 PMF_ENB, 2­49 PMF_LFSR, 2­48 PMF_PMUX, 2­49 PMM, 2­68 PMODE, 7­21 PM_ACCS_TYPE, 2­82 PM_HIT_TYPE, 2­83 Power Module A Okay bit, 5­15 Power Module B Okay bit, 5­15 Power supply connection codes, 5­19 Power­up test, 8­2 Primary cache, 3­1, 3­4 Processor status longword, 2­24 Processor­initiated transactions, 4­7 Process control block, 2­23 Process space address translation, 2­5 Process structure, 2­22 PSL, 2­24 PTE errors on reads, 9­30 PTE Error bit, 2­67 PTE Error Write bit, 2­67 PTE format, 2­6, 2­7 PTE read errors, 9­28 PTE read errors, interruptible instructions, 9­29 PTE_ER, 2­67 PTE_ER_WR, 2­67 PTE_REF, 2­62 PV System Mode bit, 2­79 PWR_MODA_OK, 5­15 PWR_MODB_OK, 5­15 P­cache, 3­1, 3­4 P­Cache Contrlol Register, 2­68 P­Cache Data Parity Register, 2­71 P­Cache Mode bit, 2­79 P­Cache Mode bits, 7­21 P­cache parity errors, 9­42 P­Cache Parity Error Address Register, 2­66 P­Cache Parity Error Enable bit, 2­69 P­Cache Parity Error Physical Address bits, 2­66 P­Cache Parity Error Status Register, 2­67 P­Cache Tag Register, 2­70 P­map, 3­9 P­Map Parity bit, 7­30 P­map parity error, 9­45 P­Map Parity Error bit, 7­24 P0 Length Longwords bits, 2­52 P0 region address translation, 2­5 P0_LENGTH_LW, 2­52 P1 Length Longwords bits, 2­54 P1 region address translation, 2­5 P1_LENGTH_LW, 2­54 Q Quadword I/O Read bit, 2­79 QW_IO_RD, 2­79 R Redundancy Enable bit, 2­68 RED_ENABLE, 2­68 Register addressing, UART, 5­4 Register descriptions, 7­4 Register mapping, 7­2 Registers BIU Control, 2­78 BIU Status, 2­87 BIU Address, 2­91 Branch Prediction Control, 2­43 B­Cache Error Tag, 2­85 Console Halt, 2­96 CPU Identification, 2­36 Diagnostic Control, 2­82 Ebox Control, 2­48 Fill Address, 2­94 Fill Syndrome, 2­92 Ibox Control and Status, 2­42 Interval Count, 2­76 Interval Count Control and Status, 2­73 Mbox Map Enable, 2­57 Mbox P0 Base, 2­51 Mbox P0 Length, 2­52 Mbox P1 Base, 2­53 Mbox P1 Length, 2­54 Mbox System Base, 2­55 Mbox System Length, 2­56 MME Faulting Address, 2­59 MME PTE Address, 2­60 MME Status, 2­61 Next Interval Count, 2­75 Patchable Control Store Control, 2­46 Physical Address Mode, 2­58 P­Cache Control, 2­68 P­Cache Data Parity, 2­71 P­Cache Parity Error Address, 2­66 P­Cache Parity Error Status, 2­67 P­Cache Tag, 2­70 Software ECC, 2­95 System Control Block Base, 2­18 System Identification, 2­37 Time­of­Day, 2­77 Translation Buffer Parity Address, 2­63 Translation Buffer Status, 2­64 Index­9 VIC Data, 2­41 VIC Memory Address, 2­39 VIC Tag, 2­40 Request Mode bits, 5­8 REQ_MODE, 5­8 Reset Status bit, 7­9 Revision Level bits, 2­98 REV_LEVEL, 2­98 Right Bank Tag Error bit, 2­67 RIGHT_BANK, 2­67 Row Index bits, 2­39 ROW_INDEX, 2­39 RSTSTAT, 7­9 RUN, 2­74 Run bit, 2­74 RUN Low bit, 5­10 RUN_L, 5­10 S SAVEPC register, 9­15 SCB, 2­18 Second Command Parity Error bit, 7­8 Second Correctable Data Error bit, 7­8 Second CSR Data Parity Error bit, 7­8 Second Uncorrectable Data Error bit, 7­8 Select control terminal bits, 5­12 Self­test description, 8­2 Self­Test Fail bit, 7­9 Self­Test Passed Low bit, 5­10 SEL_CONS_TERM, 5­12 Serial port, 5­3 Serial ROM, 5­2 SGL, 2­74 SHARED, 7­15, 7­30 Shared bit, 7­15, 7­30 Shared error, 9­47 Shared Error bit, 7­7 SHE, 7­7 SHIFT, 2­46 Shift bit, 2­46 SID, 2­37 Single Step bit, 2­74 Single­bit error syndromes, 2­93, 7­13 Software ECC bit, 2­82 Software ECC Error Register, 2­95 Software error handling, 9­1 Software interrupt requests, 2­17 Soft error interrupts, 9­40 Soft error interrupt parse tree, 9­41 Source bits, 2­61, 2­64 SPARE, 7­27 Spare bit, 7­27 SRC, 2­61, 2­64 SROM, 5­2 SROM operation, 8­2 Stack frame expanded, 2­10 minimum, 2­9 Stall buffer, 4­3 Stall error, 9­48 Stall Error bit, 7­7 STCOND_TO, 7­21 STE, 7­7 STF, 7­9 Store Conditional Timeout bits, 7­21 STP_L, 5­10 Structure, mailbox pointer, 6­3 SUBBLOCK, 2­39 Subblock Select bits, 2­39 SW_ECC, 2­82 Synchronization, 4­6 Synchronous errors, 9­44 Syndromes for single­bit errors, 7­13 Syndromes, single­bit errors, 2­93 Syndrome values, 7­13 Syndrome 0 bit, 7­12 Syndrome 1 bit, 7­12 Syndrome 2 bit, 7­12 Syndrome 3 bit, 7­12 SYND_0, 7­12 SYND_1, 7­12 SYND_2, 7­12 SYND_3, 7­12 System block diagram, 1­1 System control block, 2­18 System Control Block Base Register, 2­18 System control block layout, 2­20 System control block vector, 2­18 System environment errors, 9­44 System failure exceptions, 2­14 System Identification Register, 2­37 System Length Longwords bits, 2­56 System Page Table Physical Address bits, 2­55 System setup, boot processor, 8­3 System space address translation, 2­4 System Type bits, 2­98 System Variant bits, 2­98 System Virtual Address of P0 bits, 2­51 System Virtual Address of P1 bits, 2­53 SYS_PT_LENGTH_LW, 2­56 SYS_PT_PA, 2­55 SYS_TYPE, 2­98 SYS_TYPE parameters, 2­97 SYS_VAR, 2­98 SYS_VA_P0, 2­51 SYS_VA_P1, 2­53 T Tag, 2­40 TAG, 2­40, 2­70, 2­86 Index­10 TAGADR_P, 2­86 TAGCTL_D, 2­86 TAGCTL_P, 2­86 TAGCTL_S, 2­86 TAGCTL_V, 2­86 Tag Address bits, 7­29 Tag Address Parity bit, 2­86 Tag Address Register, 7­29 Tag bits, 2­70, 2­86 Tag Control Dirty bit, 2­86 Tag Control Parity bit, 2­86 Tag Control Shared bit, 2­86 Tag Control Valid bit, 2­86 Tag Data bits, 7­31 Tag Match bit, 2­86 Tag Parity, 2­40 Tag Parity Error bit, 2­42, 2­65 Tag read/write, 7­28 Tag Select bits, 7­26 Tag Write Data Register, 7­30 TAG_ADDR, 7­29 TAG_DATA, 7­31 TAG_MATCH, 2­86 TAG_SEL, 7­26 TBADR, 2­63 TBSTS, 2­64 TB parity error, 9­24 TB test procedures, 9­7 TDE, 7­7 Timeout Clock bit, 2­49 Timeout Occurred bit, 2­49 Timeout Test bit, 2­49 Time­of­Day bits, 2­77 Time­of­Day Register, 2­77 Time­of­Day Register Increment bit, 2­83 Time­of­Day Register Test bit, 2­83 TOD, 2­77 TODR, 2­77 TODR_INC, 2­83 TODR_TEST, 2­83 TO_CLOCK, 2­49 TO_OCCURRED, 2­49 TO_TEST, 2­49 TPERR, 2­42, 2­65 Transaction ordering, 4­9 Transfer bit, 2­74 Translation buffer, 2­7 Translation Buffer Parity Address Register, 2­63 Translation Buffer Status Register, 2­64 Transmitter during error, 9­49 Transmitter During Error bit, 7­7 Transmit check error, 9­47 Traps, 2­10 U UARTs, 5­3 UART register addressing, 5­4 UCE, 7­8 UCE2, 7­8 Uncorrectable Data Error bit, 7­8 Uncorrectable ECC error, 9­50 Uncorrectable errors, write or write­unlock, 9­39 V VALID, 7­30 Valid bit, 7­30 Valid bits, 2­70 VA_TB_PE, 2­63 VDATA, 2­41 VDATA read/write, 2­41 VIC, 3­1 Victim buffer, 3­9 VIC Access Enable bit, 2­42 VIC data bits, 2­41 VIC Data Register, 2­41 VIC Memory Address Register, 2­39 VIC parity errors, 9­26, 9­41 VIC Tag Register, 2­40 VIC_ACC_ENB, 2­42 VIC_DATA, 2­41 Virtual address space, 2­3 Virtual Address TB Parity Error bits, 2­63 Virtual instruction cache, 3­1, 3­3 VMAR, 2­39 VTAG, 2­40 W Watch chip, 5­4 WMODE, 7­22 Workstation I/O bit, 2­78 Write buffer, 4­3 Write Mode bits, 7­22 Write policy, 3­10 WS_IO, 2­78 X XFR, 2­74