Computer Organization and Design, Third Edition: The Hardware Software Interface, Third Edition (The Morgan Kaufmann Series in Computer Architecture and Design)

THIRD

EDITION

Computer Organization Design THE

HARD W ARE / SOFTWARE

INTERFACE

ACKNOWLEDGEMENTS Figures 1.9, 1.15 Courtesy of Intel.

Computers in the Real World:

Figure 1.11 Courtesy of Storage Technology Corp.

Photo of "A Laotian villager," courtesy of David Sanger.

Figures 1.7.1, 1.7.2,6.13.2 Courtesy of the Charles Babbage Institute, University of Minnesota Libraries, Minneapolis.

Photo of an "Indian villager," property of Encore Software, Ltd., India.

Figures 1.7.3, 6.13.1, 6.13.3,7.9.3,8.11.2 Courtesy of IBM. Figure 1.7.4 Courtesy ofCray Inc. Figure 1.7.5 Courtesy of Apple Computer, Inc.

Photos of "Block and students~ and "a pop-up archival satellite courtesy of Professor Barbara Block. Photos by Scott T.1ylor. Photos of "Professor Dawson and student" and "the Mica courtesy of AP/World Wide Photos.

tag,~

micromote, ~

Figure 7.33 Courtesy of AMD.

Photos of "irn.1ges of pottery fragments" and "a computer reconstruction,~ courtesy of Andrew Willis and David B. Cooper, Brown University, Division of Engineering.

Figures 7.9.1, 7.9.2 Courtesy of Museum of Science, Boston.

Photo of "the Eurostar TGV train," by los van der Kolk.

Figure 7.9.4 Courtesy of MIPS Technologies, Inc.

Photo of "the interior of a Eurostar TGV cab," by Andy Veitch.

Figure 8.3 e Peg Skorpinski.

Photo of "firefighter Ken Whitten," courtesy of'W"orld Economic Forum.

Figure 8.11.1 Courtesy of the Computer Musru.m of America.

Graphic of an "artificial Reprinted by permission.

Figure 1.7.6 Courtesy of the Computer History Museum.

Figure 8.11.3 Courtesy of the Commercial Computing Museurll. Figures 9.11.2, 9.11.3 Courtesy of NASA Ames Research Center. Figure 9.11.4 Courtesy of Lawrence Livermore National Laboratory.

retina,~

Cl The San Francisco Chronicle.

Image of "A laser scan of Michelangelo's statue of David," courtesy of Muc Levoy and Dr. Franca Falletti, director of the Galleria dell 'Accademia, Italy. "An image from the Sistine Chapel," courtesy of Luca Feuati. IR image

recorded using the scanner for IR reflectography of the INOA (National Institute for Applied Optics, http://arte.ino.it ) at the Opificio delle Pietre Dure in Florence.

T H I R D

EDITION

Computer Organization and Design THE

HARDWARE / SOFTWARE

INTERFACE

David A. Patterson University of California. Berkeley

John L Hennessy Stanford University

With a co ntribution by Peter J. Ashenden Ashenden Designs Pty Ltd

James R. Lams Microsoft Research

Daniel J. Sarin Duke Un iversity

AMSTERDAM ' BOSTON ' HEIDELBERG ' WNDON NEW YORK ' OXFORD ' PARIS ' SAN DIEGO SAN FRANCISCO ' SINGAPORE ' SYDNEY ' roJ,.'YO

ELSEVIER

Morgan Kaufmann is an imprint of Elsevier

MORGAN KAUFMANN PUBLISHERS

Senior Editor Publishing Services Manager Editorial Assistant Cover Design Cover and Chapter Illustration Text Design Composition TechnicallUustration Copyeditor Proofreader Indexer Interior printer Cover primer

Denise E. M. Penrose Simon Cnullp Summer Block Ross Caron Design ChrisAsimoudis GGS Book Services Nancy Logan and Dartmouth Publishing, Inc. Dartmouth Publishing, Inc. Ken DeUaPenta lacqui Brownstein Linda Buskus Courier Courier

Morgan Kaufmann Publishers is an imprint of Elsevier. 500 Sansome Street, Suite 400, San Francisw, CA 94111 This book is printed on acid-free paper. Cl 2005 by Elsevier Inc. All rights reserved. Designations used by companies to distinguish their products are often claimed as trademarks or registered trademarks. In aU instances in which Morgan Kaufmann Publishers is aware of a claim, the product names appear in initial c.1pital or aU capital letters. Readers, however, should wntact the appropriate companies for more complete information reg.uding trademarks and registration. No part of this publication may be reproouced, stored in a retrieval system, or transmitted in any form or by any means------electronic, mechanical, photocopying, scanning, or otherwise-----without prior written permission of the publisher. Permissions may be sought directly from Elsevier's Science & Technology Rights Department in Oxford, UK: phone: (+44 ) 1865843830, fax: (+44 ) 1865853333, e-mail: [emailprotected] also complete your request on-line via the Elsevier homepage (http://elsevier.wm) by selecting "Customer Support" and then "Obtaining Permissions." Library of Congress Cataloging-in-Publication Data Application submitted ISBN: 1-55860-604-1 For information on aU Morgan Kaufmann publications, visit our Web site at www.mkp.com. Primed in the United States of America 040506070854321

Contents

Preface

CHAPTERS

Computer Abstractions and Technology 1.1 1.2 1.3 1.4 1.5 1.6 1. 7 1.8

Introductio n 3 Below Your Program I I Under the Covers IS Real Stuff: Manufacturing Pentium 4 Chips 28 Fallacies and Pitfalls 33 Concluding Remarks 35 Historical Perspective and Further Reading 36 Exercises 36

COMPUTERS IN THE REAL WORLD Information Technology for the 4 Billion without IT 44

Instructions: Language of the Computer 2. 1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 2.10 2. I I 2. 12

Introduction 48 Operations of the Computer Hardware 49 Operands of the Computer Hardware 52 Representing Instructions in the Computer 60 Logical Operations 68 Instructions for Making Decisions 72 Supporting Procedures in Computer Hardware 79 Communicating with People 90 MIPS Addressing for 32-Bit Immediates and Addresses Translating and Starting a Program 106 How Compilers Optimize 116 How Compilers Work: An Introduction 121

Contents

2. 13 2. 14 2.1 5 2.1 6 2.1 7 2.18 2. 19 2.20

COMPUTERS IN THE REAL Helping Save Our Environment with Data 156

Arithmetic for Computers 3. 1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3. 11

ACSort Example to Put It All Together 12 1 Implementing an Object-Oriented Language 130 Arrays versus Pointers 130 Real Stuff: IA-32 Instructions 134 Fallacies and Pitfalls 143 Concluding Remarks 145 Historical Perspective and Further Reading 147 Exercises 147 WORLD

158

Introduction 160 Signed and Unsigned Numbers 160 Addition and Subtraction 170 Multiplication 176 Division 183 Floating Point 189 Rea l Stuff: Floating Point in the IA-32 217 Fallacies and Pitfalls 220 Concluding Remarks 225 Historical Perspective and Further Reading 229 Exercises 229

COMPUTERS IN THE REAL Reconstructing the Ancient World 236

WORLD

Assessing and Understanding Performance 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8

238

Introduction 240 CPU Perfonnance and Its Factors 246 Evaluating Performance 254 Rea l Stuff: Two SPEC Benchmarks and the Performance of Recent Intel Processors 259 Fallacies and Pitfalls 266 Concluding Remarks 270 Historical Perspective and Further Reading 272 Exercises 272

COMPUTERS IN THE REAL Moving People Faster and More Safely 280

WORLD

.11

Contents

The Processor: Datapath and Control

....

5. 1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 5. II 5. 12 5.13

282

Introductio n 284 Logic Design Conventio ns 289 Building a Datapath 292 A Simple Implementation Scheme 300 A Multicycle Implementatio n 3 18 Exceptions 340 Microprogramming: Simplifying Co ntrol Design 346 An Introductio n to Digital Design Using a Hardware Design Language 346 Real Stuff: The O rganization of Recent Pentium Implementations 347 Fallacies a nd Pitfalls 350 Concluding Remarks 352 Historical Perspective and Further Reading 353 Exercises 354

COMPUTERS IN THE Empowering the Disabled 366

REAL

WORLD

Enhancing Performance with Pipelining 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 6.10 6.11 6. 12 6. 13 6.14

368

An OverviewofPipelining 370 A Pipelined Datapath 384 Pipelined Cont rol 399 Data Hazards and Forwarding 402 Data Hazards and Stalls 41 3 Branch Hazards 416 Using a Hardware Description Language to Describe and Model a Pipeline 426 Exceptions 427 Adva nced Pipelining: Extracting More Performance 432 Real Stuff: The Pentium 4 Pipeline 448 Fallacies and Pitfalls 451 Concluding Remarks 452 Historical Perspective and Further Reading 454 Exercises 454

COMPUTERS IN THE REAL WORLD Mass Communication without Gatekeepers 464

vIII

Contents

Large and Fast: Exploiting Memory Hierarchy

466

7.1 7.2 7.3 7.4 7.5 7.6

Introduction 468 The Basics of Caches 473 Measuring and Improving Cache Performance 492 Virtual Memory 511 A Common Framework for Memory Hierarchies 538 Real Stuff: The Pentium P4 and the AMD Opteron Memory Hierarchies 546 7.7 Fallacies and Pitfalls 550 7.8 Concluding Remarks 552 7.9 Historical Perspective and Further Reading 555 7. 10 Exercises 555

COMPUTERS IN THE REAL Saving the World 's Art Treasures 562

WORLD

Storage, Networks, and Other Peripherals 8.1 8.2 8.3 8.4

564

Introduction 566 Disk Storage and Dependability 569 Networks 580 Buses and Other Connections between Processors, Memory, and I/O Devices 581 8.5 Interfacing I/O Devices to the Processor, Memory, and Operating System 588 8.6 I/O Performance Measures: Examples from Disk and File Systems 597 8.7 Designing an I/O System 600 8.8 Real Stuff: A Digital Camera 603 8.9 Fallacies and Pitfalls 606 8.10 Concluding Remarks 609 8.11 Historical Perspective and Further Reading 6 11 8.12 Exercises 6 11

COMPUTERS IN THE REAL Saving Uves through Better Diagnosis 622

Multiprocessors and Clusters 9. 1 9.2 9.3

WORLD

9-2

Introduction 9-4 Progra mming Multiprocessors 9-8 Multiprocessors Connected by a Single Bus

9-11

Contents

9.4 9.5 9.6 9.7 9.8 9.9 9. 10 9. I I 9. 12

Multiprocessors Connected by a Network 9-20 Clusters 9-25 Network Topologies 9-27 Multiprocessors Inside a Chip and Multithreading Real Stuff: The Coogle Cluster of PCs 9-34 Fallacies a nd Pitfalls 9-39 Concluding Remarks 9-42 Historical Perspective and Further Reading 9-47 Exercises 9-55

9-30

APPENDICES

Assemblers, Linkers, and the SPIM Simulator A. I A.2 A.3 AA A.5 A.6 A.7 A.8 A.9 A. IO A. I I A.I2

Int roductio n A-3 Assemblers A- I0 Linkers A- 18 Loading A- 19 Memory Usage A-20 Procedure Call Co nventio n A-22 Exceptio ns and Interrupts A-33 Input and Output A-38 SPIM A-40 MIPS R2000 Assembly Language A-45 Concluding Rema rks A-8 I Exercises A-82

The Basics of Logic Design 8.1 8. 2 8. 3 B.4 8. 5 B.6 B.7 B.8 8.9 8. 10 B. I I

B-2

Int roductio n B-3 Gates, Truth Tables, and Logic Equatio ns B-4 Combinational Logic B-8 Using a Hard ware Description Language B-20 Constructing a Basic Arithmetic Logic Unit B-26 Faster Additio n: Car ry Lookahead B-38 Clocks B-47 Memory Elements: Flip -fl ops, Latches, a nd Registers Memory Elements: SRAMs and DRAMs B-57 Finite State Machines B-67 Timing Methodologies B-72

B-49

A-2

Contents

B.12 Field ProgrJmlT1able Devices B.13 Concluding Remarks B-78 8.14 Exercises B-79

B-77

Mapping Control to Hardware CI C2 C3 CA C5 C6 C7

C·2

Int roduction C-3 Implementing Co mbinational Co nt rol Units C-4 Implementing Finite State Machine Control C-8 Implementing the Next-State Funct ion with a Sequencer TrJ nslating a Microprogra m to Hardware C-27 Concluding Remarks C-3 1 Exercises C-32

C-2 1

A Survey of RiSe Architectures for Desktop, Server, and Embedded Computers 1).2 D. I D.2 D.3 D.4 D.5 D.6 D.7 D.8 D.9 D.lO D. II D.12 D.13 D.14 D.15 D.16 D.17 D.18 D.19 Index

tel

Int roduction D-3 Addressing Modes a nd Instruction Formats D-5 Instructions: Th e MIPS Core Subset D-9 Instructions: Multimedia Extensions of the Desktop/Server RISCs Instructions: Digital Signal-Processing Extensions of the Embedded RI SCs D- 19 Instructions: Common Extensions to MIPS Co re D-20 Instructions Unique to MIPS64 D-25 Instructions Unique to Alpha D-27 Instructions Unique to SPARC v.9 D-29 Instructions Unique to PowerPC D-32 Instructions Unique to PA- RI SC 2.0 D-34 Instructions Unique to ARM D-36 Instructions Unique to Thumb D-38 Instructions Unique to SuperH D-39 Instructions Unique to M32 R D-40 Instructions Unique to MIPSI6 D-41 Concluding Remarks D-43 Acknowledgments D-46 References D-47

I- I

Glossary G- I Further Reading

FR- I

D-16

Preface

Tile most beautiful tiling we call experience is the mysterious. It is tile source ofall true art and sciwce. Albert Einstein, What I Believe, 1930

About This Book We believe that lea rning in computer science and engineering should refl ect the current state of the field , as well as introduce the principles that are shaping com puting. We also feel that readers in every specia lty of computing need to appreciate the organizational paradigms that determine the capabilities, performance, and , ultimately, the success of computer systems. Modern computer technology requires professionals of every computing specialty to understand both hardwa re and softwa re. The interaction between hard wa re and softwa re at a va riety of levels also offers a fram ewo rk for understanding the fundam entals of computing. Whether your primary interest is hardwa re or software, computer science or electrical engineering, the central ideas in computer o rga nization and design are the sa me. Thus, our emphasis in this book is to show the relationship between hardware and softwa re and to focus on the concepts that are the basis for current computers. The audience for this book includes those with little experience in assembly language or logic design who need to understand ba sic computer organization as well as readers with backgrounds in assembly language and/or logic design who wa nt to lea rn how to design a computer or understand how a system works and why it performs as it does.

About the Other Book Some readers may be familiar with Computer Architecture: A Quantitative Approach, popularly known as Hennessy and Patterson. (This book in turn is called Patterson and Hennessy.) Our motivation in writing that book was to describe the principles of computer architecture using solid engineering funda -

xII

Preface

mentals and quantitative cost/ performance trade-offs. We used a ll approach that combined exam ples and measu rements, based on commercial systems, to create realistic design experiences. Our goal was to demonstrate that computer architecture could be lea rned using quantitative methodologies instea d of a descriptive approach. It is intended for the serious computing professional who wants a detailed understanding of computers. A majority of the rea ders for this book do not plan to become com puter architects. The performance of future softwa re systems will be dramatically affected , however, by how well softwa re designers understand the basic hardwa re tech niques at wo rk in a system. Thus, compiler writers, operating system designers, database programmers, and m ost other softwa re engineers need a firm grounding in the principles presented in this book. Similarly, hardwa re designers must understand clea rly the effects of their work on softwa re applications. Thus, we knew that this book had to be much more than a subset of the material in Computer A rchitecture, and the material was extensively revised to match the different audience. We were so happy with the result th at the subsequent editions of Compl/ter Architecture were revised to remove most of the introductory material; hence, there is much less overlap today than with the first editions of both books.

Changes for the Third Edition We had six major goals for the third edition of Computer Organization and Design: make the book work equally well for rea ders with a software focus or with a hard wa re focus; improve pedagogy in general; enhance understanding of program performance; update the technical content to refl ect changes in the industry sin ce the publication of the second edition in 1998; tie the ideas from the book mo re closely to the real world outside the computing indust ry; and reduce the size of this book. First, the table on the next page shows the hardware and software paths th rough the material. Chapters 1, 4, and 7 are found on both paths, no matter what the experience or the focus. Chapters 2 and 3 are likely to be review material for the hard wa re-oriented , but are essential rea ding for the software-oriented, especia lly for those readers interested in lea rning more about compilers and object-oriented programming languages. The first sections of Chapters 5 and 6 give overviews for those with a software focus. Those with a hardware focus, however, will find that these chapters present core material; they may also, depending on background , want to read Appendix B on logic design first and the sections on microprogramming and how to use hardwa re description languages to specify control. Chapter 8 on input/output is key to readers with a software focus and should be read if time permits by others. The last chapter on multip rocessors and clusters is aga in a question of time for the reader. Even the history sections show this balanced focus; they include short histories of programming languages, compilers, numerical softwa re, operating systems, networking protocols, and databases.

xiII

Preface

Chapter or Appendix

Sections

Software Focus Hardware Focus

•• OG{

1.1 to 1.6

1. Computer Abstraelions and Technology

1& 1.7 (History) 2.1t02.11 •

2.12 (Compilers)

2.13 (C sort)

2. Instructions: Language of the Computer

til 2.14 (Java) til 2.19 (History)

Ar~hmetic

for Computers

O. RISC instruction set

••

3.1t03.11

arch~ectures

til 3.12 (History) til 0.1 to 0.19

4. Assessing and Understanding Performance

4.1 to 4.6

B. The Basics of Logic Design

II B.1to B.13

DC{

til 5.8 (Microcode) til 5.9 (Verilog)

II 6.7 (verilog) 6.8 to 6.9

••

til 6.13 (History)

OU(

•• ••

7.1 to 7.8

aC(

. . 7.9 (History) 8.1 to 8.2

OC(

. . 8.3 (Networks)

••

8.4t08.10

00{

. . 8.13 (History)

A. Assemblers, Linkers, and

dc{

•• •• •• ••

••

6.10t06.12

9. Multiprocessors and Clusters

ao:

OC{

OC(

C.1toC.6

6.2 to 6.6

8. Storage, NetltOrKs, and Other Peripherals

•• OU(

6.1 (Overview)

7. Large and Fast: Exploiting Memory Hierarchy

••

. . 5.13 (History)

6. Enhancing Performance with Pipelining

DC{

5.10t05.12

•• OC{

. . 4.7 (History)

5.2 to 5.7

C. Mapping Control to Hardware

DC(

5.1 (Overview)

5. The Processor: Datapath and Control

dc(

•• aO!:

2.15t02.18

•• ••

•• PG{

•• •• •• •• •• DC{

••

OC(

PC{ PC{ DC{ 00{

9.1t09.10

OO!:

. . 9.11 (History)

til A.1toA.12

the SPIM Simulator

Read carefully Review or read

•• •

DC{

Between Chapters

Computers in the Real Worfd Read

have time

Read for cullure

DC(

Reference

OQ jQC(

xlv

Preface

The next goal was to improve the exposition of the ideas in the book, based on difficulties mentioned by readers of the second edition. We added five new book elements to help. To make the book work better as a reference, we placed definitions of new terms in the margins at their first occurrence. We hope this will help readers find the sections when they wa nt to refer back to material they have already read. Another change was the in sertion of the "Check Yourself" sections, which we added to help readers to check their comprehension of the material on the first time through it. A third change is that added extra exercises in the " For More Pra ctice" section. Fourth, we added the answers to the "Check Yourself" sections and to the For More Practice exercises to help readers see for themselves if they understand the material by comparing their answers to the book. The final new book element was inspired by the "Green Card" of the IBM System/360. We believe that you will find that the MIPS Reference Data Card will be a handy reference when writing MIPS assembly language programs. Our idea is that you will remove the card from the front of the book, fold it in half, and keep it in your pocket, just as IBM S/360 programmers did in the 1960s. Third, computers are so complex today that understanding the performance of a program involves understanding a good deal about the underlying principles and the organization of a given computer. Our goa l is that readers of this book should be able to understand the performance of their progams and how to improve it. To aid in that goal, we added a new book element called "Understanding Progra m Perform ance" in several chapters. These sections often give concrete exa mples of how ideas in the chapter affect performance of real programs. Fourth, in the interva l since the second edition of this book, Moore's law has marched onward so that we now have processors with 200 million transistors, DRAM chips with a billion transistors, and clock rates of multiple gigahertz. The " Real Stuff" exa mples have been updated to describe such chips. This edition also includes AMD64/1A-32e, the 64-bit address version of the long- lived 80x86 architecture, which appears to be the nemesis of the more recent IA-64. It also refl ects the transition from parallel buses to serial networks and switches. Later chapters describe Coogle, which was born after the second edition , in terms of its cluster techn ology and in novel uses of search. Fifth, although many computer science and engineering students enj oy information technology for technology's sake, some have more altruistic interests. This latter group tends to have more women and underrepresented minorities. Consequently, we have added a new book element, "Computers in the Rea l World ;' twopage layouts found between each chapter. Our perspective is that information techn ology is more va luable for humanity than most other topics you could study-whether it is preserving our art heritage, helping the Third Wo rld, saving our environment, or even changing political systems-a nd so we demonstrate our view with concrete examples of nontraditional applications. We think readers of these segments will have a greater appreciation of the computing culture beyond

Preface

the inherently interesting technology, much like those who read the histo ry sections at the end of each chapter Finally, books are like people: they usually get larger as they get older. By using techn ology, we have managed to do all the above and yet shrink the page count by hundreds of pages. As the table illustrates, the core portion of the book for hardwa re and softwa re readers is on paper, but sections that some readers would value more than others are found on the companion CD. This technology also allows your authors to provide longer histories and more extensive exercises without concerns about lengthening the book. Once we added the CD to the book, we could then include a grea t deal of free softwa re and tuto rials that many instructors have told us they would like to use in their courses. This hybrid paper-CD publication weighs about 30% less than it did six years ago-an impressive goa l for books as well as for people.

Instructor Support We have collected a great deal of material to help instructo rs teach courses using this book. Solutions to exercises, fi gures from the book, lecnlfe notes, lecture slides, and other materials are available to adopters from the publisher. Check the publisher's Web site for m ore information:

www . mkp . com/companions/15586060 41

Concluding Remarks If you read the following acknowledgments section, you will see that we went to great lengths to correct mistakes. Since a book goes through many printings, we have the oppornmity to make even more co rrections. If you uncover any remaining, resilient bugs, please contact the publisher by electronic mail at [emailprotected] or by low-tech mail using the address found on the copyright page. The first person to repo rt a technical error will be awarded a $1.00 bounty upon its implementation in futu re printings of the book! This book is truly collabo rative, despite one of us running a major university. Together we brainstormed about the ideas and method of presentation, then indi vidually wrote about one- half of the chapters and acted as reviewer for every draft of the other half. The page count suggests we again wrote almost exactly the same number of pages. Thus, we equally share the blame for what you are about to read.

Acknowledgments for the Third Edition We'd like to again express our appreciation to Jim Larus fo r his willingness in con tributing his expertise on assembly language programming, as well as for welcom ing readers of this book to use the simulator he developed and maintains. Our

xvi

Preface

exercise editor Dan Sorin took on the Herculea n task of adding new exercises and answers. Peter Ashenden worked similarly hard to collect and orga nize the com pa nio n CD. We are grateful to the many instructors who answered the publisher's surveys, reviewed our proposals, and attended focus groups to analyze and respond to our plans for this edition. They include the following individuals: Michael Anderson (University of Hartford), David Bader (University of New Mexico), Rusty Baldwin (Air Force Institute of Technology), John Barr (Ithaca College), Jack Briner (Charleston Southern University), Mats Brorsson (KTH , Sweden), Colin Brown (Franklin University), Lori Ca rter (Point La ma Nazarene University), John Casey (Northeastern University), Gene Chase (Messiah College), Geo rge Cheney (University of Massadm setts, Lowell), Daniel Citron (Jerusalem College of Technology, Israel), Albert Cohen (I NRIA, France), Lloyd Dickman (PathScale), Jose Duato (Universidad Politecnica de Valencia , Spain), Ben Duga n (University of Washington), Derek Eager (University of Saskatchewan , Ca nada), Magnus Ekman (Chalmers University of Technology, Sweden), Ata Elahi (Southern Connecticut State University), Soundararajan Ezekiel (I ndiana University of Pennsylvania), Ernest Ferguson (Northwest Missouri State University), Michael Fry (Lebano n Valley College, Pennsylvania), R. Gaede (University of Arkansas at Little Rock), Jean-Luc Gaudiot (University of Califo rnia, Irvine), Thom as Gendreau (University of Wisconsin, La Crosse), George Geo rgiou (Califo rnia State University, Sa n Bernardino), Paul Gillard (Mem o rial University of Newfoundland , Canada), Joe Grimes (Califo rnia Polytechnic State University, SLO), Max Ha ilperin (Gustavus Adolphus College), Jayantha Herath (St. Cloud State University, Minnesota), Mark Hill (University of Wisconsin , Madison), Michael H sa io (Virginia Tech), Richard Hugh ey (University of California, Santa Cruz), To ny Jeba ra (Columbia University), Elizabeth Johnson (Xavier University), Peter Kogge (University of Notre Dam e) , Morris Lancaster (BAH ), Doug Lawrence (University of Mo ntana), David Lilja (University of Minnesota), Nam Ling (Santa Clara University, Ca lifo rnia), Paul Lum (Agilent Technologies), Stephen Mann (University of Waterloo, Ca nada), Diana Marculescu (Carnegie Mellon University), Ma rga ret McMaho n (U.S. Naval Academy Com puter Science), Uwe Meyer-Baese ( Florida State University), Chris Milner (University of Virginia), Tom Pittman (Southwest Baptist University), Jalel Rejeb (San Jose State University, Ca lifo rnia), Bill Siever (University of Missouri, Rolla), Kevin Skadron (University of Virginia), Pam Sm allwood (Regis University, Colorado), K. Stuart Smith (Rocky Mountain College), William 1. Taffe (Plym o uth State University), Michael E. Thom odakis (Texas A&M University), Ru ppa K. Thulasiram (University of Manitoba, Canada), Ye Tung (University of South Alabam a), Steve Vander leest (Calvin College), Neal R. Wagner (University of Texas at San Anto nio), a nd Kent Wilken (University of California, Davis).

xvII

Preface

We are grateful too to th ose who ca refull y read our draft manuscripts; some read successive draft s to help ensure new errors didn't creep in as we revised. They include Krste Asa novic (Massachusetts In stitute of Technology), lea n-Loup Baer ( University of Washington ), David Brooks (Harva rd University), Doug Clark (Princeton University), Dan Connors (University of Colora do at Boulder), Matt Fa rrens ( University of California, Davis), Manoj Franklin (University of Maryland College Park), lohn Greiner (Rice University), David Harris (Harvey Mudd College), Paul Hilfinger (University of California, Berkeley), Norm louppi (Hewlett Packard), David Kaeli (No rtheastern University), David Oppenheimer ( University of Califo rnia, Berkeley), Timothy Pinkston (University of Southern California), Mark Smotherman (Clemson University), and David Wood (University of Wisconsin , Ma dison ). To help us meet our goal of crea ting 70% new exercises and solutions for this edition , we recruited several graduate students recommended to us by their professors. We are grateful fo r their creativity and persistence: Michael Black ( University of Ma ryland), Lei Chen (University of Rochester), Nirav Dave (Massachusetts Institute of Technology), Wael EI Essawy (University of Rochester), Nikil Mehta (Brown University), Nicholas Nelson (University of Rochester), Aaron Smith (University of Texas, Austin), and Charlie Wa ng ( Duke University). We would like to especia lly thank Mark Smotherman for making a ca reful fin al pass to find technical and writing glitches that significa ntly improved the quality of this edition. We wish to thank the extended Mo rga n Kaufmann family for agreeing to publish this book aga in under the able lea dership of Denise Penrose. She developed the vision of the hybrid paper-CD book and recruited the many people above who played important roles in developing the book. Simo n Crump managed the book production process, and Summer Block coordinated the surveying of users and their responses. We thank also the many freelance vendors who contributed to this volume, especia lly Nan cy Logan and Dartmo uth Publishing, Inc., our compositors. The contributions of the nea rly 100 people we mentioned here have made this third edition our best book yet. Enj oy!

David A. Patterson

John L. Hennessy

Computer Abstractions and Technology Civilization advances by extending the number of importan t operations which we can perform without thinking about them. Alfred North wtIltehead An Introduction to Mallrtmlltit:s, 1911

•

1.1 1.2

Below Your Program

1.3

Under the Covers

1.4

Real Stuff: Manufacturing Pentium 4 Chip.

1.5

Fallacies and Pitfalls

1.6

Concluding Remarks

1.7

Historical Perspective and Further Reading

1.8

Exercises

Introduction

Introduction Welcome to this book! We're delighted 10 have this opportunity to convey the excitement of the world of com puter systems. This is not a dry and drc::ary field , where progress is glacial and where new ideas atrophy from neglect. No! Computers are the product of the incredibly vibrant information techno logy industry, all aspects of which are responsible for almost 10% of th e gross nati onal product of th e United Sl3tes. This unusual industry embra ces innovatio n at a breathtaking ratc. Since 1985 there have been a number of new computers wh ose introdu ction appeared to revolutionize the computing industry; th ese revolutions were cut short onl y beca use so meone else built an even better computer. This race to innovate has led to unprecedented progress since the in ception of electronic computing in the late 1940s. Had the transpo rtatio n industry kept pace with the computer industry, for example, today we could travel coast to coast in ;Ibout ;1 second fo r roughly a few cents. Take just a moment to co ntemplate how such an improvement would change society- living in T;lhiti whi le wo rking in Sa n Fr;lllcisco, go ing to Moscow for an evening at the Bolshoi Ballet-a nd you ca n appreciate the implications of such a change.

Chapter 1

Computer Abstractions and Technology

Computers have led to :I third revolution for civilization , with the information revolution taking its place alongside the agricultural and the industrial revolutions. The resulting multiplication of humankind's intellectual strength and reach naturally has affected our everyday lives profoundly and also changed the ways in which the search for new knowledge is carried out. There is now a new vein of scientific investigation, with computational scientists joining theoretical and experimental scientists in the exploration of new frontiers in astronomy, biology, chemistry, physics, ... The computer revolution continues. Each time the cost of computing improves by another factor of 10, the opportunities for computers multiply. Applications that were economically infeasible suddenly become practical. In the recent past, the following applications were "computer science fi ctio n." • Automatic teller machines: A computer placed in the wall of banks to distribute and collect cash would have been a ridiculous concept in the 1950s, when the cheapest computer cost at least $500,000 and was the size of a car. • Computers in automobiles: Until microprocessors improved dramatically in price and performance in the ea rly 1980s, computer control of ca rs was ludi crous. Today, computers reduce pollution and improve fuel effici ency via engine controls and increase sa fety through the prevention of dangerous skids and through the inflatio n of air bags to protect occupants in a crash. • Laptop computers: \-VllO would have dreamed that advances in computer systems would lead to laptop computers, allowing students to bring com puters to coffeehouses and o n airplanes? • Human genome project: The cost of computer equipment to map and analyze hum an DNA sequences is hundreds of millio ns of dollars. It 's unlikely that anyone would have considered this project had the computer costs been 10 to 100 times higher, as they would have been 10 to 20 years ago. • World Wide Web: Not in existence at the time of the first edition of this book, the World Wide Web has transformed o ur society. Am o ng its uses are distributing news, sending fl owers, buying from online ca talogues, taking electronic tours to help pick vaca tio n spots, findin g o thers who share your esoteric interests, and even m ore mundane topics like findin g the lecture no tes of the authors of your textbooks. Clea rly, advances in this technology now affect almost every aspect of our society. Hardware adva nces have allowed programmers to create wonderfully useful software, and expla in why computers are o mnipresent. Tomorrow's science fi ctio n computer applications are the cashless society, automated intelligent high ways, and genuinely ubiquito us computing: no o ne carries computers because they are available everywhere.

1.1

Introduction

Classes of Computing Applications and Their Characteristics Although a common set of hardware technologies (discussed in Sections 1.3 and 1.4) is used in computers ranging from smart home appliances to cell phones to the largest supercomputers, these different applications have different design requirements and employ the core hardwa re technologies in different ways. Broadly speaking, computers are used in three different classes of applications. Desktop computers are possibly the best-known form of computing and are characterized by the personal computer, which most readers of this book have probably used extensively. Desktop computers emphasize delivering good performan ce to a single user at low cost and usually are used to execute third -pa rty software, also called shrink-wrap software. Desktop computing is one of the largest markets for computers, and the evolution of many computing technologies is driven by this class of computing, which is only about 30 years old ! Servers are the modern form of what was once mainframes, minicomputers, and supercomputers, and are usually accessed only via a network. Servers are oriented to carrying large workloads, which may consist of either single complex applications, usually a scientific or engineering application, or handling many small jobs, such as would occur in building a large Web server. These applications are oft en based on softwa re from another source (such as a database or simulation system ), but are often m od ified or customized for a particular functi on. Servers are built from the same basic technology as desktop computers, but provide for greater expa ndability of both computing and input/output capacity. As we will see in the Chapter 4, the performance of a server ca n be measured in several different ways, depending on the application of interest. In general, servers also place a greater emphasis on dependability, since a crash is usually more costly than it would be on a single-user desktop computer. Servers span the widest range in cost and capability. At the low end, a server may be little more than a desktop machine without a screen or keyboard and with a cost of a thousand dollars. These low-end servers are typica lly used for fil e storage, small business applications, or simple web serving. At the other extreme are supercomputers, which at the present consist of hundreds to thousands of processors, and usually gigabytes to terabytes of memory and terabytes to petabytes of storage, and cost millions to hundreds of millions of dollars. Supercomputers are usually used for high-end scientific and engineering calculations, such as weather forecasting, oil exploration , protein structure determination , and other large-scale problems. Although such supercomputers represent the peak of com puting capability, they are a relatively small fraction of the servers and a relatively small fraction of the overall computer market in terms of total revenue. Embedded computers are the largest class of computers and spa n the widest range of applications and performance. Embedded computers include the microprocessors found in your washing machine and ca r, the computers in a cell phone

desktop computer A computer designed for use by an individual, usually incorporating a graphics display, keyboard, and mouse.

Server A computer used for running larger programs for multiple users often simultaneously and typically accessed only via a network.

supercomputer A class of computers with the highest performance and cost; they are configured as servers and typically cost millions of dollars. terabyte Originally 1,099,511,627,776 (2 40 ) bytes, although some communications and secondary storage systems have redefined it to mean 1,000,000,000,000 ( 10 12 ) bytes. embedded computer A computer inside another device used for rulming one predetermined application or collection of software.

Chapter 1

Computer Abstractions and Technology

o r persollal digital assistant, the computers in a video ga me or digital television , and the networks of processors that control a modern airplane or ca rgo ship. Embedded computing systems are designed to run one application or one set of related applications, which is normally integra ted with the hardware and delivered as a single system ; thus, despite the large number of embedded computers, most users never really see that they are using a computer! Embedded applications oft en have unique application requirements that com bine a minimum performance with stringent limitations on cost or power. For exa mple, consider a cell phone: the processor need only be as fast as necessary to handle its limited function, and beyond that, minimizing cost and power are the most important objectives. Despite their low cost, embedded computers oft en have the least tolera nce for fa ilure, since the results ca n vary from upsetting (when your new television crashes) to devastating (such as might occur when the com puter in a plane or car crashes). In consumer-oriented embedded applications, such as a digital home appliance, dependability is achieved primarily through simplicity-the emphasis is on doing one function, as perfectly as possible. In large embedded systems, techniques of red undan cy, which are used in servers, are oft en employed. Although this book focuses on general-purpose computers, most of the concepts apply directly, or with slight m odifications, to embedded comput ers. In several places, we will touch on some of the unique aspects of embedded computers. Figure 1.1 shows that during the last several yea rs, the growth in the number of embedded computers has been much faster (40% compounded annual growth rate) than the growth rate among desktop computers and servers (9% annually). Note that the embedded computers include cell phones, video ga rnes, digital TVs and set-top boxes, personal digital assistants, and a variety of such consumer devices. Note that this data does not include low-end embedded control devices that use 8-bit and 16-bit processors. Elaboration: Elaborations are short sections used throughout the text to provide more detail on a particular subject, which may be of interest. Dis interested readers may skip over an elaboration, s ince the subsequent material will never depend on the contents of the elaboration . Many embedded processors are designed using processor cores, a version of a processor written in a hardware description language such as Veri log or VHDL. The core allows a designer to integrate other application-spec ific hardware with the processor core for fabrication on a sing le chip. The availability of synthesis tools that can generate a chip from a Verilog specification, together with the capacity of modern s ilicon chips, has made such special-purpose processors highly attractive . Since the core ca n be synthesized for different sem iconductor manufacturing lines, using a core provides fl exibility in choosing a manufacturer as well. In the last few years, the use of core s has

1.1

Introduction

11 22

892

862

135

4 2000

131

129

2001

2002

FIGURE 1.1 The number of distinct processors sold between 1998 and 2002. These counts are obtained somewhat diffefently, so some caution is fequired in interpfeting the results. For example, the totals fOf desktops and servers count complete computer systems, because some fraction of these include ntultiple pfocessors, the number of processors sold is somewhat highef, but probably by only 10--20% in total (since the sefvers, which ntay average mOfe than one proCessof per system, afe only about 3% of the desktop sales, which afe pfedominantly single-processof systents). The totals fOf entbedded computers actually count pfocessors, ntany of which are not even visible, and in sonte cases there ntay be multiple pfOCessors pef device.

been growing very fast. For example, in 1998 only 31% of the embedded processors were cores . By 2002, 56% of the embedded processors were cores . Furthermore, while the overall growth rate in the embedded market has been 40% per year, this growth has been primarily driven by cores, where the compounded annual growth rate has been 63%!

Figure 1.2 shows the major architectures sold in these markets with counts for each architecture, across all three types of products (embedded , desktop, and server). Only 32-bit and 64-bit processors are included, although 32-bit processors are the vast majority for most of the architectures.

Chapter 1

Computer Abstractions and Technology

FIGURE 1.2 Sales of microprocessors between 1998 and 2002 by Instruction set architecture combining all uses. The "other" category refers to processors that are either applicationspecific or customized architectures. In t he case of ARM, roughly 80% of the sales are for cell phones, where an ARM core is used in conjunction wit h application-specific logic on a chip.

What You Can Learn in This Book Successful programmers have always been concerned about the performance of their programs because getting results to the user quickly is critical in creating successful software. In the 1960s and 1970s, a primary constraint on computer performance was the size of the computer's memory. Thus progra mmers oft en followed a simple credo: Minimize memory space to make progra ms fast. In the last decade, advances in computer design and memory technology have greatly reduced the importance of small memory size in most applications other th an those in embedded computing systems. Programmers interested in perfo rmance now need to understand the issues that have replaced the simple memory m odel of the 1960s: the hierarchical nature of memories and the pa rallel nanlfe of processors. Programmers who seek to build competitive versions of compilers, operating system s, databases, and even applications will therefo re need to in crease their knowledge of computer orga ni zation.

1.1

Introduction

We are honored to have the opportunity to explain what's inside this revolutionary machine, unraveling the softwa re below your program and the hard wa re under the covers of your computer. By the time you complete this book, we believe you will be able to answer the following questions: • How are p rograms written in a high -level language, such as C or Java, translated into the langll3ge of the hardwa re, and how does the hardwa re execute the resulting program? Comprehending these concepts fo rms the basis of understanding the aspects of both the hardwa re and softwa re that affect program performance. • What is the interface between the softwa re and the hardwa re, and how does softwa re instruct the hardwa re to perform needed fun ctions? These con cepts are vital to understanding how to write many kind s of soft wa re. • What determines the perform ance of a p rogram , and how ca n a programmer improve the perfo rm ance? As we will see, this depends on the original program , the softwa re translation of that program into the computer's lan gll3ge, and the effectiveness of the hardwa re in executing the program. • What techniques ca n be used by hardwa re designers to improve performance? This book will introduce the basic concepts of modern computer design. The interested rea der will find much more material on this topic in our adva nced book, A Computer Architecture: A Qllantitlltive Approach . Without understanding the answers to these questions, improving the performance of your program on a modern computer, or eva luating what features might make one computer better than another for a particular application, will be a complex process of trial and error, rather than a scientific procedure driven by insight and analysis. This first chapter lays the foundation for the rest of the book. It introduces the basic ideas and definiti ons, places the major components of softwa re and hard wa re in perspective, and int roduces integrated ci rcuits, the techn ology that fuels the computer revolution. In this chapter, and later ones, you will likely see a lot of new words, or wo rd s th at you may have hea rd, but are not sure what they mea n. Don't panic! Yes, there is a lot of special termin ology used in describing modern computers, but the termin ology acnlally helps sin ce it enables us to describe precisely a fun ction or capability. In addition , computer designers (including your authors) love using acronyms, which are easy to understand once you know what the letters stand for! To help you remember and locate terms, we have included a highlighted definition of every term , the first time it appea rs in the text. Aft er a short time of working with the terminology, you will be flu ent, and your fri ends

acronym A word constructed by taking the initial letters of string of words. For example: RAM is an acronym for Random Access Memory, and CPU is an acronym for Central Processing Unit.

Chapter 1

Computer Abstractions and Technology

will be impressed as you correctly u se wo rds such as BIOS, DIMM, C PU, cache, DRAM, ATA, PCI, and many others. To reinforce how the software and hardwa re systems used to run a program will affect performance, we use :I specia l sect ion, " Understanding Progra m Performan ce," throughout the book, with the first one appea ring below. These elements summarize impo rtant insights into program performance.

Understanding Program Performance

Check Yourself

The performance of a program depends on a combination of the effectiveness of the algorithms used in the program, the softwa re systems used to create and translate the program into ma chine instructions, and the effectiveness of the computer in executing those instructions, which may include I/O operations. The following table summarizes how the hardware and softwa re affect performance. Hardware or software component

How this component affects performance

Where Is this topic covered?

Algorithm

Detennines both the number of source-level statements and the number of I/O operations executed

Other books!

Programming language. compiler. and architecture

Detennines the number of machine instructions for each source~evel statement

Chapters 2 and 3

Processor and memory system

Detennines how fast instructions can be executed

Chapters 5. 6. and 7

I/O system (hardware and operating system)

Detennines how fast I/O operations may be executed

Chapter 8

"Check Yourself" sections are designed to help readers assess whether they have comprehended the major concepts introduced in a chapter and understand the implications of those concepts. Some "Check Yourself" questions have simple answers; others are for discussion among a group. Answers to the specific questions ca n be found at the end of the chapter. "Check Yourself" questions appear only at the end of a section, making it easy to skip them if you are sure you understand the material. I. Section 1.1 showed that the number of embedded processors sold every

yea r greatly outnumbers the number of desktop processors. Ca n you con firm or deny this insight based on your own experience? Try to count the number of embedded processors in your home. How does it compa re with the number of desktop computers in your home?

1.2

Below Your Program

2. As mentioned earlier, both the software and hardwa re affect the performance of a program. Can you think of examples where each of the following is the right place to look for a perfo rmance bottleneck? • The algorithm chosen • The programming language o r compiler • The operating system • The processor • The I/O system and devices

Below Your Program A typical application, such as a wo rd processo r o r a large database system, may consist of hundreds of th ousand s to millions of lines of code and rely on sophisticated software libraries that implement complex fun ctions in support of the application. As we will see, the hardware in a computer ca n only execute extremely simple low-level in structions. To go from a complex application to the simple instructions involves several layers of softwa re that interpret or translate high level operations into simple computer instructions. These layers of soft wa re a re orga nized primarily in a hiera rchical fashion , with applications being the outermost ring and a va riety of systems software sitting between the hardwa re and applications softwa re, as shown in Figure 1.3. There a re many types of systems software, but two types of system s softwa re a re central to every computer system today: an operating system and a compiler. An operating system interfa ces between a user's program and the hardwa re and provides a va riety of services and supervisory fun ctions. Am ong the m ost important fun ctions are • handling basic input and output operations

In Paris they simply stared when 1spoke to them in French; I never did succeed in making those idiots understand their own language. Mark Twain, The lll lloeellts Abroad, 1869

syst em s software Software that provides services that are commonly useful, including operating systems, compilers, and assemblers. operating syst em Supervising program that manages the resources of a computer for the benefit ofthe programs that run on that machine.

• allocating sto rage and memory • providing fo r sharing the computer am ong multiple applications using it simultaneously Exa mples of operating systems in use today are Windows, Linux, and MacOS. Compilers perform another vital function: the translation of a program writ ten in a high -level language, such as C or Java, into instructions that the hardwa re

compiler A program that translates high-level language statements into assembly language statements.

Chapter 1

Computer Abstractions and Technology

FIGURE 1.3 A simplified view of hardware and software as hierarchical layers, shown as concentric circles with hardware In the center and applications software outermost. In complex applications there are often multiple layers of application software as weU . For example, a database system may run on top of the systems software hosting an application, which in turn nms on top of the database.

ca n execute. Given the sophistication of modern programming languages and the simple instructions executed by the hardwa re, the translation from a high -level language p rogram to hardwa re instructions is complex. We will give a brief overview of the process and return to the subject in Chapter 2.

From a High-Level Language to the Language of Hardware

binary digit Also called a bit. O ne of the two numbers in base 2 (0 or 1) that are the com pon ents of inform ation.

To actually speak to an electronic machine, you need to send electrical signals. The easiest signals fo r machines to understand are on and off, and so the machine alphabet is just two letters. Just as the 26 letters of the English alphabet do not limit how much ca n be written , the two letters of the computer alphabet do not limit what computers ca n do. The two symbols for these two letters are the num bers 0 and 1, and we commonly think of the machine language as numbers in base 2, or binary nllmbers. We refer to each "letter" as a binary digit or bit. Computers are slaves to our commands, which are called instructions. Instructions, which are just collections of bits that the computer understands, ca n be thought of as num bers. For exam ple, the bits

1000110010100000 tell one computer to add two numbers. Chapter 3 explains why we use numbers for instructions and data; we don't want to stea l that chapter's thunder, but using numbers fo r both instructions and data is a foundation of computing.

1.2

Below Your Program

The first programmers communicated to computers in binary numbers, but this was so tediou s that they quickly invented new notations that were closer to the way humans think. At first these notations were translated to bill3ry by hand , but this process was still tiresome. Using the machine to help program the machine, the pioneers invented programs to translate from symbolic notation to binary. The first of these programs was named an assembler. This program translates a symbolic version of an in struction into the binary version. For example, the programmer would write

assembler A p rogram that translates a symbolic version of instructions into the binary version.

add A, S and the assembler would translate this notation into

1000110010100000 This in struction tells the computer to add the two numbers A and B. The name coined for this symbolic langll3ge, still used today, is assembly language. Although a tremendous improvement, assembly language is still far from the notation a scientist might like to use to simulate fluid fl ow or that an accountant might use to balan ce the books. Assembly language requires the programmer to write one line for every instruction th at the machine will follow, forcing the programmer to think like the machine. The recognition that a program could be written to translate a more powerful language into computer instructions was one of the great breakth roughs in the ea rly days of computing. Programmers today owe their productivity-a nd their sanity-to the creation of high-level programming languages and compilers that translate programs in such languages into in structions. A compiler enables a programmer to write this high -level language expression:

A+ B The compiler would compile it into this assembly language statement:

add A, S The assembler would translate this statement into the binary instruction that tells the computer to add the two numbers Aand S:

1000110010100000 Figure 1.4 shows the relationships among these programs and languages. High -level programming langll3ges offer several important benefit s. First, they allow the programmer to think in a mo re natural language, using English wo rds and algebraic notation , resulting in p rograms th at look much more like text th an like tables of cryptic symbols (see Figure 1.4). Moreover, they allow languages to

assembly language A symbolic representation of machine instructions.

high-level programming language A portable language such as C, Fortran, or Java composed of words and algebraic notation that can be translated by a com piler into assembly language.

Chapter 1

Computer Abstractions and Technology

High-level language program (in C)

swap(i nt v[ ]. i nt (int t e mp: te mp - v[n:

v[ k] - v[ H1 ]: v[ k+ 1] - te mp:

j (

Assembly language program (for MIPS)

Compiler

swa p: mLll i $2 . $5 .4 add

lw lw sw sw j r

$2 . $4. $2 $15. 0($2) $16. 4 ($2) $16. 0($2) $15. 4 ($2) $ 31

j ( Assembler

Bina ry machine language program (for MIPS)

000000001010000100000000000 11 000 00000000000 110000001100000100001 1000 11 0001 1000100000000000000000 100011 00 1111 00100000000000000100 10101 100 111 100100000000000000000 10101 10001 1000100000000000000100 00000011 111000000000000000001000

FIGURE 1.4 C program complied Into assembly language and then assembled Into binary machine language. Although the translation from high· level language to binary machine Ian· guage is shown in tm> steps, some compilers cut out the middleman and produce binary machine language directly. These languages and this program are examined in more detail in Chapter 2.

be designed acco rding to their intended use. Hence, Fortran was designed for scientific computation, Cobol for business data processing, Lisp for symbol manipu lation , and so OIL The second adva ntage of programming languages is improved programmer productivity. One of the few areas of widespread agreement in softwa re development is that it takes less time to develop p rograms when they are written in lan guages th at require fewer lines to express an idea . Conciseness is a clea r advantage of high -level languages over assembly language.

1.3

Under the Covers

The final advantage is that programming languages allow programs to be independent of the computer on which they were developed, since compilers and assemblers ca n translate high -level language programs to the binary instructions of any ma chine. These three advantages are so strong that today little programming is done in assembly language.

Under the Covers Now that we have looked below your program to uncover the underlying software, let's open the covers of the computer to lea rn about the underlying hardwa re. The underlying hardware in any computer performs the same basic fun ctions: input ting data, outputting data, processing data, and storing data. How these functions are performed is the primary topic of this book, and subsequent chapters deal with different parts of these fo ur tasks. When we come to an important point in this book, a point so important that we hope you will remember it forever, we emphasize it by identifying it as a "Big Picture" item. We have about a dozen Big Pictures in this book, with the first being the five components of a computer that perform the tasks of inputting, outputting, processing, and storing data.

The five classic components of a computer are input, output, memory, data path, and control, with the last two sometimes combined and called the processor. Figure 1.5 shows the standard organization of a computer. This organization is independent of hardware technology: You can place every piece of every computer, past and present, into one of these five categories. To help you keep all this in perspective, the five components of a computer are shown on the front page of the following chapters, with the portion of interest to that chapter highlighted.

Figure 1.6 shows a typica l desktop computer with keyboard, m ouse, screen, and a box containing even m ore hardware. What is not visible in the photograph is a network that connects the computer to other computers. This photograph revea ls two of the key components of computers: input d evices, such as the keyboard and m ouse, and o utput d evices, such as the screen. As the names suggest, input feeds the computer, and o utput is the result of computation sent to the user. Som e devices, such as networks and disks, provide both input and o utput to the computer.

BIG

The Picture

input device A mechanism through which the computer is fed information, such as the keyboard or mouse. output device A mechanism that conveys the result of a computation to a user or another computer.

Chapter 1

Computer Abstractions and Technology

Interlace

Evaluating performance

FIGURE 1.5 The organization of a computer, showing the five classic: components. The processor gels instructions and data from memory. Input WTiles data to memory, and outpul reads data from memory. Control sends the signals that determine the operations of the datapath, memory, input, and output.

Chapter 8 describes input/output 0/0) devices in more detail, but let's take an introductory tour through the computer hardwa re, starting with the external I/O devices. 1 got the idea fo r the mouse while attending a talk at a compl/ter conference. T he speaker was 50 boring that 1 started daydreaming and hit IIpon the idea. Doug Engelhart

Anatomy of a Mouse Although many users now take mice for granted, the idea of a pointing device such as a mouse was first shown by Engelbart using a resea rch prototype in 1967. The Alto, which was the inspiration fo r all workstations as well as for the Macintosh , included a mouse as its pointing device in 1973. By the 1990s, all desktop computers included this device, and new user interfaces based on graphics displays and mice beca me the norm.

1.3

Under the Covers

FIGURE 1.6 A desktop computer. The liquid crystal display (LCD) scre;,on is the primary output device, and the keyboard and mouse are the primary input devices. The box contains the processor as well as additional If0 devices. This system is a Dell Optiplex GX260.

The o riginal mouse was elect romechanical and used a large ball that when rolled across a surface would cause an x and y counter to be incremented. The amount of increase in each counter told how far the mouse had been moved. The elect romechan ical mouse has largely been repla ced by the newer all-optical mouse. The optical mouse is actually a miniature optical processo r including an LED to provide lighting, a tiny black-a nd-white ca mera, and a simple optical p rocesso r. The LED illuminates the surface undernea th the mouse; the ca mera takes 1500 sa mple pictures a second under the illumination. Successive pictures are sent to a simple optica l processor th at compa res the images and determines whether the m ouse has m oved and how far. The replacement of the elect romechanical mouse by the elect ro-optica l mouse is an illustration of a common phenomenon where the decreasing costs and high er reliability of electronics cause an electronic solution to repla ce the older electromechanical technology.

Chapter 1

Through compl/ter displays 1 have landed an airplane on the deck ofa moving carrier, observed a fluclear particle hit a potential well, j10wn in a rocket at nearly the speed oflight and watched a computer reveal its innermost workings.

Through the Looking Glass

Iva n Sutherland, the "father" of computer graphics, quoted in "Computer Software fo r Graphics:' Scientific American, 1984

catho de ray tube (CRT ) display A display, such as a television set, that displays an image using an electron beam scanned across a screen. pixel The smallest individual picnlrc element. Screen are composed of hundreds of thousands to millions of pixels, organized in a matrix. fl at panel display, liquid uysta l display A display technology using a thin layer ofliquid polymers that can be used to

transmit or block light according to whether a charge is applied. active matrix display A liquid crystal display using a transistor to control the transmission of light at each individual pixel.

Computer Abstractions and Technology

The m ost fasci nating I/O device is probably the graphics display. Based on television technology, a cathode ray tube (CRT) display scans an image one line at a time, 30 to 75 times per second. At this refresh rate, people don't notice a fli cker on the screen. The image is composed of a matrix of picture elements, or pixels, which ca n be represented as a matrix of bits, called a bit map. Depending on the size of the screen and the resolution, the display m atrix ranges in size from 5 12 X 340 to 1920 X 1280 pixels in 2003. The simplest display has I bit per pixel, allowing it to be black or white. For displays that support 256 different shades of black and white, som etimes ca lled gray-scale displays, 8 bits per pixel a re required. A color display might use 8 bits for each of the three colo rs (red, blue, a nd green), for 24 bits per pixel, permitting millions of different colors to be displayed. All laptop and handheld computers, calculators, cellular phones, and many desktop com puters use flat-panel or liquid cr yst al displays (LCDs) instead of C RTs to get a thin, low-power display. The main difference is that the LCD pixel is not the source of light; instead it cont rols the transmission of light. A typica l LCD includes rod -shaped m olecules in a liquid that fo rm a tw isting helix that bends light entering the display, from either a light source behind the display o r less oft en from refl ected light. The rods straighten out when a current is applied and no longer bend the light; since the liquid crystal m aterial is between two screens pola rized at 90 degrees, the light ca nnot pass th rough unless it is bent. Today, m ost LCD displays use a n active m a trix that has a tiny transistor switch at each pixel to precisely cont rol current and make sharper images. As in a CRT, a red green-blue mask associated with each pixel determines the intensity of the three colo r components in the fin al image; in a color active matrix LCD, there are three transistor switches at each pixel. No m atter what the display, the computer hardwa re support for graphics con sists mainly of a raster refresh buffer, or frame buffer, to store the bit map. The im age to be represented on-screen is sto red in the frame buffer, and the bit pattern per pixel is read o ut to the graphics display at the refresh rate. Figure 1.7 shows a frame buffer with 4 bits per pixel. The goa l of the bit m ap is to faithfully represent what is on the screen. The challenges in graphics systems a rise because the human eye is very good at detecting even subtle changes on the screen . For exa m ple, when the screen is being updated, the eye ca n detect the inconsistency between the po rtion of the screen that has cha nged and that which hasn't.

Opening the Box If we open the box containing the com puter, we see a fasci nating boa rd of thin green plastic, covered with dozens of small gray or black rectangles. Figure 1.8

1.3

Under the Covers

Frame buffer Raster scan CRT display

Y,+-+[QJ/ Y,

Y,+ -

+--tt!!:t-/ _ x.,

Y, +-+x.,

FtGURE 1.7 Each coordinate In the frame buffer on the left determines the shade of the corresponding coordinate for the raster scan CRT display on the right. Pixel (Xo, Yo) contains the bit pattern 0011, which is a lighter shade of gray on the screen than the bit pattern 1101 in pixel (X t , Yt ).

DVD drive power supply

~~- Zip drive

fan with cover

motherboard

--0----.1

FtGURE 1.8 Inside the personal computer of Figure 1.6 on page 17. This packaging is sometimes called a clamshell because of the way it opens with hinges on one side. To see what's inside, let's start on the top left-hand side. The shiny metal box on the top far left side is the power supply. Just below that on the far left is the fan, with its cover pulled b.1ck. To the right and below the fan is a printed circuit board (PC board), called the motherboard in a Pc, that contains most of the electronics of the computer; Figure 1.10 is a close-up of that boord. The processor is the large raised rectangle just to the right of the fan. On the right side we see the bays designed to hold types of disk drives. The top bay contains a DVD drive, the middle bay a Zip drive, and the bottom bay contains a hard disk.

Chapter 1

motherboard A plastic board containing packages of integrated circuits or chips, including processor, cache, m emory, and connectors for va devices such as n etworks and disks.

shows the contents of the desktop computer in Figure 1.6. This motherboard is shown vertica lly on the left with the power supply. Three disk drives-a DVD drive, Zip drive, a nd hard drive-appear on the right. The small recta ngles on the m otherboard contain the devices that drive our adva ncing technology, integrated circuits or chips. The board is composed of three pieces: the piece connecting to the I/O devices mentioned ea rlier, the mem ory, and the processor. The I/O devices are connected via the two large boa rds atta ched perpendicularly to the m otherboard toward the middle on the right hand side. The memory is where the programs are kept when they are running; it also contain s the data needed by the running programs. In Figure 1.8, mem ory is found on the two small boa rds that are attached perpendicularly towa rd the mid dle of the m otherboa rd. Each small mem ory board contains eight integrated circuits. The processor is the active pa rt of the boa rd , following the instructions of a program to the letter. It adds numbers, tests numbers, signals I/O devices to activate, and so on. The processor is the large square below the mem ory boards in the lower-right co rner of Figure 1.8. O ccasionally, people call the processor the CPU, for the more bureaucratic-sounding central processor unit. Descending even lower into the hardware, Figure 1. 9 reveals details of the processor in Figure 1.8. The processor comprises two main components: datapath and control, the respective brawn and brain of the processor. The datapath perform s the arithmetic opera tions, and control tells the data path , memory, and I/O devices what to do according to the wishes of the instructions of the program. Chapter 5 explains the datapath and control for a straightforwa rd implementa tion , and Chapter 6 describes the changes needed for a higher-performan ce design. Descending into the depths of any component of the hardware reveals insights into the machine. The m em ory in Figure 1.10 is built from DRAM chips. DRAM stands for dynamic random access memory. Several DRAMs are used together to contain the instructions and data of a program. In contrast to sequential access mem ories such as magnetic tapes, the RAM portion of the term DRAM mea ns that mem ory accesses take the same amount of time no matter what portion of the memory is read. Inside the processor is another type of m em ory-cache mem ory. Cache memory consists of a small, fa st memory that acts as a buffer for the DRAM memory. (The nontechnica l definition of cache is a safe place for hiding things.) Cache is built using a different mem ory technology, static random access mem ory (SRAM). SRAM is fa ster but less dense, and hence m ore expen sive, than DRAM.

integrated circuit Also called chip. A device combining dozen s to millions of transistors. memory The storage area in which programs are kept when they are running and that contains the data needed by the rUfimng programs.

central processor unit (CPU) Also called processor. The active part of the computer, which contains the datapath and COI1trol and which adds numbers, tests numbers, signals I/O devices to activate, and so on. datapath The component of the processor that performs arithmetic operations. control The component of the processor that commands the

datapath, memory, and I/O devices according to the instructions of the program. dynamic random access memory (DRAM) Memory built as an integrated circuit, it provides random access to any location. cache memory A small, fast memory that acts as a buffer for a slower, larger memory.

Computer Abstractions and Technology

1.3

Under the Covers

Control Control

110 interface

Instruction cache Data cache Enhanced floating point and multimedia

Integer datapath

Secondary cache

00'

memory interface

Control

Advanced pipelining hyperthreading support

Control

FIGURE 1.9 Inside the processor chip used on the board shown In Figure 1.8. The left-hand side is a microphotograph oflh.. Pentium 4 processor chip, and the right-hand side shows the major blocks in the processor.

You may have noticed:1 common theme in both the software and the hardware descriptions: delving into the depths of hardware or software reveals more information or, conversely, lower-level details are hidden to offer a simpler model at higher levels. The use of such layers, or ab stractions, is a principal technique for designing very sophisticated computer systems. One of the most important abstractions is the interface between the hardware and the lowest-level software. Because of its importance, it is given a special

abstraction A model that renders lower-level details of computer systems temporarily invisible in order to facilitate design of sophisticated systems.

Chapter 1

Computer Abstractions and Technology

Processor

• o ,

y Processor interface

Disk and USB interlaces Graphics

110 bus slots

DlMM (dual inline memory module) A small board that contains DRAM chips on both sides. SIMMs have DRAMs on only one side. Both DIMMs and SIMMs are meant to be plugged into m em ory slots, u su ally on a

motherboard. instruction set architecture Also called architecture. An abstract interface between the h ardware and the lowest level software of a machine that en compasses all the information n ecessary to write a machine language program that will run correctly, including instruction s, registers, mem ory access, I/O, and so a ll.

application binary interface (ABI) The user portion of the instruction set plus the operating system interfaces used by application programmers. Defines a stand ard for binary portability across computers.

implementation Hardware that obeys the architecture abstraction.

FIGURE 1.10 Close-up of PC motherboard. This board uses the Intel Pentiwn 4 processor, which is located on the left-upper quadrant of the board. It is covered by a set of metal fins, which look like a radiator. This structure is the hear sink, used to help cool the chip. The main memory is contained on one or more small boards that are perpendicuiar to the motherboard near the middle. The DRAM chips are mounted on these bo.uds (ca Ued DIMMs, for dua l inline memory moouies) and then plugged into the connectors. Much of the rest of the bo.ud comprises connectors for external If0 devices: audio/MIDI and paralleVserial at the right edge, two PCI card slots near the boltom, and an ATA connector used for attaching hard disks .

name: the instruction set architecture, or simply architecture, of a machine. The instruction set architecnlfe includes anything programmers need to know to make a binary machine language program work correctly, including instructions, I/O devices, and so on. Typica lly the operating system will enca psulate the details of doing I/O, allocating memory, and other low-level system fun ctions, so that application programmers do not need to wo rry about such details. The combin ation of the basic instruction set and the operating system interface provided for application programmers is called the application binary interface (AB!). An instruction set architecture allows computer designers to talk about fun ctions independently from the hardwa re that performs them. For exa mple, we can talk about the fun ctions of a digital clock (keeping time, displaying the time, setting the alarm) independently from the clock hardware (quartz crystal, LED displays, plastic buttons). Computer designers distinguish architecture from an implementation of an architecture along the sa me lines: an implementation is hardwa re that obeys the architecnlfe abstraction. These ideas bring us to another Big Picture.

1.3

Under the Covers

A Safe Place for Data Thus far we have seen how to input data, compute using the data, and display data. If we were to lose power to the computer, however, everything would be lost because the memory inside the computer is volatil e-that is, when it loses power, it forgets. In contra st, a cassette tape for a stereo doesn't forget the recorded music when you turn off the power because the tape is magnetic and is thus a nonvolatile memory techn ology. To distinguish between the memory used to hold programs while they are run ning and this nonvolatile memory used to store programs between runs, the term primary memory or main memory is used for the former, and secondary memory for the latter. DRAMs have dominated main memory since 1975, but magnetic disks have dominated secondary memory since 1965. In embedded applications, FLASH, a nonvolatile semiconductor memory is also used. Today the primary nonvolatile storage used on all desktop and server comput ers is the magnetic hard disk. As Figure 1.11 shows, a magnetic hard disk consists of a collection of platters, which rotate on a spindle at 5400 to 15,000 revolutions per minute. The metal platters are covered with magnetic recording material on both sides, similar to the material found on a cassette or video tape. To read and write information on a hard disk, a movable arm containing a small electromagnetic coil ca lled a read/write head is located just above each surface. The entire drive is permanently sealed to control the environment inside the drive, which, in turn, allows the disk heads to be much closer to the drive surface. Diameters of hard disks vary by more than a fa ctor of 3 today, from less than I inch to 3.5 inches, and have been shrunk over the years to fit into new products; workstation servers, personal computers, laptops, palmtops, and digital cameras have all inspired new disk form fa ctors. Traditionally, the widest disks have the highest performance, the smallest disks have the lowest unit cost, and the best cost per megabyte is usually a disk in between. Although most hard drives appear inside computers (as in Figure 1.8), hard drives ca n also be attached using external interfaces such as Firewire or USB. The use of mechanical components means that access times for magnetic disks are much slower than for DRAMs: disks typically take 5- 15 millisecond s, while DRAMs take 40--80 nanoseconds-making DRAMs about 100,000 times faster. Yet disks have much lower costs than DRAM for the sa me storage capacity because the production costs for a given amount of disk storage are lower than for the same amount of integrated circuit. In 2004, the cost per megabyte of disk is about 100 times less expensive than DRAM. Thus there are three primary differences between magnetic disks and main memory: disks are nonvolatile because they are magnetic; they have a slower access time because they are mechanical devices; and they are cheaper per megabyte because they have very high storage capacity at a modest cost.

memory The storage area in which programs are kept when they are running and th at contains the data needed by the runmng programs. volatile memory Storage, such as DRAM, that only retains data only if it is receiving power. nonvolatile memory A form of memory that retains data even in the absence of a power source and that is used to store programs between runs. M agnetic disk is nonvolatile and DRAM is not. primary memory Also called main memory. Volatile memory used to hold programs while they are running; typically consists of DRAM in today's computers. secondary memory Nonvolatile memory used to store programs and data between runs; typically consists of magnetic disks in today's computers. magnetic disk (also called hard disk) A form of nonvolatile secondary memory composed of rotating platters coated with a magnetic recording material. megabyte Traditionally 1,048,576 (220) bytes, although some communications and secondary storage systems have redefined it to mean 1,000,000 (10 6 ) bytes.

Chapter 1

Computer Abstractions and Technology

FIGURE 1.11

BIG

The Picture

A disk showing 10 disk platters and the read/ write heads.

Both hardwa re and software consist of hierarchical layers, with each lower layer hiding details from the level above. This principle of abstraction is the way both hardwa re designers and software designers cope with the complexity of computer systems. One key interface between the levels of abstraction is the instruction set arch itecture-the interface between the hardware and low-level softwa re. This abstract interface enables many implementations of varying cost and performance to run identical soft ware.

1.3

Under the Covers

Although hard drives are not rem ovable, there are several storage technologies in use that include the following: • Optical disks, including both compact disks (CDs) and digital video disks (DVDs), constitute the most common form of removable storage. • Magnetic tape provides only slow serial access and ha s been used to back up disks, in a role now often replaced by duplicate hard drives. • FLASH-based removable memo ry cards typically attach by a USB (Universal Serial Bus) connection and are oft en used to transfer fil es. • Floppy drives and Zip drives are a version of magnetic disk technology with removable fl exible disks. Floppydisks were the original primary storage for personal computers, but have now largely va nished. Optical disk technology works in a completely different way than magnetic disk techn ology. In a CD, data is reco rded in a spiral fashion, with individual bits being recorded by burning small pits-approximately I micron ( 10--6 meters) in diame,, , -into the disk surface. The disk is read by shining a laser at the CD surfa ce and determining by exa mining the refl ected light whether there is a pit or flat (reflective) surface. DVDs use the sa me approach of bouncing a laser beam off a series of pits and flat surfaces. In addition, there are multiple layers that the laser beam ca n be focused on , and the size of each bit is much smaller, which together yield a significa nt increase in capacity. CD and DVD writers in personal computers use a laser to m ake the pits in the recording layer on the CD or DVD surface. This writing process is relatively slow, taking from tens of minutes (for a full CD) to close to an hour (for a full DVD). Thus, for large quantities a different technique called pressing is used, which costs only pennies per CD or DVD. Rewritable CDs and DVDs use a different recording surfa ce that ha s a crystalline, refl ective material; pits are form ed that are not refl ective in a manner similar to that for a write-once CD or DVD. To erase the CD or DVD, the surfa ce is heated and cooled slowly, allowing an annealing process to restore the surfa ce reco rding layer to its crystalline structure. These rewritable disks are the most expensive, with write-once being cheaper; for read-only disks-used to distribute softwa re, music, or movies-both the disk cost and recording cost are much lower.

Communicating with Other Computers We've explained how we ca n input , compute, display, and save data, but there is still one missing item found in today's computers: computer networks. Just as the processor shown in Figure 1.5 on page 16 is connected to memory and I/O devices, netwo rks connect whole computers, allowing computer users to extend

floppy disk A portable form of secondary memory composed of a rotating mylar platter coated with a magnetic recording material.

Chapter 1

Computer Abstractions and Technology

the power of computing by including commun ication. Networks have become so popular that they are the backbone of current computer systems; a new machine without an optional netwo rk interface would be rid iculed. Netwo rked computers have several major adva ntages:

• Comm unication: Information is exchanged between computers at high speeds.

lo(;al area network (LAN) A network designed to carry data within a geographically confined area , typ ically within a single building. wide area network A network extended over hundreds of kilometers which can span a continent.

•

Resource sharing: Rather than each machine having its own I/O devices, devices ca n be shared by computers on the netwo rk.

•

Non/ocal access: By connecting computers over long distances, users need not be nea r the computer they are using.

Networks va ry in length and perfo rmance, with the cost of communication increasing acco rding to both the speed of communica tion and the distance that information travels. Perhaps the most popular type of network is the Ethernet. Its length is limited to about a kilometer, and the most popular version in 2004 takes about a tenth of a second to send 1 million bytes of data. Its length and speed make Ethernet useful to connect computers on the sa me fl oo r of a building; hence, it is an example of what is generica lly called a local area network. Local area networks are interconnected with switches that ca n also provide routing services and security. Wide area networks cross continents and are the backbone of the Internet, which supports the Wo rld Wide Web. They are typica lly based on optical fib ers and are leased from telecommunica tion companies. Networks have changed the face of computing in the last 25 yea rs both by becoming much more ubiquitous and by dramatic increases in perfo rmance. In the 1970s, very few individuals had access to elect ronic mail, the Internet and Web did not exist, and physica lly mailing magnetic tapes was the primary way to transfer large amounts of data between two locations. In the 1970s, local area networks were almost nonexistent, and the few existing wide area netwo rks had limited capacity and restricted access. As netwo rking technology improved, it beca me much cheaper and had a much higher capacity. For example, the first stand ardized loca l area network technology developed about 25 yea rs ago was a version of Ethernet that had a maximum capacity (also ca lled bandwidth) of 10 million bits per second , typica lly shared by tens, if not a hundred, computers. Today, local area netwo rk techn ology offers a capacity of from 100 million bits per second to a gigabit per second, usually shared by at most a few computers. Furthermore, lO-gigabit technology is in development! Optical communications techn ology has allowed similar growth in the capacity of wide area networks from hundreds of kilobits to gigabits, and from hundreds of computers connected to a wo rldwide network to millions of com put ers connected. This combination of dramatic rise in deployment of networking

1.3

Under the Covers

combined with the increases in capacity ha ve made netwo rk techn ology central to the information revolution of the last 25 years. Recently, another innovation in networking is reshaping the way computers communicate. Wireless technology has become widely deployed , and most laptops now incorporate this technology. The ability to make a radio in the same lowcost semiconductor techn ology (CMOS) used for memory and microprocessors enabled a significa nt improvement in price, leading to an explosion in deployment. Currently available wireless technologies, called by the IEEE standard name 802.11, allow for tran smission rates from I to less than 100 million bits per second. Wireless technology is quite a bit different from wire-based networks, since all users in an immed iate area share the airwaves. 1. Semiconductor DRAM and disk storage differ significa ntly. Describe the

fundam ental difference for each of the following: volatility, access time, and cost.

Check Yourself

Technologies for Building Processors and Memories Processors and memory have improved at an incredible rate because computer designers have long embraced the latest in electronic technology to try to win the race to design a better computer. Figure 1.1 2 shows the techn ologies that have been used over time, with an estimate of the relative performance per unit cost for each techn ology. This section explores the technology that has fueled the com puter industry sin ce 1975 and will continue to do so for the foreseeable future. Since this technology shapes what computers will be able to do and how quickly they will evolve, we believe all computer professionals should be familiar with the basics of integrated circuits. A tran sistor is simply an on/off switch controlled by electricity. The integrated circuit (Ie) combined dozens to hundreds of transistors into a single chip. To describe the tremendous increase in the number of transistors from hundreds to Technology used In computers

t rans istor An on/off switch controlled by an electric signal.

Relative performance/ unit cost

1 35

195 1

Vacuum tube

1965

Transistor

1975

Integrated circuit

1995 2005

Very large scale integrated circuit

2,400,000

Ultra large scale integrated circuit

6,200,000,000

900

FIGURE 1.12 Relative performance per unit cost of technologies used In computers over time. Source: Computer Museum, Boston, with 2005 extrapolated by the authors.

vacu u m tu b e An electronic component, predecessor of the transistor, that consists of a hollow glass ntbe about 5 to 10 em long from which as much air has been removed as possible and which uses an electron beam to transfer data.

Chapter 1

Computer Abstractions and Technology

1,000,000 256M 512M

100,000 16M

l ••

128M

54M

10,000 1000 100

54K

¥ "-,---,--.,----,---,--.,---,--r--,---,---,----,---,

1976 1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 Year of introduction FIGURE 1.13 Growth of capacity per DRAM chip over time. The y-axis is measured in Kbits, where K = 1024 (2 10 ). The DRAM industry quadrupled capacity almost every 3 years, a 60% increase per year, for 20 years. This "four times every three years~ estimate was called the DRAM growth rule. In recent years, the rate has slowed down somewhat and is somewhat closer to doubling every two years or four times every four years.

very large 5':ale integrated (VLSI) circuit A device containing hundreds of thousands to millions of transistors.

I thol/ght {computers} would

be a universally applicable idea, like a book is. Blit 1 didn't think it would develop as fast as it did, because 1 didn't envision we'd be able to get as many parts on a

chip as we finally got. The transistor came along l/nex-

pectedly. It all happened milch faster thall we expected.

J. Presper &kert, coinventor of ENIAC, speaking in 199 1

millions, the adjective very large scale is added to the term , crea ting the abbreviation VLSI, for very large scale integrated circuit. This rate of increasing integration has been remarkably stable. Figure 1.1 3 shows the growth in DRAM capacity since 1977. For 20 years, the industry has consistently quadrupled capacity every 3 years, resulting in atl increase in excess of 16,000 times! This increase in transistor count for an integrated circuit is popularly known as Moore's law, which states that transistor capacity doubles every 1824 months. Moore's law resulted from a prediction of such growth in IC capacity made by Gordon Moore, one of the founders of Intel during the 1960s. Sustaining this rate of progress for almost 40 years has required incredible inn ovation in the manufa cturing techniques. In Section 1.4, we discuss how integrated circuits are manufactured.

Real Stuff: Manufacturing Pentium 4 Chips Each chapter has a section entitled " Real Stuff" that ties the concepts in the book with a computer you may use every day. These sections cover the technology underlying the IBM-compatible PC, the Apple Macintosh, a common server, or atl embedded computer. For this first "Rea l Stuff" section , we look at how integrated circuits are manufactured, with the Pentium 4 as an exa mple.

1.4

Real Stuff: M anufacturing Pentium 4 Chips

Let's start at the beginning. The manufacture of a chip begins with silicon, a substance found in sand. Because silicon does not conduct elect ricity well, it is called a semiconductor. With a special chemical p rocess, it is possible to add materials to silicon that allow tiny areas to transfo rm into one of three devices: • Excellent conductors of elect ricity (using either microscopic copper or alu minum wire)

silicon A n atural element which is a sem icondu ctor. semiconductor A substance that does not conduct electricity well.

• Excellent insulators from electricity (like plastic sheathing or glass) • Areas that ca n conduct or insulate under specia l conditions (as a switch ) Transistors fa ll in the last category. A VLSI circuit , then , is just billions of combinations of conductors, in sulators, and switches manufactured in a single, small package. The manufacturing p rocess for integrated circuits is critica l to the cost of the chips and hence im portant to computer designers. Figure 1. 14 shows that process. The p rocess starts with a silicon crystal ingot, which looks like a giant sausage. Today, ingots are 8-12 inches in dia meter and about 12-24 inches long. An ingot is fin ely sliced into wafers no mo re than 0. 1 inch thick. These wafers then go th rough a series of processing steps, during which patterns of chemicals are placed

C_ _) -_. I51,,,, I

•

Tes te d dies D ~ D l8I

Bo nd die to .,~_ DDDDD ~,~_ package D D I8l D D

D ODD

•

Dicer

I. ,

20 to 40 processing s teps

I Patterned wafe rs

Tes ted wafer

Wafer tes ter

.-~

Packaged d ies

Tes te d packaged dies

1i:iI1i:iI1i:iI __ IQIIQIIQI

wafer A slice from a silicon ingot n o m ore than 0. 1 inch thick, used to create chips.

Blan k wafe rs

S ilicon ingot

P,rt tes ter

silicon crystal ingot A rod composed of a silicon crystal that is between 6 and 12 inches in diam eter and about 12 to 24 inches long.

Ship to cus to mers

FIGURE 1.14 Th e chip manufacturing pr ocess. After being sliced from the silicon ingot, blank wafers are put through 20 to 40 steps to create patterned wafers (see Figure 1.15 on page 31). These patterned wafers are then tested with a wafer tester and a map of the good parts is m.1de. Then, the wafers are diced into dies (see Figure 1.9 on page 21). ln this figure, one wafer produced 20 dies, of which 17 passed testing. (X means the die is b.1d.) The yield of good dies in this case was 17/20, or 85%. These good dies are then bonded into packages and tested one more time before shipping the packaged parts to customers. One bad packaged part was found in this final test.

defect A microscopic flaw in a wafer or in p atterning steps that can result in the failure of the die containing th at defect. die The individual rectangular sections th at are cut from a wafer, m ore informally known as chips.

yield The percentage of good dies from the total number of dies on the wafer.

Chapter 1

Computer Abstractions and Technology

on each wa fer, creating the transistors, conductors, and insulators discussed ea rlier. Today's integrated circuits contain only one layer of transistors but may have from two to eight levels of metal conductor, separated by layers of insulators. A single microscopic fla w in the wa fer itself or in one of the dozens of pattern ing steps ca n result in th at area of the wafer failing. These defects, as they are called, make it virnlally impossible to mallufacnlre a perfect wafer. To cope with imperfection, several strategies have been used, but the simplest is to pla ce many independent components on a single wafer. The patterned wa fer is then chopped up, or diced, into these components, ca lled dies and more informally kn own as chips. Figu re 1.1 5 is a photograph of a wa fer containing Pentium 4 microprocessors before they have been diced; ea rlier, Figure 1.9 on page 21 showed all individual die of the Pentium 4 and its majo r components. Dicing enables you to disca rd only those dies that were unlucky enough to con tain the fl aws, rather than the whole wa fer. This concept is quantified by the yield of a process, which is defined as the percentage of good dies from the total number of dies on the wafer. The cost of all integrated circuit rises quickly as the die size increases, due both to the lower yield and the smaller number of large dies that fit on a wa fer. To reduce the cost, a large die is often "shrunk" by u sing the next generation process, which in co rpo rates smaller sizes for both transistors and wires. This improves the yield and the die count per wafer. (An " Integrated Circuit Cost section on the CD probes these issues further. ) Once you've found good dies, they are connected to the input/output pins of a package, using a process called bonding. These packaged parts are tested a fin al time, sin ce mistakes ca n occur in packagin g, and then they are shipped to cu stomers. Another increasingly important design constraint is power. Power is a challenge for two reasons. First, power must be brought in alld distributed around the chip; modern microprocessors use hundreds of pins just for power and ground! Similarly, multiple levels of interconnect are used solely for power and ground distribution to po rtions of the chip. Second , power is dissipated as heat and must be removed. An Intel Pentium 4 at 3.06 GHz burns 82 watts, which must be removed from a chip whose surface area is just over I cm 2 1 Figure 1.1 6 shows a 3.06 GHz Pentium 4 mounted on top of its hea t sink, which in turn sits right next to the fan in the box shown in Figure 1.8 (on page 19)! What determines the power consumed by all integrated circuit? Ignoring tech nology and circuit specifics, the power is proportional to the product of the num ber of transistors switched times the frequency they are switched. Thus, in general, higher clock rates or higher transistor counts lead to higller power. For example, the Intel Itanium 2 has four times the transistors of the Intel Pentium 4; alth ougll its clock rate is only one-half that of the Pentium 4, the Itanium burns 130 watts

1.4

Real Stuff: Manufacturing Pentium 4 Chips

•• • • • • • • • ,...•..'j • • • • • , • • •• · .' ..' ,'. • • • ,• •• • ' ,; • • • • • .. , • , • • •...1 • • • • • •• •• • • •• ',' '.' .. .', , • • •.. ; •.. ' :•. J • • • • • • • •• •• , • • • .' '; ~' ~. , • •,. • • • •• • • • • • • • •• • •• , •,. • • • ..' '.' ,. • • • • • •• • • • • • •• .' •••• •• .' ,, • • • • •• • • • • • •.. • • • • • •.' • • • • •• • •, • • , • • • • • • ,, •

.. ... .. .. .. .. .. .. . .... .. .. .. ..

. .. ....'. .'. . ", ·'. . .' . '

'~'

. ~'

,• ,,

FtGURE 1.15 An 8-1nch (20~m) diameter wafer containing Intel Pentium 4 processors. The number of Pentium dies per wafer at 100% yield is 165. Figure 1.9 on page 21 is a photomicrograph of one of these PentiwlI 4 dies. The die area is 250 mm 2, and it contains about 55 miUion transistors.This die uses a 0.18 micron technology, which means that the smallest transistors are approximately 0.18 microns in size, although they are typically somewhat smaller than the actual feature size, which refers to the size of the transistors as ~drawn " versus the final manufactured size. The Pentium 4 is also made using a more advanced 0.13 micron technology. The several dozen partially rounded chips at the bowldaries of the wafer are useless; they are included becaU'ie it's easier to create the masks used to pattern the silicon.

compared to the 82 watts consumed by the Pentium 4. As we will see in later chapters, both performance and power consumption vary widely. Elaboration: In CMOS (Complementary Metal Oxide Semiconductor), which is the dominant technology for integrated circuits, the primary source of power dissipation is so-called "dynamic power"-that is, power that is consumed during switching. CMOS technology, unlike earlier technologies, does not directly consume power when it is idle-hence the use of low clock rates to allow a processor to "sleep" and conserve power. The dynamic power dissipation depends on the capacitive loading of each transistor, the voltage applied, and the frequency that the transistor is switched : Power = Capacitive load x Voltage 2 x Frequency switched

Chapter 1

Computer Abstractions and Technology

FIGURE 1.16 An Intel Pentium 4 (3.06 GHz) mounted on top of Its heat sink, which Is designed to remove the 82 watts generated within the die.

Power can be reduced by lowering the voltage, which typically occurs with a new generation of technology; in 20 years, voltages have gone from 5V to 1 .5V, significantly reducing power. The capacitive load per transistor is a function of both the number of transistors connected to an output (called the fanout) and the technology, which determines the capacitance of both wires and transistors. Although dynamic power is the primary source of power dissipation in CMOS, static power dissipation occurs because of leakage current that flows even when a transistor is off. In 2004, leakage is probably responsible for 20-30% of the power consumption . Thus, increasing the number of transistors increases power dissipation , even if the transistors are always off. A variety of design techniques and technology innovations have been deployed to control leakage.

1.5

Fallacies and Pitfalls

A key fa cto r in determining the cost of an integrated circuit is volume. \Vhich of the following are reasons why a chip made in high volume should cost less?

Check Yourself

1. With high volumes, the manufacturing process can be tuned to a particular

design, increasing the yield. 2. It is less work to design a high -volume part than a low-volume part. 3. The masks used to make the chip are expensive, so the cost per chip is lower for higher volumes. 4. Engineering development costs are high and largely independent of volume; thus, the development cost per die is lower with high -volume parts. 5. High -volume parts usually have smaller die sizes than low-volume parts and therefore have higher yield per wa fer.

Fallacies and Pitfalls The purpose of a section on fallacies and pitfalls, which will be found in every chapter, is to explain some commonly held misconceptions that you might encounter. We ca ll such misbeliefs fallacies. When discussing a fallacy, we try to give a counterexa mple. We also discuss pitfalls, or easily made mistakes. Often pitfall s are genera lizations of principles that are true in a limited context. The purpose of these sections is to help you avoid making these mistakes in the machines you may design or use. Fallacy: Computers have been built in the same, old-fashioned way for far too long, and this antiquated model ofcomputation is running Ollt of steam.

For an antiquated model of computation , it surely is improving quickly. Figure 1.1 7 plots the top performance per yea r of workstations between 1987 and 2003. (Chapter 4 explains the proper way to measure performance.) The graph shows a line indica ting an improvement of 1.54 per yea r, or doubling performance approximately every 18 months. In contrast to the statement above, computers are improving in performance faster today than at any time in their history, with over a thousandfold improvement between 1987 and 2003! Pitfall: Ignoring the inexorable progress of hardware when plann ing a new machine.

Suppose you plan to introduce a machine in three years, and you claim the machine will be a terrific seller because it's three times as fast as anything available today. Unfortunately, the ma chine will probably sell poorly because the average

Science must begin with myths, and the criticism of myths.

Sir Karl Popper, Th e Philosophy ofScience, 1957

Chapter 1

Computer Abstractions and Technology

10,000

Intel Pentium 4/3000 ~ Intel Xeonf20oo

DEC Alpha

21?2'6~4~Al~6~6~7::::';"'------

DEC Alpha 21264/600

1,000

DEC Alpha 51500 DEC Alpha 51300 IBM POWER 100......... ........ DEC AXP/5OO ~-

100

SUN-41 260

DEC Alpha 4/266

HP 90001750 IBM RS6000

M/120 ]~/~;,o;a'~M;'~P~S~M:2000:,.-...,...---,_-,---,-_;--r--;_"-"""'_r---1 MIPS

1987 1988 1989 1990 1991 199219931994 1995 1996 1997 1998 19992000 2001 2002 2003

Yoo, FIGURE 1.17 Performance Increase of workstations, 1987-2003. Here performance is given as approximately the number of times faster than the VAX-II/780, which was a commonly used yardstick. The rate of performance improvement is between 1.5 and 1.6 times per year. These performance numbers are based on SPECint performance (see Chapter 2) and scaled over time to deal with changing benchmark sets. For processors listed with xly after their name, x is the model nwnber andy is the speed in megahertz.

performance growth rate for the industry will yield ma chines with the same performance. For example, assuming a 50% yea rly growth rate in performance, a machine with performance x today ca n be expected to have performance l. S3x = 3.4x in three yea rs. Your ma chine would have no performance advantage! Many projects within computer companies are ca nceled , either because they ignore this rule or because the project is completed late and the performance of the delayed machine is below the industry average. This phenomenon may occur in any industry, but rapid improvements in cost/performan ce make it a maj or concern in the computer industry.

1.6

Concluding Remarks

Concluding Remarks Although it is difficult to predict exactly what level of cost/performance comput ers will have in the future, it's a safe bet that they will be much better than they are today. To participate in these advances, computer designers and programmers must understand a wider variety of issues. Both hardware and software designers construct computer systems in hierarchicallayers, with each lower layer hiding details from the level above. This principle of abstraction is fundam ental to understanding today's computer systems, but it does not mean that designers ca n limit themselves to knowing a single technology. Perhaps the most important exa mple of abstraction is the interface between hardware and low- level software, ca lled the instruction set architecture. Maintain ing the instruction set architecture as a constant enables many implementations of that architecture-presumably va rying in cost and performance-to run identical software. On the downside, the architecture may preclude introducing innovations that require the interface to change. Key technologies for modern processors are compilers and silicon. Clea rly, to participate you must understand some of the characteristics of both. Equal in importance to an understanding of integrated circuit technology is an understanding of the expected rates of technological change. While silicon fuels the rapid adva nce of hardware, new ideas in the organization of computers have improved price/performance. Two of the key ideas are exploiting parallelism in the processor, typica lly via pipelining, and exploiting loca lity of accesses to a memory hierarchy, typically via caches.

Road Map for This Book At the bottom of these abstractions are the five classic components of a computer: datapath, control, memory, input, and output (refer back to Figure 1.5). These five components also serve as the framework for the rest of the chapters in this book:

• Datapath: Chapters 3, 5, and 6 • Control: Chapters 5 and 6 • Memory: Chapter 7 • Inpllt: Chapter 8 • Outpllt: Chapter 8

Where . .. the ENIAC is equipped with 18,000 vacuum tubes and weighs 30 tons, computers in the future may have 1,000 vaClium wbes and perhaps weigh just 1112 tons. Papillar Mechanics, March 1949

An active field ofscience is like all immense anthill; the individual almost vanishes i1lto the mass of minds tumbling over each other, carrying information from place to place, passing itaround at the speed of light. Lewis Thomas, "Natural Science:' in The Lives ofa Cell, 1974

Chapter 1

Computer Abstractions and Technology

Chapter 6 describes how processor pipelining exploits parallelism, and Chapter 7 describes how the memory hierarchy exploits loca lity. The remaining chapters provide the introduction and the conclusion to this material. Chapter 2 describes instruction sets- the interfa ce between compilers and the machine-a nd empha sizes the role of compilers and programming languages in using the features of the instruction set. Chapter 3 describes how computers perform arithmetic operations and handle arithmetic data. Chapter 4 covers performance and thus describes how to evaluate the whole computer. Chapter 9 describes multiprocessors and is included on the CD. Appendix B, also on the CD, discusses logic design.

Historical Perspective and Further Reading For each chapter in the text, a section devoted to a historical perspective can be found on the CD that accompanies this book. We may trace the development of an idea through a series of ma chines or describe some important projects, and we provide references in case you are interested in probing furth er. The historical perspective for this chapter provides a background for some of the key ideas presented in this opening chapter. Its purpose is to give you the human story behind the technological advances and to place achievements in their historical context. By understanding the past, you may be better able to understand the forces that will shape computing in the future. Each historica l perspectives section on the CD ends with suggestions for further reading, which are also collected sepa rately on the CD under the section "Further Reading." The rest of this 'II Section 1.7 is on the CD.

Exercises The relative time ratings of exercises are shown in square brackets after each exercise number. On average, an exercise rated (10 ) will take you twice as long as one rated [5J . Sections of the text that should be read before attempting an exercise will be given in angled brackets; for exa mple, means you should have read Section 1. 3, "Under the Covers;' to help you solve this exercise. If the solution to an exercise depends on others, they will be listed in cu rly brackets; for example, {Ex.I.I } mea ns that you should answer Exercise 1.1 before trying this exercise.

In More Depth Exercises introduce a new topic or explore a topic in m ore detail. Such exercises include sufficient background to understand the concepts, as

1.8

Exercises

well as exercises to explore its implica tion or use. The In More Depth sections appea r on the CD associated with the specific chapter. Starting in Chapter 2, you will also find For More Practice Exercises. For More Practice Exercises include additional pro blems intended to give the interested reader more practice in dea ling with a subject. These exercises have been collected primarily from ea rlier editions of this book as well as exercises developed by other instructors. The For More Practice sections appea r on the CD associated with the speci fi c chapter.

Exercises 1.1 through 1.28

Find the word or phrase from the list below that best matches the description in the following questions. Use the numbers to the left of wo rd s in the a nswer. Each answer should be used only once. 1

abstraction

embedded system

assembler

instruction

bit

instruction set architecture

cache

loca l area network (LAN)

centra l processor unit (CPU )

memory

chip

operating system

compiler

semiconductor

computer family

server

control

supercomputer

datapath

transistor

desktop or personal computer

VLSI (very large scale integrated circuit)

Digital Video Disk ( DVD)

wafer

defect

wide area network (WAN)

DRAM (dynamic random access memory)

yield

1.1 121Active part of the com puter, following the instructions of the programs to the letter. It adds numbers, tests numbers, controls other com ponents, a nd so on.

1.2 121 Approach to the design of ha rdware or software. The system consists of hierarchicallayers, with each lower layer hiding details from the level above.

1.3 121 Binary digit. 1.4 121 Collection of implementations of the sa me instmction set a rchitecnlfe. They a re usually made by the same company and vary in price and performance.

Chapter 1

Computer Abstractions and Technology

1.5 [2J Component of the computer where all running progrJITIS and associated data reside. 1.6 [2J Component of the processor that performs arithmetic operations. 1.7 [2J Component of the processor that tells the datapath, memory, and I/O devices what to do according to the in structions of the program. 1.8 [2J Computer designed for use by an individual, usually incorporating a graphics display, keyboard, and mouse. 1.9 [2J Computer in side another device used for running one predetermined application or collection of software. 1.10 [2J Computer used for running larger programs for multiple users often simultaneously and typically accessed only by a network. 1.11 [2J Computer network that connects a group of computers by a common transmission cable or wireless link within a small geographic area (for example, within the sa me fl oor of a building). 1.12 [2J Computer networks that connect computers spanning great distances, the backbone of the Internet. 1.13 [2J High-performan ce machine, costing more than $1 million. 1.14 [2J Integra ted circuit commonly used to construct main memory. 1.15 [2J Microscopic fla w in a wa fer. 1.16 [2J Nickname for a die or integra ted circuit. 1.17 [2J On/off switch co ntrolled by electricity. 1.18 [2J Optical storage med ium with a storage capacity of more than 4. 7 GB. It was initially marketed for entertainment and later for computer users. 1.19 [2J Percentage of good dies from the total number of dies on the wa fer. 1.20 [2J Progra m that converts a symbolic version of an instruction into the binary versIon. 1.21 [2J Progra m that manages the resources of a computer for the benefit of the programs that run on that machine. 1.22 [2J Program that translates from a higher- level notation to assembly language. 1.23 [2J Technology in which single chip that contain s hundreds of thousands to millions of transistors.

1.8

Exercises

1.24 [2J Single softwa re command to a processor. 1.25 [2J Small, fast memory that acts as a buffer for the main m em ory. 1.26 [2J Specific interface that the hardwa re provides the low-level software. 1.27 [2J Substance that does no t conduct electricity well but is the foundation of integra ted circuits.

1.28 [2J Thin disk sliced from a silicon crystal ingot, which will be later divided into dies.

Exercises 1.29 through 1.45

Using the categories in the list below, classify the following exa mples. Use the letters to the left of the words in the answer. Unlike the previous exercises, answers in this group m ay be used more than o nce.

applications software

high-level programming language

semiconductor

input device

supercomputer

integrated circuit

output device

1.29 1.30

[11 Assembler [11 C++

1.31 [ 1J Liquid crystal display (LCD) 1.32 1.33 1.34 1.35 1.36

[11 Compiler [11 Cray- l [11 DRAM [11 IBM PC [11 Java

1.37 [ 1J Sca nner 1.38 1.39

[11 Macintosh [11 Microprocessor

1.40 [ 1J Microsoft Word 1.41 1.42

[11 Mouse [11 Operating system

personal computer

systems software

Chapter 1

Computer Abstractions and Technology

1.43 ( 1 J Printer 1.44 ( 11 Silicon 1.45 ( 11 Spreadsheet 1.46 (15) In :1 magnetic disk, the disks containing the data are constantly rotating. On average it should take half a revolutio n for the desired data o n the disk to spin under the read/write head. Assuming that the disk is rotating at 7200 revolutio ns per minute (RPM ), what is the average time for the data to rotate under the disk h ead? What is the average time if the disk is spinning at 10,000 revolutio n s per minute?

1.47 [51 A DVD drive, however, works in the Constant Linear Velocity (eLV) m ode. The read head must interact with the concentric circles at a constant rate, whether it is accessing data from the inner or o utermost portions of the disc. This is affected by varying the rotation speed of the disc, from 1600 RPM at the center, to 570 RPM at the o utside. Assuming that the DVD drive reads 1. 35 MB o f user data per second , h ow many bytes ca n the center circle store? H ow many bytes ca n the outside circle store?

1.48 [5 J If a computer issues 30 network requests per second and each request is o n average 64 KB, wiII a 100 Mbit Ethernet link be sufficient?

1.49 [5J What kinds o f n etworks do yo u use o n a regular basis? What kinds o f med ia do they use? How much bandwidth do they provide?

1.50 [ 15) End-to-end delay is an important performance m etric for n et works. It is the time between the point wh en the source starts to send data and the point wh en the data is completely delivered to the destination. Consider two h osts A and B, connected by a single link o f rate R bps. Suppose the two h osts are separated by 111 m eters, and suppose the propagation speed along the link is 5 m /sec. Host A is sending a file o f size L bits to h ost B. a. Obtain an expression for the end -to-end delay in term s o f R, L,

111,

and s.

b. Suppose there is a router between A and B, and the data from A must be forwarded to B by the router. If the forwarding process takes t sec, then what is the end -to-end delay? c. Suppose the router is configured to provide QoS (Quality o f Service) control for different kinds o f data. If the data is a multimedia strea m, such as video conferen ce data, it wiII forward it at a shorter delay o f t/2 sec. For other kinds o f data, the delay is t sec. If h ost A is sending a multimedia stream o f size 2L, what is the end -to-end delay?

1.8

Exercises

1.51 [ I S) Assume you are in a company that will market a certain IC chip. The fixed costs, including R&D, fabrication and equipments, and so on , add up to $500,000. The cost per wafer is $6000 , and each wafer ca n be diced into 1500 dies. The die yield is 50%. Finally, the dies are packaged and tested , with a cost of $ 10 per chip. The test yield is 90%; only those that pass the test wiII be sold to customers. If the retail price is 40% more than the cost, at least how many chips have to be sold to break even? 1.52 [8 ] In this exercise, you will evaluate the performance difference between two C PU architectures, C ISC (Complex Instruction Set Computing) and RI SC (Reduced Instruction Set Computing). Generally speaking, CISC CPUs have more complex instructions th an RI SC CPUs and therefo re need fewer instructions to perfo rm the same tasks. However, typically one C ISC in struction , since it is more complex, takes m ore time to complete than a RI SC instruction. Assume that a certain task needs P C ISC instructions and 2P RI SC instruct ions, and that one CISC instruction takes 8T ns to complete, and one RI SC instruction takes 2T ns. Under this assumption , which one has the better performance? 1.53 [ I S) Suppose there are five com puters connected together to form a loca l area network. The maximum data transport rate (bandwidth) that the network cable ca n p rovide is 10 Mbps. If we use a low-end device (Hub) to connect them, all the computers in the network share the 10 Mbps bandwidth. If we use a high -end device (Switch ), then any two of the com puters ca n communicate with each other with out disturbing the other computers. If you wa nt to downloa d a 10 MB fil e from a remote server, which is located outside your local network, how long will it take if using a Hub? How long wiII it take if using a Switch? Assume the other fo ur computers only communicate with each other, and each has a constant data rate of 2 Mbps. 1.54 [8 ] Sometimes softwa re optimization ca n dra m atically improve the performance of a computer system. Assume that a CPU ca n perform a multiplication operation in 10 ns, and a subtraction operation in 1 ns. How long will it take for the CPU to ca lculate the result of d = a x b - a x c? Could you optimize the equation so that it wiII take less time? 1.55 [8 ] This book covers abstract ions for computer systems at many different levels of detail. Pick another system with which you are familiar and write one or two pa ragraphs describing some of the many different levels of abstraction inherent in that system. Some possibilities include automobiles, homes, airplanes, geomet ry, the economy, a city, and the government. Be sure to identify both high-level and low-level abstractions.

Chapter 1

Computer Abstractions and Technology

1.56 (15) A less technically inclined friend has asked you to explain how computers work. Write a detailed, one-page description for your friend. 1.57 (101 In what ways do you lack a dea r understa nding of how computers work? Are there levels of abstraction with which you are particularly unfamiliar? Are there levels of abstraction with which you are familiar but still have specific questions about? Write at least o ne paragraph addressing each of these questions.

1.58 (15) In this exercise, you will lea rn more about interfaces or abstractions. For example, we can provide an abstraction for a disk like this: Performan ce characteristics: • Capacity (how much data ca n it store?) • Bandwidth (how fast ca n data be transferred between the computer and disk?) • Latency (how long does it take to find a specific position for access?) Functions the interface provides: • Read/write data • Seek to a specifi c position • Status report (is the disk ready to read/write, etc.?) Following this pattern, please provide an abstraction for a network card.

1.59 [5J " In More Depth: Integrated Circu it Cost 1.60 [15) .. In More Depth: Integra ted Circuit Cost 1.61 (10 ) " In More Depth: Integra ted Circuit Cost 1.62 [5J " In More Depth: Integrated Circuit Cost 1.63 (10 ) . 1.64 (10 )

Answers to Check Yourself

In More Depth: Integra ted Circuit Cost

'II In More Depth: Integra ted Circuit Cost

§ 1.1 , page 10: Discussion questions: lots of answers are acceptable. § 1. 3, page 27: Disk mem ory: nonvolatile, long access time (milliseconds), and cost $2-4/G B. Semiconductor mem ory: volatile, short access time (nanoseconds), and cost $200-400/ GB. § 1.4, page 33: I , 3, and 4 are valid reasons.

Computers in the Real World

Information Technology for the 4 Billion without IT

Throughout this book you will see sections what they wanted neA't, villagers sa id they entitled "Computers in the Real World." These wanted access to the Internet! First, they sect ions describe compell ing uses for comput- wanted to learn the prices before taking their ers outside of their typical functions in office crops to the nearest market, which is 35 kiloautomation and data processing. The goa l of meters away. They could also learn about the these sections is to illustrate the diversity of market abroad to make better decisions on uses for information technology. what crops to grow and to increase their bargaining power when it was time to sell them. Problem to solve: Make information tech- Second, they wanted to use Internet telephony nology ava ilable to the rest of humanity, such to talk to relatives in Laos and beyond. as farmers in rural villages, beyond a multilinThe goal was "a rugged computer and gual character set like Unicode. printer assembled from off-the-she lf components that draws less than 20 watts in normal Solution: Develop a computer, software, and use- less than 70 watts when the printer is a communicat ion system for a rural farming printing-a nd that can survive dirt, heat, and village. However, there is no electricity, no immersion in water." telephone, no technical support, and the vilThe resulting Jhai PC design uses flash lagers do not read English. memory instead of a disk drive, thereby elimiThe Jhai Foundation took on this challenge nating moving parts from the PC to make it for five villages in the Hin Heup district of more rugged and easier to maintain. Rather Laos. This American-Lao foundation was than use a power-hungry cathode ray tube, it founded to raise the standard of living for has a liquid-crystal display. To lower costs and rural Laos by developing an export economy. power, it uses an 80486 microprocessor. The It also built schools, installed wells, and power is supplied by a car battery, which can started a weaving cooperative. When asked be charged by a turning bicycle crank. An old

A laotian \llIIager who wanted access to the Internet.

des igned this person al digital ass istant, which is similar to the Palm Pilot, to meet th e needs of vill agers in third world countries. Input is through a touch screen and speech recogni tion so th at people need not be able to write to use it. It u ses three AAA batteries, which last 3 to 4 hours. The cos t is $250, and there is no speci al solution for communication. It's unclea r whether village rs in the deve loping world would spend $250 on a PDA, where even batteries are a luxury.

dot matrix printer completes the hardware, bringing the cost to about $400. The operating To learn more see these references on system is Linux, and the appli cations are the III library accounting, email, and letter writing, which "Making the Web world-wide," Tile Eco/lomist, Septem exp atriates are tailoring to the Lao language . ber 26, 2002, www.jhai.orglecono rnist The communication solutio n is to adapt th e WiFi (IEEE 802.llb) wireless network (see The Jhai Foundatio n, www.j ha i.org/ Chapter 8) . The plan is to boost the signal "Computers for the T hird Wo rld," Scielltiftc AmcriCtJIl, October 2002 u sing larger antennas and then place repea ter stations o n the hilltops betwee n the vill age and the market city. These repea ters ge t their power from solar ce lls. The loca l phone system ties to it at the far end , which completes the connection to the Internet. Twenty- five volun tee rs in Silicon Valley are developing this Th ai PC network. An alternative attem pt is the simpu ter, which stand s for "simple, inexpensive, multi Indian \llIIager using the Slmputer. lingual computer." Indian computer scientists

Instructions: Language of the Computer I speak Spanish to God, Italian to women,

French to men, and German to my horse. Charles Y, King of France

1337-1380

2.1 2.2

2.3 2.4

2.5 2.6 2.7 2.8 2.9 2.10 2.11 .. 2.12 2.13 .. 2.14 2.15 2.16

2.17 2.18 .. 2.19 2.20

Introduction 48 Operations of the Computer Hardware 49 Operands of the Computer Hardware 52 Representing Instructions in the Computer 60 Logical Operations 68 Instructions for Making Decisions 72 Supporting Procedures in Computer Hardw.e 79 Communicating with People 90 MIPS Addressing for 32·Bit Immediates and Addresses

Translating and StMing a Program 106 How Compilers Optimize 116 How Compilers Work: An Introduction 121 A C Sort Example to Put tt All Together 121 Implementing an Object-oriented Language Arrays versus Pointers 130 Real Stuff: IA-32 Instructions 134 Fallacies and Pitfalls 143 Concluding Remarks 145 Historical Perspective and Further Reading Exercises 147

130

147

The Five Classic Components of a Computer

Interlace

Chapter 2

Instructions: Language of the Computer

Introduction

instruction set The vocabulary of commands understood by a given architecnlre.

To command a computer's hardware, you must speak its language. The wo rds of a computer's language are called instructions, and its vocabulary is ca lled an instruction set. In this chapter, you will see the instruction set of a real computer, both in the form written by human s and in the form read by the computer. We introduce in structions in a top- down fashion. Starting from a notation that looks like a restricted programming language, we refin e it step-by-step until you see the real language of a real computer. Chapter 3 continues our downwa rd descent, unveiling the representation of integer and fl oating-point numbers and the hard wa re that operates on them. You might think that the languages of computers would be as diverse as th ose of humans, but in reality computer languages are quite similar, more like regional dialects than like independent languages. Hence, once you lea rn one, it is easy to pick up others. This similarity occurs because all computers are constructed from hardwa re technologies based on similar underlying principles and because there are a few basic operations that all computers must provide. Moreover, computer designers have a common goa l: to find a language that makes it easy to build the hardwa re and the compiler while maximizing performance and minimizing cost. This goa l is time-honored; the following quote was written before you could buy a computer, and it is as true today as it was in 1947: It is easy to see by formal-logical methods that there exist certain {instrllction sets} that are in abstract adequate to control and calise the execution ofany sequence ofoperations. ... The really decisive considerations from the present point of view, in selecting an {instrllction set}, are more ofa practical nattlre: simplicity ofthe equipment demanded by the {instruction set}, and the clarity ofits application to the actually important problems together with the speed of its handling of those problems. Burks, Goldstine, and yon Neumann, 1947

The "simplicity of the equipment" is as valuable a con sidera tion for com put ers of the 2oo0s as it was for th ose of the 1950s. The goal of this chapter is to teach an instruction set that follows this advice, showing both how it is rep resented in hardware and the relation ship between high-level programming lan gua ges and this more primitive one. Our examples are in the C programming language; Section 2. 14 shows how these would change for an object -o riented language like Java.

2.2

Operation s of the Compute r Hardware

By lea rning how to represent instructions, you will also discover the secret of computing: the stored-program concept. Moreover you will exercise your "foreign language" skills by writing programs in the language of the computer and running them on the simulator that comes with this book. You will also see the impact of programming languages and compiler optimization on performance. We conclude with a look at the historical evolution of instruction sets and an overview of other computer dialects. The chosen instruction set com es from MIPS, which is typical of instruction sets designed since the 1980s. Almost 100 million of these popular microprocessors were manufactured in 2002, and they are fo und in products from ATI Tech nologies, Broadcom, Cisco, NEC, Nintendo, Silicon Graphics, Sony, Texas Instruments, and Toshiba, among others. We reveal the MIPS instruction set a piece at a time, giving the rationale along with the computer strucnlfes. This top-down , step-by-step tutorial weaves the components with their explanations, making assembly language m ore palatable. To keep the overall picture in mind, each section ends with a fi gure summarizing the MIPS instruction set revealed thus far, highlighting the portions presented in that section.

Operations of the Computer Hardware Every computer must be able to perform arithmetic. The MIPS assembly language notation

adda , b , c instructs a computer to add the two va riables band c and to put their sum in a. This notation is rigid in that each MIP S arithmetic instruction performs only one operation and must always ha ve exactly three variables. For exa mple, suppose we wa nt to place the sum of variables b, c, d, and e into va riable a. (I n this section we are being deliberately vague about what a "variable" is; in the next section we'll expla in in detail.) The following sequence of instructions adds the four va riables:

adda , b , c adda , a , d adda , a , e

# The sum of band c is placed in a . # The sum of b , c , and d is now in a . # The sum of b , c , d , and e is now in a .

stored -program concept The idea that instructions and data of many types can be stored in memory as numbers, leading to the stored program computer.

There must certainly be instrllctions for performing the fundamental arithmetic operations.

Burks, Goldstine, and von Neumann, 1947

Chapter 2

Instructions: Language of the Computer

Thus, it takes three instructions to take the sum of four va riables. The words to the right of the sharp symbol (It) on each line above are comments for the human reader, and the computer ignores them. Note that unlike other programming languages, each line of this language ca n contain at most one instruction. Another difference from C is that comments always terminate at the end of a line. The natural number of opera nds for an operation like addition is three: the two numbers being added together and a place to put the sum. Requiring every instruction to have exactly three opera nds, no more and no less, conforms to the philosophy of keeping the hardware simple: hardware for a va riable number of opera nds is more complicated than hardware for a fi xed number. This situation illustrates the first of four underlying principles of hardwa re design:

Design Principle 1: Simplicity favors regularity. We can now show, in the two exa mples that follow, the relationship of progra ms written in higher-level programming languages to programs in this more primitive notation.

Compiling Two C Assignment Statements into MIPS

EXAMPLE

This segment of a C program contains the five va riables a, b, C, ct, and e . Since lava evolved from C, this example and the next few work for either high -level programming language:

b + c;

The translation from C to MIPS assembly language instructions is performed by the compiler. Show the MIPS code produced by a compiler.

ANSWER

A MIPS instruction operates on two source operands and places the result in one destination operand. Hence, the two simple statements above compile directly into these two MIPS assembly language in structions: adda , b , c subd , a , e

2.2

Operations of the Computer Hardware

Compiling a Complex C Assignment into MIPS

A somewhat complex statement contains the fi ve variables f, g, h, i, and j:

EXAMPLE

f = (g + h) - (i + j) ; \Vhat might a C compiler produce?

The compiler must break this statement into several assembly in structions since only one operation is performed per M IPS instruction. The first M IPS in struction calculates the sum of 9 and h. We must place the result somewhere, so the compiler crea tes a temporary variable, called to: add tO , g , h # temporary variable to con tain s 9 + h Although the next operation is subtract, we need to calculate the sum of i and j before we ca n subtra ct. Thus, the second instruction places the sum i and j in another temporary variable crea ted by the compiler, called ti: add tl , i , j

It temporary variable tl con tain s i + j

Finally, the subtract instruction subtracts the second sum from the first and places the difference in the va riable f, completing the compiled code: sub f , tO , ti/t f gets to - tl, which is (g + h) - (i + j) Figure 2. 1 summarizes the portions of M IPS assembly language described in this section. These instructions are symbolic representations of what the MIPS processor acnlally understands. In the next few sections, we will evolve this sym bolic representation into the real language of MIPS, with each step making the symbolic representation more concrete. MIPS assembly language Category

Instruction

Example

Meaning

Arithmetic

odd

add a.b.c sub a.b.c

a- b+c a- b c

subtract

Comments Always three operands Always three operands

FIGURE 2.1 MIPS architecture revealed In Section 2.2. The real computer operands will be unveiled in the next section. Highlighted portions in such summaries show MIPS assembly language structures imroduced in this section; for this first figure, aU is new.

ANSWER

Chapter 2

Check Yourself

Instructions: Language of the Computer

For a given function, which programming language likely takes the most lines of code? Put the three representations below in order. 1. lava

2. C 3. MIPS assembly language Elaboration: To increase portability, Java was originally envisioned as relying on a software interpreter. The instruction set of this interpreter is called Java bytecodes, which is quite different from the MIPS instruction set. To get performance close to the equivalent C program, Java systems today typically compile Java bytecodes into the

native instruction sets like MIPS . Because this compilation is normally done much later than for C programs, such Java compilers are often called Just-in-Time (JIT) compilers . Section 2 .10 shows how JITs are used later than C compilers in the startup process, and Section 2 .13 shows the performance consequences of compiling versus interpreting Java programs . The Java examples in this chapter skip the Java bytecode step and just show the MIPS code that are produced by a compiler.

Operands of the Computer Hardware

word The natural unit ofaccess in a computer, usually a group of 32 bits; corresponds to the size of a register in the MIPS architecture.

Unlike programs in high-level languages, the operands of arithmetic instructions are restricted; they must be from a limited number of special locations built directly in hardware called registers. Registers are the bricks of computer construction: registers are primitives used in hardware design that are also visible to the programmer when the computer is completed. The size of a register in the MIPS architecture is 32 bits; groups of 32 bits occur so frequently that they are given the name word in the MIPS architecture. One major difference between the variables of a programming language and registers is the limited number of registers, typically 32 on current computers. MIPS has 32 registers. (See Section 2.19 for the history of the number of registers.) Thus, continuing in our top-down , stepwise evolution of the symbolic representation of the MIPS language, in this section we have added the restriction that the three operands of MIPS arithmetic instructions must each be chosen from one of the 32 32-bit registers. The reason for the limit of 32 registers may be found in the second of our four underlying design principles of ha rdware technology:

2.3

Operands of the Computer Hardware

Design Principle 2: Sm aller is fast er. A ver y large number of registers may increase the clock cycle time simply because it takes elect ronic signals longer when they must travel farther. Guidelines such as "smaller is faster" are not absolutes; 3 1 registers may not be faster than 32. Yet, the truth behind such observations causes computer designers to take them seriously. In this case, the designer must balance the craving of pro grams fo r mo re registers with the designer's desire to keep the clock cycle fast. Another reason for not using more than 32 is the number of bits it would take in the in struction format, as Section 2.4 demonstrates. Chapters 5 and 6 show the central role that registers play in hardwa re construction; as we shall see in this chapter, effective use of registers is key to program performance. Although we could simply write instructions using numbers for registers, from o to 3 1, the MI PS convention is to use two-character na mes following a dollar sign to represent a register. Section 2.7 will explain the reasons behind these names. For now, we will use $sO , $sl , ... for registers that correspond to variables in C and Java programs and $tO , H I, ... for temporary registers needed to compile the program into MIPS instructions.

Compiling a C Assignment Using Registers It is the compiler's job to associate program variables with registers. Take, for in stance, the assignment statement from our ea rlier example:

EXAMPLE

f = (g + h) - (i + j) ; The va riables f , g, h, i , and j are assigned to the registers $sO, $sl , $s2, $s3 , and $s 4, respectively. What is the compiled MIPS code?

The compiled program is very similar to the prior exa mple, except we replace the va riables with the register names mentioned above plus two temporary registers, $tO and $t I , which co rrespond to the tempo rary va riables above: add $t O,$ sl , $s2 add $t1, $s3 , $s 4 sub $sO , $t O, $t l

# r egis t e r $t O con t ains g + h

# r egis t e r $t1 contains i # f ge t s $tO - HI, which is

+ j (g + h) - (i + j)

ANSWER

Chapter 2

Instructions: Language of the Computer

Memory Operands

d ata tran sfer instructio n A command that moves data between memory and registers. address A value used to delineate the location of a specific data element within a memory array.

Programming languages have simple va riables that contain single data elements as in these examples, but they also have more complex data structures-a rrays and structures. These complex data structures ca n contain many m ore data elements than there are registers in a computer. How ca n a computer rep resent and access such large structures? Recall the five components of a computer introduced in Chapter 1 and depicted on page 47. The processor ca n keep only a small amount of data in registers, but computer memory contains millions of data elements. Hence, data structures (arrays and structures) are kept in memory. As explained above, arithmetic operations occur only on registers in MIPS instructions; thus, MIPS must include in structions that transfer data between memory and registers. Such instructions are called da ta tra nsfer instruction s. To access a word in memory, the instruction must supply the memor y address. Memory is just a large, single-dimensional array, with the address acting as the index to that a rray, starting at O. For example, in Figure 2.2, the address of the third data element is 2, and the value ofM emory(2] is 10. The data transfer in struction that copies data from memory to a register is tra ditionally called load. The format of the load in struction is the name of the operation followed by the register to be loaded, then a constant and register used to access memo ry. The sum of the constant portion of the instruction and the con tents of the second register fo rms the mem ory address. The acnlal MIPS name fo r this instruction is 1w, standing for load word.

•

Processor

100

10 1

Address

Data

Memory

FIGURE 2.2 Memory addresses and contents of memory at those Ioc:atlons. This is a simplification of the MIPS addressing; Figure 2.3 shows the actual MIPS addressing for sequential word addresses in memory.

2.3

Operands of the Computer Hardware

Compiling an Assignment When an Operand Is in Memory

Let's assume th at A is an array of 100 wo rds and that the compiler has associated the va riables 9 and h with the registers $ s 1 and $ s 2 as before. Let's also assume that the starting address, or base address, of the array is in $ s3 . Compile this C assignment statement:

h + A[8] ;

Although there is a single operation in this assignment statement, one of the operands is in memory, so we must first transfer A[8] to a register. The ad dress of this array element is the sum of the base of the array A, found in register $ s3, plus the number to select element 8. The data should be placed in a temporary register for use in the next instruction. Based on Figure 2.2, the first compiled instruction is

$t O,8($s3l # Temporary r eg $t O ge t s A[8]

(On the next page we'll make a slight adjustment to this instruction , but we'll use this simplified version fo r now.) The following instruction ca n operate on the value in $tO (which equals A[8]) since it is in a register. The instruction must add h (contained in $ s 2) to A[ 8] ($ t O) and put the sum in the register corresponding to g (associated with $ s 1):

add

EXAMPLE

$sl ,$ s2 , $tO # g

h + A[8]

The constant in a data transfer in struction is called the offset, and the register added to form the address is ca lled the base register.

ANSWER

Chapter 2

Hardware Software Interface

alignme nt restriction A requirement that data be aligned in memory on natural boundaries

Instructions: Language of the Computer

In addition to associating va riables with registers, the compiler alloca tes data structures like arrays and structures to locations in memo ry. The compiler ca n then place the proper starting address into the data transfer instructions. Since 8-bit bytes are useful in m any programs, most architectures address individual bytes. Therefo re, the address of a word matches the address of one of the 4 bytes within the wo rd. Hence, addresses of sequential wo rd s differ by 4. For exa m ple, Figure 2.3 shows the actual MIPS addresses for Figure 2.2; the byte address of the third word is 8. In MIPS, wo rds must start at addresses that are multiples of 4. This requirement is ca lled an alignment restrictio n, and many architectures have it. (Chapter 5 suggests why alignment leads to faster data transfers.) Computers divide into those that use the address of the leftmost or "big end " byte as the wo rd address versus th ose that use the rightm ost or "little end " byte. MIPS is in the Big Endian ca mp. (Appendix A, page A-43, shows the two options to number bytes in a word.) Byte addressing also affects the array index. To get the proper byte address in the code above, the offset to be added to the base register $ s3 must be 4 X 8, or 32, so that the load address will select A[8] and not A[8/4] . (See the related pitfall of page 144 of Section 2.1 7.)

Processor

100

10 1

Address

Data

Memo ry

FIGURE 2.3 Actual MIPS memory addresses and contents of memory for those words. The changed addresses are highlighted to contrast with Figure 2.2. Since MIPS addresses each byte, word addresses are multiples of four : there are four bytes in a word.

2.3

Operands of the Computer Hardware

The in struction complementary to load is traditionally ca lled store; it copies data from a register to memory. The format of a store is similar to that of a load: the name of the operation, followed by the register to be stored, then offset to select the array element, and finall y the base register. Once again, the M IPS address is specified in part by a constant and in part by the contents of a register. The actual MIPS name is 5W, standing for store word.

Compiling Using Load and Store

Assume variable h is associa ted with register $ 5 2 and the base address of the array A is in $ 53. What is the MIP S assembly code for the C assignment statement below?

A[12]

h + A[8] ;

Although there is a single operation in the C statement, now two of the operands are in memory, so we need even more M IP S instructions. The first two in structions are the same as the prior exa mple, except this time we use the proper offset for byte addressing in the load word instruction to select A[8], and the add instruction places the sum in $tO: lw

HO , 32 ($53l

# Temporary reg $tO gets A[8]

add

HO , $s2 , HO

# Temporary reg $tO gets h + A[8]

The final instruction stores the sum into A[ 12], using 48 as the offset and register $ 53 as the base register.

EXAMPLE

1t0 . 48(1s31

# Stores h + A[8] back into A[12]

Constant or Immediate Operands Many times a program will use a constant in an opera tion- for exa mple, incrementing an index to point to the next element of an array. In fa ct, more than half of the M IP S arithmetic instructions have a constant as an opera nd when running the SPEC2000 benchmarks.

ANSWER

Chapter 2

Hardware Software Interface

Instructions: Language of the Computer

Many programs have m ore va riables than computers have registers. Consequently, the compiler tries to keep the most frequently used va riables in registers and places the rest in mem ory, using loa ds and stores to m ove va riables between registers and mem o ry. The process of putting less commonly used va riables (or th ose needed later) into m em ory is called spilling registers. The hardwa re principle relating size and speed suggests that mem ory must be slower than registers sin ce registers are smaller. This is indeed the case; data accesses are faster if data is in registers in stead of mem ory. Moreover, data is m ore useful when in a register. A MIPS arithmetic in struction ca n read two registers, o perate on them, and write the result. A MIPS data tran sfer instruction only reads one o perand or writes one opera nd , witho ut operating on it. Thus, MIPS registers take both less time to access and have higher th roughput than mem ory--a ra re combination- m aking data in registers both faster to access and simpler to use. To achieve highest performance, compilers must use registers effi ciently.

Using only the instructions we have seen so far, we would have to load a con stant from m em or y to use one. (The constants would have been placed in mem o ry when the program was loaded.) For example, to add the constant 4 to register $s3, we could use the code

$t O, AddrCons t an t4 ( $sl) add $s3 , $s3 ,$ tO

# $t O # $s3

cons t an t 4 $s3 + $tO ($tO

assuming that Add r Cons t a nt4 is the mem ory address of the consta nt 4. An alternative that avoids the load in struction is to offer versions of the a rith metic instructions in which one oper and is a constant. This quick add instruction with one consta nt opera nd is ca lled add immediate or addi . To add 4 to register $s3,we just write addi

$s3 , $s3 ,4

It$s3 = $s3 + 4

Immediate instructions illustrate the third hardwa re design principle, first mentioned in the Fallacies and Pitfalls of Chapter 1:

Design Principle 3: Make the common case fast. Constant o perand s occur frequently, and by including constants inside arithmetic instructions, they are much faster than if constants were loaded from mem ory.

2.3

Operands of the Computer Hardware

MIPS operands Name

Example

32 registers

SsO. Ssl. stO. stl.

2 memory words

Comments •

•

Fast locations for data. In MIPS, data must be in registers to perform arithmetic.

Memory(OI,

Accessed only by data transfer instructions in MIPS. MIPS uses byte addresses, so

Memory(4l, ... ,

sequential word addresses differ by 4 . Memory holds data structures, arrays, and

Memory(429496 72921

spilled registers .

MIPS assembly language Category

Arithmetic

Instruction

Example

Comments

odd

~dd

SsLSs2.Ss3

$sl _ Ss2 + Ss3

Three operands; data in registers

subtract

sub SsLSs2.Ss3

Ss l_ Ss2_Ss3

Three operands; data in registers

Ss1.Ss2.100

$sl - Ss2 + 100

Used to add constants

$s1.100(Ss2)

$sl - Memory[Ss2 + 100]

Data from memory to register

$sLl00(Ss2)

Memory(Ss2 + 100] - Ssl

Oata from register to memory

add immediate ~ddi Data transfer

Meaning

load word store word

FIGURE 2.4 MIPS architecture revealed throug h Section 2.3. Highlighted portions show M IPS assembly language st ructu res introduced in Section 2.3.

Figure 2.4 summarizes the portions of the symbolic representation of the M IPS instruction set described in this section. Loa d word and store wo rd are the instructions that copy wo rd s between memory and registers in the MIPS architecture. Other brands of computers use instructions along with load and sto re to tran sfer data. An architectu re with such alternatives is the Intel IA-32, described in Section 2. 16. Given the importance of registers, what is the rate of increase in the number of registers in a chip over time? I. Very fast: They in crease as fa st as Moore's law, which predicts doubling the number of transistors on a chip ever y 18 months. 2. Very slow: Since programs are usually distributed in the language of the computer, there is inertia in instruction set architecture, and so the number of registers increases only as fast as new instruction sets become viable. Elaboration: Alth ough the MIPS regist ers in thi s book are 32 bits wide, there is a 64bit version of the MIPS instru ction set with 3 2 64-bit regist ers . To ke ep them straight, they are officially ca lled MIPS-3 2 and MIPS-64. In thi s chapter, we use a subset of MIPS-32 . Appendix D shows the differences between MIPS·32 and MIPS-54.

Check Yourself

Chapter 2

Instructions: Language of the Computer

The MIPS offset plus base register addressing is an excellent match to structures as well as arrays, since the register can point to the beginning of the structure and the offset can select the desired element. We'll see such an example in Section 2.13. The register in the data transfer instructions was originally invented to hold an index of an array with the offset used for the starting address of an array. Thus, the base register is also called the index register. loday's memories are much larger and the software model of data allocation is more sophisticated, so the base address of the array is normally passed in a register since it won't fit in the offset, as we shall see . Section 2.4 explains that since MIPS supports negative constants, there is no need for subtract immediate in MIPS.

Representing Instructions in the Computer

binary digit Also called binary bit. One of the two numbers in base 2, 0 or 1, that are the components of information.

We are now ready to explain the difference between the way humans instruct computers and the way computers see instructions, First, let's quickly review how a computer represents numbers, Humans are taught to think in base 10, but numbers may be represented in any base, For example, 123 base 10 = 1111011base2. Numbers are kept in computer hardware as a series of high and low electronic signals, and so they are considered base 2 numbers. (Just as base 10 numbers are called decimal numbers, base 2 numbers are called binary numbers.) A single digit of a binary number is thus the "a tom" of computing, since all information is composed of binary digits or bits. This fundamental building block can be one of two values, which can be thought of as several alternatives: high or low, on or off, true or false, or I or O. Instructions are also kept in the computer as a series of high and low electronic signals and may be represented as numbers. In fact , each piece of an instruction can be considered as an individual number, and placing these numbers side by side forms the instruction. Since registers are part of almost all instructions, there must be a convention to map register names into numbers. In MIPS assembly language, registers $50 to $57 map onto registers 16 to 23, and registers $tO to $t7 map onto registers 8 to IS. Hence, $ 5 0 mea ns register 16, $ s 1 mea ns register 17, $ s 2 mea ns register 18, ... , $tO means register 8, $t1 means register 9, and so on. We'll describe the convention for the rest of the 32 registers in the following sections.

2.4

Repre senting instructi ons in the Compute r

Translating a MIPS Assembly Instruction into a Machine Instruction Let's do the next step in the refin ement of the MIPS language as an example. We' ll show the real MIPS language versio n of the in structio n represented symbolica lly as

EXAMPLE

add $tO,$sl,$s2

first as a combinatio n of decimal numbers and then of binary numbers.

The decimal representatio n is

ANSWER 18

Eadl of these segm ents of an instruction is called a fie/d. The first and last fields (containing 0 and 32 in this case) in combinatio n tell the MIPS computer that this instruction performs addition. The second fi eld gives the number of the register that is the first source operand of the addition operation ( 17 = $ s 1), and the third field gives the other source operand for the addition ( 18 = $ s 2).The fourth field contains the number of the register that is to receive the sum (8 = $to). The fifth field is unused in this instruction, so it is set to O. Thus, this instruction adds register $ s 1 to register $ s 2 and places the sum in register $ to. This instruction ca n also be represented as field s of binary numbers as opposed to decimal: 000000

10001

10010

01000

00000

100000

6 bits

5 bits

6 bits

To distinguish it fro m assembly language, we call the numeric versio n of instructio ns machine language and a sequence of such instructio ns machine code. This layout of the instructio n is called the instruction format. As you ca n see fro m counting the number of bits, this MIPS in structio n takes exactly 32 bitsthe same size as a data word. In keeping with o ur design principle that simplicity fa vors regularity, all MIPS instructio ns are 32 bits long. It would appear that you would now be reading and writing long, tedio us strings of binary numbers. We avoid that tedium by using a higher base than binary that con -

machine language Binary representation used for communication within a computer system. instruction format A form of representation of an instruction composed of fields of binary numbers.

Chapter 2

Hexadecimal

Instructions: Language of the Computer

Hexadecimal

O~,

0000_

0 100"",

s,.,

1000_

C he •

1100_

1~,

0001""0

0 101""0

9~,

1001_

1101_

2~,

001°""0 0011"'0

0 11°""0 0 111"'0

. ~,

101O""" 1011_

111O""",

1111_

",",

,,",

FIGURE 2.5 The hexadeclmal-blnary conversion table. Just replace one hexadecimal digit by the corresponding four binary digits, and vice versa. If the length of the binary number is not a multiple of 4, go from right to left.

hexadecimal Numbers in base 16.

verts easily into binary. Since almost all computer data sizes are multiples of 4, hexadecimal (base 16) numbers are popular. Since base 16 is a powerof2, we can trivially convert by replacing each group of four binary digits by a single hexadecimal digit, and vice versa. Figure 2.5 converts hexadecimal to binary, and vice versa. Because we frequently deal with different number bases, to avoid confusio n we will subscript decimal numbers with tefl , binary numbers with two, and hexadecim al numbers with hex. (If there is no subscript, the default is base 10.) By the way, C and Java use the notatio n Oxnnnn for hexadecimal numbers.

Binary-tcHIexadecimal and Back

EXAMPLE

Convert the following hexadecimal and binary numbers into the o ther base:

eca8 6420 hex 0001 0011 0101 0111 1001 1011 1101 1111 two

ANSWER

Just a table lookup o ne way:

eca8

1110 1100 1010 1000 0110 0100 0 10 OOOOtwo And then the other d irection too:

0001 0011 0101

0111 1001 1011 1101 11l1 t wo

2.4

Representing instructions in the Computer

MIPS Fields MIPS fi elds are given names to make them easier to discuss:

op 6 bits

5 bits

srta mt

f llnct

5 bits

6 bits

Here is the mea ning of each name of the fi elds in MIPS instructions:

• op: Basic operation of the instruction , traditionally ca lled the apcade. • rs: The first register source operand. • rt: The second register source operand. • rd: The register destination operand. It gets the result of the operation. • shamt: Shift amount. (Section 2.5 explains shift instructions and this term; it will not be used until then , and hence the field contains zero.) • funct: Function. This field selects the specific va riant of the operation in the op fi eld and is sometimes called the f unction code. A problem occurs when an instruction needs longer fi elds than those shown above. Fo r exa mple, the loa d word instruction must specify two registers and a constant. If the address were to use one of the 5-bit field s in the format above, the constant within the load word instruction would be limited to only 2 5 o r 32. This constant is used to select elements from arrays or data structures, and it oft en needs to be much larger than 32. This 5-bit field is too sm all to be useful. Hence, we have a confl ict between the desire to keep all instructions the sa me length and the desire to have a single instruction format. This leads us to the fin al hardwa re design principle:

Design Principle 4: Good design demands good compromises. The compromise chosen by the MIPS designers is to keep all instructions the sa me length , thereby requiring different kinds of instruction formats for different kinds of in structions. For example, the format above is ca lled R-type (for register) o r R-format. A second type of instruction format is called I-type (for immediate) o r I-format and is used by the immediate and data transfer instructions. The fields ofl -format are

6 bits

5 bits

cons t ant or add r ess

5 bits

16 bits

opcode The field that denotes the operation and format of an instruction.

Chapter 2

Instructions: Language of the Computer

The 16-bit address mea ns a load word instruction ca n load a ny word within a region of ± 2 15 or 32,768 bytes (± 213 or 8 192 words) of the address in the base re ister rs. Similarly, add immediate is limited to constants no larger than ± 2 . (Chapter 3 explains how to represent negative numbers.) We see that mo re than 32 registers would be difficult in this format, as the rs and rt fi elds would each need another bit , making it harder to fit ever ything in one wo rd. Let's look at the load word instruction from page 57:

# Temporary r eg $t O ge t s A[8]

$t O, 32 ( $s 3 l

Here, 19 (fo r $ 53 ) is pla ced in the rs field , 8 (fo r $tO ) is placed in the rt fi eld , and 32 is placed in the address field. Note th at the mea ning of the rt fi eld has changed for this instruction: in a load word instruction , the rt fi eld specifies the destination register, which receives the result of the load. Although multiple formats complicate the hardwa re, we ca n reduce the com plexity by keeping the formats similar. For exa mple, the first three fi elds of the Rtype and I-type formats a re the same size and have the sa me names; the fourth field in I-type is equal to the length of the last three fi elds of R-type. In case you were wondering, the formats are distinguished by the values in the first field: each format is assigned a distinct set of values in the first fi eld (op) so that the ha rdwa re kn ows whether to treat the last half of the instruction as three fi elds (R-type) or as a single fi eld (I-type). Figure 2.6 shows the numbers used in each fi eld for the MIPS instructions covered th rough Section 2.3. Instruction

............... R

sub (subtract)

add i mm ediate 1W(load word) sw (store word)

'dd

"".

35 ten

43 ten

'"' '"' '"' '"' '"'

'"' '"' '"' '"' '"' '"' '"'

n.a. n.a. n.a.

address

32,.,

34~

n.a . n.a .

n.a . n.a . n.a .

constant address address

FIGURE 2.6 MIPS Instruction encoding. In the table above, "reg~ means a register number between o and 31, ·address~ means a I6-bit address, and "n .a.~ ( not applic.1ble) means this field does not appear in this format. Note that add and sub instructions have the &1me value in the op field; the hardware uses the nUlct field to decide the variant of the operation: add (32) or subtract (34).

2.4

Repre senting instructi ons in the Compute r

Translating MIPS Assembly Language into Machine Language

We can now take an exa mple all the way from what the programmer writes to what the computer executes. If $t 1 has the base of the array Aand $ s2 correspond s to h, the assignment statement

A[300]

EXAMPLE

h + A[300J ;

is compiled into

ItO , 1200(1t!1 II Temporary reg ItO gets A[300]

add

$t0 ,$ s2 , $tO

ItO , 1200(1t!1 II Stores h + A[300J back into A[300J

II Temporary reg ItO gets h + A[300J

What is the MIPS machine language code for these th ree instructions?

For convenience, let's first represent the machine language instructions using decimal numbers. From Figure 2.6, we ca n determine the three ma chine language instructions:

•

1~~~:__ 1;;8~

address/ shamt

funa

---':;-- __~8,---_ -~:;C:~ c.:~- --,"':"'--

The 1w instruction is identified by 35 (see Figure 2.6) in the first field (op). The base register 9 ($t1) is specified in the second field (rs), and the destination register 8 ($t O) is specified in the third field (rt). The offset to select A[300J (1200 = 300 x 4) is found in the final field (address). The add instruction that follows is specified with 0 in the first field (op) and 32 in the last field (funct). The three register operands (18, 8, and 8) are found in the second, third, and fourth fields and correspond to $ s 2, $ to, and $tO.

ANSWER

Chapter 2

Instructi ons: Language of th e Computer

The s wins truction is identified with 43 in the first field . The rest of this final ins truction is identical to the 1winstruction. The binary equivalent to the decimal form is the following (1200 in base 10 is 0000 0100 1011 0000 base 2): 100011

0 1001

0 1000

0‫סס‬oo0

10010

0 1000

10 10 11

0 1001

0 1000

0000 0 1 00 10 11 01000

00000

0000 0 1 00 10 11

‫סס‬oo

100000

‫סס‬oo

N ote the similarity of the binary representations of the first and last instructions. The only difference is in the third bit from the left. Figure 2.7 summarizes the portions of MIP S assembly language described in this section. As we shall see in Chapters 5 and 6, the similarity of the binary representations of related instructions simplifies hardware design. These instructions a re another example of regularity in the MIPS architecture.

Check Yourself

Why doesn't MIPS have a subtract immediate instruction? I. Negative constants appear much less frequently in C and Java, so they are not the common case and do not merit special support. 2. Since the immediate field holds both negative and positive constants, add immediate with a negative number is equivalent to subtract immediate with a positive number, so subtra ct immediate is superfluous.

BIG

The Picture

Today's computers are built on two key principles: I.

Instructions are represented as numbers.

Programs are stored in memory to be read or written, just like numbers.

These principles lead to the stored-program concept; its invention let the computing genie out of its bottle. Figure 2.8 shows the power of the concept; specifically, memory can contain the source code for an editor program, the corresponding compiled machine code, the text that the compiled program is using, and even the compiler that generated the machine code. One consequence of instructions as numbers is that programs are often shipped as files of binary numbers. The commercial implication is that computers can inherit ready-made software provided they are compatible with an existing instruction set. Such "binary compatibility" often leads industry to align around a small number of instruction set architecnlfes.

2.4

Representing Instructions In the Computer

MIPS operands Name

Example

Comments

32 registers 2'" memory words

$sO, $sl, ... , $s7 stO.$tl, ... , st7

Fast locations for data . In MIPS, data must be in registers to perform arithmetic. Registers $sO - $s 7 map to 16-23 and st O- H7 map to 8-15.

Memory[O), Memory[4), ... , Memory[ 4294967292)

Accessed only by data transfer instructions in MIPS. MIPS uses byte addresses, so sequential word addresses differ by 4 . Memory holds data structures, arrays, and spilled registers .

MIPS assembly language Category Arithmetic

Instruction

Example

odd

odd $sl,$s2.$s3

subtract

,"b

0,,.

load word

transfer

store word

Name

,dd

'"'

addi

... R R I I

R.format

Hormat

Field size

Meaning

$sl,$s2.$s3 $sl,100($s2) $sl,100($s2)

Comments

,,2 ,,2

''""

,,3 ,,3

Three operands; data in registers Three operands; data in registers

$s 1 - Memory[ $s2 + 100] Memory[$s2 + 100] - $sl

Data from memory to register Data from register to memory

MIPS machine language Example

a a 8 35 43 6 bits

op op

18 18 18 18 18 5 bits

Comments

19 19 17 17 17

5 bits

ffi

,dd $s1.$s2. $s3

5 bits

0 0 100 100 100 5 bits

6 bits

All MIPS instructions 32 bits

shamt

funct

Arithmetic instruction format

address

32 34

$s1.$s2. $s3 addi $s1,$s2.1 00 $sl.1 00( $s2 ) $sl.1 00($s2 )

'"' "

Data transfer format

FIGURE 2.7 MIPS architecture revealed through Section 2.4. Highlighted portions show MIPS machine language structures introduced in Section 2.4. The two MIPS instruction formats so far are R and I. The first 16 bits are the same: both contain an op field, giv. ing the base operation: an rs field, giving one of the sources; and the rt field, which specifies the other source operand, except for lo.1d word, where it specifies the destination register. R·format divides the last 16 bits into an rd field, specifying the destination register; slulInt field, which Section 2.5 explains; and the fUllet field, which specifies the specific operation of R·format instructions. I·format keeps the last 16 bits as a single address field.

Elaboration: Representing decimal numbers in base 2 gives an easy way to represent positive integers in computer words. Chapter 3 explains how to represent negative numbers, but for now take it on faith that a 32-bit word can represent integers between _231 and +231 -lor -2,147,483,648 to +2,147,483,647, and the 16-bit constant field really holds _2 15 to +2 15 -1 or-32,768 to 32,767. Such integers are called two's complement numbers. Chapter 3 shows how we would encode addi $tO. $tO. - 1 or 1 w $tO. - 4( $ sa), which require negative numbers in the constant field of the immediate format.

Chapter 2

Instructions: Language of the Computer

------------Editor program

I I

I (machine axle ) L

--------------

Proce ss or

: I

C compiler (machine axle)

-------------I-~--~-----I

Payroll data

,~-----------_...! -------------i l ~::k_t:~

J --------------

Source code in C

__:

l_f~~:d~~..P~~~

FIGURE 2.8 The stored-program concept. Stored programs allow a computer that performs accounting to become, in the blink of an eye, a computer that helps an author WTite a book. The switch happens simply by loading memory with programs and data and then telling the computer to begin executing at a given location in memory. Treating instructions in the same way as data greatly simplifies both the memory hardware and the software of computer systems. Specifically, the memory technology needed for data can also be used for programs, and programs like compilers, for instance, can translate code written in a notation far more convenient for humans into code that the computer can understand.

"Contrariwise," continI/cd Tweedledee, "if it was so, it might be; and if it were 50, it would be; bllt as it isn't, it ain't. That's logic." Lewis Carroll, Alice's A dventures ill Wonderland , 1865

Logical Operations Although the first computers concentrated on full words, it soon became clear that it was useful to operate on fields of bits within a word or even on individual bits. Examining characters within a word, each of which are stored as 8 bits, is one exa m ple of such an operation. It follows that instructions were added to simplify, among other things, the packing and unpacking of bits into words. These instructions are called logical operations. Figure 2.9 shows logical operations in C and Java.

2.5

Logical Operations

Logical operations

C operators

Java operators

MIPS Instructions

« »

Bit.t>y.tlit AND

« »)

Bit.t>y.tlit OR

;11 ,,1 and. andi or.o r i

Shift left Shift right

Bit.tly-bit NOT

FtGURE 2.9

C and Java logical operators and their corresponding MIPS Instructions.

The first class of such operations is called sh ifts. They m ove all the bits in a word to the left or right, fillin g the emptied bits with as. For exa mple, if register $ s a contained 0000 0000 0000 00000 000 0000 0000 0000 1001 two = 9 ten and the in struction to shift left by 4 was executed , the new value would look like th is: 0000 0000 0000 0000 0000 0000 0000 1001 OOOOtwo= 144 ten The dual of a shift left is a shift right. The actual name of the two M IPS shift instructions are called shift left logical (s 11) and shift right logical (s r 1). The following instruction performs the operation above, assuming that the result should go in register H2: 511

HZ . $sO .4

# reg $tZ

reg $sO

4 bits

We delayed explaining the shamt field in the R-format. It stands fo r shift amount and is used in shift instructions. Hence, the machine language version of the instruction above is

0' o

shamt

funct

The encoding of sll is a in both the op and fun ct fields, rd contains HZ, rt con tains $sO, and shamt contains 4. The rs fi eld is unused, and thus is set to O. Shift left logica l provides a b.onus benefit. Shiftin g left by i bits gives the sa me result as multiplying by 2' (Chapter 3 explains why). For exa mrle, the above s 11 shifts by 4, which gives the sa m e result as multiplyin g by 2 o r 16.

Chapter 2

Instructi ons: Language of th e Computer

The first bit pattern above represents 9, a nd 9 X 16 = 144, the va lue of the seco nd bit pattern. Another useful operation that isolates fields is AND. (We capitalize the word to avoid confusion between the operation and the English conjunction. ) AND is a bit-by-bit operation that leaves a 1 in the result only if both bits of the opera nds are 1. For exa mple, if register $t 2 still contains

0000 0000 0000 0000 0000 1101 0000 OOOOtwo and register $ t 1 contains 0000 0000 0000 0000 00111100 0000 OOOOtwo then, after executing the MIPS instruction

# reg $tO - reg $t1 & reg $t2

and $tO, Hi, $t2

the value of register $tO would be 0000 0000 0000 0000 0000 1100 0000 OOOOtwo As you can see, AND ca n apply a bit pattern to a set of bits to force as where there is a a in the bit pattern. Such a bit pattern in conjunction with AND is tradition ally called a mask, sin ce the mask "conceals" some bits. To place a value into one of these seas of as, there is the dual to AND, called OR. It is a bit-by-bit operation that places a 1 in the result if either opera nd bit is a 1. To elaborate, if the registers $ tl and $t 2 are unchanged from the preceding exa mple, the result of the MIPS instruction

or $tO,$tl,$t2

# reg $tO

reg $tl I reg $t2

is this value in register $ t 0: 0000 0000 0000 0000 001111010000 OOOOtwo NOT A logical bit-by-bit operation with one operand that inverts the bits; that is, it replaces every 1 with a 0, and every 0 with a 1.

NOR A logical bit-by-bit operation with two operands that calculates the NOT of the OR of the two operands.

The final logical operation is a contrarian. NOT takes one opera nd and places a 1 in the result if one opera nd bit is a 0, and vice versa. In keeping with the twoopera nd format, the designers of MIPS decided to include the instruction NOR ( NOT OR) instead of NOT. If one opera nd is zero, then it is equiva lent to NOT. For exa mple, A NOR a = NOT (A OR 0) = NOT (A). If the register $tl is unchanged from the preceding exa mple and register $t3 has the value 0, the result of the MI PS instruction nor $tO,$tl,$t3 # reg $tO

( reg $t1 I reg $t3)

2.5

Logical Operations

is this va lue in register $ to:

1111 111111111111 11000011 11l1l1l1two Figure 2.9 above shows the relationship between the C and Java operators and the M IPS instructions. Constants are useful in AND and OR logica l operations as well as in arithmetic opera tions, so MIPS also provides the instructions and immediate (andi) and or immediate (ori). Constants are rare for NOR, since its main use is to invert the bits of a single opera nd; thus, the hardware has no immediate version. Figure 2.10, which summarizes the MIPS instructions seen thus far, highlights the logical instructions.

MIPS operands Name

Example

Comments

$sO, $s l , ... , $s7

Fast locations for data. In MIPS, data must be in registers to perfonn arithmetic.

registers

StO. St 1 , ... , St7

Registers $ sO - $ s 7 map to 16-23 and StO - St 7 map to 8-15 .

Memory(O].

Accessed only by data transfer instructions. MIPS uses byte addresses, so

memory

Memory(4]. ... ,

sequential word addresses differ by 4 . Memory holds data structures, arrays, and

words

Memory(429496 7292]

spilled registers .

MIPS assembly language Category

Arithmetic

Instruction

Example

Meaning

.dd

$s1.$s2.$s3

$sl - $s2

$s 1. $s2.$s3

.,d

odd ;"b add i oed

0' '0'

D,,. transfer

$s3

Three operands ; overflow detected

$sl - $s2

$;3

Three operands ; overflow detected

$ s l . $ s2.100

$s1_ $s2

+ 100

$ s1 ,$s2 ,$s3

$sl _ $ s2 & $ s3

Th ree reg . operands ; bit.t>y.t>it AND

0' '0'

$ s1 ,$s2 ,$s3

$sl _ $s21 $s3

Three reg. operands ; bit.t>y.t>it OR

$ s1 ,$s2 ,$s3

$sl--( $ s2 1$s3 )

Three reg. operands ; bit.t>y.t>it NOR

and immediate

andi

$ s1 ,$s2 ,100

$sl _ $s2 & 100

Bit.t>y.t>it AND reg with constant

or immediate

ori

$ s1 ,$s2 ,100

$sl" $ s21 1oo

Bit.t>y.t>it OR reg with constant

shift left logical

,II

$ s1 ,$s2 ,10

$sl" $ s2 «

Shift left by constant

shift right logical

'ri

$ $s1,$s2,10

$sl" $ s2»

Shift right by constant

$s1.100($s2)

$,1

$s1.100($s2)

Memory($s2

subtract add immediate

Logical

Comments

load word store word

+ constant ; overflow

Memory($ s 2

detected

100) Word from memory to register

100)- $; 1 Word from register to memory

FIGURE 2.10 MIPS architecture revealed thus far. Color indicates the portions imroouced since Figure 2.7 on page 67. The back endpapers of this book also list the MIPS machine language.

Chapter 2

The utility ofan automatic computer lies in the possibility of/Ising a given sequence of instructiollS repeatedly, the llumber oftimes it is iterated being dependent IIpon the results ofthe computation. W7lCfl the iteration is completed a dif)erent sequence of [illstrtlctions] is to be followed, so we must, ill most roses, give two parallel traillS of{instmctions] preceded by an illStruction as to which routine is to be followed. This choice call be made to depend upon the sign ofa number (zero being m::koned as plusfor machine purposes). COllSequcntly, we introduce an {instruction] (the conditional tramfer (instmctioll}) which will, depending on the sign ofa given Illlmber, ca lise the proper olle of two routines to be exeruted.

Instructions: Language of the Computer

Instructions for Making Decisions Wh at distinguish es:1 computer from a simple calculat or is its ability to make decis ion s. Based on the input d ata a nd th e va lues crea ted durin g com put ation , diffe re nt instructions execute. Decis ion m akin g is comm only rep resented in p rogra mming langu ages usin g the if state me nt , so metim es combin ed with go to st atem ents a nd labels. MIPS assembly la ngu age includ es two decisio n -m aking instru cti ons, similar to an ifst ate ment with a go to. The fir st in struction is

beq regi s t e r l , r egi s t e r 2 , L1 This instruction mea ns go to the statement labeled Ll jfthe va lue in regi s t e r l equals the value in r eg i s t e r 2. The mnemonic beq stands for branch if equal. The second instruction is

bne registe r l , r egis t e r 2 , Ll It mea ns go to the statement labeled L1 if the value in reg i s t e r 1 does flot equal the value in r egi s t er2. The mnemonic bne stands for branch if flot equal. These two instructions a re traditionally called conditional branches.

Burks, Goldstine, and vo n Neumann, 1947

Compiling if·the~lse into Conditional Branches

EXAMPLE

In the following code segment , f , g, h, i , and j a re va riables. If the fi ve va riables f through j correspond to the five registers $sO through $s 4 , what is the compiled MIPS code for this C if statement? i f (i

ANSWER

j) f

9 + h ; else f

9 - h;

Figure 2.11 is a fl owcha rt of what the MIPS code should do. The first expression compares for equality, so it would seem that we would want beq . In gen eral, the code will be more effi cient if we test for the o pposite condition to branch over the code that performs the subsequent then part of the if (the label E1 s e is defin ed below):. bne $s3 ,$ s 4,E lse

# go t o Else if

1 ~

2.6

Instructions for Making Decisions

The next assignment statem ent perform s a single operatio n , and if all the op erands are allocated to registers, it is just one instructio n:

add $sO , $sl , $s2

It f

g + h (skipped i f i ""j)

We now need to go to the end of the if statem ent. This exa mple introduces ano ther kind of branch , oft en called an unconditional branch . This instructio n says that the processo r always follows the bra nch. To distinguish between conditional and unconditio nal branches, the M IPS nam e for this type of in structio n is jump, abbreviated as j (the label Ex i t is defin ed below) . j

Exit

It go t o Exit

The assignment statem ent in the else portio n of the if statem ent ca n aga in be compiled into a single instructio n. We just need to append the label Else to this instruction. We also show the label Exit that is after this instructio n , showing the end of the if-then-else compiled code:

Else : sub $sO , $sl,$s2 Ex it :

# f

g - h (skipped i f

j )

Notice that the assembler relieves the compiler and the assembly language p rogrammer from the tedium of ca lculating addresses fo r bra nches, just as it does for calculating data addresses fo r loads and stores (see Sectio n 2.1 0) .

F J

i == j?

ii j

E1s e : f =g+ h

f =g - h

Ex i t : FIGURE 2.11 illustration of the options In the If statement above. The left box corresponds to the then part of the if statement, and the right box corresponds to the else part.

conditional branch An instruction th at requires the comparison of two values and th at allows for a subsequent transfer of control to a new address in the p rogram based on the outcome of the com parison .

Chapter 2

Hardware Software Interface

Instructions: Language of the Computer

Compilers frequently create branches and labels where they do not appear in the programming language. Avoiding the burden of writing explicit labels and branches is one benefit of writing in high -level programming languages and is a reason coding is faster at th at level.

Loops Decisions are important both for choosing between two alternatives-found in if statements- and for itera ting a computation- found in loops. The same assem bly in structions are the building blocks for both cases.

Compiling a while Loop in C

EXAMPLE

Here is a traditional loop in C: while (save[ i] i += 1 ;

Assume that i a nd k correspond to registers $ s 3 and $ s 5 and the base of the array save is in $56. What is the MIPS assembly code corresponding to this C segment?

ANSWER

The first step is to load s a ve [i] into a temporary register. Before we ca n load sa ve [i] into a temporary register, we need to have its address. Before we can add i to the base of array 5a ve to form the address, we must multiply the index i by 4 due to the byte addressing problem. Fortunately, we can use shift left logica l since shifting left by 2 bits multiplies by 4 (see page 69 in Section 2.5). We need to add the label Loop to it so that we can branch back to that instruction at the end of the loop: Loop : sll

$t1,$s3,2

# Temp reg $tl

4 * i

To get the address of save [i ], we need to add $t 1 and the base of save in $ 56: add $t1,$t1,$s6

# $tl

address of save[i]

Now we can use that address to load save [i ] into a temporary register:

ItO , O(It!)

# Temp reg $tO

save[i]

2.6

Instructions for Making Decisions

The next instruction perfo rm s the loop test, exiting if sa ve [i ] bne

$tO ,$ s5 , Exit

It go t o Exit if s a ve[iJ

The next instruction adds 1 to i : add

$s3 , $s3 , 1

Iti = i + l

The end of the loop branches back to the while test at the top of the loop. We just add the Ex it label aft er it , a nd we're done:

J Ex it :

Loop

It go to Loop

(See Exercise 2.33 for an optimization of this sequence.)

Such sequences of instructions that end in a branch a re so fund amental to compiling that they a re given their own buzzword: a basic block is a sequence of instructions with out branches, except possibly at the end , and without branch targets or branch labels, except possibly at the beginning. One of the first ea rly ph ases of compilation is breaking the progra m into basic blocks

Hardware Software Interface

The test for equality or inequality is p robably the most popular test, but sometimes it is useful to see if a va riable is less than a nother variable. For example, a for loop may wa nt to test to see if the index va riable is less than O. Such comparisons are accomplished in MIPS assembly language with an instruction that com pares two registers a nd sets a third register to I if the first is less than the second; otherwise, it is set to O. The MIPS instruction is called set on less than, or s 1 t. For example,

basic block A sequence of instructions without branches (except possibly at the end ) and without branch targets or b ranch labels (except possibly at the beginning).

slt

$tO , $s3 , $54

mea ns that register $ t 0 is set to I if the va lue in register $ s 3 is less than the value in register $s 4; otherwise, register $tO is set to O. Constant opera nds are popular in com pa risons. Since register $ze r o always has 0, we ca n already com pa re to O. To com pa re to other va lues, there is an immediate version of the set on less th an instruction. To test if register $ s 2 is less th an the constant 10, we ca n just write slti

$t0 , $s2 , lO

# $t O = 1 if $s2

< 10

Heeding von Neumann's wa rning about the simplicity of the "equipment," the MIP S architecture doesn't include branch on less tha n because it is too com plicated; either it would stretch the clock cycle time or it would take extra clock cycles per instruction. Two faster instructions a re more useful.

Chapter 2

Hardware Software Interface

Instructions: Language of the Computer

MIP S compilers use the s 1t , s 1 t i, beq , bne, and the fi xed va lue of 0 (always ava ilable by reading register $zero) to create all relative conditions: equal, not equal, less than , less than or equal, greater than, greater than or equal. (As you might expect, register $ ze ro maps to register 0. )

Case/Switch Statement

jump add ress table Also called jump table. A table of addresses of alternative instruction sequences.

Hardware Software Interface

Most programming languages have a case or switch statement that allows the progra mmer to select one of many alternatives depending on a single value. The simplest way to implement switch is via a sequence of conditional tests, turning the switch statement into a chain of i[-then-else statements. Sometimes the alternatives may be more efficiently encoded as a table of addresses of alternative instruction sequences, called a j ump address table, and the program needs only to index into the table and then jump to the appropriate sequence. The jump table is then just an array of words containing addresses that correspond to labels in the code. See the In More Depth exercises in Section 2.20 for more details on jump address tables. To support such situations, computers like M IPS include a jump register instruction (j r ), mea ning an unconditional jump to the address specified in a register. The program loads the appropriate entry from the jump table into a register, and then it jumps to the proper address using a jump register. This in struction is described in Section 2.7.

Although there are many statements for decisions and loops in programming lan guages like C and Java, the bedrock statement that implements them at the next lower level is the conditional branch.

Figure 2. 12 summarizes the portions of MIPS assembly language described in this section, and Figure 2. 13 summarizes the corresponding MIPS machine lan guage. This step along the evolution of the MIPS language has added branches and jumps to our symbolic representation, and fixes the useful value 0 pennanently in a register. Elaboration: If you have heard about delayed branches, covered in Chapter 6, don't worry: Th e MIPS assemb ler makes them invis ible to the assembly language programmer.

2.6

Instructions for Making Decisions

MIPS operands Name

Example

Comments

32 registers

$sO. $sl, ... , $s7

Fast locations for data. In MIPS, data must be in registers to perform arithmetic. Registers $sO$ s 7 map to 16-23 and stO-st7 map to 8-15. MIPS register $zero always equals O.

stO.$tl , ... ,st7 , $zero 2 30 memory words

Acc essed only by data transfer instructions in MIPS. MIPS uses byte addresses , so sequential word addresses differ by 4 . Memory holds data structures , arrays , and spilled registers .

Memory[O), Memory[4), ... , Memory[ 4294967292]

MIPS assembly language Category Arithmetic Data transfer

Instruction

Example

Meaning

odd

,dd

$s 1. $s2.$s3

$sl _ $s2

+ $s3

Three operands ; data in registers

subtract

,ob

$s 1. $s2.$s3

$sl _ $s2- $s3

Three operands ; data in registers

$s l .100($s2 )

$s 1 _ Memory[$ s 2

$sl.100($s2)

Memory($ s 2

,od

$s1.$s2.$s3

$sl - $s2 &. $s3

Three reg. operands ; bit.tly-bit AND

$sl - $s2 1 $s3

Three reg. operands ; bit.tly-bit OR

00'

$s1.$s2.$s3 $s1.$s2.$s3

$sl -

Three reg. operands ; bit.tly-bit NOR

and immediate

andi $s l .$s2. 100

Bit.tly.tlit AND reg with constant

$s l .$s2. 100

$sl - $s2 1 100

Bit.tly.tlit OR reg with constant

shift left logical

",,11

$sl - $s2 &. 100

$s 1. $s2. 10

$sl - $s2

shift right logical

;,1

$$s1.$s2.10

$sl - $s2» 10

Shift right by constant

branch on equal

b" bo,

$ sl. $ s2.L

if ( $sl """ $s2 ) goto L

Equal t est and branch

$ sl. $ s2.L

if ( $sl!= $ s2)gotoL

Not equal test and branch

,1t

$ s1. $ s2. $ s3

if ( $s2 < $ s 3) $sl = 1 ;

Compare less than ; used with beq , bne

load word store word

ood 0'

Logical

or immediate

branch on not

Comments

+ 100]

Data from memory to register

+ 1(0) _ $ s 1

Data from register to memory

($ s21$s3)

Shift left by constant

equal Conditional branch

set on less than

else $sl = 0 set on less than

,1t

$ sl. $ s2.1 00

immediate Unconditional jump

FIGURE 2.12

jump

if ( $s2 < 100)$sl_1;

Compare less than immediate; used with

else $sl = 0

beQ , bne

go to L

Jump to target address

MIPS architecture revealed through Section 2.6. Highlighted portions show MIPS structures introouced in Section 2.6.

C has many statements for decisions and loops while MIP S has few. Which of the following do or do not explain this imbalance? \-Vhy? I. More decision statements make code easier to read and understand.

2. Fewer decision statements simplify the task of the underlying layer that is responsible for execution.

Check Yourself

Chapter 2

Instructions: Language of the Computer

MIPS machine language Name

Format

,dd

"" ".-

'"' " '" and1

ori

," ,,'

Example

0 0

18 18

19 19

35 43

18 18

0 0 0

18 18 18

19 19 19

13 0

18 18 0

I I R

R R

b" b",

,it

Field size

R·fonnat

Hannat

0 4

5 0 2

6 bits

5 bits

0' 0'

Comments

0 0

17 17

32 34

100 100

17 17 17

36 37 39

18 18

18 19

'"' " "" dnd i

100 100

2500 ffi ffi

$5 1. $52.$53 $5 1. $52.$53 $5 1. $52.$53

$5 1 ,$52.100 $5 1 ,$52.100 $51.$52.10 $51.$52.10 b" S51.Ss2.100 b", $51.$s2.100 ,it $51.$s2.$s3 j 10000 (see Se (wow open tab at bar is great)

Fourth line of the keyboard poem " Hatless Atlas," 199 1 (some give names to ASCII characters: "!" is "wow," "(" is open, "I" is bar, and so on)

Communicating with People Com puters were invented to crunch numbers, but as soon as they becam e com mercially viable they were used to process text. Most computers to day use 8 -bit bytes to represent characters, with the America n Standard Code for Inform ation Interchange (ASCII ) being the representatio n that nea rly everyone follows. Figure 2.2 1 summa rizes ASCII. A series of instructio ns can extract a byte fro m a word , so load word a nd store word are sufficient fo r transferring bytes as well as wo rds. Because of the popularity

2.8

Communicating with People

.ElI.ElI.ElI.ElI.ElI• • 32

space

33 34 35 36

! •

49 50 51 52 53 54 55 56 57

# $

•

& •

40 41 42 43 44 45 46 47

( )

80 81 82

68 69

85 86 87

98 99 100 10 1 102 103 104 105 106 107 108 109 110 111

5 T U

•

(

•

Worn

5. Pseudcx:lirect addressing Address

- - -I

Memory

]6>----t~wo"'~ •

FIGURE 2.24 illustration of the live MIPS addressing modes. The operands are shaded in color. The operand of mode 3 is in memory, whereas the operand for mode 2 is a register. Note that versions of lo.1d and store access bytes, halfwords, or words. For mode I, the operand is 16 bits of the instruction itself. Modes 4 and 5 address instructions in memory, with mode 4 adding a 16-bit address shifted left 2 bits to the PC and mode 5 concatenating a 26-bit address shifted left 2 bits with the 4 upper bits of the Pc.

102

Chapter 2

Instructions: Language of the Computer

Decoding Machine Code

EXAMPLE

What is the assembly la nguage state ment co rresponding to this machine instruction?

OOaf8020hex

ANSWER

The first step in converting hexadecimal to binary is to find the op fields: (Bits:

31 28 26

0000 0000 1010 1111 1000 0000 0010 0000

We look at the op fi eld to determine the operation. Referring to Figure 2.25, when bits 31-29 are 000 and bits 28-26 are 000, it is an R-format instruction. Let's reformat the binary instruction into R-format fi elds, listed in Figu re 2.26: op

000000

"00101

shamt

funct

01111

10000

00000

100000

The bottom portion of Figure 2.25 determines the operation of an R-fo rmat in struction. In this case, bits 5-3 are 100 and bits 2-0 a re 000, which mea ns this bin ary pattern represents an add instruction. We decode the rest of the instruction by looking at the field values. The decimal values are 5 for the rs field , 15 for rt, 16 for rd (shamt is unused). Figu re 2.18 says these numbers represent registers $a 1, $t 7, and $ s O. Now we can show the assembly instruction: add $ sO , $al , $t 7

Figure 2.26 shows all the MIPS ins truction forma ts. Fig ure 2.27 shows the MIPS assemb ly language revealed in Cha pter 2; the remaining hidden portion of MIPS instructions deals mainly w ith arithmetic covered in the next cha pter.

2.9

103

MIPS Addressing for 32·111t Immedlates and Addresses

op(31:26) 28-26

0(000)

1{OO1)

2{O10)

3(011)

4(100)

j um p & lin k

branc h eq

5(101)

6(110)

7(111)

31-29 O{OOO)

R-format

Bl tz/gez

1{OO1)

odd i mm ediate

addiu

2{O10)

TlB

Fl Pt

4{1(0)

load byte

load ha lf

5(101)

s t ore by t e

s t ore half

6(110)

1wcO

lwcl

7(111)

swcO

swc l

3(011)

j ump set less th a n i mm.

,,'

s lt i u

andi

load word

lb"

s t o r e wo r d

br anch

or i

,""

blez

bgtz

xori

load uppe r i mm

,,, m

op(31:26)=010000 (TlB), rs(25:21) 23--21

0(000)

1{OO1)

2{01O)

3(011)

4(100)

5(101)

6(110)

7(111)

25--24 O{OO)

mfcO

c f cO

mtcO

ctcO

1{O1) 2(10) 3(11)

op(31:26)=OOOOOO (R·format), funet(5:0)

0(000)

1{OO1)

2{O10)

3(011)

4(100)

5(101)

6(110)

7(111)

5-3

shi ft ri gh t logic a l

O{OOO)

shi ft 1e ft logical

1{OO1)

jump reg.

j a1r

2{01O)

mfhi

mthi

mfl 0

mtl0

3(011)

mult

mu lt u

div

di vu

4{1(0)

odd

addu

sub t rac t

subu

set 1. t.

sltu

5(101)

sl1v syscall

break

ood

s r 1v

srav

no t o r (no r )

6(110) 7(111)

FIGURE 2.25 MIPS Instruction encoding. This notation gives the value of a field by row and by column. For example, the top portion of the figure shows load wo r d in row number 4 ( IOO two for bits 31- 29 of the instruction) and column number 3 (0111\..,) is defined in the bottom part of the figure. Hence, s u bt rac t in row 4 and column 2 of the bottom section means that the funct field (bits 5--0) of the instruction is 1000101\..,. The Fl Pt value in row 2, column 1 is defined in Figure 3.20 in Chapter 3. B1 tzl gel is the opcode for four instructions found in AppendixA: b1tz , bgez , b1tza 1 , and bgeza 1. Chapter 2 describes instructions given in full name using color, while Chapter 3 describes instructions given in mnemonics using color. Appendix A covers all instructions.

104

Chapter 2

Instructions: Language of the Computer

Name

Fields

Field size

6 bits

R·format

0' 0' 0'

I{ormat J.format

FIGURE 2.26

Comments

5 bits

6 bits

All MIPS instructions 32 bits

" "

shamt

furoct

Arithmetic instruction format

address/immediate

target address

Transfer. branch. imm. format Jump instruction format

MIPS Instruction formats In Chapter 2. Highlighted portions show instruction formats introduced

in this section.

Check Yourself

What is the range of addresses for conditional branches in M IPS (K = 1024)? I. Addresses between 0 and 64K - I 2. Addresses between 0 and 256K - I 3. Addresses up to about 32K before the branch to about 32K after

4. Addresses up to about 128K before the branch to about 128K after

What is the range of addresses for jump and jump and link in M IPS (M = 1024K)? 1. Addresses between 0 and 64M - I 2. Addresses between 0 and 256M - I 3. Addresses up to about 32M before the branch to about 32M after

4. Addresses up to about 128M before the branch to about 128M after 5. Anywhere within a block of 64M addresses where the PC supplies the upper 6 bits 6. Anywhere within a block of 256M addresses where the PC supplies the upper 4 bits

What is the M IPS assembly language instruction corresponding to the machine instruction with the value 0000 OOOOhex? I. J

2. R- format 3. addi 4. s 11 5. mfcO

6. Undefined opcode: there is no legal instruction that corresponds to O.

2.9

105

MIPS Addressing for 32·111t Immedlates and Addresses

MIPS operands Name

Example

Comments

32 registers

SsO-Ss7. stO-st9. Szero. SaOSa3. SVO- SVl. Sgp. Hp. Ssp. Sra. Sat

Fast locations for data. In MIPS, data must be in registers to perform arithmetic. MIPS register Sze ra always equals O. Register Sat is reserved for the a ssembler to handle large constants.

2 30 memory words

Memory[O], Memory[4]. ... , Memory[4294967292]

Accessed only by data transfer instructions. MIPS uses byte addresses, so sequential word a ddresses differ by 4. Memory holds data structures, arrays, and spilled registers, such as those saved on procedure calls.

MIPS assembly language Category

Instruction

Example

Arithmetic

.dd subtract

'dd ;ch

Data transfer

Meaning

Comments

Ss1.Ss2.Ss3

Ss l _ Ss2 + Ss3

Three register operands

Ss1.Ss2.Ss3

Ss l _ Ss2 - Ss3

Three register operands

a dd immediate load word

a ddi Ss1.Ss2.100 Ss1.100(Ss2)

Ss l _ Ss2 + 100

Used to add constants

Ss l _ Memory(Ss2 + 100)

Word from memory to register

store word

" 1h

Ssl.100(Ss2)

Memory[ Ss2 + 100]- Ssl

Word from register to memory

Ssl.100( Ss2) Ssl.100( Ss2)

Ss 1 = Memory(S s2 + 10O] Memory[Ss2 + 100] = Ssl Ss 1 _ Memory(S s2 + 10O]

Halfword memory to register Halfword register to memory

load half store half load byte store byte load upper immed.

." "

Logical

'" and immediate or immediate shift left logical

Conditional branch

Uncondi· tional jump FIGURE 2.27

;h 1b ;b

Ssl.100( Ss2) Ssl.100( Ss2)

1ui

Ss1.100

'"'

Ss1.Ss2.Ss3 Ss1.Ss2.Ss3 Ss1.Ss2.Ss3

a ndi ori

Ss1.Ss2.100 Ss1.Ss2.100 Ss1.Ss2.10

",,,

shift right logical branch on equal

;11 ,,1 h',

Ss1.Ss2.10 Ss1.Ss2. 25

branch on not equal

hoo

Ss1.Ss2. 25

set on less than

;It

Ss1.Ss2.Ss3

set less than im medi ate jump

s1ti Ss1.Ss2. 100

jump register

j j,

jump and link

jal

Memory[Ss2 + 100]- Ssl Ssl = 100 . 2 16 Ss l Ss l Ss l Ss l

_ _ _ _

Ss2 & Ss3 Ss21 Ss3 - (Ss2ISs3) Ss2 & 100

Ss l _ Ss21100 Ss l _ Ss2« 10 Ss l _ Ss2» 10 if(Ssl __ Ss2)goto PC + 4 +1OO if(Ssl !_ Ss2)goto PC + 4 +1OO if(Ss2< Ss3) Ss l_1; elseSs1 = 0

Byte from memory to register Byte from register to memory Loads constant in upper 16 bits Three reg. operands; bit·by-bit AND Three reg. operands; bit·by-bit OR Three reg. operands; bit·by-bit NOR Bit-by-bit AND reg with constant Bit-by-bit OR reg with constant Shift left by constant Shift right by constant Equal test; PCrelative branch Not equal test; PCl"e la tive Compare less than; for beq, bne

if( Ss2 < 100) Ss l = 1; else Ss 1 = 0 go to 10000

Compare less than constant

leo

gotoSra

For switch, procedure return

2500

Sra _PC+4;goto l0000

For procedure call

2500

Jump to target address

MIPS assembly language revealed In Chapter 2. Highlighted portions show portions from Sections 2.8 and 2.9.

106

Chapter 2

Instructi o ns : Lang uage of th e Computer

Translating and Starting a Program This section describes the four steps in tran sforming a C program in :I fil e on disk into a program running on a computer. Figure 2.28 shows the tran slation hierarchy. Some systems combine these steps to reduce translation time, but these are the logica l four phases that programs go through. This section follows this translation hiera rchy.

I C program I

( Compiler)

Assembly language program

( Assembler

IObject: Machine language mcx:lule I IObject: Library routine (machine language) I /

(

Linker )

"Exea.Jtable: Machine language program

(

Loader

M emory

FIGURE 2.28 A tr ans lati o n hie rarchy f or C. A hlgh·level·language program IS first complIed mto an assembly langu.1ge program and then assembled into an object mooule in machine language. The linker combines multiple modules with library routines to resolve all references. The loader then places the machine cooe into the proper memory locations for execution by the processor. To speed up the translation process, some steps are skipped or combined together. Some compilers produce object mooules directly, and some systems use linking loaders that perform the last two steps. To identify the type of file, UNIX follows a suffix convention for files: C source files are named x. c, assembly files are x. 5, object files are named x. 0, statically linked library routines are x. a , dynamically linked library routes are x. 50, and executable files by default are called d . OU t. MS-DOS uses the suffixes .C, .ASM, . OBJ, .l l B, . Dll, and. EXE to the same effect.

2.10

Translating and Starting a Program

107

Compiler The compiler transfo rms the C program into an assembly language program, a sym bolic form of what the m achine understands. High -level-language program s take ma ny fewer lines of code than assembly language, so progra mmer productivity is much higher. In 1975, m any o perating system s and assemblers were written in assembly language because m em o ries were small and compilers were ineffici ent. The 128,000fold in crease in m em o ry ca pacity per single DRAM chip has reduced progr am size concerns, and optimizing compilers today ca n produce assembly language p rogram s nearly as good as an assembly language expert, and som etimes even better for la rge program s.

assembly language A symbolic language that can be translated into binary.

Assembler As mentioned o n page 96, since assembly language is the interface to higher-level soft ware, the assembler ca n also treat common va riations of maclline language instmctio ns as if they were instructio ns in their own right. The hardware need not implement these instmctio ns; however, their appea rance in assembly language simplifies translatio n and programming. Such instmctio ns are called pseudoinstructions. As m entio ned above, the MIPS hardwa re m akes su re that register $ze r o always has the value O. That is, whenever register $zero is used , it supplies a 0, and the p rogrammer canno t change the va lue of register $ze r ooRegister $ze ro is used to create the assembly language instructio n move th at copies the contents of o ne register to a no ther. Thus the MIP S assembler accepts this in structio n even tho ugh it is no t fo und in the MIP S architecture:

move $t O, $tl

# r egis t er $t O ge t s regis t e r $t l

The assembler converts this assembly language in structio n into the machine lan guage equivalent of the following instructio n: add

$t O, $ze r o , $t l

# r egis t er $t O ge t s 0 + r egis t er $t l

The MIPS assembler also converts b 1 t (branch on less than) into the two instructions s 1 t and bne mentioned in the example o n page 96. Other examples include bgt, bge, and b 1e. It also converts branches to fa raway locatio ns into a branch and jump. As mentio ned above, the MIPS assembler allows 32-bit constants to be loaded into a register despite the 16-bit limit of the immediate instmctions. In summary, pseudoinstructio ns give MIP S a richer set of assembly language instructio ns than those implemented by the hardwa re. The o nly cost is reserving o ne register, $ a t , fo r use by the assembler. If you a re going to write assembly p rogra ms, use pseudoin structio ns to simplify your task. To understa nd the MIPS

pseudoinstruction A common variation of assembly language instructions often treated as if it were an instruction in its own right.

108

machine language Binary representation lIsed for communication within a computer system. symbol table A table that matches names oflabels to the addresses of the memory words that instructions occupy.

Chapter 2

Instructions: Language of the Computer

architecture and to be sure to get best perfo rmance, however, study the rea l MIPS instructions found in Figures 2.25 and 2.27. Assemblers will also accept numbers in a va riety of bases. In addition to binary and decimal, they usually accept a base that is more succinct than binary yet con verts easily to a bit pattern. MIPS assemblers use hexadecimal, Such features are convenient, but the primary task of an assembler is assembly into m achine code. The assembler turns the assembly language program into an object file, which is a com bination of m achine la nguage instructions, data, and information needed to place instructions p roperly in mem ory. To produce the binary version of each instruction in the assembly language program , the assembler must determine the addresses corresponding to all labels. Assemblers keep track of labels used in branches and data transfer instructions in a sym bol table. As you might expect, the table contains pairs of symbol and address. The object file for UNIX system s typically contain s six distin ct pieces: • The object file header describes the size and position of the other pieces of the object file. • The text segment contain s the machine language code. • The static data segment contains data allocated for the life of the program. (U NIX allows programs to use either static data, which is allocated th rough out the program , or dynamic data, which can grow or shrink as needed by the progra m.) • The relocation information identifies instructions and data wo rds that depend on absolute addresses when the p rogram is loaded into memo ry. • The symbol table contains the remaining labels that are not defin ed, such as external references. • The debugging information contains a concise description of how the modules were compiled so that a debugger can associate machine instructions with C source fil es a nd make data structures readable. The next subsection shows how to attach such routines that have already been assembled, such as library routines.

Linker What we have presented so far suggests that a single change to one line of one procedure requires com piling and assem bling the whole program. Complete retranslation is a terrible waste of com puting resources. This repetition is particularly wasteful for standa rd library routines because programmers would be compiling and assembling routines that by definiti on almost never change. An alternative is to

2.10

109

Translating and Starting a Program

compile and assemble each procedure independently, so that a change to one line would require compiling and assembling only one procedure. This alternative requires a new systems program, called a link editor or linker, whidl takes all the independently assembled machine language programs and "stitdles" them together. There are three steps for the linker: I. Place code and data modules symbolically in memory. 2. Determine the addresses of data and instruction labels.

linker Also called link editor. A systems program that combines independently assembled machine language programs and resolves all undefined labels into an executable file.

3. Patch both the internal and external references. The linker u ses the relocation information and symbol table in each obj ect module to resolve all undefin ed labels. Such references occur in branch instructions, jump in structions, and data addresses, so the job of this program is much like that of an editor: It finds the old addresses and replaces them with the new addresses. Editing is the origin of the name "link ed ito r," or linker for short. The reason a linker makes sense is that it is much fast er to patch code than it is to recompile and reassemble. If all external references are resolved, the linker next determines the memory locations each module will occupy. Reca ll that Figure 2. 17 on page 87 shows the MIP S convention for allocation of program and data to memory. Since the fil es were assembled in isolation, the assembler could not know where a mod ule's instructions and data will be placed relative to other m odules. When the linker places a module in memory, all absolute references, that is, memory addresses that are not relative to a register, must be relocated to refl ect its true location. The linker produces an executable file that ca n be run on a computer. Typically, this fil e has the sa me format as an object fil e, except that it contains no unresolved references. It is possible to have partially linked files, such as library routines, which still have unresolved addresses and hence result in object fil es.

executable file A ti.mctional program in the fonnat ofan object fIle that contains no unresolved references, relocation information, symbol table, or debugging information.

Unking Object Files

Link the two object files below. Show updated addresses of the first few in structions of the completed executable file. We show the instructions in assembly language just to make the example understandable; in reality, the in structions would be numbers. Note that in the object fil es we have highlighted the addresses and symbols that must be updated in the link process: the instructions that refer to the addresses of procedures A and B and the instructions that refer to the addresses of data words Xand Y.

EXAMPLE

110

Chapter 2

Instructions: Language of the Computer

Object file header Name

Text segment

Data segment Relocation information

Procedure A

Text size

lO°he
more instructions: a 1u1 to load the upper 16 bits of the displacement and an add to sum the upper address with the base register Ss 1. (Intel gives two different names to what is called Based addressing mode---Based and Indexed- but they are essentially identical and we combine them here. )

most programs, and so it made sense to be able to set a default large size. This default data size is set by a bit in the code segment register. To override the default data size, an 8-bit prefix is attached to the instruction to tell the ma chine to use the other large size for this instruction. The prefix solution was borrowed from the 8086, which allows multiple prefixes to modify instruction behavior. The three original prefixes override the default segment register, lock the bus to support a semaphore (see Chapter 9), or repeat the foll owing instruction until the register ECX counts down to O. This last prefi x was intended to be paired with a byte move instruction to move a variable number of bytes. The 80386 also added a prefix to override the default address size. The IA-32 integer operations ca n be divided into four major classes: 1. Data movement instructions, including move, push , and pop

2. Arithmetic and logic instructions, including test, integer, and decimal arithmetic operations 3. Control flow, including conditional branches, unconditional jumps, ca lls, and returns 4. String instructions, including string move and string compare The first two catego ries a re unremarkable, except that the arithmetic and logic instruction opera tions allow the destination to be either a register or a memor y location. Figure 2.4 3 shows some typica l IA-32 in struction s and their fun ctions.

140

Chapter 2

Instructions: Language of the Computer

Instruction

Function

JE name

JM P name

El P- na me

CALL name

5P- SP- 4: M[ SP] - El P+5: El P- na me:

MOVW EBX.[EDI +45]

PUS H ESI

EBX-M [E DI+45] 5P- SP 4 : M[SP] - E5 1

POP EDI

ED I-M [ SP] : SP- SP+4

ADD EAX.#6765 TE ST [ OX ./f42

EAX- EAX+6765

MOVSL

M[ EDI]- M[ ES IJ: ED I- EDI +4 : ESI- E5I +4

ti on code) lE I P- na me) : El P- 128 < na me ( El P+128 equ~ 1 (ca nd i

Set condition code (flags) with EDX and 42

FIGURE 2.43 Some typical 1A·32 Instructions and their functions. A list of frequent operations appears in Figure 2.44. The CALL s.wes t he EIP oflhe next instruct ion on t he stack. (EIP is the Intel PC.)

Co nditional branches on the IA-32 are based on condition codes o r j1ags. Condition codes are set as:1 side effect of an operation ; m ost are used to com pare the va lue of a result to O. Bran ches then test th e co ndition codes. Th e argument for condition codes is th at they occur as part of no rmal operations and are faster to test than it is to co mpare registers, as M IPS does fo r beq and bne. The argument again st condition codes is that the com pa re to 0 extends the time o f the operation , since it uses extra hardwa re aft er the operation , and th at oft en the p rogrammer must use compare in structio ns to test a va lue that is not the result of an operation. Moreover, PC- relative branch addresses must be specifi ed in the number of bytes, since unlike M IPS, 80386 instruction s are not all 4 bytes in length. String instructions are part of the 8080 ancestry of the IA-32 and are not com monly executed in most program s. They are often slower th an equiva lent soft wa re routines (see the fallacy on page 143). Figu re 2.44 lists some of the integer IA-32 instructions. Many of the in structions are ava ilable in both byte and word formats.

IA-32 Instruction Encoding Saving the worst for last, the encoding of instructions in the 80836 is complex, with many different in struction formats. Instructions for the 80386 may va ry from 1 byte, when there are no operand s, up to 17 bytes. Figure 2.45 shows the instmction format for several of the example instmctions in Figure 2.43. The opcade byte usually contains a bit saying whether the operand is

2.16

141

Real Stuff: 1A·32 Instructions

Instruction

Meaning

Contro l

Conditio nal and unco nditio nal bran c hes

JNZ, JZ

Jump if condition to EIP

+ Shit offset ; J NE (for JNZ), JE (for JZ) are alternative

names

JMP

Unconditional jump--Shit or 16-bit offset

CA LL

Subroutine call-16.tlit offset; return address pushed onto stack

RET

Pops return address f rom stack and jumps to it

LOO P

Loop branch-- 2.58 2.59

Computers in the Real World

Helping Save Our Environment with Data

Problem to solve: Monitor plants and ani-

other measurements. Data are saved in 1 MB of

mals of our environment to collect information that may influence environmental polices.

flash memory. The onboard 8-bit microprocessor estimates depth from the water pressme. It finds longitude using light intensity data and time of day. It determines sunrise, sunset, and

Solution: Develop rugged, battery-operated,

embedded computers with sensors, wireless communication, and appropriate software. Stanford biologist Barbara Block studies bluefin tuna. One poli cy question was whether the tuna on one side of the Atlantic are different from those on the other side. If so, then each region could set its own quotas. If not, then we need oceanwide quotas. To answer this question, she started implanting tuna with devices that could monitor their journeys. Every two minutes a pop-up satellite arch ival tag (PSAT) records water pressme,

therefore high noon, and calculates the time shift between local noon and Greenwich Mean Time noon, like a navigator using a sextant and chronometer. The water temperature is later matched to satellite records to determine latitude. Block does not rely on fishermen to catch the tuna and retmn PSATs. A PSAT is attached to a fish with a pin that dissolves via electrolysis after the computer tmns on a battery. The tag then floats to the surface and begins transmitting data to satellites. The floating tag can transmit for up to two weeks, sending the data directly to Block's lab.

ambient light, temperatme, time of day, and

Block and students tag a bluefln tuna, which can

A pop-up archival satellite tag and Internal

grow to 2000 pounds and 10 feet In length.

electronics.

Block discovered that bluefin tuna travel more than 10,000 miles per year; tuna tagged near the East Coast of the United States will cross the Atlantic and spawn in both the Gulf of Mexico and the Eastern Mediterranean. Her discovery changed regulations so that tuna are no longer managed separately in the Eastern and Western Atlantic. She is now developing a census of Pacific marine life using smaller tags for smaller animals and tags that transmit each time a fish surfaces. She speculates that tagged tuna could be ideal "vehicles" to monitor ocean change. Berkeley biologist Todd Dawson stud ies the ecology of the coasta l redwood, Sequoia sempervirens, particularly the interaction of sea fog with trees. For years his research involved installing 50 kilograms of gear and kilometers of wire strung to sensors. This work is often done more than 80 meters above the ground. Data could only be retrieved by climbing up to a printer-sized data logger. Berkeley computer scientist David Culler proposed a new approach. Dawson is now placing miniature wireless sensors the size of film

canisters in these trees. Each micromote is less than 3 cubic inches, can transmit up to 40 KBf sec, and can run for months on a C battery. Since micromotes are small and cheap, many can be placed in a tree. Data is collected with a compatible laptop by simply walking to the base of the tree. Dawson found that summertime fog accounts for 25% to 40% of the water that the redwoods receive for the whole year. The trees may even be drinking water directly from fog via a symbiotic relationship with fungi living on their leaves. Dawson predicts wireless sensor networks will change the way people do ecological research. To learn more see these references on

the

library

Block et aI., "Migratory movements, depth preferences, and thermal biology of atlantic bluefin tuna :' Scie/lce 293: 1310-14,2001 "Redwoods:' Prof. Dawson's laboratory site "Redwood's drinking water from fog," Tile Forestry SOI/rce, Nov. 2002 " Tagging of the Pacific Pelagics," www.toppcensus.org

Professor Dawson and student climbing a sequoia to Install fog monitors.

The Mica mlcromote with C battery. It Is about the size of a film canister.

Arithmetic for Computers Numerical precision is the very soul ofscience. Sir D'arcy Wentworth Thomp.c)n 0 ,. Growth ami Form, 19 17

•

3.1

Introduction

3.2

Signed and Unsigned Numbers

3.3

Addition and Subtraction

3.4

Multiplication

3.5

Division

3.6

Floating Point

3.7

Real Stuff: Floating Point in the IA-32

3.8

Fallacies and Pitfalls

3.9

Concluding Remarks

3.10

Historical Perspective and Further Reading

3.11

Exercises

160

170

176

183 189

217

220 225

229

The Five Classic Components of a Computer

lnlertaoe

...

Evaluating

,.".,,,"'

160

Chapter 3

Arithmetic: for Computers

Introduction Computer words are composed of bits; thus wo rds ca n be represented as binary numbers. Although the natural numbers 0, 1, 2, and so on ca n be represented either in decimal or bin ary form , what about the other numbers that commonly occur? For example: • How are negative numbers represented? • What is the largest number that ca n be represented in a computer wo rd? • What happens if an operation creates a number bigger than can be represented? • What about fractions and real numbers? And underlying all these questions is a mystery: How does hardwa re really multiply or divide numbers? The goal of this chapter is to unravel this mystery, including representation of numbers, arithmetic algorithms, hardwa re that follows these algorithms, and the implications of all this for instruction sets. These insights may even explain quirks that you have already encountered with computers. (I f you are famili ar with signed binary numbers, you may wish to skip the next section and go to Section 3.3 on page 170.)

Signed and Unsigned Numbers Numbers ca n be represented in any base; humans prefer base 10 and, as we examined in Chapter 2, base 2 is best for computers. To avoid confusion we subscript decimal numbers with ten and binary numbers with two. In any number base, the value of ith digit d is

d x Base; where i starts at 0 and increases from right to left. This lea ds to an obvious way to number the bits in the word: Simply use the power of the base for that bit. For exa mple,

3.2

161

Sig ne d and Uns ig ned Numbers

represents

23 )

11 x 8) 8

+ (0 X 2') + 11 X 2 1 ) + 11 x 20 )ten + (0 x 4 ) + 11 x 2 ) + 11 xl) ten + 0 + 2 + 1 ten

11 t en

Hence the bits are numbered 0, I , 2, 3, . . . from right to left in a wo rd. The drawing below shows the numbering of bits within a MIPS word and the placement of the number 10 I I two:

313029282726252423222120 1918 1716 1514 1 3 1 211 109876543210 10

I 0 0 0 0 I 0 0 0 0 I 0 0 0 0 I 0 0 0 0 I 0 0 00 10000 11 01 1 1 (32 bits wide)

Since wo rds are drawn vertica lly as well as ho rizontally, leftm ost and rightmost may be unclea r. Hence, the phrase least significa nt bit is used to refer to the right most bit (bit 0 above) and most significa nt bit to the leftm ost bit (bit 3 1). The MIPS word is 32 bits long, so we ca n represent 232 different 32-bit patterns. It is natural to let these combinations represent the numbers from 0 to 232 - I (4,294,967,295 ten ) :

0000 0000 0000 0000 0000 0000 0000 OOOOtwo 0000 0000 0000 0000 0000 0000 0000 0OOl two 0000 0000 0000 0000 0000 0000 0000 0010 two

1111 1111 1111 1111 1111 1111 1111 1101 two 1111 1111 1111 1111 1111 1111 1111 1110 two 1111 1111 1111 1111 1111 1111 1111 1111 two

leas t significant b it The rightmost bit in a MIPS word. most sig nifi cant bit The leftmost bit in a MIPS word.

°ten 1 ten 2t en 4 , 294 , 967 , 293 ten 4,294,9 67 , 294 ten 4,294,967,295 ten

That is, 32-bit binary numbers ca n be rep resented in terms of the bit va lue times a power of 2 (here xi mea ns the ith bit of x): (x3 1 x 2 31 ) + (x30 X 230) + (x29 X 229 ) + ... + (xl X 21) + (xO X 2°)

Base 2 is not natural to human beings; we have 10 fin gers and so find base 10 nat ural. Why didn't computers use decimal? In fact, the first commercial computer did offer decimal arithmetic. The problem was that the computer still used on and off signals, so a decimal digit was simply represented by severa l binary digits. Decimal proved so inefficient that subsequent computers reverted to all binary, con verting to base 10 only for the relatively infrequent input/output events.

Hardware Software Interface

162

Chapter 3

Arithmetic: for Computers

ASCII versus Binary Numbers

EXAMPLE

ANSWER

We could represent numbers as strings of ASCII digits instead of as integers (see Figure 2.2 1 on page 9 1). How much does storage in crease jfthe number 1 billion is represented in ASCII versus a 32-bit integer?

One billion is 1 000 000 000, so it would take 10 ASC II digits, each 8 bits long. Thus the storage expansion would be ( 10 x 8)/32 or 2.5. In addition to the expansion in storage, the hardwa re to add , subtract, multiply, and divide such numbers is difficult. Such difficulties explain why co mputing professionals are raised to believe that binary is natural and that the occasional decimal computer is biza rre. Keep in mind that the binary bit patterns above are simply representatives of numbers. Numbers really have an infinite number of digits, with almost all being o except for a few of the rightmost digits. We just don't normally show leading Os. Hardwa re ca n be designed to add, subtract, multiply, and divide these binary bit patterns. If the number that is the proper result of such operations ca nnot be represented by these rightmost hardwa re bits, overflow is sa id to have occurred. It's up to the operating system and program to determine what to do if overflow occurs. Computer program s calculate both positive and nega tive numbers, so we need a representation that distinguishes the positive from the negative. The most obvious solution is to add a separate sign , which conveniently ca n be represented in a single bit; the name for this representation is sign and m agnitude. Alas, sign and magnitude representation has several sho rtcomings. First, it's not obvious where to put the sign bit. To the right? To the left? Early computers tried both. Second, adders for sign and magnitud e may need an extra step to set the sign because we ca n't know in adva nce what the proper sign will be. Fin ally, a separate sign bit mea ns that sign and magnitude has both a positive and nega tive zero, which ca n lea d to problems for inattentive programmers. As a result of these shortcomings, sign and magnitude was soon abandoned. In the sea rch for a mo re attractive alternative, the question arose as to what would be the result for unsigned numbers if we tried to subtract a large number from a sm all one. The answer is that it would try to borrow from a string of lead ing Os, so the result would have a string of leading Is.

3.2

163

Signe d and Unsigned Numbers

Given that there was no obvio us better alternative, the final solution was to pick the representation that made the hardware simple: leading as mean positive, a nd leading Is m ea n negative. This convention for representing signed binary numbers is ca lled two's complement representation: 0000 0000 0000 0000 0000 0000 0000 OOOOtwo 0000 0000 0000 0000 0000 0000 0000 0001 two 0000 0000 0000 0000 0000 0000 0000 0010 two

0111 111111111111 0111 111111111111 0111 111111111111 1000 0000 0000 0000 1000 0000 0000 0000 1000 0000 0000 0000

- 2 , 147 ,4 83 , 645 ten - 2 , 147 ,4 83 , 646 ten - 2 , 147 ,4 83 , 647 ten - - 2 , 14 7 ,4 83 , 648 ten - - 2 , 147 ,4 83 , 647 ten - - 2 , 147 ,4 83 , 646 ten

11111111 11111111 11111111 0000 0000 0000 0000 0000 0000

1111 1101 two 11111110 two 11l1l1l1two 0000 OOOOtwo 0000 0001 two 0000 0010 two

1111 111111111111 11111111 11111101 two 1111 111111111111 11111111 11111110 two 1111 111111111111 11111111 1111 11 11 two

° t en 1 ten 2 ten

- 3 ten - 2 ten - l t en

The positive half of the numbers, from a to 2, 147,48 3,647ten (2 31 - I), use the same representation as before. The following bit pattern (1000 ... ooOOt\o,"o) represents the m ost negative number -2, 147,483,648 ten (_2 31 ). It is followed by a declining set of nega tive numbers: -2, I47,483,647 ten (1000 ... 000 Itwo) down to - Iten(i lll ... llil two)· Two's complement does have one negative number, -2, 147,483,648 ten , that has no corresponding positive number. Such imbalance was a worry to the inattentive programmer, but sign and magnitude had problems for both the programmer and the hardware designer. Consequently, every computer today uses two's complement binary representations for signed numbers. Two's complem ent representation has the adva ntage that all negative numbers have a I in the m ost significa nt bit. Consequently, hardware needs to test only this bit to see if a number is positive or nega tive (with a considered positive). This bit is often called the sign bit. By recognizing the role of the sign bit, we ca n represent positive and negative 32-bit numbers in term s of the bit va lue times a power of 2: (x3 1 X_2 31 ) + (x30 X 230) + (x29 X 229 ) + ... + (x l X2 1) + (xOX20) The sign bit is multiplied by _2 31 , and the rest of the bits are then multiplied by positive versions of their respective base values.

164

Chapter 3

Arithmetic: for Computers

Binary to Decimal Conversion

What is the decimal va lue of this 32-bit two's complement number?

EXAMPLE

ANSWER

1111 1111 1111 1111 1111 1111 1111 11 OO t wo

Substituting the number's bit values into the fo rmula above: (I X_2 31 ) + (i X 230) + (i X 229 ) + + (i X 22) + (0 X 2') + (0 X 2°)

= _2 31 + 230 = -2, 147,483,648 ten = - 4,en

+ 229 + + + 2, 147,483,644, Iten' Befo re going o n to addition and subtractio n , let's examine a few useful sho rt cuts when working with two's complem ent numbers. The first sho rtcut is a quick way to nega te a two's complem ent binary number. Simply invert every 0 to 1 and every 1 to 0, then add one to the result. This sho rt cut is based o n the observation that the sum of a number and its inverted representatio n must b e 111 ... 111 ,wo> wh ich represents - I. Since x x + x + 1 = 0 o r x + 1 = -x.

+ x - -1 , therefo re

Negation Shortcut

EXAMPLE ANSWER

N egate 2ten , and then check the result b y n egating -2 Ie n"

2ten = 0000 0000 0000 0000 0000 0000 0000 00 10two Negating this number by inverting the bits and adding o ne,

111111111111 111111111111 11111101 two 1two 111111111111 111111111111 11111110 two

- 2 ten

Going the other directio n ,

111111111111 111111111111 11111110 two is first inverted and then increm ented:

00000000000000000000000000000001 two 1two 00000000000000000000000000000010 two

2 ten

3.2

167

Signed and Unsigned Numbers

The second shortcut tells us how to convert a binary number represented in n bits to a number represented with more than n bits. For example, the immediate field in the load, store, branch, add, and set on less than instructions contain s a two's complement 16-bit number, representing -32,768,en (_2 15 ) to 32,767,en (2 15 - I). To add the immediate field to a 32-bit register, the computer must con vert that 16-bit number to its 32-bit equivalent. The shortcut is to take the most significant bit from the smaller quantity- the sign bit-and replicate it to fill the new bits of the larger quantity. The old bits are simply copied into the right portion of the new word. This shortcut is commonly ca lled sign extension.

Sign Extension Shortcut

Convert 16-bit binary versions of 2ten and -2 ten to 32-bit binary numbers.

The 16-bit binary version of the number 2 is 0000000000000010 two

ANSWER

2 ten

It is converted to a 32-bit number by making 16 copies of the value in the most significant bit (0) and placing that in the left -hand half of the word. The right half gets the old value:

00000000000000000000000000000010 t wo

Zten

Let's negate the 16-bit version of 2 using the ea rlier shortcut. Thus, 0000000000000010 two becomes 1111111111111101 two + 1 two 111111111111 111 0two Creating a 32-bit version of the negative number mea ns copying the sign bit 16 times and placing it on the left: 11111111111111111111111111111110 two

EXAMPLE

- Zten

This trick works because positive two's complement numbers really have an infinite number of Os on the left and th ose that are negative two's complement

168

Chapter 3

Arithmetic: for Computers

numbers have all infinite number of Is. The binary bit pattern representing a number hides leading bits to fit the width of the hardwa re; sign extension simply restores some of them. The third shortcut reduces the cost of checking if 0 $; x < y, which matches the index out-of-bounds check for arrays. The key is that negative integers in two's complement notation look like large numbers in unsigned notation; that is, the most significant bit is a sign bit in the former notation but a large part of the num ber in the latter. Thus, an unsigned comparison of x < y also checks if x is negative.

Bounds Check Shortcut

EXAMPLE

ANSWER

Use this shortcut to reduce an index-out-of-bounds check: jump to Index OutOfBound s if $ a 1 '2': $t2 or if $ a 1 is negative.

The checking code just uses s 1tu to do both checks:

s ltu $tO ,$ al , $t2 # Temp reg $tO=O if k) = length or k o r OO lOt\o,"OX 00 11 two '

Figure 3.8 shows the va lue of each register for each of the steps labeled acco rding to Figu re 3.6, with the fin al value of 0000 0 11O two or 6 ten. Color is used to indicate the register values that change on that step, and the bit circled is the one examined to determine the operation of the next step.

Signed Multiplication So far we have dealt with positive numbers. The easiest way to understand how to deal with signed numbers is to first convert the multiplier and multiplica nd to positive numbers and then remember the original signs. The algo rithms should then be run for 3 1 iterations, leaving the signs out of the calculation. As we lea rned in grammar school, we need negate the product only if the origin al signs disagree. It turn s out that the last algo rithm will wo rk for signed numbers provided that we remember that the numbers we are dealing with have infinite digits, and that we are only representing them with 32 bits. Hence, the shifting steps would need to extend the sign of the p roduct fo r signed numbers. \ Vhen the algorithm com pletes, the lower word would have the 32-bit product. IteratIon

o 1

Step

MultIplier

MultIplicand

Product

001

00000010

0000 0000

0011

00000010

0000001 0

2 : Shift left Multiplicand

0011

00000100

0000001 0

3 : Shift right Multipli er

oomamder register are all 64 bits wide, with only the Quotient register being 32 bits. The 32-bit divisor starts in the left half of the Divisor register and is shifted right 1 bit on each iteration. The remainder is initialized with the dividend. Control decides when to shiflthe Divisor and Quotient registers and when to write the new value into the fu.>mainder register.

3.5

185

Division

(

Start

)

1. Subtract the Divisor register from the Remainder register and place the result in the Remainder register

Remainder ~ 0

Remainder < 0 Test Remainder

2a. Shift the Quotient register to the left,

2b. Restore the original value by adding

setting the new rightmost bit to 1

the Divisor register to the Remainder register and place the sum in the Remainder register. Also shift the Quotient register to the left, setting the new least significant bit to 0

3. Shift the Divisor register right 1 bit

No: < 33 repetitions 33rd repetition?

Yes: 33 repetitions

FtGURE 3.11 A division algorithm, using the hardware In Figure 3.10. If the Remainder is positive, the divisor did go into the dividend, so step 2a generates a I in the quotient. A neg.1live Remainder after step I means that the divisor did not go into the dividend, so step 2b generates a 0 in the quotient and adds the divisor to the remainder, thereby reversing the subtraction of step I. The final shift, in step 3, aligns the divisor properly, relative to the dividend for the next iteration. These steps are repeated 33 times.

186

Chapter 3

Arithmetic: for Computers

A Divide Algorithm

Using a 4-bit version of the algorithm to save pages, let's t ry dividing 7,en by

EXAMPLE

ANSWER

21en ,

0000 0 111 two b y 00 10I\\oU'

Figure 3.1 2 shows the value of each register for each of the steps, with the quotient being 3ten and the remainder l Ien' Notice that the test in step 2 of whether the remainder is positive o r negative simply tests whether the sign bit of the Remainder register is a 0 or 1. The surprising requirement of this algorithm is that it takes 11 + 1 steps to get the proper quotient and remainder. This algorithm and hardwa re ca n be refin ed to be faster and cheaper. The speedup comes from shifting the operands and quotient at the sa me time as the subtract. This refin em ent h alves the width o f the adder and registers by n oticin g wh ere there a re unused p o rtions o f registers and adders. Figure 3.1 3 sh ows the revised h a rd wa re.

Iteration

0 1

Step

Quotient

Divisor

Remainder

Initial values

0000

0010 0000

0000 0 111

1 : Rem _ Rem - Diy

0000 0000 0000

0010 0000

@1100111

0010 0000

0000 0 111

0001 0000

0000 0 111

0000 0000

0001 0000

~ 11 0 111

0001 0000

0000 0 111

0000 0000 0000

0000 1000

0000 0 111

0000 1000

~ 111111

0000 1000

0000 0 111

0000 0000 0001

0000 0 100

0000 0 111

0000 0 100

(9:>00 0011

0000 0 100

0000 0011

0001 0001

0000 0010

0000 0011

0000 0010

@::loa 0001

0011 0011

0000 0010

0000 0001

2b: Rem < a

+Diy. sll Q. QO '" a

3 : Shift Diy right 1 : Rem _ Rem - Diy

2b: Rem < a

+Diy. sll Q. QO _ a

3 : Shift Diy right 1 : Rem _ Rem - Diy

2b: Rem < a

+Diy. sll Q. QO _ a

3 : Shift Diy right 1 : Rem _ Rem - Diy

2a:

Rem;:>:O ~ sIlQ . QO _l

3 : Shift Diy right 1 : Rem _ Rem - Diy

2a:

Rem;:>:O ~ sIlQ . QO _l

3 : Shift Diy right

FIGURE 3.12

DIvision example using the algorithm In Figure 3.11. The bit examined to deter-

mine the next step is circled in color.

3.5

187

Division

Divisor 32 bits

" " 32-b;' ALU

-:;z-

• • Rom inder

Shift right Shift left Write

Controi........ test /

64 bits

An Improved version of th e division hardware . The DJVlsor re81ster, ALU, and Quotient register are all 32 bits wide, with only the R£mainder register left at 64 bits. Compared to FIGURE 3.13

Figure 3.10, the ALU and Divisor registers are halved and the remainder is shifted left. This version also combines the Quotient register with the right half of the R£mainder register.

Signed Division So far we have ignored signed numbers in division. The simplest solution is to remember the signs of the divisor and dividend and then negate the quotient if the signs disagree. Elaboration: The one complication of signed division is that we must also set the sign of the remainder. Remember that the following equation must always hold: Dividend = Quotient x Divisor + Remainder To understand how to set the sign of the remainder, lers look at the example of dividing all the combinations of ±7ten by ±2 teo . The first case is easy: +7

+2 : Quotient = +3, Remainder = +1

Checking the results: 7=3x2+(+1)=6+1 If we change the sign of the dividend, the quotient must change as well : -7

+2 : Quotient =-3

Rewriting our basic formula to calculate the remainder : Remainder = (Dividend - Quotient x Divisor) = - 7 - (- 3 x +2) = - 7- (- 6) = - 1

So, -7

+2 : Quotient = -3, Remainder =-1

Checking the results again :

-7 = -3

x 2 + (-1)

= -6 - 1

188

Chapter 3

Arithmetic: for Computers

The reason the answer isn 't a quotient of - 4 and a remainder of +1 , wh ich would also fit t his fo rmula, is t hat t he absolute va lue of t he quotient would t hen change depending on t he sign of the dividend and t he divisorl Clearly if --(x ~

y);;t (-x ) ~ y

programming would be an even greater challenge . Thi s anomalous behavior is avoided by f ollowing t he rule that t he dividend and rema inder must have t he same s igns , no matter what t he signs of the divisor and quotient . We ca lculat e t he other combinations by following the sa me rule : + 7 ~ -2: Quotient = -3, Remainder = +1 -7 ~ -2 : Quotient = +3, Remainder =-1 Thus t he correctly signed division algorithm negates the quotient if the signs of t he operands are opposite and makes t he sign of the nonzero remainder match t he dividend.

Faster Division We used 32 adders to speed up multiply, but we cannot do the sa me trick for divide. The reason is that we need to know the sign of the difference befo re we ca n perform the next step of the algorithm, whereas with multiply we could calculate the 32 partial products immediately. There are techniques to p roduce mo re than one bit of the quotient per bit. The SRT divis ion technique tries to guess several quotient bits per step, using a table lookup based on the upper bits of the dividend and remainder. It relies on subsequent steps to correct wrong guesses. A typica l va lue today is 4 bits. The key is guessing the value to subtract. With binary division there is only a single choice. These algorithms use 6 bits from the remainder and 4 bits from diviso r to index a table that determines the guess fo r each step. The accuracy of this fast method depends on having proper values in the lookup table. The fa llacy on page 222 in Section 3.8 shows what ca n happen if the table is incorrect.

Divide in MIPS You may have already observed that the same sequential hardwa re ca n be used for both multiply and divide in Figures 3.7 and 3. 13. The only requirement is a 64-bit register that ca n shift left or right and a 32-bit ALU that add s or subtracts. Hence, MIP S uses the 32-bit Hi and 32-bit 1..0 registers for both multiply and divide. As we might expect from the algorithm above, Hi contains the remainder, and Lo contain s the quotient aft er the divide instruction completes. To handle both signed integers and unsigned integers, MIP S has two instructions: div ide (di v) and divide l/nsigned (di vu ). The M IPS assembler allows divide in structions to specify three registers, generating the mfl 0 o r mfh i instructions to place the desired result into a general-purpose register.

3.6

189

Floating Point

Summary The commo n hardware support for multiply and divide allows MIPS to provide a single pair o f 32-bit registers that a re used both for multiply and divide. Figu re 3. 14 summarizes the additio ns to the MIPS architecture fo r the last two sectio ns.

MIP S divide instructio ns igno re overflow, so software must determine if the quotient is too large. In additio n to overflow, divisio n ca n also result in an improper calculatio n: divisio n by o. Som e computers distinguish these two ano malo us events. MIPS software must check the diviso r to discover divisio n by 0 as well as overflow.

Hardware Software Interface

Elaboration: An even fa ster algorithm does not immediately add the divisor back if the remainder is negative . It simply adds the dividend to the shifted remaind er in the following step since (r + d) x 2 - d = r x 2 + d x 2 - d = r x 2 + d . This nonres toring divis ion algorithm, which takes 1 clock per step, is explored furth er in Exerci se 3 .29; the algorithm here is called res toring division.

Roating Point

Speed gets you nowhere if yo u're headed the wrong way. American proverb

Go ing beyond signed and unsigned integers, programming languages support numbers with fractio ns, which are ca lled reals in m athem atics. Here are so m e examples o f reaIs: 3.141 59265 ...It'n (rr) 2.7 1828...ten (e) O.OOOOOOOO l ten o r 1.0 ten X 10-9 (seconds in a nanosecond) 3, I 55,760,OOOten or 3.1 5576ten X 109 (seconds in a typical century) Notice that in the last case, the number didn't represent a small fra ction, but it was bigger than we could represent with a 32-bit signed integer. The alternative notatio n fo r the last two numbers is called scientific no ta tion , which has a single digit to the left of the decimal point. A number in scientific notatio n that ha s no leading Os is called a norm a lized number, which is the usual way to write it. Fo r example, 1.0ten X 10-9 is in no rmalized scientific notatio n, but O.l ten X 10--8 and 1O.0 ten X 10-10 are not.

scientific notation Anotation that renders numbers with a sin· gle digit to the left of the decimal point. normalized A number in floating-point notation that has no leading Os.

MIPS assembly language Category

Arithmetic

Instruction

Example

.dd

,dd

subtract

,ob

add immediate add unsigned subtract unsigned

addi

addu subu add immediate unsigned addiu move from coprocessor mf cO register multiply mult multiply unsigned multu divide di v divide unsigned move from Hi move from Lo

Unconditional Ijump

$ 53 $ s3 100 $ SJ $ sJ

Three operands; overflow detected Three operands; overflow detected + constant; overflow detected Three operands; overflow undetected Three operands; overflow undetected

$ sl_ $ s2 + 100 $ sl ,- $ epc

+ constant; overflow undetected Copy Exception PC + special regs

Hi, La" $ sl x ' 53

64-bit signed product in Hi, La

$ s2. $ s3

Hi, La" $ 52 x $ 53 La" $ Sl / ) 53, Hi" $ 52 mod $ s3

64-bit unsigned product in Hi, Lo La _ quotient, Hi _ remainder

$ s2. $ s3

La,, $ s~ n s::.

Unsigned quotient and remainder

$ sl. $epc $ s2. $ s3 $ s2. $ s3

Sd Sd

Hi" $ 52 mod $ s3 $ sl"Hi

Used to get copy of Hi Used to get copy of Lo

' sl"Lo Memory( $ s2 + 100)

$ sl.100{ $ s2 )

$ 5 1 _ Memory( $ 5 2 + 1001

Halfword memory to register

$ sl.100{ $ s2 )

Memory( $ 52 + 100) _ $ 5 1

Halfword register to memory

$ sl.100{ $ s2 )

$ 5 1 _ Memory( $ 5 2 + 1001

Byte from memory to register

store byte

1ho ,h 1bo ,b

$ sl.100{ $ s2 )

Byte from register to memory

load upper immediate

1ui

$ sl.100

Memory( $ 5 2 + 100) _ $ 5 1 100 • 2 16 Sd

'0' " 0"

,od

$ s1. $ s2. $ s3

"0"

$ s1. $ s2. $ s3

and immediate

andi

$ sl. $ s2.100

or immediate

$ sl. $ s2.100

load half unsigned store half

shift left logical

Conditional branch

$ s2 + $ s2_ $ s2 + $ sl + $ sl_

$ sl.100{ $ s2 )

load byte unsigned

Logical

mfl 0

$ sl. $ s2. $ s3 $ sl. $ s2. $ s3 $ sl, $ s2.100

$ 51 _ $ sl_ $ sl_ $ 51_ $ sl_

store word

transfer

mfhi

$ sl, $ s2. $ s3 $ sl, $ s2. $ s3 $ sl, $ s2.100

Comments

$ sl.100{ $ s2 )

load word

0,.

di vu

Meaning

," ,,1

$ sl, $ s2. $ s3

$ s1. $ s2.10

shift right logical branch on equal

beq

$ sl. $ s2.10 $ sl, $ s2.25

branch on not equal

boe

$ sl, $ s2.25

set on less than

,It

$ sl. $ s2. $ s3

set less than immediate slti

$ sl, $ s2.100

s et less than unsigned

5 ltu

$ sl. $ s2. $ s3

set less than immediate unsigned jump

5 It i u

$ sl, $ s2.100

J F

2500

Sco

j a1

2500

jump register jump and link

Sd Sd Sd Sd Sd Sd Sd

Memory l$ s2 + 100J

, S,3 I S,3 S,' , S,' S,'

( $ 5 2 1$ 53 )

S,' S,' S,'

I « »

100 100 10 10

if( $ S~

__ $ ~l)goto PC + 4 + 100 if( $ sl !_ $ s2)goto PC + 4 + 100 if( $ s.2 < $ s_J ) $ 51 '- 1 ; else $ 51 ,, 0 if( $ s." O ‫סס‬oo 0<XXl ‫סס‬oo ()()()() <XXX> ‫סס‬oo O<XX>two) x i 31

30 29 28 27 26 25 24 23 22 21 20 19 18

1 7 16 15 14 13 12 11 10 9 8

1 bit

11 bits

7 6 5

0 0 0

lon_ lOll)

4 3 2

0 0 0

20 bits

0000000000 1

32 bits

Now let's try going the other direction.

Converting Binary to Decimal Floating Point What decimal number is represented by this single precision fl oat?

EXAMPLE 31 30 29 28 27

ANSWER

26 25 24 23 22 21 20 19 18 17 16 1 5 14 13 12 11 10 9

0 0 0

7 6

3 2

1 0

0 0 0

The sign bit is I, the exponent fi eld contains 129, and the fra ctio n field con tains 1X r 2 = 1/4, or 0.25. Using the basic equatio n, (_ I)s X ( I

+ Fraction) X iExponent -

Bias)

= (_ 1) 1 X (i + 0.25) X 2(129-127) =-IX1.2SX2 2 =-1.2Sx4 = -5.0

In the next sections we will give the algorithms for floa ting-point addition and multiplicatio n. At their core, they use the co rresponding integer operatio ns on the significands, but extra bookkeeping is necessary to handle the exponents and normalize the result. We first give a n intuitive derivation o f the algorithm s in decimal, and then give a more detailed, binary versio n in the figures.

3.6

197

Floating Point

Elaboration: In an attempt t o increase range with out removing bits from the significand, some computers before the IEEE 754 standard used a base other than 2 . For example, the IBM 360 and 3 70 mainframe computers use base 16. Since changing the IBM exponent by one means shifting the significand by 4 bits , "normalized" base 16 numbers can have up t o 3 leading bits of Os! Hence , hexadecimal digits mean that up to 3 bits must be dropped from the s ignificand, which leads to surpris ing problems in the accuracy of fl oating-point arithmetic, as not ed in Secti on .

Roating·Point Addition Let's add numbers in scientific notatio n by ha nd to illustrate the pro blem s in fl oating-point additio n: 9.999 ten X WI + 1. 61O ten X 10- 1. Assume that we can sto re o nly four decimal digits of the significa nd and two decimal digits of the expo nent. Step 1. To be able to add these numbers properly, we must align the decim al point of the number that has the sm aller expo nent. Hence, we need a fo rm of the sm aller number , 1.6lO ten X 10- 1, th at m atches the la rger expo nent. We o bta in this by observing that there are multiple representatio ns of an unnorm alized fl oating-point number in scientific no tatio n: 1.6 1O ten X 10- 1 = 0.1 61O ten X 10° = 0.0 16 10 ten X WI

The number o n the right is the versio n we desire, since its expo nent m atches the expo nent of the larger number, 9.999 ten X WI. Thus the first step shifts the significa nd of the smaller number to the right until its co rrected expo nent m atches that of the larger number. But we ca n represent only four decimal digits so, aft er shifting, the number is really: 0.0 16 ten X 10 I

Step 2. Next com es the addition of the significa nds:

9.999 ten 0.0 16 ten 1O. 0 15 ten

The sum is 10. 0 15 ten X WI. Step 3. This sum is no t in no rmalized scientific notation , so we need to adjust it: 10.0 15 ten X 10 I = 1.00 15 ten X 10 2

Thus, aft er the addition we m ay have to shift the sum to put it into no rm alized fo rm , adjusting the expo nent appro priately. This exa mple shows

198

Chapter 3

Arithmetic: for Computers

shifting to the right, but if one number were positive and the other were negative, it would be possible for the sum to have many leading Os, requiring left shifts. Whenever the exponent is increased or decreased, we must check for overflow or underfl ow-that is, we must make sure that the exponent still fit s in its field. Step 4. Since we assumed that the significa nd ca n be only four digits long (excluding the sign), we must round the number. In our gra mmar school algorithm , the rules tnmca te the number if the digit to the right of the desired point is between 0 and 4 and add I to the digit if the number to the right is between 5 and 9. The number 1.00 15,bel. t) orfalst' (bit 16 = 0 = >bel. f ). Instructions in color are described in Chapters 2 or 3, with til Appendix A covering aU instructions.

3.6

209

Floating Point

One issue that computer designers face in supporting fl oating-point arithmetic is whether to use the same registers used by the integer in structions or to add a special set for fl oating point. Because programs no rmally perform integer operations and floa ting-point operations on different data, separating the registers will only slightly increase the number of instructions needed to execute a program. The major impact is to create a separate set of data transfer instructions to move data between floating- point registers and memory. The benefit s of separate fl oating-point registers are having twice as many registers without using up more bits in the instruction format, having twice the register bandwidth by having separate integer and fl oating-point register sets, and being able to customize registers to fl oating point ; for exa mple, some computers convert all sized operands in registers into a single internal format

Hardware Software Interface

Compiling a Floating-Point C Program into MIPS Assembly Code

Let's convert a temperanlfe in Fahrenheit to Celsius:

f loa t f2c ( f loa t f ahr) I r et ur n «5 . 0/9 . 0l * ( f ah r - 32 . 0» ;

EXAMPLE

Assume that the fl oating- point argument f ah r is passed in $fl2 and the result should go in $ f O. (Unlike integer registers, fl oating-point register 0 can contain a number.) What is the MIPS assembly code?

We assume that the compiler places the three floating- point constants in mem ory within easy reach of the global pointer $ gpoThe first two instructions load the constants 5.0 and 9.0 into floa ting- point registers:

f 2c : l weI $f16 . eons t 5($gp) # $f16 l weI $f18 . eons t 9($gp) # $f18

5. 0 (5 . 0 1n memory) 9 . 0 (9 . 0 1n memory)

ANSWER

210

Chapter 3

Arithmetic: for Computers

They are then divided to get the fraction 5.0/9.0:

div . s H1 6 . Ifl6 . HIB II I fl6 ~ 5 . 0 / 9 . 0 ( Many compilers would divide 5.0 by 9.0 at compile time and save the single constant 5.0/9.0 in m em ory, thereby avoiding the divide at runtime.) Next we load the constant 32.0 and then subtract it from fa h r ($ f 12):

lwei $f18, cons t32 ( $gp ) # $f18 = 32 . 0 sub . s $f18, $f12, $f18 # $f18 = fahr - 32 . 0 Finally, we multiply the two intermediate results, placing the product in $fO as the return result, and then return:

mul . s HO . H1 6 , $fIB It $fO J r $ ra It return

(5/9l * ( fahr - 32 . 0)

Now let's perform fl oating-point operatio ns on matrices, code commo nly found in scientific programs.

Compiling F1oating·Point C Procedure with Two-Dimensional Matrices into MIPS

EXAMPLE

Most fl oa ting-point calculations are performed in double precision. Let's perform matrix multiply of X = X + Y ~ Z. Let's assume X, Y, and Z are all square matrices with 32 elements in each dimension.

void mm (double x[][] , double I int

1 ,

y[][] ,

double z[][] )

j , k; .

, ,

0 ; 1. - 32 ; 1 - 1 + 11 . 0 ; J . - 32 ; J - J + 11 0 ; k ! - 32 ; k - k + 11 x[iHj] - xl i Hj] + y[i Hk] * z[kHj] ;

f or ( i f or ( j f or ( k

}

The array sta rting addresses are parameters, so they are in $aO, $al, and $a 2. Assume that the integer va riables are in $s 0, $ s 1, and $s 2, respectively. What is the MIPS assembly code for the body of the procedure?

3.6

211

Floating Point

Note that X[i ] [j] is used in the innermost loop above. Since the loop in dex is k, the index does not affect X [i ] [j ], so we ca n avoid load ing and storing X[i ] [j ] each iteratio n. Instead, the compiler loads X [i ] [j ] into a register outside the loop, accumulates the sum of the products o f y [ i ] [k] and z[k] [j] in that same register, and then stores the sum into x[ i] [j] upo n terminatio n o f the innermost loop. We keep the code simpler by using the assembly language pseudoinstructio ns 1 i (which loads a constant into a register), and 1 . d and 5 . d (which the assembler turns into a pair of data transfer instructio ns, 1we 1 or swe 1, to a pa ir o f floating-point registers). The body o f the procedure sta rts with saving the loop terminatio n va lue o f 32 in a temporary register and then initializing the three for loop variables:

mm : ...

Ll ; L2 ;

I I I I

i i i i

It! . $sO , $ s 1, $sZ ,

II Itl

32 0 0 0

1 ~

II J II k

~ ~

32 (row sizelloop end) 0 ; initialize 1st for loop 0 ; restart 2nd for loop 0 ; restart 3rd for loop ~

To calculate the address of x [i ] [j ], we need to know how a 32 X 32, twodimensional array is stored in m emory. As you might expect, its layout is the sa me as if there were 32 single-d imensio n arrays, each with 32 elements. So the first step is to skip over the i "single-dimensional arrays," or rows, to get the one we wa nt. Thus we multiply the index in the first dimension by the size of the row, 32. Since 32 is a power o f 2, we ca n use a shift instead: 511

HZ , $50, 5

It HZ

i * Z5 (si ze of row of x)

Now we add the second index to select the j th elem ent of the desired row: addu

HZ , HZ, $sl

i * size(row) + j

To turn this sum into a byte index, we multiply it by the size of a matrix elem ent in bytes. Since each elem ent is 8 bytes for double precision, we ca n in stead shift left by 3: 511

HZ , HZ, 3

byte offset of [i][j]

Next we add this sum to the base address o f x , giving the address of x [i ] [j], and then load the double precision number x[ i ] [j] into $f4: addu l .d

HZ , $aO, HZ It HZ = byte address of x[i][j] $f4 , O($tZ) It $f4 = 8 bytes of x[i][j]

The fo llowing five instructio ns are virnlally identical to the last fi ve: calcu late the address and then load the double precision number z [k] [j ].

ANSWER

212

Chapter 3

Arithmetic: for Computers

L3 :

sl1 $tO , $s2 , 5 It $tO = k * 25 (size of r ow o f z) addu $tO , $t O, $sl It $tO = k * size( r ow) + j sl1 $tO , $tO , 3 It $tO = byte o ff set of [kJ(j] addu $t O, $a 2 , $t O It $tO = by t e address o f z [k][j] I . d $f16 . 0(1 32-bit inputs and place the result on its output.

Add

L~-I Read L... PC I address Instruction

f---.

Instru cti on memory

FIGURE 5.6 A portion of the datapath used for fetching Instructions and Incrementing the program counter. The fetched instruction is used by other parts of the datap.1th.

duced in Chapter 2. Recall that a typical instance of such an instruction is add $11. st2. $13 ,which reads $12 and $13 and writes $11 . The processor's 32 general-purpose registers are stored in a structure called a register fil e. A register file is a collection of registers in which any register can be

r egist er fil e A state element that consists of a set of registers that can be read and written by supplying a register number to be accessed.

294

sign-extend To increase the size of a data item by replicating the high-order sign bit of the origin al data item in the h ighorder b its of the larger, destina-

tion data item. branch target address The address specified in a branch, which becomes the new program counter (PC) if the bran ch is taken. In the M IPS architecture the branch target is given by the

sum of the offset field of the instruction and the address ofthe instruction following the branch.

Chapter 5

The Processor: Datapath and Control

read or written by specify ing the number of the register in the fil e. The register fil e contain s the register state of the machine. In addition , we will need an ALU to operate on the values read from the registers. Because the R-format instructions have three register operands, we will need to read two data words from the register file and write one data word into the register fil e for each instruction. Fo r each data wo rd to be read from the registers, we need an input to the register file that specifies the register number to be rea d and an output from the register file that will ca rry the va lue that has been read from the registers. To write a data word, we will need two inputs: one to specify the register nllmber to be written and one to supply the data to be written into the register. The register file always outputs the contents of whatever register numbers are on the Rea d register inputs. Writes, however, are cont rolled by the write cont rol signal , which must be asserted fo r a write to occur at the clock edge. Thus, we need a total of four inputs (th ree for register numbers and one fo r data) and two outputs (both for data), as shown in Figure 5.7. The register number in puts are 5 bits wide to specify one of 32 registers (3 2 = 25 ) , whereas the data in put and two data out put buses are each 32 bits wide. Figu re 5.7 shows the ALU, which takes two 32-bit inputs and p roduces a 32-bit result, as well as a I-bit signal if the result is O. The four-bit cont rol signal of the ALU is described in detail in II Appendix B; we will review the ALU cont rol shortly when we need to know how to set it. Next, consider the MIPS load word and store word instructions, which have the general form 1w $tl , offse t _va 1ue ( H2) or sw $tl , offset_va 1ue ( H2). These instructions com pute a memory address by adding the base register, which is HZ, to the 16-bit signed offset field contained in the instruction . If the instruction is a store, the va lue to be sto red must also be read from the register fil e where it resides in $t 1. If the instruction is a load , the value rea d from mem o ry must be written into the register fil e in the speci fied register, which is $ t 1. Thus, we will need both the register fil e and the ALU from Figure 5.7. In addition , we will need a unit to sign-extend the 16-bit offset field in the instruction to a 32-bit signed value, and a data memory unit to read from or write to. The data memo ry must be written on store instructions; hence, it has both read and write cont rol signals, an address in put , as well as an input for the data to be written into memory. Figure 5.8 shows these two elements. The beq instruction has three operands, two registers that are compared for equality, and a 16-bit offset used to compute the branch target address relative to the branch instruction address. Its form is beq $tl , H2 , o ff set. To implement this instruction, we must compute the branch target address by adding the sign-extended offset fi eld of the instruction to the Pc. There are two details in the definiti on of branch instructions (see Chapter 2) to which we must pay attention :

5.3

295

Building a Datapath

5 5

Data {

Read register 1

4 Read data 1

Read register 2 Write register

ALU operation

z.~

Data

Regi s ters Read data 2

Write Data

) ALU ALU result

I RegWrite a. Registers

b. ALU

FIGURE 5.7 The two elements needed to Implement R·format ALU operations are the reg· Ister file and the ALU. The register file contains all the registers and has tm> read ports and one write port. The design of multiported register files is discussed in Section B.8 of Appendix B. The register file always outputs the contents of the registers corresponding to the Read register inputs on the outputs; no other control inputs are needed. In contrast, a register write must be explicitly indicated by asserting the write control signal. Remember that writes are edge-triggered, so that all the write inputs (i.e., the value to be written, the register munber, and the write control signal) must be valid at the clock edge. Since writes to the register file are edgetriggered, our design can legally read and write the same register within a clock cycle: the read will get the value written in an earlier dock cycle, while the value written will be available to a read in a subsequent dock cyde. The inputs carrying the register number to the register file are all 5 bits wide, whereas the lines carrying data values are 32 bits wide. The operation to be performed by the ALU is controlled with the ALU operation signal, which will be 4 bits wide , using the AW designed in 'II Appendix B. We will use the Zero detection output of the ALU shortly to implement branches. The overflow output will not be needed until Section 5.6, when we discuss exceptions; we omit it wltil then .

• The instruction set ardlitecture specifies that the base for the brandl address calculation is the address of the instruction following the branch. Since we compute PC + 4 (the address of the next instruction) in the instruction fetch datapath , it is easy to use this value as the base for computing the branch target address. • The architecnlfe also states that the offset field is shifted left 2 bits so that it is a word offset; this shift increases the effective range of the offset field by a factor of four. To deal with the latter complication , we will need to shift the offset field by two. In addition to computing the branch target address, we must also determine whether the next instruction is the instruction that follows sequentially or the instruction at the branch target address. When the condition is true (Le., the operands are equal), the branch target address becomes the new PC, and we say that the branch is taken . If the operands are not equal, the incremented PC should replace the current PC Oust as for any other normal instruction); in this case, we say that the branch is not taken.

bram:h taken A branch where the branch condition is satisfied and the program counter ( PC ) becomes the branch target. All unconditional branches are taken branches. branch not taken A branch where the branch condition is false and the program counter ( PC) becomes the address ofthe instruction that sequentially follows the branch.

296

Chapter 5

The Processor: Datapath and Control

I MemWrite -~, Address

data I

Data

- - --I

Write data

ReadL-_ 16 _"--I

\ 32

Sign

L-'~

extend I

memory

MemRead a. Data memory unit

b. Sign-extension unit

FIGURE 5.8 The two units needed to Implement loads and stores, In addition to the reg· Ister file and ALU of Figure 5.7, are the data memory unit and the sign extension unit. The memory unit is a stale element with inputs for the address and the write data, and a single output for the read result. There are separate read and write controls, although only one of these may be asserted on any given dock. The memory unit needs a read signal, since, unlike the register file, reading the value of an invalid address can c.1use problems, as we will see in Chapter 7. The sign extension unit has a 16·bit input that is sign-extended into a 32-bit result appearing on the output (see Chapter 3). We assume the data memory is edge-triggered for writes. Standard memory chips actually have a write enable signal that is used for writes. Although the write enable is not edge-triggered, our edge-triggered design could easily be adapted to work with real memory chips. See Section B.8 of III Appendix B for a further discussion of how real memory chips work.

Thus, the branch datapath must do two operations: compute the branch target address and compare the register contents. (Branches also affect the instruction fetch portion of the datapath, as we will deal with shortly.) Because of the complexity of handling branches, we show the structure of the datapath segment that han dles branches in Figure 5.9. To compute the branch target address, the branch datapath includes a sign extension unit, just like that in Figure 5.8, and an adder. To perform the compare, we need to use the register file shown in Figure 5.7 to supply the two register operands (although we will not need to write into the register file). In addition , the comparison ca n be done using the ALU we designed in Appendix B. Since that ALU provides an output signal that indicates whether the result was 0, we can send the two register operands to the ALU with the control set to do a subtract. If the Zero signal out of the ALU unit is asserted, we know that the two values are equal. Although the Zero output always signals if the result is 0, we will be using it only to implement the equal test of branches. Later, we will show exactly how to connect the control signals of the ALU for use in the datapath. The jump instruction operates by replacing the lower 28 bits of the PC with the lower 26 bits of the in struction shifted left by 2 bits. This shift is accomplished simply by concatenating 00 to the jump offset, as described in Chapter 2.

5.3

297

Building a Datapath

PC-t4 from instruction datapath Add Sum Shift left 2 Instruction

Read register 1

Rood

f-~ ~4'

ALU operation

data 1

Read register 2 Write register

Branch target

ALU Zero

Registers Rood

data 2

Write data

RegWrite I / 16

To branch control logic

Sign extend

FIGURE 5.9 11Ie data path for a branch uses the ALU to evaluate the branch condition and a separate adder to compute the branch target as the sum of the Incremented PC and the slgn-extended, lower 16 bits of the Instruction (the branch displacement), shifted left 2 bits. The unit labeled Shift left 2 is simply a routing of the signals between input and output that adds oolwo to the low-order end of the sign-extended offset field; no actual shift hardware is needed, since the amount of the "shift" is constant. Since we know that the offset was sign-extended from 16 bits, the shift wiU throwaway only "sign bits." Control logic is used to decide whether the incremented PC or branch target should replace the Pc, based on the Zero output of the ALU.

Elaboration: In the MIPS instruction set, branches are d el ayed , meaning that the instruction immediately following the branch is always executed, independent of whether the branch condition is true or false . When the condition is false, the execution looks like a normal branch . When the condition is true, a delayed branch first executes the instruction immediately following the branch in sequential instruction order before jumping to the specified branch target address . The motivation for delayed branches arises from how pipelining affects branches (see Section 6 .6). For simplicity, we ignore delayed branches in this chapter and implement a nondelayed beq instruction .

delayed bran ch A type of branch where the instruction immediately following the branch is always executed, independent of whether the branch condition is true or false.

298

Chapter 5

The Processor: Datapath and Control

Creating a Single Datapath Now that we have exa mined the data path components needed for the individual instruction classes, we can combine them into a single datapath and add the con trol to complete the implementation. The simplest datapath might attempt to execute all instructions in one clock cycle. This mea ns that no data path resource can be used m ore than once per instruction, so any element needed more than once must be duplicated. 'We therefore need a memory for instructions separate from one for data. Although some of the fun ctional units will need to be duplicated , many of the elements can be shared by different instruction flows. To share a datapath element between two different instruction classes, we may need to allow multiple connections to the input of an element, using a multi plexor and control signal to select among the multiple inputs.

Building a Datapath

EXAMPLE

The operations of arithmetic-logical (or R-type) instructions and the memory instructions datapath are quite similar. The key differences are the following: • The arithmetic-logical instructions use the ALU with the inputs coming from the two registers. The memory in structions ca n also use the ALU to do the address calculation, although the second input is the sign-extended 16-bit offset field from the instruction. • The va lue stored into a destination register comes from the ALU (for an R-type instruction ) or the memory (for a load). Show how to build a datapath for the opera tional portion of the memory reference and arithmetic-logical instructions that uses a single register file and a single ALU to handle both types of instructions, adding any necessary multiplexors.

ANSWER

To crea te a datapath with only a single register fil e and a single ALU, we must support two different sources for the second ALU input, as well as two different sources for the data stored into the register file. Thus, one multiplexor is placed at the ALU input and another at the data input to the register file. Figure 5. 10 shows the operational portion of the combined datapath. Now we ca n combine all the pieces to make a simple datapath for the MIPS architecture by adding the datapath for instruction fetch ( Figure 5.6 on page 293),

5.3

299

Building a Datapath

Read registe r 1

Read registe r 2 . Reg isters Read Wn~e data 2 registe r

Instruction

,..

Read data 1 ALUSrc

i0 M ," f-

Write

ALU operation

I MemWrite Ze ~

ALU ALU result

MemtoReg

fAddress

Read data

data RegWrite

I; Sign ex tend

Write

data 32

Da ta memory

," 0

I MemRead

\ FIGURE 5.10 The d a t a pa th for the memory Instructi ons and th e R·type Instructi ons. ThIS example shows how a single datapath can be assembled from the pieces in Figures 5.7 and 5.8 by adding multiplexors. l\I.u multiplexors are needed, as described as in the example.

the datapath from R-type and memory instructions ( Figure 5.10 on page 299), and the datapath fo r branches (Figure 5.9 on page 297). Figure 5. 11 shows the datapath we obtain by composing the separate pieces. The branch instruction uses the main ALU for comparison of the register operands, so we must keep the adder in Figure 5.9 for co mputing the branch target address. An additional mul tiplexor is required to select either the sequentially followin g in struction address ( PC + 4) or the branch target address to be written into the Pc. Now that we have completed this simple datapath , we ca n add the control unit. The control unit must be able to take inputs and generate a write signal for each state element, the selector control fo r each multiplexor, and the ALU cont rol. The ALU cont rol is different in a number of ways, and it will be useful to design it first before we design the rest of the control unit. Which of the following is correct for a loa d in struction? a. MemtoReg should be set to cause the data from memo ry to be sent to the register fil e. b. MemtoReg should be set to cause the correct register destination to be sent to the register fil e. c. We do not ca re about the setting of MemtoReg.

Check Yourself

300

Chapter 5

Th e Pr ocessor: Dat apath and Co ntrol

PCSrc,,-l-

) Add

ALU

Add result Shift left 2

Read

Rood

ALUSrc

address

Read

ALU operation

Ze~

Instruction

Instr uction memory

data 1

Read

) ALU

. Regis ters Read Wn.te data 2 register

M ~

Write data

'--'

RegWrite

ALU

I MemWrite

MemtoReg Address

Write

I /

data Sign

Read

data

Data

M ~

memory

MemRead

ex ten d

FIGURE 5.11 Th e simple dat apath f o r the MIPS architectu re combin es th e elem ents required by diffe rent Instructio n classes. This data path can execute the basic instructions (load/store word,ALU operations, and branches) in a single dock cycle. An additional mul· tiplexor is needed to integrate branches. The support for jumps will be added later.

A Simple Implementation Scheme In this section, we look at what might be thought of as the simplest possible implementation of our MIP S subset. We build this simple implementation using the datapath of the last section and adding a simple control fun ction. This simple implementation covers load wo rd (1 w), store word ( sw), branch equal (be q), and the arithmetic-logical in structions add, sub, and, or, and set on less than. We will later enhance the design to include a jump instruction (j ).

S.4

301

A Simple Implementation Scheme

The ALU Control As ca n be seen in Appendix B, the ALU has four control inputs. These bits were not encoded; hence, only 6 of the possible 16 possible input combinations are used in this subset. The MIPS ALU in II Appendix B shows the 6 following combinations: AW control lines

Function

0000

ANO

OOO! 0010

OR ,dd

0110

subtract

0111

set on less than

1!oo

NOR

Depending on the instmction class, the ALU will need to perform one of these first five fun ctions. (NO R is needed for other parts of the M IPS instruction set.) For load word and store word instructions, we use the ALU to compute the memory address by addition. For the R-type instructions, the ALU needs to perform one of the five actions (AND, OR, subtract, add, or set on less than), depending on the value of the 6-bit fun ct (or fun ction) field in the low-order bits of the inst ruction (see Chapter 2). For branch equal, the ALU must perform a subtraction. We ca n generate the 4-bit ALU control input using a small control unit that ha s as inputs the fun ction field of the instruction and a 2-bit control field , which we call ALUOp. ALUOp indicates whether the operation to be performed should be add (00) for loads and stores, subtract (0 1) for beq, or determined by the operation encoded in the fun ct fi eld ( 10). The output of the ALU control unit is a 4-bit signal that directly controls the ALU by generating one of the 4-bit combinations shown previously. In Figure 5.1 2, we show how to set the ALU control inputs based on the 2-bit ALUOp control and the 6-bit fun ction code. For completeness, the relationship between the ALUOp bits and the instruction opcode is also shown. Later in this chapter we will see how the ALUOp bits are generated from the main control unit. This style of using multiple levels of decoding-that is, the main control unit generates the ALUOp bits, which then are used as input to the ALU control that generates the actual signals to control the ALU unit- is a common implementa tion technique. Using multiple levels of control can red uce the size of the main control unit. Using several smaller control units may also potentially increase the speed of the control unit. Such optimizations are important, since the control unit is oft en performance-critical. There are several different ways to implement the mapping from the 2-bit ALUOp field and the 6-bit funct fi eld to the three ALU operation control bits.

302

Chapter 5

The Processor: Datapath and Control

Instruction opcode

Instruction operation

Desired AlU action

Funct field

ALU control Input

load word

XXJ()()(J(

odd

0010

XXJ()()(J(

odd

Branch equa l

store word branch equal

XXJ()()(J(

subtract

RALU

: f- V /

~ Zero

ALU result

Read Address data

," 0

Da ta Write memory data

ALU co ntrol

FIGURE 5.20 11Ie datapath In operation for a load Instruction. The oontrol lmes, datap.1th WiltS, and connectlons th.1t are actlve are hIghIighted.A store inst ruction mmld operate very similarly. The Jll.1in difference would be that the memory cont rol m>uld indicate a WTite rather than a read, the second register value read m>uld be U'ied for the data to store, and t he operation of writing the data memory value to the register fi le m>uld nOi occur.

instruction , but the ALU output is used to determine whether the PC is written with PC + 4 or the brJ nch target address. Figu re 5.2 1 shows the four steps in execution: 1. An instruction is fetched from the instruction mem ory, and the PC is incremented. 2. Two registers, $t 1 and $t Z, are read from the register fil e.

5.4

311

A Simple Implementation Sche me

'0 M

Add

ALU Add result

/ I

RegDst Branch MemRead MemtoReg ALUOp MemWrite ALUSrc RegWrite

Instruction [3 1-26J Contro l

\ Instruction [25-21]

Read register 1

Read address Instruction [20-16] Instruction [3 1-0J Instruction memory

I Instruction [15-11 ]

0 M

Read register 2 Wrne register Wrne data

IInstruction [15--0]

ze';;; Read data 2

f--O ~f-,

'"-L

Sign extend

Read data 1

Reg ist ers

Shill left 2

A LU ALU result

Address

Read data

," 0

v Data Write memory data

ALU co ntro l

Instruction [5--0J

FIGURE 5.21 The d a t a pa th In operation for a branch equal Instructi on. The control hnes, datapath umts, and connectIOns that are active are highlighted. After using the register file and ALU to perform the compare, the Zero output is used to select the next program counter from between the two candidmes.

3. The ALU performs a subtra ct on the data va lues read from the register file. The value of PC + 4 is added to the sign-extended, lower 16 bits of the instruction (o ffset ) shifted left by two; the result is the branch target address. 4. The Zero result from the ALU is used to decide which adder result to store into the Pc.

312

Chapter 5

The Processor: Datapath and Control

In the next section , we will examine ma chines that are truly sequential, namely, those in which each of these steps is a distinct clock cycle. Finalizing the Control Now that we have seen how the instructions operate in steps, let's continue with the control implementation. The control functi on ca n be precisely defin ed using the contents of Figure 5. 18 on page 308. The outputs are the control lines, and the input is the 6-bit apcade field , Op [5:0J, Thus, we ca n create a truth table fo r each of the outputs based on the binary encoding of the apcades. Figure 5.22 shows the logic in the control unit as one large truth table that combines all the outputs and that uses the opcade bits as inputs. It completely specifies the control function, and we can implement it directly in ga tes in an automated fashi on. We show this final step in Section C.2 in II Appendix C. Input or output

Signal name

Inputs

Op5 Op4 Op3 Op2 Op1 OpO

Outputs

RegDst ALUSn:

MemtoReg RegWrite MemRead MemWrite

Branch ALUOp 1 ALUOpO

single-cycle im plementatio n Also called single dock cycle implementation. An implementation in which an instruction is executed in one clock cycle.

R-format

0 0 0 0 0 0 1 0 0 1 0 0 0 1 0

1 0 0 0 1 1 0 1 1 1 1 0 0 0 0

1 0 1 0 1 1 X 1 X 0 0 1 0 0 0

0 0 0 1 0 0 X 0 X 0 0 0 1 0 1

FIGURE 5.22 The control function for the simple slngle-eycle Implementation Is completely specified by this truth table. The top half of the table gives the combinations of input signals that correspond to the four opcodes that determine the control output settings. ( ~member that Op [5:0 1 corresponds to bits 31:26 of the instruction, which is the op field. ) The bottom portion of the table gives the outputs. Thus, the output RegWrite is asserted for two different combinations of the inputs. If we consider only the four opcodes shown in this table, then we can simplify the truth table by using don't cares in the input portion. For example, we can detect an R·format instruction with the expression OpS • Opl, since this is sufficient to distinguish the R·forntat instructions from lw, sw, and beq. We do not take advantage of this simplific.1tion, since the rest of the MIPS opoodes are used in a full implementation.

5.4

313

A Simple Implementation Scheme

Now, let's add the jump instruction to show how the basic datapath and con t rol ca n be extended to handle other in structions in the instruction set.

Implementing Jumps

Figure 5.1 7 on page 307 shows the implementation of many of the instructions we looked at in Chapter 2. One class of instructions missing is that of the jump in struction. Extend the datapath and cont rol of Figure 5.1 7 to in clude the jump instruction. Describe how to set any new cont rol lines.

EXAMPLE

The jump instruction looks somewhat like a branch in struction but com putes the target PC differently and is not conditional. Like a branch , the loworder 2 bits of a jump address are always ootwo. The next lower 26 bits of this 32-bit address come from the 26- bit immediate field in the instruction , as shown in Figure 5.23. The upper 4 bits of the address that should repla ce the PC come from the PC of the jump instruction plus 4. Thus, we can implement a jump by storing into the PC the concatenation of

ANSWER

• the upper 4 bits of the current PC + 4 (these are bits 31:28 of the sequentially following in struction address) • the 26-bit immediate fi eld of the jump instruction • the bits OOtwo Figure 5.24 shows the addition of the control fo r jump added to Figu re 5.1 7. An additional multiplexo r is used to select the source for the new PC va lue, which is either the incremented PC ( PC + 4), the branch target PC, or the jump target Pc. One additional cont rol signal is needed for the additional multiplexor. This control signal, ca lled jump, is asserted only when the in struction is a jump- that is, when the opcade is 2.

Field

Bit positions

000010

add r ess

3 1:26

2 5 :0

FtGURE 5.23 Instruction format for the jump Instruction (opcode _ 2). The destination address for a jump instruction is formed by conca tenating the upper 4 bits of the curren t PC + 4 to the 26-bit address field in the jump instruction and adding 00 as the 2 low-order bits.

314

Chapter 5

~Slr UClion [25-0J

Th e Processor: Dat apath and Co ntrol

Jump address [31-01

S hi ft 26 ~f12 28

PC+4(3 1 28)

) Add

V /

Instruction [31-26]

Contro l

\ Instruction [25-21)

address Instruction [20-16)

[31-0) Instruction memory

ALU

I Instruction [15-11)

R~d

0 M

," I-1

e--

Read data 1

Zero

Read data 2

Write

d'fa

Instruction [ 15-01

R~d

Instruction

Shift c-I left 2

RegDsl Jump Branch MemRead MemloReg ALUO MemWrite ALUSrc RegWrite

-----

Add result

L.. 0

Registers

~f-.

Read

Address data

Data Wrne memory

I-- .L '

Sign extend

ALU ALU result

"--

data

ALU

contro l

Instruction [5-01

FIGURE 5.24 Th e simple contro l and dat apath are ext ended t o handle th e jump Instructi on. An addltlonal multIplexor (at the upper right ) is used to choose between the jump larget and either the branch target or the sequential instruction following this one. This multiplexor is controlled by the jump control signal. The jmnp target address is obtained by shifting the lower 26 bits of the jump instruction left 2 bits, effectively adding 00 as the low-order bits, and then concatenating the upper 4 bits of PC + 4 as the high-order bits, thus yielding a 32-bit address.

Why a Single.cycle Implementation Is Not Used Today Although the single-cycle design will work co rrectly, it would not be used in m odern designs because it is ineffi cient. To see why this is so, notice that the clock cycle must have the sam e length for every instruction in this single-cycle design, and the CPI

S.4

315

A Simple Implementation Scheme

(see Chapter 4) will therefore be 1. Of course, the clock cycle is determined by the longest possible path in the machine. This path is almost certainly a load instruction , which uses five fun ctional units in series: the instruction memory, the register file, the ALU, the data memory, and the register fil e. Although the CPl is I, the overall performance of a single-cycle implementation is not likely to be very good, since several of the instruction classes could fit in a shorter clock cycle.

Performance of Single-Cycle Machines

Assume that the operation times fo r the major fun ctional units in this implementation are the following:

EXAMPLE

• Memory units: 200 picoseconds (ps) • ALU and adders: 100 ps • Register file (read or write) : 50 ps Assuming th at the multiplexors, cont rol unit, PC accesses, sign extension unit, and wires have no delay, which of the following implementations would be faster and by how much? 1. An implementation in which ever y instruction operates in I clock cycle

of a fixed length. 2. An implementation where every instruction executes in I clock cycle using a va riable-length clock, which for each instruction is only as long as it needs to be. (Such an approach is not terribly practica l, but it will allow us to see what is being sacrificed when all the instructions must execute in a single clock of the sa me length.) To compare the performance, assume the following instruction mix: 25% loads, 10% stores, 45% ALU instructions, 15% branches, and 5% jumps.

Let's start by comparing the CPU execution times. Recall from Chapter 4 that CPU execution time = Instruction count x CPI x Clock cycle time Since CPI must be I, we can simplify this to CPU execution ti me = Inst ruction count x Clock cycle time

ANSWER

316

Chapter 5

The Processor: Datapath and Control

We need o nly find the clock cycle time for the two implementatio ns, since the in struction count and CPI are the same for both implementations. The critical path for the different instructio n classes is as follows: Instruction class

Functional units used by the Instruction class

R·type

Instruction fetch

Load word

InstnJction fetch

Store word

InstnJction fetch

Branch

Instruction fetch

Jump

Instruction fetch

ALU ALU ALU ALU

Memory access

Using these critical paths, we ca n co mpute the required length for each instruction class: Instruction class

Instruction memory

R·type

200

Load word Store word Branch

Jump

ALU operation

Data memory

100

200 200

50 50

100 100

200 200

100

II 400 ps

600 ps 550 ps 350 ps 200 ps

The clock cycle for a ma chine with a single clock for all instructio ns will be determined by the lo ngest instruction, which is 600 ps. (This timing is approximate, sin ce o ur timing model is quite simplistic. In reality, the timing of m odern digital systems is complex.) A machine with a va riable clock will have a clock cycle that va ries between

200 ps and 600 ps. We ca n find the average clock cycle length for a m achine with a va riable-length clock using the informatio n above and the instruction frequency distribution. Thus, the average time per instructio n with a va riable clock is CPU clock cycle = 600 x 25% + 550 x 10% + 400 x 45%

+ 350 x 15% + 200 x 5%

447.5 ps Since the va riable clock implem entatio n has a shorter average clock cycle, it is clearly faster. Let's find the performance ratio:

S.4

317

A Simple Implementation Sche me

CPU performancevariabl~dock

CPU execution timesingl~ dock

CPU performanCesingl~ clod:

CPU execution t ime ,.ariabl~ clod:

IC x CPU clock cyclesingl~ dock = CPU clock cyclesin*dock ) ( IC x CPU clock cyclevari.bl~dock

CPU clock cyclevari.bl~dod:

= 600 = 1. 34 447.5 The va riable clock implementation would be 1.34 times faster. Unfortunately, implem enting a va riable-speed clock for each instruction class is extremely difficult, and the overhead for such an approach could be larger than any adva ntage gained. As we will see in the next section , an alternative is to use a sho rter clock cycle that does less wo rk and then vary the number of clock cycles fo r the different instruction classes. The penalty for using the single-cycle design with a fi xed clock cycle is significa nt, but might be considered acceptable for this small instruction set. Historically, ea rly ma chines with very simple instruction sets did use this implementation technique. However, if we tried to implement the fl oating-point unit or an instruction set with more complex instructions, this single-cycle design wouldn't work well at all. An exa mple of this is shown in the " For More Practice Exercise 5.4. Because we must assume that the clock cycle is equal to the worst-case delay for all instructions, we ca n't use implementation techniques that red uce the delay of the common case but do not improve the worst-case cycle time. A single-cycle implementation thus violates o ur key design principle of making the comm on case fast. In addition, in this single-cycle implementation, each fun ctional unit ca n be used only once per clock; therefore, some fun ctional units must be duplicated, raising the cost of the implem entation. A single-cycle design is ineffici ent both in its performance and in its hardware cost! We ca n avoid these difficulties by using implementation techniques that have a shorter clock cycle-derived from the basic fun ctional unit delays-and that require multiple clock cycles for each in struction. The next section explores this alternative implem entation scheme. In Chapter 6, we'll look at another implementation technique, ca lled pipelining, that u ses a datapath very similar to the single-cycle datapath , but is much m ore efficient. Pipelining ga ins efficiency by overlapping the execution of multiple instructions, increasing hardwa re utiliza tion and improving performance. For those readers interested primarily in the high-level concepts used in processors, the material of this section is sufficient to read the introductory sections of Chapter 6 and und erstand the basic fun ctional-

318

Chapter 5

The Processor: Datapath and Control

ity of a pipel ined processo r. For those, who want to understand how the ha rdwa re really implements the control, forge ahead !

Check Yourself

Look at the cont rol signal in Figure 5.22 on page 3 12. Can any cont rol signal in the figure be replaced by the inverse of another? (Hint: Take into account the don't cares.) If so, ca n you use one signal for the other without adding an inverter?

A Multicycle Implementation

multi cyd e imple m e ntation Also called multiple dock cyde implem en ta tion. An imp le m entatio n in which a n ins tructio n is executed in multiple dock cydes.

In an earlier exam ple, we broke each instruction into a series of steps corresponding to the fun ctional unit operations that were needed. We can use these steps to create a multicyde implementation. In a multicycle implementation, each step in the execution will take I clock cycle. The multicycle implementation allows a fun ctional unit to be used more than once per instruction, as long as it is used on different clock cycles. This sha ring can help reduce the amount of hardware required. The ability to allow instructions to take different numbers of clock cycles and the ability to sha re fun ctional units within the execution of a single instruction are the major advantages of a multicycle design. Figu re 5.25 shows the abstract version of the mul-

Address

Instru cti on register

Registe r # Regi sters Registe r #

Instruction Memory or data Data

Data

Memory data register

Registe r #

f- - A

~ ) ALU

f- '---B

ALU Out

FIGURE 5.25 11Ie high-level view of the multlcycle datapath. This picture shows the key elements of the data path: a shared memory unit, a single ALU shared among instructions, and the connections among these shared lUlits. The use of shared rrUlctionallUlits requires the addition or widening of multiplexors as well as new temporary registers that hold data between dock cycles of the same instruction. The additional registers are the Instruction register (IR), the Memory data register (MDR),A, B, and AWOUl.

5.5

A Multleyele Imple mentation

ticycle datapath. If we compare Figure 5.25 to the datapath for the single-cycle version in Figure 5. 11 on page 300, we ca n see the following differences: • A single memory unit is used for both instructions and data. • There is a single ALU, rather than an ALU and two adders. • One or m ore registers are added after every major fun ctional unit to hold the output of that unit until the value is used in a subsequent clock cycle. At the end of a clock cycle, all data that is used in subsequent clock cycles must be stored in a state element. Data used by 5lIbsequent instructions in a later clock cycle is stored into one of the progra mmer-visible state elements: the register file, the PC, or the memory. In contrast, data used by the same instruction in a later cycle must be stored into one of these additional registers. Thus, the position of the additional registers is determined by the two fa ctors: what combinational units will fit in one clock cycle and what data are needed in later cycles implementing the instruction. In this multicycle design, we assume that the clock cycle ca n accommodate at most one of the following operations: a memory access, a register fil e access (two reads or one write), or an ALU operation. Hence, any data produced by one of these three fun ctional units (the mem ory, the register fil e, or the ALU) must be saved, into a temporary register for use on a later cycle. If it were not saved then the possibility of a timing race could occur, leading to the use of an in correct value. The following temporary registers are added to meet these requirements: • The Instruction register (I R) and the Memory data register (MDR) are added to save the output of the memory for an instruction read and a data read, respectively. Two separate registers are used, since, as will be clear shortly, both va lues are needed during the same clock cycle. • The A and B registers are used to hold the register opera nd values read from the register file. • The ALUOut register holds the output of the ALU. All the registers except the IR hold data only between a pair of adjacent clock cycles and will thus not need a write control signal. The IR needs to hold the instruction until the end of execution of that instruction, and thus will require a write control signal. This distinction will become more clear when we show the individual clock cycles for each instruction. Because several functional units are shared for different purposes, we need both to add multiplexors and to expa nd existing multiplexors. For exa mple, since one memory is used for both instructions and data, we need a multiplexor to select between the two sources for a memory address, namely, the PC (for in struction access) and ALUOut (for data access).

319

320

Chapter 5

Th e Processor: Datapath and Control

Replacing the three ALUs of the single-cycle datapath by a single ALU means that the single ALU must accommodate all the inputs that used to go to the three different ALUs. Handling the additional inputs requires two changes to the datapath:

I. An additional multiplexor is added for the first ALU input. The multiplexor chooses between the A register and the Pc. 2. The multiplexo r on the second ALU input is changed from a two-way to a four-way multiplexor. The two additional inputs to the multiplexor are the constant 4 (used to increment the PC) and the sign-extended and shifted offset field (used in the branch address computation).

Figure 5.26 shows the details of the datapath with these additional multiplexo rs. By introducing a few registers and multiplexors, we are able to reduce the number of memory units from two to one and eliminate two adders. Sin ce registers and multiplexors are fairly small compared to a memory unit or ALU, this could yield a substantial red uction in the hardwa re cost.

'0 M

Address Me mo ry MemData

Read register 1

Instruction [20-16]

Read register 2 Reg iste rs Write R d . te ea regis r data 2

Instruction [15-0]

Write data

In struct ion reg ister

0 M

Memo ry d ata reg iste r

," 1

Instruction (15-11 )

Instruction [15-0]

c..

L.. 'IJ'

Instruction [25-21]

..r

," 1

Read data 1

Write

A L- ~r"-'

,r;::---...

4-0

~ Zero ) ALU ALU resuR

f-ALUOut

~ Mt-- V "

2 , 3

d.~

1\ 16

S ign exte nd

S hi ft left 2

\ FIGURE 5.2 6 Multleycle da t a path for MIPS handles the basic Instruction s. Although thIS d.1lapath supports normal incrementing of the PC, a few more connections and a multiplexor will be needed for branches and jwnps; we will add these shortly. The additions versllS the single-clock d.1lapath include several registers (IR, MDR, A, B,ALUOut), a multiplexor for the memory address, a multiplexor for the top AW input, and expanding the multiplexor on the bottomALU input into a four-way selector. These small additions allow lIS to remove tm> adders and a memory unit.

5.5

A Multleyele Implementation

Because the datapath shown in Figure 5.26 takes multiple clock cycles per instruction , it will require a different set of control signals. The programmer-visible state units (the PC, the memory, and the registers) as well as the l R will need write control signals. The memory will also need a read signal. We ca n use the ALU control unit from the single-cycle datapath (see Figure 5. 13 and II Appendix C) to control the ALU here as well. Finally, each of the two-input multiplexors requires a single control line, while the four-input multiplexor requires two control lines. Figure 5.27 shows the datapath of Figure 5.26 with these control lines added. The multicycle datapath still requires additions to support branches and jumps; after these additions, we will see how the instructions are sequenced and then generate the datapath control. With the jump instruction and branch instruction, there are three possible sources for the va lue to be written into the PC: 1. The output of the ALU, which is the va lue PC + 4 during instruction fetch. This value should be stored directly into the pc. 2. The register ALUOut, which is where we will store the address of the branch target after it is computed. 3. The lower 26 bits of the Instruction register OR) shifted left by two and concatenated with the upper 4 bits of the incremented PC, which is the source when the instruction is a jump. As we observed when we implemented the single-cycle cont rol, the PC is written both un conditionally and conditionally. During a normal in crement and for jumps, the PC is written unconditi onally. If the instruction is a condi tional branch, the incremented PC is replaced with the va lue in ALUOut only if th e two designated registers are equa l. Hen ce, our implementation uses two sepa rate control signals: PCWrite, which ca uses an unconditio nal w rite of the PC, and PCWriteCond, which causes a write of the PC if the branch condition is also true. We need to connect these two control signals to the PC write control. Just as we did in the single-cycle datapath, we will use a few ga tes to derive the PC write con t rol signal from PCWrite, PCWriteCond , and the Zero signal of the ALU, which is used to detect if the two register opera nd s of a beq are equal. To determine whether the PC should be written during a conditional branch, we AND together the Zero signal of the ALU with the PCWriteCond. The output of this AND ga te is then ORed with PCWrite, which is the unconditional PC write signal. The output of this OR gate is connected to the write control signal for the Pc. Figure 5.28 shows the complete multicycle datapath and control unit, including the additional control signals and multiplexor for implementing the PC updating.

321

322

Chapter 5

lorD MemRead MemWrite IRWrite

PC f-

a M

Address Memory MemData Write d.~

Th e Processor: Datapath and Control

RegDst

RegWrite

ALUSrcA

Instruction (25-21)

Read register 1

Instruction (20-16)

Read register 2 Registers Wr~e Read register data 2

Instruction [15-0) Inst ructio n reg ister Instruction [15-0)

a M

," 1

Instruction [15-11)

r'","

, ~ L. 1" f-.

r B

..r

Wrne data

I 16

"" ~ M r-V "

~ ./ 32

Shift left 2

ALU control

"Instruction (5-01

MemtoReg

ALUOut

Sig n extend

Z. cof-oA LU A LU res

Memory data reg ister

Rood data 1

ALUSrcB

ALUOp

FIGURE 5.27 The multleycle datapath from Figure 5.26 with th e control lines shown. The signals ALUOp and ALUSrcB are l·bit control signals, while all the other control lines are I·bit signals. Neither register A nor B requires a write signal, since their contents are only read on the cycle immediately after it is WTinen. The memory data register has been added to hold the data from a load when the data returns from memory. Data from a load returning from memory cannot be WTitten directly into the register file since the clock cycle cannot accommodate the time required for both the memory access and the register file write. The MemRead signal has b«n moved to the top of the memory unit to simplify the figures. The full set of data paths and control lines for branches will be added shortly.

Before exa mining the steps to execute each instruction, let us informally exa mine the effect of all the control signals (just as we did for the single-cycle design in Figure 5. 16 on page 306). Figure 5.29 shows what each control signal does when asserted and deasserted.

Elaboration: To reduce the number of signal lines interconnecting the functional units, designers can use shared buses. A shared bus is a set of lines that connect mul· tiple units; in most cases, they include multiple sources that can place data on the bus

5.5

323

A Multleycle Imple me nta tion

PCSource

PCWriteCond ~

PCWrite

ALUOp

Outputs

lorD

ALUSrcB Contro l

MemRead

ALUSrcA

MemWrile Op [5-0]

MemtoReg

RegWrite

lR'Nrite

RegDst

Instruction [25-0]

'-./

Instruction [31-26)

0 M

," 1

Instruction [20-16]

Memory MemData

Instruction [15-0]

Write

Instru ction reg ister

d,~

Rood Instruction [15-11 ]

Memory data register

0 M

~ r- 0M rI"

Instruction [15-0]

0 M

Rood

," 1

Read data 1

' Co

Sign extend

) ALU ALU result

'"" ~

Shift left 2

2 ~

~ ALUOut

4 .... 1 M 2 ,

Write data

1\ 16

"1,

1 "

PC [31-28)

Instruction [25-21]

Address

Shift 28 lell 2

Jump address [31-0]

'\ ALU

co ntro l

\ Instruction [5-0]

FtGURE 5.28 Th e co mple t e da t a pa th fo r the multleycle Imple m e nta tion t oget he r with th e ne cessary contro l lines. The controllines of Figure 5.27 are attached to the control unit, and the control and datapath elements needed to effect changes to the PC are included. The major additions from Figure 5.27 include the multiplexor wed to select the source of a new PC value; g.1tes wed to combine the PC write signals; and the control signals PCSource, PCWrite, and PCWriteCond. The PCWriteCond signal is used to decide whether a conditional branch should be taken. Support for jwnps is included.

and multiple readers of the value. Just as we reduced the number of functional units for the data path , we can reduce the number of buses interconnecting these units by sharing the buses . For example, there are six sources coming to the ALU; however, only two of them are needed at anyone time . Thus, a pair of buses can be used to hold values that are being sent to the ALU . Rather than placing a large multiplexor in front of the

324

Chapter 5

The Processor: Datapath and Control

Actions of the l·blt control signals Signal name

Effect when deasserted

Effect when asserted

RegDst

The register file destination number for the Write register comes from the It field .

The register file destination number for the Write register comes from the rd field .

Regwrite

None.

The general.purpose register seleded by the Write register number is written with the value of the Write data input.

ALUSrcA

The first ALU operand is the PC.

The first ALU operand comes from the A register.

MemRead

None.

Content of memory at the location specified by the Address input is put on Memory data output.

MemWrite

None.

Memory contents at the location specified by the Address input is replaced by value on Write data input.

MemtoReg

The value fed to the register file Write data input comes from ALUOut.

The value fed to the register file Write data input comes from the MDR.

lorD

The PC is used to supply the address to the memory unit.

ALUOut is used to supply the address to the memory unit.

IRWrite

None.

The output of the memory is written into the IR.

PCWrite

None.

The PC is written; the source is controlled by PCSource.

PCWriteCond

None.

The PC is written if the Zero output from the ALU is also active.

Actions of the 2·blt control signals Signal name

Value (binary)

Effect

ALUOp

The ALU perfonns an add operation . The ALU perfonns a subtract operation .

ALUSrcB

01 10 00 01 10

The second input to the ALU is the constant 4 .

The second input to the ALU is the sign.extended, lower 16 bits of the IR shifted left 2 bits.

00 01

Output of the ALU (PC

PCSource

FIGURE 5.29

The funct field of the instruction determines the ALU operation . The second input to the ALU comes from the B register .

The second input to the ALU is the sign.extended, lower 16 bits of the IR.

+ 4)

is sent to the PC for writing.

The contents of ALUOut (the branch target address) are sent to the PC for writing. The jump target address (IR[25 :0) shifted left 2 bits and concatenated with PC + 4[31 :28]) is sent to the PC for writing.

The action caused by the setting of each control signal In Figure 5.28 on page 323. The top table describes the I-bit

control signals, while the bottom table describes the l-bit signals. Only those control lines that affect multiplexors h.we an action when they aredeasserted. This information is similar to that in Figure 5.16 on page 306 for the single-cycle datap.1th, but adds several new control lines (IRWrite, PCWrite, PCWriteCond, ALUSrcB, and PCSource) and removes control lines that are no longer used or have been repbced (PCSrc, Branch, and Jump).

ALU, a designer can use a shared bus and then ensure that only one of the sources is driving the bus at any point. Although this saves signal lines, the same number of control lines will be needed to control what goes on the bus. The major drawback to using such bus structures is a potential performance penalty, since a bus is unlikely to be as fast as a point-to-point connection .

5.5

A Multleyele Implementation

Breaking the Instruction Execution into Clock Cycles Given the datapath in Figu re 5.28, we now need to look at what should happen in each clock cycle of the multicycle execution, since this will determine what additional cont rol signals m ay be needed, as well as the setting of the control signals. Our goal in brea king the execution into clock cycles should be to maximize performance. We can begin by breaking the execution of any instruction into a series of steps, each taking one clock cycle, attempting to keep the a mount of work per cycle ro ughly equal. For example, we will restrict each step to contain at m ost one ALU o peration, or one register fil e access, or one mem ory access. With this restriction, the clock cycle could be as short as the longest of these o perations. Reca ll that at the end of every clock cycle any data values th at will be needed on a subsequent cycle must be stored into a register, which can be either one of the m ajor state elem ents (e.g., the PC, the register file, or the m em ory), a temporary register written on every clock cycle (e.g., A, B, MDR, or ALUOut), or a temporary register with write control (e.g., l R). Also rem ember that because o ur design is edge- triggered, we ca n continue to read the current value of a register; the new value does not appea r until the next clock cycle. In the single-cycle datapath , each instruction uses a set of datapath elements to ca rry out its execution. Many of the datapath elements o perate in series, using the o utput of another elem ent as an input. Some datapath elements o perate in parallel; for example, the PC is incremented and the instruction is read at the sa me time. A similar situation exists in the multicycle datapath. All the o perations listed in one step occur in pa rallel within 1 clock cycle, while successive steps o perate in series in different clock cycles. The limitation of one ALU o peration, one mem ory access, and one register file access determines what can fit in one step. Notice that we distinguish between rea ding from or writing into the PC o r one of the stand -a lone registers and reading from o r writing into the register fil e. In the former case, the read or write is part of a clock cycle, while reading or writing a result into the register fil e takes an additional clock cycle. The reason for this distinction is that the register fil e has additional cont rol and access overhea d com pa red to the single stand -alone registers. Thus, keeping the clock cycle short m otiva tes dedica ting separate clock cycles for register fil e accesses. The potential execution steps and their actions are given below. Each M IPS instruction needs from three to five of these steps:

1. Instruction fetch step Fetch the instruction from mem o ry a nd compute the address of the next sequen tial instruction:

IR PC

to select this value to be written into the Pc. Because the PC is incremented during the first cycle of every instruction , we ca nn ot just write the value of the PC into the EPC, since the value in the PC will be the instruction address plus 4. However, we ca n use the ALU to subtract 4 from the PC and write the output into the EPC. This requires no additional control signals or paths , since we ca n use the ALU to subtra ct, and the constant 4 is already a selectable ALU input. The data write port of the EPC, therefore, is connected to the ALU output. Figure 5.39 shows the multicycle datapath with these additions needed for implementing exceptions. Using the data path of Figure 5.39, the action to be taken for each different type of exception ca n be handled in one state apiece. In each case, the state sets the Cause register, computes and saves the original PC into the EPC, and writes the exception address into the Pc. Thus, to handle the two exception types we are considering, we will need to add only the two states, but before we add them we must determine how to check for exceptions, since these checks will control the arcs to the new states.

How Control Checks for Exceptions Now we have to design a method to detect these exceptions and to transfer control to the appropriate state in the exception states. Figure 5.40 shows the two new states ( 10 and 11 ) as well as their connection to the rest of the finite state control. Each of the two possible exceptions is detected differently: •

Undefined instrtlction: This is detected when no next state is defined from state 1 for the op va lue. We handle this exception by defining the next -state va lue for all op va lues other than 1W, SW, 0 ( R-type), j, and beq as state 10. We show this by symbolically using other to indicate that the op field does not match any of the opcodes that label arcs out of state 1 to the new state 10, which is u sed for this exception.

• Arithmetic overflow: The ALU, designed in . Appendix B, included logic to detect overfl ow, and a signal called Overflow is provided as an output from the ALU. This signal is used in the modified finit e state machine to specify an additional possible next state (state 11 ) for state 7, as shown in Figure 5.40.

343

344

Chapter 5

PCWr~eCond

PCWrile

The Processor: Datapath and Control

(\ Outputs

10.0 MemRead

Co ntro l

MemWrite MemtoReg

[5-0[

EPCWrite PCSource ALUOp ALUSrcB RegWr~e

IntCause

ALUSrcA

IRWrite

CauseWr~e

,------{o"

RagDst

S hift

Jump address

2• r:---,-C I" " .~ , ru ~ Clio ~" f t2~ 5-0 ::: J J::::=j-1----tj::~~~, ~ .ft ~2~~;~ttlT [3c ' BI = 2, bit -> BI = 0, etc.)

BD:

Bran ch relative destination.

Your simplified PowerPC implementation should be able to implement the following instructions:

ad d :

add $Rd , $Rt, $Rs addi $Rd , $Rt, Un

subtr ac t :

sub $Rd, $Rt, $Rs

($Rd ( - $Rt

$Rs)

sub i $Rd , $Rt, Itn

($Rd ( - $Rt

Un)

load: s t ore : AND , OR : Jump :

lw $Rd , Addr($Rt) sw $Rd , Addr($Rt)

($Rd ( - $Rt + $Rs) ($Rd ( - $Rt + Un)

( $Rd ( - Memory [$Rt + AddrJ) ( Memory[$Rt + Addr] ( - $Rd)

and/or $Rd , $Rt, $R s andi/ori $Rd, $Rt , #n jmp Addr (PC ( - Addr)

Branch cond itional: PC+4 )

Seq Addr

($Rd ( - $Rt AND/OR $Rs) ($Rd ( - $Rt AND/OR Un)

(CR[2] ==1? PC( - PC+SD : PC ( -

subrou tine c all : jal Addr (LN KR ( - PC +4 ; PC( - Addr) IPC ( - LNKR) subrou tine re s t ore : Ret 5.64 IDiscussion J Hypothesis: If the first implementation of an architecnlre uses microprogra mming, it affects the in stmction set architecnlre.

364

Chapter 5

The Processor: Datapath and Control

im plementation do you think the architect had in mind when designing the in struction set a rchitecture?

5.65 [Discussion ] Wilkes invented microprogra mming in large pa rt to simplify construction of control. Since 1980, there has been an explosion of computer-a ided design software whose goal is also to simplify construction of con trol. This has made cont rol design much easier. Ca n you find evidence, based either on the tools o r on real designs, that suppo rts or refut es this hypothesis? 5.66 [Discussion ] The MIPS instructions and the MIPS microinstructions have many simila rities. What would m ake it difficult for a compiler to produce M IPS microcode rather tha n macrocode? W hat changes to the microa rchitecture would m ake the microcode m ore useful fo r this applica tio n.

Answers to Check Yourself

§5.1 , page 289: 3. §5.2, page 292: false. §5.3, page 299: A. §5.4, page 3 18: Yes, Memto Reg a nd RegDst are inverses of o ne a nother. Yes, simply use the other signal and flip the o rder of the inputs to the multiplexor! §5.5, page 340: 1. False. 2. M aybe: If the signal PCSource (O] is always set to zero when it is a do n't ca re (which is m ost states), then it is identica l to PCWriteCond. §5.6, page 346: No, since the value of 11 , which was form erly unused, is now used ! §5.7, page 5.7- 13: 4 tables with 55 entries (do n't forget the primary dispatch !) §5.8, page 5.8-7: 1. 0, 1, 1, X, 0. 2. No, sin ce state is not assigned on ever y path.

Computers in the Real World

Empowering the Disabled

To overcome the obstacles faced by disabled people.

Problem to solve:

Solution: Use robotics, sensors, and com-

puter control to replace or supplement damaged limbs and organs. The picture on the right shows a system developed for a firefighter who was injured while fighting a fire. Sensors in latex fingers instantly register hot and cold, and an elecFirefighter Ken Whitten proudly displays his new tronic interface in his artificia l limb stimulates bionic arm. the nerve endings in his upper arm, which then pass the information to his brain. The bypass the photoreceptors of the eye with a $3000 system allows his hand to feel pressure signa l from a digital camera that connects and weight, so for the first time since losing directly to the visua l system. They are develhis arms in a 1986 accident, he can pick up a oping a neural interface to the visua l system can of soda without crushing it or having it ca lled the artificia l synapse ch ip. The cha lslip through his fingers. The main enabling lenge is to turn electrical signa ls into the device is an electronic interface that can trans- chemicals that cell s use to communicate. This mit signa ls to nerve endings in Whitten's ch ip is attached to ce lls, and from the ce ll 's upper arm, which then pass this information perspective the artifici al synapse is simp ly a to his brain. hole in the silicon. This hole is connected to a Harvey Fishman and Mark Peterman of reservoir of neurotransmitter. When an elecStanford have taken steps towards informa- tric field is app lied to the ch ip, the neution technology that might someday treat rotransmitter is pumped through the hole, age-related blindness. Their approach is to stimulating nearby cells. In 2003 they have

created four artificial synapses on a chip one centimeter on a side. Although this work is in its ea rly stages, the potential is not limited to eye problems. According to Fishman , "Anywhere there's a severing of a nerve connection, there 's a potential for us to reconnect it." To learn more see these references on the [41 library

-, ..... __ N1_. __"'_. . _tl_. .

A-.. .. _ .

_.It.~''

_ _ _ _ ...."..w... _ltoo-.,

ltoo

•

.. _k -_.. -.-... ._...-al_ ._..-_ . ._- -*9-_ ""--_1>,- ---- - ..... _ •

•

• Rick Smolan and Jennifer Erwitt, Glle DigittJl DrJy: How the Microchip Is ChrJnging GlIr World, Times Publishing, 1998 • Peterman et al, " The artificial synapse chip: A flexible retinal interface based on directed retinal cell growth and neurotransmitter st imulation," Artificial GrgrJlls: 27(11), November 18,2003

o ~_

...

. - . . . .... _

... ~c-.

........Il1o . . . . """"'" ~OIlA"'

0 __

opodk ....... _

"'_

Artificial retina using artificial synapse chips. From

The Sail Francisco Chronicle, January 5, 2004.

Enhancing Performance with Pipelining Thus times do shift, each thing his turn does hold; New things succeed, asformer things grow old. Robert Herrlek Ht5fXTidts; CtTC7lon;t$ jOrChristmas Ew, 1648

• •

6.1 6.2 6.3 6A 6.5 6.6 6.7

An Overview of Pipelining 370 A Pipelined Oatapath 384 Pipelined Conbol 399 Data Hazards and Forw.-ding 402 Data Hazards and Stalls 41 3 Branch Hazards 41 6 Using a H_dw_e Description la'Iguage to Describe and Model a

Pipeline

6.8 6.9 6.10 6.11 6.12 6.13 6.14

426 Exceptions 427

AdvaK:ed Pipelining: Extracting More Perfonna1Ce Real Stuff: The Pentium 4 Pipeline 448 Fallacies and Pitfalls 45 1 Concluding Remarks 452 Historical Perspective and Further Reading 454 Exercises 454

432

The Five Classic Components of a Computer

Interlace

Evatuating

"".,,,"''''''

370

Chapter 6

Enhancing Performance with Plpellnlng

Never waste time. American proverb

pipelining An implementation technique in which multiple instructions are overlapped in execution, much like to an assembly line.

An Overview of Pipelining Pipelining is an implementation technique in which multiple in structions are overlapped in execution. Today, pipelining is key to making processors fast. This section relies heavily on one analogy to give an overview of the pipelining terms and issues. If you are interested in just the big picture, you should concen trate on this section and then skip to Sections 6.9 and 6.10 to see an introduction to the advanced pipelining techniques used in recent processors such as the Pen tium III and 4. If you are interested in exploring the anatomy of a pipelined com puter, this section is a good introduction to Sections 6.2 through 6.8. Anyone who has done alot of laundry has intuitively used pipelining. The nOflpipelined approach to laundry would be 1. Place one dirty load of clothes in the washer. 2. \Vhen the washer is finish ed, place the wet load in the dryer. 3. \Vhen the dryer is fini shed, place the dry load on a table and fold. 4. \Vhen folding is finish ed, ask your roommate to put the clothes away. When your roommate is done, then start over with the next dirty load. The pipelined approach takes much less time, as Figure 6. 1 shows. As soon as the washer is fini shed with the first load and placed in the dryer, you load the washer with the second dirty load. When the first load is dry, you place it on the table to start folding, m ove the wet load to the dryer, and the next dirty load into the washer. Next you have your roommate put the first load away, you start folding the second load, the dryer has the third load, and you put the fourth load into the washer. At this point all steps-ca lled stages in pipelining-are operating con currently. As long as we have separate resources for each stage, we ca n pipeline the tasks. The pipelining paradox is that the time from placing a single dirty sock in the washer until it is dried, folded, and put away is not shorter for pipelining; the reason pipelining is fa ster for many loads is that everything is working in parallel, so more loads are fini shed per hour. Pipelining improves throughput of our laundry system without improving the time to complete a single load. Hence, pipelining would not decrease the time to complete one load of laundry, but when we have many loads of laundry to do, the improvement in throUgilput decreases the total time to complete the work. If all the stages take about the sa me amount of time and there is enough work to do, then the speedup due to pipelining is equal to the number of stages in the pipeline, in this case four: washing, drying, folding, and putting away. So, pipelined laundry is potentially four times faster than nonpipelined: 20 loads would

6.1

371

An Overvi ew of Plpellnlng

6 PM

' AM

6 PM

' AM

Time --,II~-,II~-J1~-l==-;-lI~-1==l~-'·'1=::IIII-Task order

c D

Task order

c D

~iiii. ~iii. ~iii. ~iii.

FIGURE 6.1 The laundry analogy for p1pellnlng. Ann, Brian, Cathy, and Don each have dirty clothes to be washed, dried, folded, and put away. The washer, dryer, ~folder," and "storer" each take 30 minutes for their task. Sequential laundry takes 8 hours for four loads of wash, while pipelined laundry takes just 3.5 hours. We show the pipeline stage of different loads over time by showing copies of the four resources on this two-dimensional time line, but we really have just one of each resource.

take about 5 times as long as 1 load, while 20 loads of sequential laundry takes 20 times as long as 1 load. It's only 2.3 times faster in Figure 6.1 because we only show 4 loads. Notice that at the beginning and end of the workload in the pipelined version in Figure 6. 1, the pipeline is not completely full, this start-up and wind-down affects performance when the number of tasks is not large compa red to the number of stages in the pipeline. If the number of loads is much larger th an 4, then the stages will be full most of the time and the increase in th roughput will be very close to 4. The sa me principles apply to processo rs where we pipeline instr uction execution. MIP S instructions classically take five steps: 1. Fetch instruction fro m mem ory. 2. Read registers while decoding the instruction. The format of M IPS instructions allows reading and decoding to occur simultaneously.

372

Chapter 6

Enhancing Performance with Plpellnlng

3. Execute the operation or ca lculate a n address. 4. Access an oper and in data mem ory. 5. Write the result into a register. Hence, the M IPS pipeline we explore in this chapter has five stages. The following exa mple shows that pipelining speeds up instruction execution just as it speeds up

the laundry.

Single-Cycle versus Pipelined Performance

EXAMPLE

ANSWER

To make this discussion concrete, let's create a pipeline. In this example, and in the rest of this chapter, we limit o ur attention to eight instructions: loa d wo rd (lw), store word ( sw), add ( add ) , subtract ( sub), and (and ), or (or ), set-less- than ( s 1 t ), and branch-on -equal ( beq ) . Compa re the average time between instructio ns o f a single-cycle implem entatio n, in wh ich all instruct io ns take 1 clock cycle, to a pipelined implem entation. The operatio n times for the m ajor fun ctional units in this exam ple are 200 ps fo r mem o ry access, 200 ps for ALU operation , and 100 ps fo r register file read or write. As we sa id in Chapter 5, in the single-cycle m o del every instructio n takes exactly 1 clock cycle, so the clock cycle must be stretched to accommod ate the slowest instruction .

Figure 6.2 shows the time required fo r each o f the eight instructio ns. The sin gle-cycle design must allow for the slowest instructio n- in Figure 6.2 it is 1w-so the time requi red fo r every instructio n is 800 ps. Similarly to Figure 6. 1, Figure 6.3 compa res no n pipelined and pipelined execution o f three loa d wo rd instructio ns. Thus, the time between the fi rst and fo urth instructions in the no n pipelined d esign is 3 X 800 ns o r 2400 ps. All the pipeline stages take a single clock cycle, so the clock cycle must be lo ng enough to accommodate the slowest operation . Just as the single-cycle d esign must take the worst-case clock cycle o f 800 ps even though som e in structio ns ca n be as fast as 500 ps, the pipelined executio n clock cycle must have the worst -case clock cycle o f 200 ps even though som e stages take o nly 100 ps. Pipelining still o ffers a fo urfold perform ance improvem ent: the time between the first and fo urth instructio ns is 3 X 200 ps or 600 ps.

6.1

373

An Overview of Plpellnlng

Load word (1 w)

200 ps

.. 100 ps

200 ps

Store word (sw) R.fonnat (add, sub, and, or, s 1 t)

200 ps 200 ps

100 ps 100 ps

200 ps 200 ps

200 ps

Branch (beq)

200 ps

100 ps

200 ps

Instruction fetch

Instruction class

ALU operation

Dl!IIlRl!lIII liiiillaill ·. 100 ps

800 ps

100 ps

700 ps 600 ps 500 ps

FIGURE 6.2 Total time for each Instruction calculated from the time for each component. This calculation assumes that the multiplexors, control unit, PC accesses, and sign extension unit have no delay.

Program

C 2 OO;::-__4CO~O, --_60~O,----,8~OCO_--,'~OOO::.:c_C12~OC0,--,--'4~OO=-_'C6~OO=-_'"8~OO " +

••

•" o•

FIGURE 6.21 The slngle-elock-eycle diagram corresponding to clock cycle 5 of the pipeline In Figures 6.19 and 6.20. As you can Sff, a single-clock -cycle figure is a vertical slice through a multiple-clock-cycle diagram .

6.3

399

Plpellned Control

A group of students have been debating the effici ency of the fi ve-stage pipeline when o ne student pointed o ut that no t all instructio ns are active in every stage of the pipeline. After deciding to igno re the effects of hazards, they made the following fi ve statements. \Vhich o nes are correct?

Check Yourself

I. Allowing jumps, branches, and ALU instructions to take fewer stages th an the five required by the load instructio n will increase pipeline performance under all circumstances. 2. Trying to allow some in structio ns to take fewer cycles does not help, since the th roughput is determined by the clock cycle; the number of pipe stages per instructio n affects latency, no t throughput. 3. Allowing jumps, branches, and ALU operatio ns to take fewer cycles o nly helps when no loads or stores are in the pipeline, so the benefits are small.

4. You canno t m ake ALU instructio ns take fewer cycles because of the writeback of the result, but branches and jumps ca n take fewer cycles, so there is som e oppo rtunity for improvem ent. 5. Instead of trying to m ake instructio ns take fewer cycles, we should explore making the pipeline lo nger, so that instructio ns take m ore cycles, but the cycles are sho rter. This could improve performance.

Pipelined Control Just as we added control to the simple datapath in Section 5.4, we now add control to the pipelined datapath. We sta rt with a simple design that views the problem through rose-colored glasses; in Sectio ns 6.4 th ro ugh 6.8, we remove these glasses to reveal the haza rds of the real wo rld. The first step is to label the control lines o n the existing data path. Figure 6.22 shows those lines. We borrow as much as we can from the control for the simple datapath in Figure 5. 17 on page 307. In particular, we use the same ALU control logic, branch logic, destination-register-number multiplexor, and cont rol lines. These fun ctions are defined in Figure 5.1 2 o n page 302, Figure 5. 16 o n page 306, and Figure 5. 18 o n page 308. We reproduce the key informatio n in Figures 6.23 through 6.25 to make the remaining text easier to follow. As for the single-cycle implem entatio n discussed in Chapter 5, we assume that the PC is written o n each clock cycle, so there is no separate write signal for the

In the 6600 Compl/ter, perhaps even more than in any previol/s compl/ter, the control system is the difference. James Thornton,

Design ofa Computer: The Comrol Data 6600, 1970

400

Chapter 6

Enhancing Performance with Plpellnlng

r----------------------------------' ''~

'- v

•" ':u.~_~_~--, ,•

-_.

r- _ , -

- -L-I w ", ....n_

,---1 r

Add AlU

••

bro

~ r-l.---'jro;o!

ALUSn:

~og

••

•"

o•

f-- 'O ALUOp

I" "I

~ f------I

,•

FIGURE 6.22 The plpellned datapath of Figure 6.17 with the control signals Identified. ThIs dalapalh borrows the controllogJc for PC source, register destination number, and ALU control from Chapter 5. Note that we now need the 6-bit funcl field (function code) of the instruction in the EX stage as inpul to ALU control, so these bits must also be included in the ID/EX pipeline register. RJ.>call that these 6 bits are also the 6 least significant bits of the immediate field in the instruction, so the ID/EX pipeline register can supply them from the immediate field since sign extension leaves these bits lUlchangoo.

Pc. By the sa me argument, there are no separate write signals for the pipeline registers (l FIl D, ID/EX, EXlM EM, and ME M/W B), sin ce the pipeline registers are also written during each clock cycle. To specify control for the pipeline, we need only set the cont rol va lues during each pipeline stage. Beca use each control line is associated with a component active in only a single pipeline stage, we ca n divide the cont rol lines into fi ve groups acco rding to the pipeline stage. I. Instru ction fetch: The cont rol signals to read instruction memory and to write the PC are always asserted, so there is nothing specia l to cont rol III this pipeline stage.

6.3

401

Plpellned Control

Instruction opcode

•

Instruction operation

Desired ALU action

Function code

AW control Input 0010

)()(J()(J()(

,dd ,dd

branch equal

)()(J()(J()(

subtract

0110

,dd

100000

,dd

0010

R"pe

subtract

100010

subtract

0110

R"pe

AND

100100

0000

R"pe

100101

eo'

R"pe

set on less than

101010

()()

load word

)()(J()(J()(

()()

store word

Branch equal

R.type

FIGURE 6.23

0010

0001

set on less than

0111

A copy of Figure 5.12 on page 302. This figure shows how the ALU control bits are

set depending on the ALUOp control bits and the different function codes for the R-type instruction.

Signal name

Effect when deasserted (0)

Effect when asserted (1)

RegDst

The register destination number for the Write register comes from the rt field (bits 20:16).

The register destination number for the Write register comes from the rd field (bits 15:11).

Reg'Nrite

None.

The register on the Write register input is written with the value on the Write data input.

ALUSrc

The second ALU operand comes from the second register file output (Read data 2).

The second ALU operand is the sign.extended, lower 16 bits of the instruction .

PCSrc

The PC is replaced by the output of the adder that computes the value of PC + 4 .

The PC is replaced by the output of the adder that computes the branch target .

MemRead

None.

Data memory contents designated by the address input are put on the Read data output.

MemWrite

None.

Data memory contents designated by the address input are replac ed by the value on the Write data input.

MemtoReg

The value fed to the register Write data input comes from th e ALU .

The value fed to the register Write data input comes from the data memory.

FIGURE 6.24

A copy of Figure 5.16 on page 306. The function of each of seven control signals is defined. The ALU control lines (ALUOp)

are defined in the second column of Figure 6.23. When a I-bit control to a two-way multiplexor is asserted, the multiplexor selects the input corresponding to I. Otherwise, if the control is deasserted, the multiplexor selects the 0 input. Note that PCSrc is controlled by an AND gate in Figure 6.22. If the Branch signal and the ALU Zero signal are both set, then PCSrc is I; otherwise, it is o. Control sets the Branch signal only during a beq instruction; otherwise, PCSrc is set to o.

Execution/ address calculation stage control lines

Reg

ALU

Dst

Opl

R·format

Memory access stage control lines

Write-back stage control lines

Mem

Reg

Mem to

Branch

Read

Write

Reg

OpD

ALU S"

beq

FIGURE 6.25 The values of the control lines are the same as In Figure 5.18 on page 308, but they have been shuffled Into three groups corresponding to the last three pipeline stages.

402

Chapter 6

Enhancing Performance with Plpellnlng

2. Instruction decode/register file read: As in the previous stage, the sa me thing happens at every clock cycle, so there are no optional control lines to set. 3. Execution/address calculation: The signals to be set are RegDst, ALUOp, and ALUSrc (see Figures 6.23 and 6.24). The signals select the Result register, the ALU operation, and either Read data 2 or a sign -extended immediate for the ALU.

4. Memory access: The control lines set in this stage are Bran ch, MemRead, and Mem Write. These signals are set by the branch equal, load, and sto re in structions, respectively. Recall that PCSrc in Figure 6.24 selects the next sequential address unless control asserts Branch and the ALU result was zero. 5. Write back: The two control lines are MemtoReg, which decides between sending the ALU result o r the memory value to the register fil e, and RegWrite, which writes the chosen value.

Since pipelining the datapath leaves the mea ning of the control lines unchanged, we ca n u se the sa me control values as before. Figure 6.25 has the same values as in Chapter 5, but now the nine control lines are grouped by pipeline stage. Implementing control means setting the nine control lines to these values in each stage for each instruction. The simplest way to do this is to extend the pipeline registers to include control information. Since the control lines start with the EX stage, we ca n create the control information during instruction decode. Figure 6.26 shows that these control signals are then used in the appropriate pipeline stage as the in struction moves down the pipeline, just as the destination register number fo r loads moves down the pipeline in Figure 6.1 7 on page 395. Figure 6.27 shows the full data path with the extended pipeline registers and with the control lines connected to the proper stage. What do you mean, why's it got to be built? It's a bypass. You've got to bl/ild bypasses. Douglas Adams, Hitchhikers Guide to the Galaxy, 1979

Data Hazards and Forwarding The examples in the previous section show the power of pipelined execution and how the hardwa re performs the task. It's now time to take off the rose-colored glasses and look at what happens with real programs. The instructions in Figures 6. 19 through 6.2 1 were independent ; none of them used the results calculated by any of the others. Yet in Section 6. 1 we saw that data haza rds are obstacles to pipelined execution. Let's look at a sequence with many dependences, shown in colo r:

6.4

403

Data Hazards and Forwarding

\ Instructi

Control

IDlEX

IF/ID

f--f--- .. f--- ..

EXiMEM

WB -

MEMlWB

FIGURE 6.26 The control lines for th e final thre e stages. Note that four of the nine control lines are used in the EX phase, with the remaining five control lines passed on to the EXl MEM pipeline register extended to hold the cont rol lines; three are used during the MEM stage, and the last two are passed to MEM/WB for use in the W B stage.

su b and

or add

$2 , 11 , $3 $12 , $2 , $5 $13 ,$ 6 , $2 $14, $2 , $2 $15 , 1001 $2 1

# # # # #

Regis t e r $2 writ t en by sub 1s t ope r and($2) depends on sub 2nd ope ra nd($2) depends on sub 1st($2) & 2nd( $2) depend on sub Base ($2) depends on sub

The last four instructions are all dependent on the result in register $2 of the first instruction. If register $2 had the value 10 before the subtract instruction and -20 afterwa rd s, the programmer intend s that -20 will be used in the following instructions that refer to register $ 2. How would this sequence perform with our pipeline? Figure 6.28 illustrates the execution of these instructions using a multiple-clock-cycle pipeline representation. To demonstrate the execution of this instruction sequence in our current pipeline, the top of Figure 6.28 shows the value of register $ 2, which changes during the middle of clock cycle 5, when the sub instruction writes its result. One potential hazard ca n be resolved by the design of the register fil e hardwa re: what happens when a register is rea d and written in the same clock cycle? We assume that the write is in the first half of the clock cycle and the read is in the second half, so the read delivers what is written. As is the case for many implementations of register fil es, we have no data haza rd in this case.

404

Chapter 6

Enhancing Pe rform ance with Plpellnlng

~ WB

EXIIolEM

I-

•

~ '"" ,•"

Control

~ l

-'-

... -, ,....... , ~

- -c - . I

'ri

[15...q

ALU Al.U

'-' Sign

•

c- ~ ,"•"

[20-"1

[15- " 1

,.,

"•"

. _nd

~l- kUS
--

, "•

FIGURE 6.38 The ID stage of clock cycle 3 determines that a branch must be taken, so It selects 72 as the next PC address and zeros the Instruction fetched for the next clock cycle. Clock cycle 4 shows the instruction allocation 72 being fetched and the single bubble or nap instruction in the pipeline as a result oflhe taken branch. (Since the nap is really s 11 $ 0, SO, 0, it's arguable whether or not the ID stage in clock 4 should be highlighted. )

6.6

421

Branch Hazards

Dynamic Branch Prediction Assuming a branch is not taken is one simple form of branch prediction. In that case, we predict that branches are untaken, flushing the pipeline when we are wrong. For the simple five-stage pipeline, such an approach, possibly coupled with compiler-based prediction, is probably adequate. With deeper pipelines, the branch penalty increases when measured in clock cycles. Similarly, with multiple issue, the branch penalty in creases in terms of instructions lost. This combination mea ns that in an aggressive pipeline, a simple static prediction scheme will probably waste too much performance. As we mentioned in Section 6.1 , with more hardwa re it is possible to try to predict branch behavior during program execution. One app roach is to look up the address of the instruction to see if a branch was taken the last time this instruction was executed, and , if so, to begin fetching new instructions from the sa me pla ce as the last time. This technique is called dynamic branch prediction. One implementation of that approach is a branch prediction buffer or branch history table. A branch prediction buffer is a sm all memory indexed by the lower portion of the address of the branch instruction. The memory contains a bit that says whether the branch was recently taken or not. This is the simplest so rt of buffer; we don't kn ow, in fact, if the prediction is the right one-it may have been put there by another branch th at has the sa me loworder address bits. But this doesn't affect correctness. Prediction is just a hint that is assumed to be co rrect, so fetching begins in the predicted direction. If the hint turns out to be wrong, the inco rrectly predicted instructions are deleted, the prediction bit is inverted and stored back, and the proper sequence is fetched and executed. This simple I-bit prediction scheme has a performance shortcoming: even if a branch is almost always taken , we will likely predict incorrectly twice, rather than once, when it is not taken. The following example shows this dilemma.

Loops and Prediction

Consider a loop branch that branches nine times in a row, then is not taken once. What is the prediction accuracy fo r this branch, assuming the prediction bit for this branch remain s in the prediction buffer?

dynamic branch prediction Prediction of b ranches at runtime using runtime information. branch prediction buffer Also called branch history table. A small memory that is indexed by the lower portion of the address of the branch instruction and that contains one or more bits indicating whether the b ranch was recently taken or not.

EXAMPLE

422

Chapter 6

ANSWER

Enhancing Performance with Plpellnlng

The steady-state prediction behavior will mispred ict on the first and last loop iterations. Mispredicting the last itera tion is inevitable since the prediction bit will say taken: the branch ha s been taken nine times in a row at that point. The mispred iction on the first iteration happens because the bit is flipp ed on prior execution of the last iteration of the loop, since the branch was not taken on that exiting itera tion. Thus, the prediction accuracy for this branch that is taken 90% of the time is only 80% (two in correct predictions and eight correct ones). Idea lly, the accuracy of the predictor would match the taken branch frequency for these highly regular branches. To remedy this weakness, 2-bit prediction schemes are oft en used. In a 2-bit scheme, a prediction must be wrong twice before it is changed. Figure 6.39 shows the finit e state ma chine for a 2-bit prediction scheme. A branch prediction buffer can be implemented as a small, special buffer accessed with the instruction address during the IF pipe stage. If the instruction is predicted as taken, fetching begins from the target as soon as the PC is known; as mentioned on page 418, it ca n be as ea rly as the 10 stage. Otherwise, sequential

Taken Not taken

Prwdict taken Taken

T,keo

Not taken

Taken

FIGURE 6.39 The states In a 2-b1t prediction scheme. By using 2 bits rather than l,a branch that strongly favors taken or not taken-as many branches do---will be mispredicted only once. The 2 bits are U'ied to encode the four states in the system. The two-bit scheme is a general instance of a counter-based predictor, which is incremented when the prediction is accurate and decremented otherwise, and U'ies the midpoint of its range as the division between taken and not taken.

6.6

Branch Hazards

423

fetching and executing continue. If the prediction turns out to be wrong, the prediction bits are changed as shown in Figure 6.39. Elaboration: As we described in Section 6 .1, in a five-stage pipeline we can make the control hazard a feature by redefining the branch . A delayed branch always executes the following instruction, but the second instruction following the branch will be affected by the branch . Compilers and assemblers try to place an instruction that always executes after the branch in the branch delay slot. The job of the software is to make the successor instructions valid and useful. Figure 6.40 shows the three ways in which the branch delay slot can be scheduled . The limitations on delayed-branch scheduling arise from (1) the restrictions on the instructions that are scheduled into the delay slots and (2) our ability to predict at compile time whether a branch is likely to be taken or not. Delayed branching was a simple and effective solution for a five-stage pipeline issuing one instruction each clock cycle . As processors go to both longer pipelines and issuing multiple instructions per clock cycle (see Section 6 .9), the branch delay becomes longer and a single delay slot is insufficient. Hence, delayed branching has lost popularity compared to more expensive but more flexible dynamic approaches. Simultaneously, the growth in available transistors per chip has made dynamic prediction relatively cheaper.

bram:h d elay slo t The slot directly after a delayed branch instruction, which in the MIPS architecture is filled by an instruction that does not affect the branch.

Elaboration: A branch predictor tells us whether or not a branch is taken , but still requires the calculation of the branch target. In the five-stage pipeline, this calculation takes 1 cycle, meaning that taken branches will have a 1-cycle penalty. Delayed branches are one approach to eliminate that penalty. Another approach is to use a cache to hold the destination program counter or destination instruction, using a branch target buffer.

Elaboration: The 2-bit dynamic prediction scheme uses only information about a particular branch . Researchers noticed that using information about both a local branch and the global behavior of recently executed branches together yields greater prediction accuracy for the same number of prediction bits. Such predictors are called correlating predictors. A typical correlating predictor might have two 2-bit predictors for each branch with the choice between predictors made on the basis of whether the last executed branch was taken or not taken . Thus. the global branch behavior can be thought of as adding additional index bits for the prediction lookup. A more recent innovation in branch prediction is the use of tournament predictors . A tournament pred ictor uses multiple predictors, tracking, for each branch, which predictor yields the best results. A typical tournament predictor might contain two predictions for each branch index : one based on local information and one based on global branch behavior. A selector would choose which predictor to use for any given prediction . The selector can operate similarly to a 1- or 2-bit predictor favoring whichever of the two predictors has been more accurate. Many recent advanced microprocessors make use of such elaborate predictors .

bram:h t arget buffer A structure that caches the destination PC or destination instruction for a branch. It is usually organized as a cache with tags, making it more costly than a simple prediction buffer. correlatin g predktor A branch predictor that combines local behavior of a particular branch and global information about the behavior of some recent number of executed branches. tournamen t brandl pr ed ictor A branch predictor with multiple predictions for each branch and a selection mechanism that chooses which predictor to enable for a given branch.

424

Chapter 6

Enhancing Performance with Plpellnlng

a From before

b From target

c From fall through

add $s1, $s2, $s3

sub $t4 , $t5, $16 _

add $sl , $s2, $s3

if $s2 '" 0 then

...

if $s1 '" 0 then

a dd $s l , $s2, $s3

Delay slot

if$sl ",0 then -

I Becomes

Delay slot

Becomes

sub $t4, $t5, $t6 ___

Becomes

a dd $s l , $s2, $s3 if $s2 ",Othen -

Iadd $sl, $s2, Ss3 1

if$sl ",Othen a dd $s l , $s2, $s3

I sub $t4, $t5, $t6 I

if$sl ",0 then -

I sub $t4, $t5, $t6 I FIGURE 6.40 Scheduling the branch delay slot. The top box In each paIr shows the cooe before scheduling; the bottom box shows the scheduled code. In (a), the delay slot is scheduled with an independem instruction from before the branch. This is the best choice. Strategies (b ) and (c ) are used when (a) is not possible. In the code sequences for (b ) and (c), the use of S5 1 in the branch condition prevents the add instruction (whose destination is Ss 1) from being moved imo the branch delay slot. In (b) the branchdelay slot is scheduled from the target of the branch; usually the target instruction will need to be copied because it can be reached by another path. Strategy (b) is preferred when the branch is taken with high probability, such as a loop branch. Finally, the branch may be scheduled from the not-taken fall-through as in (c). To make this optimiwtion legal for (b) or (c), it mllSt be OK to execute the 5 ub instruction when the branch goes in the wlexpected direction. By "OK" we mean that the work is wasted, but the program will still execute correctly. This is the case, for eL1mple, if St4 were an unused temporary register when the branch goes in the unexpected direction.

Pipeline Summary Thus far, we have seen th ree models of execution: single cycle, multicycle, and pipelined. Pipelined control strives for 1 clock cycle per in st ruction, like single cycle, but also for a fast clock cycle, like multicycle. Let's revisit the exa mple comparison of single-cycle and multicycle processors.

6.6

425

Branch Hazards

Comparing Performance of Several Control Schemes

Compare performance for single-cycle, multicycle, and pipelined control using the SPECint2000 in struction mix (see exa mples on pages 3 15 and 330) and assuming the sa me cycle times per unit as the exa mple on page 3 15. For pipelined execution, assume that half of the load instructions are immed iately followed by an instruction that uses the result, that the branch delay on misprediction is 1 clock cycle, and that one-quarter of the branches are mispredicted. Assume that jumps always pay 1 full clock cycle of delay, so their average time is 2 clock cycles. Ignore any other hazards.

From the example on page 315 ( Performance of Single-Cycle Machines), we get the following fun ctional unit times: • 200 ps for memory access •

100 ps for ALU operation

• 50 ps for register file read or write For the single-cycle datapath, this leads to a clock cycle of 200+50+ 100+200+50 = 600ps The exa mple on page 330 (CPI in a Multicycle CPU) has the following in struction frequencies:

• 25% loads • 10% stores 11 % branches • 2% • jumps • 52% ALU instructions Furthermore, the exa mple on page 330 showed that the CPI for the multiple design was 4.1 2. The clock cycle for the multicycle datapath and the pipelined design must be the same as the longest fun ctional unit: 200 ps. For the pipelined design, loads take 1 clock cycle when there is no load-use dependence and 2 when there is. Hence, the average clock cycles per load in struction is 1.5. Stores take 1 clock cycle, as do the ALU in structions. Branches take 1 when predicted correctly and 2 when not, so the average clock cycles per branch instruction is 1. 25. The jump CPI is 2. Hence the average CPI is 1.5 X 25% + 1 X 10%+ 1 x52 % + 1.25 X 11 '/0 +2x2% = 1.1 7

EXAMPLE

ANSWER

426

Chapter 6

Enhancing Performance with Plpellnlng

Let's compare the th ree designs by the average instruction time. For the sin gle-cycle design , it is fixed at 600 ps. For the lTIulticycle design , it is 200 x 4.1 2 = 824 ps. For the pipelined design , the average instruction time is 1.1 7 x 200 = 234 ps, m aking it almost twice as fast as either approach. The clever reader will notice that the long cycle time of the memory is a performance bottleneck for both the pipelined and lTIulticycle designs. Breaking memo ry accesses into two clock cycles and thereby allowing the clock cycle to be 100 ps would improve the performance in both cases. We explore this in the exercises. This chapter sta rted in the laundry room , showing principles of pipelining in an everyday setting. Using that analogy as a guide, we explained instruction pipelining step -by-step, sta rting with the single-cycle data path and then adding pipeline registers, forwa rding path s, data haza rd detection, branch prediction, and flu shing instructions on exceptions. Figure 6.41 shows the fin al evolved datapath a nd control.

Check Yourself

Consider three bra nch prediction schemes: branch not taken , predict taken , and dynamic prediction. Assume that they all have zero penalty when they predict correctly and 2 cycles when they are wrong. Assume that the average predict accuracy of the dyn amic predictor is 90%. Which predictor is the best choice for the following bra nches? 1. A branch that is taken with 5% frequency 2. A branch th at is taken with 95% frequency 3. A branch that is taken with 70% frequency

Using a Hardware Description Language to Describe and Model a Pipeline This section , which appears on the CD, provides a behaviora l model in Verilog of the MIPS five-stage pipeline. The initial model igno res haza rds, a nd additions to the model highlight the changes fo r fo rwa rding, data haza rd s, and branch haza rds.

6.8

427

Exeeptlon s

,,~ d_;':'lon "_.

, "

ConlrOl

'co

•"

InstNcdo n m_~

•"

A IIogl_ ..

E;;EM r-

/ "

•

Er;EM

•" r-• ,ij";: • r• r-" •

•

•"

" ""

m. mory

•

•" •

Sign

•" •

FOWIIrdl ng

"".

FtGURE 6 .41

The fin al dat apath and eontro l for thi s ehapter.

Exceptions Another fo rm of cont rol hazard involves exceptions. Fo r example, suppose the following instruction add

$1,$2 , $1

has an arithmetic overflow. We need to transfer cont rol to the exception routine immediately after this instruction because we wouldn't wa nt this invalid value to contaminate other registers o r memo ry locations. Just as we did for the taken branch in the previous section, we must flu sh the instructions that follow the add instruction from the pipeline and begin fetching instructions from the new address. We will use the sa me mechanism we used for taken branches, but this time the exception causes the deasserting of cont rol lines. \-¥hen we dea lt with bran ch mispredict, we saw how to flush the in struction in the IF stage by turning it into a nop. To flu sh instructions in the 10 stage, we use the multiplexo r already in the 10 stage that zeros control signals for stalls. A new

To make a computer with automatic program-interruption facilities behave {sequentially} was not an easy matter, because the number of instructions in various stages ofprocessing when an interrupt signal occurs may be large. Fred Brooks Jr., Plmming a Comp uter System: Project Stretch, 1962

428

Chapter 6

Enhancing Performance with Plpellnlng

control signal, ca lled ID.Flush, is ORed with the stall signal from the Hazard Detection Unit to flu sh during ID. To flush the instruction in the EX phase, we use a new signal called EX. Flush to ca use new multiplexors to zero the control lines. To start fetching instructions from location 8000 0 180hex , which is the exception location for an arithmetic overflow, we simply add an additional input to the PC multiplexor that sends 8000 0 180hex to the Pc. Figure 6.42 shows these changes. This exa mple points out a problem with exceptions: If we do not stop execu tion in the middle of the instruction, the programmer will not be able to see the original va lue of register $1 that helped cause the overfl ow because it will be clobbered as the destination register of the add instruction. Because of careful planning, the overfl ow exception is detected during the EX stage; hence, we ca n use the EX. Flush signa l to prevent the instruction in the EX stage from writing its result in the WB stage. Many exceptions require that we evennlally complete the in struction that caused the exception as if it executed normally. The easiest way to do this

--.", Hoard

IDIEX

•f-+-,

r- ~

"'"r-r---" ....\J

, '0

-·I ~H--I~ , f- V

,""".

."h'-F ' ~ I-r----,LM~B L-I~ WB f----• '" r-

•,"

Instruction memory

• - :1----+--11/

~" memory

f-H. ,"

Fawardlng unit

FIGURE 6.42 The datapath with controls to handle exceptions. The key additions mclude a new mput, with the value 8<XXl 0180 bu , In the multiplexor that supplies the new PC value; a Cause register to record the cause of the exception; and an Exception PC register to s.we the address of the instruction that caused the exception. The 8000 018Ot..x input to the multiplexor is the initial address to begin fetching instructions in the event of an exception. Although not shown, the ALU overflow signal is an input to the control unit.

6.8

429

Exception s

is to flu sh the in struction and restart it from the beginning after the exception is handled. The final step is to save the address of the offending instruction in the Excep tion Program Counter ( EPC), as we did in Chapter 5. In rea lity, we save the address + 4, so the exception handling routine must first subtract 4 from the saved value. Figure 6.42 shows a stylized version of the datapath , including the branch ha rdwa re and necessa ry accommodations to handle exceptions.

Exception in a Pipelined Computer

Given this instruction sequence, 4 0 h.. 44 h.. 4 8 h.. 4 Ch.. 50 h.. 54 h..

sub and

or add

sIt

III . 12 . 14 112 . 12 . IS

EXAMPLE

113 . 12 . 16 II . 12 . $I liS . 16 . $7 116 . 50($7)

assume the instructions to be invoked on an exception begin like this: 4 0000040 h.. 4 000004 4 h..

sw sw

125 . 1000( 10) 126 . 1004( 10)

Show what happens in the pipeline if an overflow exception occurs in the add in struction.

Figure 6.43 shows the events, starting with the add instruction in the EX stage. The overflow is detected during that phase, and 4000 0040 hex is forced into the Pc. Clock cycle 7 shows that the add and following instructions a re flu shed, and the first instruction of the exception code is fetched. Note that the address of the instruction followillg the add is saved: 4Chex + 4 = SOhex. Chapter 5 lists some other causes of exceptions: • I/O device request • Invoking an operating system service from a user progra m • Using an undefin ed instruction • Hardwa re malfunction

ANSWER

430

Chapter 6

Enhancing Pe rform ance with Plpellnlng

add ","', "

Iw " ~, 00(' 1)

., ,

•

"l"

•

ow $:'5, '000($01

.,,

'1"

"•

•

.--

or ' 13 ,

, ~~

"• 0

•

"• 0

•• •

..........

FIGURE 6 .43 The result of an exception due to arithm etic overflow In th e add Instructi on. The overflow is detected during the EX stage of clock 6, saving the address following the add in the EPC register (4C + 4 = SObu) ' Overflow causes aU the Flush signals to be set near the end of this clock cycle, deasserting control values (setting them to 0) for the add. Clock cycle 7 shows the instructions converted to bubbles in the pipeline plU'i the fetching of the first instruction of the exception routine---sw $ 25 .1 OOO{ $0 )- from instruction location 4000 0040!> more read ports and one more write port on the register file , and another ALU. Assume the bottom ALU handles address calculations for data transfers and the top ALU handles everything else.

438

Chapter 6

Enhancing Pe rform ance with Plpellnlng

Clea rly, this two- issue processor ca n improve performance by up to a factor of 2. Doing so, however, requires that twice as many instructions be overlapped in execution, and this additional overlap increases the relative performance loss from data and control hazards. For exa mple, in our simple five-stage pipeline, loads have a use latency of 1 clock cycle, which prevents one instruction from using the result without stalling. In the two- issue, five-stage pipeline, the result of a load instruction ca nn ot be used on the next clock cycle. This means that the next two instructions ca nnot use the load result without stalling. Furthermore, ALU instructions that had no use latency in the simple five-stage pipeline, now have a one- in struction use latency, since the results ca nnot be used in the paired load or store. To effectively exploit the parallelism available in a multiple- issue processor, more ambitious compiler or hardware scheduling techniques are needed, and static multiple issue requires that the compiler takes on this role.

Simple Multiple-l s5ue Code Scheduling

How would this loop be sched uled on a static two-issue pipeline for MIPS?

EXAMPLE

Loop :

lw addu

sw addi bne

ItO . O($sl)

II $t0 , HO ,$ s2 II ItO . O($sl) II $sl , $sl , - 4 II $sl , $zero,Loop II

$tO= array element add scalar in $s2 store result decrement pointer branch $sl !=O

Reorder the instructions to avoid as many pipeline stalls as possible. Assume branches are predicted, so that control hazards are handled by the hardware.

ANSWER

lo op unrolling A technique to get more performance from loops that access arrays, in which multiple copies of the loop body are made and instructions from different iterations are scheduled together.

The first three in structions have data dependences, and so do the last two. Figure 6.46 shows the best schedule for these instructions. Notice that just one pair of in structions has both issue slots used. It takes 4 clocks per loop iteration ; at 4 clocks to execute 5 instructions, we get the disappointing CP I of 0.8 versus the best case of 0.5., or an IPC of 1. 25 versus 2.0. Notice that in computing CPI or IPC, we do not count any nops executed as useful instructions. Doing so would improve CPI, but not performance! An important compiler technique to get more performance from loops is loop unrolling, a technique where multiple copies of the loop body are made. Aft er unrolling, there is more ILP available by overlapping instructions from different iterations.

6.9

439

Advanced Plpellnlng: Extracting More Performance

ALU or branch Instruction

loop: ~ddi ~ddLl

$51. $51. - 4 $tO.StO.$s2 $sl. $zero.loop

Data transfer Instruction

StD. O{ $sl)

Clock cycle 1

2 3

StD. 4 {$sl)

FtGURE 6.46 The scheduled code as It would look on a two-lssue MIPS pipeline. The empty slots are nops.

Loop Unrolling for Multiple-lssue Pipelines

See how well loop unrolling and scheduling work in the example above. Assume that the loop index is a multiple of four, for simplicity.

To schedule the loop without any delays, it niril S out that we need to make four cop ies of the loop body. Aft er unrolling and eliminating the unnecessa ry loop overhead instructions, the loop will contain four copies each of 1W, add, and SW, plus one addi and one bne. Figure 6.47 shows the unrolled and scheduled code. During the unrolling process, the compiler introduced additional registers ( $ t 1, $t2, $ t3 ). The goal of this process, ca lled register renaming, is to eliminate dependences that are not true data dependences, but could either lead to potential haza rd s or prevent the compiler from fl exibly sched uling the code. Consider how the unrolled code would look using only $tO. There would be repea ted instances of 1W $tO . 0 ( $$ s1 ), addLl $t O. $tO . $ s 2 followed by sw to. 4 ($ s 1) , but these sequences, despite using $tO, are actually completely independent- no data values fl ow between one pair of these instructions and the next pair. This is what is ca lled an antidependence or name dependence, which is an ordering forced purely by the reuse of a name, rather than a real data dependence. Renaming the registers during the unrolling process allows the compiler to subsequently move these independent instructions so as to better schedule the code. The renaming process eliminates the name dependences, while preserving the true dependences. Notice now that 12 of the 14 instructions in the loop execute as a pair. It takes 8 clocks for four loop iterations, or 2 clocks per iteration, which yields a CPI of 8/14 = 0.57. Loop unrolling and scheduling with dual issue gave us a factor of two improvement, partly from reducing the loop control in structions and partly from dual issue execution. The cost of this performance improvement is using four temporary registers rather than one, as well as a significa nt in crease in code size.

EXAMPLE

ANSWER

antidependence Also called name dependence. An ordering forced by the reuse of a name, typically a register, rather then by a true dependence that carries a value between two

440

Chapter 6

Enhancing Performance with Plpellnlng

ALU or branch Instruction

Loo p :

addi add u add u add u add u

boe

$ s l , $s l. 16 St O.StO. Ss2 Stl , $t1 . Ss2 $t 2.$t2. $s2 St3.St 3 . Ss2 $sl. $ze r o.Lo op

Data transfer Instruction

Clock cycle

St O, 0 ($ s 1)

$ t l, 1 2 {$ s l)

1. 1.

$t 2. 8 ($ s1 ) $t3. 4 ( Ss 1) $tO. 16 {$ sl) $tl. 12 ( Ss1 ) $ t 2• B{S s l) St3. 4 ($ 51)

'" '" '" '"

4 5 6 7 8

FIGURE 6.47 The unrolled and scheduled code of Figure 6.46 as It would look on a static two-lssue MIPS pipeline. The empty slots are naps. Since the first instruction in the loop decrements $ s 1 by 16, the addresses loaded are the original value of $ s 1, then that address minus 4, minus 8, and minus 12 .

The Intel IA-64 Architecture The IA-64 architecnlfe is a register-register, RI SC-style instruction set like the 64bit version of the MIPS architecture (called MIPS-64), but with several unique features to support explicit , compiler-driven exploitation of ILP. Intel ca lls the approach EPI C (Explicitly Parallel Instruction Computer). The major differences between IA-64 and the M IPS a rchitecture are the following: I. IA-64 has many more registers than MIPS, including 128 integer and 128

fl oating-point registers, as well as 8 special registers for branches and 64 1bit condition registers. In addition, IA-64 supports register windows in a fa shion similar to the original Berkeley RI SC a nd Sun SPARC architectures. 2. IA-64 places instructions into bundles that have a fi xed format and explicit designation of dependences. 3. IA-64 includes specia l instructions and capabilities for speculation and for bra nch elimination , which increase the a mount of ILP that ca n be exploited.

inst r uction group In IA-64, a sequen ce of consecutive instructions with no register data dependen ces among them. stop In IA-64, an explicit indicator of a break between independent and dependent instructions.

The IA-64 architecnlfe is designed to achieve the major benefit s of a V LI Wim plicit pa rallelism among operations in an instruction and fixed formatting of the o peration fields-while maintaining greater flexibility th an a V LI W normally allows. The IA-64 a rchitecture uses two different concepts to achieve this fl exibility: instruction groups and bundles. An instruction group is a sequence of consecutive instructions with no register data dependences among them. All the instructions in a group could be executed in pa rallel if sufficient ha rdwa re resources existed and if any dependences through memo ry were preser ved. An instruction group ca n be arbitrarily long, but the compiler must explicitly indicate the bounda ry between one instruction group and another. This bounda ry is indicated by placing a stop between two in structions th at belong to different groups.

6.9

Advanced Plpellnlng: Extracting More Pe rform ance

IA-64 in structions are encoded in bundles, which are 128 bits wide. Each bundle consists of a 5-bit template field and three instructions, each 41 bits in length. To simplify the decoding and instruction issue process, the template fi eld of a bundle specifies which of five different execution units each instruction in the bundle requires. The five different execution units are integer ALU, noninteger ALU (includes shifters and multimedia operations), memory unit, fl oating-point unit, and branch unit. The 5-bit template fi eld within each bundle describes both the presence of any stops associated with the bundle and the execution unit type required by each instruction within the bundle. The bundle format s ca n specify only a subset of all possible combinations of instruction types and stops. To enhance the amount of ILP that ca n be exploited, IA-64 provides extensive support for predication and for speculation (see the Elaboration on page 442). Predication is a technique that ca n be used to eliminate branches by making the execution of an instruction dependent on a predicate, rather than dependent on a branch. As we saw ea rlier, branches reduce the opportunity to exploit ILP by restricting the movement of code. Loop unrolling works well to eliminate loop branches, but a branch within a loop-a rising, for example, from an if-then-else statement-cannot be eliminated by loop unrolling. Predication, however, provides a method to eliminate the branch, allowing more fl exible exploitation of parallelism. For example, suppose we had a code sequence like if (p) {s tatement l} else (statement 2)

Using normal compilation methods, this segment would compile using two branches: one aft er the condition branching to the else portion and one after statement 1 branching to the next sequential statement. With predication, it could be compiled as

statement 1 (-p) statement 2

(p)

where the use of (cond i t i on) indicates that the statement is executed only if co nd i t i on is true, and otherwise becomes a no-op. Notice that predication can be used as way to speculate, as well as a method to eliminate branches. The IA-64 architecture provides comprehensive support for predication: nea rly every instruction in the IA-64 architecture ca n be predicated by specifying a predicate register, whose identity is placed in the lower 6 bits of an instruction fi eld. One consequence of full predication is that a conditional branch is simply a branch with a guarding predicate! IA-64 is the most sophisticated exa mple of an instruction set with support for compiler-based exploitation of ILP. Intel's Itanium and Itanium 2 processors implement this architecture. A brief summa ry of the cha racteristics of these processors is given in Figure 6.48.

441

predication A technique to make instructions dependent on predicates rather than on branches.

442

Chapter 6

Processor

Maximum Instr. Issues / clock

ltanium

Enhancing Performance with Plpellnlng

Functional units

Maximum Max. clock ops. per clock rate

Transistors (millions)

4 integer/ media

0 .8 GHz

1 .5 Ghz

221

III• • 130

379

701

130

810

1427

2 memory 3 branch

2FP ltanium 2

6 integer/ media

4 memory 3 branch

2FP

FIGURE 6.48 A summary of the characteristics of the Itanlum and ttanlum 2, Intel's first two Implementations of the IA·64 architecture. In addition to higher dock rates and more functional units, the Itanium 2 includes an on-chip level 3 cache, versus an off-chip level 3 cache in the ltanium.

poiso n A result generated when a speculative load yields an exception, or an instruction uses a poisoned operand. advall(:ed load In IA-64, a speculative load instruction with support to check for aliases that could invalidate the load.

Elaboration: Speculation support in the IA-64 architecture consists of separate sup. port for control speculation, which deals with deferring exceptions for speculated instructions, and memory reference speculation, which supports speculation of load instructions. Deferred exception handling is supported by adding speculative load instructions, which, when an exception occurs, tag the result as poison. When a poisoned result is used by an instruction, the result is also poison . The software can then check for a poisoned result when it knows that the execution is no longer speculative . In lA-54, we can also speculate on memory references by moving loads earlier than stores on which they may depend . This is done with an advanced load instruction . An advanced load executes normally, but uses a special table to track the address that the processor loaded from . All subsequent stores check that table and generate a flag in the entry if the store address matches the load address. A subsequent instruction must be used to check the status of the entry after the load is no longer speculative . If a store to the same address has intervened, the check instruction specifies a fix-up routine that reexecutes the load and any other dependent instructions before continuing execution; if no such store has occurred, the table entry is simply cleared, indicating that the load is no longer speculative.

Dynamic Multiple-Issue Processors superscalar An advanced pipelining technique that enables the processor to execute more than one instruction per clock cycle.

Dynamic multiple-issue processors are also known as su perscalar processors, or simply superscalars. In the simplest superscalar processors, instructions issue inorder, and the processor decides whether zero, one, or more instructions can issue in a given clock cycle. Obviously, achieving good performance on such a processor still requires the compiler to try to schedule instructions to move dependences apart and thereby improve the instruction issue rate. Even with such compiler scheduling, there is an important difference between this simple superscalar and a VLIW processor: the code, whether scheduled or not, is guaranteed by the hardware to execute correctly. Furthermore, compiled code will always run correctly

6.9

Advanced Plpellnlng: Extracting More Performance

independent of the issue rate or pipeline structure of the processor. In some VLIW designs, this has no t been the case, and recompilatio n was required when m oving across different processor models; in other static issue processors, code would run correctly across different implem entations, but often so poorly as to make compilation effectively required. Many superscalars extend the basic fram ework of dynamic issue decisions to include dynamic pipeline scheduling. Dynamic pipeline scheduling chooses which instructions to execute in a given clock cycle while trying to avoid hazards and stalls. Let's start with a simple exa mple of avoiding a data hazard. Consider the following code sequence:

lw add u sub slti

It o . Itl, $s4 , $tS ,

443

dynamk pipeline 5':heduling Hardware support for reordering the order of instruction execution so as to avoid stalls.

20($521 It O. 1t2 $s 4 , $t3 $s 4, 20

Even tho ugh the sub instructio n is ready to execute, it must wa it for the lw and a ddu to complete first, which might take many clock cycles if m emory is slow. (Chapter 7 expla ins caches, the reason that mem ory accesses are som etimes very slow. ) Dynamic pipeline scheduling allows such hazards to be avoided either fully or partially. Dynamic Pipeline Scheduling Dynamic pipeline scheduling chooses which instructio ns to execute next, possibly reordering them to avoid stalls. In such processors, the pipeline is divided into three m ajor units: an in structio n fetch and issue unit, multiple functional units ( 10 o r m o re in high -end designs in 2004), and a commit unit. Figure 6.49 shows the model. The first unit fetches instructio ns, decodes them, and sends each instructio n to a corresponding functional unit for execution. Each functional unit has buffers , called reservation stations, that hold the opera nds and the opera tio n. (I n the next section , we will discuss an alternative to reservatio n statio ns used by many recent processors. ) As soon as the buffer contain s all its operands and the fun ctio nal unit is ready to execute, the result is calculated. \-¥hen the result is com pleted, it is sent to any reservation stations waiting for this particular result as well as to the commit unit, which buffers the result until it is safe to put the result into the register file or, for a store, into m em ory. The buffer in the commit unit, oft en called the reorder buffer, is also used to supply opera nds, in much the sam e way as forwarding logic does in a statically sched uled pipeline. Once a result is com mitted to the register fil e, it ca n be fetched directly from there, just as in a normal pipeline. The combinatio n of buffering opera nds in the reserva tio n stations and results in the reorder buffer provides a form of register renaming, just like that used by

commit unit The unit in a dynamic or out-of-order execution pipeline that decides when it is safe to release the result of an operation to programmer-visible registers and memory. reservation station A buffer within a functional unit that holds the operands and the operation. reorder buffer The buffer that holds results in a dynamically scheduled processor until it is safe to store the results to memory or a register.

444

Chapter 6

Enhancing Performance with Plpellnlng

Instruction fetch

In-order issue

and deccx:le unit

Reservati on station

Integer

Functional units

...

Co mmit

Reservation station

Reservation

Floating

Load!

point

Store

station

Out-of-order execute

In-order commit

unit FIGURE 6.49 The three primary units of a dynamically scheduled pipeline. The final step of upda ting the state is also ca Ued retirement or graduatio n.

the compiler in our earlier loop unrolling example on page 439. To see how this conceptually works, consider the following steps: I. Wh en an instruction issues, if either of its operands is in the register file or the reorder buffer, it is copied to the reservation station immediately, where it is buffered until all the operands and an execution unit are available. For the issuing instruction , the register copy of the operand is no longer required , and if a write to that register occurred, the va lue could be overwritten.

2. If an operand is not in the register fil e o r reorder buffer, it must be wa iting to be p roduced by a fun ctional unit. The name of the fun ctional unit that will produce the result is tracked. When that unit eventually produces the result, it is copied directly into the waiting reservation station from the fun ctional unit bypassing the registers. These steps effectively use the reo rder buffer and the reserva tion stations to implement register renaming.

6.9

Advanced Plpellnlng: Extracting More Performance

Conceptually, you can think of a dynamically scheduled pipeline as analyzing the dataflow strucnlfe of a program, as we saw when we discussed dataflow analysis within a compiler in Chapter 2. The processor then executes the instructions in some order that preserves the data flow order of the program. To make programs behave as if they were running on a simple in-order pipeline, the instruction fetch and decode unit is required to issue instructions in order, which allows dependences to be tracked, and the commit unit is required to write results to registers and memory in program execution order. This conservative mode is called inorder completion. Hence, if an exception occurs, the computer can point to the last instruction executed, and the only registers updated will be those written by instructions before the instruction causing the exception. Although, the front end (fetch and issue) and the back end (commit) of the pipeline run in order, the functional units are free to initiate execution whenever the data they need is available. Today, all dynamically scheduled pipelines use in-order completion, although this was not always true. Dynamic scheduling is often extended by including hardware-based speculation , especially for branch outcomes. By predicting the direction of a branch , a dynamically scheduled processor can continue to fetch and execute instructions along the predicted path. Because the instructions are committed in order, we know whether or not the branch was correctly predicted before any instructions from the predicted path are committed. A speculative, dynamically scheduled pipeline can also support speculation on load addresses, allowing load-store reordering, and using the commit unit to avoid incorrect speculation. In the next section we will look at the use of dynamic scheduling with speculation in the Pentium 4 design.

445

in-order commit A commit in which the results of pipelined execution are written to the programmer-visible state in the same order that instructions are fetched.

Elaboration: A commit unit controls updates to the register file and memory. Some dynamically scheduled processors update the register file immediately during execution using extra registers to implement the renaming function and preserving the older copy of a register until the instruction updating the register is no longer speculative . Other processors buffer the result, typically in a structure called a reorder buffer, and the actual update to the register file occurs later as part of the commit. Stores to memory must be buffered until commit time either in a store buffer (see Chapter 7) or in the reorder buffer. The commit unit allows the store to write to memory from the buffer when the buffer has a valid address and valid data, and when the store is no longer dependent on predicted branches .

Elaboration: Memory accesses benefit from nonblocking caches, which continue servicing cache accesses during a cache miss (see Chapter 7) . Out-of-ord er execution processors need non blocking caches to allow instructions to execute during a miss.

out-of-o rd er execution A situation in pipelined execution when an instruction blocked from executing does not cause the following instructions to wait.

446

Chapter 6

Hardware Software Interface

BIG

The Picture

Enhancing Performance with Plpellnlng

Given that compilers ca n also schedule code around data dependences, you might ask, Why would a supersca lar p rocesso r use dynamic scheduling? There are three major reasons. First, not all stalls are pred ictable. In part icular, cache misses (see Chapter 7) cause unpredictable stalls. Dynamic scheduling allows the p rocesso r to hide some of th ose stalls by continuing to execute instructions while wa iting for the stall to end. Second , if the processor speculates on branch outcomes using dynamic branch prediction , it ca nn ot know the exact order of in struct ions at compile time, since it depends on the predicted and actual behavior of branches. Incorpo rating dynam ic speculation to exploit mo re l LP without incorporating dyna mic sched uling would significa ntly restrict the benefit s of such speculation. Third, as the pipeline latency and issue width change from one implementation to a nother, the best way to compile a code sequence also cha nges. For example, how to schedule a sequence of dependent instructions is affected by both issue width and latency. The pipeline structure affects both the number of times a loop must be un rolled to avoid stalls as well as the process of compiler-based register renam ing. Dynamic scheduling allows the ha rdwa re to hide most of these details. Thus, users and softwa re distributors do not need to worry about having multiple versions of a program fo r different implementations of the sa me instruction set. Sim ilarly, old legacy code will get much of the benefit of a new implementation without the need for recompilation.

Both pipelining and multiple- issue execution increase peak instruction throughput and attempt to exploit ILP. Data and control dependences in programs, however, offer an upper limit on sustained performance because the processor must sometimes wait for a dependence to be resolved. Software-centric approaches to exploiting ILP rely on the ability of the compiler to find and reduce the effects of such dependences, while ha rdware-centric approaches rely on extensions to the pipeline and issue mechanisms. Speculation, performed by the compiler or the hardware, can increase the amount of ILP that can be exploited , although care must be taken since speculating incorrectly is likely to reduce performan ce.

6.9

447

Advanced Plpellnlng: Extracting More Performance

Modern , high-performance microprocessors are capable of issuing several instructions per clock; unfortunately, sustaining that issue rate is ver y difficult. For exa mple, despite the existence of processors with four to six issues per clock, very few applications can sustain more than two instructions per clock. There are two primary reasons for this. First, within the pipeline, the major performance bottlenecks arise from dependences that ca nnot be alleviated, thus reducing the pa rallelism among instructions and the sustained issue rate. Alth ough little ca n be done about true data dependences, often the compiler or hardwa re does not know precisely whether a dependence exists o r not, and so must conserva tively assume the dependence exists. For example, code that makes use of pointers, particularly in ways that create more aliasing, will lead to more implied potential dependences. In contrast, the grea ter regularity of array accesses oft en allows a compiler to deduce that no dependences exist. Similarly, branches th at ca nnot be accurately predicted whether at runtime o r compile time will limit the ability to exploit ILP. Oft en additional ILP is ava ilable, but the ability of the compiler o r the hardware to find ILP that may be widely separated (sometimes by the execution of thousa nds of instructions) is limited. Second , losses in the memo ry system (the topic of Chapter 7) also limit the ability to keep the pipeline full. Some memory system stalls ca n be hidden, but limited amounts of ILP also limit the extent to which such stalls ca n be hidden.

Understanding Program Performance

State whether the following techniques or components are associated primarily with a softwa re- or hardwa re-based app roach to exploiting ILP. In so me cases, the answer may be both.

Check Yourself

I. Branch prediction

2. Multiple issue 3. VLI W 4. Supersca lar 5. Dynamic scheduling 6. Out -of-order execution 7. Speculation 8. EPIC

9. Reo rder buffer 10. Register renaming II. Predica tion

448

Chapter 6

Enhancing Performance with Plpellnlng

Real Stuff: The Pentium 4 Pipeline

microarchitecture The organization of the processor, including the major functional units, their interconnection, and control.

In the last chapter, we discussed how the Pentium 4 fetched and translated IA- 32 instructions into microoperations. The microoperations are then executed by a sophisticated , dynamically scheduled, speculative pipeline ca pable of sustaining a tl execution rate of three microoperations per clock cycle. This section focuses on that microoperation pipeline. The Pentium 4 combines multiple issue with deep pipelining so as to achieve both a low CPl and a high clock rate. When we consider the design of sophisticated , dynamica lly scheduled processors, the design of the fun ctional units, the cache and register file, instruction issue, and overall pipeline control become intermingled, making it difficult to sepa rate out the datapath fro m the pipeline. Because of this, many engineers and resea rchers have adopted the term microarchitecture to refer to the detailed internal architecture of a processo r. Figure 6.50 shows the microa rchitecture of the Pentium 4, focusing on the strucnIres fo r executing the microoperations. Another way to look at the Pentium 4 is to see the pipeline stages that a typica l instruction goes th rough. Figu re 6.5 1 shows the pipeline structu re and the typica l number of clock cycles spent in each; of course, the number of clock cycles va ries due to the nature of dynamic scheduling as well as the requirements of individual microoperations. The Pentium 4, and the ea rlier Pentium III and Pentium Pro, all use the tech nique of decoding IA-32 in structions into microoperations and executing those microoperations using a speculative pipeline with multiple functional units. In fact, the basic microa rchitecture is similar, and all these p rocessors can complete up to three microoperations per cycle. The Pentium 4 gains its performance adva ntage over the Pentium III th rough several enhancements:

1. A pipeline that is roughly twice as deep (approximately 20 cycles versus 10) and ca n run almost twice as fast in the sa me technology 2. More fun ctional units (7 versus 5) 3. Support for a larger number of outstanding operations (126 versus 40) 4. The use of a trace cache (see Chapter 7) and a much better branch predictor (4K entries versus 5 12) 5. Other enhancements to the memory system , which we discuss in Chapter 7 architectural registers The instruction set visible registers of a processor; for example, in MIPS, these are the 32 integer and 16 floating-point registers.

Elaboration: The Pent ium 4 uses a scheme fo r resolving ant idependences and incorrect speculation t hat uses a reorder buffer together wit h register renaming. Register renaming explicit ly renames t he architectural registers in a processor (8 in t he case of IA-3 2) t o a larger set of physical registers (128 in t he Pentiu m 4 ). The Penti um 4 uses

6.10

•••

Real Stuff: Th e Pentium 4 Pipeline

Instruction prefetch and decode

Branch prediction

Trace cache

Microoperation queue Register file

Dispatch and register remaining

Integer and floating-point operation queue

I Complex instruction

Memory operation queue

Integer

I Floating point

Integer

Load

I Store

I Commit unit Data cache FtGURE 6 .50 Th e mlcroarchltecture of th e Inte l Pentium 4. The extensive queues allow up to 126 microoperations to be outstanding at any point in time, including 48 loads and 24 stores. There are actually seven functional units, since the FP unit includes a separate dedicated lUlit for floating-point moves. The load and store units are actually separated into two p.uts, with the first part handling address calculation and the second part responsible for the actual memory reference. The integer ALUs operate at twice the clock frequency, allowing two integer ALU operations to be completed by each of the two imeger units in a single clock cycle. As we described in Chapter 5, the Pentium 4 uses a special cache, called the trace cache, to hold predecoded sequences of microoperations, corresponding to IA-32 instructions. The operation of a trace cache is explained in more detail in Chapter 7. The FP unit also handles the MMX multimedia and SSE2 instructions. There is an extensive bypass network among the functional units; since the pipeline is dynamic rather than static, bypassing is done by tagging results and tracking source operands, so as to allow a match when a result is produced for an instruction in one of the queues that net'ds the result. Intel is expected to release new versions of the Pentium 4 in 2004, which will probably have changes in the microorchitecture.

~r='-~~::~B-I rg =rn- ,':;;~' =rn- ~ --•

Number 01 dockcydas

•

,~~

~-I=ill,

FtGURE 6 .51 Th e Pentium 4 pipeline showing th e pipeline flow for a typi cal Instructi on and th e numbe r of clock cycles for the maj or steps In th e pipeline. The major buffers where instructions wait are also shown.

450

Chapter 6

Enhancing Performance with Plpellnlng

register rena ming t o remove antidependences . Register renaming requires t he processor to maintain a map between the architectural registers and the physica l registers , indicating which physica l register is the most current copy of an architectural regist er. By keeping t rack of the renamings that have occurred , register renaming offers another approach to recovery in the event of incorrect speculati on: simply undo the mappings that have occurred s ince the first incorrectly speculat ed instructi on. This will ca use the stat e of the processor t o return t o the last correctl y executed instructi on, keeping the correct mapping between the architectural and phys ica l regist ers .

Understanding Program Performance

The Pentium 4 combines a deep pipeline (averaging 20 or mo re pipe stages per instruction) and aggressive multiple issue to achieve high perfo rmance. By keep ing the latencies for back-to-back operations low (0 for ALU operations and 2 for loads), the impact of data dependences is reduced. What are the most serious potential perfo rmance bottlenecks fo r programs running on this processo r? The following list includes some potential performance problems, the last three of which ca n apply in some fo rm to any high-perfo rmance pipelined processo r. • The use of IA-32 instructions that do not map to three o r fewer simple microoperations • Branches that are difficult to predict, causing misprediction stalls and restarts when speculation fails • Poor instruction loca lity, which causes the trace cache not to fun ction effectively • Long dependences-typica lly caused by long-running instructions o r data cache misses-which lead to stalls Performance delays arising in accessing memory (see Chapter 7) that cause the processor to stall

Check Yourself

Are the following statements true o r false? 1. The Pentium 4 ca n issue mo re instructions per clock then the Pentium III . 2. The Pentium 4 multiple-issue pipeline directly executes IA-32 instructions. 3. The Pentium 4 uses dynamic scheduling but no speculation. 4. The Pentium 4 microa rchitecture has many more registers than IA-32 reqUIres. 5. The Pentium 4 pipeline has fewer stages than the Pentium II I. 6. The trace cache in the Pentium 4 is exactly the sa me as an instruction cache.

6.11

Fallacies and Pitfalls

Fallacies and Pitfalls Fallacy: Pipelining is easy.

Our books testify to the subtlety of correct pipeline execution. Our advanced book had a pipeline bug in its first edition , despite its being reviewed by more than 100 people and being class-tested at 18 universities. The bug was uncovered only when someone tried to build the computer in that book. The fa ct that the Verilog to describe a pipeline like that in the Pentium 4 will be thousands of lines is an indication of the complexity. Beware! Fallacy: Pipelining ideas can be implemented independent of technology.

When the number of transistors on -chip and speed of transistors made a fi vestage pipeline the best solution, then the delayed branch (see the Elaboration on page 423) was a simple solution to control hazards. With longer pipelines, superscalar execution , and dynamic branch prediction, it is now red undant. In the ea rly I990s, dynamic pipeline scheduling took too many resources and was not required for high performance, but as transistor budgets continued to double and logic became much faster than memory, then multiple functional units and dynamic pipelining made more sense. Today, all high-end processors use multiple issue, and most choose to implement aggressive speculation as well. Pitfall: Failure to consider instruction set design can adversely impact pipelining.

Many of the difficulties of pipelining arise because of instruction set complications. Here are some exa mples: • Widely variable instruction lengths and running times ca n lead to imbalance among pipeline stages and severely complica te hazard detection in a design pipelined at the instruction set level. This problem was overcome, initially in the DEC VAX 8500 in the late 198Ds, using the micropipelined scheme that the Pentium 4 employs today. Of course, the overhead of translation and maintaining correspondence between the microopera tions and the actual instructions remains. • Sophistica ted addressing modes ca n lead to different sorts of problems. Addressing m odes that update registers, such as update addressing (see Chapter 3), complicate hazard detection. Other addressing modes that require multiple memory accesses substantially complica te pipeline control and make it difficult to keep the pipeline fl owing smoothly. Perhaps the best exa mple is the DEC Alpha and the DEC NVAX. In comparable techn ology, the newer instruction set architecture of the Alpha allowed an implementation whose performance is more than twice as fast as NVAX. In another

451

452

Chapter 6

Enhancing Performance with Plpellnlng

exa mple, Bhandarkar and Clark [199 1) compared the MIPS M /2000 and the DEC VAX 8700 by counting clock cycles of the SPEC benchmarks; they concluded that, although the M IPS M/2000 executes mo re in structio ns, the VAX o n average executes 2.7 times as many clock cycles, so the MIPS is faster.

Nine-tenths of wisdom consists ofbeing wise ill time. American proverb

instruction latency The inherent execution time for an instruction.

Concluding Remarks Pipelining improves the average executio n time per instructio n. Depending on whether you start with a single-cycle o r multiple-cycle datapath, this reduction ca n be thought of as decreasing the clock cycle time or as decreasing the number of clock cycles per instruction (CPl). We started with the simple single -cycle data path, so pipelining was presented as reducing the clock cycle time of the simple datapath. Multiple issue, in comparison, clearly focuses on reducing CPI (or increasing IPC). Figure 6.52 shows the effect on CPI and clock rate for each of the microa rchitectures from Chapters 5 and 6. Performan ce is increased by m oving up and to the right, since it is the product of IPC and clock rate that determines performance for a given in struction set. Pipelining improves throughput, but not the inherent execution time, or latency, of in structions; the latency is similar in length to the multicycle approach. Unlike that approach, which uses the same hardwa re repeatedly during in struction execution, pipelining starts an instruction every clock cycle by having dedicated hardwa re. Similarly, multiple issue adds additional data path hardware to allow multiple instructions to begin every clock cycle, but at an increase in effective latency. Figure 6.53 shows the datapaths from Figure 6.52 placed according to the amount of sharing of hardwa re and instru ction lat en cy. Pipelining and multiple issue both attempt to exploit instruction-level parallelism. The presence of data and control dependences, which can become hazards, are the primary limitations on how much parallelism ca n be exploited. Scheduling and speculation, both in hardwa re and softwa re, a re the primary techniques used to red uce the performance impa ct of dependences. The switch to longer pipelines, multiple instruction issue, and dynamic scheduling in the mid- 1990s has helped sustain the 60% per yea r processor performan ce increase that we have benefited from since the ea rly 1980s. In the past , it appeared that the choice was between the highest clock rate processors and the m ost sophisticated superscalar processors. As we have seen, the Pentium 4 com bines both and achieves remarkable performance.

6.12

453

Concluding Remarks

Multiple issue with deep pipeline (Section 6.1 0)

Deeply pipelined

Multicycle (Section 5.5)

-••

Multiple-issue pipelined (Section 6 .9)

Pipelined

" ilo u

Single-eycle (Section 5.4)

Slower

Faster

Instructions per clock (IPC = 1/CPI) FtGURE 6.52 The performance consequences of simple (slngl&eycle) datapath and mul· tlcycle datapath from Chapter 5 and the plpellned execution model In Chapter 6. Remember that CPU performance is a nUlction of IPC times dock rate, and hence moving to the upper right increases performance. Although the instructions per dock cyde is slightly larger in the simple datapath, the pipelined datapath is dose, and it uses a dock rate as fast as the multicyde datapath.

Multiple issue with deep pipeline (Section 6.1 0)

]

Multiple-issue pipelined (Section 6.9) Single-cycle (Section 5.4)

Pipelined

Deeply pipelined

Multicycle (Section 5.5)

Several

Use latency in ins tru ction s FtGURE 6.53 The basic relationship between the datapaths In Figure 6.52. Notice that the x-axis is use latency in instructions, which is wltat determines the ease of keeping tlte pipeline full. The pipelined dalap.1th is shown as multiple dock cydes for instruction latency because the execution time of an instruction is not shorter; it's tlte instruction throughput tltat is improved.

454

Chapter 6

Enhancing Performance with Plpellnlng

With rem a rkable adva nces in processing, Amdahl's law suggests that another pa rt of the system will become the bottleneck. That bottleneck is the topic of the next chapter: the m em o ry system. An alternative to pushing uniprocessors to auto m atica lly exploit pa rallelism at the in structio n level is trying multiprocessors, which exploit parallelism at much coa rser levels. Parallel p rocessing is the topic of II Chapter 9, which appea rs o n the C D.

Historical Perspective and Further Reading This sectio n, which appea rs o n the CD, discusses the history of the first pipelined p rocesso rs, the ea rliest superscala rs, the development of ollt -of-order and speculative techniques, as well as impo rtant d evelopm ents in the accompanying com p iler techno logy.

Exercises 6.1 [5] If the time fo r an ALU op eratio n ca n b e sh o rten ed b y 25% (com p ared to the d escription in Figure 6.2 o n p age 373); a. W ill it affect the speedu p obtained fro m pipelining? If yes, b y h ow much ? Otherwise, why? b. W hat if the ALU op eratio n n ow takes 25% m o re time? 6.2 ( 10) A com p uter a rchitect n eeds to d esign the p ipeline o f a n ew m icrop rocesso r. She h as an example workload p rogram co re with 106 instructio n s. Each instructio n takes 100 p s to fini sh. a. How lo ng d oes it take to execute this p rogram co re on a n o npipelined p rocessor? b. The current state-of-the-a rt microp rocessor h as ab out 20 pipeline stages. Assume it is perfectly pipelined. How much sp eedu p will it achieve com pared to the n o npipelined p rocesso r? c. Rea l pipelining isn't perfect, since implem enting pipelining int roduces som e overhead p er pipeline stage. Will this overhead affect in structio n laten cy, instruction thro ugh p ut, o r b oth?

6.14

455

Exerci ses

6.3 [5J Using a drawing similar to Figure 6.5 on page 377, show the forwa rding paths needed to execute the following four instructions: add sub lw add

$3 $5 $7 $8

, , . ,

$4, $6 $3 , $2 100($5) $7 , $2

6.4 [ IOJ Idelltify all of the data dependencies in the following code. Which dependencies a re data haza rds that will be resolved via forwa rding? Which depen dencies a re data haza rds that will cause a stall? add sub lw add

$3 , $5 , $6 . $7 ,

6.5 [5J

$4 , $2 $3 , $1 200($3) $3 , $6

For More Practice: Delayed Branches

6.6 [IOJ Using Figure 6.22 on page 400 as a guide, use colored pens or markers to show which portions of the datapath a re active and which a re inactive in each of the fi ve stages of the sw instruction. We suggest that you use fi ve photocopies of Figure 6.22 to answer this exercise. (We hereby grant you permission to violate the Copyright Protection Act in doing the exercises in Chapters 5 and 6!) Be sure to include a legend to explain your colo r scheme.

II For More Practice: Understanding Pipelines by Drawing Them II For More Practice: Understa nding Pipelines by Drawing Them

6.7 [5J 6.8 [5J

6.9 [ 15J Them

6.10 [5J

For More Practice: Understanding Pipelines by Drawing For More Practice: Pipeline Registers

6.11 [ IS)

For Mo re Pra ctice: Pipelining Floating Point

6.12 [ IS) Figure 6.37 on page 41 7 and Figure 6.35 on page 41 5 a re two styles of drawing pipelines. To make sure you understand the relationship between these two styles, draw the information in Figures 6.31 th rough 6.35 on pages 4 10 th rough 41 5 using the style of Figure 6.37 on page 41 7. Highlight the active po rtions of the data paths in the figure. 6.13 (20) Figure 6.14. 10 is similar to Figure 6.14.7 on page 6.14 -9 in the II For More Practice section , but the in structions a re unidentified. Determine as much as you ca n about the fi ve in structions in the five pipeline stages. If you can not fill in a field of an instruction , state why. For some fi elds it will be easier to decode the machine instructions into assembly language, using Figure 3.1 8 on

456

Chapter 6

Enhancing Pe rformanc e with Plpellnlng

page 205 and Figure A.I D.l on page A-50 as references. For o ther fi eld s it will be easier to look at the va lues of the cont rol sign als, using Figures 6.26 th rough 6.28 on pages 403 and 405 as references. Yo u may need to ca refully exa mine Figures 6.14.5 th rough 6.14.9 to understand how collections of cont rol va lues are presented (i.e., the leftm ost bit in one cycle will become the uppermost bit in another cycle) . For example, the EX cont rol value for the subtract instruction, 1100, computed during the ID stage of cycle 3 in Figu re 6.14.6, becomes three separate va lues specifying RegDst ( I), ALUOp ( 10), and ALUSrc (0) in cycle 4.

6.14 (40) The fo llowing piece o f cod e is executed using the pipeline shown in Figure 6.30 o n page 409:

15 . add 16 . o r 17 . and 18 . su b 19 . lw

40(12 ) 13 . 12 12. 11 14. 13 12 . 11

At cycle 5, right befo re the instructio ns are executed , the p rocessor state is as fo llows: a. The PC has the va lue 1OOten' the address of the sub_ ins t r uc t ion . b. Every register has the initial value IOten plus the register number (e.g., register $8 has the initial value 18 ten ). c. Every m em ory wo rd accessed as d ata has the initial va lue WOOten plus the b yte add ress of the wo rd (e.g., Mem o ry(8 J has the initial va lue 1008ten ).

Determine the value o f every fi eld in the four pipeline registers in cycle 5.

II II

6.15 (20)

Fo r M ore Practice: Labeling Pipeline Diagra ms with Control

6.16 (20)

Fo r M o re Practice: Illustrating Diagram s with Fo rwa rding

6.17 (5 J Consider executing the fo llowing cod e o n the pipelined d atapath of Figure 6.36 on page 41 6:

add su b add add add

12 . 14. 15 . 17 . 18 .

13 . 13 . 13 . 16 . 12 .

15 17

At the end o f the fifth cycle o f execution , which registers are being read and which register will be written?

6.18 (5 J W ith rega rd to the p rogram in Exercise 6.1 7, explain what the forwa rding unit is do ing during the fifth cycle of executio n. If an y comparisons a re being m ad e, m ention them.

6.14

457

Exerci ses

6.19 [5] With regard to the program in Exercise 6.17, explain what the hazard detection unit is doing during the fifth cycle of execution. If any comparisons are being made, mention them. 6.20 [20)

For More Practice: Forwarding in Memory

6.21 [5] We ha ve a program of 10 3 instructions in the format of"1 W, add, 1 w, add, ..." The add instruction depends (and only depends) on the 1W instruction right before it. The 1 w in struction also depends (and only depends) on the add in struction right before it. If the program is executed on the pipelined datapath of Figure 6.36 on page 41 6: a. What would be the acnlal CPI? b. Without forwarding, what would be the actual CPI?

6.22 [5] Consider executing the following code on the pipelined datapath of Figure 6.36 on page 41 6: 1w sub add

$4. 1001 $2) $6, $4 , $3 $2, $3 , $5

How many cycles will it take to execute this code? Draw a diagram like that of Figure 6.34 on page 414 that illustrates the dependencies that need to be resolved , and provide another diagram like that of Figure 6.35 on page 41 5 that illustrates how the code will actually be executed (incorporating any stalls or forwa rding) so as to resolve the identified problems.

6.23 [ IS) List all the inputs and outputs of the fonvarding unit in Figure 6.36 on page 41 6. Give the names, the number of bits, and brief usage for each input and output. 6.24 [20) . . For More Practice: Illustrating Diagram s with Forwarding and Stalls 6.25 [20) (II For More Practice: Impact on Fonva rding of Moving It to ID Stage 6.26 [ IS) on Pipeline

II For More Practice: Impact of Memory Addressing Mode

6.27 ( 10 ) II For More Practice: Impact of Arithmetic Operations with Memory Operands on Pipeline 6.28 [30) Design

For More Practice: Fonvarding Unit Hardwa re

458

Chapter 6

Enhancing Performance with Plpellnlng

6.29 (1 week ) Using the simulato r provided with this book, collect statistics o n data hazards for a C program (supplied by either the instructor or with the softwa re). You will write a sub routine th at is passed the instruction to be executed , a nd this ro utine must m odel the five-stage pipeline in this chapter. Have your program collect the following statistics: •

Number of instructions executed.

•

Number of data haza rds not resolved by fo rwa rding and number resolved by forwa rding.

•

If the MIP S C compiler that you are using issues nop instructions to avoid haza rds, count the number of n 0 p in structio ns as well.

Assuming that the m em ory accesses always take 1 clock cycle, calculate the average number of clock cycles per instruction. Classify nop instructio ns as stalls inserted by softwa re, then subtract them from the number of instructio ns executed in the CPI calculatio n.

6.30 [71 In the example on page 425, we saw that the performance adva ntage of the multicycle design was limited by the lo nger time required to access mem ory versus use the ALU. Suppose the mem ory access beca me 2 clock cycles lo ng. Find the relative perfo rm ance of the single-cycle and multicycle designs. In the next few exercises, we extend this to the pipelined design , which requires lo ts m ore wo rk!

Ii Fo r M o re Practice: Coding with Conditio nal M oves (10 ) < §6.6> III For Mo re Practice: Performance Ad vantage of Conditio nal

6.31 (10 ) 6.32 M ove

6.33 [20 ) < §§6.2-6.6> In the example o n page 425, we saw that the perfo rmance adva ntage of both the multicycle and the pipelined designs was limited by the lo nger time required to access m em ory versus u se the ALU. Suppose the m em ory access becam e 2 clock cycles lo ng. Draw the m odified pipeline. List all the possible new forwa rding situations a nd all possible new haza rds and their length. 6.34 [20 ) Redo the example o n page 425 using the restructured pipeline of Exercise 6.33 to compare the single-cycle a nd multicycle. Fo r bran ches, assume the sa me prediction accuracy, but increase the penalty as appropriate. For loads, assume that the subsequent instructio ns depend o n the load with a probability of 1/ 2, 1/ 4, 1/8, 1/16, and so on. That is, the instructio n following a load by two has a 25% probability of using the loa d result as o ne of its sources. Ignoring any other data haza rds, find the relative perform ance of the pipelined design to the single-cycle design with the restrucnlfed pipeline. 6.35 [ 10 ) < §§6.4-6.6> As pointed o ut o n page 41 8, m oving the branch comparison up to the ID stage introduces a n o pportunity fo r bo th fo rwa rding a nd haza rds that cann ot be resolved by fo rwa rding. Give a set of code sequences th at show the possible

6.14

459

Exerci ses

forwa rding paths required and hazard cases that must be detected , considering only one of the two operands. The number of cases should equal the maximum length of the haza rd if no forwa rding existed.

6.36 ( 15) We have a program core consisting of five conditional branches. The program core will be executed thousa nds of times. Below are the outcomes of each branch for one execution of the program core (T fo r taken, N for not taken). Branch Branch Branch Branch Branch

I: 2: 3: 4: 5:

T-T-T N-N- N-N T-N-T-N-T- N T-T-T- N-T T-T-N-T-T-N-T

Assume the behavior of each branch remains the same for each program core execu tion. For dynamic schemes, assume each branch has its own prediction buffer and each buffer initialized to the same state before each execution. List the predictions for the following branch prediction schemes: a. Always taken b. Always not taken c. I -bit predictor, initialized to predict taken

d. 2-bit predictor, initialized to weakly predict taken What are the prediction accuracies?

6.37 ( 10 ) Sketch all the forwa rding paths for the branch inputs and show when they must be enabled (as we did on page 407). 6.38 ( 10 ) Write the logic to detect any haza rds on the branch sources, aswe did on page410. 6.39 ( 10 ) The example on page 378 shows how to maximize performance on our pipelined data path with forwa rding and stalls on a use following a load. Rewrite the following code to minimize performance on this datapath- that is, reorder the instructions so that this sequence takes the most clock cycles to execute while still obtaining the sa me result.

lw lw

12. 100(16) 13. 200(17)

add add

$4, $2 , $3 $6, $3 , $5 $8, $4 , $6

sub lw beq

17. 300(18) $7, $8 , Loop

460

Chapter 6

Enhancing Pe rformanc e with Plpellnlng

6.40 (20) Consider the pipelined datapath in Figure 6.54 on page 46 1. Ca n an attempt to flush and an attempt to stall occur simultaneously? If so, do they result in conflicting actions and/or cooperating actions? If there are any cooperating actions, how do they wo rk together? !fthere are any conflicting actions, which should take priority? Is there a simple change you can make to the datapath to ensure the necessary prio rity? You may want to consider the following code sequence to help you answer this question: beq 1w add sw

TARG ET :

II , 12 , TARG ET

# assume t ha t t he br anch is t aken

13 , 40( 14 ) 12 , 13 , $4 12 , 40( 14 ) II , II , $2

6.41 (15) The Verilog for implementing forwa rding in Figure 6.7.2 on page 6.7-4-6.7-5 did not consider fo rwa rding of a result as the value to be stored by a SW instruction. Add this to the Verilog code. 6.42 [5] The Verilog for implementing stalls in Figure 6.7.3 on page 6.7-6-6.7-7 did not consider fon varding of a result to use in an address calculation. Make this simple addition to the Verilog code. 6.43 [ IS) The Verilog code for implementing branch haza rd detection and stalls in Figure 6.7.3 on page 6.7-6-6.7-7 does not detect the possibility of data hazards for the two source registers of a BEQinstruction. Extend the Verilog in Figure 6.7.3 on page 6.7 -6-6.7-7 to handle all data hazards for branch opera nds. Write both the fon varding and stall logic needed fo r completing branches during !D. 6.44 (10 ) Rewrite theVerilog code in 6.7.3 on page 6.7-6-6.7-7 to implement a delayed branch strategy. 6.45 [20) Rewrite the verilogcode in Figure 6.7.3 on page 6.7-6-6.7-7 to implement a bran ch target buffer. Assume the buffer is implemented with a mod ule with the following definition:

module Predic t PC (cur r en t PC , ne xt PC, miss , update , des ti na ti on) ; i npu t cu rr en tP C, upda t e , II t rue if pr ev i ous pr edic ti on was un ava i lable or i ncorrec t des t in at ion ; I used wit h update t o correc t a pred i ct ion ou t pu t nex t PC , II r et urn s t he ne xt PC if pr edic ti on i s acc ur at e mlSS ; II t rue means no pred i ct io n i n bu f fe r endmodule ; Make sure you accomodate all three possibilities: a correct prediction , a miss in the buffer (that is, miss = true), and an incorrect prediction. In the last two cases, you must also update the prediction.

IF.Flustl Hazard

... /-- "-""- / dotIectlo n

'-.

rl'

'"",

- H--------,

'----,-+1-1 '" ." _ 1'\

r Control r '--+-I

•

- '1-,.-----, M~

L... M

O~,

•

,~~

WB -

lef1 2

Registers

momory f~· I'nstruct'on

" +--1

I-

Sign

..."'ndr

•,"

memory

-I

•",

forwarding

unit

FIGURE 6.54 Datapath for branch, Including hardware to flush the Instruction that follows the branch. ThIS optImIZatIon moves the branch decision from the fourth pipeline stage to the second; only one instruction thm follows the branch wm be in the pipe m thm time. The comrolline IEFlush turns the fetched instruction into a nop by zeroing the IFlID pipeline register. Although the flush line is shown coming from the control unit in this figure, in reality it comes from hardware that determines if a branch is taken, l1beled with an equal sign to the right of the registers in the ID stage. The forwarding muxes and p.1ths must also be added to this stage, but are not shown to simplify the figure.

462

Chapter 6

Enhancing Performance with Plpellnlng

6.46 ( I month ) If you have access to a simulation system such as Verilog or ViewLogic, first design the single-cycle datapath and cont rol from Chapter 5. Then evolve this design into a pipelined orga nization, as we did in this chapter. Be sure to run MIPS progra ms at each step to ensure that your refin ed design continues to operate correctly. 6.47 (10) The following code has been un rolled once but not yet sched uled. Assume the loop index is a multiple of two (i.e., $10 is a multiple of eight):

Loop :

1w sub sw 1w sub sw

12 . 14 . 14 . 15 . 16 . 16 . addi 110 . bne 110 .

0(110) 12 . $3 0(110) 4 (110) 15 . $3 4 (110) 11 0 . 8 $30 . Loop

Schedule this code for fast execution on the stand ard MIPS pipeline (assume that it supports addi instruction ). Assume initia lly $10 is 0 and $30 is 400 and that branches are resolved in the M EM stage. How does the scheduled code compa re aga inst the original unscheduled code?

6.48 (20) This exercise is similar to Exercise 6.47, except this time the code should be unrolled twice (creating three copies of the code). However, it is not known that the loop index is a multiple of three, and thus you will need to invent a mea ns of ensuring that the code still executes p roperly. (Hint : Consider adding some code to the beginning or end of the loop that takes ca re of the cases not handled by the loop.) 6.49 (20) Using the code in Exercise 6.47, unroll the code four times and schedule it fo r the static multiple-issue version of the MIPS processo r described on pages 436-439. You may assume that the loop executes fora multiple offour times. 6.50 (10) As technology leads to sm aller feature sizes, the wires become relatively slower (as compa red to the logic). As logic becomes faster with the shrinking fea ture size and clock rates increase, wire delays consume more clock cycles. That is why the Pentium 4 has several pipeline stages dedica ted to transferring data along wires from one pa rt of the pipeline to another. What are the drawbacks to having to add pipe stages fo r wire delays? 6.51 (30) New processors are int roduced more quickly than new versions of textbooks. To keep your textbook current, investigate some of the latest developments in this area and write a one-page elabo ration to insert at the end of Section 6.10. Use the World- Wide Web to explore the characteristics of the lastest processors from Intel or AM D as a starting point.

6.14

463

Exerci ses

§6. ! , page 384: 1. Stall on the LW result. 2. Bypass the ADD result. 3. No stall or bypass required. §6.2, page 399: Statements 2 and 5 are correct; the rest are incorrect. §6.6, page 426: 1. Predict not taken. 2. Predict taken. 3. Dyn amic prediction. §6.7, " page 6.7-3: Statements ! and 3 are both true. §6.7, " page 6.7-7: Only statement #3 is completely accurate. §6.8, page 432: O nly #4 is totally accurate. #2 is partially accurate. §6.9, page 447: Speculation: both; reorder buffer: hardwa re; register renaming: both; out -of-o rder execution: hardwa re; predication: softwa re; branch prediction: both; VLI W: softwa re; superscalar: hardwa re; EPI C: both, since there is substantial hardwa re support; multiple issue: both; dynamic scheduling: hardwa re. §6. !0, page 450: All the statements are false.

Answers to Check Yourself

Computers in the Real World

Mass Communication without Gatekeepers

Problem to solve: Offer society sources of

news and opin ion beyond those found in the traditional mass media. Solution: Use the Internet and World Wide

Web to se lect and publish nontraditional and non10ca1 news sources.

The Internet holds the promise of allowing citizens to communicate without the information first being interpreted by traditional mass media like television , newspapers, and magazines. To see what the future might be, we could look at countries that have widespread, high-speed Internet access. One place is South Korea. In 2002, 68% of South Korean households had broadband access, compared to 15% in the United States and 8% in Western Europe. (Broadband is generally digital subscriber loop or cable speeds, about 300 to WOO Kbps.) The main reason for the greater penetration is that 70% of households are in large cities and almost half are found in apartments. Hence, the Korean telecommunications industry could afford to quickly offer broadband to 90% of the households. What was the impact of widespread highspeed access on Korean society? Internet news

sites became extremely popular. One examp le is OhMyNews, which publishes articles from anyone after first checking that the facts in the article are correct. Many believe that Internet news services influenced the outcome of the 2002 Korean presidential election. First, they encouraged more young peop le to vote. Second, the winning candidate advocated politics that were Soulh Korea Hong Kong

t : : : : : : : : : : : : =====

Canada Belgium Un~ad

Slates

Nelherlands

'""'""

Denmark

Austria N~"Y

Germany

Switzerland

Porlugal Finland Spain

Percentage of households with broadband connections by country In 2002. Source: The Yankee Group, Boston.

closer to those popul ar on the Internet news services. Together th ey ove rca me the disad va ntage that most major medi a orga nizations endo rsed hi s opponent. Google News is another example of nontraditional access to news that goes beyond th e mass media of one country. It sea rches international news se rvices for topics, and then summ arizes and displays them by populari ty. Rather than leaving the decision o f wh at arti cles should be on the front page to loca l newspaper editors, the worldwide media decides. In addition, by providing links to stories from many countries, th e reader ge ts an international perspective rather th an a loca l one. It also is updated many times a day unlike a daily newspaper. The figure below comp ares the

JUdge Rules Out a Death Penalty lor 9/11 Suspect Rebuke for Justice Dept. Poll Shows Drop In COnndence on Bush Skillin Handling Cri ses Country on Wrong Track, SaY5 Solid Major~y

Revised Admission lor High Schools City SaY5 Students Will Get First Preference No illicit Arms Found In Iraq, U.S. Inspector Tells Congres s U.S. Practice How to Down Hijacked Jets Coetzee, Wr1ter 01 Apartheid as Bleak Mirror, Wins Nobel sexual Accu sation s Lead to an Apology by SChwarzenegger Interim Chlel Accept s Stock Exchange Shill Yankees Even with Twins Agency Warns of Fake Drugs Limbaugh Fallback Position

New York Times front page to the Google News Web site on the sa me day. The widespread imp act of these technologies reminds us that com puter enginee rs have responsibilities to their communities. We must be aware of societal values conce rning privacy, security, free speech, and so on to ensure th at new technological innovations enhance those va lues rather th an inadvertently

compromi sing th em. To learn more see these references on the

library

D "Seriously wired :' The Economist, April 17,2003. D O hMyNews, www.ohmynews.com D Coogle News, www. news.google.co lTl

Top Stories More than 1000 rally behind Schwarzenegger AP - 5 minutes ago Maria Shriver defends husband CNN Can accusations hurt Arnold 's campaign? KE SQ and 1252 related Bush: Hussein 'A Danger to the World' ABC news - 5 hours ago Bush Stands By Decision Voice of America Hunt for weapons yields no evidence The Canberra Times and 598 related World Stories Defiant UN chief announces rival blueprint for Iraq The Times (UK) - 2 hours ago France, Russia Assail US Draft on Iraq Reuters and 782 related

New York Times versus Google News on october 3, 2003 at 6 PM PT. The newsp.1per from page headlines must balance big stories with natio nal news, local news, and sports. Google News has many stories per headline from around the world, with links the reader ca n follow. Google stories vary by time of d.1 y and hence are mo re recent.

Large and Fast: Exploiting Memory Hierarchy Ideally one wonld desire an indefinitely large memory capacity such that any particular word would be immediately available. ... We are forced to recognize the possibility ofconstructing a hierarchy of memories, each ofwhich hasgreater capacity than the preceding but which is /ess quickly accessible. A. W. Burks, H. H. Goldstine, and J. von Neulllann Pmiminilry DiSCWS6iolf of tl~ Logical lNsign if an f1«troni€ Computing ErGtrument , 1946

•

7.1

Introduction

468

7.2

The Basics of Caches

7.3

Measuring and Improving Cache Performance

7.4

Virtual Memory

7.5

A Common Framework for Memory Hierarchies

7.6

Real Stuff: The Pentium P4 and the AMD Opteron Memory Hierarchies 546

7.7

Fallacies and Pitfalls

550

7.8

Concluding Remarks

552

7.9

Historical Perspective and Further Reading

7.10

Exercises

473 492

511

538

555

The Five Classic Components of a Computer

Interlace

468

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

Introduction

temporal locality The principle stating that if a data locati on is referen ced then it will tend to be referen ced again soon.

From the ea rliest days of computing, programmers have wa nted unlimited amounts of fast memory. The topics we will look at in this chapter aid program mers by creating the illusion of unlimited fast memo ry. Before we look at how the illusion is actually created , let's consider a simple analogy th at illustrates the key prin ciples and mechanisms that we lise. Suppose you were a student writing a term paper on important historica l developments in computer hardwa re. You are sitting at a desk in a library with a collection of books that you have pulled from the shelves and are exa mining. You find that several of the important computers that you need to write about are described in the books you have, but there is nothing about the EDSAC. Therefore, you go back to the shelves and look for an additional book. You find a book on ea rly British computers that covers EDSAC. On ce you have a good selection of books on the desk in front of you, there is a good p robability that many of the top ics you need can be found in them , and you may spend most of your time just using the books on the desk without going back to the shelves. Having several books on the desk in front of you saves time compared to having only one book there and constantly having to go back to the shelves to return it and take out another. The sa me principle allows us to crea te the illusion of a large memo ry that we ca n access as fast as a very small memory. Just as you did not need to access all the books in the library at once with equal probability, a program does not access all of its code or data at once with equal probability. Otherwise, it would be impossible to make most memory accesses fast and still have large memory in computers, just as it would be impossible for you to fit all the library books on your desk and still find what you wa nted quickly. This principle of locality underlies both the way in which you did your work in the library and the way th at p rograms operate. The principle of loca lity states that programs access a relatively small portion of their address space at any instant of time, just as you accessed a very small po rtion of the library's collection. There are two different types of locality:

spatial locality The locality principle stating that if a data

• Temporal locality (locality in time): If an item is referenced , it will tend to be referenced again soon. If you recently brought a book to your desk to look at, you will probably need to look at it again soon.

location is referenced, data location s with nearby addresses will tend to be referen ced soon.

• Spatial locality (loca lity in space): If an item is referenced , items whose addresses are close by will tend to be referenced soon. For exa mple, when

7.1

469

Introduction

you brought out the book on ea rly English computers to find out about EDSAC, you also noticed that there was another book shelved next to it about ea rly mechanical computers, so you also brought back that book too and , later on , found something useful in that book. Books on the sa me topic are shelved together in the library to increase spatial locality. We'll see how spa tial locality is used in memory hiera rchies a little later in this chapter. Just as accesses to books on the desk naturally exhibit locality, loca lity in progra ms arises from simple and natural program structures. For example, most progra ms contain loops, so instructions and data are likely to be accessed repeatedly, showing high amounts of temporal locality. Since in structions are normally accessed sequentially, programs show high spa tial locality. Accesses to data also exhibit a natural spatial locality. For exa mple, accesses to elements of an array or a record will naturally have high degrees of spatial locality. We take advantage of the principle of loca lity by implementing the mem ory of a computer as a memory hierarchy. A memory hierarchy consists of multiple levels of memory with different speeds and sizes. The faster memories are more expensive per bit than the slower memories and thus smaller. Today, there are three primary techn ologies used in building memory hierarchies. Main memory is implemented from DRAM (dynamic random access memory), while levels closer to the processor (caches) use SRAM (static random access mem ory). DRAM is less costly per bit than SRAM, although it is substan tially slower. The price difference arises because DRAM uses significa ntly less area per bit of memory, and DRAMs thus have larger capacity for the same amount of silicon ; the speed difference arises from several factors described in Section B.8 of Appendix B. The third technology, used to implement the largest and slowest level in the hierarchy, is magnetic disk. The access time and price per bit va ry wid ely among these technologies, as the table below shows, u sing typ ica l va lues for 2004: Memory technology

typical access time

$ per GB In 2004

SRAM DRAM Magnetic disk

0.5--5 ns 50- 70 ns 5,OOO,CX>O-20,()(X),OOO ns

$4000-.$10,000 $100- $200 $0.5Q-.$2

Because of these differences in cost and access time, it is advantageous to build memory as a hiera rchy of levels. Figure 7. 1 shows the faster memory is close to the processor and the slower, less expensive memory is below it. The goal is to present the user with as much memory as is ava ilable in the cheapest technology, while providing access at the speed offered by the fastest memory.

m emo r y hiera rch y A structure that uses multiple levels of memories; as the distance from the CPU increases, the size of the memories and the access time both increase.

470

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

Current

S peed

Fastest

CPU

Size

Cost (Sib il)

Tec hno lo gy

Smallest

Highest

SRAM

DRAM

Slowest

Biggest

Lowest

Magnetic Disk

FIGURE 7.1 Th e basic structure of a memory hierarchy. By implementing the memory system as a hierarchy, the user has the iUusion of a memory that is as large as the largest level of the hierarchy, but can be accessed as jf it were aU buill from the fastest memory.

blo(;k The minimum unit of information that can be either present or not present in the

t\Vo-level hierarchy.

hit rate The fraction ofmemory accesses found in a cache.

The memory system is orga nized as a hierarchy: a level closer to the processor is generally a subset of any level furth er away, and all the data is stored at the lowest level. By analogy, the books on your desk fo rm a subset of the library you are working in, which is in turn a subset of all the libraries on ca mpus. Furthermore, as we move away from the processo r, the levels take progressively longer to access, just as we might encounter in a hierarchy of ca mpus libraries. A memory hierarchy can consist of multiple levels, but data is copied between only two adjacent levels at a time, so we can focus our attention on just two levels. The upper level- the one closer to the processor-is smaller and faster (since it uses more expensive technology) than the lower level. Figure 7.2 shows that the minimum unit of inform ation that ca n be either present or not present in the two- level hierarchy is called a block or a line; in our library analogy, a block of information is one book. If the data requested by the processor appea rs in some block in the upper level, this is called a hit (analogous to your findin g the information in one of the books on your desk). If the data is not found in the upper level, the request is called a miss. The lower level in the hierarchy is then accessed to retrieve the block con taining the requested data. (Continuing our analogy, you go from your desk to the shelves to find the desired book.) The hit rate, or hit ratio, is the fr action of mem o ry accesses found in the upper level; it is oft en used as a measure of the perfor-

7.1

471

Introduction

Processor

~ Da ta a re translerred

FtGURE 7.2 Every pair of levels In the memory hierarchy can be thought of as having an upper and lower level. Within each level, the unit of information that is present or not is called a block. Usually we transfer an emire block when we copy something between levels.

mance of the memory hiera rchy. The miss rate (I - hit rate) is the fraction of memo ry accesses not found in the upper level. Since perfo rmance is the majo r reason for having a memo ry hierarchy, the time to service hits and misses is important. Hit time is the time to access the upper level of the memory hiera rchy, which includes the time needed to determine whether the access is a hit or a miss (that is, the time needed to look th rough the books on the desk). The miss penalty is the time to replace a block in the upper level with the corresponding block from the lower level, plus the time to deliver this block to the processor (or, the time to get another book fro m the shelves and place it on the desk). Because the upper level is sm aller and built using faster memo ry pa rts, the hit time will be much smaller than the time to access the next level in the hiera rchy, which is the major component of the miss penalty. (The time to examine the books on the desk is much smaller than the time to get up and get a new book from the shelves. ) As we will see in this chapter, the concepts u sed to build mem ory system s affect many other aspects of a co mputer, in cluding h ow th e operating system manages mem o ry and 110, h ow com pilers generate co de, and eve n how appli cations u se th e co mputer. Of course, becau se all program s spend much o f th eir time access ing memo ry, th e mem o ry system is necessa rily a majo r fac to r in determinin g perfo rm an ce. The relian ce on m emo ry hierarchies to achieve performance has mea nt that programmers, wh o used to be able to think of mem ory as a fl at, rand om access sto rage device, now need to understand

miss rate The fraction of memory accesses not found in a level of the memory hierarchy. hit time The time required to access a level of the memory hierarchy, including the time needed to determine whether the access is a h it or a miss. miss penalty The time required to fetch a block into a level of the memory hierarchy from the lower level, incl uding the time to access the block, transmit it from one level to the other, and insert it in the level that experienced the miss.

472

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

mem or y hierarchies t o get good performa nce. We sh ow how imp ortant this unde rst and ing is with t wo exa m ples. Since mem ory systems a re so critical to performance, com puter designers have devoted :I lot of attention to these system s and developed sophistica ted mechanisms for improving the performance of the memo ry system. In this chapter, we will see the major conceptual ideas, although ma ny simplifications and abst ractions have been used to keep the material manageable in length and com plexity. We cou ld easily have written hundreds of pages on memory systems, as dozens of recent doctoral theses have demonst rated.

Check Yourself

Wh ich of the following statements are generally t rue? I. Caches take advantage of tem po ral locality. 2. On a read , the value returned depends on wh ich blocks are in the cache. 3. Most of the cost of the memory hierarchy is at the highest level.

BIG

The Picture

Programs exhibit both temporal locality, the tendency to reuse recently accessed data items, and spatial locality, the tendency to reference data items that a re close to other recently accessed items. Memory hierarchies take adva ntage of temporal locality by keeping more recently accessed data item s closer to the processor. Memory hiera rchies take advantage of spatial locality by moving blocks consisting of multiple contiguou s words in memor y to upper levels of the hierarchy. Figure 7.3 shows that a memory hiera rchy uses smaller and faster mem or y technologies close to the processor. Thus, accesses that hit in the highest level of the hiera rchy can be processed quickly. Accesses that miss go to lower levels of the hiera rchy, which a re la rger but slower. If the hit rate is high enough, the memory hierarchy has an effective access time close to that of the highest (and fastest ) level and a size equal to that of the lowest (and la rgest) level. In most systems, the memo ry is a true hierarchy, meaning that data cannot be present in level i unless it is also present in level i + I.

7.2

473

The Basics of Caches

CPU

1 Increasing distance from the CPU in access time Levels in the memory hierarchy~

Level 2 ""

Level n

•

• Size of the memory at each level

FIGURE 7.3 This diagram shows the structure of a memory hierarchy: as the distance from the processor Increases, so does the size. This structure with the appropriate operating mechanisms allows the processor to have an access time that is determined prim.1rily by level I of the hierarchy and yet have a memory as large as levell!. Maintaining this illusion is the subject of this chapter. Although the local disk is normally the bottom of the hierarchy, some systems use tape or a file server over a local area network as the next levels of the hier.1rchy.

Cache: a safe place for hiding or storing things.

The Basics of Caches In our library exa mple, the desk acted as a cache-a sa fe place to store things (books) that we needed to exa mine. Cache was the name chosen to represent the level of the mem ory hierarchy between the processor and main memory in the first commercial computer to have this extra level. Today, although this remain s the dominant use of the wo rd cache, the term is also used to refer to any sto rage managed to take advantage of locality of access. Caches first appea red in resea rch computers in the ea rly 1960s and in production computers later in that sa me decade; every general-purpose computer built today, from servers to low-power embedded processors, includes caches. In this section , we begin by looking at a very simple cache in which the p rocessor requests are each one word and the blocks also consist of a single word. (Readers already familiar with cadle basics may wa nt to skip to Section 7.3 on page 492. )

Webster's New Wo rld D ictionary of the American Langllage, T hird College Edition ( 1988)

474

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

X, Xn _ 2

X _

Xn _ 1

n 1

Xo X, a. Before the reference to x"

X, b. After the reference to Xn

FIGURE 7.4 The cache just before and just after a reference to a word X n that Is not Initially In the cache. This reference causes a miss that forces the cache to fetch x" from memory and insert it into the cache.

direct-mapped cache A cache structure in which each memory location is mapped to exactly one location in the cache.

Figu re 7.4 shows such a simple cache, before and aft er requesting a data item that is no t initially in the cache. Before the request, the cache contains a collectio n of recent references X I> X2> ••• , Xn - 1' and the processor requests a word XII that is no t in the cache. This request results in a miss, and the wo rd X n is b ro ught from m em o ry into cache. In looking at the scena rio in Figure 7.4, there are two questio ns to answer: How do we know if a data item is in the cache? M oreover, if it is, how do we find it? The a nswers to these two questio ns are related. If each wo rd can go in exactly o ne place in the cache, then it is straightfo rwa rd to find the wo rd if it is in the cache. The simplest way to assign a locatio n in the cache for each word in mem o ry is to assign the cache locatio n based o n the address of the word in m em o ry. This cache structure is called direct mapped , sin ce each m em o ry loca tio n is m apped directly to exactly o ne loca tio n in the cache. The typical m apping between addresses and cache locatio ns fo r a direct-m apped cache is usually sim ple. For example, almost all direct-m apped caches use the m apping (Block address) m odulo (Number of cache blocks in the cache) This mapping is attractive beca use if the number of entries in the cache is a power of two, then m odulo ca n be computed simply by using the low-order log2 (cache size in blocks) bits of the address; hence the cache m ay be accessed directly with the low-o rder bits. For exa mple, Figure 7.5 shows how the mem o ry addresses

7.2

475

The Basics of Caches

Cache o

~ o ~ o ~ o ~ ~~ oo ~~ OO ~~~~

‫סס‬oo1

0010 1

01001

10001

11 001

111 0 1

Memory FIGURE 7.5 A dlrec:t-mapped cache with eight entries showing the addresses of memory words between 0 and 31 that map to the same cache locations. Because there are eight words in the cache, an address X maps to the cache mJrd X mooulo 8. That is, the low-order log2(8) = 3 bits are used as the cache index. Thus, addresses <XXXl1 1wo> 01OO1two> HXXl1 1wo> and ll001 m.., all map to ent ry 00l m", of the cache, while addresses 00101 1wo> 01101 1wo> 10101 1wo> and 11101 m.., all map to ent ry 101 m.., of the cache.

and 29 ten (I I 10 I two) map to loca tions I ten ( OO l two ) and Sten ( I O l tmJ ) in a direct -mapped cache of eight words. Because each cache location ca n contain the contents of a number of different memo ry locations, how do we know whether the data in the cache corresponds to a requested word? That is, how do we know whether a requested word is in the cache or not? We answer this question by adding a set of t ags to the cache. The tags contain the address information required to identify whether a word in the cache corresponds to the requested word. The tag needs only to contain the upper portion of the address, co rresponding to the bits th at are not used as an index into the cache. For example, in Figure 7.5 we need only have the upper 2 of the S address bits in the tag, since the lower 3-bit index field of the address selects the block. We exclude the index bits beca use they are redundant, since by definiti on the index fi eld of every address must have the sa me value. We also need a way to recognize that a cache block does not have va lid information. For instance, when a processor starts up, the cache does not have good between

I ten ( OOoo l two )

t ag A field in a table used for a mem ory hierarchy that contains the ad dress information required to identify whether the associated block in the hierarchy corresponds to a requested word.

476

valid bit A field in the tables of a m emory hierarchy that indicates that the associated block in the hierarchy contains valid data.

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

data, and the tag fi elds will be mea ningless. Even after executing many in structions, some of the cache entries may still be empty, as in Figure 7.4. Thus, we need to know that the tag should be ignored for such entries. The most common method is to add a valid bit to ind icate whether an entry contains a va lid address. If the bit is not set, there ca nnot be a match for this block. For the rest of this section , we will focus on explaining how reads work in a cache and how the cache control works for reads. In general, handling reads is a little simpler than handling writes, since reads do not have to change the contents of the cache. After seeing the basics of how rea ds wo rk and how cache misses can be handled , we'll exa mine the cache designs for rea l computers and detail how these caches handle writes.

Accessing a Cache Figure 7.6 shows the contents of an eight -word direct -mapped cache as it respond s to a series of requests from the p rocesso r. Since there are eight blocks in the cache, the low-o rder 3 bits of an address give the block number. Here is the action for each reference: Decimal address of reference

Binary address of reference

Hit or miss In cache

Assigned cache block (where found or placed)

10110r...,

miss (7 .6b)

(10 110_ mod 8) - 11Ot-

11010"""

miss (7.&)

(11010_ mod 8) _ 010_

10110"""

hit

(10 110_

mod 8) - 110_

26 16

11010"""

hit

(11010_

mod 8) " 010_

1()(XlO_

miss (7 .6d)

(1()(X)Q_

mod 8) " 000_

00011.,.,.,

miss (7 .6e)

(00011""" mod 8) _ 011_

16 18

1()()()()_ 10010"""

hit miss (7 .6f)

(1()(X)Q_

mod 8) " 000_

(10010""" mod 8)" 010_

When the word at address 18 (1001O,wo) is b rought into cache block 2 (O IO,wo)' the word at address 26 (i IOlO two )' which was in cache block 2 (0101\,.0)' must be replaced by th e newly requested data. This behavio r allows a cache to take adva ntage of tempo ral loca lity: recently accessed words replace less recently referenced words. This situation is directly an alogous to needing a book from the shelves and having no more space on your desk-some book alrea dy on your desk must be returned to the shelves. In a direct -mapped cache, there is only one place to put the newly requested item and hence only one choice of what to replace.

7.2

477

The Basics of Caches

.()()()

001 010 011 100 101 110 111

Data

N N N N N N

a. The initial state of the cache after

.()()()

001 010 011 100 101 110 111

N y

power~n

Dlata

Memory (11010two )

N N N y

Memortly (10110_)

.()()()

001 010 011 100 101 110 111

Data

N N N N Y

10~

Memory(1011°two )

b. After handling a miss of address (10110-,)

.()()()

001 010 011 100 101 110 111

N y

Data

10,_

Memory (10000,_) Memory (11010,_)

N N N y

Memory (10110,_)

c. After handling a miss of address (1101,\_)

d . After handling a miss of address

()()()

001 010 011 100 101 110 111

N y

Data 10~

Memory (10000"",) Memory (11010"",) Memory (00011"",)

N N y

10~

Memory (10110two )

e. After handling a miss of address (00011,_)

()()()

001 010 011 100 101 110 111

N y y

(1~)

Data

10,_

Memory (10000,_)

00,_

Memory (10010,_) Memory (00011,_)

N N y

Memory (10110,_)

f. After handling a miss of address (10010"",)

FIGURE 7.6 The cache contents are shown after each reference request that misses. with the Index and tag fields shown In binary. The cache is initially empty, with all valid bits (V entry in cache ) turned off (N). The processor requests the following addresses: 10110lwo ( miss), 1l01Oty,"O (miss ), 1011Otv."O (hit), 1101Oty,"O (hit ), HXXXlty,"O (miss), OOOlltwo (miss), HXXXlty,"O (hit), and l00lOty,"O (miss). The figures show the cache contents after each miss in the sequence has been handled. When address l00lO two (18) is referenced, the entry for address llOlO two (26) must be replaced, and a reference to 1101Otv."O wiU cause a subsequent miss. The tag field wiU contain only the upper portion of the address. The full address of a word contained in cache block i with tag field j for this cache is j x 8 + ;, or equivalently the concatenation of the tag field j and the index;. For example, in cache f above, index 010 has tag 10 and corresponds to address 10010.

478

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

We know where to look in the cache fo r each possible address: the low-o rder bits of an address ca n be used to find the unique cache entry to which the address could map. Figure 7.7 shows how a referenced address is divided into • a cache index, which is used to select the block • a tag fi eld , which is used to compa re with the value of the tag fi eld of the cache

Address (showing bit positions) 3 130

13 1211 ••• 21 0

• • •

Byt. offset

Hit

T, g

Dot

Index

Valid Tag

D' fa

0 1 2 •••

••• •••

1021 1022 1023

-0

Y FIGURE 7.7 For this cache, the lower portion of the address Is used to select a cache entry consisting of a data word and a tag. The tag from the cache is compared against the upper portion of the address to determine whether the entry in the cache corresponds to t he requested address. Because the cache has 2 10 (or 1024) words and a block size of I word, 10 bits are used to index the cache, leaving 32 - 10 - 2 = 20 bits to be compared against the tag. If the tag and upper 20 bits of the address are equal and the valid bit is on, then t he request hits in the cache, and the word is supplied to the processor. Otherwise, a miss occurs.

7.2

479

The Basics of Caches

The index of a cache block, together with the tag contents of that block, uniquely specifies the memory address of the word contained in the cache block. Because the index field is used as an address to access the cache and because an n-bit field has 2" values, the total number of entries in a direct-mapped cache must be a power of two. In the M IPS architecnlfe, since words are aligned to multiples of 4 bytes , the least significa nt 2 bits of every address specify a byte within a word and hence are ignored when selecting the word in the block. The total number of bits needed for a cache is a function of the cache size and the address size because the cache includes both the storage for the data and the tags. The size of the block above was one word, but normally it is severa l. Assuming the 32-bit byte address, a direct-mapped cache of size 2~ blocks with 2m -word (2 m +2 _byte) blocks will require a tag field whose size is 32 - (n + m + 2) bits because n bits are used for the index, m bits are used for the word within the block, and 2 bits are used for the byte part of the address. The total number of bits in a direct-mapped cache is 2" X (block size + tag size + valid fi eld size). Since the block size is 2m words (2 m +S bits) and the address size is 32 bits, the number of bits in such a cache is 2~ X (m X 32 + (32 - 11 - /1l - 2) + 1) = 2" X (m X 32 + 3 1 - 11 - /1l). However, the naming convention is to excludes the size of the tag and valid field and to count only the size of the data.

Bits in a Cache

How many total bits are required for a direct-mapped cache with 16 KB of data and 4-word blocks, assuming a 32-bit address?

EXAMPLE

We know that 16 KB is 4K words, which is 2 12 words, and, with a block size of 4 words (2 2) , 2 10 blocks. Each block has 4x 32 or 128 bits of data plus a tag, which is 32 - 10 - 2 - 2 bits, plus a valid bit. Thus, the total cache size is

ANSWER

2 10 X(128+(32-1O-2-2)+ 1) = 2 lO x 147 = 147 Kbits or 18.4 KB for a 16 KB cache. For this cache, the total number of bits in the cache is about 1.1 5 times as many as needed just for the storage of the data.

480

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

Mapping an Address to a Multiword Cache Block

EXAMPLE

Consider a cache with 64 blocks and a block size of 16 bytes. What block number does byte address 1200 map to?

ANSWER

We saw the formula o n page 474. The block is given by ( Block address) modulo (Number o f cache blocks) where the address o f the block is

Byte address Bytes per block Notice that this block address is the block containing all addresses between

Byte address X Bytes per block Bytes per block

and

Byte address X Bytes per block + ( Bytes per block - 1) Bytes per block

Thus, with 16 bytes per block, byte address 1200 is block address

l l~~ J =

which maps to cache block number (75 modulo 64) = 11. In fact, this block maps all addresses between 1200 and 12 15. Larger blocks explo it spatial locality to lower miss rates. Ai; Figure 7.8 shows, increasing the block size usually decreases the miss rate. The miss rate m ay go up eventually if the block size becomes a significa nt fractio n o f the cache size because the number of blocks that ca n be held in the cache will become small , and there will be a great deal o f competition for those blocks. Ai; a result, a block will be bumped o ut o f the cache before many o f its words are accessed. Stated alternatively, spatial locality among the words in a block decreases with a very large block; consequently, the benefits in the miss rate become smaller.

7.2

10%

Miss mte

481

The Basics of Caches

~-----o--

1 4K

128

Block size FIGURE 7.8 Miss rate versus block size. Note that miss rate actually goes up if the block size is too large relative to the cache size. Each line represents a cache of different size. (This figure is independent of associativity, discussed soon.) Unfortunately, SPEC2000 traces would take too long if block size were included, so these data are based on SPEC92.

A more serious problem associa ted with just increasing the block size is that the cost of a miss increases. The miss penalty is determined by the time requi red to fetch the block from the next lower level of the hiera rchy and loa d it into the cache. The time to fetch the block has two parts: the latency to the first word and the transfer time fo r the rest of the block. Clea rly, unless we change the memory system , the transfer time-a nd hence the miss penalty-will increase as the block size grows. Furthermore, the imp rovement in the miss rate starts to decrease as the blocks become larger. The result is that the increase in the miss penalty overwhelms the decrease in the miss rate for large blocks, and cache performance thus decreases. Of course, if we design the memory to transfer larger blocks more effi ciently, we ca n increase the block size and obtain further improvements in cache performance. We discuss this topic in the next section . Elaboration: The major disadvant age of increasing t he block s ize is t hat t he cache miss penalty increases . Alt hough it is hard t o do anyth ing about t he latency component of t he miss penalty, we may be able to hide some of t he transfer ti me so t hat t he miss penalty is effectively sma ller. The s implest method for doing t his, ca lled early restart , is s imply to resume execution as soon as t he requested word of the block is returned, rather than wait f or the ent ire block. Many processors use t his t echnique f or instru ction

482

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

access, where it works best. Instruction accesses are largely sequential, so if the memo ory system can deliver a word every clock cycle, the processor may be able to restart operation when the requested word is returned, with the memory system delivering new instruction words just in time. This technique is usually less effective for data caches because it is likely that the words will be requested from the block in a less predictable way, and the probability that the processor will need another word from a different cache block before the transfer completes is high . If the processor cannot access the data cache because a transfer is ongoing, then it must stall. An even more sophisticated scheme is to organize the memory so that the requested word is transferred from the memory to the cache first. The remainder of the block is then transferred, starting with the address after the requested word and wrapping around to the beginning of the block. This technique, called requested word first, or critical word first, can be slightly faster than early restart, but it is limited by the same properties that limit early restart.

Handling Cache Misses (;ache miss A request for data from the cache that cannot be filled because the data is not present in the cache.

Before we look at the cache of a real system, let's see how the control unit deals with cache misses. The control unit must detect a miss and process the miss by fetching the requested data from memory (or, as we shall see, a lower-level cache). If the cache reports a hit , the computer continues using the data as if nothing had happened. Consequently, we can use the same basic control that we developed in Chapter 5 and enhanced to accommodate pipelining in Chapter 6. The memories in the datapath in Chapters 5 and 6 are simply replaced by caches. Modifying the control of a processor to handle a hit is trivial; misses, however, require some extra work. The cache miss handling is done with the processor control unit and with a separate controller that initiates the memory access and refills the cadle. The processing of a cache miss creates a stall, similar to the pipeline stalls discussed in Chapter 6, as opposed to an interrupt, which would require saving the state of all registers. For a cache miss, we can stall the entire processor, essentially freezing the contents of the temporary and programmer-visible registers, while we wait for memory. In contrast, pipeline stalls, discussed in Chapter 6, are more complex because we must continue executing some instructions while we stall others. Let's look a little more closely at how instruction misses are handled for either the multicycle or pipelined datapath; the same approach can be easily extended to handle data misses. If an instruction access results in a miss, then the content of the Instruction register is invalid. To get the proper instruction into the cache, we must be able to instruct the lower level in the memory hierarchy to perform a read. Since the program counter is incremented in the first clock cycle of execution in both the pipelined and multicycle processors, the address of the instruction that generates an instruction cache miss is equal to the value of the program counter minus 4. Once we have the address, we need to instruct the main memory

7.2

The Basics of Caches

483

to perform a read. We wa it for the memor y to respond (since the access will take multiple cycles), and then write the words into the cache. We ca n now define the steps to be taken on an instruction cache miss: I. Send the original PC va lue (current PC - 4) to the memory. 2. Instruct main memory to perform a read and wa it for the memory to com plete its access. 3. Write the cache entry, putting the data from memory in the data portion of the entry, writing the upper bits of the address (from the ALU ) into the tag fi eld, and turning the valid bit on. 4. Restart the instruction execution at the first step, which will refetch the instruction, this time findin g it in the cache. The control of the cache on a data access is essentially identical: on a miSS, we simply stall the processor until the memory responds with the data.

Handling Writes Writes work somewhat differently. Suppose on a sto re instruction, we wrote the data into only the data cache (without changing main mem ory); then, after the write into the cache, memory would have a different value from that in the cache. In such a case, the cache and memo ry are sa id to be inconsistent. The simplest way to keep the main memory and the cache consistent is to always write the data into both the memo ry and the cache. This scheme is called write-through. The other key aspect of writes is what occurs on a write miss. We first fetch the words of the block from memory. Aft er the block is fetched and placed into the cache, we can overwrite the word that caused the miss into the cache block. \Ve also write the word to main memory u sing the full address. Although this design handles writes very simply, it would not provide very good performance. With a write-through scheme, every write causes the data to be written to main memory. These writes will take a long time, likely at least 100 processor clock cycles, and could slow down the processor considerably. For the SPEC2000 integer benchmarks, for example, 10% of the instructions are stores. If the CPI without caclle misses was 1.0, spending 100 extra cycles on every write would lead to a C PI of 1.0 + 100 x 10% = II , reducing performance by more than a fa ctor of 10. One solution to this problem is to use a write buffer. A write buffer stores the data while it is waiting to be written to memory. After writing the data into the cache and into the write buffer, the processor can continue execution. When a write to main memory completes, the entry in the write buffer is freed. If the write buffer is full when the processor reaches a write, the processor must stall until there is an empty position in the write buffer. Of course, if the rate at which the

write-through A scheme in which writes always update both the cache and the memory, ensuring that data is always consistent between the two.

write buffer A queue that holds data while the data are waiting to be written to memory.

484

write-back A scheme that handles writes by updating values only to the block in the cache, then writing the modified block to the lower level of the hierarchy when the block is replaced.

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

memory can complete writes is less than the rate at which the processor is generating writes, no amount of buffering can help because writes are being generated faster than the memory system can accept them. The rate at which writes are generated may also be less than the rate at which the memory can accept them , and yet stalls may still occur. This can happen when the writes occur in bursts. To reduce the occurrence of such stalls, processors usually increase the depth of the write buffer beyond a single entry. The alternative to a write-through scheme is a scheme called write-back. In a write-back scheme, when a write occurs, the new value is written only to the block in the cache. The modified block is written to the lower level of the hierarchy when it is replaced. Write-back schemes can improve performance, especially when processors can generate writes as fast or faster than the writes can be handled by main memory; a write-back scheme is, however, more complex to implement than write-through. In the rest of this section, we describe caches from real processors, and we examine how they handle both reads and writes. In Section 7.5, we will describe the handling of writes in more detail. Elaboration: Writes introduce several complications into caches that are not present for reads . Here we discuss two of them : the policy on write misses and efficient implementation of writes in write-back caches . Consider a miss in a write-through cache . The strategy followed in most writethrough cache designs, called fetch
7.2

The Basics of Caches

In a write-back cache, because we cannot overwrite the block, stores either require two cycles (a cycle to check for a hit follo'Ned by a cycle to actually perform the write) or require an extra buffer, called a store buffer, to hold that data-effectively allowing the store to take only one cycle by pipelining it. When a store buffer is used, the processor does the cache lookup and places the data in the store buffer during the normal cache access cycle . Assuming a cache hit, the new data is written from the store buffer into the cache on the next unused cache access cycle. By comparison, in a write-through cache, writes can always be done in one cycle. There are some extra complications with multiword blocks, however, since we cannot simply overwrite the tag when we write the data . Instead, we read the tag and write the data portion of the selected block. If the tag matches the address of the block being written, the processor can continue normally, since the correct block has been updated . If the tag does not match, the processor generates a write miss to fetch the rest of the block corresponding to that address. Because it is always safe to overwrite the data, write hits still take one cycle. Many write-back caches also include write buffers that are used to reduce the miss penalty when a miss replaces a dirty block. In such a case, the dirty block is moved to a write-back buffer associated with the cache while the requested block is read from memory. The write-back buffer is later written back to memory. Assuming another miss does not occur immediately, this technique halves the miss penalty when a dirty block must be replaced .

An Example Cache: The Intrinsity FastMATH processor The Intrinsity FastMATH is a fast embedded microprocessor that uses the MIPS architecture and a simple cache implementation. Near the end of the chapter, we will examine the more complex cache design of the Intel Pentium 4, but we start with this simple, yet real, example for pedagogical reasons. Figure 7.9 shows the orga nization of the Intrinsity FastMATH data cache. This processor has 12-stage pipeline, similar to that discussed in Chapter 6. When operating at peak speed, the processor can request both an instruction word and a data word on every clock. To satisfy the demands of the pipeline without stalling, separate instruction and data caches are used. Each cache is 16 KB, or 4K words, with 16-word blocks. Read requests for the cache are straightforward. Because there are separate data and instruction caches, separate control signals will be needed to read and write each cache. (Remember that we need to update the instruction cache when a miss occurs.) Thus, the steps for a read request to either cache are as follows: I. Send the address to the appropriate cache. The address comes either from the PC (for an instruction) or from the ALU (for data). 2. If the cache signals hit, the requested word is available on the data lines. Since there are 16 words in the desired block, we need to select the right

485

486

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

Addre ss (showing bit positions) 31

•••

Hit

141 3 "' 65 "' 2 1 0

Tag

B~e

Data

offset Block offset

Index 18 bits

512 bits

•

• ••

Tag

Data ••• •• • •• • ••• ••• •• •

256 entries

•• • •• •

:J2

32 • ••

... I M", 32

FIGURE 7.9 11Ie 16 KB caches In the Intrinslty FastMATH each contain 256 blocks with 16 words per block. The tag field IS 18 bIts wide and the index field is 8 bits wide, while a 4-bit field (bits 5- 2) is used to index the block and select the mJrd from the block using a 16-to-1 multiplexor. In practice, to elimin.1le the multiplexor, caches U'ie a separate large RAM for the data and a smaller RAM for the tags, with the block offset supplying the extra address bits for the large data RAM. In this case, the large RAM is 32 bits wide and mU'it have 16 tinles as many words as blocks in the cache.

one. A block index field is used to control the multiplexor (shown at the bottom of the fi gure), which selects the requested word from the 16 words in the indexed block. 3. If the cache signals miss, we send the address to the main memory. Wh en the memo ry returns with the data, we write it into the cache and then read it to fulfill the request. For writes, the Intrinsity FastM ATH offers both write-through and write-back, leaving it up to the operating system to decide which strategy to use for an application. It has a one-entry write buffer.

7.2

487

The Basics of Caches

Instruction miss rate

Data miss rate

Effective combined miss rate

0.496

11.496

3 .2%

FtGURE 7.10 Approximate Instruction and data miss rates for the Intrlnslty FastMATH processor for SPEC2000 benchmarks. The combined miss rate is the effective miss rate seen for the combination of the 16 KB instruction cache and 16 KB data cache. It is obtained by weighting the instruc-

tion and data individual miss rates by the frequency of instruction and data references.

\-¥hat cache miss rates are attained with a cache strucnlre like that used by the lntrinsity FastMATH? Figure 7.10 shows the miss rates for the instruction and data caches for the SPEC2000 integer benchmarks. The combined miss rate is the effective miss rate per reference for each program after accounting for the differing frequency of instruction and data accesses. Although miss rate is an important characteristic of cache designs, the ultimate measure will be the effect of the memory system on program execution time; we'll see how miss rate and execution time are related shortly. Elaboration: A combined cache with a total size equal to the sum of the two split caches will usually have a better hit rate . This higher rate occurs because the combined cache does not rigidly divide the number of entries that may be used by instructions from those that may be used by data . Nonetheless, many processors use a split instruction and data cache to increase cache bandwidth. Here are miss rates for caches the size of those found in the Intrinsity FastMATH processor, and for a combined cache whose size is equal to the total of the two caches: •

Total cache size : 32 KB

•

Split cache effective miss rate: 3 .24%

•

Combined cache miss rate: 3 .18%

The miss rate of the split cache is only slightly worse . The advantage of doubling the cache bandwidth , by supporting both an instruction and data access simultaneously, easily overcomes the disadvantage of a slightly increased miss rate . This observation is another reminder that we cannot use miss rate as the sole measure of cache performance, as Section 7 .3 shows .

Designing the Memory System to Support Caches Cache misses are satisfied from main memory, which is constructed from DRAMs. In Section 7.1, we saw that DRAMs are designed with the primary emphasis on density rather than access time. Although it is difficult to reduce the latency to fetch the first word from memory, we can reduce the miss penalty if we increase the bandwidth from the memory to the cache. This reduction allows

split cache A scheme in which a level of the memory hierarchy is composed of two independent caches that operate in parallel with each other with one handling instructions and one handling data.

488

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

larger block sizes to be used while still maintaining alow miss penalty, similar to that for a smaller block. The processor is typica lly connected to memory over a bus. The clock rate of the bus is usually much slower than the processor, by as much as a fa ctor of 10. The speed of this bus affects the miss penalty. To understand the impa ct of different organizations of memory, let's define a set of hypothetical memory access times. Assume •

1 memory bus clock cycle to send the address

•

15 memory bus clock cycles for each DRAM access initiated

•

1 memory bus clock cycle to send a word of data

If we have a cache block of four words and a one-word-wide bank of DRAMs, the miss penalty would be 1 + 4 X 15 + 4 X 1 = 65 memory bus clock cycles. Thus, the number of bytes transferred per bus clock cycle for a single miss would be 4 x 4 = 0.25

65 Figure 7.11 shows three options for designing the memory system. The first option follows what we have been assuming: memory is one word wide, and all accesses are made sequentially. The second option in creases the bandwidth to memory by widening the memory and the buses between the processor and mem ory; this allows parallel access to all the words of the block. The third option increases the bandwidth by widening the mem ory but not the interconnection bus. Thus, we still pay a cost to transmit each word, but we ca n avoid paying the cost of the access latency m ore than once. Let's look at how much these other two options improve the 65-cycle miss penalty that we would see for the first option ( Figure 7. 11 a). Increasing the width of the memory and the bus will increase the memory bandwidth proportionally, decreasing both the access time and transfer time portions of the miss penalty. With a main memory width of two words, the miss penalty drops from 65 memory bus clock cycles to 1 + 2 X 15 + 2 X 1 = 33 memory bus clock cycles. With a four-word-wide memory, the miss penalty is just 17 memory bus clock cycles. The bandwidth for a single miss is then 0.48 (almost twice as high) bytes per bus clock cycle for a memory that is two words wide, and 0.94 bytes per bus clock cycle when the memory is four words wide (almost four times higher). The major costs of this enhancement are the wider bus and the potential increase in cache access time due to the multiplexor and control logic between the processor and cache. Instead of making the entire path between the memory and cache wider, the memory chips can be orga nized in banks to read or write multiple words in one

7.2

489

The Basics of Caches

CPU

< >

Multiplexor

Cache

Cache Cache

'---

Memory

b. Wide memo ry org anizati on

Memory

bank 0

bank 1

bank 2

bank 3

c. Interleaved memory organization

Memory

a. One-word-wide memory organizati on FtGURE 7.11 The primary method of achieving higher memory bandwidth Is to Increase the physical or logical width of the memory system. In this figure, memory bandwidth is improved two ways. The simplest design, (a), uses a memory where all components are one word wide; (b ) shows a wider memory, bus, and cache; while (c) shows a narrow bus and cache with an interleaved memory. In (b ), the logic between the cache and processor consists of a multiplexor used on reads and control logic to update the appropriate words of the cache on writes.

access time rather than reading or writing a single word each time. Each ba nk could be one wo rd wide so th at the width of the bus and the cache need not change, but sending an address to several banks permits them all to rea d simultaneously. This scheme, which is ca lled interleaving, retains the adva ntage of incurring the full memory latency only once. For example, with four ba nks, the time to get a four-word block would consist of 1 cycle to transmit the address and rea d request to the banks, 15 cycles fo r all four banks to access memory, and 4 cycles to send the four words back to the cache. This yields a miss penalty of 1 + 1 X 15 + 4 X I = 20 memory bus clock cycles. This is an effective bandwidth per miss of 0.80 bytes per clock, or about three times the bandwidth for the one-wa rd-wide mem -

49.

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

ory and bus. Banks are also valuable on writes. Each bank can write independently, quadrupling the write bandwidth and leading to fewer stalls in a writethrough cache. As we will see, an alternative strategy for writes makes interleaving even more attractive. Elaboration: Memory chips are organized to produce a number of output bits, usually 4 to 32, with 8 or 16 being the most popular in 2004 . We describe the organization of a RAM as d x w, where d is the number of addressable locations (the depth) and w is the output (or width of each location). One path to improving the rate at which we transfer data from the memory to the caches is to take advantage of the structure of DRAMs. DRAMs are logically organized as rectangular arrays, and access time is divided into row access and column access . DRAMs buffer a row of bits inside the DRAM for column access . They also come with optional timing signals that allow repeated accesses to the buffer without a row access time. This capability, originally called page mode, has gone through a series of enhancements. In page mode, the buffer acts like an SRAM; by changing column address, random bits can be accessed in the buffer until the next row access. This capability changes the access time significantly, since the access time to bits in the row is much lower. Figure 7 .12 shows how the density, cost, and access time of DRAMS have changed over the years. The newest development is DDR SDRAMs (double data rate synchronous DRAMs). SDRAMs provide for a burst access to data from a series of sequential locations in the DRAM . An SDRAM is supplied with a starting address and a burst length . The data in the burst is transferred under control of a clock signal, which in 2004 can run at up to

Year Introduced

Chip size

$ per MB

Total access time to a new row/ column

Column access time to existing row

1980

64 Kbit

$1500

250 ns

150 ns

1983

256 Kbit

$500

185 ns

100 ns

1985

1 Mbit

$200

135 ns

40 ns

1989

4 Mbit

110 ns

40 ns

1992

16 Mbit

$50 $15

90 ns

30 ns

1996

64 Mbit

60 ns

12 ns

1998

128 Mbit

60ns

10 ns

2000

256 Mbit

$10 $4 $1

55ns

2002

512 Mbit

$0.25

50",

2004

1024 Mbit

$0.10

45ns

5 "' 3 "'

FIGURE 7.12 DRAM size Increased by multiples of four approximately once every three years until 1996, and thereafter doubling approximately every two years. The improvements in access time have been slower but continuous, and cost almost tracks density improvements, although cost is often affected by other issues, such as availability and demand. The cost per meg.1byte is not adjusted for inflation.

7.2

491

The Basics of Caches

300 MHz. The two key advant ages of SDRAMs are t he use of a clock t hat eliminates t he need to synchron ize and t he elimination of the need t o supply successive addresses in t he burst. The DDR part of t he name means data transf ers on both t he leading and falling edge of t he clock, t hereby getting twice as much bandwidth as you might expect based on t he c lock rat e and t he data width. To de liver such high bandwidt h, t he intern al DRAM is organized as interl eaved memory ba nks . The advant age of t hese optimizations is that t hey use t he c ircuitry already large ly on t he DRAMs, add ing little cost to the system wh ile achieving a sign ificant improvement in ba ndwidth . The intern al architecture of DRAMs and how t hese opt imizations are implemented are described in Secti on 8 .8 of II Appendix 8 .

Summary We began the previous section by examining the simplest of caches: a direct -m apped cadle with a o ne-word block. In such a cadle, both hits and misses are simple, since a word ca n go in exactly o ne location and there is a separate tag for every word. To keep the cache and mem o ry consistent, a write-through scheme can be used, so that every write into the cache also causes mem o ry to be updated. The alternative to writethrough is a write-back scheme that copies a block back to memo ry when it is replaced; we'll discuss this scheme furth er in upcoming sectio ns. To take advantage of spatial locality, a cache must have a block size larger than o ne word. The use of a larger block decreases the miss rate and improves the effi ciency of the cache by reducing the am o unt of tag storage relative to the am ount of data storage in the cache. Although a larger block size decreases the miss rate, it ca n also increase the miss penalty. If the miss penalty increased linea rly with the block size, la rger blocks could easily lead to lower perform ance. To avoid this, the bandwidth of m ain m em o ry is increased to tra nsfer cache blocks m o re effi ciently. The two commo n m ethods fo r doing this are m aking the m em ory wider and interleaving. In bo th cases, we reduce the time to fetch the block by minimizing the number of times we must start a new mem o ry access to fetch a block, a nd , with a wider bus, we can also decrease the time needed to send the block from the mem o ry to the cache. The speed of the m em o ry system affects the designer's decisio n on the size of the cache block. \Vhich of the following cache designer guidelines are generally valid? 1. The shorter the m em o ry latency, the sm aller the cache block.

2. The shorter the m em ory latency, the larger the cache block. 3. The higher the mem ory bandwid th , the sm aller the cache block. 4. The higher the mem o ry ba ndwidth, the larger the cache block.

Check Yourself

492

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

Measuring and Improving Cache Performance In this section , we begin by looking at how to measure and a nalyze cache performance; we then explo re two different techniques for improving cache performance. One focuses on reducing the miss rate by reducing the pro bability that two different mem ory blocks will contend fo r the same cache location. The second technique reduces the miss penalty by adding an additional level to the hiera rchy. This technique, called multilevel caching, first appea red in high -end computers selling for over $1 00,000 in 1990; since then it has become common on desktop computers selling fo r less than $1000! CPU time ca n be divided into the clock cycles that the CPU spends executing the program and the clock cycles that the CPU spends wa iting for the memory system. Normally, we assume that the costs of cache accesses that are hits are part of the normal CPU execution cycles. Thus,

CPU time = (CPU execution clock cycles + Memory-stall clock cycles) x Clock cycle time The memory-stall clock cycles come primarily from cache misses, and we make that assumption here. We also restrict the discussion to a simplified model of the memo ry system. In rea l processors, the stalls generated by reads and writes ca n be quite complex, and accurate performance prediction usually requires very detailed simulations of the processor and mem ory system. Memory-stall clock cycles ca n be defin ed as the sum of the stall cycles coming from rea ds plus th ose coming from writes: Memory-stall clock cycles = Rea d-stall cycles + Write-stall cycles The rea d-stall cycles ca n be defined in terms of the number of rea d accesses per program , the miss penalty in clock cycles for a rea d, and the read miss rate: Read-stall cycles =

Reads X Rea d miss rate X Rea d miss penalty Program

Writes are mo re complicated. For a write-th rough scheme, we have two sources of stalls: write misses, which usually require that we fetch the block befo re continu ing the write (see the Elaboration on page 484 for more details on dealing with writes), and write buffer stalls, which occur when the write buffer is full when a write occurs. Thus, the cycles stalled for writes equals the sum of these two:

7.3

493

Measuring and Improving Cache Performance

Write-stall cycles

Writes X I" ·· I" . . I ) 'v nte miss rate X 'v nte miss pena ty ( Program

+ Write buffer stalls Because the write buffer stalls depend on the timing of writes, and not just the frequency, it is not possible to give a simple equation to compute such stalls. Fortunately, in systems with a reasonable write buffer depth (e.g., four or m ore words) and a mem ory capable of accepting writes at a rate that significantly exceeds the average write frequency in progra ms (e.g., by a factor of two) , the write buffer stalls will be small, and we ca n safely ignore them. If a system did not meet these criteria, it would not be well designed; instead, the designer should have used either a deeper write buffer or a write-back organization. Write-back schemes also have potential additional stalls arising from the need to write a cache block back to memory when the block is replaced. We will discuss this m ore in Section 7.5. In m ost write- th ro ugh cache o rga nizations, the rea d and write miss penalties a re the same (the time to fetch the block from mem ory). If we assume that the write buffer stalls a re negligible, we can combine the reads and writes by using a single miss rate and the miss penalty: Mem ory-stall clock cycles

Mem ory accesses X Miss rate X Miss penalty Progra m

We ca n also factor this as Mem ory-stall clock cycles = Instructions X Misse~ X Miss penalty Program InstructIOn Let's consider a simple example to help us understand the impact of cache perform ance on processor performance.

Calculating Cache Performance

Assume an instruction cache miss rate for a program is 2% and a data cache miss rate is 4%. If a processo r has a CPI of 2 without a ny mem ory stalls and the miss penalty is 100 cycles fo r all misses, determine how much faster a p rocesso r would run with a perfect cache that never missed. Use the instruction frequencies for SPECint2000 from Chapter 3, Figure 3.26, on page 228.

EXAMPLE

494

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

The number of memory miss cycles for instructions in terms of the Instruction count (I ) is

ANSWER

I nstruction miss cycles = I X 2 % X 100 = 2.00 X I

The frequency of all loads and sto res in SPECint2000 is 36%. Therefore. we ca n find the number of mem ory miss cycles for data references: Data miss cycles = I X 36 % X 4 % X 100 = 1.44 X I

The total number of memory-stall cycles is 2.00 I + 1.44 I = 3.44 I. This is m ore than 3 cycles of m emory stall per instructio n. Accordingly, the CP! with mem ory stalls is 2 + 3.44 = 5.44. Since there is no cha nge in instruction count or clock rate, the ratio of the C PU execution times is CPU time with stalls CPU time with perfect cache

I X C PI stall X Clock cycle I X CPl prrfect X Clock cycle

CP! stall

5.44

CPI prrfect

The perfo rmance with the perfect cache is better by 5.44 2

2.72 .

Wh at happens if the processor is m ade faster, but the m em o ry system is no t? The a mo unt of time spent o n mem o ry stalls will take up a n increasing fractio n of the executio n time; Amdahl's law, which we examined in Chapter 4, reminds us of this fact. A few simple examples show how serious this problem ca n be. Suppose we speed up the computer in the previo us example by reducing its CPl from 2 to 1 witho ut changing the clock rate, which might be done with an imp roved pipeline. The system with cache misses would then have a CPI of 1 + 3.44 = 4.44, and the system with the perfect cache would be 4.44 = 4.44 times fa ster 1

The am ount of execution time spent o n m em ory stalls would have risen fro m 3.44 = 63 % 5.44 to

7.3

495

Measuring and Improving Cache Performance

3.44 4.44

77%

Similarly, increa sing clock rate without changing the mem ory system also increases the performance lost due to cache misses, as the next example shows.

Cache Performance with Increased Clock Rate

Suppose we increase the performance of the computer in the previous exam ple by doubling its clock rate. Sin ce the main m em ory speed is unlikely to change, assume that the absolute time to handle a cache miss does not change. How much faster will the computer be with the faster clock, assum ing the sa me miss rate as the previous exa mple?

EXAMPLE

Measured in the fa ster clock cycles, the new miss penalty will be twice as many clock cycles, or 200 clock cycles. Hence:

ANSWER

Total miss cycles per instruction = (2% X 200)

+ 36% X ( 4% X 200) = 6.88

Thus, the faster computer with cache misses will have a CPI of 2 + 6.88 = 8.88, compared to a CPI with cache misses of 5.44 for the slower computer. Using the formula for CPU time from the previous exa mple, we ca n compute the relative performance as Performance with fa st clock Performance with slow clock

Execution time with slow clock Execution time with fast clock

IC X CPIslow dock X Clock cycle IC

CPI

5.44 8.88 X ~

fast dock X

Clock 9'cle 2

1.23

Thus, the computer with the fa ster clock is about 1.2 times faster rather than 2 times faster, which it would have been if we ignored cache misses.

496

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

As these exa m ples illustrate, relative cache penalties increase as a processor becomes faster. Furtherm ore, if a processor improves both clock rate and CPI , it suffers a double hit:

I. The lower the CPI , the more pronoun ced the impact of stall cycles. 2. The main mem ory system is unlikely to improve as fast as processor cycle time, prim arily because the perfo rm ance of the underlying DRAM is not getting much faster. When ca lculating CPI , the cache miss penalty is measured in processor clock cycles needed for a miss. Therefore, if the main mem ories of two p rocesso rs have the same absolute access times, a higher processor clock rate leads to a larger miss penalty.

Thus, th e importan ce of cache perfo rm an ce for processo rs with low C PI and high clock rates is greater, and co nsequently the danger of neglecting cache behavio r in assessing the performance of such processors is greater. As we will see in Sectio n 7.6, th e u se of fast, pipelined processo rs in desktop PCs and wo rkstation s has led to the use of sophist ica ted cache system s even in comput ers sellin g fo r less th an a $1000. Th e previou s examples and equations assume th at th e hit tim e is n ot a facto r in determining cache perfo rmance. Clea rly, if the hit time increases, the tot al time to access a word from th e m emo ry system will in crease, possibly cau sin g an increase in the p rocesso r cycle time. Alth ough we will see addi tion al exa mples of wh at ca n in crease hit time sh ortly, o ne exa mple is in creasing the cache size. A larger cache could clea rly have a lo nger access time, ju st as if your desk in the librar y was ver y large (say, 3 square meters), it wo uld t ake lo nger to loca te a book on the desk. With pipelines deeper than fi ve stages, an in crease in hit tim e likely adds another stage to th e pipelin e, sin ce it may take multiple cycles for a cache hit. Alth ough it is mo re co mplex to ca lcu late the performan ce imp act of a deep er pipelin e, at so me po int the increase in hit time for a larger cache could d ominate the imp rovem ent in hit rate, lea d ing t o a decrease in p rocesso r perfo rman ce. The next subsection discusses alternative cache organizations that decrease miss rate but may sometimes increase hit time; additional exa mples appea r in Fallacies and Pitfalls (Section 7.7).

Reducing Cache Misses by More Rexible Placement of Blocks So far, when we place a block in the cache, we have used a simple placement scheme: A block can go in exactly one place in the cache. As mentioned ea rlier, it

7.3

Measuring and Improving Cache Performance

is ca lled direct mapped because there is a direct mapping from any block address in memory to a single locatio n in the upper level of the hiera rchy. There is actually a whole range of schemes fo r placing blocks. At o ne extreme is direct mapped, where a block ca n be placed in exactly o ne locatio n. At the other extreme is a schem e where a block can be placed in any location in the cache. Such a scheme is ca lled fully associative because a block in m emory may be associated with any entry in the cache. To find a given block in a fullyassociative cache, all the entries in the cache must be sea rched because a block ca n be placed in anyone. To make the sea rch practical, it is done in parallel with a com parator associated with each cache entry. These comparators significantly increase the hardware cost, effectively ma king fully associative placement practical o nly for caches with small numbers of blocks. The middle range of designs between direct mapped and full y associative is ca lled set associative. In a set-associative cache, th ere are a fi xed number of locatio ns (at least two) where each block ca n be placed; a set -associative cache with n locatio ns for a block is ca lled an n-way set -associative cache. An n-way set -associative cache con sists of a number of sets, each of which con sists of n blocks. Ea ch block in the m em o ry m aps to a unique set in the cache given by th e ind ex fi eld, and a block can be placed in any element of that set. Thus, a setassocia tive placement combin es direct-mapped placement and full y associa tive placement: a block is directly mapped into a set, and th en all the blocks in the set are sea rched for a match. Rem ember that in a direct-mapped cache, the position of a m em ory block is given by (Block number) modulo (Number of cache blocks) In a set -associative cache, the set containing a m emory block is given by (Block number) modulo (Number of sets in the cache) Since the block m ay be placed in any element of the set, all the tags of all the elements of the set must be sea rched. In a full y associa tive cache, the block can go anywhere and all tags of all the blocks in the cache must be sea rched. For exa mple, Figure 7. 13 shows where block 12 m ay be placed in a cache with eight blocks to tal, according to the block placement policy fo r direct-mapped, two-way set -associative, and fully associative caches. We can think of every block placement strategy as a va riatio n on set associativity. Figure 7. 14 shows the possible associativity strucnlfes fo r an eight-block cache. A direct-mapped cache is simply a o ne-way set -associative cache: each

497

full y associative cache A cache structure in which a block can be placed in any location in the cache.

set-associative cache A cache that has a fixed number oflocations (at least two) where each block can be placed.

498

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

Set a ssociative

Direct mapped Block # 0 1 2 34 5 67

Data

T, g

Set #

T,g

Data

Fully ass ociativ e

T, g

FIGURE 7.13 The location of a memory block whose address Is 12 In a cache with 8 blocks varies for dlrect-mapped, setassociative, and fully associative placement. In direct-mapped placement, there is only one cache block where memory block 12 can be found, and that block is given by ( 12 mooulo 8) = 4. In a lm>-wayset-associative cache, there would be four sets, and memory block 12 must be in set ( 12 mod 4) = 0; the memory block could be in either element of the set. In a fully associative placement, the memory block for block address 12 can appear in any oflhe eight cache blocks.

cache entry holds one block and each set has one element. A full y associa tive cache with 111 entries is simply an /1l-way set-associative cache; it has one set with 111 blocks, and an ent ry ca n reside in any block within that set. The adva ntage of increasing the degree of associa tivity is th at it usually decreases the miss rate, as the next exa mple shows. The main disa dva ntage, which we discuss in more detail shortly, is an increase in the hit time.

7.3

499

Measuring and Improving Cache Performance

One-way set associative (di rec t mapped) Block

Tag Data

Two-way set ass ociati ve

2 3

5 6

Set

Tag Data Tag Data

o 1

2 3

7 Fo ur- way se t associati ve Sol

o 1

Tag Data Tag Data Tag Data Tag Data

I EE I EE I Eight-way s et associati ve (full y associati ve)

Tag Data Tag Data Tag Data Tag Data Tag Data Tag Data Tag Data Tag Data

00_ -----,DOD

OOL-I

FtGURE 7.14 An elght-block cache configured as direct mapped, two-way set associative, four-way set associative, and fully associative. The total size of the cache in blocks is equal to the munber of sets times the associativity. Thus, for a fixed cache size, increasing the associativity decreases the nwnber of sets, while increasing the munber of elements per set. With eight blocks, an eightway set-associative cache is the same as a fully associative cache.

Misses and Associativity in Caches

Assume there are three small caches, each consisting of four one-word blocks. One cache is fully associative, a second is two-way set associative, and the third is direct mapped. Find the number of misses for each cache organization given the following sequence of block addresses: 0, 8, 0,6,8.

EXAMPLE

500

Chapter 7

ANSWER

Large and Fast: Exploiting Memory Hierarchy

The direct-mapped case is easiest. First, let's determine to which cache block each block address maps: Block address

Cache block

(0 modulo 4) '" 0

(6 modulo 4) '" 2

(S modulo 4) '" 0

Now we ca n fill in the cache contents after each reference, using a blank entry to mea n that the block is invalid , colored text to show a new entry added to the cache for the associate reference, and a plain text to show an old entry in the cache:

Address of memory block accessed 0

Contents of cache blocks after reference

•

miss

Memory(O]

miss

Memory(8]

miss

Memory(O]

miss

Memory[O)

Memory[6)

miss

Memory(8]

Memory[6]

The direct-mapped cache generates five misses for the fi ve accesses. The set-associative cache has two sets (with indices 0 and 1) with two elements per set. Let's first determine to which set each block address maps: Block address

Cache set

(0 modulo 2) '" 0

(6 modulo 2) _ 0

(8 modulo 2) _ 0

Because we have a choice of which entry in a set to replace on a miss, we need a replacement rule. Set-associa tive caches usually replace the least recently used block within a set; that is, the block that was used furth est in the past is replaced. (We will discuss replacement rules in m ore detail shortly. ) Using this replacement rule, the contents of the set-associative cache after each reference looks like this:

7.3

501

Measuring and Improving Cache Performance

Address of memory block accessed 0

Contents of cache blocks after reference Set 0

Set 0

miss

Memory(O]

miss

Memory[O)

Memory[8)

hit

Memory[O)

Memory[8]

miss

Memory[O)

Memory[6)

miss

Memory(8]

Memory[6]

Set 1

Notice that when block 6 is referenced, it replaces block 8, since block 8 has been less recently referenced than block o. The two-way set -associative cache has four misses, one less th an the direct -mapped cache. The fully associative cache has four cache blocks (in a single set); any memory block can be stored in any cache block. The fully associative cache has the best performance, with only three misses:

Address of memory block accessed 0

Contents of cache blocks after reference Block 0

Block 1

Block 2

miss

Memory(O]

miss

Memory[O)

Memory[8)

hit

Memory[O)

Memory[8]

miss

Memory[O)

Memory[8]

Memory(6)

hit

Memory[O)

Memory[8]

Memory[6]

Block 3

For this series of references, three misses is the best we ca n do because three unique block addresses are accessed. Notice that if we had eight blocks in the cache, there would be no replacements in the two-way set -associative cache (check this for you rself) , and it would have the same number of misses as the fully associative cache. Similarly, if we had ]6 blocks, all three caches would have the sa me number of misses. This change in miss rate shows us that cache size and associativity are not independent in determining cache performance.

How much of a reduction in the miss rate is achieved by associativity? Figure 7.] 5 shows the improvement for the SPEC2000 bendlmarks for a 64 KB data cache with a ]6-wo rd block, and associativity ranging from direct mapped to eight -way. Going from one-way to two-way associativity decreases the miss rate by about ]5%, but there is little furth er improvement in going to higher associativity.

502

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

Associativity

Data miss rate

10.396

8 .6%

8 .3%

8 .1%

FIGURE 7.15 The data cache miss rates for an organization like the Intrlnslty FastMATH processor for SPEC2000 benchmarks with associativity varying from one-way to eight· way. These results for 10 SPEC2000 programs are from Hennessy and Patterson [2003 ].

Locating a Block in the Cache Now, let's consider the task of findin g:l block in :I cache that is set associative. Just as in a direct -mapped cache, each block in a set -associative cache includes an address tag that gives the block address. The tag of every cache block within the appropriate set is checked to see if it m atches the block address from the processor. Figure 7.1 6 shows how the address is decomposed. The index value is used to select the set containing the address of interest, and the tags of all the blocks in the set must be sea rched. Because speed is of the essence, all the tags in the selected set a re sea rched in par allel. As in a fully associa tive cache, a sequential search would m ake the hit time of a set -associative cache too slow. If th e to tal cache size is kept the sa m e, increasin g the associativity increases the number of blocks per set , which is the number of simultaneou s compa res needed to perfo rm the sea rch in pa rallel: each increase by a facto r of two in associa tivity doubles the number of blocks per set a nd halves the number of sets. Acco rdingly, each facto r-o f-two in crease in associa tivity decreases the size of the index by I bit a nd increases the size of the tag by I bit. In a fully associative cache, there is effectively o nly o ne set, and all the blocks must be checked in pa rallel. Thus, there is no ind ex, a nd the entire address, excluding the block offset, is compa red aga inst the tag of ever y block. In o ther wo rds, we sea rch th e entire cache witho ut any indexing. In a direct -m apped cache, such as in Figu re 7.7 o n page 478, only a single com pa rato r is needed, because the entry ca n be in only o ne block, and we access the cache simply by indexing. Figure 7.1 7 shows that in a four-way set-associa tive cache, fo ur comparato rs a re needed, together with a 4-to- 1 multiplexor to choose

T, g

Index

Block Offset

FIGURE 7.16 The three portions of an address In a set·assoclatlve or dlrect-mapped cache. The index is used to seleC1 t he set, then the tag is used to choose the block by comparison with the blocks in the selected set. The block offset is the address of t he desired data within the block.

7.3

503

Measuring and Improving Cache Performance

Addre ss 3 1 30 "'1 2 111 098 "' 3210

I 22

I I 8

T,g Index Index

Tao

Data

V T,o

Data

Tao

Data

Tao

Data

0 1

2 C-

lOT

C-

lOT

C-

253 25' 255 22 ~

4-t0-1 multiplexor

Hit

Data

FIGURE 7.17 The Implementation of a four-way set-associative cache requires four comparators and a 4-to-l multiplexor. The comparators determine which element of the selected set (if any) matches the tag. The output of the comparators is used to select the data from one of the four blocks of the indexed set, using a multiplexor with a decoded select signal. In some implementations, the Output enable signals on the data portions of the cache RAMs can be used to select the entry in the set that drives the output. The Output enable signal comes from the comparators, causing the element that matches to drive the data outputs. This organization eliminates the need for the multiplexor.

among the four potential members of the selected set. The cache access consists of indexing the appropriate set and then searching the tags of the set. The costs of an associative cache are the extra comparators and any delay imposed by having to do the compare and select from among the elements of the set. The choice among direct-mapped, set -associative, or fully associative mapping in any memory hierarchy will depend on the cost of a miss versus the cost of implementing associativity, both in time and in extra hardware.

504

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

Size of Tags versus Set Associativity

EXAMPLE

ANSWER

Increasing associativity requires more comparato rs, and m o re tag bits per cache block. Assuming a cadle of 4K blocks, a four-word block size, and a 32-bit add ress, find the total number of sets and the total number of tag bits for caches that are direct m apped, two-way and fo ur-way set associative, and fully associative.

Since there are 16 (=24) bytes per block, a 32-bit address yields 32 - 4 = 28 bits to be used fo r ind ex and tag. The direct -mapped cache has the sa me number of sets as blocks, and hence 12 bits of index, sin ce log2(4K) = 12; hence, the total number of tag bits is (28 - 12) X 4K = 16 X 4K = 64 Kbits. Each degree of associativity decreases the number of sets by a factor of two and thus decreases the number of bits used to index the cadle by one and increases the number of bits in the tag by o ne. Thus, fo r a two-way set-associative cache, there are 2K sets, and the total number of tag bits is (28- 11 ) X2x 2K= 34 X2K= 68 Kbits. Fo r a four-way set-associative cache, the total number of sets is IK, and the total number of tag bits is (28 - 10) X4 X IK = 72 X IK = 72 Kbits. For a fully associative cache, there is only o ne set with 4K blocks, a nd the tag is 28 bits, lea ding to a to tal of 28 X 4K X I = 11 2K tag bits .

Choosing Which Block to Replace

least recentl y used (LRlJ) A replacement scheme in which the block replaced is the one that has been unused for the longest time.

When a miss occurs in a direct-m apped cache, the requested block ca n go in exactly o ne positio n , and the block occupying that position must be repla ced. In an associative cache, we have a choice of where to place the requested block, and hence a choice of which block to replace. In a fully associative cache, all blocks are ca ndidates for repla cem ent. In a set-associative cache, we must choose am o ng the blocks in the selected set. The most commo nly used schem e is least r ecently used (LRU), which we used in the previous exa m ple. In an LRU schem e. The block repla ced is the o ne th at has been unused for the lo ngest time. LRU replacem ent is implem ented by keeping track of when each element in a set was used relative to the o ther elem ents in the set. Fo r a two-way set -associative cache, tracking when the two elements were used can be implem ented by keeping a single bit in each set and setting the bit to indicate a n elem ent whenever that elem ent is referenced. As associativity increases, implem enting LRU gets ha rder; in Sectio n 7.5, we will see an alternative schem e for replacement.

7.3

505

Measuring and Improving Cache Performance

Reducing the Miss Penalty Using Multilevel Caches All m odern computers m ake use of cach es. In m ost cases, these caches are implemented o n the sam e die as the m icrop rocessor that form s the p rocessor. To further close the gap between the fast clock rates o f m o dern p rocessors and the relatively lo n g time required to access DRAM s, m any microp rocessors support an additio n al level o f caching. This second -level cach e, which can b e o n the sa m e chip o r off-chip in a sep a rate set o f SRAMs, is accessed when ever a miss occurs in the prima ry cach e. If the second -level cach e contains the desired d ata, the miss penalty for the first-level cach e wiII be the access time o f the second -level cache, wh ich will b e much less than the access time o f m ain m em or y. If neither the pri m ary n o r seconda ry cach e contains the d ata, a m ain m em o ry access is required , a nd a larger miss pen alty is incurred. How significant is the perfo rma nce improvem ent fro m the use of a secondary cach e? The next exa mple shows us.

Performance of Multilevel Caches Suppose we h ave a p rocesso r with a base C PI o f 1.0, assuming all referen ces hit in the prima ry cache, and a clock rate o f 5 G Hz. Assume a m ain mem o ry access time of 100 ns, including all the miss h andling. Suppose the miss rate per instructio n at the primary cach e is 2% . How much faster wiII the p rocesso r be if we add a secondary cach e that h as a 5 ns access time for either a hit or a miss and is large en o u gh to reduce the miss rate to m ain m em ory to 0.5% ?

EXAMPLE

The miss pen alty to m a in mem ory is

ANSWER

100 ns ~d~; 0.2 n.

(;

dock cyd..

500 clock cycles

506

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

The effective CPI with o ne level of caching is given by T o tal CPI = Base CPI + Memory-stall cycles per instructio n For the processor with o ne level o f caching, Total C PI = 1.0 + Memory-stall cycles per instructio n = 1.0

+ 2% x 500 =

11.0

With two levels of cache, a miss in the primary (or first-level) cach e ca n be satisfied either by the secondary cach e or by main m em ory. The miss penalty for an access to the second-level cach e is 5 ns 0.2

25 clock cycles

ns dock cyd~

If the miss is satisfied in the secondary cache, then this is the entire miss penalty. If the miss needs to go to main mem ory, then the total miss penalty is the sum o f the secondary cache access time and the main m emory access time. Thus, for a two- level cach e, total CPI is the sum of the stall cycles from both levels of cach e and the base CPI: Total C PI = 1 + Primary stalls per instructio n + Secondary stalls per instructio n = 1 + 2% x 25 + 0.5% x 500 = 1 + 0.5

+ 2.5 = 4.0

Thus, the processor with the secondary cache is faster by 11.0 = 2.8

4.0 Alternatively, we could have computed the stall cycles by summing the stall cycles of those references that hit in the secondary cach e ((2% - 0.5%) x 25 = 0.4) and those references that go to main memory, which must include the cost to access the seco nda ry cache as well as the main memory access time (0.5% x (25 + 500) = 2.6). The sum , 1.0 + 0.4 + 2.6, is again 4.0.

7.3

507

Measuring and Improving Cache Performance

The design considerations for a primary and secondary cache are significa ntly different because the presence of the other cache changes the best choice versus a single-level cache. In particular, a two-level cache structure allows the primary cache to focus on minimizing hit time to yield a shorter clock cycle, while allowing the secondary cache to focus on miss rate to reduce the penalty of long mem o ry access times. The interaction of the two caches permits such a focus. The miss penalty of the primary cache is significantly reduced by the presence of the secondary cache, allowing the primar y to be smaller and have a higher miss rate. For the secondary cache, access time becomes less important with the presence of the primary cache, since the access time of the secondary cache affects the miss penalty of the pri mary cache, rather than directly affecting the primary cache hit time o r the processo r cycle time. The effect of these changes on the two caches ca n be seen by comparing each cache to the optimal design for a single level of cache. In comparison to a singlelevel cache, the primary cache of a multilevel cach e is often smaller. Furthermore, the primary cache oft en uses a smaller block size, to go with the smaller cache size and reduced miss penalty. In comparison , the secondary cache will oft en be larger than in a single-level cache, since the access time of the secondary cache is less crit ical. With a larger total size, the secondary cache often will use a larger block size than appropriate with a single-level cache

In Chapter 2, we saw that Qu ickso rt had an algorithmic adva ntage over Bubble Sort that could not be overcome by language or compiler optimization. Figure 7.18(a) shows instructions executed by item sea rched for Radix Sort versus Qu icksort. Indeed, for large arrays, Radix Sort has an algo rithmic advantage over quicksort in terms of number of operations. Figure 7. 18(b) shows time per key instea d of instructions executed. We see th at the lines start on the sa me trajectory as Figure 7.18 (a), but then the Radix Sort line diverges as the data to so rt increases. What is going on? Figure 7.1 8(c) answers by looking at the cache misses per item sorted: Qu icksort consistently has many fewer misses per item to be so rted. Alas, standard algorithmic analysis ignores the impact of the memo ry hierarchy. As faster clock rates and Moore's law allow architects to squeeze all of the performance out of a strea m of instructions, using the memory hierarchy well is critical to high performance. As we sa id in the int roduction, understanding the behavior of the memory hierarchy is critical to understanding the performance of programs on today's computers.

multilevel cach e A m em ory hierarchy with multiple levels of caches, rather than just a cache and m ain memory.

Understanding Program Performance

508

Chapter 7

Large and Fast: Exploiting Mem ory Hie rarchy

''''

"'00

Radix

sort

~ '00 • '00

0 0

•0 ~ '00

'00

Quicksort

, , ,

" " " "" '" '"

1024 2049 4096

Size (K Items to sort)

""" !

Radix

sort

"00

-•

"00

•

"•

'00 '00

=: : :---: •

Quicksort

•

, , , , Ii , ! -•• , ••• , ,, , ,• ,•

" " Size"(K Items "" to'"sort)'"

....--

1024 2049 4096

Radix sort

Quicksort

:: ~ .~ " " Size"(K Items "" to'"sort)'"

1024 2049 4096

FIGURE 7.18 Comparing Quicksort a nd Radi x Sort by (a) Instructi ons executed per Item sorte d, (b ) tim e per Item sorted, and (e) cache misses per Ite m sorted. This data is from a paper by LaMarca and Ladner [1 9% 1. Although the numbers would change for newer computers, the idea still holds. Due to such results, new versions of Radix Sort have been invented that take memory hierarch)' into accOlUlt, to regain its algorithmic advantages (see Section 7.7). The oosic idea of cache optimizations is to use aU the data in a block repeatedly before it is replaced on a miss.

7.3

Measuring and Improving Cache Performance

509

Elaboration: Multilevel caches create several complications . First, there are now several different types of misses and corresponding miss rates. In the example on page 499, we saw the primary cache miss rate and the global miss rate- the fraction of references that missed in all cache levels. There is also a miss rate for the secondary cache, which is the ratio of all misses in the secondary cache divided by the number of accesses. This miss rate is called the local miss ra te of the secondary cache. Because the primary cache filters accesses, especially those with good spatial and temporal locality, the local miss rate of the secondary cache is much higher than the global miss rate . For the example on page 499, we can compute the local miss rate of the secondary cache as: 0 .5%/2% = 25%! Luckily, the global miss rate dictates how often we must access the main memory. Additional complications arise because the caches may have different block sizes to match the larger or smaller total size . Likewise, the associativity of the cache may change. On-chip caches are often built with associativity of four or higher, while off-chip caches rarely have associativity of greater than two. On chip Ll caches tend to have lower associativity than one chip L2 caches since fast hit time is more important for Ll caches. These changes in block size and associativity introduce complications in the modeling of the caches, which typically mean that all levels need to be simulated together to understand the behavior.

Elaboration: With out-of-order processors, performance is more complex, since they execute instructions during the miss penalty. Instead of instruction miss rate and data miss rates, we use misses per instruction, and this formula : Memory stall cycles _ Misse.s X (Total miss latency - Overlapped miss latency) Instruction Instruction There is no general way to calculate overlapped miss latency, so evaluations of memory hierarchies for out-of-order processors inevitably require simulation of the processor and memory hierarchy. Only by seeing the execution of the processor during each miss can we see if the processor stalls waiting for data or simply finds other work to do. A guideline is that the processor often hides the miss penalty for an Ll cache miss that hits in the L2 cache, but it rarely hides a miss to the L2 cache.

Elaboration: The performance challenge for algorithms is that the memory hierarchy varies bet.....e en different implementations of the same architecture in cache size, associativity, block size, and number of caches . To copy with such variability, some recent numerical libraries parameterize their algorithms and then search the parameter space at runtime to find the best combination for a particular computer.

global m iss ra te The fraction of references that miss in all leve1s of a multilevel cache. local m iss ra te The fraction of references to one level ofa cache that miss; used in multilevel hierarchies.

510

Chapter 7

Check Yourself

Large and Fast: Exploiting Memory Hierarchy

Which of the following is generally true about a design with multiple levels of caches? 1. First-level caches are more concerned about hit time, and second -level caches are more concerned about miss rate.

2. First-level caches are more concerned about caches are more concerned about hit time.

iniSS

rate, and second -level

Summary In this section, we focused on three topics: cache performance, using associativity to reduce miss rates, and the use of multilevel cache hierarchies to reduce miss penalties. Since the total number of cycles spent on a program is the sum of the processor cycles and the m em ory-stall cycles, the mem ory system can have a significant effect o n program execution time. In fact, as processors get faster (by lowering CPI o r by increasing the clock rate or both), the relative effect o f the m emory-stall cycles increases, m aking good mem ory system s critical to achieving high performance. The number of m em ory-stall cycles depends o n both the miss rate and the miss penalty. The challenge, as we will see in Sectio n 7.5, is to reduce o ne of these factors without significantly affecting other critical factors in the mem ory hierarchy. To reduce the miss rate, we exa mined the use o f associative placement schemes. Such schemes ca n reduce the miss rate o f a cache by allowing more flexible placement o f blocks within the cache. Fully associative schem es allow blocks to be placed anywhere, but also require that every block in the cache be searched to sat isfy a request. This sea rch is usually implem ented by having a comparator per cache block and searching the tags in parallel. The cost o f the comparators m akes large fully associative caches impractical. Set-associative caches are a practical alternative, since we need o nly search am ong the elem ents of a unique set that is chosen by indexing. Set -associative caches have higher miss rates but are fa ster to access. The amount of associativity that yields the best performance depends on both the techn ology and the details o f the implem entatio n. Finally, we looked at multilevel caches as a technique to reduce the miss penalty by allowing a larger secondary cache to handle misses to the primary cache. Second -level caches have become comm o npla ce as designers find that limited silicon and th e goa ls o f high clock rates prevent primary caches from becoming large. Th e secondary cache, which is often 10 or m ore times larger than the primary cache, handles many accesses that miss in the primary cache. In su ch cases, the miss penalty is that of the access time to the secondary cache (typically < 10 processor cycles) versus the access time to m emory (typ ica lly> 100 processor cycles). As with associa tivity, the design trad e-o ffs between th e size o f th e

7.4

Virtual M emory

511

secondary cache a nd its access time depend o n a number of aspects of the implem entatio n.

Virtual Memory In the previous sectio n , we saw how caches p rovided fast access to recently used portio ns of a program 's code and data. Similarly, the m ain m em o ry ca n act as a "cache" fo r the second ary sto rage, usually implem ented with m agnet ic disks. This technique is ca lled virtual memory. Histo rically, there were two m ajo r m o tivatio ns for virtual m em or y: to allow effi cient and sa fe sharing of m em or y am ong multiple progra ms, and to rem ove the progra mming burdens of a sm all, limited am o unt of m ain m em o ry. Fo ur deca des aft er its inventio n, it's the fo rmer reason that reigns today. Consider a collectio n of p rogra ms running at o nce on a computer. The total mem o ry required by all the program s m ay be much larger than the am o unt of m ain m em o ry ava ilable o n the computer, but o nly a fractio n of this m em ory is actively being used at any point in time. Ma in m em o ry need contain only the active portio ns of the m any progr am s, just as a cache contain s o nly the active portio n of o ne p rogram. Thus, the principle of locality enables virnlal m em o ry as well as caches, and virnlal m em ory allows us to effi ciently share the processo r as well as the m a in m em o ry. Of course, to allow multiple p rogram s to sha re the sa m e mem o ry, we must be able to protect the program s fro m each o ther, ensuring that a program ca n o nly rea d and write the portio ns of m ain m em o ry that have been assigned to it. We ca nno t kn ow which progra m s w ill sha re th e m em ory w ith o ther progr am s whe n we co mpile them. In fact, the program s sh aring the m em o ry change dynamically while the p rogram s a re running. Beca use of this dynamic interactio n, we would like to compile each p rogra m into its own address spacesepa rate range of m em o ry loca tio ns accessible o nly to this program. Virtual m em ory implem ent s the translatio n of a p rogram 's address space to physical addresses. This tra nslatio n p rocess enforces protection of a progr am's address space from o ther p rogram s. The second m o tivatio n fo r virtual mem o ry is to allow a single user program to exceed the size of primary mem o ry. Fo rmerly, if a progr am beca m e too large fo r mem ory, it was up to the programmer to m ake it fit. Programmers divided p rogra ms into pieces and then identified the pieces that were mutually exclusive. These overlays were loaded o r unloa ded under user program cont rol during executio n , with the p rogrammer ensuring th at the program never tried to access an

. . . a system has been de vised to make the core drum combination appear to the programmer as a single level store, the requisite transfers taking place automatically. Kilburn et aL,"One-level storage system," 1962 virtual memory A technique that uses main memory as a "cache" for secondary storage.

physical address An address . . m mam memory. protection A set of mechanisms for ensuring that multiple processes sharing the processor, memory, or I/O devices cannot interfere, intentionally or unintentionally, with one another by reading or writing each other's data. These mechanisms also isolate the operating system from a user process.

512

page fault An event that occurs when an accessed page is not present in main memory.

virtual address An address that corresponds to a location in virtual space and is translated by address mapping to a physical address when memory is accessed.

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

overlay that was not loaded and that the overlays loaded never exceeded the total size of the mem ory. Overlays were traditionally organized as modules, each con taining both code and data. Ca lls between procedures in different modules would lead to overlaying of one module with another. As you ca n well imagine, this responsibility was a substantial burden on programmers. Virtual memory, which was invented to relieve programmers of this difficulty, automatically manages the two levels of the mem ory hiera rchy represented by main memory (sometimes called physical memory to distinguish it from virtual memory) and secondary storage. Although the con cepts at work in virtual mem or y and in caches are the sa me, their differing hist orical roots have led to the use of different termin ology. A virtual mem ory block is ca lled a page, and a virtual m em ory miss is called a page fault . With virtual mem ory, the processor produces a virtual address, which is translated by a combination of hardware and softwa re to a physical address, whi ch in turn ca n be used to access main memory. Figure 7. 19 shows the virtually addressed memory with pages mapped t o main mem or y. This process is ca lled address mapp ing or address translation. Today, th e two memory hiera rchy levels controlled by virtual m em or y are DRAMs

Virtual addresses

address translation Also

Physical addresses Address translation

called address mapping. The process by which a virtual address is mapped to an address lIsed to access memory.

~/ /'

• •

~ ::.!:

Disk addresses

FIGURE 7.19 In virtual memory, blocks of memory (called pages) are mapped from one set of addresses (called vITfualaddresses) to another set (called physical addresses). The processor generates virtual addresses while the memory is accessed U'iing physical addresses. Both the virtual memory and the physical memory are broken into pages, so that a virtual p.1ge is really mapped to a physical page. Of course, it is also possible for a virtual page to be absent from main memory and not be mapped to a physical address, residing instead on disk. Physical P.1ges can be shared by having two virtual addresses point to the same physical address. This capability is used to allow two different programs to share data or code.

7.4

513

Virtual M emory

and m agnetic disks (see Ch apter 1, p ages 5, 13 and 23). If we return to our library analogy, we ca n think o f a virtual address as the title of a book and a physical address as the loca tion of th at book in the library, su ch as might be given by th e Library of Congress ca ll number. Virtual memory also simplifies loading the program for execution by provid in g relocation. Reloca tion maps the virtual addresses used by a program to different physical addresses before the addresses are used to access mem ory. This relocation allows u s to load the program anywhere in main memory. Furthermore, all virtual memo ry systems in use today reloca te th e program as a set of fixed -size blocks (pages), thereby elimin ating the need to find a contiguous block of memo ry to alloca te to a program; in stead , the operating system need only find a sufficient number of pages in m ain mem ory. Fo rmerly, relocation p roblem s required special hardwa re and special suppo rt in the operating system ; today, virtual memo ry also p rovides this fun ction. In virtual memo ry, the address is broken into a virtual page number and a page offset. Figure 7.20 shows the translation of the virtual page number to a physical page number. The physical page number constitutes the upper portion of the physical address, while the page offset, which is not changed, constitutes the lower

Virtual address

313029 2827 ······················ 15 141312111 09 8 ··········· 321 0 Virtual page number

(

Page offset

Translation

29 2827 ·········

••• ••• ••• •••

15 141312111 09 8 ····

Physical page number

••••••

321 0

Page offset

Physica l address FtGURE 7.20 Mapping from a virtual to a physical address. The palle size is 2 12 - 4 KB. The number of physical palles allowed in memory is 2 18, since Ihe physical palle number has 18 bils in it. Thus, main memory can have at most 1 GB, while Ihe virtual address space is 4 GB.

514

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

portion. The number of bits in the page-offset field determines the page size. The number of pages addressable with the virtual address need not match the number of pages addressable with the physical address. Having a larger number of virtual pages than physical pages is the basis for the illusion of an essentially unbounded amount of virtual memory. Many design choices in virtual memory systems are motivated by the high cost of a miss, which in virtual memory is traditionally called a page fault. A page fault will take millions of clock cycles to process. (The table on page 469 shows that main memory is about 100,000 times faster than disk. ) This enormous miss penalty, dominated by the time to get the first word for typical page sizes, leads to several key decisions in designing virtual memory systems: • Pages should be large enough to try to amortize the high access time. Sizes from 4 KB to 16 KB are typical today. New desktop and server systems are being developed to support 32 KB and 64 KB pages, but new embedded systems are going in the other direction, to 1 KB pages. • Organizations that reduce the page fault rate are attractive. The primary technique used here is to allow fully associative placement of pages in memory. • Page faults can be handled in software because the overhead will be small compared to the disk access time. In addition , software can afford to use clever algorithms for choosing how to place pages because even small reductions in the miss rate will pay for the cost of such algorithms. • Write-through will not work for virtual memory, since writes take too long. Instead, virtual memory systems use write-back. The next few subsections address these factors in virtual memory design. Elaboration: Although we normally think of virtual addresses as much larger than physical addresses, the opposite can occur when the processor address size is small relative to the state of the memory technology. No single program can benefit, but a collection of programs running at the same time can benefit from not having to be swapped to memory or by running on parallel processors. Given that Moore's law applies to DRAM, 32-bit processors are already problematic for servers and soon for desktops.

segmentation A variable-size address mapping scheme in which an address consists of t",'O parts: a segment number, which is mapped to a physical address, and a segment offset.

Elaboration: The discussion of virtual memory in this book focuses on paging, which uses fixed-size blocks. There is also a variable-size block scheme called segmentation. In segmentation, an address consists of two parts : a segment number and a segment offset. The segment register is mapped to a physical address, and the offset is added to find the actual physical address. Because the segment can vary in size, a bounds check is also needed to make sure that the offset is within the segment. The major use

7.4

Virtual M emory

515

of segmentation is to support more powerful methods of protection and sharing in an address space . Most operating system textbooks contain extensive discussions of segmentation compared to paging and of the use of segmentation to logically share the address space . The major disadvantage of segmentation is that it splits the address space into logically separate pieces that must be manipulated as a two-part address : the segment number and the offset. Paging, in contrast, makes the boundary between page number and offset invisible to programmers and compilers. Segments have also been used as a method to extend the address space without changing the word size of the computer. Such attempts have been unsuccessful because of the awkwardness and performance penalties inherent in a two-part address of which programmers and compilers must be aware. Many architectures divide the address space into large fixed-size blocks that simplify protection between the operating system and user programs and increase the efficiency of implementing paging. Although these divisions are often called "segments," this mechanism is much simpler than variable block size segmentation and is not visible to user programs; we discuss it in more detail shortly.

Placing a Page and Finding It Again Because of the incredibly high penalty for a page fault , designers reduce page fault frequency by optimizing page placement. If we allow a virtual page to be mapped to any physical page, the operating system can then choose to replace any page it wants when a page fault occurs. For example, the operating system can use a sophisticated algorithm and complex data structures , which track page usage, to try to choose a page that will not be needed for a long time. The ability to use a clever and flexible replacement scheme reduces the page fault rate and simplifies the use of fully associative placement of pages. As mentioned in Section 7.3, the difficulty in using fully associative placement is in locating an entry, since it can be anywhere in the upper level of the hierarchy. A full search is impractical. In virtual memory systems, we locate pages by using a table that indexes the memory; this structure is called a page table and resides in memory. A page table is indexed with the page number from the virtual address to discover the corresponding physical page number. Each program has its own page table, which maps the virtual address space of that program to main memory. In our library analogy, the page table corresponds to a mapping between book titles and library locations. Just as the card catalog may contain entries for books in another library on campus rather than the local branch library, we will see that the page table may contain entries for pages not present in memory. To indicate the location of the page table in memory, the hardware includes a register that points to the start of the page table; we call this the page table register. Assume for now that the page table is in a fixed and contiguous area of memory.

page table The table containing the virtual to physical address translations in a virnlal memory system. The table, which is stored in memory, is typically indexed by the virtual page number; each entry in the table contains the physical page number for that virtual page if the page is currently in memory.

516

Chapter 7

Hardware Software Interface

Large and Fast: Exploiting Memory Hierarchy

The page table, together with the program counter and the registers, specifies the state of a program. If we wa nt to allow another program to use the processor, we must save this state. Later, after restoring this state, the program ca n continue execution. 'lVe oft en refer to this state as a process. The process is considered active when it is in possession of the processor; otherwise, it is considered inactive. The operating system ca n make a process active by loading the process's state, includ ing the program counter, which will initiate execution at the va lue of the saved program counter. The process's address space, and hence all the data it ca n access in mem ory, is defined by its page table, which resides in memory. Rather than save the entire page table, the opera ting system simply loads the page table register to point to the page table of the process it wants to make active. Ea ch process has its own page table, since different processes use the sa me virtual addresses. The operating system is responsible fo r allocating the physical memory and updating the page tables, so that the virtual address spaces of different processes do not collide. Ai; we will see shortly, the use of separate page tables also provides protection of one process from another.

Figure 7.2 1 uses the page table register, the virtual address, and the indica ted page table to show how the hardwa re ca n form a physical address. A valid bit is used in each page table entry, just as we did in a cache. If the bit is off, the page is not present in main memory and a page fault occurs. If the bit is on , the page is in memory and the entry contains the physical page number. Because the page table contains a mapping for every possible virtual page, no tags are required. In cache terminology, the index that is used to access the page table consists of the full block address, which is the virtual page number.

Page Faults If the valid bit for a virtual page is off, a page fault occurs. The operating system must be given control. This transfer is done with the exception mechanism, which we discuss later in this section. Once the operating system gets control, it must find the page in the next level of the hiera rchy (usually magnetic disk) and decide where to place the requested page in main mem ory. The virtual address alone does not immediately tell us where the page is on disk. Returning to our library analogy, we ca nn ot find the location of a library book on the shelves just by kn owing its title. Instead, we go to the ca talog and look up the book, obtaining an address for the location on the shelves, such as the Library of Congress call number. Likewise, in a virnlal memory system, we must keep tra ck of the location on disk of each page in virtual address space.

7.4

517

Virtual M emory

Page table register Virtua l address

31 3 0 2. 28 2 7 .......................... 15 14 1 3 1 2 11 1 0 Virtual page number

•

Page offset

2lJ

Valid

8 ......... 3 2 1 0

Physical page number

Page table

18 If 0 then page is not present in memory

2. 28 27 ................................. .. 1 5 14 1 3 1 2 11 10 Physical page number

•

8 ·· ...... 3 2 1 0

Page offset

Physical address FtGURE 7.21 The page table Is Indexed with the virtual page number to obtain the corresponding portion of the physical address. The starting address of the page table is given by the p.1ge table pointer. In this figure, the page size is 212 bytes, or 4 KB. The virtual address space is i 32 bytes, or 4 GB, and the physical address space is 230 bytes, which allows main memory of up to 1 GB. The nwnber of entries in the p.1ge table is 220, or I million entries. The valid bit for each emry indicates whether the mapping is legal. If it is off, then the page is not present in memory. Although the page table entry shown here need only be 19 bits wide, it would t ypicaUy be rounded up to 32 bits for ease of indexing. The ext ra bits would be used to store additional information that needs to be kept on a per-page basis, such as protection.

Because we do not know ahead of time when a page in memory will be chosen to be replaced , the operating system usually creates the space on disk for all the pages of a process when it creates the process. This disk space is called the swap space. At that time, it also crea tes a data structure to record where each virtual page is stored on disk. This data structure may be part of the page table or may be an auxiliary data strucnlfe indexed in the same way as the page table. Figure 7.22

swap space The sp ace on the disk reserved for the full virntal memory space of a process.

518

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

Virtual page number Page table Physical page or Valid disk address

Physica l memory

/, 1 1 1 1 0 1 1 0 1 1 0 1

// ~

Disk storage

-, [

I I

FIGURE 7.22 The page table maps each page In virtual memory to either a page In main memory or a page stored on disk, which Is the next level In the hierarchy. The virtual page number is used to index the page table. If the valid bit is on, th e page table supplies the physical page number (i.e., the starting address of th e page in memory) corresponding to the virtual page. If the valid bit is off, the page currently resides only on disk, at a specified disk address. In many systems, the table of physical page addresses and disk page addresses, while logically one table, is stored in two separate data structures. Dual tables are justified in part because we must keep the disk addresses of all the pages, even if they are currently in main memory. Remember th at the pages in main memory and the pages on disk are identical in size.

shows the organization when a single table holds either the physical page number o r the disk address. The operating system also creates a data structure that tracks which processes and which virtual addresses use each physical page. When a page fault occurs, if all the pages in main memory are in use, the operating system must choose a page to replace. Because we want to minimize the number of page faults, most operating systems try to choose a page that they hypothesize will not be needed in the nea r future. Using the past to predict the future, operating systems follow the least recently used ( LRU ) replacement scheme, which we mentioned in Section 7.3. The operating system searches for the least recently used page, making the assumption that a page that has not been used in a long time is less likely to be needed than a more recently accessed page. The replaced pages are written to swap space on the disk. In case you are wondering, the operating system is just another

7.4

519

Virtual M emory

process, and these tables controlling memory are in memory; the details of this seeming contradiction will be explained shortly. For example, suppose the page references (in order) were 10, 12,9,7, II , 10, and then we referenced page 8, which was not present in memory. The LRU page is 12; in LRU replacement, we would replace page 12 in main memory with page 8. If the next reference also generated a page fault, we would replace page 9, since it would then be the LRU among the pages present in memory.

Implementing a completely accurate LRU scheme is too expensive, since it requires updating a data stmcture on every memory reference. Instead, most operating systems approximate LRU by keeping track of which pages have and which pages have not been recently used. To help the operating system estimate the LRU pages, some computers provide a use bit or refer ence bit, which is set whenever a page is accessed. The operating system periodically clears the reference bits and later records them so it can determine which pages were touched during a particular time period. With this usage information, the operating system can select a page that is among the least recently referenced (detected by having its reference bit off). If this bit is not provided by the hardware, the operating system must find another way to estimate which pages have been accessed.

Elaboration: With a 32-bit virtual address, 4 KB pages, and 4 bytes per page table entry, we can compute the total page table size: Number of page table entries

2 20

Size of page table

2 20 page table entries x 22

b~~S

page ta e entry

4 MB

That is, we would need to use 4 MB of memory for each program in execution at any time . On a computer with tens to hundreds of active programs and a fixed-size page table, most or all of the memory would be tied up in page tables! A range of techniques is used to reduce the amount of storage required for the page table . The five techniques below aim at reducing the total maximum storage required as well as minimizing the main memory dedicated to page tables: 1 . The simplest technique is to keep a limit register that restricts the size of the page table for a given precess . If the virtual page number becomes larger than the contents of the limit register, entries must be added to the page table. This technique

Hardware Software Interface reference bit Also called use bit. A field that is set whenever a page is accessed and that is used to implement LRU or other replacement schemes.

520

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

allows the page table to grow as a process consumes more space . Thus, the page table will only be large if the process is using many pages of virtual address space. This technique requires that the address space expand in only one direction . 2 . Allowing growth in only one direction is not sufficient, since most languages require two areas whose size is expandable : one area holds the stack and the other area holds the heap . Because of this duality, it is convenient to divide the page table and let it grow from the highest address down, as well as from the lowest address up . This means that there will be two separate page tables and two separate limits. The use of two page tables breaks the address space into two segments. The high-order bit of an address usually determines which segment and thus which page table to use for that address . Since the segment is specified by the high-order address bit, each segment can be as large as one-half of the address space . A limit register for each segment specifies the current size of the segment, which grows in units of pages . This type of segmentation is used by many architectures, including MIPS. Unlike the type of segmentation discussed in the Elaboration on page 514, this form of segmentation is invisible to the application program, although not to the operating system . The major disadvantage of this scheme is that it does not work well when the address space is used in a sparse fashion rather than as a contiguous set of virtual addresses . 3 . Another approach to reducing the page table size is to apply a hashing function to the virtual address so that the page table data structure need be only the size of the number of physical pages in main memory. Such a structure is called an inverted page table . Of course, the lookup process is slightly more complex with an inverted page table because we can no longer just index the page table . 4. Multiple levels of page tables can also be used to reduce the total amount of page table storage. The first level maps large fixed-size blocks of virtual address space, perhaps 64 to 256 pages in total. These large blocks are sometimes called segments, and this first-level mapping table is sometimes called a segment table, though the segments are invisible to the user. Each entry in the segment table indicates whether any pages in that segment are allocated and, if so, points to a page table for that segment. Address translation happens by first looking in the segment table, using the highest-order bits of the address . If the segment address is valid, the next set of high-order bits is used to index the page table indicated by the segment table entry. This scheme allows the address space to be used in a sparse fashion (multiple noncontiguous segments can be active) without having to allocate the entire page table . Such schemes are particularly useful with very large address spaces and in software systems that require noncontiguous allocation . The primary disadvantage of this two-level mapping is the more complex process for address translation . 5 . To reduce the actual main memory tied up in page tables, most modern systems also allow the page tables to be paged . Although this sounds tricky, it works by using the same basic ideas of virtual memory and simply allowing the page tables to reside in the virtual address space. In addition, there are some small but critical

7.4

521

Virtual M emory

problems , such as a never-ending series of page faults , which must be avoided. How these problems are overcome is both very detailed and typically high ly processor specific. In brief, these problems are avoided by placing all the page tables in the address s pace of the operating system and plac ing at least some of the page tables for the system in a porti on of main memory that is physica lly addressed and is always present and thu s never on disk.

What about Writes? The difference between the access time to the cache and main memory is tens to hundreds of cycles, and write-through schemes ca n be used , although we need a write buffer to hide the latency of the write from the processor. In a virtual mem ory system , writes to the next level of the hiera rchy (disk) take millions of processor clock cycles; therefore, building a write buffer to allow the system to write th rough to disk would be completely impractical. Instea d, virtual memo ry systems must use write- back, performing the individual writes into the page in memory and copying the page back to disk when it is repla ced in the memory. This copying back to the lower level in the hiera rchy is the source of the other name for this technique of handling writes, namely, copy back.

A write- back scheme has a nother major adva ntage in a virtual memor y system. Because the disk transfer time is small compa red with its access time, copying ba ck an entire page is much mo re effi cient tha n writing individual words back to the disk. A write-back operation , alth ough more effi cient th an transferring individual words, is still costly. Thus, we would like to know whether a page needs to be copied back when we choose to replace it. To track whether a page has been written since it was read into the memory, a dirty bit is added to the page table. The dirty bit is set when any word in a page is written. If the operating system chooses to repla ce the page, the dirty bit indicates whether the page needs to be written out before its location in memory can be given to another page.

Making Address Translation Fast: The TLB Since the page tables are stored in main memory, every memory access by a program can take at least tw ice as long: one memory access to obtain the physical address and a second access to get the data. The key to improving access performance is to rely on locality of reference to the page table. When a translation for a virtual page number is used, it wiII probably be needed again in the near funlre because the references to the words on that page have both temporal and spatial locality.

Hardware Software Interface

522

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

Accordingly, modern processo rs include :I special cache that keeps track of recently used translations. This special address translation cache is traditionally referred to as a tran slation-Iookaside buffer (TLB), although it would be more accurate to call it a translation cache. The TLB corresponds to that little piece of paper we typica lly use to record the location of a set of books we look up in the ca rd catalog; rather than continually sea rching the entire catalog, we reco rd the locatio n o f several books and use the scrap o f p aper as a cache o f Library o f Con gress call numbers. Figu re 7.23 sh ows that each tag entry in the TLB ho lds a p ortion of the virtual p age number, and each data entry o f the TLB h olds a physical page number. Because we will no lon ger access the page table o n every referen ce, instead accessing the TLB, the TLB will need to include o ther bits, su ch as the dirty and the referen ce bit.

tran slation-Iookaside buffer (TLB) A cache that keeps track of recently lIsed address mappings to avoid an access to the page table.

TLB Virtual page number Valid Dirty Ref 1 1 1 1

0 1 1 0

Tag

Physical page address

1 1 1 1

Physical memory

000 1 0 1 Page table Physical page Valid Dirty Ref or disk address 1 1 1 1

1 0 0 0

1 0 0 1

~>< ;;

Dis k storage

0 00 1 0 1 1 0 1

0 00 1 1 1 1 1 1

000 1 1 1 FIGURE 7.23 The TLB acts as a cache on the page table for the entries that map to physical pages only. The TLB contmns a subset of t he virtual-to-physical page mappings that are in the page table . The T LB mappings are shown in color. Because the TLB is a cache, it must have a tag field. If there is no matching ent ry in the TLB for a p.1ge, the page table must be examined. The page table either supplies a physical page number for the page (which can then be used to build a TLB entry) or indic.1tes that the p.1ge resides on disk, in which case a p.1ge fault occurs. Since the page table has an entry for every virt ual page, no tag field is needed; in other words, it is /lot a cache,

7.4

Virtu a l M emory

On every reference, we look up the virtual page number in the TLB. If we get a hit, the physical page number is used to form the address, and the co rresponding reference bit is turned on. If the processor is performing a write, the dirty bit is also turned on. If a miss in the TLB occurs, we must determine whether it is a page fault or merely a TLB miss. If the page exists in memory, then the TLB miss indica tes only that the translation is missing. In such cases, the processor can handle the TLB miss by loading the translation from the page table into the TLB and then trying the reference again. If the page is not present in memory, then the TLB miss indicates a true page fault. In this case, the processor invokes the opera ting system using an exception. Because the TLB ha s many fewer entries than the number of pages in main memory, TLB misses will be much more frequent than true page faults. TLB misses ca n be handled either in hardwa re or in softwa re. In practice, with care there ca n be little performance difference between the two approaches because the basic operations are the sa me in either case. After a TLB miss occurs and the missing tran slation has been retrieved from the page table, we will need to select a TLB entry to replace. Because the reference and dirty bits are contained in the TLB entry, we need to copy these bits back to the page table entry when we replace an entry. These bits are the only portion of the TLB entry that ca n be changed. Using write-back-that is, copying these entries back at miss time rather than when they are written- is very efficient, since we expect the TLB miss rate to be small. Some systems use other techniques to approximate the reference and dirty bits, eliminating the need to write into the TLB except to load a new table entry on a miss. Some typical va lues for a TLB might be • TLB size: 16-5 12 entries • Block size: 1-2 page table entries (typica lly 4-8 bytes each) • Hit time: 0.5- 1 clock cycle • Miss penalty: 10-100 clock cycles • Miss rate: 0.0 1%- 1% Designers have used a wide variety of associativities in TLBs. Some systems use small, fully associative TLBs because a fully associative mapping has a lower miss rate; furthermore, sin ce the TLB is small , the cost of a full y associative mapping is not too high. Other systems use large TLBs, often with small associativity. With a fully associative mapping, choosing the entry to replace becomes tricky since implementing a hardwa re LRU scheme is too expensive. Furthermore, since TLB misses are much m ore frequent than page faults and thus must be handled more cheaply, we cannot afford an expensive softwa re algorithm, as we ca n for page

523

524

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

faults. As a result, many systems provide some suppo rt fo r randomly choosing an entry to replace. We'll examine replacement schemes in a little more detail in Section 7.5. The Intrinsity FastMATH TLB

To see these ideas in a rea l processor, let's take a closer look at the TLB of the Intrinsity FastMATH . The memory system uses 4 KB pages and a 32-bit address space; thus, the virnlal page number is 20 bits long, as in the top of Figu re 7.24. The physica l add ress is the sa me size as the virtual address. The TLB contains 16 entries, is fully associative, and is shared between the instruction and data references. Each entry is 64 bits wide and contain s a 20-bit tag (which is the virtual page number fo r that TLB entry), the co rresponding physica l page number (also 20 bits), a valid bit, a dirty bit, and other bookkeeping bits. Figu re 7.24 shows the TLB and one of the caches, while Figure 7.25 shows the steps in processing a read or write request. \-Vhen a TLB miss occurs, the MIPS hardwa re saves the page number of the reference in a specia l register and generates an exception. The exception invokes the operating system , which handles the miss in softwa re. To find the physical address fo r the missing page, the TLB miss rou tine indexes the page table using the page number of the virtual address and the page table register, which indicates the starting address of the active process page table. Using a special set of system in structions that ca n upd ate the TLB, the operating system places the physical address from the page table into the TLB. A TLB miss takes about 13 clock cycles, assuming the code and the page table entry are in the instruction cache and data cache, respectively. (We will see the M IPS TLB code on page 534) A true page fault occurs if the page table ent ry does not have a va lid physical address. The hardwa re maintain s an index th at indicates the recom mended entry to replace; the recommended entry is chosen randomly. There is an extra complication for write requests: namely, the write access bit in the TLB must be checked. This bit prevents the p rogram from writing into pages for which it has only read access. If the program attem pts a write and the write access bit is off, an exception is generated. The write access bit forms pa rt of the protection mechanism , which we discuss shortly.

Integrating Virtual Memory, TLBs, and Caches Our virtual memory and cache systems work together as a hierarchy, so that data ca nn ot be in the cache unless it is present in main memory. The operating system plays an important role in maintaining this hierarchy by flu shing the contents of any page from the cache, when it decides to migrate that page to disk. At the sa me time, the as modifies the page tables and TLB, so that an attempt to access any data on the page will generate a page fault. Under the best of ci rcumstances, a virtual address is translated by the TLB and sent to the cache where the appropriate data is found , retrieved, and sent back to the processo r. In the wo rst case, a reference can miss in all three components of

7.4

525

Virtu a l M emory

Vi rt ual address

31 30 29

14131211109 Virtual page number

·· 3 2 1 0

Page offset

}" T.g

Valid Dirty

Physical page number

TLB

TL B hit ......

• 0 0 0

Physical page number _I Page offset Phys ica l addre ss Block Physical address lag " Cache index offset

V B

B_.

offset

B 12 Valid

00..

T.g

Cache

FtGURE 7.24 11Ie TLB and cach e Imple me nt th e process of going fro m a virtu a l address t o a dat a Item In the Intrlnslty Fast· MATH. This figure shows the org.1nization of the TLB and the data cache assuming a 4 KB page size. This diagram focuses on a read; Figure 7.25 describes howto handle writes. Note that unlike Figure 7.9 on page 486, the tag and data RAMs are split. By addressing the long but narrow data RAM with the cache index concatenated with the block offset, we select the desired word in the block without a 16:1 multiplexor. While the cache is direct mapped, the TLB is fully associative. Implementing a fully associative TLB requires that every TLB tag be compared against the virtual p.1ge number, since the entry of interest can be anywhere in the TLB. If the valid bit of the matching entry is on, the access is a TLB hit, and bits from the physical page number together with bits from the page offset form the index that is used to access the cache. (The Intrinsity actually has a 16 KB page size; the FJaboration on page 528 explains how it works. )

526

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

Virtual address

TLB access

TLB miss exception

No Physical address

- - - - - -I Try to read data from cache

Write protection Cache miss stall while read block

Yes

exception

-------I Try to to write data cache

Deliver data

to the CPU Cache miss stall while read block

I-_N_o (

Write data into cache, update the dirty bit, and put the data and the address into the write buffer

FIGURE 7.25 Processing a read or a write through In the Intrinslty FastMATH TLB and cache. If the TLB generates a hit, the cache can be accessed with the resulting physical address. For a read, th e cache generates a hit or miss and supplies the data or causes a stall while the data is brought from memory. If the operation is a write, a portion of the cache entry is overwritten for a hit and the data is sent to the write buffer if we assume write-th rough . A WTite miss is jU'itlike a read miss except that the block is modified after it is read from memory. Write-back requires writes to set a dirty bit for the cache block, and a write buffer is loaded with the whole block only on a read miss or write miss if the block to be replaced is dirty. Notice that a TLB hit and a cache hit are independent events, but a cache hit can on ly occur after a TLB hit occurs., which means that the data must be present in memory. The relationship between TLB misses and cache misses is examined further in the following example and the exercises at the end of this chapter.

7.4

527

Virtual M emory

the memory hierarchy: the TLB, the page table, and the cache. The following example illustrates these interactions in more detail..

Overall Operation of a Memory Hierarchy

In a memory hierarchy like that of Figure 7.24 that includes a TLB and a cache organized as shown, a memory reference can encounter three different types of misses: a TLB miss, a page fault, and a cache miss. Consider all the combinations of these three events with one or more occurring (seven possibilities). For each possibility, state whether this event can actually occur and under what circumstances.

EXAMPLE

Figure 7.26 shows the possible circumstances and whether they can arise in practice or not.

ANSWER

Elaboration: Figure 7 .26 assumes that all memory addresses are translated to physical addresses before the cache is accessed . In this organization, the cache is physically indexed and physically tagged (both the cache index and tag are physical, rather than virtual, addresses) . In such a system, the amount of time to access memory, assuming a cache hit, must accommodate both a TLB access and a cache access; of course, these accesses can be pipelined . Alternatively, the processor can index the cache with an address that is completely or partially virtual. This is called a virtually addressed cache, and it uses tags that are virtual addresses; hence, such a cache is virtually indexed and virtually tagged . In such caches, the address translation hardware (TLB) is unused during the normal cache access, since the cache is accessed with a virtual address that has not been translated to a physical address . This takes the TLB out of the critical path, reducing cache

11111.

Possible? If so, under what circumstance?

hit

miss

Possible, although the page table is never really checked if TLB hits.

miss

hit

TLB misses, but entry found in page table; after retry, data is found in cache .

miss

hit

miss

TLB misses, but entry found in page table; after retry, data misses in cache .

miss

TLB misses and is followed by a page fault; after retry, data must miss in cache .

hit

miss

Impossible: cannot have a translation in TLB if page is not present in memory.

hit

miss

hit

Impossible: cannot have a translation in TLB if page is not present in memory.

miss

hit

Impossible: data cannot be allowed in cache if the page is not in memory.

FIGURE 7.26 The possible combinations of events In the TLB, virtual memory system, and cache. Three of these combinations are impossible, and one is possible (TLB hit, virtual memory hit, cache miss) but never detected.

virtua ll y addressed cach e A cache that is accessed with a virtual address rather than a physical address.

528

alias ing A situation in which the same object is accessed by two addresses; can occur in virtual memory when there are two virtual addresses for the same physical page.

physically addressed cach e A cache that is addressed by a physical address.

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

latency. When a cache miss occurs, however, the processor needs to translate the address to a physical address so that it can fetch the cache block from main memory. When the cache is accessed with a virtual address and pages are shared between programs (which may access them with different virtual addresses), there is the possibility of aliasing. Aliasing occurs when the same object has two names-in this case, two virtual addresses for the same page . This ambiguity creates a problem because a word on such a page may be cached in two different locations, each corresponding to different virtual addresses. This ambiguity would allow one program to write the data without the other program being aware that the data had changed . Completely virtually addressed caches either introduce design limitations on the cache and TLB to reduce aliases or require the operating system, and possibly the user, to take steps to ensure that aliases do not occur. Figure 7.24 assumed a 4 KB page size, but it's really 16 KB . The lntrinsity FastMATH uses such a memory system organization . The cache and TLB are still accessed in parallel, so the upper 2 bits of the cache index must be virtual. Hence, up to four cache entries could be aliased to the same physical memory address. As the L2 cache on the chip includes all entries in the L1 caches, on a L1 miss it checks the other three possible cache locations in the L2 cache for aliases. If it finds one, it flushes it from the caches to prevent aliases from occurring. A common compromise between these two design points is caches that are virtually indexed (sometimes using just the page offset portion of the address, which is really a physical address since it is untranslated), but use physical tags. These designs, which are virtually indexed but physically tagged, attempt to achieve the performance advantages of virtually indexed caches with the architecturally simpler advantages of a physically addressed cache. For example, there is no alias problem in this case. The L1 data cache of the Pentium 4 is an example as would the lntrinsity if the page size was 4 KB. To pull off this trick, there must be careful coordination between the minimum page size, the cache size, and associativity.

Elaboration: The FastMATH TLB is a bit more complicated than in Figure 7 .24. MIPS includes two physical page mappings per virtual page number, thereby mapping an evenodd pair of virtual page numbers into two physical page numbers. Hence, the tag is 1 bit narrower since each entry corresponds to two pages. The least significant bit of the virtual page number selects between the two physical pages. There are separate bookkeeping bits for each physical page. This optimization doubles the amount of memory mapped per TLB entry. As the Elaboration on page 530 explains, the tag field actually includes an B-bit address space ID field to reduce the cost of context switches . To support the variable page sizes mentioned on page 537, there is also a 32-bit mask field that determines the dividing line between the virtual page address and the page offset.

Implementing Protection with Virtual Memory One of the most important functions for virtual memory is to allow sharing of a single main memory by multiple processes, while providing memory protection among these processes and the operating system. The protection mechanism must

7.4

529

Virtual M emory

ensure that although multiple processes are sharing the sa me main memory, one renegade process cann ot write into the address space of another user process or into the opera ting system either intentionally or unintentionally. For exa mple, if the program that maintains student grades were running on a computer at the same time as the programs of the students in the first programming course, we wouldn't wa nt the errant program of a beginner to write over someone's grades. The write access bit in the TLB ca n protect a page from being written. Without this level of protection, computer viruses would be even more widespread.

To enable the operating system to implement protection in the virtual memory system, the hardwa re must provide at least the three basic capabilities summarized below. I. Support at least two modes that indica te whether the running process is a user process or an operating system process, variously called a super visor process, a kernel process, or an executive process. 2. Provide a portion of the processor state that a user process ca n read but not write. This includes the user/supervisor mode bit, which dictates whether the processor is in user or supervisor mode, the page table pointer, and the TLB. To write these elements the operating system uses specia l instructions that are only available in supervisor mode. 3. Provide mechanisms whereby the processor can go from user mode to superviso r mode, and vice versa. The first direction is typically accomplished by a system call exception, implemented as a specia l instruction (syscall in the M IPS instruction set) that transfers control to a dedicated location in supervisor code space. As with any other exception , the program counter from the point of the system call is saved in the exception PC (EPC), and the processor is placed in superviso r mode. To return to user mode from the exception, use the retllrn from exception ( ERET) instruction, which resets to user mode and jumps to the address in EPe. By using these mechanisms and storing the page tables in the operating system's address space, the opera ting system ca n change the page tables while preventing a user process from changing them, ensuring that a user process ca n access only the sto rage provided to it by the operating system.

We also wa nt to prevent a process from read ing th e data of another process. For example, we wouldn't want a student program to read th e grades while they were in the processor's memory. Once we begin sharing main mem ory, we must provide the ability fo r a process to protect its data from both read ing and writ in g by another process; otherwise, sharing the main mem ory will be a mixed blessing!

Hardware Software Interface kernel mode Also called su per visor mode. A mode indicating that a rulming process is an opernting system process.

syst em call A special instruction that transfers control from user mode to a dedicated location in supervisor code space, invoking the exception mechanism in the process.

530

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

Remember that each process has its own virtual address space. Thus, if the operating system keeps the page tables organized so that the independent virtual pages map to disjoint physical pages, one process will not be able to access another's data. Of course, this also requires that a user process be unable to change the page table mapping. The operating system can assure safety if it prevents the user process from modifying its own page tables. Yet , the operating system must be able to modify the page tables. Placing the page tables in the protected address space of the operating system satisfies both requirements. \Vhen processes want to share information in a limited way, the operating system must assist them, since accessing the information of another process requires changing the page table of the accessing process. The write access bit can be used to restrict the sha ring to just read sharing, and, like the rest of the page table, this bit can be changed only by the operating system. To allow another process, say PI, to read a page owned by process P2, P2 would ask the operating system to create a page table entry for a virtual page in PI's address space that points to the same physical page that P2 wants to share. The operating system could use the write protection bit to prevent PI from writing the data , if that was P2's wish. Any bits that determine the access rights for a page must be included in both the page table and the TLB because the page table is accessed only on a TLB miss. Elaboration: When the operating system decides to change from running process Pi wn text switch A changing of the internal state of the processor to allow a different process to use the processor that includes saving the state needed to renlfll to the currently executing process.

to running process P2 (called a context switch or process switch), it must ensure that P2 cannot get access to the page tables of Pi because that would compromise protection . If there is no TLB, it suffices to change the page table register to point to P2's page table (rather than to Pi's); with a TLB, we must clear the TLB entries that belong to Pi-both to protect the data of Pi and to force the TLB to load the entries for P2. If the process switch rate were high, this could be quite inefficient. For example, P2 might load only a few TLB entries before the operating system switched back to Pi. Unfortunately, Pi would then find that all its TLB entries were gone and would have to pay TLB misses to reload them . This problem arises because the virtual addresses used by Pi and P2 are the same, and we must clear out the TLB to avoid confusing these addresses . A common alternative is to extend the virtual address space by adding a process identifier or task identifier. The lntrinsity FastMATH has an B-bit address space ID (AS ID) field for this purpose . This small field identifies the currently running process; it is kept in a register loaded by the operating system when it switches processes. The process identifier is concatenated to the tag portion of the TLB, so that a TLB hit occurs only if both the page number and the process identifier match . This combination eliminates the need to clear the TLB. except on rare occasions. Similar problems can occur for a cache, since on a process switch the cache will contain data from the running process . These problems arise in different ways for physically addressed and virtually addressed caches, and a variety of different solutions, such as process identifiers, are used to ensure that a process gets its own data .

7.4

Virtual M emory

Handling TLB Misses and Page Faults Although the tra tlslation of virtual to physical addresses with a TLB is straightforwa rd when we get a TLB hit, handling TLB misses and page faults a re more com plex. A TLB miss occurs when no entry in the TLB matches a virtual address. A TLB miss ca n indicate one of two possibilities: I. The page is present in memory, and we need only create the missing TLB entry. 2. The page is not present in memory, and we need to transfer control to the operating system to deal with a page fault. How do we know which of these two circumstances has occurred? \Vhen we p rocess the TLB miss, we will look for a page table entry to bring into the TLB. If the matching page table entry has a va lid bit that is turned off, then the corresponding page is not in memo ry and we have a page fault, rather than just a TLB miss. If the valid bit is on, we can simply retrieve the desired entry. A TLB miss ca n be handled in soft wa re or hardwa re because it will require only a short sequence of operations to copy a valid page table entry from memory into the TLB. MIPS traditioll3lly ha ndles a TLB miss in softwa re. It brings in the page table entry from memory and then reexecutes the instruction that caused the TLB miss. Upon reexecuting it will get a TLB hit. If the page table entry indicates the page is not in memo ry, this time it will get a page fault exception. Handling a TLB miss or a page fault requires using the exception mechanism to interrupt the active process, transferring control to the operating system, and later resuming execution of the interrupted process. A page fault will be recognized sometime during the clock cycle u sed to access memory. To restart the instruction after the page fault is handled , the p rogram counter of the instruction that ca used the page fault must be saved. Just as in Chapters 5 and 6, the exception progra m counter (EPC) is used to hold this value. In addition, a TLB miss or page fault exception must be asserted by the end of the sa me clock cycle that the memory access occurs, so that the next clock cycle will begin exception p rocessing rather than continue normal instruction execu tion. If the page fault was not recognized in this clock cycle, a load instruction could overwrite a register, and this could be disast rous when we t ry to resta rt the instruction. For example, consider the in struction 1w $1 , 0 ( $1 ): the computer must be able to prevent the write pipeline stage from occurring; otherwise, it could not pro perly resta rt the instruction, sin ce the contents of $1 would have been dest royed. A similar complication arises on stores. \Ve must prevent the write into memory from actually completing when there is a page fault; this is usually done by deasserting the write cont rol line to the memory.

531

532

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

CPO register number

Description

14 13

INhere to restart after exception

BadVAddr

Address that caused exception

Index

Location in TLB to be read or written

Random

Pseudorandom location in TLB

EntryLo

P~sical

EntryHi

Context

EPC Cause

Cause of exception

page address and flags

Virtual page address

Page table address and page number

FIGURE 7.27 MIPS control registers. These are considered to be in coprocessor 0, and hence are read using mf cO and written using mt cO.

Hardware Software Interface exceptio n enable Also called interrupt enable. A signal or action that controls whether the process responds to an exception or not; necessary for preventing the occurrence of exceptions during intervals before the processor has safely saved the state needed to restart.

Between the time we begin executing the exception handler in the opera ting system and the time that the operatin g system has saved a11 the state of the process, the operating system is particularly vulnerable. For exa mple, if another exception occurred when we were processing the first exception in the operating system, the control unit would overwrite the exception program counter, m aking it impossible to return to the instruction that caused the page fault ! We can avoid this disaster by providing the ability to disable and en able exception s. When an exception first occurs, the processor sets a bit that disables all other exceptions; this could happen at the sa me time the processor sets the supervisor m ode bit. The operating system will then save just enough state to allow it to recover if another exception occurs-namely, the exception program counter and Cause register. EPC and Cause are two of the special control registers that help with exceptions, TLB misses, and page fault s; Figure 7.27 shows the rest. The operating system ca n then reenable exceptions. These steps make sure that exceptions will not ca use the processor to lose any state and thereby be unable to restart execution of the interrupting instruction.

O nce the operating system knows the virtual address that caused the page fault , it must complete three steps: I. Look up the page table entry using the virtual address and find the location of the referenced page on disk. 2. Choose a physical page to replace; if the chosen page is dirty, it must be written out to disk before we can bring a new virnlal page into this physical page. 3. Start a read to bring the referenced page from disk into the chosen physical page.

7.4

Virtual M emory

533

Of course, this last step will take millions of p rocessor clock cycles (so will the second if the replaced page is dirty); accordingly, the operating system will usually select another process to execute in the p rocessor until the disk access completes. Because the operating system has saved the state of the process, it ca n freely give cont rol of the processor to another process. \-¥hen the read of the page from disk is complete, the operating system can restore the state of the process that originally caused the page fault and execute the instruction that returns from the exception. This instruction will reset the processor from kernel to user mode, as well as restore the program counter. The user process then reexecutes the instruction that faulted , accesses the requested page successfully, and continues execution. Page fault exceptions for data accesses are difficult to implement properly in a processo r because of a combination of three characteristics: I. They occur in the middle of instructions, unlike instruction page faults. 2. The instruction cannot be completed before handling the exception. 3. Aft er handling the exception , the instruction must be restarted as if nothing had occurred. Making instructions restartable, so that the exception ca n be handled and the instruction later continued, is relatively easy in an architecture like the MIPS. Because each instruction writes only one data item and this write occurs at the end of the instruction cycle, we ca n simply prevent the instruction from complet ing (by not writing) and restart the instruction at the beginning. Fo r p rocessors with much more complex in structions that may tou ch many memor y locations and write many data items, making in structions restartable is much hard er. Processing one in struction m ay generate a number of page faults in the middle of the instruction. For exa mple, some p rocessors have block move in structions that touch thousa nds of data words. In such processo rs, instructions oft en ca nn ot be restarted from th e beginnin g, as we do fo r MIPS instructions. Instead , the in struction must be interrupted and later continued midstrea m in its execution. Resumin g an instruction in the middle of its execu tion usually requires saving some special state, p rocessing the exception , and resto ring th at specia l state. Making this wo rk properly requires ca reful and detailed coordination between the exception -handling code in the operating system and the hardwa re. Let's look in more detail at MIPS. Wh en a TLB miss occurs, the MIPS hardware saves the page number of the reference in a special register called Bild VAdd ra nd generates an exception. The exception invokes the operating system, which handles the miss in software. Control is transferred to address 8000 Ooo~ex' the location of the TLB miss handIer. To find the physical address for the missing page, the TLB miss routine indexes

restartable instruction An instruction that can resume execution after an exception is resolved without the exception's affecting the result of the instruction.

handler Name of a software routine invoked to "handle" an exception or interrupt.

534

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

the page table using the page number of the virtual address and the page table register, which indicates the starting address of the active process page table. To make this indexing fast, MIPS hardwa re places everything you need in the special Contex t register: the upper 12 bits have the address of the base of the page table and the next 18 bits have the virtual address of the missing page. Each page table entry is one word, so the last 2 bits are O. Thus, the first two instructions copy the Context register into the kernel temporary register $kl and then load the page table entry from that address into $ kl. Recall that $ kO and $ kl are reserved for the operating system to use without saving; a major reason for this convention is to make the TLB miss handler fast. Below is the MIPS code for a typical TLB miss handler: TLBmiss : mfcO $kl , Context

lkl . Dllkl)

mtcO $kl , EntryLo tlbwr eret

# # # # #

copy address of PTE into temp $kl put PTE into temp $kl put PTE into special register EntryLo put EntryLo into TLB entry at Random return from TLB miss exception

As shown above, MIPS has a special set of system in structions to update the TLB. The instruction tl bwr copies from control register EntryLo into the TLB entry selected by the control register Random. Random implements random replacement, so it is basically a free-running counter. A TLB miss takes about a dozen clock cycles. Note that the TLB miss handler does not check to see if the page table entry is va lid. Because the exception for TLB entry missing is much m ore frequent than a page fault, the operating system loads the TLB from the page table without exa m ining the entry and restarts the instruction. If the entry is invalid, another and different exception occurs, and the operating system recognizes the page fault. This method m akes the frequent case of a TLB miss fast, at a slight performance penalty for the infrequent case of a page fault. O nce the process that generated the page fault has been interrupted, it tran sfers control to 8000 0 180hex , a different address than TLB miss handler. This is the general address for exception; TLB miss has a special entry point to lower the penalty for a TLB miss. The operating system uses the exception Cause register to diagnose the cause of the exception. Because the exception is a page fault, the operating system knows that extensive processing will be required. Thus, unlike a TLB miss, it saves the entire state of the active process. This state includes all the general-purpose and fl oating-po int registers, the page table address register, the EPC, and the exception Cause register. Since exception handlers do not usually use the floa ting-point registers, the general entry point does not save them , leaving that to the few handlers that need them. Figure 7.28 sketches the MIPS code of an exception handler. Note that we save and restore the state in MIPS code, taking care when we enable and disable exceptions, but we invoke C code to handle the particular exception.

7.4

535

Virtu a l M emory

Save state

Save GPR

addi

" " Save Hi,

mfhi

mfl 0

" " mfcO "

Save Exception Registers

mfcO

"move

Set sp

Skl. $sp. -XCPSIZE If save space on s t ac~ f or stat e Ssp. XCCSP(Skll If save Ssp on s t ac~ SVO. XCCVO(Skll If save SvO on s t ac~ II save Svl. $di. $si . Hi .... on Sra, XCCRA($U ) II save Sra on s t ac~ II copy Hi II copy Lo SVO. XCCH](Skll II save Hi value on s ta c ~ Sv!. XCT L](Skll II save Lo value on sta ck SaO. II copy ca use r egis t er SaO. XCCCR(Skll II save Scr value on stac~ II save Svl. .... Sa 3. II copy Status Regi ster Sa 3. XCT SR(Skll II save Ssr on stac~ Ssp. II sp - sp - XCPSIZE

s ta c ~

,,0 ,,1

,,, ,,, '"

Enable nested exceptions

and i mt cO

SvO. Sa3. MAS Kl SvO. Ssr

If $vO - $s r & MASKl. ena ble exceptions If $sr - val ue th at enables exceptions

Call C exception handler

Set $gp

move move

Call C code

Sgp. GPINIT SaO. ',p xcpt deliver

II set $gp to poi nt to heap ar ea II ar~I - pointer to except ion sta ck II ca 1 C code t o hand l e exceptio n

Restoring state

Res t ore mos t GPR. Hi. 10 Restore Status Register

move

" " "1; ,,' mtcO

Sa t. ',p Sra, XCT RA($a t ) SaO. SVO. Sv l. SvO. SvO.

XCT AO($U) XCCSR($at ) MASK2 SvO.

,,, "1

II II II II II

temporary va lu e of $sp r es t ore $ra fr om sta ck r es t ore HO. • •••• r es t ore $aO fr om sta ck load old Ssr f ro m stac~ mask t o disable exceptions Z $vO - $s r & MASK2. disenable exceptions II set Statlls Regi st er

,.1

Exception return

Res t ore $sp and rest of GPR used as tempora ry reg1 st ers Res t ore ERC and retllrn FIGURE 7.28

mt cO eret

Ssp. XCCSP($at ) SVO. XCCVO($at ) Sv l. XCCVl($at ) Skl. XCCEPC($at) Sa t . XCT AT($a t ) SkI.Sepc

,,,

If If If If If If If

r estore $sp f rom sta c k r estore $vO f rom sta c k r es t ore $vl f rom sta c k copy old Sepc f rom stac k r es t ore $at f rom stac k r estore $epc r etll rn t o interrupted in structio n

MIPS code to save and restore state on an exception.

The virtual address that caused the fault depends on whether the fault was an instmction or data fault. The address of the instmction that generated the fault is in the EPC. If it was an instmction page fault, the EPC contains the virtual address of the faultin g page; o therwise, the faulting virtual address can be computed by examining the instmction (whose address is in the EPC) to find the base register and offset field.

536

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

Elaboration: This s implified vers ion assumes that the stack pointer (sp) is valid . To unmapped A portion of the address space that cannot have page faults.

avoid the problem of a page fault during thi s low-leve l exception code , MIPS set s aside a porti on of its address s pace that cannot have page faults , called unmapped. The operating syst em places exception entry point code and the exception stac k in unmapped memory. M IPS hardware tran s lates virtual addresses 8 000 000Clt.ex t o BFFF FFFFhex to phys ical addresses simply by ignorin g the upper bits of the virtual address , thereby placing these addresses in the low part of phys ical memory. Thus , the operatin g syst em places exception entry po ints and exception stacks in unmapped memory.

Elaboration: The code in Figure 7 .28 shows the MIP&32 exception return sequence. MIPS-l uses rf e and j r instead of er et.

Summary Virtual memory is the name for the level of memory hierarchy that manages cach ing between the main memory and disk. Virtual memory allows a single p rogram to expand its address space beyond the limits of main memory. Mo re importantly, in recent computer systems virtual memory supports sharing of the main mem o ry among multiple, simultaneously active processes, which together require far more total physical m ain memory th an exists. To support sharing, virtual mem o ry also provides mechanism s for memory p rotection. Managing the memory hierarchy between main memo ry and disk is challenging beca use of the high cost of page faults. Several techniques are used to reduce the miss rate: I. Blocks, ca lled pages, are made large to take adva ntage of spatial loca lity and to reduce the miss rate. 2. The mapping between virtual addresses and physical addresses, which is implemented with a page table, is made fully associative so that a virtual page can be placed anywhere in main memory. 3. The operating system uses techniques, su ch as LRU and a reference bit, to choose which pages to replace. Writes to disk are expensive, so virtual memory uses a write-ba ck scheme and also tracks whether a page is unchanged (using a dirty bit) to avoid writing unchanged pages back to disk. The virtual m emory mechanism provides address translation from a virtual address used by th e p rogram to the physica l address space used for accessing memory. This address translation allows protected sharin g of the m ain memory and provides several additional benefit s, such as simplifying mem ory allocation. To ensure that p rocesses are protected from each other requires that only th e

7.4

537

Virtual M emory

operating system ca n cha nge the address translations, which is implemented by preventing user programs from changing the page tables. Controlled sharing of pages among processes ca n be implemented with the help of the operating system and access bits in the page table that indicate whether the user program has read or write access to a page. If a processor had to access a page table resident in memory to translate every access, virtual memory would have too much overhead and caches would be pointless! Instead, a TLB acts as a cache for translations from the page table. Add resses are then translated from virtual to physical using the tran slations in the TLB. Caches, virtual memory, and TLBs all rely on a common set of principles and policies. The next section discusses this common framework.

Alth ough virtual mem or y was invented to enable a small mem or y to act as a large one, the performance difference between disk and mem ory m ea ns that if a program routinely accesses m ore virtual m em ory than it ha s physical mem o ry it will run very slowly. Su ch a program would be continuously swapping pages between memory and disk, ca lled thrashing. Thrashin g is a disaster if it occurs, but it is rare. If yo ur program thrashes, the easiest solutio n is to run it o n a computer with m ore mem or y or buy m ore mem or y for your computer. A more co mplex ch oice is to reexamine your algorithm and data stru ctures to see if you ca n change th e loca lity and thereby reduce the number of pages that your program uses simultaneously. This set of pages is info rmally ca lled the

working set. A more common performance problem is TLB misses. Since a TLB might han dle only 32-64 page entries at a time, a program could easily see a high TLB miss rate, as the processor may access less than a quarter megabyte directly: 64 X 4 KB = 0.25 MB. For exa mple, TLB misses are often a challenge for Radix Sort. To try to alleviate this problem, most computer architectures now support variable page sizes. For example, in addition to the standard 4 KB page, MIP S hardware supports 16 KB, 64 KB, 256 KB, 1 MB, 4 MB, 16 MB, 64 MB, and 256 MB pages. Hence, if a program uses large page sizes, it ca n access more memory directly without TLB misses. The practical challenge is getting the operating system to allow programs to select these larger page sizes. Once again, the more complex solution to reducing TLB misses is to reexamine the algorithm and data structures to red uce the working set of pages.

Understanding Program Performance

538

Chapter 7

Check Yourself

Large and Fast: Exploiting Memory Hierarchy

Match the m em ory hierardw element on the left with the closest phrase on the right: 1. L1 cache

A cach e fo r a cach e

2. L2 cache

b. A cache fo r disks

3. Main memory

c. A cache for a main memory

4. TLB

d. A cache fo r page table entries

A Common Framework for Memory Hierarchies By now, you 've recognized that the different ty pes of mem o ry hiera rchies sh are a grea t d eal in commo n. Althou gh m any of the as pects o f m em ory hiera rchies differ

quantitatively, many of the policies and features that determine how a hierarchy fun ctio n s are simila r qualitatively. Figure 7.29 shows h ow som e o f the quantitative

characteristics of memory hierarchies ca n differ. In the rest of this section, we will discu ss the commo n op eratio n al asp ects o f m em ory hierarchies a nd h ow these d etermine their beh avio r. We will exa mine these p olicies as a series o f four questio n s that apply b etween any two levels o f a m em o ry hierarchy, althou gh for sim plicity we will primarily u se termin ology fo r cach es.

Question 1: Where Can a Block Be Placed? We h ave seen that block placement in the upper level o f the hierarchy can u se a range o f schemes, from direct mapped to set associative to fully associative. As m ention ed ab ove, this entire range of sch em es can be tho ught o f as variatio ns on a set -associative scheme where the number o f sets a nd the number of blocks p er set va ries: Scheme name Direct mapped

Number of sets Number of blocks in cache

Set associative

Number of blocks in cache Associativity

Fully associative

Blocks per set 1 Associat ivity (typically 2- 16) Number of blocks in the cache

The advantage o f increasing the d egree o f associativity is th at it u su ally d ecreases the miss rate. The imp rovem ent in miss rate com es fro m reducing misses that compete fo r the same loca tio n. We will exa mine these in m ore d eta il sh ortly. First, let's look at how much improvem ent is ga ined. Figure 7.30 sh ows the d ata fo r a workload consisting o f the SPEC2000 b en chmarks with cach es o f 4

7.5

539

A Common Framework for Memory Hierarchies

Feature

typical values for Ll caches

Typical values for L2 caches

typical values for paged memory

typical values for a TlB

250 - 2000

4000 - 250,000

1 6 ,000-250 ,000

16-512

250,000- 1,000 ,000 ,000

0 .25-16

4000-64,000

4-32

Total size in blocks

'EM;'

Block size in bytes

32-64

500-8000 32- 1 28

Miss penalty in clocks

10- 25

100-1000

1 0 ,ooo ,O()(}-'100 ,OOO ,OOO

10-1000

Miss rates (global for L2)

296-5%

0 .196-296

0 .00001%-0.000196

0 .0196-296

Total size in kilobytes

FIGURE 7.29 The key quantitative design parameters that characterize the major elements of memory hierarchy In a computer. These are typical values for these levels as of 2004. Although the range of values is wide, this is partially because many of the values that have shifted over time are related; for example, as caches become larger to overcome larger miss penalties, block sizes also grow.

15% 12%

~ 2 KB

•

6% 3%

- - - - - 1 "......

_ ~

, KB 8 KB

~ _ _'~ 5KB

ol--:.2"~~32~2R~~B~;,-~';~6~4 K~B~;_~';/~1~2B~K~B~~~ One-way

Two-way

Four-way

Eight-way

Associativity FIGURE 7.30 The data cache miss rates for each of eight cache sizes Improve as the associativity Increases. While the benefit of going from one-way (direct-mapped) to two-way set associative is significant, the benefits of further associativity are smaller (e.g., 1%- 10% going from t\loU-way to four-way versus 20%--30% improvement going from one-way to two-way). There is even less improvement in going from four-way to eight-way set associative, which, in turn, comes very dose to the miss rates of a fully associative cache. Smaller caches obtain a significantly larger absolute benefit from associativity because the base miss rate of a small cache is larger. Figure 7.1 5 explains how this data was collected.

KB to 5 12 KB, va rying from direct mapped to eight-way set associative. The largest ga ins are obtained in going from direct mapped to two-way set associative, which yields between a 20% and 30% reduction in the miss rate. As cache sizes grow, the relative improvement from associativity in creases only slightly; since the overall miss rate of a larger cache is lower, the opportunity fo r improving the miss rate decreases and the absolute improvement in the miss rate from associativity

540

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

shrinks significantly. The potential disadva ntages of associa tivity, as we men tioned ea rlier, are increased cost and slower access time.

Question 2: How Is a Block Found? The choice of how we locate a block depends on the block placement scheme, since that dictates the number of possible loca tions. We ca n summa rize the schemes as follows: Associativity

location method

Comparisons required

Direct mapped

index

Set associative

index the set , 5eilR:h among elements

degree of associativity

Full

5eilR:h all cache entries

size of the cache

separate lookup table

The choice among direct-mapped, set -associative, or fully associative mapping in any memo ry hiera rchy will depend on the cost of a miss versus the cost of implementing associativity, both in time a nd in extra ha rdwa re. Including the L2 cache on the chip enables much higher associativity, because the hit times a re not as critica l and the designer does not have to rely on sta ndard SRAM chips as the building blocks. Fully associative caches are prohibitive except fo r sm all sizes, where the cost of the comparators is not overwhelming and where the absolute miss rate improvements are greatest. In virtual memory system s, a sepa rate mapping table (the page table) is kept to index the memory. In addition to the storage required for the table, using an index table requires an extra memory access. The choice of full associativity for page placement and the extra table is motivated by four facts: I. Full associativity is benefi cial, since misses a re very expensive. 2. Full associativity allows softwa re to use sophisticated replacement schemes that are designed to reduce the miss rate. 3. The full map ca n be easily indexed with no extra ha rdware and no sea rch ing required. 4. The large page size mea ns the page table size overhead is relatively small. (The use of a sepa rate lookup table, like a page table for virnlal memo ry, is not practica l fo r a cache beca use the table would be much larger than a page table and could not be accessed quickly.) Therefore, virtual memory systems almost always use full y associative placement. Set-associative placement is often used for caches and TLBs, where the access combines indexing and the sea rch of a small set. A few systems have used direct -

7.5

A Common Framework for Memory Hierarchies

mapped caches because of their adva ntage in access time and simplicity. The adva ntage in access time occurs because findin g the requested block does not depend on a comparison. Such design choices depend on many details of the implementation , such as whether the cache is on-chip, the technology used for implementing the cache, and the critical role of cache access time in determining the processor cycle time.

Question 3: Which Block Should Be Replaced on a Cache Miss? When a miss occurs in an associative cache, we must decide which block to repla ce. In a fully associative cache, all blocks are candid ates for repla cement. If the cache is set associative, we must choose among the blocks in the set. Of course, repla cement is easy in a direct-m apped cache because there is only one ca ndidate. We have alrea dy mentioned the two primary strategies for replacement in set associative o r full y associative caches: • Random: Ca ndidate blocks are randomly selected, possibly using some hardwa re assistance. For example, MIP S supports random replacement for TLB misses. • Least recently used (LRU ): The block replaced is the one th at has been unused for the longest time.

In practice, LRU is too costly to implement for hierarchies with more than a small degree of associativity (two to four, typically), since tracking the usage information is costly. Even for four-way set associativity, LRU is often approximated- for exa mple, by keeping track of which of a pair of blocks is LRU (which requires 1 bit), and then tracking which block in each pair is LRU (which requires 1 bit per pair). For larger associa tivity, either LRU is approximated o r random replacement is used. In caches, the repla cement algorithm is in hardwa re, which means that the scheme should be easy to implement. Rand om replacement is simple to build in hardwa re, and for a two-way set -associative cache, random replacement has a miss rate about 1.1 times higher than LRU replacement. As the caches become larger, the miss rate for both repla cement strategies falls, and the absolute difference becomes small. In fact, random replacement can sometimes be better than the simple LRU approximations that are easily implemented in hardwa re. In virtual memory, some form of LRU is always approximated since even a tiny reduction in the miss rate ca n be important when the cost of a miss is enorm ous. Reference bits or equivalent fun ctionality is often provided to make it easier for the operating system to track a set of less recently used pages. Beca use misses are

541

542

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

so expensive and relatively infrequent, approximating this information primarily in software is acceptable.

Question 4: What Happens on a Write? A key characteristic of any memory hiera rchy is how it deals with writes. We have already seen the two basic options:

• Write-through: The information is written to both the block in the cache and to the block in the lower level of the memory hierarchy (main memory for a cache). The caches in Section 7.2 used this scheme. •

Write-back (also ca lled copy-back): The information is written only to the block in the cache. The modified block is written to the lower level of the hierarchy only when it is replaced. Virtual memory systems always use write-back, for the reasons discussed in Section 7.4.

Both write-back and write-through have their adva ntages. The key advantages of write-back are the following: •

Individual words can be written by the processor at the rate that the cache, rather than the m em ory, ca n accept them.

• Multiple writes within a block require o nly one write to the lower level in the hiera rchy. • When blocks are written back, the system ca n make effective use of a high bandwidth transfer, since the entire block is written. Write-through has these advantages: • Misses are simpler and ch eaper because they never require a block to be written back to the lower level. • Write-through is easier to implem ent than write-back, altho ugh to be practica l in a high -speed system, a write-through cach e will need to use a write buffer. In virtual m emory systems, o nly a write-back policy is practical because o f the lo ng laten cy o f a write to the lower level o f the hiera rchy (disk) . As processors con tinue to increase in performance at a fast er rate than DRAM-based main m emory, the rate at which writes are gen erated by a processor will exceed the rate at which the mem ory system ca n process them , even allowing for physically and logically wider m em o ries. Consequently, more and m ore cach es are using a write-back strategy.

7.5

543

A Common Framework for Memory Hierarchies

While caches, TLBs, and virtual m em ory m ay initially look very different, they rely o n the sa me two prin ciples of locality and can be understood by looking at h ow they dea l with fo ur questio ns:

Question 1: Answer:

Where ca n a block be placed ? O n e place (direct m apped ), a few places (set associative), or any place (fully associative) .

Question 2: Answer:

How is a block found? There a re four m ethod s: indexing (as in a direct -m apped cache), limited sea rch (as in a set -associative cach e), full sea rch (as in a fully associative cache), and a sepa rate lookup table (as in a p age table) .

Question 3: Answer:

Wh at block is re pla ced o n a miss? Typically, either the least recently used or a random block.

Question 4: Answer:

How a re writes h andled ? Each level in the hierarchy can use either write- th ro u gh or write-b ack.

The Three Cs: An Intuitive Model for Understanding the Behavior of Memory Hierarchies In this sectio n , we look at a m od el that p rovides in sight into the sources o f misses in a mem ory hiera rchy and h ow the misses will be affected by ch anges in the hiera rchy. We will explain the ideas in terms o f cach es, alth ou gh the ideas ca rryover directly to any other level in the hierarchy. In this m odel, all misses a re classified into on e o f three categories (the three Cs): • Compulsory misses: These are cache misses caused by the first access to a block that h as never b een in the cach e. These are also called cold-start misses. • Cap acity misses: These a re cach e misses caused when the cach e ca nno t con tain all the blocks need ed during executio n o f a p rog ram . Cap acity misses occur wh en blocks are replaced and then later retrieved . • Conmct misses: These a re cach e misses that occur in set -associative o r direct -m apped caches wh en multip le blocks com pete fo r the sa me set. Con fl ict misses are those misses in a d irect-m apped o r set -associative cach e that a re eliminated in a fully associative cach e of the sa m e size. These cach e misses are also ca lled collision misses.

The

BIG

Picture

three Cs model Acache model in which all cache misses are classified into one of three categories: compulsory misses, capacity misses, and conflict misses. compulsor y miss Also called cold start miss. A cache miss caused by the first access to a block that has never been in the cache. capacity miss A cache miss that occurs because the cache, even with full associativity, cannot contain all the block needed to satisfy the request. confl ict miss Also called collision miss. A cache miss that occurs in a set-associative or direct-mapped cache when mul· tiple blocks compete for the same set and that are eliminated in a fully associative cache of the same size.

544

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

10%

7% 6% Miss rate per type

5% 4% 3% 2% Capacity

1% 0% 4

128

256

512

1024

Cache size (KB) FIGURE 7.31 The miss rate can be broken Into three sources of misses. This graph shows the total miss rate and its components for a range of cache sizes. This data is for the SP EC2000 integer and floating-point benchmarks and is from the same source as the data in Figure 7.30. The compulsory miss component is 0.006% and cannot be seen in this graph. The next component is the capacity miss rate, which depends on cache size. The conflict portion, which depends both on associativity and on cache size, is shown for a range of associativities from one-way to eight-way. In each case, the labeled section com,>· sponds to the increase in the miss rate that occurs when the associativity is changed from the next higher degree to the labeled degree of associativity. For example, the section labeled fWrd size. The P4 has one TLB for instructions and a separate identical TLB for data, while the Opteron has both an L1 TLB and an L2 TLB for instructions and identical LI and L2 TLBs for data. Both processors provide support for large pages, which are used for things like the operating system or mapping a frame buffer. The large-page scheme avoids using a large mUlIber of entries to map a single object that is always present .

547

548

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

Characteristic

Intel Pentium P4

Ll cache organization Split instruction and data caches Ll cache size

AMDOpteron Split instruction and data caches

8 KB for data , 96 KB trac e cache for 64 KB each for instructions/ data RiSe instructions (1 2 K RISC operations)

Ll cache a ssociativity 4.way set associative

2·way set associative

Ll replacement

Approximated LRU replacement

LRU replacement

Ll block size

64 bytes

Ll write policy

Write.through

Write-back

L2 cache organization

Unified (instruction and data)

Unified (instruction and data )

L2 cache size

5 1 2 KB

1024 KB (1 MB)

L2 cache associativity Sway set associative

16.way set associative

L2 replacement

Approximated LRU replacement

L2 block size

128 bytes

64 bytes

L2 write policy

Write.tlack

Write-back

FIGURE 7.35 First-level and second-level caches In the Intel Pentium P4 and AMD Opteron. The primary caches in the P4 are physically indexed and tagged; for a discussion of the alternatives, see the Elaboration on page 527.

nonblocking (;ache A cache that allows the processor to m ake references to the cache while the cache is handling an earl ier m iss.

requested word fi rst on a miss, as described in the Elaboration on page 490. Both allow the processor to continue to execute instructions that access the data cache during a cache miss. This technique, called a nonblocking cach e, is commonly used as designers attem pt to hide the cache miss latency by using out -of-order processors. They implement two flavors of nonblocking. H it under miss allows additional cache hits during a miss, while miss IInder miss allows multiple outstanding cache misses. The aim of the first of these two is hiding some miss latency with other work, while the aim of the second is overlapping the latency of two different misses. Overlapping a large fraction of miss times for multiple outstanding misses requires a high -bandwidth memory system capable of handling multiple misses in pa rallel. In desktop systems, the memory may only be able to take limited adva ntage of this capability, but large ser vers and multip rocesso rs often have memory systems capable of handling more than one outstanding miss in pa rallel. Both microprocesso rs prefetch instructions and have a built -in hardwa re prefetd l mechanism for data accesses. They look at a pattern of data misses and use this information to try to predict the next address to start fetching the data before the miss occurs. SUd l techniques genera lly work best when accessing arrays in loops. A significant challenge facing cadle designers is to support processo rs like the P4 a nd Opteron that ca n execute more than one memory instruction per dock cyde. Multiple requests ca n be supported in the fi rst-level cache by two different techniques. The cadle can be multiported, allowing more than one simultaneous access to the same cache block. Multiported caches, however, are often too expensive, since

7.6

Real Stuff: The Pentium P4 and the AMD Opteron Memory Hierarchies

the RAM cells in a multiported memory must be much larger than single-ported cells. The alternative scheme is to break the cache into banks and allow multiple, independent accesses, provided the accesses are to different banks. The technique is similar to interleaved main memory (see Figure 7.11 on page 489). To reduce the memory traffic in a multiprocessor configuration, Intel has other versions of the P4 with much larger on-dlip caches in 2004. For example, the Intel Pentium P4 Xeon comes with third-level cache 011 chip of 1 MB and is intended for dual-processor servers. A more radical example is the Intel Pentium P4 Extreme Edition , which comes with 2 MB of L3 cache but no support for multiprocessing. These two chips are much larger and more expensive. For example, in 2004 a Precision Workstation 360 with a 3.2 GHz P4 costs about $1900. Upgrading to the Extreme Edition processor adds $500 to the price. The Dell Precision Workstation 450, which allows dual processors, costs about $2000 for a 3.2 GHz Xeon with 1 MB of L3 cache. Adding a second processor like that one adds $1500 to the price. The sophisticated memory hierarchies of these chips and the large fraction of the dies dedicated to caches and TLBs show the significant design effort expended to try to close the gap between processor cycle times and memory latency. Future advances in processor pipeline designs, together with the increased use of multiprocessing that presents its own problems in memory hierarchies, provide many new challenges for designers. Elaboration: Perhaps the largest difference between the AMD and Intel chips is the use of a trace cache for the P4 instruction cache, while the AMD Opteron uses a more traditional instruction cache . Instead of organizing the instructions in a cache block sequentially to promote spatial locality, a trace cache finds a dynamic sequence of instructions including taken branches to load into a cache block. Thus, the cache blocks contain dynamic traces of the executed instructions as determined by the CPU rather than static sequences of instructions as determined by memory layout. It folds branch prediction (Chapter 6) into the cache, so the branches must be validated along with the addresses in order to have a valid fetch . In addition, the P4 caches the micro-operations (see Chapter 5) rather than the IA-32 instructions as in the Opteron . Clearly, trace caches have much more complicated address mapping mechanisms, since the addresses are no longer aligned to power-of-two multiples of the word size . Trace caches can improve utilization of cache blocks, however. For example, very long blocks in conventional caches may be entered from a taken branch, and hence the first portion of the block occupies space in the cache that might not be fetched . Similarly, such blocks may be exited by taken branches, so the last portion of the block might be wasted . Given that taken branches or jumps occur every 5-10 instructions, effective block utilization is a real problem for processors like the Opteron , whose 64byte block would likely include 16-24 80x86 instructions . Trace caches store instructions only from the branch entry point to the exit of the trace, thereby avoiding such

549

550

Chapter 7

Large and Fast: Exploiting Mem ory Hie rarchy

header and trailer overh ead . A down s ide of t race caches is that they pot entially st ore the same instructi ons multiple times in the cache : conditional branches making different choices result in the same instructi ons being part of sepa ra te traces, which each appear in the cache . To account for both the larger s ize of the micro--operati ons and the redundancy inherent in a trace cache , Intel claims that the miss rat e of the 9 6 KB trace cache of the P4, which holds 12K micro-operati ons , is about that of an 8 KB cache , which holds about 2K-3K IA-3 2 instructi ons .

Fallacies and Pitfalls As one of the most naturally qua ntitative aspects of computer architecnlfe, the

memo ry hierarchy would seem to be less vulnerable to fallacies and pitfalls. Not only have there been many fallacies propagated and pitfalls encountered, but some have led to major negative outcomes. We start with a pitfall th at often traps students in exercises and exams. Pitfall: Fo rgetting to account fo r byte address ing or the cache block size in siml/lating a cache.

When simulating a cache (by hand or by computer), we need to make sure we account for the effect of byte addressing and multiword blocks in determining which cache block a given address maps into. Fo r example, if we have a 32-byte direct-mapped cache with a block size of 4 bytes, the byte address 36 maps into block I of the cache, since byte address 36 is block address 9 and (9 modulo 8) = I. On the other hand, if address 36 is a word address, then it maps into block (36 mod 8) = 4. Make sure the problem clea rly states the base of the address. In like fashion, we must account for the block size. Suppose we have a cache with 256 bytes and a block size of 32 bytes. \ Vhich block does the byte address 300 fall into? If we brea k the address 300 into fi elds, we ca n see the answer: 31

o I

Cache Bloc k bloc k offset number Bloc k address

7.7

551

Fallacies and Pitfalls

Byte address 300 is block address 00 l 33 2

The number of blocks in the cache is

l23526 J

Block number 9 falls into cache block number (9 mod ulo 8) = 1. This mistake catches many people, including the authors (in ea rlier drafts) and instructors who forget whether they intended the addresses to be in words, bytes, or block numbers. Remember this pitfall when you tackle the exercises.

Pitfall: Ignoring memory system behavior when writing programs or when generIlting code in a compiler. This could easily be written as a fallacy: "Progra mmers ca n ignore memory hiera rchies in writing code." We illustrate with an exa mple using matrix multiply, to complement the sort comparison in Figure 7. 18 on page 508. Here is the inner loop of the version of matrix multiply from Chapter 3: for (i = O; i! =500 ; i = i + ll for (j =O; j! = 500 ; j = j + ll for (k=O; k! = 500 ; k= k+ l) x[i][j] = x[i][j] + y[i][k] * z[k][j];

When run with inputs that are 500 x 500 double precision matrices, the CPU runtime of the above loop on a MIPS CPU with a 1 MB secondary cache was about half the speed compared to when the loop order is changed to k , j , i (so i is innermost)! The only difference is how the program accesses memory and the ensuing effect on the memory hierarchy. Further compiler optimizations using a technique called blocking ca n result in a runtime that is another four times faster for this code!

Pitfall: Using average memory access time to evaluate the memory hierarchy ofan out-of-order processor. If a processor stalls during a cache miss, then you can separa tely calculate the memory-stall time and the processor execution time, and hence evaluate the memory hierarchy independently using average memory access time. If the processor continues to execute instructions and may even sustain more cache misses during a cache miss, then the only accurate assessment of the mem ory hiera rchy is to simulate the out -of-order processor along with the memory hierarchy.

552

Chapter 7

Large and Fast: Exploiting Memory Hierarchy

Pitfall: Extending an address space by adding segments on top ofan unsegmented address space.

During the 1970s, ma ny program s grew so large that no t all the code and data could be addressed with just a 16-bit address. Computers were then revised to offer 32-bit addresses, either thro ugh an unsegm ented 32-bit address space (also called a flat address space) o r by adding 16 bits of segm ent to the existing 16-bit address. From a m arketing point of view, adding segments th at were program mer-visible a nd that fo rced the programmer and com piler to decom pose p rogram s into segm ents could solve the addressing pro blem. Unfo rtunately, there is t rouble any time a programming language wants an address that is larger than one segment, such as indices for large arrays, unrestricted pointers, or reference parameters. Mo reover, adding segments ca n turn every address into two wordsone for the segment number and one for the segment offset-causing problems in the use of addresses in registers. Given the size of DRAMs and Moo re's law, many of today's 32-bit systems are facing similar problems.

Concluding Remarks The difficulty of building a memory system to keep pace with faster processo rs is undersco red by the fact th at the raw material fo r main memory, DRAMs, is essen tially the sa me in the fastest computers as it is in the slowest and cheapest. Figure 7.36 compares the memory hierarchy of microprocessors aimed at desktop, server, and embedded applications. The L1 caches are similar across applica tions, with the primary differences being L2 cache size, die size, processor clock rate, and instructions issued per clock. It is the principle of locality that gives us a chance to overcome the long latency of memory access-a nd the soundness of this strategy is demonstrated at all levels of the memo ry hierarchy. Although these levels of the hierarchy look quite different in quantitative term s, they follow similar strategies in their operation and exploit the sa me p roperties of locality. Because processor speeds continue to imp rove faster than either DRAM access times or disk access times, memory will in creasingly be the factor that limits performance. Processors in crease in perfo rm ance at a high rate, and DRAMs are now doubling their density about every two years. The access time of DRAMs, however, is improving at a much slower rate-less than 10% per yea r. Figure 7.37 plots processor performance aga inst a 7% annual performance imp rovement in DRAM latency. \-Vhile latency improves slowly, recent enhancements in DRAM technology (double data rate DRAMs and related techniques) have led to greater

7.8

553

Concluding Remarks

AMD Opteron

Intrlnsity FastMATH

Intel Pentium 4

Intel PXA250

UltraSPARC IV

Instruction set architedure

1A·32 , AM D64

MIPS32

IA-32

ARM

SPARC v9

Intended application Die size (mm 2) (2004)

server

embedded

deskt op

10WilOwer embedded

server

122 2

217

Instructions issued/clock

193 3

3 RISC ops

356 4 ,2

Clock rate (2004)

2 .0 GHz

3 .2 GHz

0 .4 GHz

1.2 GHz

Instruction cache

6 4 KB, 2.way set associative

16 KB, direct mapped

12000 RISC op trace cache (-96 KB)

32 KB , 32.way set associative

32 KB, 4.way set associative

Latency (clocks)

Data cache

6 4 KB, 2.way set associative

16 KB,

8 KB, 4--way set associative

32 KB , 32.way set associative

64 KB, 4.way set associative

3 16

128/128

32/32

128/512

1 KB

81 disk reads (DO, P) and two disk writes ( DO', P' ), which in'·olve just two disks. Increasing the size of the parity group increases the &wings of the shortcut. RAIO 5 uses the &'\Ille shortcut.

578

Chapter 8

The key insight to reduce this overhea d is that parity is simply a sum of information; by watching which bits change when we write the new information , we need only change the corresponding bits on the parity disk. Figure 8. 7 shows the shortcut. We must read the old data from the disk being written , compare old data to the new data to see which bits change, read the old parity, change the corresponding bits, then write the new data and new parity. Thus, the small write involves four disk accesses to two disks in stead of accessing all disks. This orga ni zation is RAID 4. Distributed Block-Interleaved Parity (RAID 5)

RAID 4 effici ently suppo rts a mixtu re of large reads, large writes, and small reads, plus it allows small writes. One drawback to the system is that the parity disk must be updated on every write, so the parity disk is the bottleneck for back-to-back writes. To fi x the parity-write bottleneck, the parity info rm ation can be sprea d th roughout all the disks so that there is no single bottleneck for writes. The distributed parity orga niza tion is RAID 5. Figu re 8.8 shows how data are distributed in RAID 4 versus RAID 5. As the o rganization on the right shows, in RAID 5 the parity associated with each row of data blocks is no longer restricted to a single disk. This orga nization allows multiple writes to occur simultaneously as long as the parity blocks are not located to the sa me disk. Fo r example, a write to block 8 on the right must also access its parity block P2, thereby occupying the first and third disks. A second write to block 5 on the right, implying an update to its parity block PI , accesses the second and fourth disks and thus could occur concurrently with the write to block 8. Those sa me writes to the o rga nization on the left result in changes to blocks PI and P2, both on the fifth disk, which is a bottleneck. P + Q Redundancy (RAID 6) Parity-based schemes p rotect against a single self-identifying failure. When a single failure correction is not sufficient, parity ca n be generalized to have a second ca lculation over the data and another check disk of information. This second check block allows recovery from a second failure. Thus, the sto rage overhead is twice that of RAID 5. The small write shortcut of Figure 8.7 works as well, except now there are six disk accesses instead of four to update both P and Q information. RAID Summary RAID I and RAID 5 are widely used in servers; one estimate is 80% of disks in servers are found in some RAID system. One weakness of the RAID system s is repair. First, to avoid making the data ull3vailable during repair, the array must be designed to allow the failed disks to

8.2

Disk Storage and Dependability

~ [4]

0 ~ @] ~ .. .

0 [5] @]

0 0

@] @J

§]

.. .

. ..

RAID 4

579

~ ~ ~ ~ ~ ~

~ [4]

Q] [5]

0 0

~ ~

[5] ~

@] @J

@] @] @]

~ ~ ~

§]

. ..

.. .

...

RAIDS

FIGURE 8.8 Block.jnterleaved parity (RAID 4) versus distributed block·lnterleaved par· Ity (RAID S). By distributing parity blocks to all d isks, some small w rites can be performed in parallel.

be replaced with out having to turn off the system. RAIDs have enough redun dancy to allow continuous operation, but hot swapping disks places demands on the physical and electrica l design of the array and the disk interfaces. Second , another failure could occur during repair, so the repair time affects the chances of losing data: the longer the repair time, the greater the chances of another failure that will lose data. Rather than having to wa it for the operato r to bring in a good disk, some systems include standby spares so that the data ca n be reconstructed immediately upon discovery of the failure. The operator ca n then repla ce the failed disks in a more leisurely fashion. Third, although disk manufacturers quote very high MTTF for their products, those numbers are under nominal conditions. If a particular disk array has been subject to temperature cycles due to, say, the failure of the air conditioning system , o r to shaking due to a poo r rack design, construction , or installation , the failure rates will be much higher. The calculation of RAID reliability aSSllmes independence between disk failures, but disk failures could be correlated because such damage due to the environment would likely happen to all the disks in the array. Finally, a human operator ultimately determines wh ich disks to remove. As Figure 8. 5 shows, operators are only human, so they occasionally remove the good disk instea d of the b roken disk, leading to an unrecoverable disk failure. Although RAID 6 is rarely used today, a cautious operator might wa nt its extra redundancy to protect against expected hardware failures plus a safety margin to p rotect against human erro r and correlated failures due to p roblems with the environment.

hot swa pping Replacing a hardware component while the system is running.

standby spares Reserve hardware resources that can immedi· ately take the place of a failed component.

580

Chapter 8

Check Yourself

Which of the following a re true about dependability? I. If :l system is up, then all its components a re accomplishing their expected

service. 2. Ava ilability is a quantitative measure of the percentage of time a system is accomplishing its expected service. 3. Reliability is a quantitative measure of continuous service accomplishment by a system.

4. The major source of outages today is softwa re. \Vhich of the following a re true about RAID levels 1,3, 4, 5, and 6? I. RAID systems rely on redundancy to achieve high ava ilability. 2. RAID 1 (mirroring) has the highest check disk overhead. 3. For small writes, RAID 3 (bit-interleaved pa rity) has the worst throughput. 4. For large writes, RAID 3, 4, and 5 have the same th roughput.

Elaboration: One issue is how mirroring intera ct s with striping . Suppose you had, say, f our disks worth of data to st ore and eight physical disks to use. Would you create four pairs of disks-each organized as RAID 1-and then stripe data across the four RAID 1 pairs? Alternatively, would you creat e two set s offour disks-each organized as RAID o-and then mirror writes to both RAID 0 sets? The RAID terminology has evolved to call the former RAID 1 + 0 or RAID 10 ("striped mirrors ") and the latter RAID 0 + 1 or RAID 0 1 (" mirrored stripes").

Networks Networks a re growing in po pula rity over time, and unlike other I/O devices, there a re many books and courses on them. For readers who have not taken courses or read books on netwo rking, Section 8. 3 on the " CD gives a quick overview of the topics and terminology, including internetwo rking, the O S! model, protocol fam ilies such as TCPIl P, long- haul networks such as ATM , loca l area networks such as Ethernet , and wireless netwo rks such as IEEE 802.11.

8.4

Buses and other Connections between Processors, Memory, and i/ O Devlc:es

Buses and Other Connections between Processors, Memory, and I/O Devices In a computer system , the va rious subsystems must have interfaces to one another. Fo r example, the memory and processo r need to communicate, as do the processo r and the I/O devices. For many yea rs, this has been done with a bus. A bus is a shared communica tion link, which uses one set of wires to connect multiple subsystems. The two major advantages of the bus organization are versatility and low cost. By definin g a single connection scheme, new devices ca n easily be added , and peripherals ca n even be moved between computer systems that use the sa me kind of bus. Furthermore, buses are cost-effective because a single set of wires is shared in multiple ways. The major disa dvantage of a bus is that it crea tes a communication bottleneck, possibly limiting the maximum I/O th roughput. When I/O must pass th rough a single bus, the bandwidth of that bus limits the maximum I/O th roughput. Designing a bus system capable of meeting the demands of the processor as well as connecting large numbers of I/O devices to the machine presents a m ajor challenge. One reason bus design is so difficult is that the maximum bus speed is largely limited by physical factors: the length of the bus and the number of devices. These physica l limits prevent us from running the bus arbitrarily fast. In addition , the need to support a range of devices with widely va rying latencies and data transfer rates also makes bus design challenging. As it becomes difficult to run many parallel wires at high speed due to clock skew and refl ection , the industry is in transition from parallel shared buses to high-speed serial point -to- point interconnections with switches. Thus, such networks are gradually repla cing buses in our systems. As a result of this transition , this section has been revised in this edition to emphasize the general problem of connecting I/O devices, processors, and mem o ry rather than focus exclusively on buses.

Bus Basics Classically, a bus generally contains a set of cont rol lines and a set of data lines. The cont rol lines are used to signal requests and acknowledgments, and to indicate what type of information is on the data lines. The data lines of the bus ca rry information between the source and the destination. This information may con sist of data, complex commands, or addresses. For exa mple, if a disk wa nts to write so me data into memory from a disk sector, the data lines will be used to indicate the address in memory in which to place the data as well as to ca rry the

581

582

bus transaction A sequence of bus operations that includes a request and may include a response, either of which m ay carry data. A tran saction is initiated by a single request and m ay take m any individual bus operations.

processor-memory bus A bus that connects processor and m em ory and that is short, gen erally high speed, and m atched to the memory system so as to m aXImize mem ory-processor bandwidth. backplane bus A bus that is design ed to allow processors, m em ory, and I/O devices to coexist on a single b us.

synchronous bus A bus that includes a clock in the control lines and a fixed protocol for communicating that is relative to the clock. asynchronous bus A bus that u ses a handshaking protocol for coordinating usage rather than a clock; can accommodate a wide variety of devices of differing

speeds.

Chapter 8

actual data from the disk. The control lines will be used to indica te what type of information is contained on the data lines of the bus at each point in the transfer. Some buses have two sets of signal lines to separately communicate both data and address in a single bus transmission. In either case, the control lines are used to indicate what the bus contains and to implement the bus p rotocol. And because the bus is sha red, we also need a protocol to decide who uses it next; we will discuss this problem shortly. Let's consider a typica l bus transaction. A bus transaction includes two parts: sending the address and receiving o r sending the data. Bus transactions are typica lly defin ed by what they do to memory. A read transaction transfers data from memory (to either the processo r or an I/O device), and a write transaction writes data to the memory. Clea rly, this terminology is confusing. To avoid this, we'll try to use the terms inpl/t and Ol/tput, which are always defin ed from the perspective of the processo r: an input operation is inputting data from the device to memory, where the p rocessor ca n rea d it, and an output operation is outputting data to a device from memory where the p rocesso r wrote it. Buses are traditionally classified as processor-memory buses or 110 bl/ses. Processo r-memory buses are short, generally high speed, and matched to the memory system so as to maximize memory-p rocesso r bandwidth. I/O buses, by contrast, ca n be lengthy, ca n have many types of devices connected to them , and often have a wide range in the data bandwidth of the devices connected to them. I/O buses do not typically interface directly to the memory but use either a processor- mem o ry or a backplane bus to connect to memory. Other buses with different characteristics have emerged fo r special fun ctions, such as graphics buses. The I/O bus serves as a way of expanding the machine and connecting new peripherals. To make this easier, the computer industry has developed several standards. The standards serve as a specification for the computer manufacturer and for the peripheral manufacnlfer. A standard ensures the computer designer that peripherals will be ava ilable fo r a new machine, and it ensures the peripheral builder that users will be able to hook up their new equipment. Figure 8.9 sum marizes the key characteristics of the two dominant I/O bus stand ards: Firewire and USB. They connect a va riety of devices to the desktop computer, from keyboa rds to cameras to disks. The two basic schemes for communica tion on the bus are synchronous and asynchronous. If a bus is synchronous, it includes a clock in the cont rol lines and a fi xed protocol for communicating that is relative to the clock. For exa mple, for a processor-memory bus performing a rea d from memory, we might have a protocol that transmits the address and rea d comm and on the first clock cycle, using the cont rol lines to indicate the type of request. The memo ry might then be required to respond with the data word on the fifth clock. This type of protocol ca n be implemented easily in a small finite state machine. Because the protocol is predetermined and involves little logic, the bus ca n run very fast and the interface logic will be small.

8.4

Buses and other Connections between Processors, Memory, and I/ O Devlc:es

Characteristic: Bus type Basic data bus width (signals) Clocking Theoretical peak bandwidth

Hot plugable Maximum number of devices Maximum bus length (copper wire) Standard name

FtGURE 8.9

Flrewlre (1394)

USB 2.0

I/O

asynchronous

50 MBjsec (Firewire 400) or 100 MBjsec (Firewire 800)

0 .2 MBjsec (low speed) , 1.5 MBjsec (full speed) , or 60 MBjsec (high speed)

ye' 63

y" 127

4 .5 meters

5 meters

IEEE 1394, 1394b

USE Implementors Forum

583

Key c:haracteristlc:s of two dominant I/O bus standards.

Synch ronous buses have two major disa dva ntages, however. First, every device on the bus must run at the same clock rate. Second , beca use of clock skew problems, synch ronous buses cannot be long if they are fast (see " Appendix B for a discussion of clock skew). Processor-memory buses are often synchronous because the devices commun icating are close, small in number, and prepa red to operate at high clock rates. An asynchronous bus is not clocked. Because it is not clocked , an asynchronous bus can accommodate a wide va riety of devices, and the bus ca n be lengthened without worrying about clock skew or synchronization p roblems. Both Firewire and USE 2.0 are asynch ronous. To coo rdin ate the transmission of data between sender and receiver, an asynchronous bus uses a hand shaking protocol. A hand shaking p rotocol consists of a series of steps in which the sender and receiver p roceed to the next step only when both pa rties agree. The protocol is implemented with an additional set of cont rol lines. A simple exa mple will illustrate how asynchronous buses wo rk. Let's consider a device requesting a word of data from the memo ry system . Assume that there are three cont rol lines: I. ReadReq: Used to indicate a read request fo r memory. The address is put on the data lines at the sa me time. 2. DataRdy: Used to indicate th at the data wo rd is now rea dy on the data lines. In an output transaction , the memo ry will assert this signal since it is providing the data. In an input transaction, an I/O device would assert this signal, since it would provide data. In either case, the data is pla ced on the data lines at the sa me time.

3. Ack: Used to acknowledge the ReadReq or the DataRdy signal of the other pa rty.

hand shaking p ro tocol A series of steps used to coordinate asynch ronous b us transfers in which the sender and receiver proceed to the next step only when both parties agree that the current step has been completed.

584

Chapter 8

In an asynch ronous protocol, the cont rol signals Rea dReq and DataRdy are asserted until the other pa rty (the memory or the device) indica tes th at the CO Il t rollines have been seen and the data lines have been read; this indication is made by asserting the Ack line. This complete process is called handshaking. Figure 8.1 0 shows how such a protocol operates by depicting the steps in the communication. Although much of the bandwidth of a bus is decided by the choice of a syn ch ronous o r asynch ronous protocol and the timing characteristics of the bus, several other factors affect the bandwidth that can be attained by a single transfer. The most important of these are the data bus width , and whether it supports block transfers o r it transfers a word at a time.

ReadReq

3 Data

. ok

r---1(I~

---'

L""",'-----/ , ""7:s'-'

DataRdy The steps in the protocol begin immediately after the device signals a request by raising ReadReq and putting the address on the Data lines: 1 . When memory sees the ReadReq line, it reads the address from the data bus and raises Ac k to indicate it has been seen. 2 . I/O device sees the Ac k line high and releases the ReadReq and data lines. 3 . Memory sees that ReadReq is low and drops the Ac k line to acknowledge the ReadReq signal. 4 . This step starts when the memory has the data ready. It places the data from the read request on the data lin es and raises DataRdy. 5 . The I/O devi ce sees DataRdy, reads the data from the bus , and signals that it has the data by raising Ao k.

6 . The memory sees th e Ac k signal , drops DataRdy, and releases the data lines. 7 . Finally, the I/O device, seeing DataRdy go low, drops the Ac k line, which indicates that the transmission is completed . A new bus transaction can now begin.

FIGURE 8.10 The asynchronous handshaking protocol consists of seven steps to read a word from memory and receive It In an I/ O device. The signals in color are those asserted by the I/O device, while the memory asserts the signals shown in black. The arrows label the seven steps and the event that triggers each step. The symbol showing two lines (high and low) at the same time on the data lines indicates tha t the data lines have valid data at this point. (The symbol indicates tha t the data is valid, but the value is not known .)

8.4

585

Buses and other Connecti ons between Processor s, Me mo ry, and I/ O Devlc:es

Elaboration: Another method for increasing the effective bus bandwidth is to release the bus when it is not being used for transmitting information . This type of protocol is called a split transaction protocol . The advantage of such a protocol is that, by freeing the bus during the time data is not being transmitted, the protocol allows another requestor to use the bus. This can improve the effective bus bandwidth for the entire system if the memory is sophisticated enough to handle multiple overlapping transactions . Multiprocessors sharing a memory bus may use split transaction protocols.

The Buses and Networks of the Pentium 4 Figure 8.11 shows the I/O system of a PC based on the Pentium 4. The processor connects to peripherals via two main chips. The chip next to the processor is the memory controller hub, commonly called the north bridge, and the one connected to it is the I/O controller hub, called the south bridge.

Pentium 4 processor

DDR 400 (3.2 GBtsec) Main memory DIMMs

DDR 400 (3.2 GBtsec)

System bus (800 MHz, 604 GBtsec) AGP8X Memory (2.1 GB/sec) Graphics controller output hob CSA (north bridge) (0.266 GBlsec) 1 Gbit Ethernet ./ 82875P

Serial ATA (150 MBlsec)

(266 MBlsec) Parallel ATA (100 MB/sec)

Serial ATA (150 MBlsec)

Parallel ATA (100 MBlsec)

Disk

Disk AC/97

Stereo (surroundsound)

(1 MB/sec) USB 2.0 (60 MBlsec)

...

11O controller hob (south bridge) 82801 EB

( CDfDVD

/' T",.

" /

(20 MB/sec)

10/100 Mbit Ethern:y

( PCI bus (132 MBlsec)

FtGURE 8.11 Organizati on of th e I/O syst em on a Pentium 4 PC us ing the Intel 875 c:hlp set. Note that the maximum transfer rate between the north bridge ( memory hub ) and south bridge (IIO hub) is 266 MB/ sec, which is why Intel put the AGP bus and Gig.1bit Ethernet on the north bridge.

split transaction protocol A protocol in which the bus is released during a bus transaction while the requester is waiting for the data to be transmitted, which frees the bus for access by another requester.

586

Chapter 8

The no rth bridge is basica lly a DM A controller, connecting the processor to memory, the AG P graphic bus, and the south bridge chip. The south bridge con nects the north bridge to a co rnucopia of I/O buses. Intel and others offer a wide va riety of these chip sets to connect the Pentium 4 to the outside world. To give a flavor of the options, Figure 8.12 shows two of the chip sets. As Moore's law continues, an increasing number of I/O controllers that were form erly ava ilable as optional ca rds that connected to I/O buses have been coopted into these chip sets. For example, the south bridge chip of the Intel 875 includes a striping RA ID controller, and the north bridge chip of the Intel 845GL includes a graphics controller.

Target segment System bus (64 bit)

875P chip set

845GL chip set

Perfonnance PC

Value PC

800/533 MHz

400 MHz

Memory controller hub ("north bridge") Package sire, pins

42.5 x 4 2 .5 mm, 1005

37.5 x 37.5 mm, 760

DDR 400/333/266 SDRAM

DDR 266/ 200, PC133 SDRAM

2x72

1 x 64

4, 128/256/512 Mbits

2, 128/256/512 MBits

Maximum memory capacity

4 GB

2 GB

Memory error correE 15

g.Q)

< • w_

Interrupt mask

Branch delay

Pending interrupts

Exception code

FIGURE 8.13 Th e Cause and Statu s registe rs. This version of the Cause register corresponds to the MIPS-32 architecture. The earlier MIPS I architecture had three nested sets of kernel/user and interrupt enable bits to support nested interrupts. Section A.7 in iii Appendix A has more detials aboU1these registers.

each bit in the pending interrupt fi eld of the Cause register. To enable the corresponding interrupt, there must be a 1 in the mask fi eld at that bit position. Once an interrupt occurs, the operating system ca n find the reason in the exception code fi eld of the Status register: 0 mea ns an interrupt occurred, with other values for the exceptions mentioned in Chapter 7. Here are the steps that must occur in handling an interrupt: 1. Logically AND the pending interrupt fi eld and the interrupt mask fi eld to see which enabled interrupts could be the culprit. Copies are made of these two registers using the mfcO instruction. 2. Select the higher priority of these interrupts. The software convention that the leftmost is the highest priority.

3. Save the interrupt mask fi eld of the Status register. 4. Change the interrupt mask fi eld to disable all interrupts of equal or lower priority. 5. Save the processor state needed to handle the interrupt. 6. To allow higher-priority interrupts, set the interrupt enable bit of the Cause register to 1. 7. Call the appropriate interrupt routine. 8. Before restoring state, set the interrupt enable bit of the Cause register to O. This allows you to restore the interrupt mask field.

8.5

Interfacing I/ O Devices to the Processor, Memory, and Operating System

• Appendix A shows an exception handler for a simple I/O task on pages A-36 to A-37. How do the interrtlpt priority levels (I PL) co rrespond to these mechanisms? The IPL is an operating system invention. It is stored in the memory of the process, and every process is given an IPL. At the lowest IPL, all interrupts are permitted. Conversely, at the highest IPL, all interrupts are blocked. Raising and lowering the IPL involves changes to the interrupt mask fi eld of the Status register.

Elaboration: The two least sign ificant bits of the pending interrupt and interrupt mask fields are for software interrupts , which are lower priority. These are typically used by higher-priority interrupts to leave work for lower-priority interrupts to do once the immediate reason for the interrupt is handled . Once the higher-priority interrupt is fini shed, the lower-priority t asks will be noticed and handled .

Transferring the Data between a Device and Memory We have seen two different methods that enable a device to communicate with the processo r. These two techniques-polling and I/O interrupts- form the basis for two methods of implementing the transfer of data between the I/O device and memo ry. Both these techniques wo rk best with lower-bandwidth devices, where we are more interested in reducing the cost of the device controller and interface than in providing a high -bandwidth transfer. Both polling and interrupt -driven transfers put the burden of moving data and m anaging the transfer on the processo r. Aft er looking at these two schemes, we will examine a scheme mo re suitable for higher-perfo rmance devices o r collections of devices. We ca n use the processor to transfer data between a device and memory based on polling. In real-time applications, the processor loads data fro m I/O device registers and sto res them into memory. An alternative mechanism is to make the transfer of data interrupt driven. In this case, the OS would still transfer data in small numbers of bytes from or to the device. But because the I/O operation is interrupt driven, the OS simply works on other tasks while data is being read from or written to the device. When the OS recognizes an interrupt from the device, it reads the staniS to check for errors. If there are none, the OS can supply the next piece of data, fo r example, by a sequence of mem ory-mapped writes. When the last byte of an I/O request has been transmitted and the I/O operation is completed, the OS ca n inform the program. The processor and OS do all the work in this process, accessing the device and memory for each data item transferred. Interrupt-driven I/O rel ieves the processor from having to wa it for every I/O event, alth ough if we used this method for transferring data from o r to a hard disk, the overhead could still be intolerable, since it could consume a large fraction of the processo r when the disk was transferring. For high -bandwidth devices like hard disks, the transfers consist primarily of relatively large blocks of data

593

594

di rect memo r y access (DMA) A mechanism that provides a device controller the ability to transfer data directly to or from the memory without involving the processor. bus master A unit on the bus that can initiate bus requests.

Chapter 8

(hundreds to thousand s of bytes). Thus, computer designers invented a mechanism for offi oading the processor and having the device controller transfer data directly to or from the memo ry without involving the processo r. This mechanism is called direct memory access (OM A). The interrupt mechanism is still used by the device to communicate with the processor, but only on completion of the I/O tran sfer or when an error occurs. DM A is implemented with a specialized cont roller that transfers data between an I/O device and memory independent of the processor. The DM A controller becomes the bus master and directs the reads or writes between itself and mem o ry. There are three steps in a DM A transfer: I. The processor sets up the DM A by supplying the identity of the device, the operation to perfo rm on the device, the memo ry address that is the source or destill3tion of the data to be transferred, and the number of bytes to transfer. 2. The DM A starts the operation on the device and arbitrates for the bus. Wh en the data is ava ilable (from the device o r memory), it transfers the data. The DM A device supplies the memory address for the read or the write. If the request requires more than one transfer on the bus, the DMA unit generates the next memory address and initiates the next transfer. Using this mechanism , a DM A unit ca n complete an entire transfer, which may be th ousands of bytes in length , without bothering the processor. Many DM A controllers contain some memory to allow them to dea l flexibly with delays either in transfer or those incurred while waiting to become bus master. 3. Once the DM A transfer is complete, the cont roller interrupts the processo r, which can then determine by inter rogating the DM A device or exa mining memory whether the entire operation completed successfully. There may be multiple DMA devices in a computer system. For example, in a system with a single processo r-memory bus and multiple I/O buses, each I/O bus cont roller will often contain a DM A processo r that handles any transfers between a device on the I/O bus and the memory. Unlike either polling or interrupt -driven I/O, DM A ca n be used to interface a hard disk without consuming all the processo r cycles fo r a single I/O. Of course, if the processor is also contending for memor y, it will be delayed when the memory is busy doing a DMA transfer. By using caches, the processo r ca n avoid having to access memory most of the time, thereby leaving m ost of the memory bandwidth free for use by I/ O devices.

Elaboration: To furth er reduce the need to interrupt the processor and occupy it in handling an I/O request that may involve doing several actual operations . the I/O cont roller can be made more intelligent. Intelligent controllers are often called I/O proces-

8.5

595

Interfacing I/ O Devices to the Processor, Memory, and Operating System

sors (as well as I/O controllers or channel controllers) . These specialized processors basically execute a seri es of I/O operations , called an I/O program . The program may be st ored in the I/O processor, or it may be stored in memory and fetched by the I/O processor. When us ing an I/O processor, the operating syst em typically set s up an I/O program that indicates the I/O operati ons t o be done as well as the s ize and tran sf er address f or any reads or writes . The I/O processor then takes the operati ons from the I/O program and interrupts the processor only when the entire program is completed. DMA processors are essentially special-purpose processors (usually s inglechip and nonprogrammable), while I/O processors are often implemented with generalpurpose microprocessors , which run a spec ialized I/O program .

Direct Memory Access and the Memory System When OM A is incorporated into an I/O system , the relationship between the memory system and processo r changes. Without OM A, all accesses to the memory system come from the processo r and thus proceed th rough address translation and cache access as if the p rocesso r generated the references. With DMA, there is another path to the memory system-one that does not go through the address translation mechanism or the cache hierarchy. This difference generates some problems in both virtual memory systems and systems with caches. These problems are usually solved with a combination of hardwa re techniques and softwa re support. The difficulties in having DMA in a virtual mem ory system arise because pages have both a physical and a virtual address. DMA also creates problems for systems with caches because there ca n be two copies of a data item: one in the cache and one in memory. Because the DMA processor issues memory requests directly to the memo ry rather than th rough the processor cache, the value of a memory location seen by the OM A unit and the processo r may differ. Consider a read from disk that the OMA unit places directly into memory. If some of the locations into which the DM A writes are in the cache, the p rocesso r will receive the old value when it does a read. Similarly, if the cache is write-back, the DM A may rea d a value directly from memory when a newer va lue is in the cache, and the va lue has not been written back. This is ca lled the stale data problem or coherence problem. In a system with virtual memory, should DM A wo rk with virtual addresses or physical addresses? The obvious problem with virtual addresses is that the DM A unit will need to translate the virtual addresses to physical addresses. The major p roblem with the use of a physica l address in a DMA transfer is that the transfer ca nnot easily cross a page boundary. If an I/O request crossed a page boundary, then the memory locations to which it was being transferred would not necessa rily be contiguous in the virtual memo ry. Consequently, if we use physical addresses, we must constrain all DMA transfers to stay within one page.

Hardware Software Interface

596

Chapter 8

O ne method to allow the system to initiate DM A transfers that cross page bounda ries is to make the DM A work on virtual addresses. In such a system , the DM A unit has a small number of map entries th at provide virtual-to-physical mapping for a transfer. The operating system provides the mapping when the I/O is initiated. By using this mapping, the DM A unit need not worry about the location of the virtual pages involved in the tra nsfer. Another technique is for the operating system to break the DM A transfer into a series of transfers, each confined within a single physical page. The transfers a re then chained together a nd ha nded to a n I/O p rocessor or intelligent DM A unit that executes the entire sequence of transfers; alternatively, the operating system ca n individually request the transfers. Whichever method is used, the operating system must still cooperate by not remapping pages while a DM A transfer involving that page is in progress.

We have looked at three different methods for transferring data between an I/O device and memory. In moving from polling to an interrupt-driven to a DM A interface, we shift the burden for managing an I/O operation from the processor to a progressively more intelligent I/O cont roller. These methods have the advan tage of freeing up p rocessor cycles. Their disa dva ntage is that they increase the cost of the I/O system. Because of this, a given computer system ca n choose which point along this spectrum is appropriate for the I/O devices connected to it. Before discussing the design of I/O systems, let's look briefly at performance measures of them.

Hardware Software Interface

The coherency problem for I/O data is avoided by using one of three major tech niques. One approach is to route the I/O activity th rough the cache. This ensures that reads see the latest value while writes update any data in the cache. Routing all I/O th rough the cache is expensive and potentially has a large negative performance impact on the processor, since the I/O data is rarely used immediately and may displa ce useful data that a running program needs. A second choice is to have the OS selectively invalidate the cache fo r an I/O read or force write-backs to occur for an I/O write (often called cach ej1ushillg) . This approach requires some sm all amount of hardwa re support and is probably mo re effi cient if the soft wa re ca n perfo rm the fun ction easily and effi ciently. Because this flushing of large parts of the cache need only happen on DM A block accesses, it will be relatively infrequent. The third approach is to provide a hardware mechanism for selectively flu shing (or invalidating) cache entries. Hardwa re invalidation to ensure cache coherence is typical in multip rocesso r system s, and the sa me technique ca n be used for I/O; we discuss this topic in detail in Chapter 9.

8.6

597

I/O Performance Measures: Examples from Disk and File Systems

In ranking of the three ways of doing I/O, which statements are true? I. If we wa nt the lowest latency for an I/O operation to a single I/O device, the order is polling, DMA, and interrupt driven. 2. In terms of lowest impact on processo r utiliza tion from a single I/O device, the order is DMA, interrupt driven, and polling

I/O Performance Measures: Examples from Disk and File Systems How should we compare I/O systems? This is a complex question because I/O performance depends on many aspects of the system and different applications st ress different aspects of the I/O system. Furtherm ore, a design ca n make com plex trade-offs between response time and th roughput , making it impossible to measure just one aspect in isolation. For example, handling a request as ea rly as possible generally minimizes response time, although greater th roughput can be achieved if we try to handle related requests together. Acco rdingly, we may increase th roughput on a disk by grouping requests th at access loca tions that are close together. Such a policy will in crease the response time for some requests, probably leading to a larger va riation in response time. Although th roughput will be higher, some benchmarks constrain the maximum response time to any request, making such optimiza tions potentially p roblematic. In this section, we give so me examples of measurements proposed fo r determining the performance of disk systems. These benchmarks are affected by a va riety of system features, including the disk technology, how disks are con nected, the memory system , the processo r, and the fil e system provided by the operating system. Before we discuss these benchmarks, we need to address a confusing point about termin ology and units. The perform ance of I/O systems depends on the rate at which the system transfers data. The transfer rate depend s on the clock rate, which is typically given in G Hz = 10 9 cycles per second. The transfer rate is usually quoted in GB/sec. In I/O systems, G Bs are measured using base 10 (i.e., 1 GB = 109 = 1,000,000,000 bytes), unlike main memory where base 2 is used (i.e., 1 GB = 230 = 1,073,741 ,824). In addition to adding confusion , this difference intro duces the need to convert between base 10 (l K = 1000) and base 2 ( IK = 1024) because many I/O accesses are for data blocks that have a size that is a power of two. Rather than complica te all our examples by accurately converting one of the two measurements, we make note here of this distinction and the fa ct that treating the two measu res as if the units were identica l int roduces a small er ror. We illustrate this er ror in Section 8.9.

Check Yourself

598

Chapter 8

Transaction Processing I/O Benchmarks transaction processing A type of application that involves han dling small sh ort operations (called transaction s) that typically require both I/O and com putation. Tran sacti on processing applications typically have bo th response time requirements and a perform an ce m easurem ent based on the throughput of transactions . ItO rate Performance m easure of VOs per unit time, such as reads per second.

data rate Performance measure of bytes p er unit time, su ch as GB/second.

Transaction processing (TP) applications involve both a response time requirement and a performance measurement based on th roughput. Furthermore, most of the I/O accesses are small. Because of this, TP applications are chiefl y con cerned with I/O rate, measured as the number of disk accesses per second, as opposed to data rate, measured as bytes of data per second. TP applications gen erally involve changes to a large database, with the system meeting some response time requirements as well as gracefully handling certain types of failures. These applications are extremely critical and cost -sensitive. For exa mple, banks normally use TP systems because they are concerned about a range of characteristics. These include making sure transactions aren't lost, handling transactions quickly, and minimizing the cost of processing each transaction. Although depend ability in the face of failure is an absolute requirement in such systems, both response time and th roughput are critical to building cost -effective systems. A number of transaction processing benchmarks have been developed. The best -known set of benchm ar ks is a series developed by the Transaction Processing Coun cil (TPC). TPC-C, initially created in 1992, simulates a complex query environment. TPC- H models ad hoc decision support- the queries are unrelated and knowledge of past queries cannot be used to optimize future queries; the result is that quer y execution times ca n be very long. TPC- R simulates a business decision sup port system where users run a standard set of queries. In TPC- R, prekn owledge of the queries is taken fo r granted , and the DBMS ca n be optimized to run these queries. TPC-W is a Web-based transaction benchmark that simulates the activities of a business-oriented transactional Web server. It exercises the database system as well as the underlying Web server softwa re. The TPC benchmarks are described at www.tpc.org. All the TPC benchmarks measure perfo rm ance in transactions per second. In addition , they include a response time requi rement , so that th roughput performance is measured only when the response time limit is met. To model rea l-wo rld systems, higher transaction rates are also associated with larger systems, both in terms of users and the size of the database that the transactions are applied to. Finally, the system cost fo r a benchmark system must also be included, allowing accurate comparisons of cost -performance.

File System and Web I/O Benchmarks File system s, which are stored on disks, have a different access pattern. For exa mple, measurements of UNIX fil e systems in an engineering environment have found that 80% of accesses are to fil es of less than 10 KB and that 90% of all fil e accesses are to data with sequential addresses on the disk. Furthermore, 67% of the accesses were reads, 27% were writes, and 6% were read -modify-write accesses, which read data, modify it , and then rewrite the sa me location. Such

8.6

599

I/O Performance Measures: Examples from Disk and File Systems

measurements have led to the creation of synthetic fil e system benchmarks. One of the most popular of such benchmarks has five phases, using 70 fil es:

• MakeDir: Constructs a directory subtree that is identical in structure to the given directory subtree

• Copy: Copies every fil e from the source subtree to the target subtree • SCIlllDir: Recursively traverses a directo ry subtree and examines the staniS of every fil e in it

• ReadA Il: Scans ever y byte of ever y file in a subtree once • Make: Compiles and links all the fil es in a subtree As we will see in Section 8.7, the design of an I/O system involves kn owing what

the wo rkload is. In addition to processor benchmarks, SPEC offers both a file server benchmark (SPECS FS) and a Web server benchmark (S PECWeb). SPECSFS is a benchmark for measuring NFS (Netwo rk File System) performance using a script of fil e server requests; it tests the perfo rmance of the I/O system , including both disk and net work I/ O, as well as the processo r. SPECS FS is a throughput -oriented benchm ark but with important response time requirements. SPECWeb is a Web server bench mark that simulates multiple clients requesting both static and dynamic pages from a server, as well as clients posting data to the server.

I/O Performance versus Processor Performance Amdahl's law in Chapter 2 reminds us that neglecting I/O is dangerous. A simple example demonstrates this.

Impact of I/O on System Performance

Suppose we have a benchmark that executes in 100 seconds of elapsed time, where 90 seconds is C PU time and the rest is I/O time. If C PU time improves by 50% per yea r for the next fi ve years but I/O time doesn't improve, how much faster will our program run at the end of fi ve yea rs?

We know that Elapsed time 100

I/O time

CPU time + I/O time 90 + I/O time 10 seconds

EXAMPLE

ANSWER

600

Chapter 8

The new CPU times and the resulting elapsed times are computed in the following table: After II years

I/O time

Elapsed time

90 seconds

10 s econds

100 s econds

10%

CPU time

I/O time

jUl ~

60 seconds

10 s econds

70 seconds

14%

..6.ll " 40 seconds

10 s econds

50 seconds

20%

~ " 27 seconds

10 s econds

37 seconds

27%

~ " 18 seconds

10 s econds

28 seconds

36%

..11i " 12 seconds

10 s econds

22 seconds

45%

1.5 1.5

1.5

The improvement in CPU performance over five years is 90 = 7.5 12

However, the improvement in elapsed time is only 100 = 4.5 22

and the I/O time has increased from 10% to 45% of the elapsed time.

Check Yourself

Are the following true or false? Unlike processor benchmarks, I/O benchmarks I. concentrate on throughput rather than latency 2. ca n require that the data set scale in size or number of users to achieve performance milestones 3. come from organizations rather than from individuals

Designing an I/O System There are two primary types of specifica tions that designers encounter in I/O systems: latency constraints and bandwidth constraints. In both cases, knowledge of the traffic pattern affects the design and analysis. Latency constraints involve ensuring that the latency to complete an I/O operation is bounded by a certain amount. In the simple case, the system may be

8.7

601

Designing an I/ O System

unloaded , and the designer must ensure that some latency bound is met either because it is critical to the application or because the device must receive certain guaranteed service to prevent errors. Examples of the latter are similar to the analysis we looked at in the previous section. Likewise, determining the latency on an unloaded system is relatively easy, sin ce it involves tracing the path of the I/O operation and summing the individual latencies. Finding the average latency (o r distribution of latency) under a load is a much more complex problem. Such problems are ta ckled either by queuing theory (when the behavior of the workload requests and I/O service times can be approximated by simple distributions) o r by simulation (when the behavior of I/O events is complex). Both topics are beyond the limits of this text. Designing an I/O system to meet a set of bandwidth constraints given a workload is the other typical problem designers fa ce. Alternatively, the designer may be given a partially configured I/O system and be asked to balance the system to main tain the ma ximum bandwidth achievable as dictated by the preconfigured portion of the system. This latter design problem is a simplified version of the first. The genera l approach to designing such a system is as follows: I. Find the weakest link in the I/O system, which is the component in the I/O

path that will constrain the design. Depending on the workload, this com ponent ca n be anywhere, including the CPU, the memory system , the backplane bus, the I/O controllers, or the devices. Both the workload and configuration limits may dictate where the weakest link is located. 2. Configure this component to sustain the required bandwidth. 3. Determine the requirements for the rest of the system and configure them to support this bandwidth. The easiest way to understa nd this methodology is with an exa mple.

I/O System Design

Consider the following computer system: • A CPU that sustains 3 billion in structions per second and averages 100,000 instructions in the operating system per I/O operation • A memory backplane bus capable of sustaining a transfer rate of 1000 MB/sec • SCSI Ultra320 controllers with a transfer rate of 320 MB/sec and accommodating up to 7 disks • Disk drives with a read/write bandwidth of 75 MB/sec and an average seek plus rotational latency of 6 ms

EXAMPLE

602

Chapter 8

If the workload consists of 64 KB reads (where the block is sequential on a tra ck) and the user program needs 200,000 in structions per I/O operation , find the maximum sustainable I/O rate and the number of disks and SCS I controllers required. Assume that the reads ca n always be done on an idle disk if one exists (i.e., ignore disk conflicts).

ANSWER

The two fixed components of the system are the mem ory bus and the CPU. Let's first find the I/O rate that these two components ca n sustain and determine which of these is the bottleneck. Each I/O takes 200,000 user instructions and 100,000 OS instructions, so Maximum I/O rate of CPU = In struction execution pte Instructions per I/O

3 X I.!LQ'_--:

(200

+ 100) X 10 3

10,000 --lL.Q.L second

Ea ch I/O transfers 64 KB, so Maximum I/O rate ofbus = Bus bandwidtb = 1DOD X IJf = IS,625-.lLili.... Bytes per I/O 64 X 10 3 second The CPU is the bottleneck, so we ca n now configure the rest of the system to perform at the level dictated by the CPU, 10,000 I/Os per second. Let's determine how many disks we need to be able to accommodate 10,000 I/Os per second. To find the number of disks, we first find the time per I/O operation at the disk: Time per I/O at disk = Seek

+ rotational time + Transfer time

= 6 ms

= 6.9 ms

75 MB/sec

Thus, each disk ca n complete 1000 ms/6.9 ms or 146 I/O s per second. To sat urate the CPU requires 10,000 I/Os per second, or 10,000/ 146'" 69 disks. To compute the number of SCSI buses, we need to check the average transfer rate per disk to see if we ca n sa nlfate the bus, which is given by Transfer rate = Transfer size = 64 KB '" 9.56 MB/sec Transfer time 6.9 ms The maximum number of disks per SCSI bus is 7, which won't saturate this bus. This m ea ns we will need 69/7, or 10 SCS I buses and controllers.

8.8

Real Stuff: A Digital Camera

Notice the significa nt number of simpli fying assumptions that are needed to do this exa mple. In practice, many of these simplifications might not hold for critical I/O-intensive applications (such as databases). For this reason, simulation is often the only rea listic way to pred ict the I/O performance of a rea list ic workload.

Real Stuff: A Digital Camera Digital ca meras are basica lly embedded computers with removable, writable, non volatile, storage, and interesting I/O devices. Figure 8. 14 shows our example.

FtGURE 8.14 The Sanyo VPC·SXSOO with Flash memory card and IBM Mlcrodrlve. Although newer cameras offer more pixels per picture, the principles are the same. This 1360 x 1024 pixel digital camera stores pictures either using CompactFlash memory or using a IBM Microorive. This photo was taken using a 340 MB microorive and a 8 MB Compact Flash memory. As Figure 8.15 shows, in 2004 the cap.1cities are as large as 1 GB to 4 GRit is 4.3 inches wide x 2.5 inches high x 1.6 inches deep, and it weighs 7.4 ounces. In addition to taking a still picture and converting it to !pEG format every 0.9 seconds, it can record a Quick Time video dip at VGA size (640 x 480). One technological advantage is the use of a custom system on a chip to reduce size and power, so the camera only needs two AA batteries to operate versus four in other digital cameras.

603

604

Chapter 8

\ Vhen powered on, the microprocessor first runs diagnostics on all components and writes any er ro r messages to the liquid crystal display (LCD) on the back of the camera. This ca mera uses a 1.8- in ch low-temperature polysilicon TFT color LC D. When photographers take picnlfes, they first hold the shutter halfway so that the microprocessor can take a light reading. The microprocesso r then keeps the shutter open to get the necessar y light, which is ca ptured by a charged couple device (CeO) as red , green , and blue pixels. For the ca mera in Figure 8.14, the ceo is a 1/2-inch , 1360 X 1024 pixel, progressive-sca n chip. The pixels are sca nned out row by row and then passed th rough routines for white balance, color, and aliasing co rrection , and then stored in a 4 M B frame buffer. The next step is to compress the image into a standard format, such as IPEG, and store it in the removable Flash memory. The photographer picks the compression, in this camera called either fin e or normal, with a com pression ratio of 10 to 20 times. A fin e-quality compressed image takes less than 0.5 MB, and a normal-quality compressed image takes about 0.25 MB. The microprocessor then updates the LCD display to show that there is room for one less pictu re. Although the previous paragraph covers the basics of a digital camera, there are many more features that are included: showing the recorded images on the color LCD display; sleep mode to save battery life; monito ring battery energy; buffering to allow reco rding a rapid sequence of uncompressed images; and , in this camera, video recording using MPEG form at and audio recording using WAY format. This camera allows the photographer to use a Microdrive disk in stea d of Com pact Flash memory. Figure 8.1 5 compares Compact Flash and the IBM Microdrive.

Characteristics

Sandlsk Type I CompactFlash SDCFB·128·768

Sandlsk Type II CompactFlash SDCFB·I0000768

Hitachi 4 GB Mlcrodrlve DSCM·I0340

Fonnatted data capacity (MB)

128

1000

4000

Bytes per sector

512

Data transfer rate (MB/sec)

4 (burst)

4- 7

Lin k speed to buffer (MB/sec)

Power standbyloperating (W)

0 .15/0 .66

0 .1 5/0.66

0 .07/0 .83

Size : height x width x depth (inches)

1.43 x 1.68 x 0 .13

1.43 x 1.68 x 0 .1 3

1.43 x 1.68 x 0 .16

Weight in grams (454 grams/pound)

11.4

13 .5

Write cycles before sector wear-out

300 .000

300.000

not applicabl e

Mean time between failures (hours)

> 1.000.000

> 1 .000.000

(see caption)

Best price (200 4)

$40

$200

$480

FIGURE 8.15 Characteristics of three storage alternatives for digital cameras. H itachi matches the Type II form factor in the Microdrive, while t he Com pact Flash card uses that space to include many more Flash chips. H itachi does not quote MTTF for the 1.0·inch drives, bU1 t he service life is five years or 8800 powered-on hours, whichever is first. They rotate at 3600 RPM and have 12 ms seek t imes.

8.8

605

Real Stuff: A Dig ital Came ra

The Compact Flash standard package was proposed by Sa ndisk Corporation in 1994 for the PCMCIA-ATA ca rds of portable PCs. Beca use it follows the ATA interface, it simulates a disk interface including seek commands, logical tracks, and so on. It includes a built-in controller to support many types of Flash memory and to help with chip yield for Flash memories by mapping out bad blocks. The elect ronic brain of this ca mera is an embedded com puter with several special fun ctions embedded on the chip. Figure 8. 16 shows the block diagram of a chip similar to the one in the ca mera. Such chips have been called systems on a chip (SOC) beca use they essent ia lly integrate into a single chip all the parts that were found on a small printed circuit boa rd of the past. SOC generally reduces size and lowers power com pa red to less integrated solutions. The manufacturer cla ims the SOC enables the ca mera to operate on half the number of batteries and to offer a smaller fo rm factor than competitors' ca meras.

2-channel video D/A 10 bits CCD

16b~s l SDRAM SDRAM I

I I controller I

Signal processor 32

MJPEG

NTSCJPA L encoder

V 16

b~s

LCDITV

bits

Signal bus

Bus bridge

Smart Media

SSFDC controller

Flash (program)

RISC

16 bits

DRAM controller

UART

Audio D/A. AID

, 2

IrDA

RS-232

IrCA port

PC MCIA controller

, , , DRAM ,

SlO PlO PWM

DMA controller

MIC Speaker

CPU bus

1 1_____ 1

PCMCIA

Others

card

FtGURE 8.16 The syst em on a chip ( SOC) found In Sa nyo dig ita l c amer as. This block diagram is for the predecessor of the SOC in the camera in Figure 8.14. The successor SOC, called Super Advanced IC, uses three buses instead of two, operates at 60 MHz, consumes 800 mW, and fits 3.IM transistors in a 10.2 x 10.2 mm die using a 0.35-micron process. Note that this embedded system has twice as many transistors as the state-of-the-art, high-performance microprocessor in 1990! The SOC in the figure is limited to processing 1024 x 768 pixels, but its su((;essor supports 1360 x 1024 pixels. (See Okada, M.11suda, Yamada, and Kobayashi [1999 1).

606

Chapter 8

For higher performance, it has two buses. The 16-bit bus is for the ma ny slower I/O devices: Sma rt Media interface, program and data memory, and DM A. The 32- bit bus is for the SDRAM , the signal p rocesso r (which is connected to the CeO), the Motion IPEG encoder, and the NTSC/PAL encoder (which is con nected to the LC D). Unlike desktop microprocessors, note the la rge va riety of I/O buses that this chip must integrate. The 32-bit RI se MPU is a proprietary design and runs at 28.8 MH z, the sa me clock rate as the buses. This 700 mW chip con tains 1.8M transistors in a 10.5 X 10.5 mm die implemented using a O.35-micron process.

Fallacies and Pitfalls Fallacy: The rated mean time to failure of disks is 1,200,000 hours or almost 140 years, 50 disks practically never fail. The current marketing practices of disk manufacturers ca n mislea d users. How is such a n MTTF calculated? Ea rly in the process ma nufacturers will put thousands of disks in a room , run them for a few months, and count the number that fail. They compute MTIF as the total number of hours that the disks were cumulatively up divided by the number that fail ed. O ne problem is that this number far exceeds the lifetime of a disk, which is commonly assumed to be fi ve yea rs or 43,800 hours. For this large MTTF to make some sense, disk ma nufacturers argue that the calculation co rresponds to a user who buys a disk, and then keeps replacing the disk every five years-the planned lifetime of the disk. The claim is that if m any customers (a nd their great grand children ) did this for the next century, on average they would replace a disk 27 times before a failure, o r about 140 yea rs. A mo re useful measure would be percentage of disks that fail. Assume 1000 disks with a 1,200,000-hour MTTF and that the disks are used 24 hours a day. If you replaced failed disks with a new one having the same reliability characteristics, the number that would fail over five yea rs (43,800 hours) is Failed disks = 1000 drives X 43.800 hours/drive = 36 1,200,000 hours/failure Stated alternatively, 3.6% would fail over the 5-yea r period. Pitfall: Using the peak transfer rate ofa portion of the I/O system to make performance projections or performance comparisons.

Many of the components of an I/O system , from the devices to the cont rollers to the buses, a re specified using their pea k bandwidths. In practice, these peak band width measurements are oft en based on unrea listic assumptions about the system

8.9

Fallacies and Pitfalls

or are unattainable beca use of other system limitations. For exa mple, in quoting bus performance, the pea k transfer rate is sometimes specified using a memory system that is impossible to build. For networked systems, the software overhea d of initiating communication is ignored. The 32-bit, 33 MHz PCI bus has a peak bandwidth of about 133 MB/sec. In practice, even for long tra nsfers, it is difficult to sustain m ore th an about 80 MB/sec for realistic memory systems. As mentioned above, users of wireless net works typically achieve only about a third of the peak bandwidth. Amdahl's law also reminds us that the th roughput of an I/O system will be lim ited by the lowest-performance component in the I/O path. Fallacy: Magnetic disk storage is on its last legs and will be replaced shortly.

This is both a fallacy a nd a pitfall. Such claims have been made constantly for the past 20 yea rs, though the string of failed altern atives in recent yea rs seems to have reduced the level of claim s for the death of magnetic storage. Among the un successful contenders are magnetic bubble memories, optica l storage, and holographic storage. None of these systems has matched the combination of characteristics that favo r magnetic disks: high reliability, nonvolatility, low cost, reaso nable access time, and rapid imp rovement. Magnetic storage techn ology continues to improve at the same-o r faster- pace that it has sustained over the past 25 yea rs. Pitfall: Using magnetic tapes to back lip disks.

Once again , this is both a fallacy a nd a pitfall. Magnetic tapes have been part of computer systems as long as disks because they use similar technology as disks, and hence historically have followed the sa me density improvements. The historic cost -performance difference between disks and tapes is based on a sealed, rotating disk having lower access time than sequen tial tape access but removable spools of magnetic tape mea n many tapes ca n be used per rea der and they can be very long and so have high capacity. Hence, in the past a single m agnetic tape could hold the contents of many disks, and since it was 10 to 100 times cheaper per gigabyte th an disks, it was a useful backup medium. The claim was that magnetic tapes must track disks since innovations in disks must help tapes. This claim was important beca use tapes were a small market and could not afford a separate large research a nd develo pment effort. One reason the market is small is that desktop owners generally do not back up disks onto tape, a nd so while desktops a re by far the largest ma rket for disks, desktops are a small market for tapes. Alas, the la rger ma rket has led disks to improve much more quickly th an tapes. Starting in 2000 to 2002, the la rgest popular disk was larger than the largest popular tape. In that sa me time frame, the price per gigabyte of ATA disks dro pped below that of tapes. Tape apologists now claim that tapes have compatibility requirements that a re not imposed on disks; tape readers must rea d or write the current and previous generation of tapes, and must rea d the last four generations

607

608

Chapter 8

of tapes. As disks are closed systems, disk heads need only read the platters enclosed with them, and this advantage explains why disks are improving much m ore rapidly. Today, so me orga nizations have dropped tapes altogether, using networks and remote disks to replicate the data geographically. The sites are picked so that disasters would not take out both sites, enabling instantaneous recovery time. (Long recovery time is another serious drawback to the serial nature of magnetic tapes.) Such a solution depends on adva nces in disk capacity and network bandwidth to make economic sense, but these two are getting much greater investment and hence have better recent records of accomplishment than tape. Fallacy: A 100 M E/sec blls can transfer 100 MB ofdata in 1 second.

First, you genera lly ca nnot use 100% of any computer resource. For a bus, you would be fortunate to get 70% to 80% of the peak bandwidth. Time to send the address, time to acknowledge the signa ls, and stalls while wa iting to use a busy bus are am ong the reasons you ca nn ot use 100% of a bus. Second , the definition of a megabyte of storage and a megabyte per second of bandwidth do not agree. As we discussed on page 597, I/O bandwidth measures are usually quoted in base 10 (Le., 1 MB/sec = 10 6 bytes/sec), while 1 MB of data is typically a base 2 measure (Le., 1 MB = 220 bytes). How significa nt is this distinction? If we could use 100% of the bus for data tran sfer, the time to tran sfer 100 MB of data on a l OO-MB/sec bus is actually

JOQ X 2

"6 =

100 X 10

~ = 1.048576'" 1.05 second 1,000,000

A similar but larger error is introduced when we treat a gigabyte of data transferred or stored as equiva lent, mea ning 109 versus 2 30 bytes. Pitfall: Trying to provide featllres only within the network versus end to end.

The concern is providing at a lower level features that ca n only be accomplished at the highest level, thus o nly partially satisfying the communication demand. Sa ltzer, Reed, and Clark [ 1984 ) give the end-to-end argument as The function in question can completely and correctly be specified only with the knowledge and help of the application standing at the endpoints of the communication system. Therefore, providing that questioned function as a featllre of the communication system itself is not possible.

Their exa mple of the pitfall was a network at MIT that used several gateways, each of which added a checksum from one gateway to the next. The programmers of the application assumed the checksum guaranteed accuracy, incorrectly believing that the message was protected while stored in the memory of each ga teway. One gateway developed a transient failure that swapped one pair of bytes per million

8.10

Concluding Remarks

bytes transferred. Over time the source code of one operating system was repeatedly passed th rough the gateway, thereby corrupting the code. The only solution was to correct the infected source fil es by comparing to paper listings and repairing the code by hand! Had the checksums been calculated and checked by the application running on the end systems, sa fety would have been assured. There is a useful role fo r intermediate checks, however, provided th at end -toend checking is ava ilable. End -to-end checking may show that something is b roken between two nodes, but it doesn't point to where the problem is. Intermediate checks ca n discover what is broken. You need both for repair. Pitfall: M ovingfunctions from the CPU to the I/O processor, expecting to improve performance without a careful analysis.

There are many examples of this pitfall trapping people, although I/O p rocesso rs, when properly used, ca n certainly enhance performance. A frequent instance of this fa llacy is the use of intelligent I/O interfaces, which , because of the higher overhead to set up an I/O request, ca n turn out to have wo rse latency than a p rocesso r-directed I/O activity (although if the processo r is freed up sufficiently, system th roughput may still increase). Frequently, performance falls when the I/O processo r has much lower performance than the main processor. Consequently, a sm all atTI ount of main processo r time is repla ced with a larger amount of I/O processor time. Workstation designers have seen both these phenomena repeatedly. Myer and Sutherland [1968 ) wrote a classic paper on the trade-off of complexity and performance in I/O cont rollers. Borrowing the religious concept of the "wheel of rein ca rnation," they eventually noticed they were caught in a loop of continuously increasing the power of an I/O p rocesso r until it needed its own sim pler coprocessor: We approached the task by starting with a simple scheme and then adding commands and features that we felt would enhance the power of the machine. Gradually the {display] processor became more complex . ... Finally the display processor came to resemble a fllll-fiedged computer with some special graphics featllres. A nd then a strange thing happened. We felt compelled to add to the processor a second, subsidiary processor, which, itself, began to grow in complexity. It was then that we discovered the disturbing truth. Design ing a display processor can become a never-ending cyclical process. In fact, we found the process 50 frustrating that we have come to call it the "wheel of reincarnation."

Concluding Remarks I/O systems are eva luated on several different characteristics: dependability; the variety of I/O devices supported; the maximum number of I/O devices; cost; and

609

610

Chapter 8

performance, measured both in latency and in throughput. These goa ls lea d to widely va rying schemes for interfa cing I/O devices. In the low-end and midrange systems, buffered DM A is likely to be the dominant transfer mechanism. In the high-end systems, latency and bandwidth may both be important, and cost may be secondary. Multiple paths to I/O devices with limited buffering oft en characterize high-end I/O systems. Typically, being able to access the data on an I/O device at any time (high availability) becomes more impo rtant as systems grow. As a result, red undan cy and erro r co rrection mechanisms become mo re and mo re prevalent as we enlarge the system. Storage and networking demands are growing at unprecedented rates, in part because of increasing demands for all information to be at your fingertips. One estimate is that the amount of information created in 2002 was 5 exabytesequiva lent to 500,000 copies of the text in the U.S. Library of Congress-a nd that the total amount of information in the world doubled in the last three years (Lyman and Va rian 2003J. Future directions of I/O include expanding the reach of wired and wireless networks, with nea rly every device potentially having an IP address , and the continu ing transformati on from parallel buses to serial networks and switches. However, consolidation in the disk industry may lead to a slowdown in improvement in disk capacity to ea rlier rates, which have doubled every yea r between 2000 and 2004.

Understanding Program Performance

The performance of an I/O system , whether measured by bandwidth or latency, depends on all the elements in the path between the device and memory, includ ing the operating system that generates the I/O commands. The bandwidth of the buses, the memory, and the device determine the maximum tran sfer rate from or to the device. Similarly, the latency depends on the device latency, together with any latency imposed by the memory system or buses. The effective bandwidth and response latency also depend on other I/O requests that may cause contention for some resource in the path. Finally, the operating system is a bottleneck. In some cases, the OS takes a long time to deliver an I/O request from a user program to an I/O device, leading to high latency. In other cases, the opera ting system effectively limits the I/O bandwidth because of limitations in the number of concurrent I/O operations it ca n support. Keep in mind that while performance ca n help sell an I/O system , users overwhelmingly demand dependability and capacity from their I/O systems.

8.12

611

Exerci ses

Historical Perspective and Further Reading The history of I/O systems is a fascinating one. This II Section 8.11 gives a brief history of magnetic disks, RAID, databases, the Internet, the Wo rld Wide Web, and how Ethernet continues to triumph over its challengers.

Exercises 8.1 [ 10 ] Here are two different I/O systems intended for use transaction processing:

III

• System A ca n suppo rt 1500 I/O operations per second. • System B can support 1000 I/O operations per second. The systems use the same processor that executes 500 million instructions per second. Assume th at each transaction requires 5 I/O operations and that each I/O operation requi res 10,000 in structions. Ignoring response time and assuming that transactions may be arbitrarily overlapped, what is the maximum transaction per-second rate that each machine ca n sustain?

8.2 [ 15] The latency of an I/O operation for the two systems in Exercise 8.1 differs. The latency for an I/O on system A is equal to 20 m s, while for system B the latency is 18 ms for the first 500 I/Ds per second and 25 ms per I/O for each I/O between 500 and 1000 I/Ds per second. In the workloa d, every 10th transaction depends on the immediately preceding transaction and must wait for its completion. What is the maximum tratlsa ction rate that still allows every transaction to complete in I second and that does not exceed the I/O bandwidth of the machine? (For simplicity, assume that all transaction requests arrive at the begin ning of a I-second interva l.) 8.3 [5] Suppose we wa nt to use a laptop to send 100 files of approximately 40 MB each to another computer over a 5 Mbit/sec wireless connection. The laptop battery currently holds 100,000 oules of energy. The wireless netwo rking ca rd alone consumes 5 wa tts while transmitting, while the rest of the laptop always consumes 35 watts. Before each fil e transfer we need 10 seconds to choose which file to send. How many complete fil es ca n we transfer before the laptop's battery runs down to zero?

612

Chapter 8

8.4 [ IO J Consider the laptop's hard disk power consumption in Exercise 8.3 . Assume that it is no longer constant, but va ries between 6 wa tts when it is spinning and 1 watt when it is not spinning. The power consumed by the lap top apart from the hard disk and wireless ca rd is a constant 32 watts. Suppose that the hard disk's transfer rate is 50 MB/sec, its delay before it can begin transfer is 20 ms, and at all other times it does not spin. How many complete files ca n we t ra nsfer before the laptop's battery runs down to zero? How much energy would we need to send all 100 fil es? (Consider that the wireless card cannot send data until it is in memory. )

8.5 [5J The following simplified diagram shows two potential ways of numbering the sectors of d ata o n a disk (only two tracks are shown and each t rack h as eight secto rs) . Assuming that typica l read s a re contigu o us (e.g., all 16 secto rs a re read in order), which way o f numbering the sectors will be likely to result in high er perfo rman ce? W h y?

8.6 [20 J < §8.3 > In this exercise, we will run a p rogram to evaluate the b ehavior o f a disk drive. Disk sectors are addressed sequentially within a track, tracks sequen tially within cylinders, and cylinders sequentially within the disk. Determining h ead switch time and cylinder switch time is difficult b ecause o f ro tatio nal effects. Even determining platter count , sectors/ track, a nd ro tatio nal delay is difficult b ased on o bservation o f typical disk wo rkloa d s. The key is to fac to r out disk ro tatio n al effects b y m aking consecutive seeks to individual sectors with addresses that differ by a linea rl y increasing am ount sta rting with 0, 1, 2, and so forth. The Skip py algorithm , fro m work b y Nish a Talagala and colleagu es of u.c. Berkeley [2000 J, is

8.12

Exerci ses

fd = open( "r aw disk device " ) ; fo r (i = 0 ; i < measuremen t s ; i ++ ) ( Ii t ime t he f ollowing sequence , and ou t pu t lsee k(fd , i * SINGL E_S ECTOR , SEE K_CUR) ; wri t e(fd , bu ff er , SI NGL E_S ECTOR) ;

I close( f d) ; The basic algo rithm skips th rough the disk, increasing the distance of the seek by one sector before every write, and outputs the distance and time for each write. The raw device interface is used to avoid fil e system optimizations. SI NGL E_ SECT OR is the size of a single secto r in bytes. The SEE K_CU R argument to 1see k moves the fil e pointer an amount relative to the current pointer. A technical repo rt describing Skippy and two other disk drive benchmarks (run in seconds or minutes rather than hours o r days) is at http://su nsite.berkeley.edli/DienstIUI/2.01 Describe/ncstrl.lIcb/CSD-99-1063. Run the Skippy algo rithm on a disk drive of your choosing. a. What is the number of heads? b. The number of platters? c. What is the rotational latency?

d. What is the head switch time (the time to switch the head that is reading from one disk surfa ce to another with out moving the arm; that is, in the same cylinder)? e. What is the cylinder switch time? (It is the time to move the arm to the next sequential cylinder.)

8.7 (20J Figure 8.1 7 shows the output from running the benchmark Skippy on a disk. a. What is the number of heads? b. The number of platters? c. What is the rotational latency?

d. What is the head switch time (the time to switch the head that is reading from one disk surface to another with out moving the arm; that is, in the sa me cylinder)? e. What is the cylinder switch time (the time to m ove the arm to the next sequential cylinder)?

613

614

Chapter 8

MOCK DISK

-.s•

••>-

•

(2)

0+----,----,----,----,--o

100

150

200

Dis tance (sectors) FIGURE 8.17

Example output of Skippy for a hypothetical disk.

8.8 [ IO J Consider two RAID disk systems that a re meant to store 10 terabytes of data (not counting any redund ancy). System A uses RAID 1 technology, a nd System B uses RAID 5 technology with fo ur disks in a " protection group." 3.

How m any m o re ter abytes of storage a re needed in System A than in System 81

b. Suppose an application writes one block of data to the disk. If reading or writing a block takes 30 in S, how much time will the write take on System A in the worst case? How abo ut on System B in the wo rst case? c. Is System A m o re reliable that System B? Why o r why not?

8.9 ( 15 J < §8.3> What ca n happen to a RAID 5 system if the power fails between the write update to the data block and the write update to the check block so that only one of the two is successfully written? What could be done to prevent this from happening? 8.10 [5J < §8.3> The speed of light is approximately 3 X 108 meters per second , a nd electrica l signals travel at about 50% of this speed in a conductor. When the

8.12

Exerci ses

term high speed is applied to a netwo rk, it is the bandwidth that is high er, not necessarily the velocity of the electrical signals. How much of a factor is the actual "fli ght time" fo r the electrical signals? Consider two computers that are 20 meters apart and two computers that are 2000 kilometers apart. Compare your results to the latencies reported in the example on page 8. 3-7 in " Section 8. 3.

8.11 [5J The number of bytes in transit on a network is defin ed as the flight time (described in Exercise 8.10) multiplied by the delivered bandwidth. Ca lculate the number of bytes in transit for the two networks described in Exercise 8.1 0, assuming a del ivered bandwidth of 6 MB/sec. 8.12 [5 J A secret agency simultaneously monitors 100 cellular phone con versa tions and multiplexes the data onto a network with a bandwidth of 5 MB/sec and an overhea d latency of 150 p s per 1 KB message. Ca lculate the transmission time per message and determine whether there is sufficient bandwidth to support this application. Assume that the phone conversation data consists of 2 bytes sa mpled at a rate of 4 KHz. 8.13 [5 J Wireless networking has a much high er bit erro r rate ( BER) than wired netwo rking. One way to cope with a high er BER is to use an error co rrecting code (ECC) on the transmitted data. A very simple ECC is to triplicate each bit , encoding each zero as 000 and each one as 111. When an encoded 3-bit pattern is received, the system chooses the most likely original bit. a. If the system received 00 1, what is the most likely value of the original bit? b. If 000 was sent but a double-bit erro r causes it to be received as 110, what will the receiver believe was the origin al bit 's value? c. How many bit errors can this simple ECC co rrect?

d. How many bit errors can this ECC detect? e. If l out of every 100 bits sent over the netwo rk is inco rrect, what percentage of bit errors would a receiver using this ECC not detect?

8.14 [5 J There are two types of parity: even and odd. A bina ry word with even parity and no errors will have an even number of Is in it, while a word with odd parity and no erro rs will have an odd number of 1's in it. Compute the parity bit fo r each of the following 8-bit wo rds if even parity is used: a. 0 1100 111 b. 0 1010101

8.15 [ 10 1 a. If a system uses even parity, and the wo rd 0 111 is read from the disk, ca n we tell if there is a single-bit error?

615

616

Chapter 8

b. If :l system uses odd pa rity, and the word 010 1 appears on the processormemory bus, we suspect that a single-bit error has occurred. Ca n we tell which bit the er ro r occurs in? \-Vhy o r why not? c. If a system uses even parity and the word 010 1 appea rs on the processor-

memory bus, can we tell if there is a double-bit er ror?

8.16 (10 ) A program repea tedly performs a three-step process: It rea ds in a 4 KB block of data from disk, does some processing on th at data, and then writes o ut the result as an other 4 KB block elsewh ere on the disk. Each block is contigu ou s

and randomly located on a single track on the disk. The disk drive rotates at 10,000 RPM , has an average seek time of8 ms, and has a transfer rate of 50 MB/sec. The controller overhea d is 2 m s. No o ther p rogra m is u sing the disk o r p rocesso r, and there is n o overlapping o f disk o peration with p rocessing. The p rocessing step takes 20 millio n clock cycles, a nd the clock rate is 5 G H z. W h at is the overall sp eed o f the system in blocks p rocessed p er second?

8.17 [5] The O SI network p ro tocol is a hiera rch y o f layers o f ab st ractio n , creating a n interface b etween network application s and the physica l wires. This is similar to the levels o f abstractio n u sed in the ISA interface between softwa re and h a rd wa re. Na me three ad va ntages to u sing abstractio n in n etwo rk p ro tocol design.

8.18 [5] Suppose we h ave a system with the fo llowing ch a racteristics: I. A m em o ry and bus system supporting block access o f 4 to 16 32-bit words.

2. A 64-bit syn ch ro n o u s bus clocked at 200 MHz, with each 64-bit transfer taking 1 clock cycle, and 1 clock cycle required to send an address to m em ory. 3. Two clock cycles need ed between each bus op eratio n. (Assume the bus is idle befo re an access.) 4. A m em ory access time for the first four word s of 200 n s; each addition al set o f four wo rds ca n b e rea d in 20 n s. Assume th at the bus and m em o ry system s d escribed ab ove are u sed to handle disk accesses fro m disks like the o n e d escribed in the exa mple o n p age 570. If the I/O is allowed to con sume 100% o f the bus and m em o ry b andwidth, wh at is the m aximum number o f simultan eou s disk tran sfers that ca n b e su stained fo r the two block sizes?

8.19 [5] In the system d escribed in Exercise 8.18 , the mem ory system took 200 n s to rea d the first fo ur words, a nd each additio n al four word s required 20 n s. Assuming th at the m em ory system takes 150 n s to rea d the first four words and 30 n s to read each additio n al fo ur word s, find the su stained bandwidth and the laten cy for a read of 256 words for tran sfers that u se 4 -word blocks and fo r tran s-

8.12

Exerci ses

fers that use 16-word blocks. Also compute the effective number of bus transactions per second for each case.

8.20 [5 J Exercise 8.19 demonstrates that using larger block sizes results in an increase in the maximum sustained bandwidth that can be achieved. Under what conditions might a designer tend to favor smaller block sizes? Specifically, why would a designer choose a block size of 4 instead of 16 (assuming all of the chara cteristics are as identified in Exercise 8.19)? 8.21 [15) This question exa mines in more detail how increasing the block size for bus transactions decreases the total latency required and increases the maximum sustainable bandwidth. In Exercise 8.19 , two different block sizes are con sidered (4 words and 16 words). Compute the total latency and the maximum bandwidth for all of the possible block sizes (between 4 and 16) and plot your results. Summarize what you lea rn by looking at your graph. 8.22 [15) This exercise is similar to Exercise 8.21. This time fix the block size at 4 and 16 (as in Exercise 8.19), but compute latencies and bandwidths for reads of different sizes. Speci fi ca lly, consider reads of from 4 to 256 words, and use as many data points as you need to construct a mea ningful graph. Use your graph to help determine at what point block sizes of 16 result in a reduced latency when compared with block sizes of 4. 8.23 (10 ) This exercise examines a design alternative to the system described in Exercise 8.18 that may improve the performance of writes. For writes, assume all of the characteristics reported in Exercise 8.18 as well as the following: The first 4 words are written 200 ns after the address is available, and each new write takes 20 ns. Assume a bus transfer of the most recent data to write, and a write of the previous 4 words ca n be overlapped. The performance analysis reported in the exa mple would thus remain unchanged for writes (in actuality, some minor changes might exist due to the need to com pute error correction codes, etc., but we' ll ignore this). An alternative bus scheme relies on separate 32-bit address and data lines. This will permit an address and data to be tran smitted in the sa me cycle. For this bus alternative, what will the latency of the entire 256-word transfer be? What is the sustained bandwidth? Con sider block sizes of 4 and 8 words. When do you think the alternative scheme would be heavily favored?

8.24 Con sid er an asynchronous bus used to interfa ce an I/O device to the memory system described in Exercise 8.18. Each I/ O request asks for 16 wo rds of data from the mem ory, which , along with th e I/O device, has a 4-word bus. Assume the same type of handshaking protocol as appears in Figure 8.10 on page 584 except that it is extended so that the memory ca n co ntinue th e

617

618

Chapter 8

transa ction by sending additional blocks of data until the transaction is co m plete. Modify Figure 8.10 (both the steps and diagram) to indica te how such a transfer might take place. Assuming that each handshaking step takes 20 tl S and memory access takes 60 ns, how long does it take to co mplete a transfer? What is the maximum sustained bandwidth for this asynchronous bus, and how does it compare to the synch ronous bus in the exa mple?

8.25 (1 day-l weekI mark I/O Performance

II For Mo re Practice: Writing Code to Ben ch -

8.26 (3 days-l weekI .

In Mo re Depth: Ethernet Simulation

8.27 (151 We wa nt to compare the maximum bandwidth for a synchronous and an asynchronous bus. The synchronous bus has a clock cycle time of 50 ns, and each bus transmission takes 1clock cycle. The asyn chronous bus requires 40 ns per handshake. The data portion of both buses is 32 bits wide. Find the bandwidth for each bus when performing one-word reads from a 2oo-ns memory. 8.28 (20) Suppose we have a system with the following characteristics: 1. A memory and bus system supporting block access of 4 to 16 32-bit words.

2. A 64-bit synchronous bus clocked at 200 MHz, with each 64-bit transfer taking 1 clock cycle, and 1 clock cycle required to send an address to mem ory. 3. Two clock cycles needed between each bus operation. (Assume the bus is idle before an access.) 4. A memory access time for the first four words of 200 ns; each additional set of four words can be read in 20 ns. Assume that a bus transfer of the most recently read data and a read of the next four words ca n be overlapped. Find the sustained bandwidth and the latency for a read of 2S6 words for transfers that use 4-word blocks and for transfers th at use 16-word blocks. Also compute the effective number of bus transactions per second for each case. Recall that a single bus tran saction consists of an address transmission followed by data.

8.29 (10 ) Let's determine the impact of polling overhead forthree different devices. Assume that the number of clock cycles for a polling opera tionincluding tran sferring to the polling routine, accessing the device, and restarting the user program-is 400 and that the processor executes with a Soo-M Hz clock. Determine the fraction of CPU time consumed for the following three cases, assuming that you poll often enough so that no data is ever lost and assuming that the devices are potentially always busy: 1. The mou se must be polled 30 times per second to ensure that we do not

miss any movement made by the user.

8.12

619

Exerci ses

2. The floppy disk transfers data to the processor in 16-bit units and has a data rate of 50 KB/sec. No data transfer can be missed. 3. The hard disk transfers data in four-word chunks and ca n transfer at 4 MB/ sec. Again, no transfer can be missed.

8.30 [ I S) For the I/O system described in Exercise 8.45, find the ma ximum instantaneous bandwidth at which data can be transferred from disk to memory using as many disks as needed. How many disks and I/ O buses (the minimum ofeach) do you need to achieve the bandwidth? Since you need only achieve this bandwidth for an instant, latencies need not be considered.

II (10 ) II

8.31 (20)

In More Depth: Disk Arrays versus Single Disk

8.32

In More Depth: Disk Arrays Bandwidth

8.33 [5] Suppose you are designing a microprocessor that uses special instructions to access I/O devices (instead of mapping the devices to m em ory addresses). What specia l instructions would you need to include? What additional bus lines would you need this microprocessor to support in order to address I/O devices? 8.34 An important advantage of interrupts over polling is the ability of the processor to perform other tasks while waiting for communication from an I/O device. Suppose that a I GHz processor needs to read 1000 bytes of data from a particular I/O device. The I/O device supplies I byte of data every 0.02 ms. The code to process the data and store it in a buffer takes 1000 cycles. a. If the processor detects that a byte of data is ready through polling, and a polling iteration takes 60 cycles, how many cycles does the entire operation take? b. If instead, the processor is interrupted when a byte is ready, and the processor spends the time between interrupts on another ta sk, how many cycles of this other ta sk ca n the processor complete while the I/O communication is taking place? The overhead for handling an interrupt is 200 cycles.

II For More Pra ctice: Finding I/O Bandwidth Bottlenecks [ I S) II For More Pra ctice: Finding I/O Bandwidth Bottlenecks [ I S) II For More Practice: I/O System Operation

8.35 (20) 8.36 8.37

8.38 (10 ) Write a paragraph identifying some of the simplifying assumptions made in the analysis below: Suppose we have a processor that executes with a Soo-M Hz clock and the number of clock cycles for a polling operation- including transferring to the polling routine, accessing the devise, and restarting the user program-is 400. The hard disk

620

Chapter 8

tran sfers data in four-wo rd chunks and ca n transfer at 4 MB/sec. Assume that you poll oft en enough that no data is ever lost and assume that the hard disk is potentially always busy. The initial senlp of a DMA transfer takes 1000 clock cycles for the processor, and the handling of the interrupt at DM A completion requires 500 clock cycles for the processor. The hard disk has a transfer rate of 4 MB/sec and uses DM A. Ignore any impact from bus contention between the processor and the DM A controller. Therefore, if the average transfer from the disk is 8 KB, the fraction of the 500- MHz processor consumed if the disk is actively transferring 100% of the time is 0.2%.

8.39 (81 Suppose we have the same hard disk and processor we used in Exercise 8.18, but we use interrupt-driven I/O. The overhead for each transfer, including the interrupt, is 500 clock cycles. Find the fra ction of the processor con sumed if the hard disk is only transferring data 5% of the time. 8.40 [8] Suppose we have the sa me processor and hard disk as in Exercise 8.18. Assume that the initial setup of a DMA transfer takes 1000 clock cycles for the processor, and assume the handling of the interrupt at DM A completion requires 500 clock cycles for the processor. The hard disk has a transfer rate of 4 MB/sec and uses DMA. If the average transfer from the disk is 8 KB, what fraction of the 500MHz processor is consumed if the disk is actively transferring 100% of the time? Ignore any impa ct from bus contention between the processo r and DM A controller. 8.41 [2 days-I week] . Explore I/O

For Mo re Pra ctice: Using SPIM to

8.42 [3 days-I week] to Perform I/O

tel

For More Practice: Writing Code

8.43 [3 days-I week] to Perform I/O

For More Practice: Writing Code

8.44 [ IS) Redo the exa mple on page 60 1, but instead assume that the reads are random 8-KB reads. You ca n assume that the reads are always to an idle disk, if one is available. 8.45 (20) Here are a variety of building blocks used in an I/O system that has a synchronous processor-memory bus running at 800 MHz and one or more I/O adapters that interface I/O buses to the processor-memory bus.

• Memory system: The memory system has a 32-bit interface and handles four-word transfers. The memory system ha s separate address and data lines and , for writes to memory, accepts a word every clock cycle for 4 clock cycles and then takes an additional 4 clock cycles before the words have been stored and it ca n accept another transaction.

8.12

621

Exerci ses

• DMA in terfaces: The I/O adapters use DM A to transfer the data between the I/O buses and the processor- memory bus. The DMA unit arbitrates for the p rocesso r-memory bus and sends/receives four-word blocks from/to the memo ry system. The DM A cont roller ca n accommodate up to eight disks. Initiating a new I/O operation (including the seek and access) takes 0.1 ms, during which another I/O ca nnot be initiated by this cont roller (but out standing operations ca n be handled). •

I/O bus: The I/O bus is a synchronous bus with a sustainable bandwidth of 100 MB/sec; each tran sfer is one wo rd long.

• Disks: The disks have a measured average seek plus rotational latency of 8 m s. The disks have a read/write bandwidth of 40 MB/sec, when they are transferring. Find the time required to read a 16 KB sector from a disk to memory, assuming that this is the only activity on the bus.

8.46 [5 J In order to perfo rm a disk o r network access, it is typically necessary fo r the user to have the operating system communicate with the disk or net work controllers. Suppose that in a particular 5 G Hz computer, it takes 10,000 cycles to trap to the as, 20 ms for the OS to perform a disk access, and 25 fls for the as to perfo rm a network access. In a disk access, what percentage of the delay time is spent in trapping to the aS? How about in a network access? 8.47 [5 J Suppose th at in the computer in Exercise 8.46 we can somehow reduce the time for the OS to communica te with the disk cont roller by 60%, and we ca n reduce the time for the OS to communicate with the netwo rk by 40% . By what percentage ca n we reduce the total time for a netwo rk access? By what percentage can we reduce the total time for a disk access? Is it worthwhile for us to spend a lot of effo rt improving the as trap latency in a computer that performs many disk accesses? How about in a computer that perfo rm s many network accesses? §8. 2, Page §8. 3, Page §8.4, Page §8.5, Page §8.6, Page

580: Dependability: 2 and 3. RAID: All are true. 8.3- 10: 1. 587: 1 and 2. 597: 1 and 2. 600: 1 and 2.

Answers To Check Yourself

Computers in the Real World

Saving Lives through Better Diagnosis

Problem: Find a way to examme internal

organs to diagnose psychological problems without the use of invasive surgery or harmful radiation. Solution: The development of magnetic res-

onance imaging (M RI), a three-dimensional scanning technology, has been one of the most important breakthroughs in modern medical technology. MRI uses a combination of radiofrequency pulses and magnetic fields to scan tissue. The organ to be imaged is scanned in a series of two-dimensional slices, which are then composed to create a three-dimensional tillage. In addit ion to this computationa lly intensive task of composing the slices to create a volumetric image , extensive computation is used to extract the initial two-dimensional images, since the signa l-to-noise ratio is often

low. The development of MRI has allowed the scanning of soft tissues, such as the brain , for which X-rays are not as effective and exploratory surgery is dangerous. Without a costeffective computing capability, MRI would remain slow and expensive. The two illustrations shows a series of MRI images of the human brain; the images below represent two-dimensional slices, while those on the facing page show a three-dimensional reconstruction. Once an image is in digital form, a physician can manipulate the image, removing outer layers, examining the image from different viewpoints, or looking at the three-dimensional structure to help in diagnosis. The major benefits of MRI are twofold: •

It can reduce the need for unnecessary exploratory surgery. A physician may be able to determine whether a patient ex-

MRiImag&s of a human brain, In two-dImensional view

periencing headaches has a brain tumor, which requires surgery, or simply needs medication for a headache. •

By providing a smgeon with an accmate three-dimensional image, MRI can im-

accurately perform surgery, particularly when using minimally invasive techniques. •

application that uses MRI to examine brain function, primarily by analyzing blood flow in various portions of the brain. FMRI is being used for a number of applications, including eA"ploring the physiological bases for cognitive problems such as dyslexia, pain management, planningfor neurosurgery, and understanding neurological disorders.

prove the smgical planning process and hence the outcome. For example, in operating on the brain to remove a tumor without accurate images of the tumor, the surgeon likely would have to enter the brain and then create a plan on the fly depending on the size and exact placement of the tumor. Fwthermore, minimally invasive techniques (e.g. endoscopic surgery), which have become quite effective, would

To learn more see these references on

be impossible without accurate images.

the

There are many new interesting uses of MRI technology, which rely on faster and more cost effective computing. Some of the most promIsmg are •

•

Functional MRI (FMRI): a new type of

library

MRI scans from the National Institutes of Health's Visible Human project Principles of MRI and its application to medical imaging (long and reasonably detailed, but only a little mathematics) Using MRI to do real-time cardiac imaging and a ngiography (i maging of blood vessels)

real-time imaging of the heart and blood vessels to enhance diagnosis of cardiac and cardiovascular disease;

Functio nal MRI, www.fmri.org/fmri.htm

Combining real-time images and MRI images during surgery to help surgeons

Visualization and imaging (including MRI and CT images): high -performan ce co mputing for complex Images

MRllmages of a human brain In three dimensions

G-l

Glossary

a bsolute address A variable's or ro utine's actual address in memo ry. a bstraction A m odel that renders lowerlevel details of computer systems tempora rily invisible in order to facilitate design of sophisticated system s. acronym A wo rd constructed by taking the initial letters of string of wo rds. For exam ple: RAM is an acro nym for Random Access Memory, and CPU is an acro nym fo r Cen tral Processing Unit. active matrix display A liquid crystal display using a transisto r to contro l the transmission of light at each individual pixeL address translation Also called address m apping. The process by which a vi rtual address is mapped to an address used to access m em ory. add ress A value used to delineate the location of a specific data element within a m em ory array. add ressing mode One of several addressing regimes delimited by their va ried use of o perands and/o r addresses. advanced load In IA-64, a speculative load instruction with support to check for aliases that could invalidate the load. aliasing A situation in which the same o bject is accessed by two addresses; can occur in virtual memory when there are two vi rtual addresses fo r the same physical page. alignment restriction A requirement that data be align ed in memory on natural boundaries

Amdahl's law A rule stating that the perfo rmance enhancement possible with a given improvement is limited by the amo unt that the improved feature is used. antid ependence Also called name depend ence. An o rdering fo rced by the reuse of a nam e, typically a register, rather then by a true dependence that carries a value between nvo instructions. antifu se A structure in an integrated circuit that when programmed makes a permanent connectio n between nvo wires. application binary int erface (ABI) The user portio n of the instructio n set plus the o pera ting system interfaces used by a pplicatio n progra mmers. Defin es a sta nda rd fo r bina ry po rtability across co mputers. architectural registers The instructio n set visible registers of a processor; fo r example, in M IPS, these are the 32 integer and 16 floating- point registers. arithmetic m ean The average of the execution times that is directly propo rtional to total executio n time. assem bler directive An operatio n that tells the assembler how to translate a program but does not produce mac hine instructions; always begins with a period. assembler A program that translates a symbolic versio n of instructio ns into the binary versIOn. assembl y language A symbolic language that can be translated into binary.

G-2

Glossary

asserted sign al A signal that is (logically) true, o r 1. asynchro no u s bus A bus that uses a ha ndshaking protocol fo r coordinating usage rather than a clock; can acco mmodate a wide variety of devices of differing speeds. a tomic swap opera tion An o peration in which the processo r can both read a location and write it in the sam e bus o peration , preventing any other processo r o r I/O device fro m reading o r writing memory until it completes. backpatching A method fo r tra nslating fro m assembly language to machine instructions in which the assembler builds a (possibly incomplete) binary representatio n of every instructio n in o ne pass over a program and then returns to fill in previo usly undefined labels. backplane bus A bus that is design ed to allow processors, memory, and I/O devices to coexist o n a single bus. barrier synchroniza tion A synchronization schem e in which processors wait at the barrier and do not proceed until every processor has reached it. basic block A sequence of instructio ns witho ut bran ches (except possibly at the end ) a nd witho ut branch targets o r branch labels (except possibly at the beginning). behavioral specification Describes how a digital system operates functio nally. biased no tatio n A notation that represents the m ost negative value byOO ooolWOand the most positive value by II Il lwo, with o typically having the value 10 OOt....u' thereby biasing the number such that the number plus the bias has a nonnegative representation. binary digit Also called a bit. One of the two numbers in base 2 (0 o r I) that are the components of info rmatio n.

bit erro r rate The fr actio n in bits of a message o r collectio n of messages that is incorrect. block The minimum unit of informatio n that can be either present o r not present in the two-level hierarchy. blocking assignment In Verilog, an assignment that com pletes before the executio n of the neA1: statem ent. bra nch delay slot The slo t directly aft er a delayed branch instructio n, which in the M IPS architecture is filled by an instructio n that does no t affect the branch. branch not taken A branch where the branch conditio n is false and the program counter (PC) becomes the address of the instructio n that sequentially follows the branch. bra nch prediction A m ethod of resolving a branch hazard that assumes a given o utcom e fo r the branch and proceeds fro m that assumptio n rather than waiting to ascertain the ac tual o utcome. branch prediction buffer Also called branch histo ry table. A small memory that is indexed by the lower portion of the add ress of the branch instructio n and that contains one o r m ore bits indicating whether the branch was recently taken o r no t. bra nch taken A branch where the branch condition is satisfied and the program counter (PC) becomes the branch target.A11 unconditional branches are taken branches. branch target address The address specifi ed in a branch, which becom es the new program counter ( PC) if the branch is taken. l n the M IPS architecture the branch target is given by the sum of the offset field of the instructio n and the address of the instructio n following the branch. bra nch target buffer A structure that caches the destination PC o r destination instructio n fo r a branch. It is usually o rganized as a

G -3

Glossary

cache with tags, m aking it more costly than a simple prediction buffer. bu s In logic design , a collection of data lines that is treated together as a single logical signal; also, a shared collectio n oflines with multiple sources and uses. bu s master A unit o n the bus that can initiate bus requests. bu s tran saction A sequence of bus o perations that includes a request and m ay include a response, either of which m ay carry data. A transaction is initiated by a single request and may take many individual bus operations. cache coherency Consistency in the value of data benvee n the versions in the caches of several processors. cache coherent NUMACC-NUMA A no nuniform memory access multiprocessor that maintains coherence for all caches. cach e memory A sm all, fast memory that acts as a buffer for a slower, larger m em ory. cach e miss A request for data from the cache that cannot be filled because the data is not present in the cache. callee A procedure that executes a series of stored instructio ns based o n parameters provided by the caller and then returns control to the caller. callee-saved register A register saved by the routine making a procedure calL caller The program that instigates a procedure and provides the necessary param eter values. caller-saved register A register saved by the routine being called. capacit y miss A cache miss that occurs because the cache, even with full associativity, cannot contain all the block needed to satisfy the request. carrier signal A continuo us sign al of a single frequency capable of being m odulated by a second data-carryi ng signaL

cathode ray tube (CRT ) display A display, such as a televisio n set, that displays an image using an electro n beam scanned across a screen. central processo r unit (CP U) Also called processo r. The active part of the computer, which contains the datapath and control and which adds numbers, tests numbers, sign als I/O devices to activate, and so o n. clock cycle Also called tick, clock tick, clock period, clock, cycle. The time fo r one clock period, usually of the processo r clock, which nms at a constant rate. clock cycles per instruction (CPI) Average number of clock cycles per instruction fo r a program o r program fra gm ent. clock period The length of each clock cycle. clock skew The difference in absolute time between the times when two state elements see a clock edge. clocking m ethodology The approach used to determine when data is valid and stable relative to the clock. cluster A set of computers connected over a local area netwo rk ( LAN) that functio n as a single large multiprocessor. combina tional logic A logic system whose blocks do not contain m em ory and hence compute the same o utput give n the sam e input. commit unit The unit in a dynamic or outof-order executio n pipeline that decides when it is safe to release the result of a n operation to programmer-visible registers and memory. compiler A program that translates highlevel language statem ents into asse mbly language statem ents. compulsory miss Also called cold sta rt miss. A cache miss caused by the first access to a block that has never been in the cache. conditional branch An instruction that requires the comparison of two values and

G-4

Glossary

that allows fo r a subsequent t ransfer of cont rol to a new address in the program based o n the o utcome of the comparison. confli ct miss Also called collision miss. A cache miss that occu rs in a set-associative o r direct-mapped cache when multiple blocks compete for the sam e set and that are eliminated in a fully associative cache o f the sam e Size. con stellation A cluster that uses an SMP as the building block. context switch A changi ng of the intern al state of the processor to allow a different process to use the processor that includes saving the state need ed to return to the currently executing process. cont ro l The compo nent o f the processor that commands the d atapath, memo ry, and I/O devices acco rding to the instructions of the program . cont ro l hazard Also called b ra nch hazard . An occurrence in which the proper instruction cannot execute in the proper clock cycle because the instructio n that was fet ched is not the o ne that is needed; that is, the flow o f instructio n addresses is not w hat the pipeline expected. cont ro l signal A sign al used fo r multiplexo r selectio n o r for directing the o peration o f a functio nal unit; contrasts with a d ata signal, which contains info rm atio n that is operated on b y a functio nal unit. cor relating p redict or A branch predicto r that combines local behavio r o f a particular branch and global info rmatio n abo ut the behavio r o f som e recent number of executed branches. C P U execution time Also called C P U time. The actual time the CPU spends computing fo r a specific task. crossbar net work A network that allows an y no d e to communicat e w ith an y other node in o ne pass thro ugh the network.

D flip -fl op A flip-flo p with o ne d at a input that sto res the value o f that input signal in the internal memory when the clock ed ge occu rs. d ata hazard Also called pipeline d at a hazard. An occurrence in w hich a planned instructio n cannot execute in the proper clock cycle because data that is needed to execute the instructio n is no t yet available. d ata par allelism Parallelism achieved by havi ng m assive d ata. d ata rat e Perfo rmance m easure o f bytes per unit time, such as G B/seco nd. d ata segm ent The segment o f a UNIX object or executable file that contains a binary representation o f the initialized d ata used b y the p rogram . d ata t ransfer inst r uction A command that moves data between m em ory and registers. d atapa th The compo nent o f the processor that perfo rms a rithmetic oper atio ns. d ata path elem ent A fun ctional unit used t o operate on or h old data within a processor. In the M IPS implementatio n the d at apath elements include the instructio n and d ata memo ries, the register file, the arithmetic logic unit (ALU) , and adders. d eassert ed signal A sign al that is (logically) false, o r O. d ecoder A logic block that has an II- bit input and 2/1 o utputs where only o ne o utput is asserted fo r each input combination. d efect A microscopic flaw in a wafer o r in patterning steps that can result in the failure o f the die containing that d efect. d elayed branch A type of branch where the instructio n immediately fo llowing the branch is always executed , independent of whether the branch condition is true or false. d eskt op com puter A computer d esign ed fo r use by an individual, usually incorporating a graphics display, keyboard, and mouse.

G -5

Glossary

die The individual rectangular sectio ns that are cut from a wafer, more info rmally known as chips. DIMM (d ual inline m em o r y mod ule) A sm all board that contains DRAM chips o n both sides. SIMMs have DRAMs on only o ne side. Both DlMMs and SIMMs are m eant to be plugged into memo ry slo ts, usually on a m otherboard. di rect m emo r y access (DMA) A mechanism that provides a device controller the ability to transfer data directly to o r from the memory without involving the processor. direct-m a pped cache A cache structure in which each memo ry locatio n is mapped to exactly o ne locatio n in the cache. directory A repository fo r informatio n o n the state of every block in m ain memory, including which caches have copies of the block, whether it is dirty, and so on. Used fo r cache coherence. dispatch An operatio n in a microprogrammed control unit in which the next microinstructio n is selected on the basisof oneor mo re fields of a macroinstructio n, usually by creating a table containing the addresses of the target microinstmctions and indexing the table using a field of the macroinstmction. The dispatch tables are typically implemented in ROM or programmable logic array (PLA). The term dispatch is also used in dynamically scheduled processors to refer to the process of sending an instruction to a queue. dist ributed memory Physical m emory that is divided into modules, with som e placed near each processor in a multiprocessor. distrib uted shared m emo ry (DSM) A memo ry scheme that uses addresses to access remote data when demanded rather than retrieving the data in case it might be used. dividend A number being divided. divisor A number that the dividend is divided by.

do n't-care term An element of a logical function in which the output does not depend on the values of all the inputs. Do n'tcare terms may be specified in different ways. double p recision A floating- point value represented in two 32-bit wo rds. d ynamic branch prediction Predictio n of branches at runtime using runtime info rmatio n. d ynamic multiple issu e An approac h to implementing a multiple-issue processor where m any decisio ns are made during executio n by the processor. d ynamic pipeline scheduling Hardware support fo r reordering the o rder of instructio n executio n so as to avoid stalls. d ynamic rando m access m emo r y (DRAM) M em o ry built as an integrated circuit, it provides rando m access to any locatio n. edge-t riggered d ockin g A clocking schem e in which all state changes occur on a clock edge. embedd ed computer A computer inside a nother device used fo r runnin g o ne predetermined applicatio n o r collectio n of softwa re. er ror-d etecting cod e A code that enables the detectio n of an error in data, but no t the precise locatio n , and hence correctio n of the err or. Ethernet A computer network whose length is limited to abo ut a kilom eter. O riginally capable of transferring up to 10 million bits per second, nelver versions can run up to 100 millio n bits per second and even 1000 millio n bits per second. It treats the wire like a bus with multiple m asters and uses collisio n detectio n and a back-off schem e for handling simultaneous accesses. exception Also called interrupt. An unscheduled event that disrupts program executio n; used to detect overflow.

G-6

Glossary

exception enable Also called intermpt en able. A signal or action that controls whether the process respo nds to an exceptio n o r not; necessary for preventing the occurrence of exceptions during intervals before the processor has safely saved the state needed to restart. executable fil e A functional program in the fo rmat of an object file that contains no unresolved references, relocation information , symbol table, or debugging information. exponent In the numerical representation system of floating- point arithmetic, the value that is placed in the eAvonent field. external label Also called global labeL A label referring to an o bject that can be referenced from files other than the one in which it is defined. fal se sharing A sharing situation in which two unrelated shared variables are located in the sam e cach e block and the full block is exchanged between processors even though the p rocessors are accessing different variables. field progr amma ble d evices (FPD) An integrated circuit containing combinati on al logic, and p ossibly m em ory d evices, that is configur able by the end u ser. field prog ramma ble ga te array A configurable integrated circuit containing bo th combinati on al logic blocks and flip -flop s. finite st ate machine A sequential logic functio n con sisting of a set o f inputs and outputs, a next-state functio n that m aps the current state and the inputs to a n elV st at e, and an output functio n that m ap s th e current state and p ossibly the inputs to a set o f asserted o utputs. firmware Micro code implem ented in a m em ory structure, typically ROM o r RAM. flat pan el display, liquid crystal display A display technology using a thin layer o fliquid polymers that can be used to transmit or block light according to whether a charge is applied.

flip-flop A m em ory elem ent fo r which the o utput is equ al to the value o f the stored state inside the elem ent and fo r which the internal state is ch anged o nly o n a clock edge. floa ting point Computer arithmetic that represents numbers in which the binary point is n ot fixed . floppy disk A portable fo rm o f secondary m em ory composed of a rotating m ylar plat t er coated with a m agn etic r ecording m ateriaL flush (i n structio n s) To discard instructio ns in a pipeline, u su ally due to an un exp ected event. formal parameter A variable that is the argument to a p rocedure or m ac ro; replaced by that argument o nce the m ac ro is eAl'anded . fo rward r efer ence A label that is u sed befo re it is d efined . forwarding Also called bypassing. A m etho d of resolving a data h azard b y retrieving the missing d ata elem ent fro m intern al buffers rather than waiting fo r it to arrive from programmer -visible registers or m em ory. fraction The value, gen er ally b etween 0 and 1, placed in the fractio n fi eld. frame pointer A value d eno ting the locatio n of the saved r egisters and local variables fo r a give n procedure. fully asso ciative cach e A cach e structure in which a block can be placed in an y location in the cach e. fully connected n et work A n etwork that connects processor-m em ory nodes by supplyi ng a dedicated communication link between every n ode. ga te A d evice that implem ents b asic logic functio ns, such as AND or O R. gen eral-purpose r egister (GPR) A regist er that can b e used fo r addresses or fo r data with virtually any instruction. globa l miss r a te The frac tio n o f refer en ces that miss in all levels o f a multilevel cach e.

G -7

Glossary

global pointer The register that is reserved to point to static data. guard The first of two eA1:ra bits kept on the right during intermediate calculations of floating-point numbers; used to improve rounding accuracy. handler Name of a software routine invoked to "handle" an exception o r interrupt. handshaking protocol A series of steps used to coordinate async hrono us bus transfers in which the sender and receiver proceed to the neA1: step only when both parties agree that the current step has been completed. hardware description language A programming language for describing hardware used for generating simulations of a hardware design and also as input to synthesis tools that can generate actual hardware. hardware synthesis tools Computer-aided design software that can generate a gate-level design based on behavioral descriptions of a digital system . hardwired control An implem entation of finite state machine control typically using programmable logic arrays ( PLAs) o r collections of PLAs a nd random logic. h exadecimal Numbers in base 16. high-level programming language A po rtable language such as C, Fo rtran, or Java composed of wo rds and algebraic notation that can be translated by a compiler into assembly language. hit rate The fraction of memory accesses found in a cache. hit time The time required to access a level of the m em ory hierarchy, including the time needed to determine whether the access is a hit or a miss. hold time The minimum time during which the input must be valid after the clock edge. hot swapping Replacing a hardware com ponent while the system is running.

ItO instru ctions A dedicated instructio n that is used to give a command to an 110 device and that specifies both the device number and the command wo rd (o r the location of the command wo rd in m em ory). ItO rate Performance measure ofllOs per unit time, such as reads per second. ItO requests Reads o r writes to 110 devices. implementation Hardware that obeys the architecture abstrac tio n. imprecise interrupt Also called imprecise exceptio n. Interrupts or exceptio ns in pipelined computers that are not associated with the exact instructio n that was the cause of the interrupt or exceptio n. in-order commit A commit in which the results of pipelined execution are written to the programmer-visible state in the same order that instructions are fetched. input device A m echanism through which the computer is fed information , such as the keyboard o r mouse. instruction format A form of representatio n of an instructio n composed of field s of binary numbers. instruction group In IA-64, a sequence of consecutive instructio ns with no register data dependences am ong them. instruction latency The inherent executio n time fo r an instructio n. instruction mix A measure of the dynamic frequency of instructions across o ne or many program s. instruction set architecture Also called architecture. An abstract interface between the hardwa re and the lowest level software of a machine that encompasses all the info rmation necessary to write a m achine language program that will run correctly, including instructions, registers, m em ory access, 110, and so o n. instruction set The vocabulary of com mands understood by a given architecture.

Glossary

instructio n -level parallelism The parallelism amo ng instructio ns. integrated circuit Also called chip. A d evice combining d ozens to millions of transisto rs. interrupt An exceptio n that comes from o utside of the processor. (Some architectures use the teon interrupt fo r all exceptio ns.) interrupt-d riven 110 An I/O schem e that employs interrupts to indicate to the processor that an I/O device needs attentio n. interrupt handler A piece of code that is run as a result of an exception or an interrupt. issue packet The set of instructions that issues together in 1 d ock cycle; the pac ket m ay be determined statically by the compiler or dynamically by the processor. issue slots The positio ns fr om which instructions could issue in a give n d ock cycle; by analogy these correspond to positio ns at the sta rting blocks fo r a sprint. Java bytecod e Instructio n fro m an instruction set designed to interpret Java programs. Java Vi r tual Machine OVM ) The progra m that interprets Java bytecodes. j ump address table Also called jump table. A table o f addresses of alternative instruction sequences. jump-and-link instruction An instructio n that jumps to an address and simultaneo usly saves the address of the fo llowing instruction in a register ( $ r a in M IPS). Just In Tinle Compiler (JIT) The name com mo nly given to a compiler that operates at runtime, translating the interpreted code segments into the native code of the computer. kernel mod e Also called s uper viso r mo de. A mode indicating that a running process is an o perating system process. latch A memory element in which the o utput is equal to the value o f the sto red state inside the element and the state is changed whenever the appropriate inputs change and the d ock is asserted.

latency (pipeline) The number o f stages in a pipeline o r the number o f stages between two instructio ns during execution. least rece ntly used (LRU) A replacem ent schem e in which the block replaced is the o ne that has been unused fo r the lo ngest time. lea st significant bit The rightmost bit in a M IPS wo rd. level- sen sitive d ocking A timing metho d o logy in which state changes occur at either high o r low clock levels but are no t instantaneous, as such changes are in ed ge- triggered d esign s. linker Also called link edito r. A system s program that combines independently assembled machine language program s and resolves all undefined labels into an executable file. load er A system s progra m that places an object program in main memory so that it is ready to execute. load -store ma chine Also called registerregister machine. An instructio n set architecture in w hich all operatio ns are benveen registers and data memory may o nly be accessed via lo ads or stores. load -use d ata hazard A specific fo rm o f d ata hazard in which the d ata requested by a load instructio n has no t yet become available when it is requested. local area net work (LAN) A network d esign ed to car ry d ata within a geographically confined area, typically within a single building. local label A label referring to an object that can be used o nly within the file in which it is d efin ed. local miss rate The fraction o f referen ces to o ne level o f a cache that miss; used in multilevel hierarchies. lock A synchronizatio n d evice that allows access to data to o nly o ne processor at a time.

G-.

Glossary

lookup tables (lUIs) In a field program m able device, the nam e given to the cells because they consist of a small amount oflogic and RAM. loop unrolling A technique to get more performance from loops that access arrays, in which multiple copies of the loop body are m ade and instructions fro m different iterations are scheduled together. machine language Binary representation used for communication within a computer system . macro A pattern-m atching and replacem ent facility that provides a simple mechanism to nam e a frequently used sequence of instructions. magnetic disk (also called hard disk) A form of nonvolatile secondary memory composed of rotating platters coated with a m agnetic recording material. megabyte Traditionally 1,048,576 (220) bytes, although some communications and secondary storage system s have redefined it to mean 1,000,000 (10 6) bytes. memory The storage area in which programs are kept when they are running and that contains the data needed by the runIlIng program s. memory hierarchy A structure that uses multiple levels of m emories; as the distance from the CPU increases, the size of the mem ories and the access time both IIlcrease. memory- mapped I/O An I/O scheme in which portions of address space are assigned to I/O devices and reads and writes to those addresses are interpreted as com m ands to the I/O device. MESI cache coherency protocol A write-invalidate protocol whose name is an ac ron ym for the four states of the pro tocol: M odified , Exclusive, Shared , Invalid.

message passing Communicating between multiple processors by explicitly sending and receiving information. metastability A situation that occurs if a sign al is sampled when it is not stable for the required set-up and hold times, possibly causing the sampled value to fall in the indeterminate region bet"veen a high and low value. microarchitecture The organization of the processo r, including the major fun ctional units, their interconnection, and control. microcode The set of microinstructions that control a processor. microinstruction A representation of cont rol using low- level instructions, each of which asse rts a set of control signals that are ac tive on a given clock cycle as well as specifies what microinstruction to execute next. micro-operations The RI SC-like instructions directly executed by the hardware in recent Pentium implementations. microprogram A symbolic representation of control in the form of instructions, called microinstructions, that are executed on a simple micromachine. microprogrammed control A method of specifying control that uses microcode rather than a finite state representation. million instructions per second (MI PS) A measurem ent of program execution speed based on the number of millions of instructions. M IPS is computed as the instruction count divided by the product of the execution time and 106. minterms Also called product terms. A set oflogic inputs joined by conjunction (AND operations); the product terms form the first logic stage of the progra mmable logic array ( PLA). mirroring Writing the identical data to multiple disks to increase data availability.

CHO

Glossary

miss penalty The time required to fetch a block into a level of the memory hierarch y fro m the lower level, includ ing the time to access the block, transmit it from o ne level to the other, and insert it in the level that experienced the miss. miss rate The fractio n of memory accesses not fo und in a level ofthe memory hierarchy. most significant bit The leftm ost bit in a M IPS wo rd. motherboa rd A plastic board containing packages of integrated circ uits o r chips, includ ing processor, cache, memory, and connectors fo r I/O devices such as netwo rks and disks. multicomputer Parallel processors with multiple p rivate addresses. multicyde implem entation Also called multiple clock cycle implementatio n. An implem entation in which an instruction is executed in multiple clock cycles. multilevel cache A memory hierarchy with multiple levels o f caches, rather than just a cache and main mem ory. multiple issue A scheme whereby multiple instructio ns are launched in I clock cycle. multiprocessor Parallel p rocessors with a single shared address. multistage network A netwo rk that supplies a sm all switch at each node. NAND gate An inverted AND gate. net work bandwidth Infor mally, the peak transfer rate of a netwo rk; can refer to the speed of a single link or the collective t ransfer rate o f all links in the netwo rk. next-sta te function A combinatio nalli.lllction that, give n the inp uts and the current state, determines the next state o f a finite state machine. non blocking a ssignment An assignment that continues after evalua ting the right-hand side, assigning the left-hand side the value only after all right-hand sides are evaluated.

nonblockingcache A cache that allows the p rocesso r to make references to the cache while the cache is hand ling an earlier miss. nonuniform m emor y a ccess (NUMA) A type of single-add ress space multip rocessor in which some m em ory accesses are faster than o thers depending w hich p rocessor asks fo r which wo rd. nonvolatile m emory A fo rm o f m emory that retains d ata even in the absence o f a power source and that is used to sto re p rograms between runs. Magn etic d isk is no nvolatile and DRAM is not. nonvolatile Storage device where data retains its value even when power is removed. nop An instructio n that does no operation to change state. NOR A logical bit-by-bit operatio n with two operands that calculates the N OT o f the O R o f the two o perands. NOR gate An inverted O R gate. normalized A number in floating- po int no tatio n that has no leading Os. NOT A logical bit-by- bit o peratio n with o n e o perand tha t inverts the bits; that is, it replaces every 1 with a 0, a nd every 0 with a I. object-oriented la nguage A p rogramming language that is o riented aro und objects rather than actio ns, o r data versus logic. opcode The field that de no tes the o peratio n and fo rmat o f an instructio n. o perating system Supervising p rogram that ma nages the resources o f a computer fo r the benefit o f the p rograms that run on that mac hine. out-of-order execution A situatio n in pipelined execution when an instruction blocked fro m executing d oes no t cause the fo llowing instructio ns to wait. output d evice A mechanism that conveys the result o f a computatio n to a user o r ano ther computer.

G-ll

Glossary

overflow (floa ting- point) A situation in which a positive exponent becom es too large to fit in the exponent field. package Basically a directo ry that contains a group of related classes. page fault An event that o ccurs when an accessed page is n ot present in m ain m em o ry. page table The table containing the virtual to physical address translatio ns in a virtual m em o ry system. The table, which is stored in m em o ry, is typically indexed by the virtual page number;each entry in the table contains the physical page number for that virtual page if the page is currently in m em ory. parallel processing program A single program that runs o n multiple processors simultan eo usly. PC- relative addressing An addressing regime in which the address is the sum o f the program counter (PC) and a constant in the instructio n. physical a ddress An addr ess in m ain m em o ry. phys icall y addressed cache A cache that is addressed by a physical address. pipeline stall Also called bubble. A st all initiated in o rder to r esolve a h azard. pipelining An implem entation technique in which multiple instru ctio ns are over lapped in exec utio n, much like to an asse m bly line. pixel The sm allest individual picture elem ent. Screen are composed o f hundred s o f thousands to millio ns o f pixels, o rganized in a m atrix. poison A result gen erated wh en a speculative load yields an exceptio n, o r an instruction uses a p o isoned operand. polling The process o f periodically check ing the status o f an I/O device t o determine the n eed t o service the device. precise interrupt Also called precise ex ception. An interrupt or exceptio n that is

alway associat ed with the correct instructio n in pipelined computers. predication A technique to m ake instructio ns d ependent o n predicates rather than o n branch es. prefet ching A t echnique in w hich d ata blocks n eeded in the future are brought into the cach e early by the use of special instructio ns that specify the address o f the block. prima r y m emory Also called m ain m em ory. Volatile m em o ry used to ho ld programs while they are running; typically consists of DRAM in to day's computers. procedure A stored subroutine that per fo rms a specific task based o n the paramet ers with which it is provided. procedu re call frame A block of m em ory that is used to hold values passed to a procedure as arguments, to save registers that a procedure m ay m odify but that the p rocedure's caller d oes not want chan ged , and to provide space for variables local to a procedure. procedure frame Also called activation record. The segm ent o f the stac k conta ining a procedure's saved r egisters a nd local variables. processor-m emory bus A bus that con nects processor and m em o ry and that is sho rt, gen er ally high speed , and m at ch ed to the m em ory system so as t o m aximize m em ory- processor bandw idth. program counter (P C) The regist er con t aining the address of the instructio n in the program being executed programmable a rray logic (PAL) Contains a programmable andplan e followed by a fixed o r-plan e. programma ble logic ar ray (PLA) A structured-logic elem ent composed of a set of inputs and correspo nding input complem ents and two stages oflogic: the first generating product terms o f the inputs and input com plem ents and the second generating sum

CH2

Glossary

terms of the product terms. Hence, PLAs implement logic functio ns as a sum of products. programmable logic d evice (PLO) An integrated circuit containing combinational logic whose functio n is configured by the end user. programmable ROM (PROM) A form of read-only memory that can be programmed when a designer knows its contents. propaga tion time The time required for an input to a flip -flop to propagate to the outputs of the flip -flop. protected A Java keyword that restricts invocation of a method to other methods in that package. protection A set of mechanisms for ensuring that multiple processes sharing the processor, memory, o r I/O devices canno t interfere, intentionally or unintentionally, with one another by reading or writing each other's data. These mechanisms also isolate the o perating system from a user process. protection group The group of data disks or blocks that share a common check disk o r block. pseudo instructio n A commo n variatio n of assembly language instmctio ns often treated as if it were an instmctio n in its own right. publk A Java keyword that allows a method to be invoked by any other method. quotient The primary result of a divisio n; a number that when multiplied by the divisor and added to the remainder produces the dividend. read-only m emo r y (ROM ) A memo ry whose contents are designated at creation time, after which the contents can o nly be read. ROM is used as stmctured logic to implem ent a set oflogic functio ns by using the terms in the logic functio ns as address inputs and the o utputs as bits in each wo rd of the m em ory.

receive m essage routine A routine used by a processor in mac hines with private memories to accept a m essage from another processor. recursive procedures Procedures that call themselves either directly or indirectly through a chain of calls. redundant arrays of inexpensive disks (RAID) An o rganization of disks that uses an array of small and inexpensive disks so as to increase both perfonnance and reliability. reference bit Also called use bit. A field that is set whenever a page is accessed and that is used to implem ent LRU or other replacement schemes. reg In Verilog, a register. register fil e A state elem ent that consists of a set of registers that can be read and written by supplying a register number to be accessed. register renaming The renaming of registers, by the compiler o r hardwa re, to remove antidependences. register-use conventio n Also called procedu re call conventio n. A software protocol goveming the use of registers by procedures. relocatio n info rmatio n The segment of a UN IX o bject file that identifies instmctio ns and data words that depend o n absolute addresses. remainder The secondary result of a division; a number that when added to the product of the quotient and the divisor produces the dividend. reorder buffer The buffer that holds results in a dynamically scheduled processor until it is safe to sto re the results to memory or a register. reservation station A buffer within a fun ctio nal unit that holds the o perands and the o peration. response time Also called execution time. The total time required for the computer to complete a task, including disk accesses,

G -13

Glossary

m em ory accesses, I/O activities, operating system overhead, CPU execution time, and so on. restartable instruction An instmction that can resume execution aft er an exception is resolved without the exception 's affecting the result of the instruction. return address A link to the calling site that allows a procedure to retum to the proper address; in M IPS it is stored in register $ r a. rotation latency Also called delay. The time required fo r the desired sector of a disk to ro tate under the read/write head; usually ass umed to be half the rotation time. round Method to make the intermediate floating-point result fit the floating-point format; the goal is typically to find the nearest number that can be represented in the format. scientific notation A notation that renders numbers with a single digit to the left of the decimal point. secondary memory Nonvolatile memory used to store program s and data between mns; typically consists of m agnetic disks in today's computers. secto r One of the segm ents that make up a trac k on a magnetic disk; a sector is the sm allest amount of infor mation that is read or written on a disk. seek The process of positioning a read/write head over the proper track on a disk. segm entation A variable-size address m apping scheme in which an address consists oftwo parts: a segment number, which is m apped to a physical address, and a segm ent offset. selector value Also called cont rol value. The control sign al that is used to select one of the inp ut values of a multi plexor as the out put of the multiplexor. semiconducto r A substance that does not conduct electricity welL

send message routine A ro utine used by a processor in machines with private memories to pass to another processor. sen sitivity list The list of signals that specifies when an always block should be reevaluated. separate compilation Splitting a program across many files, each of which can be com piled without knowledge of what is in the other files. sequential logic A group oflogic elements that contain memory and hence whose value depends on the inputs as well as the current contents of the memory. Server A com puter used for mnning larger program s fo r multiple users often simultaneously and typically accessed only via a network. set-associative cach e A cache that has a fixed number oflocations (at least two) where each block can be placed. set-up time The minimum time that the input to a memory device must be valid befo re the clock edge. sh ared memo ry A m em ory for a parallel processor with a single address space, implyi ng implicit communication with loads and stores. sign-extend To increase the size of a data item by replicating the high-order sign bit of the original data item in the high-order bits of the larger, destination data item . silicon A natural elem ent which is a semiconductor. silicon crystal ingot A rod com posed of a silicon crystal that is between 6 and 12 inch es in diameter and abo ut 12 to 24 inches long. simple programmable logic d evice (SPLD) Programmable logic device usually containing either a single PAi or PLA. single precision A fl oating-point value represented in a single 32-bit wo rd.

CH4

Glossary

single-cycle implementation Also called single clock cycle implementatio n. An implem entation in which an instruction is executed in one clock cycle. small computer system s interface (SCSI) A bus lIsed as a standard fo r I/O devices. snooping carne coherency A method for m aintaining cache coherency in which all cache cont rollers m onitor o r snoop on the bus to detennine whether or not they have a copy of the desired block. source language The high-level language in which a program is originally written. spatial locality The locality principle stating that if a data location is referenced , data locations with nearb y addresses will tend to be referenced soon. speculation An approach whereby the compiler o r processo r guesses the o utcome of an instructio n to remove it as a dependence in executing other instructio ns. split cache A scheme in which a level of the m em ory hierarchy is composed of two independent caches that o perate in parallel with each other with o ne handling instructions and o ne handling data. split transaction protocol A protocol in which the bus is released during a bus transaction while the requester is waiting fo r the data to be transmitted , which frees the bus fo r access by another requester. stack pointer A value denoting the m ost recently allocated address in a stack that shows where registers sho uld be spilled or where old register values can be fo und. stack segment The po rtio n of mem ory used by a program to hold procedure call fram es. stack A data structure fo r spilling registers organized as a last-in-first-out queue. standby spares Reserve hardware resources that can immediately take the place of a failed component.

state element A memory element. static da ta The po rtion of memory that contains data whose size is known to the compiler and whose lifetime is the program's entire execution. static method A method that applies to the whole class rather to an individual object. It is unrelated to static in C. static multiple issue An approac h to implementing a multiple-issue processor where many decisio ns are made by the com piler before execution. static random access m emory (SRAM ) A memory where data is stored statically (as in flip -flops) rather than dynamically (as in DRAM). SRAMs are faster than DRAMs, but less dense and more expensive per bit. sticky bit A bit used in rounding in additio n to gua rd and round that is set whenever there are no nzero bits to the right of the round bit. stop In IA-64, an explicit indicator ofa break between independent and dependent instructions. sto red- program concept The idea that instructio ns and data of m any types can be stored in m em ory as numbers, leading to the stored program computer. striping Allocatio n of logically sequential blocks to separate disks to allow higher performance than a single disk can deliver. structural hazard An occurrence in which a planned instruction cannot execute in the proper clock cycle because the hardware cannot support the combinatio n of instructions that are set to execute in the given clock cycle. stru ctural specification Describes how a digital system is organized in terms of a hierarchical connectio n of elem ents. sum of products A form of logical representatio n that employs a logical sum (OR) of products (ten ns joined using the AND o perato r).

G -15

Glossary

supercomputer A class of computers with the highest perfo rmance and cost; they are configured as servers and typically cost millions of dollars. superscalar An advanced pipelining tech nique that enables the processor to execute more than o ne instructio n per clock cycle. swap space The space o n the disk reserved for the full virtual mem ory space of a process. switch ed n etwork A network of dedicated point-to-point links that are connected to each o ther with a switch. symbol table A table that matches names oflabels to the addresses of the memory words that instructions occupy. symmetric multiprocessor (SMP) o r unifo r m m emor y access (UM A) A multiprocessor in which accesses to main memory take the sam e amo unt of time no matter which processor requests the access and no matter which word is asked. synchro nization The process of coordinating the behavio r of two or mo re processes, which may be running on different processors. synch ronizer failure A situation in which a flip -flo p enters a metastable state and where some logic blocks reading the o utput of the flip -flo p see a 0 while o thers see a 1. synchronous bus A bus that includes a clock in the control lines and a fixed protocol fo r communicating that is relative to the clock. synchro nous system A memory system that employs clocks and where data signals are read o nly when the clock indicates that the signal values a re stable. system call A special instruction that transfers control from user mode to a dedicated locatio n in supervisor code space, invoking the exceptio n m echanism in the process. system CPU time The CPU time spent in the o perating system perfo rming tasks o n behalf of the program.

system perfo r mance evaluation coo perative (SPEC) benchmark A set of standard CPU-intensive, integer and floating point benchmarks based on real programs. system s softwa re Soft""are that provides services that are commo nly useful , including operating systems, compilers, and assemblers. tag A field in a table used fo r a m em ory hierarchy that contains the address info rmatio n required to identify whether the associated block in the hierarchy correspo nds to a requested wo rd. tempora l locality The principle stating that if a data locatio n is referenced then it will tend to be referenced again soon. terabyte Originally 1,099,5 11 ,627,776 (240) bytes, altho ugh some communications and secondary storage systems have redefined it to mean 1,000,000,000,000 (10 12 ) bytes. text segm ent The segm ent of a UN IX o bject file that contains the m ac hine language code for routines in the source file. three Cs model A cache model in which all cache misses are classified into o ne of three categories: compulsory misses, capacity misses, and conflict misses. tournament branch predictor A branch predictor with multiple predictions for each branch and a selectio n mechanism that chooses which predictor to enable for a given branch trace cadle An instruction cache that holds a sequence of instructio ns with a given starting address; in recent Pentium implementations the trace cache holds microoperatio ns rather than lA-32 instructio ns. track One of thousands of concentric circles that m akes up the surface of a magn etic disk. transaction processing A type of applicatio n that involves handling small short o perations (called transactions) that typically require both I/O and computatio n. Trans-

CH6

Glossary

action processing applicatio ns typically have both response time requirements and a performance m easurement based on the thro ughput of transactions. tran sistor An on/off switch contro lled by an electric sign aL translation-Iookaside buffer (TLB) A cache that keeps track of recently used address mappings to avoid an access to the page table. underflow (floating- point) A situation in which a negative exponent becom es too large to fit in the exponent fi eld. units in the last place (ulp) The number of bits in error in the least significant bits of the significand between the ac tual number and the number that can be prepresented. unmapped A portion of the address space that cannot have page faults. unresolved refereO(;e A reference that requires m ore information fro m an o utside source in order to be complete. untaken branch One that falls th ro ugh to the successive instruction. A taken branch is one that causes transfer to the branch target. user CPU time The CPU time spent in a program itself. vacuum tube An electronic component, predecessor of the transistor, that consists of a hollow glass tube about 5 to 10 cm long fro m which as much air has been removed as possible and which uses an electron beam to transfer data. valid bit A field in the tables of a memo ry hierarchy that indicates that the associated block in the hierarchy contains valid data. vector processor An architecture and com piler model that was popularized by supercomputers in which high-level operatio ns work o n linear arrays of numbers. vectored interrupt An interrupt fo r which the address to which cont rol is transferred is

determined by the cause of the exception. verilog One of the two m ost commo n hardwa re descriptio n la nguages. very large scale integrated (VLSI) circuit A device containing hundreds of tho usands to millio ns of transistors. VHDL One of the two most commo n hardwa re descriptio n la nguages. virtual address An address that corresponds to a location in vi rtual space and is translated by address mapping to a physical address when memo ry is accessed. virtual machine A virtual computer that appears to have nondelayed branches and loads and a richer instruction set than the actual hardware. virtual memo ry A technique that uses main memo ry as a "cache" for secondary storage. virtually addressed cache A cache that is accessed wi th a vi rtual address rather than a physical address. volatile m emo r y Storage, such as DRAM , that o nly retains data only if it is receiving power. wafer A slice from a silicon ingot no more than 0.1 inch thick, used to create chips. weighted arithmetic mean An average of the execution time of a wo rkload with weighting factors designed to reflect the presence of the programs in a wo rkload; computed as the sum of the products of weighting factors and executio n times. wide a rea network A netwo rk extended over hundreds of kilom eters which can span a continent. wire In Verilog, specifies a combinatio nal sign aL word The natural unit of access in a com puter, usually a group of 32 bits; correspo nds to the size of a register in the M IPS architecture.

G -17

Glossary

workload A set of programs run on a com puter that is either the ac tual collection of applications run by a user o r is constructed fro m real program s to approximate such a m ix. A typical wo rkload specifies both the programs as well as the relative frequencies. write buffer A queue that holds data while the data are waiting to be written to memory. write-back A scheme that handles writes by updating values only to the block in the cache, then writing the m odified block to the lower level of the hierarchy when the block is replaced.

write- invalida te A type of snooping protocol in which the writing processor causes all copies in other caches to be invalidated befo re changing its local copy, which allows it to update the local data until a nother processor asks fo r it. write-through A schem e in which writes always update both the cache and the mem ory, ensuring that data is always consistent between the two. yield The percentage of good dies from the total number of dies o n the wafer.

Index

CD information is listed by chapter and section number followed by page ranges (CD9. 1: 1-2). In More Depth references are listed by chapter number followed by page ranges (lMD4:56). Page references preceded by a single letter refer to appendices.

A Absolute addresses, AI3 Abstractions, 21-22, 24 Accumulator architecnlres, CD2.19: 1-2 Accumulator instructions, IMD2:7 Acronyms, 9--10 ACS, CD6.13A Activation record, 86 Active matrix display, 18 Ada, 173 add,49--51,301 Adder, 292 add immediate, 58 add immediate unsigned, 172 Addition, 170-176 carry lookahead, B38--47 floating point, 197-20 I Address (addressing) absolute, AI3 base, 55 calculation, 385, 390, 392, 402 exception, 342-343 in large-scale parallel processors, CD9.4:23- 25 memory,54 PC-relative, 98 physical, 511,512,513-514

translation, 512, 521-524 virnlal,512 Addressing, MIPS branches and jumps, 97- 99, 294-295 decoding machine language, 100--104 mode summary, 100 32-bit immediate operands, 95--96 Addressing modes IA-32,138 MIPS, 100 RISC, D5-9 add unsigned, 172 Advanced Research Project Agency (ARPA ), CD7.9;9, CD8.3:5, CD8.11;7 Advance load, 442 Agarwala, Tilak, CD6.13:4 Aho, AI, CD2.19:8 Aiken, Howard, CD 1.7:3 Air bags, 281 Algol, CD2.19;6--7 Aliasing, 528 Alignment restriction, 56 Allan, Fran, CD2.19:8 Allocate-on-miss, 484 Alpha architecnlre, CD5.12:3, D27-28 Alto, 16, CDI.7:7-8, CD7.9: 10, CD8.11:7 ALU. See Arithmetic logic unit ALUOp, 301-305 ALUOut, 319, 320, 327 AMD,136 Amdahl, Gene, CD5.12: I Amdahl's law, 179,267,494, CD9.2;9, CD9.9:40, IMD4:5-6 AMD Opteron, memory hierarchy, 546--550

and (AND ), 70, 301, 321, B6 AND gate, CD3.1O:5 and immediate, 71 Andreessen, Marc, CD8.11;7 Antidependence, 439 Antifuse, B77 Antilock brakes, 281 Apple II, CDI.7:5 Application binary interface (ABl), 22 Applications software, II Archeological sites, 236--237 Architectural registers, 448 Architecture. See Instruction set ardlitecture Arithmetic addition, 170-176 division, 183-189 fallacies and pitfalls, 220-224 floating point, 189, 191-220 mean, 257-258 multiplication, 176--182 signed and unsigned numbers, 160-170 subtraction, 170-176 Arithmetic-logical instructions, 292- 293,298 multiple-cycle implementation, 327,329 single-cycle implementation, 300--318 Arithmetic logic unit (ALU ), 177, 179, 184,187,201 adders and, 292, 294 ALVOp, 301-305 ALVOut, 319, 320, 327 constructing, B26-38 control, 301-303, C4-8

1-2

Index

datapaths and, 286, 292, 294, 296 MIPS, B32-38 muiticycle implementation, 318- 340 I-bit, B26-29 single-cycle implementation, 300-318 32-bit, B29-36 ARM, D36--38 ARPANET, CD8.3:5, CD8.l1:7 Arrays of logic elements, B18-19 versus pointers, I 3()....1 34 Art, restoration of, 562-563 ASCII (American Standard Code for Information Interchange), 90--91 versus binary numbers, 162 Assembler directives, A5 Assemblers, 13, 107-108, A4, 10-17 Assembly language, 13, 107,1\3-10 See also MIPS assembly language disadvantages of, A9-IO when to use, A7-9 Asserted signal, 290, B4 Assert signal, 290 Associativity, in caches, 499-502 Asynchronous bus, 582-583 Asynchronous inputs, B75- 77 Atanasoff, John, CD 1.7:3 AT&T Bell Labs, CD7.9:8-9 Atomic swap operation, CD9.3: 18 Automatic storage class, 85 Availability, 573 Average Memory Access Time (AMAT), IMD7:1

B Bachman, Charles, CD8.11:4, 5 Backpatching, A13 Backplane, 582 Backus, John, CD2.l9:6, 7 Barrier synchronization, CD9.3: 15 Base address, 55, 100 Base register, 55 Base stations, CD8.3:9 Base 2 to represent numbers, 16()....161 Basic block, 75

Basket, Forrest, CD7.9:9 Behavioral specification, B21 Bell Labs, CD7.9:8-9 Benchmarks, 254--255 EEMBC, 255, IMD4: 17- 18 kernel, CD4.7:2, IMD4:7--8 SPEC CPU, 254--255, 259-266, CD4.7:2- 3, IMD4:7-8 S PE~eb99,262-266

synthetic, CD4.7:1-2, IMD4: 11-12 Berkeley Computer Corp. ( BCC), CD7.9:8,9 Berkeley Software Distribution (BSD), CD7.9:9 Berners-Lee, Tim, CD8.11:7 Biased notation, 170, 194 Bigelow, Julian, CDI.?:3 Big Endian, 56, A43 Big-interleaved parity (RAID 3), 576-577 BINAC,CDI.7:4 Binary digits (numbers), 12,60 adding and subtracting, 170-176 ASCII versus, 162 converting to decimal floating point, 196 converting to decimals, 164 hexadecimal-binary conversion table, 62 scientific notation, 191 use of, 16()....161 Binary point, 191 Bit(s), 12,60 in a cache, 479 dirty,521 fields, IMD2: 13-14 least significant, 161 map, 18 most significant, 161 reference/use, 519 sign, 163 sticky, 215 Bit error rate (BER), CD8.3:9 Blaauw, Gerrit, CD6.13:2 Block, Barbara, 156-157 Blocking assignment, B24

Block-interleaved parity (RAID 4), 577-578 Blocks defined,470 finding, 540-541 locating in Glches, 502-504 placement of, 538-540 reducing cache misses with, 496-502 replacing, 504, 541 -542 Bonding, 30 Boolean algebra, B6 Booth's algorithm, IMD3:5---9 Bounds check shortcut, 168 Branch (es) addressing in, 97- 99, 294--295 delayed, 297,382, 418-419,A41 delay slot, 423 history table, 421 loop, 421-422 multiple-cycle implementation, 327-328,336 not taken, 295, 418 prediction, 382, 421-423 prediction buffer, 421 taken, 295 target address, 294---296 target buffer, 423 Branch equal (beq), 294, 297,300-318 Branch/control hazards, 379-382, 416--424 delayed, 297,382,418-419 dynamic branch prediction, 421-423 not taken, 295, 418 untaken, 381 Verilog and, CD6.7:S--9 Brooks, Fred, Jr., CD6.13:2 Bubble Sort, 129 Burks, Arthur \Y., 48, CDI.7:3, CD3.10: I Buses, 291-292 advantages/disadvantages of, 581 asynchronous, 582-583 backplane, 582 basics of, 581-585 defined,581,BI8-19 master, 594 Pentium 4, 585-587

'·3

Index

processor-memory or I/O, 582 shared,322-324

synchronous, 582-583 transaction, 582

Bypassing, 376-377 Byte addressing, 56 Byte order, A4J

c c bit fields, IMD2:13- 14

converting floating points to M IPS 3£Sembly code, 209-213 development of, CD2.19:7 logical operations, 68-71 overflows, 172

procedures, 81-88 sort example, 121-129 strings, 92-93 translating hierarchy, 106-111

while loop in, 74-75 Cache coherency

multiprocessor, CD9.3: 12- 20 protocols, CD9.3:13, 16--18 snooping, CD9.3:13 synchronization using, CD9.J: 18-20 Cache-coherent nonuniform memory access (CC-NUMA), CD9.4:22 Caches accessing, 47~82 associativity, 499--502 basics of, 473-491 bits in, 479 blocks, locating in, 502-504 blocks used to reduce misses,

496-502 defined,473 direct-mapped, 474-475, 497 example of simple, 474-476 fully associative, 497 lntrinsity FastMATH processor example, 485-487 mapping address to multiword block,480

memory, 20 memory system design to support, 487-491 misses, handling, 482--483, 496--502 multilevel,492,505-510 nonblocking, 445, 548 performance, measuring and improving, 492-51 I performance with increased clock rate, 495-496 reducing miss penalty using multilevel,505-509 set associative, 497, 504 split, 487 tags, 475, 504 three Cs model, 543-545 valid bit, 476 writes, handling, 483--485 Callee, 80, A23 Caller, 80, A23 Cal TSS, CD7.9:8 Capacity misses, 543 Carnegie Mellon University, CD6.13:5 Carrier signal, CD8.3:8 Carrylookahead, B38-47 Carry save adders, 181, lMD3:17-18 Case statement, 76 Cathode ray tubes (CRTs), 18 Cause register, 342 CauseWrite, 342 Central processor unit (C PU ), 20 execution time, 244-245 performance, 245, 246--253 time, 244-245 Cerf, Vint,CD8.11:7 Chamberlin, Donald, CD8.11:5 Characters, Java, 93-95 Chavin de Huantar, 236-237 Chips, 20, 30 Clearing words in memory arrays and, 130-132 comparing both methods, 133-134 pointers and, 132-133 Clock cycles, 245, 847 finite state machines, 332

multicycle implementation, 31S--340 single-cycle implementation, 300-318 Clock cycles, breaking execution into arithmetic-logical instruction, 327,329 branches, 327-328 decode instruction and register fetch, 326--327 fetch instruction, 325-326 jump, 328 memory read, 329 memory reference, 327, 328 Clock cycles per instruction (CPI), 248-251 in multicycle CPU, 330-331 Clocking methodology, 290-292, 847 edge-triggered, 290-291, B47 level-sensitive, B74-75 timing methodologies, B72-77 Clock period, 245, B47 Clock rate, 245 Clocks, B47-49 Clock skew, B73-74 CLU, CD2.19:7 Clusters, CD9.1:4, CD9.5:25-26 CMOS (Complementary Metal Oxide Semiconductor), 31, 264 Coarse-grained multithreading, CD9.7:31-33 Cobol, CD2.19:6, CD8.11:4 Cocke, John, CD2.19:8, CD6.13:2, 4 Codd, Ted, CD8.11:4, 5 Code generation, CD2.12:9 Code motion, 119 Code size, fallacy of using, IMD4:18- 19 Coherence problem, 595 Cold-start misses, 543 Collision misses, 543 Color, 292 Colossus, CD 1.7:3 Combinational control units, C4---8 Combinational elements, 289 Combinational logic, B5, 8-20, 23-25 Compact disks (C Ds ), 25

1-4

Index

Compaq Computers, CD8.11:6 Comments, 50 Commit unit, 443 Common subexpression elimination, 117 Compilers C,107 functions of, 11-12 historical development of, CD2.19:7-8 how they work, CD2.12:1 -9 Java, 114-115 optimization, 116-121 structure of, 116 translating high-level language into instructions that hardware can execute,12-15,A5--6 Compulsory misses, 543 Computers applications, 5- 7 components of, 15-16 historical development of, CD1.7:1- 1O organization of, 16 what it looks like inside, 18-22 Computer technology, advances in, 4 Conditional branches, 72-73 Condition codes, 140 Conflict misses, 543 Constant folding, 118 Constant propagation, 118 Constants, 57, 58 loading 32-bit, 96 Constellations, CD9.5:26 Context switch, 530 Control,20 hardwired, 348, CD5.12:2 pipelined,399-402 Control Data Corp. (C DC), CDI.7:5, CD6.13:2 Control hazards. See Branch hazards Controller time, 570 Control signals list of, 306, 324 write, 290, 294

Colltrol unit adding, 299 combinational, C4--8 designing main, 303-312 exceptions, 340- 346 fallacies and pitfalls, 350--352 finite state machines, 330, 331-340, C8-20 interrupts, 340--341 jumps,313-314 microprogramming, 330, CD5.7:4-10 multicycle implementation, 318-340 single-cycle implementation, 300-318 Coonen, Jerome T., CD3.10:7 Copy back, 521 Copy propagation, 118 Corbato, John, CD7.9:7, II Correlating predictors, 423 Cosmic Cube, CD9.11 :52 C++,CD2.19:7 CPU. See Central processor unit Cray, Seymour, CD1.7:5, CD3.1O:4, CD6.13:2 Cray Research, Inc., CDI.7:5, CD3.1O:4-5, CD6.13:5 Critical word first, 482 Crossbar network, CD9.6:30 CTSS (Compatible Time-Sharing System ), CD7.9:7- 11 Culler, David, 157 Culler Scientific, CD6.13:4 Cutler, David, CD7.9:9 Cydrome Co., CD6.13:4, 5 Cylinder, use of term, 569

D Dahl, Ole-Johan, CD2.19:7 Databases, history of, CD8.11:4-5 Data General, CD8.11:6 Data hazards defined,376--379 forwarding, 402--412

load-use, 377 stalls, 413-416 Data parallelism, CD9.11:48 Datapath,20 building a, 292-300 elements, 292 fallacies and pitfalls, 350--352 jumps, 313-314 logic design conventions, 289-292 multicycle implementation, 318-340 operation of, 306-312 pipelined,384-399 single-cycle implementation, 300-318 Data rate, 598 Data segment, A 13, 20 Data selector, 286 Data transfer instructions, 54-55 Data types, Verilog, B21-22 Dawson, Todd, 157 Dead code elimination, 118 Dead store elimination, 118 Deasserted signal, 290, B4 Deassert signal, 290 Debugging information, A 13 DEC (Digital Equipment Corp.), CDI.7:5, CD4. 7:2, CD7.9:9, CD8.11:11 Decimal numbers, 60,161 converting binary numbers to, 164 converting binary to decimal floating point, 196 dividing, 183 multiplying, 176--177 scientific notation, 189 Decision-making instructions, 72-74 Decoders, B8-9 Decoding, 333 Dedicated register, CD2.19:2 Defects, 30 Delayed branch, 297, 382, 418-419, A41 Dell Computer Corp. SPEC CPU benchmarks, 254-255, 259-266 SPECweb99 benchmark, 262-266

'·5

Index

DeMorgan's laws, 86 DeMorgan's theorems, 811 Dependability, disk, 569-580 Dependence detection, 406 Dependent instructions, 403 Desktop computers,S performance benchmarks, 255 Destination register, 64 Deutsch, Peter, CD7.9:8 D flip-flop, 851-53 Dhrystone synthetic benchmark, CD4.7:2, IMD4:11- 12 Dies, 30 Digital cameras, 236- 237,603--606 Digital signal-processing extensions, D19 DIMMs. See Dual inline memory modules

Directives, data layout, A 14---15 Direct-mapped cache, 474--475, 497 Direct memory access (DMA), 594-596 Directories, CD9.4:24 Dirty bit, 521 Disabled people, technology for, 366- 367 Disk(s) arrays, IMD8:2 controller, 570, 571 drives, 19, 20 fallacies and pitfalls, 606-609 read time, 570-571 storage and dependability, 569-580, CD8.11:1-4 Dispatch, 350 Displacement addressing, 100 Distributed block-interleaved parity (RAID 5), 578 Distributed memory, CD9.4:22 Distributed shared memory (DSM), CD9.4:24 divide, 188-189 Dividend,183 divide unsigned, 188--189 Division, 183-189 Divisor, 183 Don't-care terms, 303, B16-18 Double, 192 Double extended precision, 218

Double precision, 192 Double data rate synchronous DRAMs (DDD SDRAMs), 490-491 DRAM. See Dynamic random access memory Dual inline memory modules (DIMMs),22 DVD drive, 19,20 DVDs (digital video disks ), 25 Dynamically linked libraries (DLLs), 112-114 Dynamic branch prediction, 382, 421-423 Dynamic data, A22 Dynamic multiple issue, 433, 442-445 Dynamic pipeline scheduling, 443--445 Dynamic random access memory (DRAM), 20, 469, 487-488, 49D-491,513,B60,63-65 historical development of, CD7.9:3-4

E Early restart, 481-482 Eckert, J. Presper, CD 1.7:1,2,4, CD7.9: I Eckert-Mauch1y Computer Corp., CDI.7:4 Edge-triggered clocking methodology, 290-291, B47 EDSAC (Electronic Delay Storage Automatic Calculator), CDI.7:2, CD5.12:1 EDVAC ( Electronic Discrete Variable Automatic Computer ), CDI.7:1-2 EEMBC benchmarks, 255, IMD4: 17- 18 802.11 stand ard, CD8.3:9-10 Eispack, CD3.1O:3 Elaborations, 8 Elapsed time, 244 Ellison, Larry, CD8.11:5 Embedded computers, 6-8, CDI.7:8--9,A7 performance benchmarks, 255, IMD4:17- 18 EMC,CD8.11:6 Emulation, CD5.12:1-2 Encoder, B9

Energy efficiency problems, 263- 265 Engelbart, Doug, 16 ENIAC (Electronic Numerical Integrator and Calculator), CDI.7: 1- 2, 3, CD7.9:1 Environmental problems, tedUlology and,156-157 EPc\Vrite, 342 Error-correcting codes, B65 Error-detecting codes, B65--67 Ethernet, 26, CD8.3:5, CD8.11 :7-8, IMD8:1 - 2 Evolution versus revolution, CD9.1O:45-47 Exception enable, 532 Exception program counter (EPC ), 173, 341- 342,429-431 Exceptions,173,A33-38 address, 342-343 control checking of, 343-346 defined,340-341 handling of, 341-343, A35, 36-38 imprecise, 432 pipeline, 427-432 Executable file, 109 Execution time, 242, 244---245 use of total, 257- 259 Executive process, 529 Exponent, 191 Extended accumulator, CD2.19:2 External labels, A II

F Failures mean time between failures (MTBF ),573 mean time to failure (MTTF), 573, 574 mean time to repair (MTTR), 573, 574 reasons for, 574 synchronizer, B76 Fallacies, 33 False sharing, CD9.3: 14 Fanout, 32 Fast carry, B38-47

1-6

Index

Fetch-on-missfwrite,484 Field programmable devices (FPDs), 877-78 Field programmable gate arrays ( FPGAs),877 Fields defined,61 M IPS, 63--64 File system benchmarks, 598-599 Fine-grained multithreading, CD9.7:31-33 Finite state machines, 330, 331-340, 847-72, C8-20 Firewire, 582, 583 Firmware, CD5.12:2 Fisher, Josh, CD6.13:4 Fishman, H arvey, 366-367 Flags, 140 FLAS H , 23, 25 Flat-panel display, 18 Flip-flops, 290, 850--53 Floating point, 189, 191-220 addition, 197-201 converting binary to decimal floating point, 196 defined, 191 historical development of, CD3.1O:1-9 IA-32,217-220 M IPS, 206- 213 multiplication, 202- 205 representation, 191-197 rounding, 214--215 Floating Point Systems, CD6.13:4, 5 Floating vectors, CD3.10:2 Floppy disks, 25, CDI.7:6 Floppy drives, 25 Flush instructions, 418 Flynn, Michael, CD6.13:3 Formal parameter,Al6 Forrester, J., CD7.9: I FORTRAN, CD2.19:6, 7-8 overflows, 172, 173 Forwarding, 376--377,402--412, CD6.7:3 Forward reference, All

Fraction, 191, 193 Frame buffer, 18 Frame pointer, 86 Front end of compiles, CD2.12:1-9 Fully associative cache, 497 Fully connected network, CD9.6:28 Function code, 63

G Gates, 87-8, C4--8 Gateways, CD8.3:6 General-purpose register (GPR), 135, 138,CD2.19:2-3 Generate, carry lookahead, 839--47 Geometric mean, IMD4:9-11 Gibson, Garth, CD8.11:6 Global common subexpression elimination,118 Global labels, All Global miss rate, 509 Global optimization, 117-121, CD2.12:4-6 Global pointer, 85 Goldstine, Herman H., 48, CD 1.7:1 - 2, 3, CD3.1O:1 Google, CD9.8:34---39 News, 465 Gosling, James, CD2.19:7 Graph coloring, CD2.12:7-8 Graphics display, 18 Gray, Jim, CD8.11:5 Gray-scale display, 18 Guard,214-215

functions of, IS performance affected by, 10 synthesis tools, 821 H ardwired control, 348, CD5.12:2 H arvard architecture, CD 1.7:3 H azards See a/50 Pipelining hazards detection unit,413--415 Heap,allocating space for data on, 87-88 Heat sink, 22 Held, Gerald, CD8.11:5 Hewlett-Packard, CD2.19:5, CD3.1O:6-7, CD4.7:2 PA- RI SC 2.0, D34-36 H exadecimal-binary conversion table, 62 H i,181 H igh-level optimization, 116-117 H igh-level programming languages advantages of, 14 -IS architectures, CD2.19:4 defined,13 translating into instructions that hardware can exeulte, 12- 15 H it(s) Average Memory Access Time (AMAT ), IMD7: I defined,470 rate/ratio, 470--471 time, 471 H itachi, SuperH , D39-40 H old time, 8 53 H ot swapping, 579 H ubs, CD8.3:7

I H IBM H alf words, 94 H amming code, 867 H andler, 533 H andshaking protocol, 583-584 H ard disk, magnetic, 23 Hard drive, 19,20 H ardware description language, 820-25

disk storage, CD8.11: 1--4 early computers, CDI.7:5 floating points, CD3.1O:2, 3--4 floppy disks, CDI.7:6, CD8.11:2 history of programming languages, CD2.19:6 microprogramming, CD5.12: 1-2 multiple issue, CD6.13:4

Index

PowrPC, 032- 33, IMD 2: I 7-20, IMD3:1O RAID, CD8.l1:6 Stretch computer, CD6.13: 1- 2 virtual memory, CD7.9:5-7, 10 Winchester disks, CD8.11 :2, 4 IEEE 754 f1oation-point standard, 193---196, CD3.10:7-9 If-then-else statements, compiling into conditional branches, 72-73 Immediate addressing, 100 Implementation, 22, 24 Imprecise interrupts/exceptions, 432, CD6.13:3 IMS, CD8.11:4 Induction variable elimination, 119-120 Infinity, 193 Ingres, CD8.11:5 In-order commit, 445 In-order completion, 445 Input devices, 15,566, AJ8--40 Input don't cares, B16 Input operation, 582 Inputs, asynchronous, B75-77 Instruction decode, 385, 390, 392, 402 Instruction encoding, MIPS f1oatingpoint, 208 Instruction group, 440 Instruction fetch, 385, 388-389, 392, 400 Instruction format, 61 Instruction latency, 452 Instruction-level parallelism (I LP ), CD9.7:33,433, CD6.13:5 Instruction mix, 253 Instruction register (IR), 319, 321 Instruction sets addressing, 95-1 05 architecture, 22, 24 compiler optimization, 116-121 decision-making instructions, 72-74 defined,48 designing, for pipelining, 374-375 historical development of, CD2.19:1-9 logical operations, 68-71

operands of hardware, 52--60 operations of hardware, 49-52 to process text, 90-95 representing instructions to computer, 60-68 styles, IMD2:7- 9 supporting procedures, 79-90 translating and starting a program, 106--115 Integers, signed versus unsigned, 165 Integrated cirruits (ICs) costs, IMDI:I- 2 defined, 20, 27-28 how they are manufactured, 28--33 Integrated Data Store (IDS ), CD8.11:4 Intel, CD 1.7:5, 6, CD8.11:8 See a/50 Pentium 4 8086,135,CD2.19:2,4,5 8087,135, CD3.1O:7 80286,135, CD2.19:5 80386, CD2.19:5 80486,135, CD2.19:5 iSC 860 and Paragon, CD9.11 :52 Pentium and Pentium Pro, 135, CD2.19:5,448-450 SPEC CPU benchmarks, 254--255, 259-266 SPECweb99 benchmark, 262-266 Intel IA-32, 59 addressing modes, 138 complexity of, 347-348 conclusions, 142- 143 fallacies and pitfalls, 143-144 floating point, 217-220 historical development of, 134-137, CD2.19:4-5 instruction encoding, 140-142 integer operations, 138-140 registers, 137- 138 Intel IA-64, 435, CD5.12:3 architecture, 440-442, CD6.13:4-5 Intel Streaming SIMD Extensions (SSE), 135-136 Intel Streaming SIMD Extension 2 (SSE2),136

floating points, 220 Interface message processor (IMP ), CD8.3:5 Interference graph, CD2.12:7 Interleaving, 489 Intermediate representation, CD2.12:2-3 Internet, CD8.11:7 news services, 464-465 Internetworking, CD8.3:1-4 Interrupt-driven I/O, 590-591 Interrupts,173,A33-38 handler, A33 imprecise, 432, CD6.13:3 priority levels, 591 - 593 use of term, 340-341 Intrinsity FastMATH processor example, 485-487,524 Invalid operations, 193

I/O buses, 582 communicating with processor, 590-591 designing a system, 60Q.-603 devices, 15,566, A38-40 digital camera example, 603--606 diversity of, 568 fallacies and pitfalls, 606--609 giving commands to devices, 589-590 historical development of, CD8.11:1-9 instructions, 590 interfacing devices to processor, memory, and operating system, 588- 596 interrupt-driven, 590-591 interrupt priority levels, 591-593 measuring performance, 567 memory-mapped,589-590 performance, 597--600 rate, 598 requests, 568 transferring data between devices and memory, 593-595 Issue packet,435 Issue slots, 434

1-8

Index

J Java bytecode, 114, CD2.14: I, 2 characters and strings, 93-95 compiling, CD2.14:4-6 development of, CD2.19:7 interpreting, CD2.14:1 - 3 invoking methods, CD2.14:6 logical operations, 68-71 translating hierarchy, 114-115 sort and swap, CD2.14:6--13 while loop, CD2.14;3-4, 5--6 Java Virtual Machine (JVM ), 115, CD2.14:3 Jhai Foundation, PC network, 44-45 Jobs, Steven, CDI.7;5 Johnson, Reynold B., CD8.11;1 Joy, Bill, CD7.9:9 J-type,97 jump, 73, 77, 80, 89, 296 addressing in, 97-99 datapath and control and, 3 I 3-3 14, 321,328,336 Jump address table, 76, 77, IMD2:15-16 jump-and-link, 79--80, 89 jump register, 76 Just-in- Time (lIT ) compiler, 115

K Kahan, William, CD3.1O:5-7, 8, 9 Kahn, Robert, CD8.11:7 Karnaugh maps, BI8 Katz, Randy, CD8.11:6 Kay, Alan, CD2.19:7 Kernel benchmarks, CD4.7:2, IMD4:7-8 Kernel process, 529 Knuth, Donald, CD2.19:8

L Labels, externallglobal and local, All Lampson, Butler, CD7.9:8, II

Laptop computers, performance versus power versus energy efficiency, 263-265 Latches, B59-53 Latency instruction, 452 pipeline, 383 Leaf procedures, 83, 93 Least recently used (LRU ), 504, 518,519 Least significant bit, 161 Level-sensitive docking, B74-75 Link editor, 109 Linkers, 108-1 11,A4, 18-19 Linpack, CD3.1O;3, CD4.7:2 Linux, 11, CD7.9;11 Liquid crystal displays (LeDs ), 18 Lisp,CD2.19:6 Little Endian, 56, A43 Live range, CD2.12;7 Livermore Loops, CD4.7:2 LO,181 Load, 54, 57 advanced,442 byte,91,164 byte unsigned, 164 half,94,164 halfword unsigned, 164 linked, CD9.3: 19 locked, CD9.3:19 upper immediate, 95 word, 54, 57, 59, 294, 300-3 18 Loader, 112 Loading,AI9-20 Loading 32-bit constant, 96 Load-use data hazard, 377 Local area networks (LANs ), 26, CD8.3:5-8, CD8.11 :7-8 Locality, principle of, 468-469 Local labels, All Local miss rate, 509 Local optimization, 117-121, CD2.12;3-4 Lock, CD9.1:5 Lock variables, CD9.3; 18

Logic arrays of logic elements, 818-19 combinational, 85, 8-20, 23-25 equations, 86--7, C 12-1J sequential, 85, 55-57 two-level,810-14 Logical operations, 68-71, 86, IMD2;21-22 Logic design conventions, 289-292 Long-haul networks, CD8.3:5 Long instruction word (LIW), CD6.13:4 Lookup tables (LUTs ), 878 Loops, 74-75 branch, 421-422 unrolling, 117,438-440 Lorie, Raymond, CD8.11:5

M Machine code, 61 Machine language, 61, A3 decoding, 100--104 MIPS floating-point, 207 object file and, 108 MacOS, II Macros,A4,15-17 Magnetic disks, 23, 569 differences between main memory and,24 memory hierarchies and, 469, 513 Magnetic resonance imaging (MRI), 622--623 Magnetic tape, 25 Main memory, 23 differences between magnetic disks and,24 Make the common case fast, 267, 285 Mark machines, CD 1.7:3 Mask,70 Mauchly, John, CD 1.7: 1,2,4 McCarthy, John, CD2.19:6, CD7.9:7, II McKeeman, William, CD2.19:8 Mealy, George, 338 Mealy machine, 338, 340, 868

'·9

Index

Mean time between failures (MTB F), 573 Mean time to failure (MTTF), 573, 574,606 Mean time to repair (MTTR), 573, 574 Megabyte, 23 Memories, 290 Memory,S access, 385, 390, 392, 402 allocation, 87-88 Average Memory Access Time (AMAT), IM07:!

board,20 cache, 20 cards, 25 consistency model, CD9.3: 15 defined, 20, 23 direct memory access (DMA),

594-596 distributed, CD9.4:22, 24 dynamic random access (DRAM), 20,

469,487-488,490-491,513,860, 63--65

historical development of, CD7.9: 1-12 main, 23 mapping, 512 nonvolatile, 23 operands, 54-55 primary, 23 random access (RAM ), 20 read only (ROM ), 814, 16, C13-19 secondary, 23 shared, CD9.! :4-5, CD9.4:22, 24 static random access (SRAM), 20,

469,857--60 transferring data between devices and,593-595

unit, 292 usage, A20-22 virtual,511-538 volatile, 23 Memory data register (MDR), 3 19, 328 Memory elements latches, flip-flops, and register files, 849-57 SRAMs and DRAMs, 857-67

Memory hierarchy caches, 473-51 I defined,469 fallacies and pitfalls, 550-552 framework for, 538-545 historical development of, CD7.9:S--7 levels, 470-471 methods for building, 469-470 overall operation of, 527-528 Pentium P4 and AMD Opteron, 546-550 trends for, 553-555 virtual,511-538 Memory-mapped I/O, 589-590 Memory-memory instructions, IMD2:8 Memory reference, 327, 328, 334-335 MESI cache coherency protocol, CD9.3;16,18 Message passing, CD9.1 :6, CD9.4:22- 23 Metastability, 875-76 MFWPS (million floating-point operations per second), IMD4: 15-17 Microarchitecture,448 Microcode, 348 Microinstructions, 348--349, CD5.7;1 fields, CD5.?;3, 5-9 format, CD5.7:2--4 Microoperations, 348 Microprocessors first, CDI.7;5 future of, CD9.10;44-45 Microprogramming controller, 348, CD5.12:2 creating a program, CD5.7;4-1 0 defined, 330, 346 fallacies and pitfalls, 350-352 historical development of, CD5.12:1-4 implementing the program, CD5.?;10-12 microinstruction format defined, CD5.7;2-4 simplifying design with, CD5.7:1-13 Microsoft Corp., CDI.7:5, CD7.9;1 0, CD8.11:5,6

Minicomputers, first, CD 1.7:5 Minterms, 812 M IPS, 49 addressing,9S--105 allocation of memory, 87 arithmetic logic unit (ALU), 832-38 compiling statements into, 50-51 decision-making instructions, 72- 73 exception code, 535 fields, 63- 64 floating point, 206- 213 implementation, 28S--289 instruction encoding table, 64, 103 instruction set, 49 logical operations, 68-71 machine language, summary of, 67, 78,90 mapping registers into numbers, 6Q.-68 operands, summary of, 59, 67, 71,89, 105, 169 registers, 52-53, 79-80, 85, 88, 532 RISC core subset, D9-16, 20-24 RISC instructions for M IPSI6, D41-43 RISC instructions for M IPS64, D2S--27 translating assembly into machine language, 65-66 M IPS assembly language add,49-51 add immediate, 58, 59 add immediate unsigned, 172 add unsigned, 172 AND, 69, 70 and immediate, 71, 89 conditional and unconditional branches, 72-73 divide, 188--189 divide unsigned, 188-189 floating point, 207 jump, 73, 80 jump address table, 76 jump-and-link,79-80 load word, 54-59, 294 move from hi, 181 move from 10, 181 multiply, 181

1-10

Index

multiply unsigned, 181 nor (NOR), 69, 70 or (OR), 69, 70 or immediate, 71,89 set on less than, 75 set on less than immediate, 77,165 set on less than immediate unsigned, 165 set on less than unsigned, 165 shifts, 69 store word, 54-59, 294 subtract, 49-51 subtract unsigned, 172 summary of, 51, 59, 67,71,77,89, 105,169,175,190,207,226-228 xor, IMD2:21-22 M IPS assembly language, R2000 addressing modes, A45-47 arithmetic and logical instructions, A51-57 assembler syntax, A47-49 branch instructions, A59-63 comparison instructions, A57-59 constant manipulating instructions,A57 data movement instructions, A70- 73 encoding instructions, A49 exception and interrupt instructions, A80-81 floating-point instructions, A73-80 instruction format, A49-5 I jump instructions, A63--64 load instructions, A66-68 store instructions, A68-70 trap instructions,A64-66 M IPS (million instructions per second) equation, 268 peak versus relative, IMD4: 13-14 problem with using as a performance measure, 268-270 Mirroring, 575 Miss, 470 Misses Average Memory Access Time (AMAT), IMD7: I

cache,482-483,496-502 capacity,543 cold-start, 543 collision, 543 compulsory, 543 conflict, 543 TBL,531 Miss penalty, 471 reducing, using multilevel caches, 505-509 Miss ratefratio, 471 global, 509 local,509 Mitsubishi, M32R, D4O-41 Moore, Edward, 338 Moore, Gordon, 28 Moore machine, 338, B68 Moore's law, 28, 181 Mosaic, CD8.11:7 Most significant bit, 161 Motherboard, 19,20 Motorola PowrPC, D32-33, IMD2:17-20, IMD3:1O 68881, CD3.10:8 Mouse, 16--17 move from hi, 181 move from 10,181 Move from system control, 173 M32R, D40-41 Multicomputers, CD9.11 :52 MULTICS (Multiplexed Information and Computing Service), CD7.9:8 Multicyc1e implementation, 318-340 Multiflow Co., CD6.13:4 Multilevel caching, 492, 505-510 Multimedia extensions of desktop/server RIses, DI6--19 Multiple instruction multiple data (MIMD ), CD9.11 :51- 53 Multiple instruction single data (M ISD), CD9.11:51 Multiple issue defined,433 dynamic, 433, 442--445

IBM's work on, CD6.13:4 static, 433, 435-442 Multiplexors, 286, 89-10 Multiplicand, 176 Multiplication, 176-182 floating point, 202- 205 Multiplier, 176 multiply, 181 multiply unsigned, 181 Multiprocessors connected by a network, CD9.4:2Q-.-25 connected by a single bus, CD9.3:11-20 defined, CD9.1 :4, CD9.11 :52 future of, CD9.1 0:43-44 history development of, CD9.11:47- 55 inside a chip and multithreading, CD9.7:3Q-.-34 networks, CD9.4:2Q-.-25, CD9.6:27-30 programming, CD9.2:8- 1O types of, CD9.1:4--8 Multistage network, CD9.6:29-30 Multithreading, CD9.7:30-34

N Name dependence, 439 NaN (Not a Number ), 193 NAND gate, B8 NCR,CD8.11:6 Negation shortcut, 166 Nested procedures, 83-85 Netscape, CD8.11:7 Network bandwidth defined, CD9.6:27 fully connected, CD9.6:28 total, CD9.6:27-28 Networks,25-27 characteristics of, CD8.3: I crossbar, CD9.6:30 internetworking, CD8.3: 1-4 local area, CD8.3:5-8

1·11

Index

long-haul, CD8.3:5 multiprocessors connected by, CD9.4:20-25, CD9.6:27-30 multistaged, CD9.6:29-30 Pentium 4, 585--587 wireless local area, CD8.3:8-1O Next-state function, 331, B67, C 12- 13, 21- 27 No-a1locate-on-write,484 No-fetch-on-write, 484 Nonblocking assignment, 824 Nonblocking caches, 445, 548 Nonuniform memory access (NUMA ) multiprocessors, CD9.1 :6, CD9.4:22 Nonvolatile memory, 23 Nonvolatile storage device, 569 Nop, 413-414 nor (NOR ), 70, 301, 88 Normalized number, 189 Northrop, CD 1.7:4 NOT, 70, 86 Numbers ASC II versus binary, 162 base to represent, 160-161 converting binary to decimal, 164 loads, 164 negative and positive, 165 shortcuts, 166-168 sign and magnitude, 162 sign bit, 163 signed and unsigned, 160-170 two's complement representation, 163 Nygaard, Kristen, CD2.19:7

o Oak, CD2.19:7 Object files defined,108,AIO format,AI3-14 linking, 109-111 Object-oriented language defined, 130, CD2.14: I Javas, CD2.14: 1-13 Offset, 55, 56

Opcode, 63, 303, 305, 306 Open Source Foundation, CD7.9:9 Open Systems Interconnect (OSI) CD8.3:2 Operands for computer hardware, 52--60 constant or immediate, 57 memory, 54-55 MIPS, summary of, 59, 67, 71,89, 105, 169 MIPS floating point, 207 Operating systems examples of, II functions of, 11- 12, 588-589 historical development of, CD7.9:7-11 Operations, for computer hardware, 49-52 Operators, Verilog, 821-22 Optical disks, 25 Optimizations high-level,116-117 local and global, 117- 120, CD2.12:3--6 summary of, 120-121 or(OR), 70, 30 I, 321,86 Oracle, CD8.11:5 or immediate, 71 Out-of-order execution, 445 Output devices, 15,A38-40 Output don't cares, 816 Output operation, 582 Overflow, CD3.1O:5 adding and subtracting and, 171-174 division and, 189 exceptions, detection of, 343 floating point and, 192 multiplying and, 181 Overlays, 51 1-512

p Packets, CD8.3:5 Page,512 placing and find, 515-516 Page faults, 512, 514, 516-521,531

Page offset, 513, 514 Page table, 515-516 Palmer, John F., CD3.1O:7 Parallel processing program, CD9.1 :4, CD9.2:8-1O, CD9.4:22- 23 addressing, CD9.4:23- 25 fallacies and pitfalls, CD9.9:39-42 PA-RISC 2.0, D34-36 Parity big-interleaved (RAID 3), 576-577 block-interleaved (RAID 4), 577- 578 distributed block-interleaved (RAID 5), 578 Parsing, CD2.12:1 Pascal, CD2.19:6-7 Patterson, David, CD8.11:6 PC-relative addressing, 98,100 PCSpim,A42, CDA:I - 3 PCSrc control and signal, 305 PCWrite, 321 PCWriteCond, 321 Peer-to-peer architecture, CD8.3:9-1O Pentium 4 buses and networks of, 585--587 implementation of, 348-350 manufacturing of, 28-33 memory hierarchies, 546-550 pipeline, 448-450 Pentium processors SPEC CPU benchmarks, 254-255, 259-266 SPECweb99 benchmark, 262-266 Performance See also Pipelining benchmarks, 254-255, IMD4:7-8, 11 - 18 of caches, 253 of caches, measuring and improving, 492-511 comparing, 252-253, 256-259, 425-426 CPU, 245, 246-253 defined,241- 244 equation, 248-249 evaluating, 254-259 factors affecting, 251

1-12

Index

fallacies and pitfalls, 266-270 historical review, CD4.7:1-4 how hardware and software affect, 10 1/0,597-600 measuring, 244-246 per unit cost of technologies, 27 of the pipeline, 253 relative, 243-244 reports, 255---256 single-cycle machines and, 315-318 SPEC CPU benchmarks, 254--255, 259-266, CD4.7:2-3, IMD4:7--8 SPECweb99 benchmark, 262- 266 system, 245 versus power and energy efficiency, 263-265 Personal computers, early, CDI.7:5--8 Peterman, Mark, 366--367 Physical addresses, 511, 512, 513-514 Physically addressed cache, 528 Physical page number, 513 Pipeline stalls, 377-379, 413-416, CD6.7:5---7 Pipelining advanced methods for extracting more performance, 432-445 control, 399-402 datapath,384--399 defined,370 designing instruction sets for, 374--375 exceptions, 427-432 fallacies and pitfalls, 451-384 forwarding, 376-377, 402-412, CD6.7:3 graphic representation, 395-399 historical development of, CD6.13:1-13 instruction execution sped up by, 372-374 latency, 383 overview of, 370 Pentium 4 example, 448 stalls, 377- 379, 413-416, CD6.7:5- 7 Verilog used to describe and model, CD6.7:1- 9

Pipelining hazards, branch/control, 379-382,416--424, CD6.7:8-9 Pipelining hazards data defined,376--379 forwarding, 402-412 load-use, 377 stalls, 413-416 strucnlral,375 Pitfalls, 33-34 Pixels, 18 Pointers, arrays versus, 130-134 Poison, 442 Polling, 590 Pop, 80 Pop-up satellite archival tags (PSATs), 156-157 Positive numbers, multiplying, 176--180 Power, 30-32 consumption, problems with, 263-265 PowerPC addressing, IMD 2:17-20 instructions, D32- 33 multiply-add instruction, IMD3: 10 Prediction, 382, 421-423 IA-64,441 Primary memory, 23 Procedure call conventions, A22-33 Procedure call frame, A23 Procedures allocating space for data on heap, 87--88 allocating space for data on stack, 86 C,81--88 defined,79 frame, 86 inlining, 116 leaf, 83, 93 nested,83-85 preserved versus not preserved, 85 recursive, A26, 29 steps, 70 Processor, 20 communicating with, 590--591 cores, 6--7 -memory buses, 582 Process switch, 530

Product of sums, 810-12 Product terms, 812 Program counter (PC), 80, 292 Programmable logic arrays (PLAs), 812- 14, C7, 19-20 Programmable logic devices ( PLDs), 877 Programmable read only memory (PROM),814,16 Programming languages, history of, CD2.19:6-7 Propagate, carry lookahead, 839-47 Propagation time, 877 Protection group, 576 Protocol families/suites, CD8.3: 1- 2 Protocol stack, CD8.3:3 Pseudodirect addressing, 100 Pseudoinstructions, 107, AI7 Push,80 Putzolu, Gianfranco, CD8.II:5

Q Quicksort, 129,507- 508 Quotient, 183

R Radio communication, CD8.3:8-9 Radix Sort, 507- 508 RAID (redundant arrays of inexpensive disks) big-interleaved parity (RAID 3), 576-577 block-interleaved parity (RAID 4), 577- 578 distributed block-interleaved parity (RAID 5), 578 error detecting and correcting code (RAID 2), 575 historical development of, CD8.II:5--6 mirroring (RAID 1),575 no redundancy (RAID 0), 575 P + Q redundancy (RAID 6), 578 summary of, 578-579 use of term, 574--574

1·13

Index

Random access memory (RAM ), 20 Raster cathode ray tubes (CRTs), 18 Raster refresh buffer, 18 Rau, Bob, CD6.13:4 Read only memory (ROM ), B14, 16, C13-19 Read/write head, 23 Reals, 189 Receive message routine, CD9.1:6 Recursive procedures, A26, 29 Reduced instruction set computer (RISC), CD2.19:4 addressing modes and instruction formats, D5-9 Alpha, D27-28 architecture, CD5.12:3 ARM,D36-38 desktop versus embedded, D3-5 digital signal-processing extensions, 019 M IPS, D9-16, 20-24 M IPSI6, D41-43 M IPS64, D25- 27 M32R, D4Q.....41 multimedia extensions, D 16-19 PA-RISC 2.0, D34-36 PowrPC, D32- 33 SPARCv.9, D29-32 SuperH , D39-40 Thumb, D38--39 Redundancy. See RAID (redundant arrays of inexpensive disks ) Reference bit, 519 Refresh rate, 18 reg, 821-22 Register addressing, 100 Register file, 293-294, 849, 53-55 read,385,390, 392,402 Registers, 52-53, 59, 88, 290, 532 allocation, CD2.12:7- 9 architectural,448 dedicated, CD2.19:2 destination, 64 general-purpose, 135, 138, CD2.19:2-3

global pointer, 85 IA-32,137- 138 jump, 76, 80 mapping into numbers, 60-68 number, 294 renaming, 439 special-purpose, CD2.19:2 spilling, 58, 80 Register use conventions, A22-33 Relational databases, CD8.11 :4-5 Reliability,573 Relocation, 513 Relocation information,Al3 Remainder, 183 Remington-Rand, CDI.7:4 Remote access times, CD9.1:7 Reorder buffer, 443 Reproducibility,25S--256 Requested word first, 482 Reservation stations, 443 Response time, 242, 244 Restartable instruction, 533 Restorations, 572 Return address, 80 Rings, CD7.9:7 Ring topology, CD9.6:27 Ripple carry, B39, 44-45 RISe. See Reduced instruction set computer Ritchie, Dennis, CD2.19:7, CD7.9:8, II Rotational delay, 570 Rotational latency, 570 Rounding, 214--215, CD3.10:2--4 Routers, CD8.3:6 R-type instructions, 292-293, 298

5 Sandisk Corp., 605 ScalUling, CD2.12: I Scientific notation, 189, 191 Secondary memory, 23 Sectors, 569 Seek,569 Seek time, 569-570 Segmentation, 514--515

Selector value, B9 Selinger, Patricia, CD8.11:5 Semantic analysis, CD2.12: I Semaphores, CD9.3:18 Semiconductor, 29 Send message routine, CD9.1:6 Sensitivity list, B24 Separate compilation, A 18 Sequential elements, 290 Sequential logic, B5, 55-57 Servers, 5 Set associative cache, 497, 504 set on less than, 75, 77,165,301 set on less than immediate, 77, 165 set on less than immediate unsigned, 165,169 set on less than unsigned, 165, 169 Set-up time, 853 Shadowing, 575 Shared memory, CD9.4:22, 24 Shared-memory processors, CD9.1:4-5 Shared virtual memory, CD9.4:24 Shift amount, 69 Shifts, 69 Sign and magnitude, 162, 191 Sign bit, 163 Signed division, 187-188 Signed multiplication, 180 Signed numbers, 160--170 Sign extension, 164, 167- 168,294, 296 Significand, 193 Silicon, 29 Silicon crystal ingot, 29 Silicon Graphics. See M IPS Simple programmable logic devices (SPLDs),877 Simplicity, 285 Simputer,45 Simula-67, CD2.19:7 Simultaneous multithreading (SMT), CD9.7:31-34 Single address space multiprocessors, CD9.1:4-6 Single bus, multiprocessors connected by a, CD9.3: 11-20

1-14

Index

Single-cycle implementation scheme, 300-318 pipelined performance versus, 372-374 Single instruction multiple data (SIMD ), CD9.11:47-49,51 Single instruction single data (SISD), IMD 2.12, CD9.11:47 Single precision, 192 Small computer systems interface (SCSI),573 Smalltalk, CD2.19:7 Smith, Jim, CD6.13:2 Snooping cache coherency, CD9.3: 13 Software applications, 11-12 performance affected by, 10 systems, II third-party of shrink-wrap, 5 sort body for for loop, 126-127 code for the body of, 124-126 full procedure, 127-128 Java, CD2.14:6-14 passing parameters, 127 preserving registers, 127 register allocation, 123 Source language, A6 SPARCv.9, D29-32 Spatial locality, 468-469 SPEC (System Performance Evaluation Corp.) CPU benchmarks, 254-255, 259-266, CD4.7:2-3, IMD4:7--S file server benchmarks, 599 Web server benchmarks, 599 SPEC ratio, 259 Speculation, 434-435 SPECweb99 benchmark, 262-266 Speedup, IMD4:5 Spilling registers, 58, 80 Spilt caches, 487 SPIM,A40-45, CDA:I-2 command-line options,A42, CDA: 1-3 Spin waiting, CD9.3:19, 20

Split transaction protocol, 585 SRAM. See Static random access memory SRT division, 188 Stack,80 allocating space for data on, 86 instructions, CD2.19:3-4, IMD2:8-9 pointer, 80 segment,A22 Stale data problem, 595 Stallman, Richard, CD2.19:8 Standby spares, 579 Stanford DASH multiprocessor, CD9.11:52 State elements, 289-290, B47-48 Static data segment, 87, A20- 22 Static multiple issue, 433, 43~2, CD6.13:4 Static random access memory (SRAM ), 20,469, B57--60 Static storage class, 85 Stewart, Robert G., CD3.1O:7 Sticky bit, 215 Stone, Harold S., CD3.1O:7 Stonebraker, Mike, CD8.11:5 Stop, 440 Storage for digital cameras, 603--606 disk, 569-580, CD8.11: 1-4 Storage classes, types of, 85 store, 57 Store buffer, 445, 485 store byte, 91 store conditional, CD9.3:19-20 Stored-program concept, 49, 215 store half, 94 store word, 57-59, 294, 300--318 Strength reduction, 118 Stretch computer, CD6.13: 1-2 Strings C,92-93 Java, 93-95 Striping, 575 Stroustrup, Bjarne, CD2.19:7 Structural hazards, 375

Structural specification, B21 Structured Query Language (SQL), CD8.11:4-5 Subroutines, CD5.7:2 subtract,49-51,301 Subtraction, 170-176 subtract unsigned, 172 Sum of products, B10--12 Sun Microsystems, CD4.7:2, CD7.9:9 SPARCv.9, D29-32 Supercomputers defined,5 first CD1.7:5 SuperH , D39-40 Superscalar processors, 348, 442--445, CD6.IJ:4 Supervisor process, 529 swap code for the body of, 122-123 full procedure, 123 Java, CD2.14:6-14 register allocation, 122 space, 517 Switched networks, CD8.3:5 Switches, CD8.3:7 Switch statement, 76 Sybase, CD8.11:5 Symbol table, 108,AI2, 13 Symmetric multiprocessors (SMPs), CD9.1:6 Synchronization barrier, CD9.3: 15 coherency and, CD9.3:18-20 defined, CD9.1:5 failure, B76 Synchronizers, B75-77 Synchronous bus, 582-583 Synchronous system, B48 Synthetic benchmarks, CD4.7: 1-2, IMD4:11-12 System call, 529, A43-45 System CPU time, 245 System performance, 245 System R, CD8.11 :4, 5 Systems software, II

1·15

Index

T Tags, cache, 475, 504 Tail recursion, IMD2: 10--11 Target language, A6 Taylor, George S., CD3.1O:8- 9 Taylor, Robert, CD7.9:9--1 0 TCP/IP, CD8.3:4, CD8.11:7 Temporal locality, 468 Terabytes,S Text segment, 87, Al 3, 20 Thacker, Chuck, CD7.9:8 Thinking Machines, CD9.11 :52 Thompson, Ken, CD7.9:8, II Thornton, J. E., CD6.1 3:2 Thrashing, 537 Thread-level parallelism (TLP ), CD9.7:33 Three Cs model, 543-545 Throughput, 242 Thumb, D38-39 Time, definitions of, 244 Time-sharing systems, CD7.9:7- 11 Timing methodologies, B72-77 Tomasulo, Robert, CD6.13:2, 3 Tomasulo's algorithm, CD6.1 3:2 Torvald, Linus, CD7.9: 10 Tournament branch predictors, 423 Trace cache, 349 Tracks,569 Traiger, Irving, CD8.11:5 Trains, computer controlled, 280-281 Transaction processing (TP ), 598 Transaction Processing Coundl (TPC), 598 Transfer time, 570 Transistors, 27, 29 Translating microprogram to hardware, C27- 31 Translation hierarchy for C, 106 assembler, 107-108 compiler, 107 linker, 108-111 loader, 112

Translation hierarchy for Java, 114 compiler, 114--115 Java Virtual Machine, liS Just in Time compiler, liS Translation-lookaside buffer (TLB), 522-534, CD7.9:5 Transportation, technology and, 280--281 Truth tables, 302- 303, B5, C5, 14, IS, 16 Tucker, Stewart, CD5.12: 1-2 Turing, Alan, CDI.7:3 TVM (transmission voie-machine), 280--281 Two-level logic, B10--14 Two's complement representation, 163

TYP" checking, CD2.12:1 examples of, 85

u Ullman, Jeff, CD2.19:8 Unconditional branches, 73 Undefined instruction, exception detection of, 343 Underflow, 192, CD3.10:5 Unicode, 93-94 Uniform memory access (UMA ) multiprocessors, CD9.1:6, CD9.4:22 Units in the last place (ulp ), 215 UN IVAC I ( Universal Automatic Computer), CD 1.7:4 UN IX development of, CD2.19:7, CD7.9:8-11 loader, I 12 object file for, 108 Unmapped,536 Unresolved references,A4 Unsigned numbers, 160--170 Untaken branch hazards, 381 USB, 582, 583 Use bit, 519 User CPU time, 245

v Valid bit, cache, 476 VAX, CD5.12:2-3, CD7.9:9 Vectored interrupts, 342 Vector processing, CD9.11 :49-51 Verilog, CD5.8: 1- 7 combinational logic in, B23-25 data types and operators, B21- 22 description of, B20--25 MIPS arithmetic logic unit (ALU ), B36-38 program structure, B23 sequential logic, B55--57 used to describe and model a pipeline, CD6.7: 1-9 Very large scale integrated (VIS Is) circuits, 20, 27- 28, 29 Very long instruction word (VLIW ), CD6.13:4 VHDL,B20,21 Virtual address, 512 Virtually addressed cache, 527 Virtual machine, simulation of, A41--42 Virtual memory address translation, 512, 521- 524 defined,511 design, 514--521 implementing protection with, 528-530 overlays, 51 1- 512 page, 512 page, placing and find, 515--516 page faults, 512, 514, 516-521 page offset, 5IJ, 514 page table, 515--516 reasons for, 511-512 translation-Iookaside buffer (T LB), 522-534 write-backs, 521 Virtual page number, 513 Volatile memory, 23 von Neumann, John, 48, CD 1.7:1 - 2, 3,

1-16

Index

CD3.1O:1 - 2,3 Vyssotsky, Victor, CD2.19:8

w Wafers, 29-30 Wall-clock time, 244 WARP, CD6.13:5 Web server benchmarks, 599 Weighted arithmetic mean, 258 \Vhetstone synthetic benchmarks, CD4.7: 1-2, IMD4: 11-12 while loop, 74-75, 98-99 in Java, CD2.14:3-4, 5--6 \Vhirlwind project, CDI.7:4, CD7.9: I \Vide area networks (WANs ), 26, CD8.11:11 \ViFi, 44-45, CD8.3:8 \Vilkes, Maurice, CD 1.7:2, CD5.12: I, CD7.9:6 \Vilkinson, James H ., CD3.IO:2

Windows, II wire, B21- 22 Wired Equivalent Privacy, CD8.3: 10 Wireless local area networks (WLANs ), CD8.3:8- IO Wireless technology, 27, 156-157 Wirth, Niklaus, CD2.19:6 Wong, Gene, CD8.11:5 Word, in M IPS architecture, 52 Working set, 537 Workload,254 World Wide Web, CD8.11:7 Wozniak, Stephen, CDI.7:5 Write-around,484 Write-back, 385, 392, 402, 484--485, 521,

542 Write buffer, 483-484 Write control signal, 290, 294 Write invalidate, CD9.3:14, 17 Writes handling cache, 483-485

handling virtual memory, 521 Write-through, 483, 542

x Xerox Palo Alto Research Center (PARC), 16, CD 1.7:7-8, CD7.9:9-IO, CD8.11:7, 8 xor, IMD2:21-22 xspim,A42, CDA:I-4

y Yield,30

z Zip drive, 19, 20, 25 Zone bit recording (ZBR), 569 Zuse, Konrad, CD 1.7:3

Computer Organization and Design, Third Edition: The Hardware Software Interface, Third Edition (The Morgan Kaufmann Series in Computer Architecture and Design) - PDF Free Download (2025)

References