8/10/2019 IBM Microsoft Segmented Memory

http://slidepdf.com/reader/full/ibm-microsoft-segmented-memory 1/5

byJeff Prosise

SEGME NTED MEMORYI've often heard that Inte l pr ocessors h av esegmente d m emory ar chitectu r es , and thatsegmen ts limit the am oun t of code anddata a p rogra m can h a ve . Can you e x-plain what seg me ntati on is and elabora teupon the pros an d c on s of a segmentedarch itect ure?

Donal d E. Whitel oc kTurne rsvnl e , Ne w Je r sey

~ In it simple st form, a segment in

.. - the Intel architect ure is a 64K block of memory . On the 8088, 8086 ,

and 80286 proces sors , memory i n quan-tities of mor e than 64K can 't be addre ssed as one flat , linear addre ss space; instea d ,memor y must be broke n down i nto smaller 64K segments and t hen ad dressed o ne

nent at a time.Why segments? Intel wanted to e n-

able thE 8086 to address a f ull megaby teof memor y. Since th e number of differe ntmem ory addre sses you can have i s 2",where n is the number of bits yo u'r e givento r epr esent memory, it w as necess ary to

build a machin e that was capa ble of 20-

Figure 1: An Inte l micr oprocessor running in

real mode f orm s a 20 -bit address by shifting

the content s of a 1 6-bit segment register 4

bits left (to f or m a p ara graph address ) and'-:ling a 1 6-b it offset . Here , the segment

a ress , 1234 h, is p a ir ed with the offsetaddr e ss 0005 h to f orm the a bsolut e addr ess

123 45h.

Tutor

• SEGMENTED MEMORY: Although memo ry in pesis laid out in one biglinear mass, it's accessedas if it were divided intosmall, discrete blocks.Here's the story of howand why.

bit addr ess ing (220 = 1MB).Intel also wanted to k ee p its intern al

regis ter s 16 bits wid e for com patibil itywith ear lier Intel micro pr ocesso r s. Thisimm ediately cr eated a se rious confli ct16-bit addr ess ing limit s you to 216

, or 64K of R AM. So that both goals could beachieve d , Intel opted to give the 80862 0addre ss lines ( per mitting up to 220 bytesof da ta to be addresse d ) but l eave t her egister s 16 bits wide, a nd set four of them asid e for use as segmen t registers -r egist er s whose sole purp ose

is to hold segment add resses. Using thisscheme, the CPU could form a 20-b itaddre ss by combinin g the contents of asegment register , which hold s a paragr a phaddr ess , with an offse t r egister , whichhold s a byte add ress . (A parag rap h is 16 bytes of memo ry; thus, you multipl y by16 to tr anslate a paragraph addre ss to a

byte ad d r ess). In other word s, the seg-ment r egister contains the addre ss wher ethe segment be gins, and the addr ess inthe off set r egister is interpret ed as an of fsetr elative to the base o f the segment.

Figur e 1 shows how two 16-bit values

ar e combin ed to form the addre ss of auniqu e location in memor y. The segmentvalue is multipli ed by 16 to produc e thenumb er 12340h (in hexadecimal notation ,all it takes to multipl y a number by 16 i sto add a zer o at the end , just like multi- plyin g by 10 in our d ecimal numberingsystem ). The off set value 0005h is thenad d ed to the r esult , yieldin g the a bsolut eaddres s 12345 h.

By convention , mem or y addr esses arerare ly referred to in abso lute form. In-stead, they're e x pressed in segment offsetform , with the segm ent and off set valueswritten in hexadecim al and se par ated bya colon. The addre ss 12345 h, f or exa n1p le,would normally be writt en as 1234:000 5.The ad vant age to using this notation i sthat it clearl y s pells out the values used toarr ive a t the resultant ad dr ess.

That's not the onl y way to arrive at12345 h,of cour se. The ex press ion 1234:lXX l5

tr eats it as the fifth b yte in the segment based at paragraph 1 234 h. But it 's alsothe 21st (hex 15) byte in the segmentstarting at 1233h (1233:0 015 ), the 37th(hex 25) byte in the segment at 1232h(1232:0025 ), and so on. For any given byte in memor y, ther e ar e thousand s of d iffere nt ways to formul ate its add ress . Infact , it's a mistak e to believe that mem-or y in a PC eq uip ped with 640K o f RAMis d ivid ed into ten evenly s paced seg -ments. Segment s can begin on any 16- byte boundary. Th ey can also over la p,leavi ng you 4 ,096 w ays to fOll11Ulatea n

add ress ( unless of cour se it's protec ted mode , in which ca se ther e ar e as many

Index

Figure 2: In p rotec te d mod e , segment

registers ar e load e d with selectors rather tha n physic al s egm e nt addresse s . The 13 -bit

inde x field holds a descr ip tor number , which

references a desc r iptor in the Global

Descriptor T a ble (GO T) if bit 2 (the Ta ble

Inde x bit) is 0 or in the Lo cal Descriptor Ta ble

(LOT) if Tabl e Inde x is 1 . The RP L field h old s

a number from 0 to 3, representing ther e que s ted pr ivilege level , with 0 rankinghighe s t an d 3 lowest .

8/10/2019 IBM Microsoft Segmented Memory

http://slidepdf.com/reader/full/ibm-microsoft-segmented-memory 2/5

way s to for mulate an addres s as there are paragr a phs of memor y).

One w ay to view segments is to think of them as 64K window s onto memorythat can b e moved around as needed toreferen ce s pecif ic tar gets inside a larger ,mor e extensive addr ess s pace. On the 8086 ,regi ster s CS, DS, ES, and SS hold seg-ment addr esses. By changing a segmentregister 's value , you can move the 64K windo w ass ociated with that register to adifferent loc ation in m emor y. In assem- bly language, the instruction

mean s" go to the segment that ES point sto and get me th e byte at off set 1000h ,then pl ace i t in register AL. " As you mayhave guessed , the 64K limit on segmentsize arises from the fact that the large stnumber th at can be f ormed with a 16-bitquantit y, a nd thu s the highe st off set thatcan be addre ssed within a segment , is65,535 -exactl y 64K minus 1 .

286 SEGMENTATION

The same concept of segmentation thatapplie s to the 8086 proc essor also applie sto a 286, 386, or 486 running in realmode . Memor y addre ssing is limited to1MB , and 20-bit addr esses are formed from 16 - bit value s.

In cert ain cases, you can acces s morethan 1MB of memory in r eal mode byloading a segment regi ster with a valuegreater th an FOOOhand combining it withan offset address high enough to producean absolute addre ss higher than 100000h.For example , the segment:offset addressFFFF: 1000h actually ref erences ab soluteaddre ss 100FFOh , which is clearl y abovethe 1MB boundar y. Since an 8086 , withits 20 address line s, isn't physically ca- pable of addre ssing more than 1MB of RAM , the address wrap s around , becom-ing OOOO:OFFOh .But on 286s , 386s , and 486s, sele ctively enabling the A20 ad-dress line (which is normally used only in protected mode ) allow s the CPU to reachinto the f ir st 64K (minus 16 bytes) of upper m emor y, even though it 's runningin real mode . This 64K region i s knownas the H igh M emory A r ea, or HMA . Mi-crosoft 's protot y pe Ext ended Memory Speci-fication (XM S) dri ver HIMEM.SYS , whichwas di scussed in the December 11 ,1990 ,

Reserved

63 484746 454443 414039 16 15 0

1.1.1_-10-15 16-bit limit (segment size)

16 -39 24-bit base address of segment40 Accessed (A) bit

a = selector for the descriptor has been transferred into a se gm e nt regi st er 1 = selector for the descriptor has not been transfer red i nto a s e gmen t r e gister

41 -43 Types of operations permittedon the segment000 = data segment; read-only ; expand up001 = data segment; read /write ; expand up010 = data segment; read-only ; expand down011 = data segment; read /write ; expand down100 = code segment ; execute-only ; nonconfo r ming101 = code segment ; execute /read ; nonconform ing110 = code segment ; execute-only ; conform ing111 = code segment ; execute /read ; conforming

44 DescriptorType (DT) bit

a = system segment or system gate1 = memory segment

45 -46 Descriptor Privilege Level (DPL) bits00 = privilege level a01 = privilege level 110 = privilege level 211 = privilege level 3

47 Present (P) bita = segment not present in memory1 = segment present in memory

48 -63 Reserved; should be zeroed for compatibility with 386

Figure 3: The 286 segment descriptor defines the location in memory an d the s ize of a

segment in protected mode . Base addresses are 24 b its long , allow ing s eg ments to b e located

anywhere in the 286 's 16MB physical address space . Limits are 16 bits long, re st r icting the

maximum segment size to 64K . Bits 40 through 47 store au xiliary inform ation tha t de f ines the

segment 's privilege level, type , and current status . Expand down segm e nts are segm e nts that

extend downward in memory , from higher to lower addresses . They are t ypically us e d to hold

stacks, which also grow downward in memory. Conforming segments ar e special cod e

segments that may be called by processes running with equal or lower p rivilege levels .

Tutor column , provides a mechanism for controlling the A20 address line and for arbitrating acces ses to the HMA amongcompeting processe s.

But protected mode is quite different .In protected mode , memory is still seg-mented , but segment registers hold quan-tities known a s selec tors rather than segmentaddresses. Selectors reference descriptorsstored in descriptor tables located in RAM,and descriptors,in turn , spell out the physicallocations (and sizes) of correspondingsegments of memory.

Figure 2 shows the format of a selec-tor . The 13-bit index field serves as anindex into one of the system's descriptor tables. One table can hold up to 8 ,192different de scriptor s. The operating sys-tem typic ally sets up sever al such table s-a

Global De scriptor Table, or G DT, whichold s descriptor s for segments of memor y reser ved for opera ting system useand a series of Local Descrip tor Ta bles,or LDTs , normally used to map the memoryset aside for a pplication pr ogr ams to govemaccess to d ifferent p arts of me mory. If bit2, the Table Inde x bit, in the selector is 0,then the selector re fere nces the GDT; if it's set to I , refer ence is mad e ins tead tothe current LDT (which LDT is current isdetermined by anoth er reg ister, the LDTR r egister , located in the CPU).

Bits 0 and 1 hold the R equeste d Privi-lege Level, or RPL . Privi lege levels playa role in 286/386 memory protec tion ~r estrictin g segment accesses to proc es!>(that is , appli cation progra ms) with equalor higher privilege leve ls than the seg-

8/10/2019 IBM Microsoft Segmented Memory

http://slidepdf.com/reader/full/ibm-microsoft-segmented-memory 3/5

,,,,,L ,." "descriptor table •1FFF ~

II~: Jill 1 A O O O O

5

84F~ 84E__ 840. : 4 C

1

o

Offset. 1 : 1 . 6

Selector

i r II ·· · ··. . ..

Protected-modememory

Descriptor

43 39 1615 0

•••....'Ii'i.'IL. ""

Limit = 7FFFhSegment is 32K inlength .

' ig~re 4: In protected mode , the path to a location in memory is keyed to a descriptor table,

ere the segment 's base address and limit are defined. Here , the selector references

.scriptor number 84Eh in the LDT. The base address of the segment , 1AOOOOh ,is stored in

e descriptor its el f , as is the segment 's limit, 7FFFh . The offset value 4B05h is added to the3se to form absolute address 1A4B05h. The ma ximum size of the base , limit, and offset is

lrocessor-dependent .

ment s they 'r e tr ying to access . In OS/2,for example , r egion s of memor y that belongto the oper ating sys tem ar e assigned thehighest-po ssible pri vilege level , prevent-ing application pro gr ams, which run atlower privil ege leve ls, f rom corruptin goper ating system d ata. That's one r easonit's called pr ot ect ed mode.

Figure 3 show s the format of th e 286segment de scriptor . Bit field s within the

M-bit descript or d efine wher e a segmentlies in memor y and how f ar i t extend s.

gment si zes a re adjusta ble on the 286 , _ ..It the maximum size of a segment isstill MK-th e same limit im pose d on the8086. Wh y? The d escriptor's limit field,

which define s the size of the segment , isonly 16 bit s wid e. The limit i s the highe stoff set that can be addr essed within thesegment; any attempt to access memory beyond the po sted limit re sult s in a pro-tection fault . On the other h and , 24 bitsare set asid e for the segment 's base ad-dres s, permittin g the segment to be lo-cated anywhere within the 286' s 16MB physical addre ss s pace .

Bit s 40 throu gh 47 in the 286 segmentdescriptor contain au xiliary informationsuch as whether or not the selector for thedescriptor has been loaded into a segmentr egister , whether the corr es pondin g seg-ment is a cod e or d ata segment and what

The d escr i ptor type (whether it points toa memor y segme nt or to some other ty peof sy stem obje ct, such as a call gate, w hichallo ws processes to call r outines wi th higher privilege levels), the descriptor privi legelevel, and whether the segmen t is cur-rentl y present in memory are also fo und here . This type of inf or mation, eso ter ic asit may seem, is crucial to a CPU ope rat-ing in a multit asking environ ment wher eeach pro cess must be pr otected fro m other sacti ve in the sys tem at the same time.

Figur e 4 d iagr ams the process a 28 6goes thr ough to convert an address sto r ed in selector:offse t for m to a physical ad -dre ss in memory. In this examp le, thesele ctor referen ces a d escriptor i n the Lo calDescript or Table, whose base address isstor ed in the LDTR register . The se lec-tor 's ind ex field holds the hexadecima lvalu e 84E, which points to t he 84Ethdescript or in the LDT. The descripto r 's base add ress f ield holds the 24 - bit valueIAOOOOh ,which says that the correspo nd-ing segment is based at abso lute addresslAOOOOh ,a location approximate ly mid-

way betwee n the 1MB and 2MB ma rk s.The limit field hold s 7FFFh , indicatingthat the segment is 32K in length. F i-nall y, the off set val ue 4B05h is appl ied relati ve to the base of the segme nt Gus t asit is in real mode), yie lding the ab soluteaddr ess IA4B05h.

At a glance , all this may seem conf us-ing. But a closer look reveals some of theinher ent ad vantages to structur ing me m-or y this way. By setting up a un iq ue LDTfor each process, a protected -mode oper -atin g system c an load multiple processesinto memor y and k eep them physica lly

isolat ed f r om one another . The premise isthat a process can't acces s any region o f memory th at's not m a pped into its LD T.The ope r ating system can also prev ent proce sses from accessing t he GDT byrunnin g them at lower privilege leve lsthan the GDT allows. In the end, y ouhave a solid foundati on for a mult itask ingenvir onment , wh er e no program can corr uptthe system b y ove r wr iting memory t hathas been allocated to another pr ogr am.

386 SEGMENTATION

Segm entation on the 386 is remarkab ly

simil ar to segm entation on the 286 , butwith a twist: the add ressing ca pa bilitiesof the 386 are far super ior to those of t he286 , providing more f lexibility i n the way protected-mod e memory can be str ucture d .In f act, the 386 was the fir st Inte l CPU to

8/10/2019 IBM Microsoft Segmented Memory

http://slidepdf.com/reader/full/ibm-microsoft-segmented-memory 4/5

THE 386 SEGMENT DESCRIPTOR FORMAT! ii f l

~63

Base 0-23 Limit 0-15Base 24-31

0-15

16-3940

Lowe r 16 bits of 20-bit limitLowe r 24 bits of 32-bit bas e a dd r ess

Accesse d ( A) bita = se lector for t he desc riptor has bee n trans f e rr e d

into a segment register 1 = se lector for the descripto r has not been transferred

into a seg ment reg ister

Types of oper a t ions p e rmitte d on the se gm e nt000 = data se gment ; rea d-only; expan d up001 = data segment; read /write ; expan d up010 = data segment ; rea d-only; ex pand down011 = data segment ; rea d/wr ite ; expa nd down100 = code seg me nt ; exec ute -only; nonco nfo rming101 = code seg ment ; exe cute /r ea d; non conf orming110 = co de seg ment; execute-o nly; confo r ming111 = code segment ; execute /read ; conforming

Descri ptor Type ( DT) bita = system seg men t or sys te m ga te1 = me mo ry seg me nt

Desc r iptor Privilege Level ( DPL ) bits00 = privilege level a01 = privilege level 110 = privilege level 211 = pr ivilege leve l 3

Pr esent ( P ) bita = seg ment not presen t in m em or y1 = segment present in mem ory

Upper 4 bits of 20 -bit limit Ava ilab le ( AVL) bit; availa ble to sys tem s pr ogr a mmer sRese r ve d; should b e zer oe d for comp a tibility withfutu r e m icro processo r s

Default (D) bit; always 1 on the 386Granularity (G) bit

a = byte gra nu lar ity1 = pag e gr a nula rity

Upp e r 8 bits of 32- bit base addr ess

48 -5152

53

Figure 5: The 38 6 se gm e nt descriptor , an e xtension of the 286 descriptor , permits segments

to be lo cated anywhere in a 4GB address space and individual segments to be up to 4GB in

length. Although t he 20-b it limit field would at first seem to restrict segment sizes to 1MB ,

setting the de sc riptor 's g r a nularity b it (bit 55 ) cau ses th e value in t he limit f ie ld to be

interp rete d as the nu mb e r of 4K pa ge s ra ther than the numbe r of byte s , resulting in a 4GB

ceiling o n se gm e nt size.

remove the 64K size limit on segments,

mak ing it an attrac tive p latfor m for o per -ating systems that fare better on pr oces -sor s not bound by seg mented m emor y,such as Unix.

Figure 5 maps t he conte nts of the 386segment descriptor, w hich is simpl y anextension of t he 286 descr ipto r . Note thed own ward co m pati bility betwee n the twoforma ts: every field t hat's defined on the286 is also defined on the 386 in the same

loca tion . As a res ult , a pr ogr am written to

run in p r otected mod e on the 286 will r unf ine on a 386, as long as the unuse d bitsin the d escript or ar e zeroed out , a pr ecau-tion Int el r eco mm ended to 286 systems progr ammer s from the beginnin g, to ensur ecompati bility with f utur e CPUs.

The 386 assig ns 8 addition al bits ( bits56 thr ough 63) to a segment's base ad-dress, expanding the ra nge of me mory itcan be located in from 16M B to 4GB . It

also assign s 4 additional bits (bits48 t hrough51) to the limit , expanding the effect ivesegment size from 64K to 1MB. But thelimit on segment size doesn't stop there.Setting the Granul arity bit in the 386descriptor (undefined in the 286 descrip -tor) cause s the valu e in the limit field to be interpreted a s the number of pagesrather than the numb er of byt es. Since a page is 4 ,096 byte s (4K) in len gth, pagegranularity increase s the maximum sizeof a segment on the 3 86 to an almos tunfat homab le 4GB . Your ma chine maynever have that much memory , but if your un an o perati ng system that virtua lizesmemory, like OS/2 , it doesn't matter : your programs can be fool ed into thinking there'sthat much memory in stalled .

How do you acce ss individual of lwithin one of these huge segments ? Simple:with 32 - bit register s. All general registerson the 386 are 32 bit s wide . From a programmer 's point o f view , addr essesare sti ll spel led out with selectors a nd offsets; but while the selector i s still 16

bits wide , the off set is 32. Con structsuchas ES:[EDI] are common in 386 assam bly language, where EDI is the 32- J-

ver sion of the 16-bit DI register 8086 ill

286 p r ogrammers are accustomed to.

VIRTUAL MEMOR Y

Are seg ments really all that bad? Onctu pon a time, when the majority of app li-cat ions were still written in a ssemb lylanguage, they did m ake life difficult for deve lope r s. Segments permeate every fibe r of a program's being. It's inherently mo r ediff icult to write a program that dea ls

with mult iple code or data segments tha none that limits its code and data to 64K -thesize of a single segment . That 's why, for exa mpl e, good-old BASICA limited codeand d ata to 64K and some language inte r- preterss till do. Fort unatel y, compiler s capab leof handling far code and data objects -co -and d ata stored in remote segment s-no .do most of the work for us.

But segments aren 't alway s the bad

8/10/2019 IBM Microsoft Segmented Memory

http://slidepdf.com/reader/full/ibm-microsoft-segmented-memory 5/5

guy s. In prote cted mode, they playa cruci alrole in helpin g oper ating system s to vir-tualize memo ry. Virtu al memor y require sthat physical memor y be partitioned into

easily mana ged units that can be swapped to and from th e hard disk. Segment s 64K or less in length fit this role quit e well .On both th e 286 and 386, the descriptor 'sPresent bit ( bit 47) is set to 1 when asegment i s pr esent in memory and to 0when it is swa pped out .

When a process a ttempt s to acces s aswapped segment , the CPU generates anexce ption that i s tr a pped by the operatingsys tem. Th e oper ating sys tem in turn get sthe opportunit y to load the segment ba ck into mem ory, swa pping other segment sout , if necessa ry, to make room.

The pr o blem with huge segment s onthe 386 is that they can be too lar ge to beswa pped to the har d disk. The y can also be small eno ugh to fit on the hard disk buttoo lar ge to be swapped in and out effi-ciently. That's why the 386 supports asecond fo rm of virtu al memor y manage-

t known as. paging. Paging dividesh .. of memory into a series of evenlys paced 4K pa ges. If segment sizes pro-hibit swappin g at the segment le vel, page scan be swapp ed inste ad .

The 386- s pecific v ersion of OS/2 dueout tater thi s year will u se paging rather

thar segment ation to virtual ize memory,'na':>lingdeveloper s to set up huge 3861ments without sacr ificing the virtual-~mory capabilities o f the CPU.If you'd like to know more abo ut

~gmentation , paging , protection , and virtualnemory m anagement on the 286 and 386,nere 's a list of recommended reading:

Inside the 80286 , by Ed Stra uss (1986,Prentice -Hall) ; Pr og r ammin g the 80386 , by John H . Crawford and P atrickP. Gel singer (1987, SYBEX) ; and The 80386 Book , by Ross P. Nel son (1988 , Microsof t Press).

ASK THE TUTO RThe T utor solves practical prob lems and explains techniques for using yo ur hard-ware and software more productive ly.Questions aboutOOS and sy stem s in generalare answered here. To have yo ur question~'1swered, write to Tutor, PC Magazine ,

IePark Avenue, New York , NY 100 16,or uploa d it to PC MagNet (see page 8 for access instruction s). We'r e unabl e to an-swerquestion s individually. •

Haven' t YouHeard the News?

You don 't have to buy th e whole newsstand to get

copies of your latest article or r eview. Order customized reprint s from Ziff..Da vis Publishing Co.

and let potential cl ients r ead all about it. *

To f ind out how yo u can have your arti cleor review r e printed, c ontact Claudia Hardi son-R eprints Manage r ; Ziff ,Dav isPub lishing Com pany ,

One Park Ave., New York,NY 10016,212,50 3,5447 .