PortableSystemsGroup

NTOS/2FileSystemDesignNote
Author:GaryD.Kimura
Revision1.0,Sep11,1989

Copyright (c) Microsoft Corporation - Use subject to the Windows Research Kernel License

NTOS/2FileSystemDesignNote
1.Introduction..............................................................................................................1
1.1FunctionofFileSystem.....................................................................................1
1.2NTOS/2EnvironmentfortheFileSystem.......................................................1
2.FileSystemOperations............................................................................................2
2.1FSD/FSPDispatchandCommunication...........................................................2
2.2FileSystemInitialization...................................................................................3
2.3BeforetheFirstOperationonaVolume...........................................................3
2.4OpenandCreateFileOperations......................................................................5
2.5Read,Write,Set,andQueryFileOperations....................................................6
2.6CloseFileOperation..........................................................................................7
3.LooseEnds...............................................................................................................7

Copyright (c) Microsoft Corporation - Use subject to the Windows Research Kernel License

NTOS/2FileSystemDesignNote
1.Introduction
ThisdesignnotedescribestheimplementationofanNTOS/2filesystem.AllNTOS/2file
systemssharesimilarpropertiesintheircommunicationwiththeNTOS/2I/Osystem,andtheir
basicinternalflowofcontrol.Theinformationpresentedhereisbasedonknowledgegained
whileimplementingtheFATfilesystemforNTOS/2,butalsocontainsinformationapplicable
toallNTOS/2filesystems.
Beforereadingthisdesignnote,thereadershouldbefamiliarwiththe"NTOS/2I/OSystem
Specification"documentandthesectionwithinthe"NTOS/2MemoryManagementDesign
Note"regardingI/Osupport.TheI/OsystemspecificationpresentsahighlevelviewoftheNT
OS/2userAPIcallsandthesupportroutinesitprovidesforusebythevariousNTOS/2file
systemsanddevicedrivers.ThisdesignaugmentstheI/Osystemdocumentbydescribingin
detailhowafilesystemactuallytiesintotheI/Osystem,intermsoftheI/Osystem'smajordata
structuresandflowofcontrol.
ThisdesignnotepresentsatourofthecommunicationthatoccursbetweentheNTOS/2I/O
systemandthefilesystem.Itisintendedasaguidetofilesystemprogrammersinunderstanding
howthefilesysteminteractswiththeI/Osystem,andavoidsdiscussingfilesysteminternals,
suchasitsinternaldatastructuresoritsresourcelockingmechanismandgranularity.
1.1FunctionofFileSystem
Thetaskofthefilesystemistohandleuser(andsystem)generatedrequeststoreadandwrite
filestoorfromadiskvolume.TheuserpassesallI/OrequeststotheI/Osystem(viacallssuch
asNtReadFile)whichinturnpassestherequesttotheappropriatefilesystemordevicedriverin
astructurecalledanI/Orequestpacket(IRP).TheIRPcontainsafunctioncodeandparameters
appropriateforthefunction.Forexample,toopenafiletheIRPcontainsacodeindicatingan
openfileoperationandafilenameparameter1.
InthecaseofafilesystemtheIRPalsocontainsapointertothedeviceobjectdenotingthetarget
devicethatthefilesystemshoulddirectallsectorreadandwriterequeststowards.Thatis,to
actuallyreadasectoronadevicethefilesystemcallsbacktotheI/Osystemwithinformation
fromtheIRPindicatingwhichdevicetoread.
Inthissense,thefilesystemisnottiedtoanyphysicaldevice.FromthestandpointoftheI/O
system,afilesystemactsmorelikeafilter.InNTOS/2,diskdriversonlydealwithreadingand
writingphysicaldevicesectors.Theprimaryjobofthefilesystemistoaddstructuretothis
device.SoauserorsystemrequesttomanipulateafilegoesfromtheI/Osystemtothefile
systemwhereitispassedbacktotheI/Osystemasactualreadsandwritesofsectorsonadisk.

TheIRPactuallycontainsadditionalopenfileparametersthatarenotdiscussedinthisexample.

Copyright (c) Microsoft Corporation - Use subject to the Windows Research Kernel License

NTOS/2FileSystemDesignNote
1.2NTOS/2EnvironmentfortheFileSystem
AnNTOS/2filesystemexportsthreeentrypointsandtheseentrypointsareonlycalledbythe
I/Osystem(i.e.,auserneverdirectlycallsthefilesystem).Thereisaninitializationroutine,a
dispatchroutine,andanunloadroutine.
TheinitializationroutineiscalledatsystemstartupduringI/Oinitialization.Thisroutineis
responsibleforcreatingthefilesystem'sdeviceobjectandforinitializinganyglobaldata
structuresandprocesses/threadsusedbythefilesystem.Thestructureofafilesystem'sdevice
objectiscoveredlaterinthisdesignnote.
ThedispatchroutineiscalledbytheI/OsystemtoprocessallIRPstargetedforthefilesystem.
Theseincluderequeststomountavolume,openorcreateafile,readandwritetoafile,and
closeafile.Logically,thedispatchroutineeithercompletestheI/Orequestandthenreturns
controltotheI/Osystem,oritreturnstotheI/OsystemimmediatelyandindicatesthattheI/O
willbecompletedatalatertime(i.e.,itsreturnstatusisSTATUS_PENDING).
TheunloadroutineiscalledbytheI/Osystemwhenthefilesystemisbeingremovedfroma
runningNTOS/2system.Theunloadroutineisresponsibleforcleaningupallofitsglobaldata
structures,processes,andthreads.Aftercallingunload,thefilesystemisessentiallyunavailable
foruseuntilitsinitializationroutineiscalledagain.
2.FileSystemOperations
ThissectiondescribestheimplementationdetailsofanNTOS/2filesystem.Itdescribesthe
basicflowofcontrolthroughthefilesystemanditscommunicationwiththeI/Osystem.
2.1FSD/FSPDispatchandCommunication
ThetermFileSystemDriver(FSD)referstoasetofroutinescalledbytheI/Osystem(viathe
filesystem'sdispatchroutine).Thesearekernelmoderoutinesthatexecuteinthesamecontext
inwhichtheyarecalledbytheI/Osystem.Thatis,thereisnotaspecialprocess/threador
contextswitchassociatedwithexecutingtheFSD.ExecutionpathsthroughtheFSDshouldbe
fairlyshortandnotrequirelongwaits(e.g.,foradiskI/Otocomplete).
TheworkhorseofthefilesystemistheFileSystemProcess(FSP).ThetermFSPreferstothe
setofthreadsexecutingwithinadedicatedfilesystemprocess.UnliketheFSD,theFSPhasthe
entireuseraddressspaceavailableforuse,andcandedicatethreadstosystematicallyattendto
thetasksassociatedwithanI/Orequest.AnFSPthreadcanwaitforI/Otocompleteandcan
easilymaintaincontextunavailabletotheFSD.
Figure1illustratestherelationshipbetweentheFSDandtheFSP.TheFSDiscalledbytheI/O
systemwithanIRPcontainingtherequestedoperation.Then,basedontheamountofworkand
contextrequiredbytherequest,theFSDeithersendsthenecessaryreadandwriterequeststothe
targetdeviceobject(asassociatedIRPs)oritenqueuestheIRPtotheworkqueueusedbythe

Copyright (c) Microsoft Corporation - Use subject to the Windows Research Kernel License

NTOS/2FileSystemDesignNote
FSP.IftherequestisqueuedtotheFSP,theFSPdequeuestheIRPandperformsthework
requiredtocompletetherequest.

Figure1FSD/FSPLayout

Asaruleofthumb,filereadandwriterequestscanusuallygothroughtheFSDdirectlytothe
targetdeviceobject.Allotherrequests,suchasfilecreate,thatrequirefilesystembufferingor
datastructureeditinggothroughtheFSP.
Withinasinglefilesystem,therearemultipleworkqueuesandanFSPthreaddedicatedtoeach
queue.TheFSPthreadisanormalthread(i.e.,itismerelythenameofathreadthatishandling
workqueueactivity).Thereisaqueuetohandlemountrequestsandaqueueforeachmounted
volume.Theconstructionanduseofeachworkqueueisdescribedinthefollowingsubsections.
2.2FileSystemInitialization
ThefilesysteminitializationprocedureiscalledatsystemstartupduringI/Oinitialization.This
procedureisresponsibleforcreatingthedeviceobjectforthefilesystem,creatingtheFSP
process,andstartingtheFSPmountthread.Italsoinitializesalloftheglobaldatastructures
usedbythefilesystem.Itiscalledwithapointertothedriverobjectforthefilesystemcreated
bytheI/Osystem.
Thedeviceobjectcreatedatinitializationiscalledthefilesystemdeviceobjectandcontainsthe
nameofthefilesystem(e.g.,"\Fat")andaworkqueue.AllmountrequestsaresentbytheFSD
dispatchertothisworkqueueandareprocessedbyadedicatedFSPmountthread.Figure2
illustratestherelationshipbetweenthefilesystemdeviceobjectandtheFSPmountthread.

Figure2FileSystemInitialization

Insummary,theinitializationphasecreatesafilesystemdeviceobjectandsetsapointertoitin
theDriverObject.ItalsocreatestheFSPprocessandFSPmountthread.TheFSPmountthread

Copyright (c) Microsoft Corporation - Use subject to the Windows Research Kernel License

NTOS/2FileSystemDesignNote
onlyprocessesrequeststhatareplacedinthefilesystemdeviceobject'sworkqueue,andthe
onlytypeofrequestsvalidinthatworkqueuearemountvolumerequests.
2.3BeforetheFirstOperationonaVolume
WhenausercallsNtCreateFile,oneofthefirstthingstheI/Osystemmustdecideiswhetherthe
volumetargetedbytherequestismounted.Ifthisisthefirstoperationonthevolume,thenthe
volumeisnotmountedandtheI/OsystemmustsendanIRPtothefilesystemrequestingthatthe
volumebemounted.Afterthevolumeismounted,theI/Osystemcanthenforwardthecreate
filerequest.
Onamountrequest,theI/OsystemcallsthefilesystemFSDdispatchroutinepassingasinput
parametersapointertothefilesystemdeviceobjectandanIRP(SeeFigure3a).TheIRP
containstwoparametersofinterest2.Thereisapointertoavolumeparameterblock(VPB)and
toatargeteddeviceobject.TheVPBisastructureusedtodenoteadiskvolume.Itwillcontain,
afterthevolumeismounted,thevolumelabelanditsserialnumber.Withinthesystem,thereis
oneVPBforeverymountedvolume.
Thetargeteddeviceobjectisthedeviceobjectthatistobeusedbythefilesystemwhenitissues
areadorwritesectorrequest.TheVPBalsocontainsapointertoadeviceobject(thisfieldisset
bytheI/Osystem)calledtherealdevice(notshowninFigure3).Inmostcases,thetargeted
deviceobjectandrealdeviceareidenticalandsimplydenotethediskdrivercontainingthe
volume(e.g.,afloppyorharddisk).However,toaccommodatemultivolumedisksetsanddisk
strippers,thetargeteddeviceobjectandtherealdevicecanbedifferentdeviceobjects.Thefile
systemonlysendsrequesttothetargeteddeviceobjectandnevertotherealdevice.
Figure3ashowstheparametersanddatastructuressenttothefilesystemFSDdispatchroutine
onamountrequest.Whenitiscalled,theFSDdispatcherwillenqueuethemountrequesttothe
FSPmountthread'sworkqueue.NotethattheworkqueueislocatedbytheFSDdispatchervia
thefilesystemdeviceobjectwhichispassedinasaninputparameter.

Figure3ProcessingaMountRequest

Toprocessthemountrequest,theFSPmountthreadissuesthenecessaryreadsectorrequeststo
thetargeteddeviceobjecttodecideifthevolumebelongstothisfilesystem(e.g.,whetheritisa
FatfilesystemoraPinballfilesystem).Ifthemountissuccessful,thefilesystemcreatesanew
AnIRPactuallycontainsmorethantwoparameters,butonlythesetwoaregermanetothis
discussion.
22

Copyright (c) Microsoft Corporation - Use subject to the Windows Research Kernel License

NTOS/2FileSystemDesignNote
deviceobjectforthevolume.Thisdeviceobjectreplacesthedeviceobjectoriginallyreferenced
bytheVPB.Itcontainsaworkqueueforthevolume,andavolumecontrolblock(VCB).Upon
completionofasuccessfulmountoperation,thefilesystemalsoinsertsintheVPBthevolume
labelandserialnumber.Figure3bshowstheresultsafterasuccessfulmountrequest.
TheFSPmountthreadalsocreatesanewFSPvolumethreadtoprocesssubsequentI/Orequests
targetedforthevolume.Thatis,whentheI/OsystemcallstheFSDdispatchertodoI/Otoa
mountedvolume,itwillpassasinputapointertothevolumedeviceobjectandnotthefile
systemdeviceobject.TheFSDdispatcherwillthensendtherequesttotheworkqueueforthe
appropriateFSPvolumethread.
Aslightlydifferentchainofeventstakesplaceifthevolumebeingmountedhaspreviouslybeen
mountedbythefilesystem.Thatis,thefilesystemalreadypossessesaVCBforthevolume.
ThissituationcanoccurwithremovablemediawheretheI/Osystemisattemptinganoperation
onwhatitthinksisanewvolume,wheninrealitythevolumewasmountedinadifferentdriveat
anearliertime.
Tohandletheremountcase,thefilesystemwillkeeptrackofeverydeviceobjectthatis
mountedandsearchthislistwheneveranewmountrequestisprocessed.Ifamatchwitha
previousvolumeisfound,thefilesystemwillthensimplyreplacethenewVPBitisgivenas
inputwiththeoldVPB.ThissimplyrequireschangingtherealdevicepointerfoundintheVPB
andthetargetdeviceobjectpointerfoundintheoldVCB.Theremountprocessisillustratedin
Figure4.

Figure4VolumeRemount

Insummary,thefilesystem'smountprocedureisusedtoestablishthatthetargetdeviceobject
containsaproperfilesystemstructure.Ifthefilesystemrecognizestheondiskstructure,it
createsanewdeviceobjectforthevolumeinplaceofthefilesystemdeviceobject,setsthe
volumelabelandserialnumberintheVPB,andstartsupanFSPvolumethread.
2.4OpenandCreateFileOperations
Theopenandcreatefile3operationtakesasinputafileobjectdenotingthefilebeingopenedand
thedeviceobject(createdbythefilesystem)whichistargetedfortheoperation.TheFSD
dispatcherqueuestheopenandcreatefilerequeststotheworkqueuefortheappropriateFSP
volumethread.Figure5aillustratestheinputgiventotheFSDdispatcher.
Thisdiscussionalsoappliestoopeningandcreatingdirectories.

Copyright (c) Microsoft Corporation - Use subject to the Windows Research Kernel License

NTOS/2FileSystemDesignNote

Figure5OpeningandCreatingaFile

Thefileobjectpassedtothefilesystemcontainsthefilenamebeingopenedrelativetotheroot
ofthevolumeanditcontainstherequestedaccessmode.Itisthejobofthefilesystemtoopen
thefileonthetargeteddeviceobject.Theinformationavailabletothefilesystemisthedevice
objectofthevolume(passedasinputfromtheI/Osystem),theVCBandVPBassociatedwith
thevolume,andapointertothetargeteddeviceobject.TheFSPvolumethreadisfreetoissue
asmanyreadandwritesectorcommandstothetargeteddeviceobjectasnecessarytoopen
(create)thefile.
Whenthefileissuccessfullyopened,thefilesystemcreatesafilecontrolblock(FCB)4anda
contextcontrolblock(CCB)whicharebothusedtostoreinformationabouttheopenedfile.
BothoftheserecordsarereferencedbythefileobjectthroughitstwoFsContextpointers.The
fileobjectalsocontainsasectionobjectpointerwhichpointstoareservedlongwordintheFCB.
Thislongwordisusedbythememorymanagementsystem.SeeFigure5b.
TheFCBandCCBareinternalfilesystemcontrolstructures.AnFCBiscreatedforevery
openedfileonavolume.Itcontainsthefilename,mappinginformation,andapointertothe
VCBcontainingthefile5.OnlyoneFCBiscreatedperopenedfile.TheFCBissharedby
multiplefileobjectsifthefileobjectsdenotethesamefile.ACCBiscreatedforeveryfile
object6andcontainsinformationspecifictothefileobject,suchascurrentfileposition
information.
Insummary,whenopeningafile,thefilesystemispassedasinputafileobjectdenotingthefile
andadeviceobjectdenotingthevolumewherethefileistoexist.Iftheopenoperationis
successful,thefilesystemcreatesanFCBandaCCBrecord,bothreferencedbythefileobject.
2.5Read,Write,Set,andQueryFileOperations
Thebasicoperationsonafile,afteritisopened,areread,write,set,andquery.Theseoperations
takeasinputafileobjectandapointertothedeviceobjectforthevolume.SeeFigure6.

Iftheoperationopensadirectory,thenthestructureiscalledadirectorycontrolblock(DCB).

ThisisasimplificationoftheactualFCBstructure.

Actuallyasanoptimizationitisonlycreatedwhenabsolutelynecessary.

Copyright (c) Microsoft Corporation - Use subject to the Windows Research Kernel License

NTOS/2FileSystemDesignNote

Figure6Read,Write,Set,andQueryFileInput

WhenprocessinganIRPofthistype,theFSDdispatchercaneithersendtheIRPtotheFSP
workqueueandlettheFSPvolumethreadhandletherequest,oritcancompletetherequest
itself.TheFSPcanactuallyhandlealltypesI/Orequests;however,forefficiency,itisbetterto
havetheFSDhandletherequestswhenpossible.ThecriterionusedtodetermineiftheFSDcan
handlearequestaretheamountoftimerequiredtoprocesstherequestandthetypeofresource
locksneededtoreadoralterthefilesystem'sinternaldatastructures.Ifalotofextraprocessing
isrequiredoriftoomanytimeconsuminglocksareneededtoperformtheoperation,thenthe
FSPshouldhandletherequest.
Theseoperationsdonotaltertheinfrastructureconnectingthefileobject,deviceobject,FCB,or
CCB.However,theystillmayneedtoacquireanexclusiveresourcelockonthestructuresin
ordertoaltertheirotherfields(suchassettingadeleteflagorexpandingfileallocation).
2.6CloseFileOperation
Thelastoperationonafileisthecloseoperation.Thisoperationtakesthesameinputasthe
precedingread,write,set,andqueryoperations,butunlikethoseoperations,thecloseoperation
modifiestheinfrastructure.
Todoacloseoperation,thefilesystemfirstperformsanynecessaryI/Otothetargetdevice
objecttoeithercloseordeletethefile.ItthenremovestheFCB7andCCBfromitsinternaldata
structure.Afterthecloseoperationthefileobjectisnolongervalidandmustberemoved(or
recycled)bytheI/Osystem.
3.LooseEnds

TheFCBisremovedonlyitifisnolongerreferencebyanyfileobject.

Copyright (c) Microsoft Corporation - Use subject to the Windows Research Kernel License

NTOS/2FileSystemDesignNote
RevisionHistory
OriginalDraft1.0,September7,1989

Copyright (c) Microsoft Corporation - Use subject to the Windows Research Kernel License