10/4/2015

LSFSdevicetechnicaldescription|StarWindKnowledgeBase

LSFSdevicetechnicaldescription
Published:August7,2015byTarasShved
ThisarticledescribesLogStructuredFileSystemdevices.Itprovidestechnicalproperties,requirements,
limitations,featuresandbenefitsoftheLSFSdevice,whichshouldgivetheendusernecessaryknowledgeand
understandingforitsimplementationintothesystem.

Technicalproperties
1. LSFSisalogstructuredfilesystemthatkeepstrackofthechangesthataremadeinajournal.Thisfile
systemkeepsnodata,changesonly.
2. Olddataarestoredonthefilesystemuntilonlinedefragmentationcleansitup.
3. LSFSisasnapshotbasedfilesystemandusessnapshotsbasedontheconceptofjournalingfilesystem.
Everysnapshotisincrementalandoccupiesadditionalspacethatisequaltochangesmadesinceprevious
snapshotcreation.
4. Thejournaldividesintofilesegmentsforconvenience.Theminimumsizeoffilesegmentis128MB,
maximum512MB.LSFSalwayskeepsoneadditionalemptyfilesegment.
5. JunkratepredeterminesmaximumallowedLSFSgrowth(overprovisioning)comparisontodeclaredLSFS
size.Thedefaultrateis60%causingLSFSfilesegmentsmightuse2.5timesmorespacethaninitialLSFS
size.Additionallymetadataoccupiesupto20%ofinitialLSFSsizeresultingatotaloverprovisioningof
LSFSdevicesat200%.
6. Acontributionofmetadataisusuallysmall,butsomepatternscausehighmetadatagrowthanditmay
occupyasmuchspaceastheusefuldata.Theworstpatternis4krandomwrite.Nevertheless,whenthe
wholediskisfull,theratiobetweenmetadataandusefuldatashouldbestabilizedand3timesgrowthis
maximumpossible.Informationaboutusefuldatasize,metadataandfragmentationcanbefoundin
StarWindManagementConsolewhenLSFSdeviceisselected.
7. Nomatterwhichdataaccesspatternisused,underlyingstoragealwaysreceives4MBblocks.Read
patternsmayvarystartingfrom4kblocks.Accordingtomentionedaboveunderlyingstoragemaybea
cheapspinningdrive.
8. Ifavailablediskspaceonthephysicalstorageislow,theLSFSusesmoreaggressivedefragmentation
policyandslowsdownaccessspeedfortheenduser.Ifthereisnospaceonthephysicaldrive,thenLSFS
devicebecomesreadonly.
9. 4kistheblock,thatLSFSuses.Thismeansthatwritespeedwillbelowifwriterequestsarenotaligned
anddeduplicationwillnotbeperformed.Butitshouldntbetheproblemsinceallbrandnewdrivesuse4k
blocks.StartingfromWindowsVistaitispossibletousesuchdrives.HyperValignsVHDX,butVHDmight
notbealigned.ESXcannotworkwith4kdrives,butVMFSshouldbealignedaswell,thusthereisno
problem.

Limitsandrequirements
1. RequiredfreeRAM(itisnotrelatedtoL1cache):
4.6GBofRAMper1TBinitialLSFSsize(withdeduplicationdisabled)
7.6GBofRAMper1TBinitialLSFSsize(withdeduplicationenabled)
2. LSFSmaximumsizeis11TB
3. Overprovisioningis200%(LSFSfilescanoccupy3timesmorespacecomparedtoinitialLSFSsize).
Snapshotsrequirehavingadditionalspacetostorethem.

data:text/htmlcharset=utf8,%3Cheader%20class%3D%22entryheader%20pageheader%22%20style%3D%22boxsizing%3A%20borderbox%3B%20displa

1/2

10/4/2015

LSFSdevicetechnicaldescription|StarWindKnowledgeBase

4. ThephysicalblocksizeofLSFSdeviceis4k.512bor4karethesizesofvirtualblocks.Itisrecommended
touse4kblocksforMicrosoftWindowsEnvironmentsand512bblocksforVMwareESX.

Featuresandbenefits
1. Defragmentationworkscontinuouslyinabackground.Filesegmentwillbedefragmented,whendata
capacityisoutofallowedvalue.Datafromtheoldfilesegmentedwillbemovedtoanotheremptyfile
segmentandtheoldfragmentedfilewillbedeleted.Maximumallowedjunkratebeforedefragmentation
processis60%.ThisvaluecanbechangedusingthecontextmenuofthedeviceinStarWindManagement
Console.
2. Itispossibletorunfulldefragmentationviadevicecontextmenu(forstandalonedevicesonly).Manually
starteddefragmentationignoresdefaultdefragmentationrateandcleansupalljunkblocksinsidefile
segments.
3. HAbasedonLSFSusessnapshotsforsynchronizationpurposes.Snapshotsarecreatedevery5minutes
bydefaultandthenremoved.HAsynchronizesonlylatestchangesafteranyfailure,becauseeachHA
partnerhashealthysnapshotbeforethefailureandthereisnoneedtosynchronizeallthedatabutlatest
changesonlythatweremadeafterthesnapshotwascreated.Fullsynchronizationtakespartonlyafter
initialreplicacreation.Evenincaseoffullsynchronization,onlyusefuldataisreplicatedskippingjunkdata.
4. LSFScreatesrestorepointsduringausualjob.Thisisajournalpart,whichwillbeusedinthecaseof
failure.RestorepointscanbeobservedviaSnapshotManagerinDeviceRecoveryMode.
5. Deduplicationanalyzesuniquechunksofdataidentifyingandstoringthem.Astheanalysiscontinues,other
chunksarebeingcomparedtothestoredcopyandwheneveramatchoccurs,theredundantchunkis
replacedwithareferenceshortcutthatpointstothestoredchunk.Giventhatthesamebytepatternmay
occurdozens,hundreds,oreventhousandsoftimes(thematchingfrequencyisdependentonthechunk
size),theamountofdatatobestoredortransferredcanbegreatlyreduced.Forexample,similarVMsthat
resideontopofLSFSdevicewillbededuplicatedwell,butpagefileofeachVMwillnot.Accordingtoour
observationsdeduplicationisnotapplicabletopagefiles,thusthesizeofeverypagefilewillbeaddedas
uniquedata.Longstoryshort,10similarVMs12GBeach(2GBforpagefile)occupy(122)+2*10=30GB.

data:text/htmlcharset=utf8,%3Cheader%20class%3D%22entryheader%20pageheader%22%20style%3D%22boxsizing%3A%20borderbox%3B%20displa

2/2