Kernelbasis
OrhowtopileaKernelwithoutcausingtheBigCrunch
T.CastilloGironaDepartmentofAppliedPhysics,
U.P.C. Thisdocumentwillbeavailableathttps://dfa.upc.es/pub/bscw.cgi/d4651 Abstract WewilldiscusshowGNU/LinuxKernelcanpiledusingagenericmethod,tryingtoavoiddistribution-dependanttechniques,andhowtheKernelimplementsitsdevicedrivermodel.Thisway,wewillbeabletofigureoutwhatisgoingonbehindthescenes.Toconclude,thistalkwillfocusonKernelerrorslikeOOPSandPANICmessages. Contents 1WhatistheKernel?
3 1.1Kerneltypes.............................3 1.1.1MonolithicKernels......................4 1.1.2Micro-kernelKernels.....................4 1.2ExecutionRings...........................4 1.3PreemptiveKernels..........................4 1.4ReentrantKernels..........................4 1.5Kernelversions............................5 2CompilingtheLinuxKernel
5 2.1pilingprocess........................5 2.1.1GettingtheKernelsources.................5 2.1.2ConfiguringtheKernel....................5 2.1.3CompilingtheKernelImage.................5 2.1.4Compilingandinstallingthemodules...........6 2.1.5Theinitrdimage.......................6 2.1.6Installingthenewkernel’simage..............6 2.2SavingandrestoringtheKernel’sconfiguration..........6 2.3Initrd’sentrails............................7 2.3.1Initrdexample:readingsomeargs.............8 2.4PatchingtheGNU/LinuxKernel’ssources.............9 3TheLinuxDeviceDrivermodel 10 3.1AboutModules............................10 3.1.1Dealingwithmodules....................11 3.1.2Loadingamodule......................11 3.1.3Unloadingamodule.....................12
1 3.2Devicedrivers.............................123.2.1Characterdevices.......................123.2.2Blockdevices.........................123.2.3Majorandminornumbers..................13 4ExportedSymbols 13 4.1Whatisasymbol?
..........................13 4.2ApeekinsideanLKMobjectfile..................14 4.3Unknownsymbols..........................15 5DealingwithKernelOOPSandKernelPANICmessages 15 5.1KernelOOPS.............................16 5.1.1AKernelOOPScase-study.................16 5.2KernelPANIC............................17 5.3Overridingtheinitprogram.....................18
2 1WhatistheKernel?
Figure1:TheGNU/LinuxMicro-kernelTheGNU/LinuxKernelisaprograminchargeofdealingdirectlywithputer’shardware,assigningI/Oresourcestoprocesses,schedulingtheseones,amongmanyotherlow-leveltasks.Inasimpleway,itcouldbeconsideredasabridgebetweenuser-landandhardware-land. 1.1Kerneltypes Basically,wecansplitallexistingKernelsintwomainfamilies:theMonolithiconesandtheMicro-kernelones.
3 1.1.1MonolithicKernels AlllayersarestoredinsidetheKernelprogram,loadedentirelyinthememory,runninginKernelMode.Bearinmindthattheunnecessaryonesareloaded,too,andthatcanbeinefficient. 1.1.2Micro-kernelKernels Thesekernelsposedofdifferentlayers,eachoneinchargeofsomespecifictask.Anylayercanbeunloadedfrommemory,savingresourceswhentheyaren’tnecessaryanymore.Obviously,anylayercanusesomeexportedsymbols(discussedbelow)fromotherones,withnoneedtore-implementthem. TheGNU/LinuxKernelbelongstotheseones. 1.2ExecutionRings Normally,anyCPUhasaminimumoftwomainexecutionmodes,orexecutionrings.Thefirstoneiscalleduser-mode,user-spaceor,inIntelarchitectures,Ring3.Thelastoneisknownaskernel-mode,kernel-spaceorsimplyRing0. Inthefirstone,there’snodirectesstohardwareresources.Aslongasacertainuser-spaceprocessneedstogainsomeesstoaharddisk,tothemainmemory,andsoon,itneedstocallsomekernel’sroutine(so-calledsystemcalls)inordertojumpfromRing3toRing0. Clearly,theGNU/LinuxKernelprogramrunsalwaysinRing0. KernelThreads Therearesomespecialkindofprocesses,well-knownasKernelThreads,whichrunalwaysinRing0.Generally,theyarecreatedatsystemstartupandremaintheirexecutionuntilthesystemhalts. TheycanbefoundonanyGNU/Linuxsystembyrunningthepsmand.KernelThreadsappearbetweenbrackets: ...root210Mar08?
00:00:00[migration/0]root310Mar08?
00:00:00[ksoftirqd/0]root1110Mar08?
00:00:00[kthread]... 1.3PreemptiveKernels TheearlierGNU/LinuxKernelversionswerenotpreemptive,thatis,aslongasacertainprocesspwasrunninginRing0,therewasnopossibilitytosuspenditsexecutionuntilitreturnedtoRing3. CurrentversionsoftheGNU/LinuxKernelcanplishthat(finally).Thus,asfarasweareconcerned,Linuxisapreemptivekernel. 1.4ReentrantKernels AreentrantKernelallowsmorethanoneprocessrunninginsideRing0executionmode.Obviously,ifthereisonlyoneprocessor,therecouldbealotofsuspendedprocesseswaitinginsideRing0.
4 TheGNU/LinuxKernelisreentrant,also. 1.5Kernelversions 2.41.182.52.2332.6.284.55 2CompilingtheLinuxKernel TherearealotofmethodsinorderpileanewLinuxKernel’simage.Someofthemaredistribution-dependant.Wewillconsiderthestandardone,usefulinallGNU/Linux’sdistributions. 2.1pilingprocess Thepilationprocessconsistsofthesixsteps,discussedbelow: 2.1.1GettingtheKernelsources Normally,allGNU/LinuxdistributionshavetheirownLinuxsourcecodeasameta-package.Forexample,Debianhasthepackagelinux-source-
2.X.Yavailableonitssourcerepositories. Thus,wecaninstalleithertheGNU/LinuxKernelincludedinthedistrorepositoriesorthe”official”oneavailableat: /pub/linux/kernel/ 2.1.2ConfiguringtheKernel InordertoconfiguretheGNU/LinuxKernel,onecanchoosebetweendifferent”make”methods.TheeasiestoneisbyusingtheN-curseslibraries.It’squickly,canbeexecutedinavirtualterminalacrossremotesecureshellconnectionswithnolatencies,andithasitsownUserInterface(UI).Thiscanbedonebytypping:#cd/usr/src/linux-2.6.XX.Y#makemenuconfig 2.1.3CompilingtheKernelImage JustafterchoosingthedesiredKerneloptions,itistimepilethenewKernel’simage.Todothis:#make[-jN]6bzImage ThenewGNU/LinuxKernel’simagewillbeplacedatplatform-dependantdirectoryarch,i.e:arch/x8664/boot/bzImageintheparticularcaseofEM64Tarchitectures,andarch/i386/boot/bzImageintheX86ones. 1Even:stableversion2Odd:unstable,underdevelopment3Developmentrelease4Stablerelease5Stablepatch6Ncouldbe2..n,wherenisthenumberofprocessorspresentontheLinuxbox
5 2.1.4Compilingandinstallingthemodules Tobuildalloptionsmarkedasmodules(M)inthefirststep,simplyexecute:#make[-jN]modules It’snecessarytoinstallallofthemintherightplace.Todothisrun:#makemodulesinstall Afterawhile,allmoduleswillbecopiedto/lib/modules/2.X.YY/ 2.1.5Theinitrdimage SometimestheGNU/LinuxKernelisn’tcapableofbootingbyitselfbecauseofsomenon-loadeddrivers-suchasharddiskcontrollers,file-systems,andsoon-.So,itisinneedofaninitrdimage,which,inturn,isinchargeofdoingsomeprevioustasksinordertohelpitsstartup. Therearetwomainfile-systemsdesignedtocreateanewinitrdimagefile:cramfsandext2.Wecanchooseeither,itdoesn’tmatter.Inaddition,wecanuseag-zippedcpiofileformat. Generally,initrdcreationcanbeachievedbyusingascriptnamedmkinitrdormkinitramfs.IntheparticularcaseofGNU/LinuxDebiandistributions,wecancreateourowninitrdimagefilebasedonthepiledkernelbyrunning:#mkinitramfs-o/boot/initrd-
2.X.Y.Z2.X.Y.Z 2.1.6Installingthenewkernel’simage ThelaststepconsistsoninstallingthenewKernel’simageto/bootdirectory,namingitordinglyandupdatingtheBootLoader7-ifneeded-.#cparch/platform/boot/bzImage/boot/vmlinuz-myVersion#vim/boot/grub/menu.lst...#vim/etc/lilo.conf... Don’tettoaddanewinitrdentryifyouareusingone.LetustakealookattheFigure2-GRUBloaderexample-andFigure3-liloloaderexample-. titleDebianEtch2.6.25.6PreemptiveVoluntaryticknesslessroot(hd0,0)kernel/vmlinuz-2.6.25.6-kproot=/dev/md2initrd/initrd-2.6.25.6-kp Figure2:/boot/grub/menu.lstconfigurationfile 2.2SavingandrestoringtheKernel’sconfiguration Obviously,theremustbeasimplewayforrecoveringorsavingtheGNU/LinuxKernel’sconfigurationselectedinthefirststep.AllGNU/Linuxdistrosstoretheirconfigurationconcerningtherunningkernelinafileplacedat/boot/config-
2.X.Y. 7BearinmindthatincaseofusingLILO,youhavetorunmandlilorightaftermodifyingoraddingentriesin/etc/lilo.confconfigurationfile.
6 image=/boot/vmlinuz-2.6.25.6-kpinitrd=/boot/initrd-2.6.25.6-kproot=/dev/md2label=2.6.25.6-nokpread-only Figure3:Thesameconfigurationbutinthe/etc/lilo.conffileNowadays,modernkernelscanplishthisbyenablingCONFIGIKCONFIGsymbol,aswell.Thus,wecanrecovertheentireKernel’soptionsbytypinginashell:#zcat/proc/config.gz>/usr/src/linux-
2.X.Y.Z/.configOrbyusingtheextractconfigscript,availableinsidetheGNU/LinuxKernelsources:#cd/usr/src/linux-
2.X.Y.Z/#scripts/extractconfig/boot/vmlinuz-
2.X.Y.Z>.config Figure4:EnablingKernel’sconfigthrough/proc/config.gzinterface 2.3Initrd’sentrails Itisfeasibletomodifyanexistinginitrdimagefilewithoutcreatinganewonefromscratch.Sometimes,weneedtoaddanewmoduletotheinitrdimagefiledirectly,ormodifysomemodule’sparameters. Asweknow,aninitrdimagefileitisbasedonext2,cramfsfile-systemsorcpiofileformat.Thus,wecanmount,copy,modify,addordeletefilesinside
7 theinitrdimagefile.Letussupposethatwehaveaninitrdimagefilecalled/boot/initrd.img- 2.6.18-test.Firstofall,wemustcheckitsformat: #file/boot/initrd.img-2.6.18-test/boot/initrd.img-2.6.18-test:presseddata... Inthiscase,theinitrdimagefilepressedusinggzip.So,obviously,wehavetog-unzipit: #gunzip-f-S""-c/boot/initrd.img-2.6.18-test>/boot/initrd.img-2.6.18-test-gunzipped #file/boot/initrd.img-2.6.18-test-gunzipped/boot/initrd.img-2.6.18-test-gunzipped:ASCIIcpioarchive... #mkdir/tmp/initrd#cd/tmp/initrd#cat/boot/initrd.img-2.6.18-test-gunzipped|cpio-id 20505blocks#ls-l binconfetcinitlibmodulessbinscripts Rightafterg-unzippingandrecoveringtheentiredirectorystructurefromcpiofile,wecanadd,deleteormodifyanyexistingfileinside/tmp/initrddirectory.Allmodulesarestoredinside/lib/modules/2.X.Y.Zdirectory. TheinitscriptwillbeexecutedassoonastheinitrdimagefileispressedatstartuptimebytheGNU/LinuxKernel.Intheparticularcaseofext2initrds,insteadof”init”scriptfilewecanfindoutanotherequivalent,usuallynamedaslinuxrc. Oncetheinitrdstructurehasbeenmodifieditisnecessarytogobackwardsintheformerprocess,thatis,createthenewcpiofile,g-zippingitandcopyingittotherightplace. #find./|cpio-Hnewc-o>/tmp/new-initrd.cpio20505blocks #gzip-c/tmp/new-initrd.cpio>/boot/initrd.img-2.6.18-test Atthisstep,thenewinitrdimagefilehasbeenupdated,anditcanbeusedimmediately.Afterusingit,theGNU/LinuxKernelfreesallallocatedmemory,andsoitshowsthroughksyslog:Freeinginitrdmemory:4405kfreed 2.3.1Initrdexample:readingsomeargs Inthissubsectionwearegoingtochangetheinitrdbehaviour:wewantreadacertainparameterwhichwillbeappliedonlytoadesiredkernelmodule,called”mymodule”.Obviously,thismodulewon’tbeloadedbecauseitdoesn’texist. Firstofall,letustakealookattheinitscript,justbeforetheshellfunctionloadmodules:
8 ...echo-ne"TCG:please,insertthemoduleparametersformymodule:\>"readmargloadmodules$marg...Clearly,wehaveaddedsomecodetothescriptinordertopassanextraparametertotheshellfunctionloadmodules,readingitusingreadshell-statement. monfunctionsarelocatedinsidescripts/directory.Theshellfunctionloadmodulesisimplementedinscript/functionsfile.Thus,wemustchangesomeaspectsofit: ...if["$m"="mymodule"];then echo-ne"\tPersonalisedmodule$mwitharg$marg\n"Dosometasks...elsemodprobe-q$mfi... Toconclude,wehavetoaddmymoduletoconf/modules:#echo"mymodule">>conf/modules Afterre-generatingonceagaintheinitrdg-zippedcpiofile,thenewinitrdisreadytouse.So,nexttimethesystembootsup,wewillseewhatisshowninFigure5. Figure5:initrdalteredbehaviour 2.4PatchingtheGNU/LinuxKernel’ssources WhentherunningKernelmustbeupgraded,itisfeasibletopatchthecurrentsourcesandpilingthemasdescribedearlier.Todothisweneedtogetthepatch,thenapplyittothecurrentkernelsourcetree.Forexample,inordertoupgradeourKernelfrom2.6.18.1versionto2.6.18.2:#wget/pub/linux/kernel/v2.6/patch-2.6.18.2.bz2 Then,wehavetoapplythepatch:#cd/usr/src/linux-2.6.18.1#bzcat/path/to/patch-2.6.18.2.bz2|patch-p1-r/tmp/rejs.rej...patchingfilesound/core/hwdep.cpatchingfilesound/core/info.c
9 ...patchingfilesound/pci/emu10k1/emu10k1main.c Ifsomethinggoeswrong,wecantakealookat/tmp/rejs.rejfileinordertodeterminewhichfilescouldn’tbepatched. Beforetryingtopilethenewpatchedkernelmainstream,wehavetocleanupanyobjectfilesrunning:#makemrproper 3TheLinuxDeviceDrivermodel LookingbackatFigure1,theGNU/LinuxKernelposedofmanydifferentlayers,eachoftheminchargeofsomeconcretetasks.Someoftheselayerscanpiledasmodules,andsomeothersnot.Usually,pilingtheGNU/LinuxKernelwecanchoosewhenalayerwillbeamoduleornot.Bearinmindanynon-modulelayerwillbelinkedandinsertedphysicallyinsidetheKernel’simagefile.Asaresult,thenewKernelwillupymorememory.Clearly,theimage’sresultantfilewillbelargerinsizethananotheronewithmorepiledasmodules. TalkingaboutTheGNU/LinuxDeviceDriverModelintroducestheconceptofLinuxKernelModule(LKM). 3.1AboutModules It’sjustanELFobjectfile,notlinked,storedontheharddrive.Inmodernkernels,thesefilesarerecognisedbecauseofitsextension,ko.Intheolder2.4xKernel’sbranch,theyhad*.oextension.Thesefilescanbedriversornot.Inotherwords,notallmodulesaredrivers.Forexample,therearesomeLKMswhichallowuserstomountafile-system,likevfat.koorext3.ko.Asamatteroffact,theyaren’tdriversatall. InallGNU/Linuxkernels,piledmodulesaresavedin/lib/modules/2.X.Y.Zdirectory. Inorderpilesomelayerasamodule,wehavetoselectMinthemakemenuconfigscreen,asshowninFigure6. Figure6:CompilingsomelayersasLKMs 10 3.1.1Dealingwithmodules InordertodealwiththeLKMs,theGNU/LinuxKerneloffersagroupoftools,well-knownasthemodutils.Withthesesortmands,wecanload,unloadorjustlistwhichmodulesareloaded.Infact,allloadedmodulesarelinkedinsidetheGNU/Linuxkernelprogram.Bearinmindthatthislinkprocedurewilltakeplaceassoonasanewmoduleis”loaded”usingthemodutils,andunlinkedassoonasitisunloaded. 3.1.2Loadingamodule Therearetwomethodsforloadingmodules:#insmod/path/to/module.koarg1=value8...argN=valueNor#modprobemodulearg1=value...argN=valueN Thereareint,short,long,char*,int[],short[],long[]andchar**parameters.Forexample,inordertopassthreeparameterstoamodulecalledmymodule.ko,twoofthemanintvalueandthelastoneanarrayofintegers,wemustrunsomethinglike:#modprobemymoduleio=0x220irq=5dma=1,
5 Generally,inanyGNU/Linuxdistributionthere’safileinordertoconfigurewhatmoduleswillbeloadedautomaticallywiththeirownparameters.Inaddition,theGNU/LinuxKerneltriestoauto-loadsomemodulesassoonastheyareneeded,nomatterwhatmodulesarewrittendowninthemodule’sconfigurationfile.Thisoptionmustbeexplicitlyconfiguredinthemakemenuconfigscreen,asshowninFigure7. Figure7:Enablingkernelmodulesauto-loadingoption Beforeloadingthedesiredmodule,allmodulesneededtoloaditperfectlymustbelinkedintheruntimeKernel.Incaseofusinginsmod,itcouldbedifficulttorealizewhatmoduleswouldbeloadedpreviouslytoavoidunknownsymbolmessages-seebelow-.Thus,weneedtousemodprobe.Thisprogramtriestoloadalldependencieslookingat/lib/modules/2.X.Y.Z/modules.depfile,loadingthemjustbeforeloadingthegivenmodule. Inparticular,themodule”msdos.ko”,inchargeofallowingesstomsdosfile-systems,needsthesymbolsexportedbythemodulefat.kowhich,inturn,hasnodependencies: /lib/modules/2.6.18-kmodest/kernel/fs/msdos/msdos.ko:/lib/modules/2.6.18-kmodest/kernel/fs/fat/fat.ko .../lib/modules/2.6.18-kmodest/kernel/fs/fat/fat.ko: 8Todeterminetheallowedmodule’sparameters,run#modinfomodule. 11 Modules.depisgeneratedandupdatedafteranycalltomakemodulesinstall,asdiscussedearlier.mandneededtore-generateitbyhandis:#depmod-ae 3.1.3Unloadingamodule Tounloadamodulewehavetousemand:#rmmodmodule Sometimes,itcouldbenotpossibletounloadamoduleusingrmmodbecauseofitsreferencecounter.Clearly,somemoduleshavedependencies,andtheseonescannotbeunloadedaslongasothersdependantsmodulesareloadedandinuse.Themandreportsusifacertainmoduleisbeingused:ModuleSizeUsedbycdrom325441idecd ThereferencecounterofacertainLKMismaintainedinternallybytheKernel,but,sometimes,anLKMdevelopercanprefertowritesomecodeforcontrollingitdirectly,thankstotrymoduleget()andmoduleput()Cfunctions. Inaddition,theGNU/LinuxKernelcanpiledwithouttheMODULEUNLOADsymbol:inthiscase,unloadingamoduleisnotallowed. 3.2Devicedrivers Inshort,therearethreemaindevicedriverstypes:
1.Characterdevicedrivers.
2.Blockdevicedrivers.
3.Networkinterfacecards. Infact,allofthembutNICshavetheirassociatedspecialdevicefileunder/devdirectory,controlledbyUDEVsubsysteminmodernGNU/LinuxKernelimplementations.IntheparticularcaseofNICs,there’sno/deventrybecauseoftheirownimplementationissofarawayfromthemainideaofstreaming,likecharacterorblockdevicesare. 3.2.1Characterdevices Allchardevicedriversareessedwithnobuffers,andtheyallowtoreadorwriteanynumberofbytesperread/writeess.Generally,thisdevicedriversimplementhooksforthecallsopen(),close(),lseek(),read(),write(),tl(),ioctl()... Insidethe/devdirectory,thesedevicesarerecognisedbythecharacter”c”:crw-------1rootroot4,1Mar1308:54/dev/tty1crwxrwxrwx1rootroot1,3Jan262006/dev/nullcrw-rw-rw-1rootroot1,5Jan262006/dev/zero 3.2.2Blockdevices Theyareassociatedtoharddrives,harddiskcontrollers,etcetera.Thisparticularcaseofdevicedrivesworkalwaysusingintermediatebuffers,andthereadsorwritesarealwaysdonebybyte-blocks.Internally,therearealotof 12 differencesbut,asfarastheuserisconcerned,eithercharorblockdevicesseemquitesimilar. Theycanberecognisedinside/devdirectorybythecharacter”b”:brw-rw----1rootdisk8,0Feb1708:11/dev/sdabrw-rw----1rootdisk8,1Jan312005/dev/sda1 3.2.3Majorandminornumbers TheGNU/Linuxanisesalldevicedriversthroughmajorandminornumbers.Asamatteroffact,thisisthewayforidentifyingaparticulardevicedriverfromaKernel’spointofview.Thefilenameinside/devdirectorycanbechanged,obviously. Themajornumbermeanstheclassorgroupforthisparticulardevicedriver,suchasaHarddisk.Thankstoit,theGNU/LinuxKernel,assoonasanopen()systemcalliscalledbyarunningprocess,knowswhichdriverwillbeabletodispatchit. Forexample,theSATAharddisksdabelongstofamilywithmajornumberof8:brw-rw----1rootdisk8,0Mar1308:43/dev/sda Theminornumberisdirectlyrelatedtothemajorone.AllowstotheGNU/LinuxKerneltofine-tunewhichparticulardevicewillrespondtothecallforthatparticulardevicedriverfamily.Forexample,inthecaseofaharddisk,itcouldbethefirstdisk-partition,sucha/dev/sda1:brw-rw----1rootdisk8,0Mar1308:43/dev/sda1brw-rw----1rootfloppy8,33Mar1308:43/dev/sdc1 4ExportedSymbols TheGNU/LinuxKernelbelongstoMicrokernelfamily.So,somelayerscanbestackedofotherones,usingsomefunctionspreviouslyimplementedinlowerlayers.Thankstoit,anLKMdeveloper-orjustaKerneldeveloper-canre-usesomewell-knowntestedcodeorfunctionsinsidetheKernel.Clearly,thismatterpresentsareallayer-dependancyissue.AssoonassomeKernellayerneedssomeaidsimplemented(offered)byanotherone,theremustexistatotalagreementbetweenthesefunctionalitiesinordertoavoiderroneousbehaviours,likeKernelOOPS,unknownsymbolmessages,andsoon(seebelow). 4.1Whatisasymbol?
Asymbolcanbeafunction,avariable,adatastructure,etc.Forexample,dolookupfunction-analysedinsectionKernelOOPS-,isasymbol.Thissymbolisanexportedone,too.Thatis,anyexternallayerfromwhereitisdefined,canuseit. Inordertocheckifthissymbolisreallyexported,wecanread/proc/kallsymsfile.ThisfilestoresallKernel’sexportedsymboltable,placedinmemorybetweenstartksymtabandksymtab.Thefield’sformatislikenm’soutput.#cat/proc/kallsyms|grepdolookupc0164a7ftdolookup 13 ...c0102b7ftcheckuserspacec0102b94Tresumeuserspacec0102bacTsysenterentryc0102bb3tsysenterpastespc0102c2cTsystemcallc0102c55tnosinglestepc0102c6ctsyscallcallc0102c77tsyscallexitc0102c86trestoreallc0102c9etrestorenocheckc0102c9etrestorenochecknotrace... Figure8:TheKernel’sexportedsymboltablecutshort Yes,itexists!
Wecanfinditatkernel’saddress0xc0164a7f,andthe”t”fielddenotesitisalocalsymbolstoredinthetextsegment. 4.2ApeekinsideanLKMobjectfile Allmoduleshavetheirownsymbols,ofcourse.Thesesymbolscanbeexportedornot,dependingonthedesignortheconceptinwiththismoduleisinvolved.Supposewehaveamodule,saykmodest.ko,withalotoffunctions,variables,datastructuresandsoon.pilingitandobtainingthemodule’sobjectfile,wecangetalistofitssymbolsbyrunningnm:#nmkmodest.ko00000004Bbytestoread00000000TcleanupmoduleUcopyfromuserUcopytouser00000000DcurrentcmdUdofsync00000557Tdoinsertfd000003b2Tdoremapfd0000043cTdorestoretask0000028cTfddeinstallbytask000002f6Tfdinstallbytask00000020Bfdsinfo0000003dTfillfileinformation... Asshownintheprevioussymbollist,wecanrealizethatsomesymbolsareundefined(U).Thesesymbolsmustbeexportedinordertousethismodule.Thus,thecopyfromuser,copytouseranddofsyncsymbolswillbecalledfromthismodule.IftheseoneswillnotavailableatKernel’sexportedsymboltable,thismodulecannotbeloadedthroughinsmodormodprobeatall. 14 4.3Unknownsymbols Theundefinedsymbolscaneasunknownones.Forexample,inthisparticularcase: #insmod./kmodest.koinsmod:errorinserting’./kmodest.ko’: #dmesg|tail-3kmodest:Unknownsymboldofsynckmodest:Unknownsymbolsyskillkmodest:Unknownsymbolgetfilesstruct -1Unknownsymbolinmodule Whenthereareunknownsymbolsreportedbyinsmod,wecanfixthemprobablyloadingapreviousmoduleusingmodprobe.Sometimes,either,weneedtopatchtheKernelsourcesinordertoexportthesesymbolsexplicitly.Forexample,inordertosolvethepreviousissue,weneedthecurrentKernelsources.Withcscope9wecanfindoutwherethisnon-exportedfunctionsareandexportthembyaddingthemacroEXPORTSYMBOL. Finally,wehavetopilethekerneland,sometimes,rebootthesysteminordertoloadthenewone.Wecantakealookat/proc/kallsymsalways,inordertotestifacertainsymbolhasbeenexported.BearinmindthatsomesymbolswillbeexportedassoonasacertainchainofmoduleswillbelinkedintotheruntimeKernelviamodprobe. ...asmlinkagelongsyskill(intpid,intsig) ......}EXPORTSYMBOL(syskill); Figure9:Exportingsyskillsymbolbyhand 5DealingwithKernelOOPSandKernelPANICmessages WhenaKernelOOPSmessageappears,itispossibletocontinueusingthesystem.Butwhenacriticalerrorhappens,theKernelPANICmessageappears,hangingupthepletely.Dependingonitsconfiguration,thisKernelPanicmessagecanbefollowed,aftersomeseconds,inasystemreboot. 9AtooldesignedforbrowsinginsideCsourcecodefiles,suchasGNU/LinuxKernel.Itisfeasibletorun,insidetheGNU/LinuxKernelsourcedirectory,mand:#makecscopeAfterawhile,wecanbrowsethesourcesinfortablewaybyrunning:#cscope-k 15 (...)Pid:3378,m:findNottainted(2.6.24-etchnhalf.1-686#1)EIP:0060:[]EFLAGS:00000286CPU:1EIPisatdlookup+0xbe/0xd9EAX:dbc649e4EBX:dbc649d4ECX:00000011EDX:c17e2100ESI:efbcdf04EDI:dbc649e4EBP:d9b347c8ESP:efbcddc4(...)
[]dolookup+0x24/0x14e[]linkpathwalk+0x73f/0xb46[]linkpathwalk+0x44/0xb3[]dopathlookup+0x162/0x1c4[]getname+0x59/0xad[]userwalkfd+0x2f/0x40[]vfslstatfd+0x16/0x3d[]syslstat64+0xf/0x23[]syscallcall+0x7/0xb[]skbicvwalk+0x21e/0x262(...)
Figure10:KernelOOPSindlookup()function
5.1KernelOOPS
Thiskindoferrormessagesappearassoonasthere’saNULLde-referencepointer.Generally,allinformationreportedbytheGNU/LinuxKernelcanbeanalysedinordertomakeoutwheretheproblemcausingthisOOPSis.Alldataiswritteninthelogfilesthroughsyslog,andthismessageisprintedoutthankstoprintk()Kernel’sfunction-sosimilartoprintfinuser-landdevelopment-.
Normally,itisfeasibletocontinueusingthesystem.But,sometimes,theOOPSmessagepointstoahardwareerror,likecorruptedmemoryDIMMs.Obviously,theoffendingprocessiskilledimmediately.
5.1.1AKernelOOPScase-study
Let’stakealookattheKernelOOPSscreeninFigure10,showinguparealissueurredtoaLinuxBoxrunningDebianEtch”and-a-half”,X86architecture.
Despitethefacttherearealotof”strange”messagesinhere,themostimportantonesare,basically,theEIPregister,theoffendingprocessandthecall-trace.So,firstofall,wehavetowritedownthisbasicinformationgatheredfromtheKernelOOPSmessage:Offendingprocess:fimand,PID3378.Currentinstructionexecuted:dlookup(),atoffset0xbe.Call-trace:lookattheFigure.
Well,itmakessense,doesn’tit?
The”find”mand,withPID3378,firesaKernelOOPSmessagewhilerunninginRing0thedolookup()function,justinside0xbeoffset.Checkingoutthecall-trace,itseemssorelatedtothe”find”mandexecuted:allKernelfunctionsexecutedherehavebeendesignedforwalkinginsideafile-systemthroughtheVFS(VirtualFile-Systemlayer,shown 16 inFigure1).Generally,afimanddoesnotfiresaKernelOOPSmessage.It’squite strange,soprobablythefimandisn’treallyinvolvedinthisawfulmatteratall.ItseemstousmorerelatedtosomeprobleminsidetheGNU/LinuxKernel,maybe. Thus,weneedtoanalysethedolookupfunction.WeneedthecurrentKernelsourcesinordertodothat,ofcourse!
Usingcscope,wefindoutthisfunctiondeclaredininclude/linux/dcache.hheaderfile.Hereisitssignature:externstructdentry*dlookup(structdentry*,structqstr*); AdentrystructurestoresallaboutafileordirectoryinsidethefilesystemwalkedbytheVFS.Thisfunctionsimplylooksforthedentry’sparenttransferedasaparameterandreturnsapointerincaseoffindingit.TheGNU/LinuxKernelalwaysworkswithdirectoriesorfilesusingdentryobjects,directlymappedtothememory,inordertoimprovetime-consumption.Thesekindofdentryobjectsarewell-knownasdcachetypedefs. Inshort,weknowthatfimand,whilesearchingforasomeparentdirectory,firedaKernelOOPSmessage.Howwasitpossible?
Clearly,becauseofsomephysicalerrormemory.Ifalldcache/dentryobjectsarekeepinmemoryforimprovingsystemtroughtput,andtheKernelOOPSwassaying:Nottainted,thentherewasaninvalidmemoryess.Why?
ItcouldbeaBUG.Whynot?
But,asamatteroffact,thesystemwasrunningperfectlyforalongtime,so...theanswershouldberelatedtophysicalDIMMmemoryerrors.Theactiontakeninordertodeterminetherealmemoryerrorcausewasrunningatrivialmemorytester(memtest86+).Afterabit,thefirstmemoryerrorsappeared.Whatarelief;-)!
!
5.2KernelPANIC AKernelPANICmessagemeansthere’snothingthesystemcandoinordertorecovertheminimalstabilityandisinneedofanurgentreboot.Sometimesitisinneedofpleteshutdownofanyattachedhardware,too-suchasanexternalUltra-SCSIdiskcontroller-. WhenthismessageappearsalongtheruntimelifeoftheGNU/LinuxKernelprogram,there’ssomehardwarerelatederrororanundocumentedBUG.Whenitursatboottime,itcanberelatedtoanotlinkeddriver-suchasahci.koSATAcontroller,oranotknownfile-systemduetosomemissingmodule.- InordertorebootthesystemafternsecondsjustafterappearingtheKernelPANICmessage,itisnecessarytoconfigurethistimeoutin/etc/sysctl.conf:kernel.panic=10Itisfeasibletochangethisbehaviouratruntimetyping:#echon10>/proc/sys/kernel/panic Otherwise,thesystemwillremainpoweredonandhangedupforever. Figure11:NotloadedHarddiskcontroller 10A0valuemeansnoreboot 17 Figure12:Notfile-systemsupportformountingtherootpartition 5.3Overridingtheinitprogram Sometimesweneedtorestoreroot’spasswordwehaveotten.Todothis,wemustpassaparametertotheGNU/LinuxKernelthroughtheBootLoader:init=/bin/bash.Assoonaswecanesstothebashprompt,wemustremounttheslashpartitioninordertoallowwriteess:mount/-oremount,rw.Finally,wecanusethemandasusual. 18 Glossary cpio Afile-formatsimilartotar,allfilespackedinone.,
6 EM64Tarchitecture 64bitarchitectures,IntelCoreprocessors.,
5 GNU/Linuxdistro(s) Arecopilationoftools,softwareandaGNU/LinuxKernelinonepackage,
5 KernelImage TheentireGNU/LinuxKernelprogram,gzipped.Itwillbepressedatruntime,inmemory.,
5 layer Asub-systeminchargeofdoingsometasks.Forexample,worklayer.,
3 N-Curseslibraries LibrariesdesignedforbuildingapplicationsrunninginsideTerminalswithmoreorlessGraphicalUserInterfacecapabilities.,
5 X86architectures(a.k.ai386)32bitarchitectures,AMDorIntelprocessors.,
5 19
T.CastilloGirona
U.P.C. Thisdocumentwillbeavailableathttps://dfa.upc.es/pub/bscw.cgi/d4651 Abstract WewilldiscusshowGNU/LinuxKernelcanpiledusingagenericmethod,tryingtoavoiddistribution-dependanttechniques,andhowtheKernelimplementsitsdevicedrivermodel.Thisway,wewillbeabletofigureoutwhatisgoingonbehindthescenes.Toconclude,thistalkwillfocusonKernelerrorslikeOOPSandPANICmessages. Contents 1WhatistheKernel?
3 1.1Kerneltypes.............................3 1.1.1MonolithicKernels......................4 1.1.2Micro-kernelKernels.....................4 1.2ExecutionRings...........................4 1.3PreemptiveKernels..........................4 1.4ReentrantKernels..........................4 1.5Kernelversions............................5 2CompilingtheLinuxKernel
5 2.1pilingprocess........................5 2.1.1GettingtheKernelsources.................5 2.1.2ConfiguringtheKernel....................5 2.1.3CompilingtheKernelImage.................5 2.1.4Compilingandinstallingthemodules...........6 2.1.5Theinitrdimage.......................6 2.1.6Installingthenewkernel’simage..............6 2.2SavingandrestoringtheKernel’sconfiguration..........6 2.3Initrd’sentrails............................7 2.3.1Initrdexample:readingsomeargs.............8 2.4PatchingtheGNU/LinuxKernel’ssources.............9 3TheLinuxDeviceDrivermodel 10 3.1AboutModules............................10 3.1.1Dealingwithmodules....................11 3.1.2Loadingamodule......................11 3.1.3Unloadingamodule.....................12
1 3.2Devicedrivers.............................123.2.1Characterdevices.......................123.2.2Blockdevices.........................123.2.3Majorandminornumbers..................13 4ExportedSymbols 13 4.1Whatisasymbol?
..........................13 4.2ApeekinsideanLKMobjectfile..................14 4.3Unknownsymbols..........................15 5DealingwithKernelOOPSandKernelPANICmessages 15 5.1KernelOOPS.............................16 5.1.1AKernelOOPScase-study.................16 5.2KernelPANIC............................17 5.3Overridingtheinitprogram.....................18
2 1WhatistheKernel?
Figure1:TheGNU/LinuxMicro-kernelTheGNU/LinuxKernelisaprograminchargeofdealingdirectlywithputer’shardware,assigningI/Oresourcestoprocesses,schedulingtheseones,amongmanyotherlow-leveltasks.Inasimpleway,itcouldbeconsideredasabridgebetweenuser-landandhardware-land. 1.1Kerneltypes Basically,wecansplitallexistingKernelsintwomainfamilies:theMonolithiconesandtheMicro-kernelones.
3 1.1.1MonolithicKernels AlllayersarestoredinsidetheKernelprogram,loadedentirelyinthememory,runninginKernelMode.Bearinmindthattheunnecessaryonesareloaded,too,andthatcanbeinefficient. 1.1.2Micro-kernelKernels Thesekernelsposedofdifferentlayers,eachoneinchargeofsomespecifictask.Anylayercanbeunloadedfrommemory,savingresourceswhentheyaren’tnecessaryanymore.Obviously,anylayercanusesomeexportedsymbols(discussedbelow)fromotherones,withnoneedtore-implementthem. TheGNU/LinuxKernelbelongstotheseones. 1.2ExecutionRings Normally,anyCPUhasaminimumoftwomainexecutionmodes,orexecutionrings.Thefirstoneiscalleduser-mode,user-spaceor,inIntelarchitectures,Ring3.Thelastoneisknownaskernel-mode,kernel-spaceorsimplyRing0. Inthefirstone,there’snodirectesstohardwareresources.Aslongasacertainuser-spaceprocessneedstogainsomeesstoaharddisk,tothemainmemory,andsoon,itneedstocallsomekernel’sroutine(so-calledsystemcalls)inordertojumpfromRing3toRing0. Clearly,theGNU/LinuxKernelprogramrunsalwaysinRing0. KernelThreads Therearesomespecialkindofprocesses,well-knownasKernelThreads,whichrunalwaysinRing0.Generally,theyarecreatedatsystemstartupandremaintheirexecutionuntilthesystemhalts. TheycanbefoundonanyGNU/Linuxsystembyrunningthepsmand.KernelThreadsappearbetweenbrackets: ...root210Mar08?
00:00:00[migration/0]root310Mar08?
00:00:00[ksoftirqd/0]root1110Mar08?
00:00:00[kthread]... 1.3PreemptiveKernels TheearlierGNU/LinuxKernelversionswerenotpreemptive,thatis,aslongasacertainprocesspwasrunninginRing0,therewasnopossibilitytosuspenditsexecutionuntilitreturnedtoRing3. CurrentversionsoftheGNU/LinuxKernelcanplishthat(finally).Thus,asfarasweareconcerned,Linuxisapreemptivekernel. 1.4ReentrantKernels AreentrantKernelallowsmorethanoneprocessrunninginsideRing0executionmode.Obviously,ifthereisonlyoneprocessor,therecouldbealotofsuspendedprocesseswaitinginsideRing0.
4 TheGNU/LinuxKernelisreentrant,also. 1.5Kernelversions 2.41.182.52.2332.6.284.55 2CompilingtheLinuxKernel TherearealotofmethodsinorderpileanewLinuxKernel’simage.Someofthemaredistribution-dependant.Wewillconsiderthestandardone,usefulinallGNU/Linux’sdistributions. 2.1pilingprocess Thepilationprocessconsistsofthesixsteps,discussedbelow: 2.1.1GettingtheKernelsources Normally,allGNU/LinuxdistributionshavetheirownLinuxsourcecodeasameta-package.Forexample,Debianhasthepackagelinux-source-
2.X.Yavailableonitssourcerepositories. Thus,wecaninstalleithertheGNU/LinuxKernelincludedinthedistrorepositoriesorthe”official”oneavailableat: /pub/linux/kernel/ 2.1.2ConfiguringtheKernel InordertoconfiguretheGNU/LinuxKernel,onecanchoosebetweendifferent”make”methods.TheeasiestoneisbyusingtheN-curseslibraries.It’squickly,canbeexecutedinavirtualterminalacrossremotesecureshellconnectionswithnolatencies,andithasitsownUserInterface(UI).Thiscanbedonebytypping:#cd/usr/src/linux-2.6.XX.Y#makemenuconfig 2.1.3CompilingtheKernelImage JustafterchoosingthedesiredKerneloptions,itistimepilethenewKernel’simage.Todothis:#make[-jN]6bzImage ThenewGNU/LinuxKernel’simagewillbeplacedatplatform-dependantdirectoryarch,i.e:arch/x8664/boot/bzImageintheparticularcaseofEM64Tarchitectures,andarch/i386/boot/bzImageintheX86ones. 1Even:stableversion2Odd:unstable,underdevelopment3Developmentrelease4Stablerelease5Stablepatch6Ncouldbe2..n,wherenisthenumberofprocessorspresentontheLinuxbox
5 2.1.4Compilingandinstallingthemodules Tobuildalloptionsmarkedasmodules(M)inthefirststep,simplyexecute:#make[-jN]modules It’snecessarytoinstallallofthemintherightplace.Todothisrun:#makemodulesinstall Afterawhile,allmoduleswillbecopiedto/lib/modules/2.X.YY/ 2.1.5Theinitrdimage SometimestheGNU/LinuxKernelisn’tcapableofbootingbyitselfbecauseofsomenon-loadeddrivers-suchasharddiskcontrollers,file-systems,andsoon-.So,itisinneedofaninitrdimage,which,inturn,isinchargeofdoingsomeprevioustasksinordertohelpitsstartup. Therearetwomainfile-systemsdesignedtocreateanewinitrdimagefile:cramfsandext2.Wecanchooseeither,itdoesn’tmatter.Inaddition,wecanuseag-zippedcpiofileformat. Generally,initrdcreationcanbeachievedbyusingascriptnamedmkinitrdormkinitramfs.IntheparticularcaseofGNU/LinuxDebiandistributions,wecancreateourowninitrdimagefilebasedonthepiledkernelbyrunning:#mkinitramfs-o/boot/initrd-
2.X.Y.Z2.X.Y.Z 2.1.6Installingthenewkernel’simage ThelaststepconsistsoninstallingthenewKernel’simageto/bootdirectory,namingitordinglyandupdatingtheBootLoader7-ifneeded-.#cparch/platform/boot/bzImage/boot/vmlinuz-myVersion#vim/boot/grub/menu.lst...#vim/etc/lilo.conf... Don’tettoaddanewinitrdentryifyouareusingone.LetustakealookattheFigure2-GRUBloaderexample-andFigure3-liloloaderexample-. titleDebianEtch2.6.25.6PreemptiveVoluntaryticknesslessroot(hd0,0)kernel/vmlinuz-2.6.25.6-kproot=/dev/md2initrd/initrd-2.6.25.6-kp Figure2:/boot/grub/menu.lstconfigurationfile 2.2SavingandrestoringtheKernel’sconfiguration Obviously,theremustbeasimplewayforrecoveringorsavingtheGNU/LinuxKernel’sconfigurationselectedinthefirststep.AllGNU/Linuxdistrosstoretheirconfigurationconcerningtherunningkernelinafileplacedat/boot/config-
2.X.Y. 7BearinmindthatincaseofusingLILO,youhavetorunmandlilorightaftermodifyingoraddingentriesin/etc/lilo.confconfigurationfile.
6 image=/boot/vmlinuz-2.6.25.6-kpinitrd=/boot/initrd-2.6.25.6-kproot=/dev/md2label=2.6.25.6-nokpread-only Figure3:Thesameconfigurationbutinthe/etc/lilo.conffileNowadays,modernkernelscanplishthisbyenablingCONFIGIKCONFIGsymbol,aswell.Thus,wecanrecovertheentireKernel’soptionsbytypinginashell:#zcat/proc/config.gz>/usr/src/linux-
2.X.Y.Z/.configOrbyusingtheextractconfigscript,availableinsidetheGNU/LinuxKernelsources:#cd/usr/src/linux-
2.X.Y.Z/#scripts/extractconfig/boot/vmlinuz-
2.X.Y.Z>.config Figure4:EnablingKernel’sconfigthrough/proc/config.gzinterface 2.3Initrd’sentrails Itisfeasibletomodifyanexistinginitrdimagefilewithoutcreatinganewonefromscratch.Sometimes,weneedtoaddanewmoduletotheinitrdimagefiledirectly,ormodifysomemodule’sparameters. Asweknow,aninitrdimagefileitisbasedonext2,cramfsfile-systemsorcpiofileformat.Thus,wecanmount,copy,modify,addordeletefilesinside
7 theinitrdimagefile.Letussupposethatwehaveaninitrdimagefilecalled/boot/initrd.img- 2.6.18-test.Firstofall,wemustcheckitsformat: #file/boot/initrd.img-2.6.18-test/boot/initrd.img-2.6.18-test:presseddata... Inthiscase,theinitrdimagefilepressedusinggzip.So,obviously,wehavetog-unzipit: #gunzip-f-S""-c/boot/initrd.img-2.6.18-test>/boot/initrd.img-2.6.18-test-gunzipped #file/boot/initrd.img-2.6.18-test-gunzipped/boot/initrd.img-2.6.18-test-gunzipped:ASCIIcpioarchive... #mkdir/tmp/initrd#cd/tmp/initrd#cat/boot/initrd.img-2.6.18-test-gunzipped|cpio-id 20505blocks#ls-l binconfetcinitlibmodulessbinscripts Rightafterg-unzippingandrecoveringtheentiredirectorystructurefromcpiofile,wecanadd,deleteormodifyanyexistingfileinside/tmp/initrddirectory.Allmodulesarestoredinside/lib/modules/2.X.Y.Zdirectory. TheinitscriptwillbeexecutedassoonastheinitrdimagefileispressedatstartuptimebytheGNU/LinuxKernel.Intheparticularcaseofext2initrds,insteadof”init”scriptfilewecanfindoutanotherequivalent,usuallynamedaslinuxrc. Oncetheinitrdstructurehasbeenmodifieditisnecessarytogobackwardsintheformerprocess,thatis,createthenewcpiofile,g-zippingitandcopyingittotherightplace. #find./|cpio-Hnewc-o>/tmp/new-initrd.cpio20505blocks #gzip-c/tmp/new-initrd.cpio>/boot/initrd.img-2.6.18-test Atthisstep,thenewinitrdimagefilehasbeenupdated,anditcanbeusedimmediately.Afterusingit,theGNU/LinuxKernelfreesallallocatedmemory,andsoitshowsthroughksyslog:Freeinginitrdmemory:4405kfreed 2.3.1Initrdexample:readingsomeargs Inthissubsectionwearegoingtochangetheinitrdbehaviour:wewantreadacertainparameterwhichwillbeappliedonlytoadesiredkernelmodule,called”mymodule”.Obviously,thismodulewon’tbeloadedbecauseitdoesn’texist. Firstofall,letustakealookattheinitscript,justbeforetheshellfunctionloadmodules:
8 ...echo-ne"TCG:please,insertthemoduleparametersformymodule:\>"readmargloadmodules$marg...Clearly,wehaveaddedsomecodetothescriptinordertopassanextraparametertotheshellfunctionloadmodules,readingitusingreadshell-statement. monfunctionsarelocatedinsidescripts/directory.Theshellfunctionloadmodulesisimplementedinscript/functionsfile.Thus,wemustchangesomeaspectsofit: ...if["$m"="mymodule"];then echo-ne"\tPersonalisedmodule$mwitharg$marg\n"Dosometasks...elsemodprobe-q$mfi... Toconclude,wehavetoaddmymoduletoconf/modules:#echo"mymodule">>conf/modules Afterre-generatingonceagaintheinitrdg-zippedcpiofile,thenewinitrdisreadytouse.So,nexttimethesystembootsup,wewillseewhatisshowninFigure5. Figure5:initrdalteredbehaviour 2.4PatchingtheGNU/LinuxKernel’ssources WhentherunningKernelmustbeupgraded,itisfeasibletopatchthecurrentsourcesandpilingthemasdescribedearlier.Todothisweneedtogetthepatch,thenapplyittothecurrentkernelsourcetree.Forexample,inordertoupgradeourKernelfrom2.6.18.1versionto2.6.18.2:#wget/pub/linux/kernel/v2.6/patch-2.6.18.2.bz2 Then,wehavetoapplythepatch:#cd/usr/src/linux-2.6.18.1#bzcat/path/to/patch-2.6.18.2.bz2|patch-p1-r/tmp/rejs.rej...patchingfilesound/core/hwdep.cpatchingfilesound/core/info.c
9 ...patchingfilesound/pci/emu10k1/emu10k1main.c Ifsomethinggoeswrong,wecantakealookat/tmp/rejs.rejfileinordertodeterminewhichfilescouldn’tbepatched. Beforetryingtopilethenewpatchedkernelmainstream,wehavetocleanupanyobjectfilesrunning:#makemrproper 3TheLinuxDeviceDrivermodel LookingbackatFigure1,theGNU/LinuxKernelposedofmanydifferentlayers,eachoftheminchargeofsomeconcretetasks.Someoftheselayerscanpiledasmodules,andsomeothersnot.Usually,pilingtheGNU/LinuxKernelwecanchoosewhenalayerwillbeamoduleornot.Bearinmindanynon-modulelayerwillbelinkedandinsertedphysicallyinsidetheKernel’simagefile.Asaresult,thenewKernelwillupymorememory.Clearly,theimage’sresultantfilewillbelargerinsizethananotheronewithmorepiledasmodules. TalkingaboutTheGNU/LinuxDeviceDriverModelintroducestheconceptofLinuxKernelModule(LKM). 3.1AboutModules It’sjustanELFobjectfile,notlinked,storedontheharddrive.Inmodernkernels,thesefilesarerecognisedbecauseofitsextension,ko.Intheolder2.4xKernel’sbranch,theyhad*.oextension.Thesefilescanbedriversornot.Inotherwords,notallmodulesaredrivers.Forexample,therearesomeLKMswhichallowuserstomountafile-system,likevfat.koorext3.ko.Asamatteroffact,theyaren’tdriversatall. InallGNU/Linuxkernels,piledmodulesaresavedin/lib/modules/2.X.Y.Zdirectory. Inorderpilesomelayerasamodule,wehavetoselectMinthemakemenuconfigscreen,asshowninFigure6. Figure6:CompilingsomelayersasLKMs 10 3.1.1Dealingwithmodules InordertodealwiththeLKMs,theGNU/LinuxKerneloffersagroupoftools,well-knownasthemodutils.Withthesesortmands,wecanload,unloadorjustlistwhichmodulesareloaded.Infact,allloadedmodulesarelinkedinsidetheGNU/Linuxkernelprogram.Bearinmindthatthislinkprocedurewilltakeplaceassoonasanewmoduleis”loaded”usingthemodutils,andunlinkedassoonasitisunloaded. 3.1.2Loadingamodule Therearetwomethodsforloadingmodules:#insmod/path/to/module.koarg1=value8...argN=valueNor#modprobemodulearg1=value...argN=valueN Thereareint,short,long,char*,int[],short[],long[]andchar**parameters.Forexample,inordertopassthreeparameterstoamodulecalledmymodule.ko,twoofthemanintvalueandthelastoneanarrayofintegers,wemustrunsomethinglike:#modprobemymoduleio=0x220irq=5dma=1,
5 Generally,inanyGNU/Linuxdistributionthere’safileinordertoconfigurewhatmoduleswillbeloadedautomaticallywiththeirownparameters.Inaddition,theGNU/LinuxKerneltriestoauto-loadsomemodulesassoonastheyareneeded,nomatterwhatmodulesarewrittendowninthemodule’sconfigurationfile.Thisoptionmustbeexplicitlyconfiguredinthemakemenuconfigscreen,asshowninFigure7. Figure7:Enablingkernelmodulesauto-loadingoption Beforeloadingthedesiredmodule,allmodulesneededtoloaditperfectlymustbelinkedintheruntimeKernel.Incaseofusinginsmod,itcouldbedifficulttorealizewhatmoduleswouldbeloadedpreviouslytoavoidunknownsymbolmessages-seebelow-.Thus,weneedtousemodprobe.Thisprogramtriestoloadalldependencieslookingat/lib/modules/2.X.Y.Z/modules.depfile,loadingthemjustbeforeloadingthegivenmodule. Inparticular,themodule”msdos.ko”,inchargeofallowingesstomsdosfile-systems,needsthesymbolsexportedbythemodulefat.kowhich,inturn,hasnodependencies: /lib/modules/2.6.18-kmodest/kernel/fs/msdos/msdos.ko:/lib/modules/2.6.18-kmodest/kernel/fs/fat/fat.ko .../lib/modules/2.6.18-kmodest/kernel/fs/fat/fat.ko: 8Todeterminetheallowedmodule’sparameters,run#modinfomodule. 11 Modules.depisgeneratedandupdatedafteranycalltomakemodulesinstall,asdiscussedearlier.mandneededtore-generateitbyhandis:#depmod-ae 3.1.3Unloadingamodule Tounloadamodulewehavetousemand:#rmmodmodule Sometimes,itcouldbenotpossibletounloadamoduleusingrmmodbecauseofitsreferencecounter.Clearly,somemoduleshavedependencies,andtheseonescannotbeunloadedaslongasothersdependantsmodulesareloadedandinuse.Themandreportsusifacertainmoduleisbeingused:ModuleSizeUsedbycdrom325441idecd ThereferencecounterofacertainLKMismaintainedinternallybytheKernel,but,sometimes,anLKMdevelopercanprefertowritesomecodeforcontrollingitdirectly,thankstotrymoduleget()andmoduleput()Cfunctions. Inaddition,theGNU/LinuxKernelcanpiledwithouttheMODULEUNLOADsymbol:inthiscase,unloadingamoduleisnotallowed. 3.2Devicedrivers Inshort,therearethreemaindevicedriverstypes:
1.Characterdevicedrivers.
2.Blockdevicedrivers.
3.Networkinterfacecards. Infact,allofthembutNICshavetheirassociatedspecialdevicefileunder/devdirectory,controlledbyUDEVsubsysteminmodernGNU/LinuxKernelimplementations.IntheparticularcaseofNICs,there’sno/deventrybecauseoftheirownimplementationissofarawayfromthemainideaofstreaming,likecharacterorblockdevicesare. 3.2.1Characterdevices Allchardevicedriversareessedwithnobuffers,andtheyallowtoreadorwriteanynumberofbytesperread/writeess.Generally,thisdevicedriversimplementhooksforthecallsopen(),close(),lseek(),read(),write(),tl(),ioctl()... Insidethe/devdirectory,thesedevicesarerecognisedbythecharacter”c”:crw-------1rootroot4,1Mar1308:54/dev/tty1crwxrwxrwx1rootroot1,3Jan262006/dev/nullcrw-rw-rw-1rootroot1,5Jan262006/dev/zero 3.2.2Blockdevices Theyareassociatedtoharddrives,harddiskcontrollers,etcetera.Thisparticularcaseofdevicedrivesworkalwaysusingintermediatebuffers,andthereadsorwritesarealwaysdonebybyte-blocks.Internally,therearealotof 12 differencesbut,asfarastheuserisconcerned,eithercharorblockdevicesseemquitesimilar. Theycanberecognisedinside/devdirectorybythecharacter”b”:brw-rw----1rootdisk8,0Feb1708:11/dev/sdabrw-rw----1rootdisk8,1Jan312005/dev/sda1 3.2.3Majorandminornumbers TheGNU/Linuxanisesalldevicedriversthroughmajorandminornumbers.Asamatteroffact,thisisthewayforidentifyingaparticulardevicedriverfromaKernel’spointofview.Thefilenameinside/devdirectorycanbechanged,obviously. Themajornumbermeanstheclassorgroupforthisparticulardevicedriver,suchasaHarddisk.Thankstoit,theGNU/LinuxKernel,assoonasanopen()systemcalliscalledbyarunningprocess,knowswhichdriverwillbeabletodispatchit. Forexample,theSATAharddisksdabelongstofamilywithmajornumberof8:brw-rw----1rootdisk8,0Mar1308:43/dev/sda Theminornumberisdirectlyrelatedtothemajorone.AllowstotheGNU/LinuxKerneltofine-tunewhichparticulardevicewillrespondtothecallforthatparticulardevicedriverfamily.Forexample,inthecaseofaharddisk,itcouldbethefirstdisk-partition,sucha/dev/sda1:brw-rw----1rootdisk8,0Mar1308:43/dev/sda1brw-rw----1rootfloppy8,33Mar1308:43/dev/sdc1 4ExportedSymbols TheGNU/LinuxKernelbelongstoMicrokernelfamily.So,somelayerscanbestackedofotherones,usingsomefunctionspreviouslyimplementedinlowerlayers.Thankstoit,anLKMdeveloper-orjustaKerneldeveloper-canre-usesomewell-knowntestedcodeorfunctionsinsidetheKernel.Clearly,thismatterpresentsareallayer-dependancyissue.AssoonassomeKernellayerneedssomeaidsimplemented(offered)byanotherone,theremustexistatotalagreementbetweenthesefunctionalitiesinordertoavoiderroneousbehaviours,likeKernelOOPS,unknownsymbolmessages,andsoon(seebelow). 4.1Whatisasymbol?
Asymbolcanbeafunction,avariable,adatastructure,etc.Forexample,dolookupfunction-analysedinsectionKernelOOPS-,isasymbol.Thissymbolisanexportedone,too.Thatis,anyexternallayerfromwhereitisdefined,canuseit. Inordertocheckifthissymbolisreallyexported,wecanread/proc/kallsymsfile.ThisfilestoresallKernel’sexportedsymboltable,placedinmemorybetweenstartksymtabandksymtab.Thefield’sformatislikenm’soutput.#cat/proc/kallsyms|grepdolookupc0164a7ftdolookup 13 ...c0102b7ftcheckuserspacec0102b94Tresumeuserspacec0102bacTsysenterentryc0102bb3tsysenterpastespc0102c2cTsystemcallc0102c55tnosinglestepc0102c6ctsyscallcallc0102c77tsyscallexitc0102c86trestoreallc0102c9etrestorenocheckc0102c9etrestorenochecknotrace... Figure8:TheKernel’sexportedsymboltablecutshort Yes,itexists!
Wecanfinditatkernel’saddress0xc0164a7f,andthe”t”fielddenotesitisalocalsymbolstoredinthetextsegment. 4.2ApeekinsideanLKMobjectfile Allmoduleshavetheirownsymbols,ofcourse.Thesesymbolscanbeexportedornot,dependingonthedesignortheconceptinwiththismoduleisinvolved.Supposewehaveamodule,saykmodest.ko,withalotoffunctions,variables,datastructuresandsoon.pilingitandobtainingthemodule’sobjectfile,wecangetalistofitssymbolsbyrunningnm:#nmkmodest.ko00000004Bbytestoread00000000TcleanupmoduleUcopyfromuserUcopytouser00000000DcurrentcmdUdofsync00000557Tdoinsertfd000003b2Tdoremapfd0000043cTdorestoretask0000028cTfddeinstallbytask000002f6Tfdinstallbytask00000020Bfdsinfo0000003dTfillfileinformation... Asshownintheprevioussymbollist,wecanrealizethatsomesymbolsareundefined(U).Thesesymbolsmustbeexportedinordertousethismodule.Thus,thecopyfromuser,copytouseranddofsyncsymbolswillbecalledfromthismodule.IftheseoneswillnotavailableatKernel’sexportedsymboltable,thismodulecannotbeloadedthroughinsmodormodprobeatall. 14 4.3Unknownsymbols Theundefinedsymbolscaneasunknownones.Forexample,inthisparticularcase: #insmod./kmodest.koinsmod:errorinserting’./kmodest.ko’: #dmesg|tail-3kmodest:Unknownsymboldofsynckmodest:Unknownsymbolsyskillkmodest:Unknownsymbolgetfilesstruct -1Unknownsymbolinmodule Whenthereareunknownsymbolsreportedbyinsmod,wecanfixthemprobablyloadingapreviousmoduleusingmodprobe.Sometimes,either,weneedtopatchtheKernelsourcesinordertoexportthesesymbolsexplicitly.Forexample,inordertosolvethepreviousissue,weneedthecurrentKernelsources.Withcscope9wecanfindoutwherethisnon-exportedfunctionsareandexportthembyaddingthemacroEXPORTSYMBOL. Finally,wehavetopilethekerneland,sometimes,rebootthesysteminordertoloadthenewone.Wecantakealookat/proc/kallsymsalways,inordertotestifacertainsymbolhasbeenexported.BearinmindthatsomesymbolswillbeexportedassoonasacertainchainofmoduleswillbelinkedintotheruntimeKernelviamodprobe. ...asmlinkagelongsyskill(intpid,intsig) ......}EXPORTSYMBOL(syskill); Figure9:Exportingsyskillsymbolbyhand 5DealingwithKernelOOPSandKernelPANICmessages WhenaKernelOOPSmessageappears,itispossibletocontinueusingthesystem.Butwhenacriticalerrorhappens,theKernelPANICmessageappears,hangingupthepletely.Dependingonitsconfiguration,thisKernelPanicmessagecanbefollowed,aftersomeseconds,inasystemreboot. 9AtooldesignedforbrowsinginsideCsourcecodefiles,suchasGNU/LinuxKernel.Itisfeasibletorun,insidetheGNU/LinuxKernelsourcedirectory,mand:#makecscopeAfterawhile,wecanbrowsethesourcesinfortablewaybyrunning:#cscope-k 15 (...)Pid:3378,m:findNottainted(2.6.24-etchnhalf.1-686#1)EIP:0060:[
The”find”mand,withPID3378,firesaKernelOOPSmessagewhilerunninginRing0thedolookup()function,justinside0xbeoffset.Checkingoutthecall-trace,itseemssorelatedtothe”find”mandexecuted:allKernelfunctionsexecutedherehavebeendesignedforwalkinginsideafile-systemthroughtheVFS(VirtualFile-Systemlayer,shown 16 inFigure1).Generally,afimanddoesnotfiresaKernelOOPSmessage.It’squite strange,soprobablythefimandisn’treallyinvolvedinthisawfulmatteratall.ItseemstousmorerelatedtosomeprobleminsidetheGNU/LinuxKernel,maybe. Thus,weneedtoanalysethedolookupfunction.WeneedthecurrentKernelsourcesinordertodothat,ofcourse!
Usingcscope,wefindoutthisfunctiondeclaredininclude/linux/dcache.hheaderfile.Hereisitssignature:externstructdentry*dlookup(structdentry*,structqstr*); AdentrystructurestoresallaboutafileordirectoryinsidethefilesystemwalkedbytheVFS.Thisfunctionsimplylooksforthedentry’sparenttransferedasaparameterandreturnsapointerincaseoffindingit.TheGNU/LinuxKernelalwaysworkswithdirectoriesorfilesusingdentryobjects,directlymappedtothememory,inordertoimprovetime-consumption.Thesekindofdentryobjectsarewell-knownasdcachetypedefs. Inshort,weknowthatfimand,whilesearchingforasomeparentdirectory,firedaKernelOOPSmessage.Howwasitpossible?
Clearly,becauseofsomephysicalerrormemory.Ifalldcache/dentryobjectsarekeepinmemoryforimprovingsystemtroughtput,andtheKernelOOPSwassaying:Nottainted,thentherewasaninvalidmemoryess.Why?
ItcouldbeaBUG.Whynot?
But,asamatteroffact,thesystemwasrunningperfectlyforalongtime,so...theanswershouldberelatedtophysicalDIMMmemoryerrors.Theactiontakeninordertodeterminetherealmemoryerrorcausewasrunningatrivialmemorytester(memtest86+).Afterabit,thefirstmemoryerrorsappeared.Whatarelief;-)!
!
5.2KernelPANIC AKernelPANICmessagemeansthere’snothingthesystemcandoinordertorecovertheminimalstabilityandisinneedofanurgentreboot.Sometimesitisinneedofpleteshutdownofanyattachedhardware,too-suchasanexternalUltra-SCSIdiskcontroller-. WhenthismessageappearsalongtheruntimelifeoftheGNU/LinuxKernelprogram,there’ssomehardwarerelatederrororanundocumentedBUG.Whenitursatboottime,itcanberelatedtoanotlinkeddriver-suchasahci.koSATAcontroller,oranotknownfile-systemduetosomemissingmodule.- InordertorebootthesystemafternsecondsjustafterappearingtheKernelPANICmessage,itisnecessarytoconfigurethistimeoutin/etc/sysctl.conf:kernel.panic=10Itisfeasibletochangethisbehaviouratruntimetyping:#echon10>/proc/sys/kernel/panic Otherwise,thesystemwillremainpoweredonandhangedupforever. Figure11:NotloadedHarddiskcontroller 10A0valuemeansnoreboot 17 Figure12:Notfile-systemsupportformountingtherootpartition 5.3Overridingtheinitprogram Sometimesweneedtorestoreroot’spasswordwehaveotten.Todothis,wemustpassaparametertotheGNU/LinuxKernelthroughtheBootLoader:init=/bin/bash.Assoonaswecanesstothebashprompt,wemustremounttheslashpartitioninordertoallowwriteess:mount/-oremount,rw.Finally,wecanusethemandasusual. 18 Glossary cpio Afile-formatsimilartotar,allfilespackedinone.,
6 EM64Tarchitecture 64bitarchitectures,IntelCoreprocessors.,
5 GNU/Linuxdistro(s) Arecopilationoftools,softwareandaGNU/LinuxKernelinonepackage,
5 KernelImage TheentireGNU/LinuxKernelprogram,gzipped.Itwillbepressedatruntime,inmemory.,
5 layer Asub-systeminchargeofdoingsometasks.Forexample,worklayer.,
3 N-Curseslibraries LibrariesdesignedforbuildingapplicationsrunninginsideTerminalswithmoreorlessGraphicalUserInterfacecapabilities.,
5 X86architectures(a.k.ai386)32bitarchitectures,AMDorIntelprocessors.,
5 19
声明：
该资讯来自于互联网网友发布，如有侵犯您的权益请联系我们。
相关文章
