Thanksformakingthisthenumber#1pageinGoogleon'sed'!

AlsoseemyUnixPage Mytwitterhandleis@grymoire Lastupdate:SatJul2107:05:49EDT2012CheckoutmynewSedReferenceChartandMyblogpostaboutsedand\t Imovedthecreditstotheendtomakethiseasierforthoselookingupanswers

SedAnIntroductionandTutorialbyBruceBarnett
TableofContents
TheAwfulTruthaboutsed Theessentialcommand:sforsubstitution Theslashasadelimiter Using&asthematchedstring Using\1tokeeppartofthepattern SubstituteFlags /gGlobalreplacement Issedrecursive? /1,/2,etc.Specifyingwhichoccurrence /pprint Writetoafilewith/wfilename Combiningsubstitutionflags Argumentsandinvocationofsed Multiplecommandswithecommand Filenamesonthecommandline sedn:noprinting sedfscriptname sedinshellscript QuotingmultiplesedlinesintheCshell QuotingmultiplesedlinesintheBourneshell Asedinterpreterscript SedComments Passingargumentsintoasedscript Usingsedinashellhereisdocument Multiplecommandsandorderofexecution AddressesandRangesofText Restrictingtoalinenumber Patterns Rangesbylinenumber Rangesbypatterns Deletewithd Printingwithp Reversingtherestrictionwith! Relationshipsbetweend,p,and! Theqorquitcommand Groupingwith{and} Operatinginapatternrangeexceptforthepatterns Writingafilewiththe'w'command Readinginafilewiththe'r'command SunOSandthe#CommentCommand Adding,Changing,Insertingnewlines Appendalinewith'a' Insertalinewith'i' Changealinewith'c' Leadingtabsandspacesinasedscript Addingmorethanoneline

Addinglinesandthepatternspace Addressrangesandtheabovecommands MultiLinePatterns Printlinenumberwith= Transformwithy Displayingcontrolcharacterswithal WorkingwithMultipleLines Usingnewlinesinsedscripts TheHoldBuffer Exchangewithx ExampleofContextGrep HoldwithhorH Keepingmorethanonelineintheholdbuffer GetwithgorG FlowControl Testingwitht Analternatewayofaddingcomments Thepoorlydocumented Passingregularexpressionsasarguments CommandSummary InConclusion MoreReferences Copyright2001,2005,2007,2011BruceBarnettandGeneralElectricCompany Allrightsreserved Youareallowedtoprintcopiesofthistutorialforyourpersonaluse,andlinktothispage,butyouarenotallowedtomake electroniccopies,orredistributethistutorialinanyformwithoutpermission.

IntroductiontoSed
Howtousesed,aspecialeditorformodifyingfilesautomatically.Ifyouwanttowriteaprogramtomakechangesinafile,sedis thetooltouse. ThereareafewprogramsthataretherealworkhorseintheUnixtoolbox.Theseprogramsaresimpletouseforsimple applications,yethavearichsetofcommandsforperformingcomplexactions.Don'tletthecomplexpotentialofaprogramkeep youfrommakinguseofthesimpleraspects.Thischapter,likealloftherest,startwiththesimpleconceptsandintroducesthe advancedtopicslateron. Anoteoncomments.WhenIfirstwrotethis,mostversionsofseddidnotallowyoutoplacecommentsinsidethescript.Lines startingwiththe'#'charactersarecomments.Newerversionsofsedmaysupportcommentsattheendofthelineaswell.

TheAwfulTruthaboutsed
Sedistheultimatestreameditor.Ifthatsoundsstrange,pictureastreamflowingthroughapipe.Okay,youcan'tseeastreamifit's insideapipe.That'swhatIgetforattemptingaflowinganalogy.Youwantliterature,readJamesJoyce. Anyhow,sedisamarvelousutility.Unfortunately,mostpeopleneverlearnitsrealpower.Thelanguageisverysimple,butthe documentationisterrible.TheSolarisonlinemanualpagesforsedarefivepageslong,andtwoofthosepagesdescribethe34 differenterrorsyoucanget.Aprogramthatspendsasmuchspacedocumentingtheerrorsthanitdoesdocumentingthelanguage hasaseriouslearningcurve. Donotfret!Itisnotyourfaultyoudon'tunderstandsed.Iwillcoversedcompletely.ButIwilldescribethefeaturesintheorder thatIlearnedthem.Ididn'tlearneverythingatonce.Youdon'tneedtoeither.

Theessentialcommand:sforsubstitution
Sedhasseveralcommands,butmostpeopleonlylearnthesubstitutecommand:s.Thesubstitutecommandchangesalloccurrences

oftheregularexpressionintoanewvalue.Asimpleexampleischanging"day"inthe"old"fileto"night"inthe"new"file:
s e ds / d a y / n i g h t /< o l d> n e w

Oranotherway(forUnixbeginners),
s e ds / d a y / n i g h t /o l d> n e w

andforthosewhowanttotestthis:
e c h od a y|s e ds / d a y / n i g h t /

Thiswilloutput"night". Ididn'tputquotesaroundtheargumentbecausethisexampledidn'tneedthem.Ifyoureadmyearliertutorialonquotes,you wouldunderstandwhyitdoesn'tneedquotes.However,Irecommendyoudousequotes.Ifyouhavemetacharactersinthe command,quotesarenecessary.Andifyouaren'tsure,it'sagoodhabit,andIwillhenceforthquotefutureexamplestoemphasize the"bestpractice."Usingthestrong(singlequote)character,thatwouldbe:

s e d' s / d a y / n i g h t / '< o l d> n e w

Imustemphasizethatthesededitorchangesexactlywhatyoutellitto.Soifyouexecuted
e c h oS u n d a y|s e d' s / d a y / n i g h t / '

Thiswouldoutputtheword"Sunnight"bacausesedfoundthestring"day"intheinput. Therearefourpartstothissubstitutecommand:
s / . . / . . / d a y n i g h t S u b s t i t u t ec o m m a n d D e l i m i t e r R e g u l a rE x p r e s s i o nP a t t e r nS e a r c hP a t t e r n R e p l a c e m e n ts t r i n g

Thesearchpatternisonthelefthandsideandthereplacementstringisontherighthandside. We'vecoveredquotingandregularexpressions..That's90%oftheeffortneededtolearnthesubstitutecommand.Toputitanother way,youalreadyknowhowtohandle90%ofthemostfrequentusesofsed.Therearea...fewfinepointsthatanfuturesed expertshouldknowabout.(Youjustfinishedsection1.There'sonly63moresectionstocover.:)Oh.Andyoumaywantto bookmarkthispage,....justincaseyoudon'tfinish.

Theslashasadelimiter
Thecharacterafterthesisthedelimiter.Itisconventionallyaslash,becausethisiswhated,more,andviuse.Itcanbeanything youwant,however.Ifyouwanttochangeapathnamethatcontainsaslashsay/usr/local/binto/common/binyoucouldusethe backslashtoquotetheslash:
s e d' s / \ / u s r \ / l o c a l \ / b i n / \ / c o m m o n \ / b i n / '< o l d> n e w

Gulp.Somecallthisa'PicketFence'andit'sugly.Itiseasiertoreadifyouuseanunderlineinsteadofaslashasadelimiter:
s e d' s _ / u s r / l o c a l / b i n _ / c o m m o n / b i n _ '< o l d> n e w

Somepeopleusecolons:
s e d' s : / u s r / l o c a l / b i n : / c o m m o n / b i n : '< o l d> n e w

Othersusethe"|"character.
s e d' s | / u s r / l o c a l / b i n | / c o m m o n / b i n | '< o l d> n e w

Pickoneyoulike.Aslongasit'snotinthestringyouarelookingfor,anythinggoes.Andrememberthatyouneedthree delimiters.Ifyougeta"Unterminated`s'command"it'sbecauseyouaremissingoneofthem.

Using&asthematchedstring

Sometimesyouwanttosearchforapatternandaddsomecharacters,likeparenthesis,aroundornearthepatternyoufound.Itis easytodothisifyouarelookingforaparticularstring:
s e d' s / a b c / ( a b c ) / '< o l d> n e w

Thiswon'tworkifyoudon'tknowexactlywhatyouwillfind.Howcanyouputthestringyoufoundinthereplacementstringif youdon'tknowwhatitis? Thesolutionrequiresthespecialcharacter"&."Itcorrespondstothepatternfound.

s e d' s / [ a z ] * / ( & ) / '< o l d> n e w

Youcanhaveanynumberof"&"inthereplacementstring.Youcouldalsodoubleapattern,e.g.thefirstnumberofaline:
%e c h o" 1 2 3a b c "|s e d' s / [ 0 9 ] * / && / ' 1 2 31 2 3a b c

Letmeslightlyamendthisexample.Sedwillmatchthefirststring,andmakeitasgreedyaspossible.Thefirstmatchfor'[09]*'is thefirstcharacterontheline,asthismatcheszeroofmorenumbers.Soiftheinputwas"abc123"theoutputwouldbeunchanged (well,exceptforaspacebeforetheletters).Abetterwaytoduplicatethenumberistomakesureitmatchesanumber:

%e c h o" 1 2 3a b c "|s e d' s / [ 0 9 ] [ 0 9 ] * / && / ' 1 2 31 2 3a b c

Thestring"abc"isunchanged,becauseitwasnotmatchedbytheregularexpression.Ifyouwantedtoeliminate"abc"fromthe output,youmustexpandthetheregularexpressiontomatchtherestofthelineandexplicitlyexcludepartoftheexpressionusing "(",")"and"\1",whichisthenexttopic. Aquickcomment.Theoriginalseddidnotsupportthe"+"metacharacter.GNUseddoesifyouusethe"r"commandlineoption, whichenablesextendedregularexpressions.The"+"means"oneormorematches".Sotheabovecouldalsobewrittenusing

%e c h o" 1 2 3a b c "|s e dr' s / [ 0 9 ] + / && / ' 1 2 31 2 3a b c

Using\1tokeeppartofthepattern
Ihavealreadydescribedtheuseof"("")"and"1"inmytutorialonregularexpressions.Toreview,theescapedparentheses(that is,parentheseswithbackslashesbeforethem)rememberportionsoftheregularexpression.Youcanusethistoexcludepartofthe regularexpression.The"\1"isthefirstrememberedpattern,andthe"\2"isthesecondrememberedpattern.Sedhasuptonine rememberedpatterns. Ifyouwantedtokeepthefirstwordofaline,anddeletetherestoftheline,marktheimportantpartwiththeparenthesis:
s e d' s / \ ( [ a z ] * \ ) . * / \ 1 / '

Ishouldelaborateonthis.Regularexprssionsaregreedy,andtrytomatchasmuchaspossible."[az]*"matcheszeroormore lowercaseletters,andtriestobeasbigaspossible.The".*"matcheszeroormorecharactersafterthefirstmatch.Sincethefirst onegrabsallofthelowercaseletters,thesecondmatchesanythingelse.Thereforeifyoutype

e c h oa b c d 1 2 3|s e d' s / \ ( [ a z ] * \ ) . * / \ 1 / '

Thiswilloutput"abcd"anddeletethenumbers. Ifyouwanttoswitchtwowordsaround,youcanremembertwopatternsandchangetheorderaround:
s e d' s / \ ( [ a z ] * \ )\ ( [ a z ] * \ ) / \ 2\ 1 / '

Notethespacebetweenthetworememberedpatterns.Thisisusedtomakesuretwowordsarefound.However,thiswilldo nothingifasinglewordisfound,oranylineswithnoletters.Youmaywanttoinsistthatwordshaveatleastoneletterbyusing
s e d' s / \ ( [ a z ] [ a z ] * \ )\ ( [ a z ] [ a z ] * \ ) / \ 2\ 1 / '

orbyusingextendedregularexpressions
s e dr' s / \ ( [ a z ] + \ )\ ( [ a z ] + \ ) / \ 2\ 1 / '#U s i n gG N Us e d

The"\1"doesn'thavetobeinthereplacementstring(intherighthandside).Itcanbeinthepatternyouaresearchingfor(inthe

lefthandside).Ifyouwanttoeliminateduplicatedwords,youcantry:
s e d' s / \ ( [ a z ] * \ )\ 1 / \ 1 / '

Ifyouwanttodetectduplicatedwords,youcanuse
s e dn' / \ ( [ a z ] [ a z ] * \ )\ 1 / p '

This,whenusedasafilter,willprintlineswithduplicatedwords. Thenumericvaluecanhaveuptoninevalues:"\1"thru"\9."Ifyouwantedtoreversethefirstthreecharactersonaline,youcan use

s e d' s / ^ \ ( . \ ) \ ( . \ ) \ ( . \ ) / \ 3 \ 2 \ 1 / '

SubstituteFlags
Youcanaddadditionalflagsafterthelastdelimiter.YoumighthavenoticedIuseda'p'attheendoftheprevioussubstitute command.Ialsoaddedthe'n'option.Letmefirstcoverthe'p'andothersubstituteflags.Theseflagscanspecifywhathappens whenamatchisfound.Letmedescribethem.

/gGlobalreplacement
MostUnixutiltiesworkonfiles,readingalineatatime.Sed,bydefault,isthesameway.Ifyoutellittochangeaword,itwill onlychangethefirstoccurrenceofthewordonaline.Youmaywanttomakethechangeoneverywordonthelineinsteadofthe first.Foranexample,let'splaceparenthesesaroundwordsonaline.Insteadofusingapatternlike"[AZaz]*"whichwon'tmatch wordslike"won't,"wewilluseapattern,"[^]*,"thatmatcheseverythingexceptaspace.Well,thiswillalsomatchanything because"*"meanszeroormore.ThecurrentversionofSolaris'ssed(asIwrotethis)cangetunhappywithpatternslikethis,and generateerrorslike"Outputlinetoolong"orevenrunforever.Iconsiderthisabug,andhavereportedthistoSun.Asawork around,youmustavoidmatchingthenullstringwhenusingthe"g"flagtosed.Aworkaroundexampleis:"[^][^]*."The followingwillputparenthesisaroundthefirstword:
s e d' s / [ ^] * / ( & ) / '< o l d> n e w

Ifyouwantittomakechangesforeveryword,adda"g"afterthelastdelimiterandusetheworkaround:
s e d' s / [ ^] [ ^] * / ( & ) / g '< o l d> n e w

Issedrecursive?
Sedonlyoperatesonpatternsfoundintheincomingdata.Thatis,theinputlineisread,andwhenapatternismatched,the modifiedoutputisgenerated,andtherestoftheinputlineisscanned.The"s"commandwillnotscanthenewlycreatedoutput. Thatis,youdon'thavetoworryaboutexpressionslike:
s e d' s / l o o p / l o o pt h el o o p / g '< o l d> n e w

Thiswillnotcauseaninfiniteloop.Ifasecond"s"commandisexecuted,itcouldmodifytheresultsofapreviouscommand.Iwill showyouhowtoexecutemultiplecommandslater.

/1,/2,etc.Specifyingwhichoccurrence
Withnoflags,thefirstpatternischanged.Withthe"g"option,allpatternsarechanged.Ifyouwanttomodifyaparticularpattern thatisnotthefirstoneontheline,youcoulduse"\("and"\)"tomarkeachpattern,anduse"\1"toputthefirstpatternback unchanged.Thisnextexamplekeepsthefirstwordonthelinebutdeletesthesecond:
s e d' s / \ ( [ a z A Z ] * \ )\ ( [ a z A Z ] * \ )/ \ 1/ '< o l d> n e w

Yuck.Thereisaneasierwaytodothis.Youcanaddanumberafterthesubstitutioncommandtoindicateyouonlywanttomatch thatparticularpattern.Example:
s e d' s / [ a z A Z ] */ / 2 '< o l d> n e w

Youcancombineanumberwiththeg(global)flag.Forinstance,ifyouwanttoleavethefirstwordalonealone,butchangethe second,third,etc.toDELETED,use/2g:
s e d' s / [ a z A Z ] */ D E L E T E D/ 2 g '< o l d> n e w

Don'tget/2and\2confused.The/2isusedattheend.\2isusedininsidethereplacementfield. Notethespaceafterthe"*"character.Withoutthespace,sedwillrunalong,longtime.(Note:thisbugisprobablyfixedbynow.) Thisisbecausethenumberflagandthe"g"flaghavethesamebug.Youshouldalsobeabletousethepattern

s e d' s / [ ^] * / / 2 '< o l d> n e w

butthisalsoeatsCPU.Ifthisworksonyourcomputer,anditdoesonsomeUnixsystems,youcouldremovetheencrypted passwordfromthepasswordfile:
s e d' s / [ ^ : ] * / / 2 '< / e t c / p a s s w d> / e t c / p a s s w o r d . n e w

Butthisdidn'tworkformethetimeIwrotethise.Using"[^:][^:]*"asaworkarounddoesn'thelpbecauseitwon'tmatchannon existentpassword,andinsteaddeletethethirdfield,whichistheuserID!Insteadyouhavetousetheuglyparenthesis:
s e d' s / ^ \ ( [ ^ : ] * \ ) : [ ^ : ] : / \ 1 : : / ' < / e t c / p a s s w d> / e t c / p a s s w o r d . n e w

Youcouldalsoaddacharactertothefirstpatternsothatitnolongermatchesthenullpattern:
s e d' s / [ ^ : ] * : / : / 2 ' < / e t c / p a s s w d> / e t c / p a s s w o r d . n e w

Thenumberflagisnotrestrictedtoasingledigit.Itcanbeanynumberfrom1to512.Ifyouwantedtoaddacolonafterthe80th characterineachline,youcouldtype:
s e d' s / . / & : / 8 0 '< f i l e> n e w

Youcanalsodoitthehardwaybyusing80dots:
s e d' s / ^ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . / & : / '< f i l e> n e w

/pprint
Bydefault,sedprintseveryline.Ifitmakesasubstitution,thenewtextisprintedinsteadoftheoldone.Ifyouuseanoptional argumenttosed,"sedn,"itwillnot,bydefault,printanynewlines.I'llcoverthisandotheroptionslater.Whenthe"n"optionis used,the"p"flagwillcausethemodifiedlinetobeprinted.Hereisonewaytoduplicatethefunctionofgrepwithsed:
s e dn' s / p a t t e r n / & / p '< f i l e

Writetoafilewith/wfilename
Thereisonemoreflagthatcanfollowthethirddelimiter.Withit,youcanspecifyafilethatwillreceivethemodifieddata.An exampleisthefollowing,whichwillwritealllinesthatstartwithanevennumber,followedbyaspace,tothefileeven:
s e dn' s / ^ [ 0 9 ] * [ 0 2 4 6 8 ]/ & / we v e n '< f i l e

Inthisexample,theoutputfileisn'tneeded,astheinputwasnotmodified.Youmusthaveexactlyonespacebetweenthewand thefilename.Youcanalsohavetenfilesopenwithoneinstanceofsed.Thisallowsyoutosplitupastreamofdataintoseparate files.Usingthepreviousexamplecombinedwithmultiplesubstitutioncommandsdescribedlater,youcouldsplitafileintoten piecesdependingonthelastdigitofthefirstnumber.Youcouldalsousethismethodtologerrorordebugginginformationtoa specialfile.

Combiningsubstitutionflags
Youcancombineflagswhenitmakessense.Also"w"hastobethelastflag.Forexamplethefollowingcommandworks:
s e dn' s / a / A / 2 p w/ t m p / f i l e '< o l d> n e w

NextIwilldiscusstheoptionstosed,anddifferentwaystoinvokesed.

Argumentsandinvocationofsed
previously,Ihaveonlyusedonesubstitutecommand.Ifyouneedtomaketwochanges,andyoudidn'twanttoreadthemanual, youcouldpipetogethermultiplesedcommands:
s e d' s / B E G I N / b e g i n / '< o l d|s e d' s / E N D / e n d / '> n e w

Thisusedtwoprocessesinsteadofone.Asedguruneverusestwoprocesseswhenonecando.

Multiplecommandswithecommand
Onemethodofcombiningmultiplecommandsistouseaebeforeeachcommand:
s e de' s / a / A / 'e' s / b / B / '< o l d> n e w

A"e"isn'tneededintheearlierexamplesbecausesedknowsthattheremustalwaysbeonecommand.Ifyougivesedone argument,itmustbeacommand,andsedwilleditthedatareadfromstandardinput. AlsoseeQuotingmultiplesedlinesintheBourneshell

Filenamesonthecommandline
Youcanspecifyfilesonthecommandlineifyouwish.Ifthereismorethanoneargumenttosedthatdoesnotstartwithanoption, itmustbeafilename.Thisnextexamplewillcountthenumberoflinesinthreefilesthatdon'tbeginwitha"#:"
s e d' s / ^ # . * / / ' f 1f 2f 3|g r e pv' ^ $ '|w cl

Thesedsubstitutecommandchangeseverylinethatstartswitha"#"intoablankline.Grepwasusedtofilteroutemptylines.Wc countsthenumberoflinesleft.Sedhasmorecommandsthatmakegrepunnecessary.ButIwillcoverthatlater. Ofcourseyoucouldwritethelastexampleusingthe"e"option:

s e de' s / ^ # . * / / ' f 1f 2f 3|g r e pv' ^ $ '|w cl

Therearetwootheroptionstosed.

sedn:noprinting
The"n"optionwillnotprintanythingunlessanexplicitrequesttoprintisfound.Imentionedthe"/p"flagtothesubstitute commandasonewaytoturnprintingbackon.Letmeclarifythis.Thecommand
s e d ' s / P A T T E R N / & / p 'f i l e

actslikethecatprogramifPATTERNisnotinthefile:e.g.nothingischanged.IfPATTERNisinthefile,theneachlinethathas thisisprintedtwice.Addthe"n"optionandtheexampleactslikegrep:
s e dn' s / P A T T E R N / & / p 'f i l e

Nothingisprinted,exceptthoselineswithPATTERNincluded.

sedfscriptname
Ifyouhavealargenumberofsedcommands,youcanputthemintoafileanduse
s e dfs e d s c r i p t< o l d> n e w

wheresedscriptcouldlooklikethis:
#s e dc o m m e n t-T h i ss c r i p tc h a n g e sl o w e rc a s ev o w e l st ou p p e rc a s e s / a / A / g s / e / E / g

s / i / I / g s / o / O / g s / u / U / g

Whenthereareseveralcommandsinonefile,eachcommandmustbeonaseparateline. Alsoseehere

sedinshellscript
Ifyouhavemanycommandsandtheywon'tfitneatlyononeline,youcanbreakupthelineusingabackslash:
s e de' s / a / A / g '\ e' s / e / E / g '\ e' s / i / I / g '\ e' s / o / O / g '\ e' s / u / U / g ' < o l d> n e w

QuotingmultiplesedlinesintheCshell
Youcanhavealarge,multilinesedscriptintheCshell,butyoumusttelltheCshellthatthequoteiscontinuedacrossseveral lines.Thisisdonebyplacingabackslashattheendofeachline:
# ! / b i n / c s hf s e d' s / a / A / g \ s / e / E / g\ s / i / I / g\ s / o / O / g\ s / u / U / g ' < o l d> n e w

QuotingmultiplesedlinesintheBourneshell
TheBourneshellmakesthiseasierasaquotecancoverseverallines:
# ! / b i n / s h s e d' s / a / A / g s / e / E / g s / i / I / g s / o / O / g s / u / U / g ' < o l d> n e w

Asedinterpreterscript
Anotherwayofexecutingsedistouseaninterpreterscript.Createafilethatcontains: #!/bin/sedf s/a/A/g s/e/E/g s/i/I/g s/o/O/g s/u/U/g Clickheretogetfile:CapVowel.sed Ifthisscriptwasstoredinafilewiththename"CapVowel"andwasexecutable,youcoulduseitwiththesimplecommand:
C a p V o w e l< o l d> n e w

Comments
Sedcommentsarelineswherethefirstnonwhitecharacterisa"#."Onmanysystems,sedcanhaveonlyonecomment,andit mustbethefirstlineofthescript.OntheSun(1988whenIwrotethis),youcanhaveseveralcommentlinesanywhereinthe

mustbethefirstlineofthescript.OntheSun(1988whenIwrotethis),youcanhaveseveralcommentlinesanywhereinthe script.ModernversionsofSedsupportthis.Ifthefirstlinecontainsexactly"#n"thenthisdoesthesamethingasthe"n"option: turningoffprintingbydefault.Thiscouldnotdonewithasedinterpreterscript,becausethefirstlinemuststartwith"#!/bin/sedf" asIthink"#!/bin/sednf"generatedanerror.ItworksasIwritethisupdatein2008.Notethat"#!/bin/sedfn"doesnotwork becausesedthinksthefilenameofthescriptis"n".However,

" # ! / b i n / s e dn f "

doeswork.

Passingargumentsintoasedscript
PassingawordintoashellscriptthatcallssediseasyifyourememberedmytutorialontheUnixquotingmechanism.Toreview, youusethesinglequotestoturnquotingonandoff.Asimpleshellscriptthatusessedtoemulategrepis: #!/bin/sh sedn's/'$1'/&/p' Howeverthereisaproblemwiththisscript.Ifyouhaveaspaceasanargument,thescriptwouldcauseasyntaxerrorAbetter versionwouldprotectfromthishappening: #!/bin/sh sedn's/'"$1"'/&/p'

Clickheretogetfile:sedgrep.sed Ifthiswasstoredinafilecalledsedgrep,youcouldtype
s e d g r e p' [ A Z ] [ A Z ] '< f i l e

Thiswouldallowsedtoactasthegrepcommand.

Usingsedinashellhereisdocument
Youcanusesedtoprompttheuserforsomeparametersandthencreateafilewiththoseparametersfilledin.Youcouldcreatea filewithdummyvaluesplacedinsideit,andusesedtochangethosedummyvalues.Asimplerwayistousethe"hereis" document,whichusespartoftheshellscriptasifitwerestandardinput:
# ! / b i n / s h e c h on' w h a ti st h ev a l u e ?' r e a dv a l u e s e d ' s / X X X / ' $ v a l u e ' / '< < E O F T h ev a l u ei sX X X E O F

Clickheretogetfile:sed_hereis.sed Whenexecuted,thescriptsays:
w h a ti st h ev a l u e ?

Ifyoutypein"123,"thenextlinewillbe:
T h ev a l u ei s1 2 3

Iadmitthisisacontrivedexample."Hereis"documentscanhavevaluesevaluatedwithouttheuseofsed.Thisexampledoesthe samething:
# ! / b i n / s h e c h on' w h a ti st h ev a l u e ?' r e a dv a l u e c a t< < E O F T h ev a l u ei s$ v a l u e

E O F

However,combining"hereis"documentswithsedcanbeusefulforsomecomplexcases. Notethat sed's/XXX/'$value'/'<<EOF willgiveasyntaxerroriftheusertypesaspace.Betterformwouldbetouse sed's/XXX/'"$value"'/'<<EOF

Multiplecommandsandorderofexecution
Asweexploremoreofthecommandsofsed,thecommandswillbecomecomplex,andtheactualsequencecanbeconfusing.It's reallyquitesimple.Eachlineisreadin.Eachcommand,inorderspecifiedbytheuser,hasachancetooperateontheinputline. Afterthesubstitutionsaremade,thenextcommandhasachancetooperateonthesameline,whichmayhavebeenmodifiedby earliercommands.Ifyoueverhaveaquestion,thebestwaytolearnwhatwillhappenistocreateasmallexample.Ifacomplex commanddoesn'twork,makeitsimpler.Ifyouarehavingproblemsgettingacomplexscriptworking,breakitupintotwosmaller scriptsandpipethetwoscriptstogether.

AddressesandRangesofText
Youhaveonlylearnedonecommand,andyoucanseehowpowerfulsedis.However,allitisdoingisagrepandsubstitute.That is,thesubstitutecommandistreatingeachlinebyitself,withoutcaringaboutnearbylines.Whatwouldbeusefulistheabilityto restricttheoperationtocertainlines.Someusefulrestrictionsmightbe:

Specifyingalinebyitsnumber. Specifyingarangeoflinesbynumber. Alllinescontainingapattern. Alllinesfromthebeginningofafiletoaregularexpression Alllinesfromaregularexpressiontotheendofthefile. Alllinesbetweentworegularexpressions. Sedcandoallthatandmore.Everycommandinsedcanbeproceededbyanaddress,rangeorrestrictionliketheaboveexamples. Therestrictionoraddressimmediatelyprecedesthecommand: restrictioncommand

Restrictingtoalinenumber
Thesimplestrestrictionisalinenumber.Ifyouwantedtodeletethefirstnumberonline3,justadda"3"beforethecommand:
s e d' 3s / [ 0 9 ] [ 0 9 ] * / / '< f i l e> n e w

Patterns
ManyUnixutilitieslikeviandmoreuseaslashtosearchforaregularexpression.Sedusesthesameconvention,providedyou terminatetheexpressionwithaslash.Todeletethefirstnumberonalllinesthatstartwitha"#,"use:
s e d' / ^ # /s / [ 0 9 ] [ 0 9 ] * / / '

Iplacedaspaceafterthe"/expression/"soitiseasiertoread.Itisn'tnecessary,butwithoutitthecommandishardertofathom.Sed doesprovideafewextraoptionswhenspecifyingregularexpressions.ButI'lldiscussthoselater.Iftheexpressionstartswitha backslash,thenextcharacteristhedelimiter.Touseacommainsteadofaslash,use:

s e d' \ , ^ # ,s / [ 0 9 ] [ 0 9 ] * / / '

Themainadvantageofthisfeatureissearchingforslashes.Supposeyouwantedtosearchforthestring"/usr/local/bin"andyou wantedtochangeitfor"/common/all/bin."Youcouldusethebackslashtoescapetheslash:
s e d' / \ / u s r \ / l o c a l \ / b i n /s / \ / u s r \ / l o c a l / \ / c o m m o n \ / a l l / '

Itwouldbeeasiertofollowifyouusedanunderlineinsteadofaslashasasearch.Thisexampleusestheunderlineinboththe searchcommandandthesubstitutecommand:
s e d' \ _ / u s r / l o c a l / b i n _s _ / u s r / l o c a l _ / c o m m o n / a l l _ '

Thisillustrateswhysedscriptsgetthereputationforobscurity.Icouldbeperverseandshowyoutheexamplethatwillsearchfor alllinesthatstartwitha"g,"andchangeeach"g"onthatlinetoan"s:"
s e d' / ^ g / s / g / s / g '

Addingaspaceandusinganunderscoreafterthesubstitutecommandmakesthismucheasiertoread:
s e d' / ^ g /s _ g _ s _ g '

Er,Itakethatback.It'shopeless.Thereisalessonhere:UsecommentsliberallyinasedscriptunderSunOS.Youmayhaveto removethecommentstorunthescriptunderadifferentoperatingsystem,butyounowknowhowtowriteasedscripttodothat veryeasily!CommentsareaGoodThing.Youmayhaveunderstoodthescriptperfectlywhenyouwroteit.Butsixmonthsfrom nowitcouldlooklikemodemnoise.

Rangesbylinenumber
Youcanspecifyarangeonlinenumbersbyinsertingacommabetweenthenumbers.Torestrictasubstitutiontothefirst100 lines,youcanuse:
s e d' 1 , 1 0 0s / A / a / '

Ifyouknowexactlyhowmanylinesareinafile,youcanexplicitlystatethatnumbertoperformthesubstitutionontherestofthe file.Inthiscase,assumeyouusedwctofindoutthereare532linesinthefile:
s e d' 1 0 1 , 5 3 2s / A / a / '

Aneasierwayistousethespecialcharacter"$,"whichmeansthelastlineinthefile.
s e d' 1 0 1 , $s / A / a / '

The"$"isoneofthoseconventionsthatmean"last"inutilitieslikecate,vi,anded."cate"Linenumbersarecumulativeif severalfilesareedited.Thatis,
s e d' 2 0 0 , 3 0 0s / A / a / 'f 1f 2f 3> n e w

isthesameas
c a tf 1f 2f 3|s e d' 2 0 0 , 3 0 0s / A / a / '> n e w

Rangesbypatterns
Youcanspecifytworegularexpressionsastherange.Assuminga"#"startsacomment,youcansearchforakeyword,removeall commentsuntilyouseethesecondkeyword.Inthiscasethetwokeywordsare"start"and"stop:"
s e d' / s t a r t / , / s t o p /s / # . * / / '

Thefirstpatternturnsonaflagthattellssedtoperformthesubstitutecommandoneveryline.Thesecondpatternturnsofftheflag. Ifthe"start"and"stop"patternoccurstwice,thesubstitutionisdonebothtimes.Ifthe"stop"patternismissing,theflagisnever turnedoff,andthesubstitutionwillbeperformedoneverylineuntiltheendofthefile. Youshouldknowthatifthe"start"patternisfound,thesubstitutionoccursonthesamelinethatcontains"start."Thisturnsona switch,whichislineoriented.Thatis,thenextlineisreadandthesubstitutecommandischecked.Ifitcontains"stop"theswitch isturnedoff.Switchesarelineoriented,andnotwordoriented.

Youcancombinelinenumbersandregularexpressions.Thisexamplewillremovecommentsfromthebeginningofthefileuntilit findsthekeyword"start:"
s e de' 1 , / s t a r t /s / # . * / / '

Thisexamplewillremovecommentseverywhereexceptthelinesbetweenthetwokeywords:
s e de' 1 , / s t a r t /s / # . * / / 'e' / s t o p / , $s / # . * / / '

Thelastexamplehasarangethatoverlapsthe"/start/,/stop/"range,asbothrangesoperateonthelinesthatcontainthekeywords.I willshowyoulaterhowtorestrictacommandupto,butnotincludingthelinecontainingthespecifiedpattern.ItisinOperating inapatternrangeexceptforthepatternsButIhavetocoversomemorebasicprinciples. BeforeIstartdiscussingthevariouscommands,Ishouldexplainthatsomecommandscannotoperateonarangeoflines.Iwilllet youknowwhenImentionthecommands.InthisnextsectionIwilldescribethreecommands,oneofwhichcannotoperateona range.

Deletewithd
Usingrangescanbeconfusing,soyoushouldexpecttodosomeexperimentationwhenyouaretryingoutanewscript.Auseful commanddeleteseverylinethatmatchestherestriction:"d."Ifyouwanttolookatthefirst10linesofafile,youcanuse:
s e d' 1 1 , $d '< f i l e

whichissimilarinfunctiontotheheadcommand.Ifyouwanttochopofftheheaderofamailmessage,whichiseverythingupto thefirstblankline,use:
s e d' 1 , / ^ $ /d '< f i l e

Youcanduplicatethefunctionofthetailcommand,assumingyouknowthelengthofafile.Wccancountthelines,andexprcan subtract10fromthenumberoflines.ABourneshellscripttolookatthelast10linesofafilemightlooklikethis: #!/bin/sh #printlast10linesoffile #Firstargumentisthefilename lines=`wcl$1|awk'{print$1}'` start=`expr$lines10` sed"1,$startd"$1 Clickheretogetfile:sed_tail.sh Therangefordeletionscanberegularexpressionspairstomarkthebeginandendoftheoperation.Oritcanbeasingleregular expression.Deletingalllinesthatstartwitha"#"iseasy:
s e d' / ^ # /d '

Removingcommentsandblanklinestakestwocommands.Thefirstremoveseverycharacterfromthe"#"totheendoftheline, andtheseconddeletesallblanklines:
s e de' s / # . * / / 'e' / ^ $ /d '

Athirdoneshouldbeaddedtoremoveallblanksandtabsimmediatelybeforetheendofline:
s e de' s / # . * / / 'e' s / [^ I ] * $ / / 'e' / ^ $ /d '

Thecharacter"^I"isaCTRLIortabcharacter.Youwouldhavetoexplicitlytypeinthetab.Notetheorderofoperationsabove, whichisinthatorderforaverygoodreason.Commentsmightstartinthemiddleofaline,withwhitespacecharactersbefore them.Thereforecommentsarefirstremovedfromaline,potentiallyleavingwhitespacecharactersthatwerebeforethecomment. Thesecondcommandremovesalltrailingblanks,sothatlinesthatarenowblankareconvertedtoemptylines.Thelastcommand deletesemptylines.Together,thethreecommandsremovealllinescontainingonlycomments,tabsorspaces. Thisdemonstratesthepatternspacesedusestooperateonaline.Theactualoperationsedusesis:

Copytheinputlineintothepatternspace. Applythefirst sedcommandonthepatternspace,iftheaddressrestrictionistrue. Repeatwiththenextsedexpression,again operatingonthepatternspace. Whenthelastoperationisperformed,writeoutthepatternspace andreadinthenextlinefromtheinputfile.

Printingwithp
Anotherusefulcommandistheprintcommand:"p."Ifsedwasn'tstartedwithan"n"option,the"p"commandwillduplicatethe input.Thecommand
s e d' p '

willduplicateeveryline.Ifyouwantedtodoubleeveryemptyline,use:
s e d' / ^ $ /p '

Addingthe"n"optionturnsoffprintingunlessyourequestit.Anotherwayofduplicatinghead'sfunctionalityistoprintonlythe linesyouwant.Thisexampleprintsthefirst10lines:
s e dn' 1 , 1 0p '< f i l e

Sedcanactlikegrepbycombiningtheprintoperatortofunctiononalllinesthatmatcharegularexpression:
s e dn' / m a t c h /p '

whichisthesameas:
g r e pm a t c h

Reversingtherestrictionwith!
Sometimesyouneedtoperformanactiononeverylineexceptthosethatmatcharegularexpression,orthoseoutsideofarangeof addresses.The"!"character,whichoftenmeansnotinUnixutilities,invertstheaddressrestriction.Yourememberthat
s e dn' / m a t c h /p '

actslikethegrepcommand.The"v"optiontogrepprintsalllinesthatdon'tcontainthepattern.Sedcandothiswith
s e dn' / m a t c h /! p '< / t m p / b

Relationshipsbetweend,p,and!
Asyoumayhavenoticed,thereareoftenseveralwaystosolvethesameproblemwithsed.Thisisbecauseprintanddeleteare oppositefunctions,anditappearsthat"!p"issimilarto"d,"while"!d"issimilarto"p."Iwantedtotestthis,soIcreateda20line file,andtriedeverydifferentcombination.Thefollowingtable,whichshowstheresults,demonstratesthedifference:
R e l a t i o n sb e t w e e nd ,p ,a n d! S e d R a n g e C o m m a n d R e s u l t s s e dn 1 , 1 0 p P r i n tf i r s t1 0l i n e s s e dn 1 1 , $ ! p P r i n tf i r s t1 0l i n e s s e d 1 , 1 0 ! d P r i n tf i r s t1 0l i n e s s e d 1 1 , $ d P r i n tf i r s t1 0l i n e s s e dn 1 , 1 0 ! p P r i n tl a s t1 0l i n e s s e dn 1 1 , $ p P r i n tl a s t1 0l i n e s s e d 1 , 1 0 d P r i n tl a s t1 0l i n e s s e d 1 1 , $ ! d P r i n tl a s t1 0l i n e s s e dn 1 , 1 0 d N o t h i n gp r i n t e d s e dn 1 , 1 0 ! d N o t h i n gp r i n t e d s e dn 1 1 , $ d N o t h i n gp r i n t e d

s e dn 1 1 , $ ! d N o t h i n gp r i n t e d s e d 1 , 1 0 p P r i n tf i r s t1 0l i n e st w i c e , T h e nn e x t1 0l i n e so n c e s e d 1 1 , $ ! p P r i n tf i r s t1 0l i n e st w i c e , T h e nl a s t1 0l i n e so n c e s e d 1 , 1 0 ! p P r i n tf i r s t1 0l i n e so n c e , T h e nl a s t1 0l i n e st w i c e s e d 1 1 , $ p P r i n tf i r s t1 0l i n e so n c e , t h e nl a s t1 0l i n e st w i c e

Thistableshowsthatthefollowingcommandsareidentical:
s e dn' 1 , 1 0p ' s e dn' 1 1 , $! p ' s e d' 1 , 1 0! d ' s e d' 1 1 , $d '

Italsoshowsthatthe"!"command"inverts"theaddressrange,operatingontheotherlines.

Theqorquitcommand
Thereisonemoresimplecommandthatcanrestrictthechangestoasetoflines.Itisthe"q"command:quit.thethirdwayto duplicatetheheadcommandis:
s e d' 1 1q '

whichquitswhentheeleventhlineisreached.Thiscommandismostusefulwhenyouwishtoaborttheeditingaftersome conditionisreached. The"q"commandistheonecommandthatdoesnottakearangeofaddresses.Obviouslythecommand

s e d' 1 , 1 0q '

cannotquit10times.Instead
s e d' 1q '

or
s e d' 1 0q '

iscorrect.

Groupingwith{and}
Thecurlybraces,"{"and"},"areusedtogroupthecommands. Hardlyworththebuildup.Allthatproseandthesolutionisjustmatchingsquigqles.Well,thereisonecomplication.Sinceeach sedcommandmuststartonitsownline,thecurlybracesandthenestedsedcommandsmustbeonseparatelines. Previously,Ishowedyouhowtoremovecommentsstartingwitha"#."Ifyouwantedtorestricttheremovaltolinesbetween special"begin"and"end"keywords,youcoulduse:
# ! / b i n / s h #T h i si saB o u r n es h e l ls c r i p tt h a tr e m o v e s# t y p ec o m m e n t s #b e t w e e n' b e g i n 'a n d' e n d 'w o r d s . s e dn' / b e g i n / , / e n d /{ s / # . * / / s / [^ I ] * $ / / / ^ $ /d p } '

Clickheretogetfile:sed_begin_end.sh Thesebracescanbenested,whichallowyoutocombineaddressranges.Youcouldperformthesameactionasbefore,butlimit thechangetothefirst100lines:

# ! / b i n / s h #T h i si saB o u r n es h e l ls c r i p tt h a tr e m o v e s# t y p ec o m m e n t s #b e t w e e n' b e g i n 'a n d' e n d 'w o r d s . s e dn' 1 , 1 0 0{ / b e g i n / , / e n d /{ s / # . * / / s / [^ I ] * $ / / / ^ $ /d p } } '

Clickheretogetfile:sed_begin_end1.sh Youcanplacea"!"beforeasetofcurlybraces.Thisinvertstheaddress,whichremovescommentsfromalllinesexceptthose betweenthetworeservedwords:

# ! / b i n / s h s e d' / b e g i n / , / e n d /! { s / # . * / / s / [^ I ] * $ / / / ^ $ /d p } '

Clickheretogetfile:sed_begin_end2.sh

Operatinginapatternrangeexceptforthepatterns
YoumayrememberthatImentionedyoucandoasubstitureonapatternrange,likechanging"old"to"new"betweenabegin/end pattern:
# ! / b i n / s h s e d' / b e g i n / , / e n d /s / o l d / n e w / '

Anotherwaytowritethisistousethecurlybracesforgrouping:
# ! / b i n / s h s e d' / b e g i n / , / e n d /{ s / o l d / n e w / } '

Ithinkthismakesthecodeclearertounderstand,andeasiertomodifyf,asyouwillseebelow. Ifyoudidnotwanttomakeanychangeswheretheword"begin"occured,youcouldsimpleaddanewconditiontoskipoverthat line:

# ! / b i n / s h s e d' / b e g i n / , / e n d /{ / b e g i n / n#s k i po v e rt h el i n et h a th a s" b e g i n "o ni t s / o l d / n e w / } '

However,skippingoverthelinethathas"end"istrickier.Ifyouusethesamemethodyouusedfor"begin"thenthesedengine willnotseethe"end"tostoptherangeitskipsoverthataswell.Thesolutionistodoasubstitureonalllinesthatdon'thavethe "end"byusing

# ! / b i n / s h s e d' / b e g i n / , / e n d /{ / b e g i n / n#s k i po v e rt h el i n et h a th a s" b e g i n "o ni t / e n d /! { s / o l d / n e w / } } '

Writingafilewiththe'w'command
Youmayrememberthatthesubstitutecommandcanwritetoafile.Hereagainistheexamplethatwillonlywritelinesthatstart withanevennumber(andfollowedbyaspace):
s e dn' s / ^ [ 0 9 ] * [ 0 2 4 6 8 ]/ & / we v e n '< f i l e

Iusedthe"&"inthereplacementpartofthesubstitutioncommandsothatthelinewouldnotbechanged.Asimplerexampleisto usethe"w"command,whichhasthesamesyntaxasthe"w"flaginthesubstitutecommand:
s e dn' / ^ [ 0 9 ] * [ 0 2 4 6 8 ] /we v e n '< f i l e

Rememberonlyonespacemustfollowthecommand.Anythingelsewillbeconsideredpartofthefilename.The"w"command alsohasthesamelimitationasthe"w"flag:only10filescanbeopenedinsed.

Readinginafilewiththe'r'command
Thereisalsoacommandforreadingfiles.Thecommand
s e d' $ re n d '< i n > o u t

willappendthefile"end"attheendofthefile(address"$)."Thefollowingwillinsertafileafterthelinewiththeword "INCLUDE:"
s e d' / I N C L U D E /rf i l e '< i n> o u t

Youcanusethecurlybracestodeletethelinehavingthe"INCLUDE"commandonit:
# ! / b i n / s h s e d' / I N C L U D E /{ rf i l e d } '

Clickheretogetfile:sed_include.sh Theorderofthedeletecommand"d"andthereadfilecommand"r"isimportant.Changetheorderanditwillnotwork.Thereare twosubtleactionsthatpreventthisfromworking.Thefirstisthe"r"commandwritesthefiletotheoutputstream.Thefileisnot insertedintothepatternspace,andthereforecannotbemodifiedbyanycommand.Thereforethedeletecommanddoesnotaffect thedatareadfromthefile. Theothersubtletyisthe"d"commanddeletesthecurrentdatainthepatternspace.Onceallofthedataisdeleted,itdoesmake sensethatnootheractionwillbeattempted.Thereforea"d"commandexecutedinacurlybracealsoabortsallfurtheractions.As anexample,thesubstitutecommandbelowisneverexecuted:

# ! / b i n / s h #t h i se x a m p l ei sW R O N G s e de' 1{ d

} '

s / . * / /

Clickheretogetfile:sed_bad_example.sh TheearlierexampleisacrudeversionoftheCpreprocessorprogram.Thefilethatisincludedhasapredeterminedname.Itwould beniceifsedallowedavariable(e.g"\1)"insteadofafixedfilename.Alas,seddoesn'thavethisability.Youcouldworkaround thislimitationbycreatingsedcommandsonthefly,orbyusingshellquotestopassvariablesintothesedscript.Supposeyou wantedtocreateacommandthatwouldincludeafilelikecpp,butthefilenameisanargumenttothescript.Anexampleofthis scriptis:

%i n c l u d e' s y s / p a r a m . h '< f i l e . c> f i l e . c . n e w

Ashellscripttodothiswouldbe:
# ! / b i n / s h #w a t c ho u tf o ra' / 'i nt h ep a r a m e t e r #u s ea l t e r n a t es e a r c hd e l i m i t e r s e de' \ _ # I N C L U D E< ' " $ 1 " ' > _ { r' " $ 1 " ' d } '

Letmeelaborate.Ifyouhadafilethatcontains
T e s tf i r s tf i l e # I N C L U D E< f i l e 1 > T e s ts e c o n df i l e # I N C L U D E< f i l e 2 >

youcouldusethecommand
s e d _ i n c l u d e 1 . s hf i l e 1 < i n p u t | s e d _ i n c l u d e 1 . s hf i l e 2

toincludethespecifiedfiles. Clickheretogetfile:sed_include1.sh

SunOSandthe#CommentCommand
Aswedigdeeperintosed,commentswillmakethecommandseasiertofollow.Theolderversionsofsedonlyallowonelineasa comment,anditmustbethefirstline.SunOS(andGNU'ssed)allowsmorethanonecomment,andthesecommentsdon'thaveto befirst.Thelastexamplecouldbe:
# ! / b i n / s h #w a t c ho u tf o ra' / 'i nt h ep a r a m e t e r #u s ea l t e r n a t es e a r c hd e l i m i t e r s e de' \ _ # I N C L U D E< ' " $ 1 " ' > _ { #r e a dt h ef i l e r' " $ 1 " ' #d e l e t ea n yc h a r a c t e r si nt h ep a t t e r ns p a c e #a n dr e a dt h en e x tl i n ei n d

} '

Clickheretogetfile:sed_include2.sh

Adding,Changing,Insertingnewlines
Sedhasthreecommandsusedtoaddnewlinestotheoutputstream.Becauseanentirelineisadded,thenewlineisonalineby itselftoemphasizethis.Thereisnooption,anentirelineisused,anditmustbeonitsownline.Ifyouarefamiliarwithmanyunix

utilities,youwouldexpectsedtouseasimilarconvention:linesarecontinuedbyendingthepreviouslinewitha"\".Thesyntaxto thesecommandsisfinicky,likethe"r"and"w"commands.

Appendalinewith'a'
The"a"commandappendsalineaftertherangeorpattern.Thisexamplewilladdalineaftereverylinewith"WORD:"
# ! / b i n / s h s e d' / W O R D /a \ A d dt h i sl i n ea f t e re v e r yl i n ew i t hW O R D '

Clickheretogetfile:sed_add_line_after_word.sh Youcouldeliminatetwolinesintheshellscriptifyouwish:
# ! / b i n / s h s e d' / W O R D /a \ A d dt h i sl i n ea f t e re v e r yl i n ew i t hW O R D '

Clickheretogetfile:sed_add_line_after_word1.sh Ipreferthefirstformbecauseit'seasiertoaddanewcommandbyaddinganewlineandbecausetheintentisclearer.Theremust notbeaspaceafterthe"\".

Insertalinewith'i'
Youcaninsertanewlinebeforethepatternwiththe"i"command:
# ! / b i n / s h s e d' / W O R D /i \ A d dt h i sl i n eb e f o r ee v e r yl i n ew i t hW O R D '

Clickheretogetfile:sed_add_line_before_word.sh

Changealinewith'c'
Youcanchangethecurrentlinewithanewline.
# ! / b i n / s h s e d' / W O R D /c \ R e p l a c et h ec u r r e n tl i n ew i t ht h el i n e '

Clickheretogetfile:sed_change_line.sh A"d"commandfollowedbya"a"commandwon'twork,asIdiscussedearlier.The"d"commandwouldterminatethecurrent actions.Youcancombineallthreeactionsusingcurlybraces:

# ! / b i n / s h s e d'

/ W O R D /{ i \ A d dt h i sl i n eb e f o r e a \ A d dt h i sl i n ea f t e r c \ C h a n g et h el i n et ot h i so n e } '

Clickheretogetfile:sed_insert_append_change.sh

Leadingtabsandspacesinasedscript
Sedignoresleadingtabsandspacesinallcommands.Howeverthesewhitespacecharactersmayormaynotbeignoredifthey startthetextfollowinga"a,""c"or"i"command.InSunOS,both"features"areavailable.TheBerkeley(andLinux)stylesedis in/usr/bin,andtheAT&Tversion(SystemV)isin/usr/5bin/. Toelaborate,the/usr/bin/sedcommandretainswhitespace,whilethe/usr/5bin/sedstripsoffleadingspaces.Ifyouwanttokeep leadingspaces,andnotcareaboutwhichversionofsedyouareusing,puta"\"asthefirstcharacteroftheline:
# ! / b i n / s h s e d' a \ \ T h i sl i n es t a r t sw i t hat a b '

Addingmorethanoneline
Allthreecommandswillallowyoutoaddmorethanoneline.Justendeachlinewitha"\:"
# ! / b i n / s h s e d' / W O R D /a \ A d dt h i sl i n e \ T h i sl i n e \ A n dt h i sl i n e '

Addinglinesandthepatternspace
Ihavementionedthepatternspacebefore.Mostcommandsoperateonthepatternspace,andsubsequentcommandsmayacton theresultsofthelastmodification.Thethreepreviouscommands,likethereadfilecommand,addthenewlinestotheoutput stream,bypassingthepatternspace.

Addressrangesandtheabovecommands
YoumayrememberinmylasttutorialIwarnedyouthatsomecommandscantakearangeoflines,andotherscannot.Tobe precise,thecommands"a,""i,""r,"and"q"willnottakearangelike"1,100"or"/begin/,/end/."Thedocumentationstatesthatthe readcommandcantakearange,butIgetanerrorwhenItrythis.The"c"orchangecommandallowsthis,anditwillletyou changeseverallinesintoone:
# ! / b i n / s h s e d' / b e g i n / , / e n d /c \ * * * D E L E T E D * * * '

Ifyouneedtodothis,youcanusethecurlybraces,asthatwillletyouperformtheoperationoneveryline:
# ! / b i n / s h #a d dab l a n kl i n ea f t e re v e r yl i n e s e d' 1 , ${ a \

} '

MultiLinePatterns
MostUNIXutilitiesarelineoriented.Regularexpressionsarelineoriented.Searchingforpatternsthatcoversmorethanonelineis notaneasytask.(Hint:Itwillbeveryshortly.) Sedreadsinalineoftext,performscommandswhichmaymodifytheline,andoutputsmodificationifdesired.Themainloopofa sedscriptlookslikethis: 1. Thenextlineisreadfromtheinputfileandplacesinthepatternspace.Iftheendoffileisfound,andifthereareadditional filestoread,thecurrentfileisclosed,thenextfileisopened,andthefirstlineofthenewfileisplacedintothepatternspace. 2. Thelinecountisincrementedbyone.Openinganewfiledoesnotresetthisnumber. 3. Eachsedcommandisexamined.Ifthereisarestrictionplacedonthecommand,andthecurrentlineinthepatternspace meetsthatrestriction,thecommandisexecuted.Somecommands,like"n"or"d"causesedtogotothetopoftheloop.The "q"commandcausessedtostop.Otherwisethenextcommandisexamined. 4. Afterallofthecommandsareexamined,thepatternspaceisoutputunlesssedhastheoptional"n"argument. Therestrictionbeforethecommanddeterminesifthecommandisexecuted.Iftherestrictionisapattern,andtheoperationisthe deletecommand,thenthefollowingwilldeletealllinesthathavethepattern:
/ P A T T E R N /d

Iftherestrictionisapairofnumbers,thenthedeletionwillhappenifthelinenumberisequaltothefirstnumberorgreaterthanthe firstnumberandlessthanorequaltothelastnumber:
1 0 , 2 0d

Iftherestrictionisapairofpatterns,thereisavariablethatiskeptforeachofthesepairs.Ifthevariableisfalseandthefirstpattern isfound,thevariableismadetrue.Ifthevariableistrue,thecommandisexecuted.Ifthevariableistrue,andthelastpatternison theline,afterthecommandisexecutedthevariableisturnedoff:

/ b e g i n / , / e n d /d

Whew!Thatwasamouthful.Ifyouhavereadcarefullyuptohere,youshouldhavebreezedthroughthis.Youmaywanttorefer back,becauseIcoveredseveralsubtlepoints.Mychoiceofwordswasdeliberate.Itcoverssomeunusualcases,like:
#w h a th a p p e n si ft h es e c o n dn u m b e r #i sl e s st h a nt h ef i r s tn u m b e r ? s e dn' 2 0 , 1p 'f i l e

and
#g e n e r a t ea1 0l i n ef i l ew i t hl i n en u m b e r s #a n ds e ew h a th a p p e n sw h e nt w op a t t e r n so v e r l a p y e s|h e a d1 0|c a tn|\ s e dne' / 1 / , / 7 /p 'e' / 5 / , / 9 /p '

Enoughmentalpunishment.Hereisanotherreview,thistimeinatableformat.Assumetheinputfilecontainsthefollowinglines:
A B C D E F G H I J

Whensedstartsup,thefirstlineisplacedinthepatternspace.Thenextlineis"CD."Theoperationsofthe"n,""d,"and"p" commandscanbesummarizedas:
+ + + + | P a t t e r n N e x t |C o m m a n d|O u t p u t N e wP a t t e r n N e wN e x t| | S p a c e I n p u t| | S p a c e I n p u t | + + + + | A B C D |n |< d e f a u l t > C D E F | | A B C D |d |C D E F | | A B C D |p |A B C D E F | + + + +

The"n"commandmayormaynotgenerateoutputdependingupontheexistenceofthe"n"flag. Thatreviewisalittleeasiertofollow,isn'tit?BeforeIjumpintomultilinepatterns,Iwantedtocoverthreemorecommands:

Printlinenumberwith=
The"="commandprintsthecurrentlinenumbertostandardoutput.Onewaytofindoutthelinenumbersthatcontainapatternis touse:
#a d dl i n en u m b e r sf i r s t , #t h e nu s eg r e p , #t h e nj u s tp r i n tt h en u m b e r c a tnf i l e|g r e p' P A T T E R N '|a w k' { p r i n t$ 1 } '

Thesedsolutionis:
s e dn' / P A T T E R N /= 'f i l e

EarlierIusedthefollowingtofindthenumberoflinesinafile
# ! / b i n / s h l i n e s = ` w clf i l e|a w k' { p r i n t$ 1 } '`

Usingthe"="commandcansimplifythis:
# ! / b i n / s h l i n e s = ` s e dn' $ = 'f i l e`

The"="commandonlyacceptsoneaddress,soifyouwanttoprintthenumberforarangeoflines,youmustusethecurlybraces:
# ! / b i n / s h #J u s tp r i n tt h el i n en u m b e r s s e dn' / b e g i n / , / e n d /{ = d } 'f i l e

Sincethe"="commandonlyprintstostandardoutput,youcannotprintthelinenumberonthesamelineasthepattern.Youneed toeditmultilinepatternstodothis.

Transformwithy
Ifyouwantedtochangeawordfromlowercasetouppercase,youcouldwrite26charactersubstitutions,converting"a"to"A," etc.Sedhasacommandthatoperateslikethetrprogram.Itiscalledthe"y"command.Forinstance,tochangetheletters"a" through"f"intotheiruppercaseform,use:
s e d' y / a b c d e f / A B C D E F / 'f i l e

Icouldhaveusedanexamplethatconvertedall26lettersintouppercase,andwhilethiscolumncoversabroadrangeoftopics, the"column"prefersanarrowerformat. Ifyouwantedtoconvertalinethatcontainedahexadecimalnumber(e.g.0x1aff)touppercase(0x1AFF),youcoulduse:

s e d' / 0 x [ 0 9 a z A Z ] * /y / a b c d e f / A B C D E F 'f i l e

Thisworksfineifthereareonlynumbersinthefile.Ifyouwantedtochangethesecondwordinalinetouppercase,youareout ofluckunlessyouusemultilineediting.(HeyIthinkthereissomesortofthemehere!)

Displayingcontrolcharacterswithal
The"l"commandprintsthecurrentpatternspace.Itisthereforeusefulindebuggingsedscripts.Italsoconvertsunprintable charactersintoprintingcharactersbyoutputtingthevalueinoctalprecededbya"\"character.Ifounditusefultoprintoutthe currentpatternspace,whileprobingthesubtletiesofsed.

WorkingwithMultipleLines
Therearethreenewcommandsusedinmultiplelinepatterns:"N,""D,"and"P."Iwillexplaintheirrelationtothematching"n," "d,"and"p"singlelinecommands. The"n"commandwillprintoutthecurrentpatternspace(unlessthe"n"flagisused),emptythecurrentpatternspace,andreadin thenextlineofinput.The"N"commanddoesnotprintoutthecurrentpatternspaceanddoesnotemptythepatternspace.Itreads inthenextline,butappendsanewlinecharacteralongwiththeinputlineitselftothepatternspace. The"d"commanddeletedthecurrentpatternspace,readsinthenextline,putsthenewlineintothepatternspace,andabortsthe currentcommand,andstartsexecutionatthefirstsedcommand.Thisiscalledstartinganew"cycle."The"D"commanddeletes thefirstportionofthepatternspace,uptothenewlinecharacter,leavingtherestofthepatternalone.Like"d,"itstopsthecurrent commandandstartsthecommandcycleoveragain.However,itwillnotprintthecurrentpatternspace.Youmustprintityourself, astepearlier.Ifthe"D"commandisexecutedwithagroupofothercommandsinacurlybrace,commandsafterthe"D" commandareignored.Thenextgroupofsedcommandsisexecuted,unlessthepatternspaceisemptied.Ifthishappens,thecycle isstartedfromthetopandanewlineisread. The"p"commandprintstheentirepatternspace.The"P"commandonlyprintsthefirstpartofthepatternspace,uptothe NEWLINEcharacter.Neitherthe"p"northe"P"commandchangesthepatternsspace. Someexamplesmightdemonstrate"N"byitselfisn'tveryuseful.thefilter
s e de' N '

doesn'tmodifytheinputstream.Instead,itcombinesthefirstandsecondline,thenprintsthem,combinesthethirdandfourthline, andprintsthem,etc.Itdoesallowyoutouseanew"anchor"character:"\n."Thismatchesthenewlinecharacterthatseparates multiplelinesinthepatternspace.Ifyouwantedtosearchforalinethatendedwiththecharacter"#,"andappendthenextlineto it,youcoulduse

# ! / b i n / s h s e d' #l o o kf o ra" # "a tt h ee n do ft h el i n e / # $ /{ #F o u n do n e-n o wr e a di nt h en e x tl i n e N #d e l e t et h e" # "a n dt h en e wl i n ec h a r a c t e r , s / # \ n / / } 'f i l e

Youcouldsearchfortwolinescontaining"ONE"and"TWO"andonlyprintoutthetwoconsecutivelines:
# ! / b i n / s h s e dn' / O N E /{ #f o u n d" O N E "-r e a di nn e x tl i n e N #l o o kf o r" T W O "o nt h es e c o n dl i n e #a n dp r i n ti ft h e r e . / \ n . * T W O /p } 'f i l e

Thenextexamplewoulddeleteeverythingbetween"ONE"and"TWO:"
# ! / b i n / s h s e d' / O N E /{ #a p p e n dal i n e N #s e a r c hf o rT W Oo nt h es e c o n dl i n e / \ n . * T W O /{ #f o u n di t-n o we d i tm a k i n go n el i n e s / O N E . * \ n . * T W O / O N ET W O / } } 'f i l e

Youcaneithersearchforaparticularpatternontwoconsecutivelines,oryoucansearchfortwoconsecutivewordsthatmaybe splitonalineboundary.Thenextexamplewilllookfortwowordswhichareeitheronthesamelineoroneisontheendofaline andthesecondisonthebeginningofthenextline.Iffound,thefirstwordisdeleted:

# ! / b i n / s h s e d' / O N E /{ #a p p e n dal i n e N #" O N ET W O "o ns a m el i n e s / O N ET W O / T W O / #" O N E #T W O "o nt w oc o n s e c u t i v el i n e s s / O N E \ n T W O / T W O / } 'f i l e

Let'susethe "D"command,andifwefindalinecontaining "TWO"immediatelyafteralinecontaining "ONE,"thendeletethefirstline:

# ! / b i n / s h s e d' / O N E /{ #a p p e n dal i n e N #i fT W Of o u n d ,d e l e t et h ef i r s tl i n e / \ n . * T W O /D } 'f i l e

Clickheretogetfile:sed_delete_line_after_word.sh Ifwewantedtoprintthefirstlineinsteadofdeletingit,andnotprinteveryotherline,changethe"D"toa"P"andadda"n"asan argumenttosed:

# ! / b i n / s h s e dn' #b yd e f a u l t-d on o tp r i n ta n y t h i n g / O N E /{ #a p p e n dal i n e N #i fT W Of o u n d ,p r i n tt h ef i r s tl i n e / \ n . * T W O /P } 'f i l e

Clickheretogetfile:sed_print_line_after_word.sh Itisverycommontocombineallthreemultilinecommands.Thetypicalorderis"N,""P"andlastly"D."Thisonewilldelete everythingbetween"ONE"and"TWO"iftheyareononeortwoconsecutivelines:

# ! / b i n / s h s e d' / O N E /{ #a p p e n dt h en e x tl i n e N #l o o kf o r" O N E "f o l l o w e db y" T W O " / O N E . * T W O /{ # d e l e t ee v e r y t h i n gb e t w e e n s / O N E . * T W O / O N ET W O / # p r i n t P # t h e nd e l e t et h ef i r s tl i n e D } } 'f i l e

Clickheretogetfile:sed_delete_between_two_words.sh EarlierItalkedaboutthe"="command,andusingittoaddlinenumberstoafile.Youcanusetwoinvocationsofsedtodothis

(althoughitispossibletodoitwithone,butthatmustwaituntilnextsection).Thefirstsedcommandwilloutputalinenumberon oneline,andthenprintthelineonthenextline.Thesecondinvocationofsedwillmergethetwolinestogether:
# ! / b i n / s h s e d' = 'f i l e|\ s e d' { N s / \ n // } '

Clickheretogetfile:sed_merge_two_lines.sh Ifyoufinditnecessary,youcanbreakonelineintotwolines,editthem,andmergethemtogetheragain.Asanexample,ifyou hadafilethathadahexadecimalnumberfollowedbyaword,andyouwantedtoconvertthefirstwordtoalluppercase,youcan usethe"y"command,butyoumustfirstsplitthelineintotwolines,changeoneofthetwo,andmergethemtogether.Thatis,a linecontaining

0 x 1 f f ft a b l e 2

willbechangedintotwolines:
0 x 1 f f f t a b l e 2

andthefirstlinewillbeconvertedintouppercase.Iwillusetrtoconvertthespaceintoanewline:
# ! / b i n / s h t r''' \ 0 1 2 'f i l e | s e d'{ y / a b c d e f / A B C D E F / N s / \ n // } '

Clickheretogetfile:sed_split.sh Itisn'tobvious,butsedcouldbeusedinsteadoftr.Youcanembedanewlineinasubstitutecommand,butyoumustescapeit withabackslash.Itisunfortunatethatyoumustuse"\n"intheleftsideofasubstitutecommand,andanembeddednewlineinthe righthandside.Heavysigh.Hereistheexample:

# ! / b i n / s h s e d' s // \ / '|\ s e d'{ y / a b c d e f / A B C D E F / N s / \ n // } '

Clickheretogetfile:sed_split_merge.sh SometimesIaddaspecialcharacterasamarker,andlookforthatcharacterintheinputstream.Whenfound,itindicatestheplace ablankusedtobe.Abackslashisagoodcharacter,exceptitmustbeescapedwithabackslash,andmakesthesedscriptobscure. Saveitforthatguywhokeepsaskingdumbquestions.Thesedscripttochangeablankintoa"\"followingbyanewlinewould be: #!/bin/sh sed's//\\\ /'file Clickheretogetfile:sed_addslash_before_blank.sh Yeah.That'stheticket.OrusetheCshellandreallyconfusehim!

#!/bin/cshf sed'\ s//\\\\ /'file Clickheretogetfile:sed_addslash_before_blank.csh Afewmoreexamplesofthat,andhe'llneveraskyouaquestionagain!IthinkI'mgettingcarriedaway.I'llsummarizewithachart thatcoversthefeatureswe'vetalkedabout:

+ + + + | P a t t e r n N e x t |C o m m a n d|O u t p u t N e wP a t t e r n N e wN e x t| | S p a c e I n p u t| | S p a c e I n p u t | + + + + | A B C D |n |< d e f a u l t > C D E F | | A B C D |N |A B \ n C D E F | | A B C D |d |E F | | A B C D |D |E F | | A B C D |p |A B A B C D |( C o r r e c t e d ) | A B C D |P |A B A B C D |( C o r r e c t e d ) + + + + | A B \ n C D E F |n |< d e f a u l t > E F G H | | A B \ n C D E F |N |A B \ n C D \ n E F G H | | A B \ n C D E F |d |E F G H | | A B \ n C D E F |D |C D E F | | A B \ n C D E F |p |A B \ n C D A B \ n C D E F | | A B \ n C D E F |P |A B A B \ n C D E F | + + + +

Usingnewlinesinsedscripts
Occasionallyonewishestouseanewlinecharacterinasedscript.Well,thishassomesubtleissueshere.Ifonewantstosearch foranewline,onehastouse"\n."Hereisanexamplewhereyousearchforaphrase,anddeletethenewlinecharacterafterthat phrasejoiningtwolinestogether.
( e c h oa ; e c h ox ; e c h oy )|s e d' / x $ /{ N s : x \ n : x : } '

whichgenerates
a x y

However,ifyouareinsertinganewline,don'tuse"\n"insteadinsertaliteralnewlinecharacter:
( e c h oa ; e c h ox ; e c h oy )|s e d' s : x : X \ : '

generates
a X y

TheHoldBuffer
Sofarwehavetalkedaboutthreeconceptsofsed:(1)Theinputstreamordatabeforeitismodified,(2)theoutputstreamordata afterithasbeenmodified,and(3)thepatternspace,orbuffercontainingcharactersthatcanbemodifiedandsendtotheoutput stream. Thereisonemore"location"tobecovered:theholdbufferorholdspace.Thinkofitasasparepatternbuffer.Itcanbeusedto "copy"or"remember"thedatainthepatternspaceforlater.Therearefivecommandsthatusetheholdbuffer.

Exchangewithx
The"x"commandeXchangesthepatternspacewiththeholdbuffer.Byitself,thecommandisn'tuseful.Executingthesed command
s e d' x '

asafilteraddsablanklineinthefront,anddeletesthelastline.Itlookslikeitdidn'tchangetheinputstreamsignificantly,butthe sedcommandismodifyingeveryline. Theholdbufferstartsoutcontainingablankline.Whenthe"x"commandmodifiesthefirstline,line1issavedintheholdbuffer, andtheblanklinetakestheplaceofthefirstline.Thesecond"x"commandexchangesthesecondlinewiththeholdbuffer,which containsthefirstline.Eachsubsequentlineisexchangedwiththeprecedingline.Thelastlineisplacedintheholdbuffer,andis notexchangedasecondtime,soitremainsintheholdbufferwhentheprogramterminates,andnevergetsprinted.Thisillustrates thatcaremustbetakenwhenstoringdataintheholdbuffer,becauseitwon'tbeoutputunlessyouexplicitlyrequestit.

ExampleofContextGrep
Oneuseoftheholdbufferistorememberpreviouslines.Anexampleofthisisautilitythatactslikegrepasitshowsyouthelines thatmatchapattern.Inaddition,itshowsyouthelinebeforeandafterthepattern.Thatis,ifline8containsthepattern,thisutility wouldprintlines7,8and9. Onewaytodothisistoseeifthelinehasthepattern.Ifitdoesnothavethepattern,putthecurrentlineintheholdbuffer.Ifit does,printthelineintheholdbuffer,thenthecurrentline,andthenthenextline.Aftereachset,threedashesareprinted.The scriptchecksfortheexistenceofanargument,andifmissing,printsanerror.Passingtheargumentintothesedscriptisdoneby turningoffthesinglequotemechanism,insertingthe"$1"intothescript,andstartingupthesinglequoteagain:

# ! / b i n / s h #g r e p 3-p r i n t so u tt h r e el i n e sa r o u n dp a t t e r n #i ft h e r ei so n l yo n ea r g u m e n t ,e x i t c a s e$ #i n 1 ) ; ; * )e c h o" U s a g e :$ 0p a t t e r n " ; e x i t ; ; e s a c ; #Ih o p et h ea r g u m e n td o e s n ' tc o n t a i na/ #i fi td o e s ,s e dw i l lc o m p l a i n #u s es e dnt od i s a b l ep r i n t i n g #u n l e s sw ea s kf o ri t s e dn' ' / $ 1 / '! { # n om a t c h-p u tt h ec u r r e n tl i n ei nt h eh o l db u f f e r x #d e l e t et h eo l do n e ,w h i c hi s #n o wi nt h ep a t t e r nb u f f e r d } ' / $ 1 / '{ #am a t c h-g e tl a s tl i n e x #p r i n ti t p #g e tt h eo r i g i n a ll i n eb a c k x #p r i n ti t p #g e tt h en e x tl i n e n #p r i n ti t p #n o wa d dt h r e ed a s h e sa sam a r k e r a \ #n o wp u tt h i sl i n ei n t ot h eh o l db u f f e r x } '

} '

Clickheretogetfile:grep3.sh Youcouldusethistoshowthethreelinesaroundakeyword,i.e.:
g r e p 3v t 1 0 0< / e t c / t e r m c a p

HoldwithhorH
The"x"commandexchangestheholdbufferandthepatternbuffer.Botharechanged.The"h"commandcopiesthepatternbuffer intotheholdbuffer.Thepatternbufferisunchanged.Anidenticalscripttotheaboveusestheholdcommands:

# ! / b i n / s h #g r e p 3v e r s i o nb-a n o t h e rv e r s i o nu s i n gt h eh o l dc o m m a n d s #i ft h e r ei so n l yo n ea r g u m e n t ,e x i t c a s e$ #i n 1 ) ; ; * )e c h o" U s a g e :$ 0p a t t e r n " ; e x i t ; ; e s a c ; #a g a i n-Ih o p et h ea r g u m e n td o e s n ' tc o n t a i na/ #u s es e dnt od i s a b l ep r i n t i n g s e dn' ' / $ 1 / '! { #p u tt h en o n m a t c h i n gl i n ei nt h eh o l db u f f e r h } ' / $ 1 / '{ #f o u n dal i n et h a tm a t c h e s #a p p e n di tt ot h eh o l db u f f e r H #t h eh o l db u f f e rc o n t a i n s2l i n e s #g e tt h en e x tl i n e n #a n da d di tt ot h eh o l db u f f e r H #n o wp r i n ti tb a c kt ot h ep a t t e r ns p a c e x #a n dp r i n ti t . p #a d dt h et h r e eh y p h e n sa sam a r k e r a \ } '

Clickheretogetfile:grep3a.sh

Keepingmorethanonelineintheholdbuffer
The"H"commandallowsyoutocombineseverallinesintheholdbuffer.Itactslikethe"N"commandaslinesareappendedto thebuffer,witha"\n"betweenthelines.Youcansaveseverallinesintheholdbuffer,andprintthemonlyifaparticularpatternis foundlater. Asanexample,takeafilethatusesspacesasthefirstcharacterofalineasacontinuationcharacter.Thefiles/etc/termcap, /etc/printcap,makefileandmailmessagesusespacesortabstoindicateacontinuingofanentry.Ifyouwantedtoprinttheentry beforeaword,youcouldusethisscript.Iusea"^I"toindicateanactualtabcharacter:

# ! / b i n / s h #p r i n tp r e v i o u se n t r y s e dn' / ^ [^ I ] / ! { #l i n ed o e sn o ts t a r tw i t has p a c eo rt a b , #d o e si th a v et h ep a t t e r nw ea r ei n t e r e s t e di n ? ' / $ 1 / '{ #y e si td o e s .p r i n tt h r e ed a s h e s i \ #g e th o l db u f f e r ,s a v ec u r r e n tl i n e x #n o wp r i n tw h a tw a si nt h eh o l db u f f e r p #g e tt h eo r i g i n a ll i n eb a c k x } #s t o r ei ti nt h eh o l db u f f e r h } #w h a ta b o u tl i n e st h a ts t a r t #w i t has p a c eo rt a b ? / ^ [^ I ] /{ #a p p e n di tt ot h eh o l db u f f e r H } '

Clickheretogetfile:grep_previous.sh Youcanalsousethe"H"toextendthecontextgrep.Inthisexample,theprogramprintsoutthetwolinesbeforethepattern, insteadofasingleline.Themethodtolimitthistotwolinesistousethe"s"commandtokeeponenewline,anddeletingextra lines.Icallitgrep4:

# ! / b i n / s h #g r e p 4 :p r i n t so u t4l i n e sa r o u n dp a t t e r n #i ft h e r ei so n l yo n ea r g u m e n t ,e x i t c a s e$ #i n 1 ) ; ; * )e c h o" U s a g e :$ 0p a t t e r n " ; e x i t ; ; e s a c ; s e dn' ' / $ 1 / '! { #d o e sn o tm a t c h-a d dt h i sl i n et ot h eh o l ds p a c e H #b r i n gi tb a c ki n t ot h ep a t t e r ns p a c e x #T w ol i n e sw o u l dl o o kl i k e. * \ n . * #T h r e el i n e sl o o kl i k e. * \ n . * \ n . * #D e l e t ee x t r al i n e s-k e e pt w o s / ^ . * \ n \ ( . * \ n . * \ ) $ / \ 1 / #n o wp u tt h et w ol i n e s( a tm o s t )i n t o #t h eh o l db u f f e ra g a i n x } ' / $ 1 / '{ #m a t c h e s-a p p e n dt h ec u r r e n tl i n e H #g e tt h en e x tl i n e n #a p p e n dt h a to n ea l s o H #b r i n gi tb a c k ,b u tk e e pt h ec u r r e n tl i n ei n #t h eh o l db u f f e r .T h i si st h el i n ea f t e rt h ep a t t e r n , #a n dw ew a n tt op l a c ei ti nh o l di nc a s et h en e x tl i n e #h a st h ed e s i r e dp a t t e r n x #p r i n tt h e4l i n e s p #a d dt h em a r k a \

} '

Clickheretogetfile:grep4.sh Youcanmodifythistoprintanynumberoflinesaroundapattern.Asyoucansee,youmustrememberwhatisintheholdspace, andwhatisinthepatternspace.Thereareotherwaystowritethesameroutine.

GetwithgorG
Insteadofexchangingtheholdspacewiththepatternspace,youcancopytheholdspacetothepatternspacewiththe"g" command.Thisdeletesthepatternspace.Ifyouwanttoappendtothepatternspace,usethe"G"command.Thisaddsanewline tothepatternspace,andcopiestheholdspaceafterthenewline. Hereisanotherversionofthe"grep3"command.Itworksjustlikethepreviousone,butisimplementeddifferently.Thisillustrates thatsedhasmorethanonewaytosolvemanyproblems.Whatisimportantisyouunderstandyourproblem,anddocumentyour solution:

# ! / b i n / s h #g r e p 3v e r s i o nc :u s e' G ' i n s t e a do fH #i ft h e r ei so n l yo n ea r g u m e n t ,e x i t c a s e$ #i n 1 ) ; ; * )e c h o" U s a g e :$ 0p a t t e r n " ; e x i t ; ; e s a c ; #a g a i n-Ih o p et h ea r g u m e n td o e s n ' tc o n t a i na/ s e dn' ' / $ 1 / '! { #p u tt h en o n m a t c h i n gl i n ei nt h eh o l db u f f e r h } ' / $ 1 / '{ #f o u n dal i n et h a tm a t c h e s #a d dt h en e x tl i n et ot h ep a t t e r ns p a c e N #e x c h a n g et h ep r e v i o u sl i n ew i t ht h e #2i np a t t e r ns p a c e x #n o wa d dt h et w ol i n e sb a c k G #a n dp r i n ti t . p #a d dt h et h r e eh y p h e n sa sam a r k e r a \ #r e m o v ef i r s t2l i n e s s / . * \ n . * \ n \ ( . * \ ) $ / \ 1 / #a n dp l a c ei nt h eh o l db u f f e rf o rn e x tt i m e h } '

Clickheretogetfile:grep3c.sh The"G"commandmakesiteasytohavetwocopiesofaline.Supposeyouwantedtotheconvertthefirsthexadecimalnumberto uppercase,anddon'twanttousethescriptIdescribedinanearliercolumn

# ! / b i n / s h #c h a n g et h ef i r s th e xn u m b e rt ou p p e rc a s ef o r m a t #u s e ss e dt w i c e #u s e da saf i l t e r #c o n v e r t 2 u c< i n> o u t

s e d' s // \ / '|\ s e d'{

} '

y / a b c d e f / A B C D E F / N s / \ n //

Clickheretogetfile:convert2uc.sh Hereisasolutionthatdoesnotrequiretwoinvocationsofsed:
# ! / b i n / s h #c o n v e r t 2 u cv e r s i o nb #c h a n g et h ef i r s th e xn u m b e rt ou p p e rc a s ef o r m a t #u s e ss e do n c e #u s e da saf i l t e r #c o n v e r t 2 u c< i n> o u t s e d' { #r e m e m b e rt h el i n e h # c h a n g et h ec u r r e n tl i n et ou p p e rc a s e y / a b c d e f / A B C D E F / #a d dt h eo l dl i n eb a c k G #K e e pt h ef i r s tw o r do ft h ef i r s tl i n e , #a n ds e c o n dw o r do ft h es e c o n dl i n e #w i t ho n eh u m o n g e o u sr e g u l a re x p r e s s i o n s / ^ \ ( [ ^] * \ ). * \ n [ ^] *\ ( . * \ ) / \ 1\ 2 / } '

Clickheretogetfile:convert2uc1.sh CarlHenrikLundesuggestedawaytomakethissimpler.Iwasworkingtoohard.
# ! / b i n / s h #c o n v e r t 2 u cv e r s i o nb #c h a n g et h ef i r s th e xn u m b e rt ou p p e rc a s ef o r m a t #u s e ss e do n c e #u s e da saf i l t e r #c o n v e r t 2 u c< i n> o u t s e d' { #r e m e m b e rt h el i n e h # c h a n g et h ec u r r e n tl i n et ou p p e rc a s e y / a b c d e f / A B C D E F / #a d dt h eo l dl i n eb a c k G #K e e pt h ef i r s tw o r do ft h ef i r s tl i n e , #a n ds e c o n dw o r do ft h es e c o n dl i n e #w i t ho n eh u m o n g e o u sr e g u l a re x p r e s s i o n s /. *//#d e l e t ea l lb u tt h ef i r s ta n dl a s tw o r d } '

Clickheretogetfile:convert2uc2.sh Thisexampleonlyconvertstheletters"a"through"f"touppercase.Thiswaschosentomakethescripteasiertoprintinthese narrowcolumns.Youcaneasilymodifythescripttoconvertallletterstouppercase,ortochangethefirstletter,secondword,etc.

FlowControl
Asyoulearnaboutsedyourealizethatithasitsownprogramminglanguage.Itistruethatit'saveryspecializedandsimple language.Whatlanguagewouldbecompletewithoutamethodofchangingtheflowcontrol?Therearethreecommandsseduses forthis.Youcanspecifyalabelwithantextstringprecededbyacolon.The"b"commandbranchestothelabel.Thelabelfollows thecommand.Ifnolabelisthere,branchtotheendofthescript.The"t"commandisusedtotestconditions.BeforeIdiscussthe

"t"command,Iwillshowyouanexampleusingthe"b"command. Thisexampleremembersparagraphs,andifitcontainsthepattern(specifiedbyanargument),thescriptprintsouttheentire paragraph.

# ! / b i n / s h s e dn' #i fa ne m p t yl i n e ,c h e c kt h ep a r a g r a p h / ^ $ /bp a r a #e l s ea d di tt ot h eh o l db u f f e r H #a te n do ff i l e ,c h e c kp a r a g r a p h $bp a r a #n o wb r a n c ht oe n do fs c r i p t b #t h i si sw h e r eap a r a g r a p hi sc h e c k e df o rt h ep a t t e r n : p a r a #r e t u r nt h ee n t i r ep a r a g r a p h #i n t ot h ep a t t e r ns p a c e x #l o o kf o rt h ep a t t e r n ,i ft h e r e-p r i n t / ' $ 1 ' /p '

Clickheretogetfile:grep_paragraph.sh

Testingwitht
Youcanexecuteabranchifapatternisfound.Youmaywanttoexecuteabranchonlyifasubstitutionismade.Thecommand"t label"willbranchtothelabelifthelastsubstitutecommandmodifiedthepatternspace. Oneuseforthisisrecursivepatterns.Supposeyouwantedtoremovewhitespaceinsideparenthesis.Theseparenthesesmightbe nested.Thatis,youwouldwanttodeleteastringthatlookedlike"(((())))."Thesedexpressions
s e d' s / ( [^ I ] * ) / g '

wouldonlyremovetheinnermostset.Youwouldhavetopipethedatathroughthescriptfourtimestoremoveeachsetor parenthesis.Youcouldusetheregularexpression
s e d' s / ( [^ I ( ) ] * ) / g '

butthatwoulddeletenonmatchingsetsofparenthesis.The"t"commandwouldsolvethis:
# ! / b i n / s h s e d' : a g a i n s / ( [^ I ] * ) / / ta g a i n '

Anearlierversionhada'g'afterthe's'expression.Thisisnotneeded. Clickheretogetfile:delete_nested_parens.sh

Analternatewayofaddingcomments
Thereisonewaytoaddcommentsinasedscriptifyoudon'thaveaversionthatsupportsit.Usethe"a"commandwiththeline numberofzero:
# ! / b i n / s h s e d' / b e g i n /{ 0 i \ T h i si sac o m m e n t \

} '

I tc a nc o v e rs e v e r a ll i n e s \ I tw i l lw o r kw i t ha n yv e r s i o no fs e d

Clickheretogetfile:sed_add_comments.sh

Thepoorlydocumented
Thereisonemoresedcommandthatisn'twelldocumented.Itisthe""command.Thiscanbeusedtocombinedseveralsed commandsononeline.Hereisthegrep4scriptIdescribedearlier,butwithoutthecommentsorerrorcheckingandwith semicolonsbetweencommands: #!/bin/sh sedn' '/$1/'!{Hxs/^.*\n\(.*\n.*\)$/\1/x} '/$1/'{HnHxpa\ }' Clickheretogetfile:grep4a.sh Yessireebob!Definitelycharacterbuilding.IthinkIhavemademypoint.AsfarasIamconcerned,theonlytimethesemicolonis usefuliswhenyouwanttotypethesedscriptonthecommandline.Ifyouaregoingtoplaceitinascript,formatitsoitis readable.Ihavementionedearlierthatmanyversionsofseddonotsupportcommentsexceptonthefirstline.Youmaywantto writeyourscriptswithcommentsinthem,andinstallthemin"binary"formwithoutcomments.Thisshouldnotbedifficult.After all,youhavebecomeasedgurubynow.Iwon'teventellyouhowtowriteascripttostripoutcomments.Thatwouldbeinsulting yourintelligence.AlsosomeoperatingsystemsdoNOTletyouusesemicolons.Soifyouseeascriptwithsemicolons,andit doesnotworkonanonLinuxsystem,replacethesemicolonwithanewlinecharacter.(Aslongasyouarenotusingcsh/tcsh,but that'sanothertopic.

Passingregularexpressionsasarguments
Intheearlierscripts,Imentionedthatyouwouldhaveproblemsifyoupassedanargumenttothescriptthathadaslashinit.In fact,regularexpressionmightcauseyouproblems.Ascriptlikethefollowingisaskingtobebrokensomeday:
# ! / b i n / s h s e d' s / ' " $ 1 " ' / / g '

Iftheargumentcontainsanyofthesecharactersinit,youmaygetabrokenscript:"/\.*[]^$"Forinstance,ifsomeonetypesa"/" thenthesubstiturecommandwillseefourdelimitersinsteadofthree.Youwillalsogetsyntaxerrorsifyouprovidea"]"withouta "]".Onesolutionistohavetheuserputabackslashbeforeanyofthesecharacterswhentheypassitasanargument.However,the userhastoknowwhichcharactersarespecial. Anothersolutionistoaddabackslashbeforeeachofthosecharactersinthescript

# ! / b i n / s h a r g = ` e c h o" $ 1 "|s e d' s : [ ] \ [ \ ^ \ $ \ . \ * \ / ] : \ \ \ \ & : g ' ` s e d' s / ' " $ a r g " ' / / g '

Clickheretogetfile:sed_with_regular_expressions1.sh Ifyouweresearchingforthepattern"^../,"thescriptwouldconvertthisinto"\^\.\.\/"beforepassingittosed.

CommandSummary
AsIpromisedearlier,hereisatablethatsummarizesthedifferentcommands.Thesecondcolumnspecifiesifthecommandcan havearangeorpairofaddresses(witha2)orasingleaddressorpattern(witha1).Thenextfourcolumnsspecifieswhichofthe fourbuffersorstreamsaremodifiedbythecommand.Somecommandsonlyaffecttheoutputstream,othersonlyaffectthehold buffer.Ifyourememberthatthepatternspaceisoutput(unlessa"n"wasgiventosed),thistableshouldhelpyoukeeptrackof

thevariouscommands.
+ + | C o m m a n d A d d r e s s M o d i f i c a t i o n st o | | o rR a n g e I n p u t O u t p u t P a t t e r n H o l d | | S t r e a m S t r e a m S p a c e B u f f e r| + + | = Y | | a 1 Y | | b 2 | | c 2 Y | | d 2 Y Y | | D 2 Y Y | | g 2 Y | | G 2 Y | | h 2 Y | | H 2 Y | | i 1 Y | | l 1 Y | | n 2 Y * | | N 2 Y Y | | p 2 Y | | P 2 Y | | q 1 | | r 1 Y | | s 2 Y | | t 2 | | w 2 Y | | x 2 Y Y | | y 2 Y | + +

The"n"commandmayormaynotgenerateoutput,dependingonthe"n"option.The"r"commandcanonlyhaveoneaddress, despitethedocumentation. CheckoutmynewSedReferenceChart

InConclusion
Thisconcludesmytutorialonsed.Itispossibletofindshorterformsofsomeofmyscripts.However,Ichosetheseexamplesto illustratesomebasicconstructs.Iwantedclarity,notobscurity.Ihopeyouenjoyedit.

MoreReferences
Thisconcludesmytutorialonsed.OtherofmyUnixshelltutorialscanbefoundhere.Othershelltutorialscanbefoundat Heiner'sSHELLdoradoandChrisF.A.Johnson'sUnixShellPage TheWikipediaEntryonSED SEDoneliners Anddon'tforgetTheSEDFAQ ThisdocumentwasoriginallyconvertedfromNROFFtoTEXTtoHTML. Pleaseforgiveerrorsinthetranslation. Ifyouareconfused,grabtheactualscriptifpossible.Notranslationsoccurredinthescripts.

Thanksforthefeedback,gang
ThankstoKeelanEvans,FredrikNilsson,andKurtMcKeeforspottingsometypos. ThankstoWimStolkerandJose'Sebrosaaswell. ThankstoOlivierMengue. ThankstoAndrewM.Goth. ThankstoDavidP.Brown. ThankstoAxelSchulzeforsomecorrections ThankstoMartinJanforthecorrectionsinsedformat(grin)

AbigthanksforFazlRahmanforspottingdozensoferrors. ThankstoCarlHenrikLundewhosuggestedanimprovementtoconvert2uc1.sh AbigthankstoBryanHyunHuhwhospottedanerrorinthetableandreferencechart Thanksforinputfrom

M a r t e nJ a n G o r d o nW i l s o n T o mK o n a n t z P e t e rB r a t t o n G r a n tR o o t K e i t hB r i g g s Z o l t a nM i k l o s P e g g yR u s s e l l L o r e n sK o c k k u m . n e t J o h nP o u l i n R i h a r d s C o r e yR i c h a r d s o n E r i cM a t h i s o n I l d a rM u l y u k o v T o mZ h u

Thisdocumentwastranslatedbytroff2htmlv0.21onSeptember22,2001andthenmanuallyeditedtomakeitcompliantwith: