Regular Expressions

This article will give you an introduction to the world of regular expressions. I'll start off with explaining what regular expressions are and introduce it's syntax with some examples of varying complexity.

Concept
A regular expression is a text pattern consisting of a combination of alphanumeric characters and special characters known as metacharacters. A close relative is in fact the wildcard expression which are often used in file management. The pattern is used to match against text strings. The result of a match is either successful or not, however when a match is successful not all of the pattern must match, this is explained later in the article. You'll find that regular expressions are used in different ways: Regular text match, search and replace etc. Regular expressions are often simply called RegExp, but for consistency I'll be referring to it with it's full name. Due to the versatility of the regular expression it is widely used in text processing and parsing. UNIX users are probably familiar with them trough the use of the programs such as grep. Text editors such as (X)Emacs and vi also use them heavily. Probably the most known use of regular expressions are in the programming language Perl, you'll find that Perl sports the most advanced regular expression implementation to this day.

Usage
Now you're probably wondering why you should bother to learn regular expressions. Well, if you're a normal computer user, your benefits from using them are somewhat small, unless you use BrownRecluse. However if you're a developer, you'll find that knowing regular expressions will make your life so much better.

Syntax of Regular Expressions

The following section will explain in details the syntax of Regular Expressions. Examples are provided for clarity. Introduction Regular Expressions are a widely-used method of specifying patterns of text to search for. Special metacharacters allow you to specify, for instance, that a particular string you are looking for occurs at the beginning or end of a line, or contains n recurrences of a certain character. Regular expressions look ugly for novices, but really they are very simple (well, usually simple), it's a handy and powerful tool. Simple matches Any single character matches itself, unless it is a metacharacter with a special meaning described below. A series of characters matches that series of characters in the target string, so the pattern "Walt" would match "Walt" in the target string. Quite simple right? You can cause characters that normally function as metacharacters or escape sequences to be interpreted literally by escaping them by preceding them with a backslash \, for instance:

metacharacter ^ match beginning of string, but \^ match character ^, \\ match \ and so on. Examples: foobar matchs foobar anywhere in the text. ^foobar matchs foobar at the beginning of the text. \^foobar matchs ^foobar anywhere in the text. Escape sequences Characters may be specified using escape sequences syntax much like that used in C and Perl: "\n'' matches a newline, "\t'' a tab, etc. More generally, \xnn, where nn is a string of hexadecimal digits, matches the character whose ASCII value is nn. If You need wide (Unicode) character code, You can use '\x{nnnn}', where 'nnnn' is one or more hexadecimal digits. \xnn char with hex code nn \x{nnnn} char with hex code nnnn (one byte for plain text and two bytes for Unicode) \t tab (HT/TAB), same as \x09 \n newline (NL), same as \x0a \r car.return (CR), same as \x0d \f form feed (FF), same as \x0c \a alarm (bell) (BEL), same as \x07 \e escape (ESC), same as \x1b Examples: foo\x20bar matchs foo bar (note space in the middle) \tfoobar matchs foobar predefined by tab Character classes You can specify a character class, by enclosing a list of characters in brackets [ ], which will match any one character from the list. If the first character after the [ is ^, the class matches any character not in the list. Examples: foob[aeiou]r finds strings foobar, foober etc. but not foobbr, foobcr etc. foob[^aeiou]r find strings foobbr, foobcr etc. but not foobar, foober etc. Within a list, the dash - character is used to specify a range, so that a-z represents all characters from a to z. If You want a dash - itself to be a member of a class, put it at the start or end of the list, or escape it with a backslash. If You want the right bracket ] you may place it at the start of the list or escape it with a backslash. Examples: [-az] matchs a, z and [az-] matchs a, z and [a\-z] matchs a, z and [a-z] matchs all twenty six characters from a to z [\n-\x0D] matchs any of #10, #11, #12, #13 [\d-t] matchs any digit, - or t []-a] matchs any char from ] to a

Metacharacters

Metacharacters are special characters which are the essence of Regular Expressions. There are different types of metacharacters, described below. Metacharacters - line separator ^ start of a line $ end of a line \A start of text \Z end of text . any character in line Examples: ^foobar matchs string foobar only if it is at the beginning of a line. foobar$ matchs string foobar only if it is at the end of a line. ^foobar$ matchs string foobar only if it is the only string in the line. foob.r matchs strings like foobar, foobbr, foob1r and so on. Metacharacters - predefined classes \w \W \d \D \s \S an alphanumeric character (including "_"). a nonalphanumeric character. a numeric character. a non-numeric character. any space (same as [ \t\n\r\f]) a non space character

You may use \w, \d and \s within custom character classes. Examples: foob\dr matchs strings like foob1r, foob6r and so on but not foobar, foobbr and so on foob[\w\s]r matchs strings like foobar, foob r, foobbr and so on but not foob1r, foob=r and so on. Metacharacters - word boundaries A word boundary (\b) is a spot between two characters that has a \w on one side of it and a \W on the other side of it (in either order), counting the imaginary characters off the beginning and end of the string as matching a \W. Examples: \b Match a word boundary. \B Match a non-(word boundary). Metacharacters - iterators Any item of a regular expression may be followed by another type of metacharacters, an iterators. Using this metacharacters You can specify number of occurences of previous character, metacharacter or subexpression. * zero or more ("greedy"), same as {0,} + one or more ("greedy"), same as {1,} ? zero or one ("greedy"), same as {0,1} {n} exactly n times ("greedy") {n,} at least n times ("greedy")

{n,m} at least n but not more than m times ("greedy") *? zero or more ("non-greedy"), same as {0,}? +? one or more ("non-greedy"), same as {1,}? ?? zero or one ("non-greedy"), same as {0,1}? {n}? exactly n times ("non-greedy") {n,}? at least n times ("non-greedy") {n,m}? at least n times but not more than m times ("non-greedy") Digits in curly brackets of the form {n,m}, specify the minimum (n) number of times to match the item and the maximum (m). The form {n} is equivalent to {n,n} and matches exactly n times. The form {n,} matches n or more times. There is no limit to the size of n or m, but large numbers will chew up more memory and slow down execution. If a curly bracket occurs in any other context, it is treated as a regular character. Examples: foob.*r matchs strings like foobar, foobalkjdflkj9r and foobr foob.+r matchs strings like foobar, foobalkjdflkj9r but not foobr foob.?r matchs strings like foobar, foobbr and foobr but not foobalkj9r fooba{2}r matchs the string foobaar fooba{2,}r matchs strings like foobaar, foobaaar, foobaaaar etc. fooba{2,3}r matchs strings like foobaar, or foobaaar but not foobaaaar A little explanation about greediness. Greedy takes as many as possible, non-greedy takes as few as possible. For example, 'b+' and 'b*' applied to string 'abbbbc' return 'bbbb', 'b+?' returns 'b', 'b*?' returns empty string, 'b{2,3}?' returns 'bb', 'b{2,3}' returns 'bbb'. Metacharacters - alternatives You can specify a series of alternatives for a pattern using the pipe | character to separate them, so that fee|fie|foe will match any of fee, fie, or foe in the target string, as would f(e|i|o)e. The first alternative includes everything from the last pattern delimiter "(", "[", or the beginning of the pattern, up to the first "|", and the last alternative contains everything from the last "|" to the next pattern delimiter. For this reason, it is common practice to include alternatives in parentheses, to minimize confusion about where they start and end. Alternatives are tried from left to right, so the first alternative found for which the entire expression matches, is the one that is chosen. This means that alternatives are not necessarily greedy. For example: when matching foo|foot against "barefoot", only the foo part will match, as that is the first alternative tried, and it successfully matches the target string. This might not seem important, but it is important when you are capturing matched text using parentheses. Also remember that the | character is interpreted as a literal character within square brackets, so if You write [fee|fie|foe] You're really only matching any of the feio| characters. Examples: foo(bar|foo) matchs strings foobar or foofoo Metacharacters - subexpressions The parenthesis construct ( ... ) may be used to define subexpressions which are numbered based on the left to right order of their opening parenthesis. The first subexpression has number '1' (whole match has number '0') and they are backreferenced (see next section) by a single digit preceeding a backslash. For example, \1

Examples: (foobar){8,10} matchs strings which contain 8, 9 or 10 instances of foobar foob([0-9]|a+)r matchs foob0r, foob1r, foobar, foobaar, foobaar etc. Metacharacters - backreferences Metacharacters \1 through \9 are interpreted as backreferences. \<n> matches previously matched subexpressions as explained in the previous section. Matching the string /123/image123.jpg with /([09]+)/images\1.jpg will make sure that the number 123 in the above example is also present after images. If we would write the mask as /[0-9]+/images[0-9]+.jpg then both numbers do not need to be the same. Examples: (.)\1+ matchs aaaa and cc (.+)\1+ also match abab and 123123 (['"]?)(\d+)\1 matchs a number in double quotes, or a number in single quotes, or a number without quotes.

Modifiers
Modifiers are for changing the behaviour of regular expressions. They may be embedded within the regular expression itself using the (?...) construct where ... is one or more of the following... i Do case-insensitive pattern matching (default) -i Do case-sensitive pattern matching g set greedy mode (default) -g set non-greedy mode Examples: (?i)FooBar matchs FooBar, FOOBAR, foObAr etc. (?-i)FooBar matchs only FooBar (?-i)Foo(?i)Bar matchs FooBar, FooBAR, Foobar etc. (?g)[0-9]+ matchs any number of any length. (?-g)[0-9]+ matchs only a single digit number. foobar? what does it mean? foobar is a derivative of the acronym FUBAR, changed for symmetry. FUBAR stands for F***ed Up Beyond All Recognition. Sometimes used as a whole by programmers to describe data loss, foo and bar are also often used separately when a quick and random name is needed. Programmers (mostly those dirty UNIX guys) got tired of trying to think of random words, so they use foo and bar.