Introduction

    Regular expression defines a matching method. First, it can be used to check whether a string contains a substring which matches a certain pattern, and then the substring can be retrieved. Second, regular expression can be used to do complex replacement.

    It is very simple to study regular expression syntax, and the few abstract concepts can be understood easily too. Many articles does not introduce its concepts from simple ones to abstract ones step by step, so some persons may feel it is difficult to study. On the other hand, each regular expression engine's document will describe its special function, but this part of special function is not what we should study first.

    Every example in this article has a link to test page. Now let's begin!

---

1. Regular Expression Basic Syntax

1.1 Common Characters

    Letters, numbers, the underline, and punctuations with no special definition are "common characters". When regular expression matches a string, a common character can match the same character.

    Example1: When pattern "c" matches string "abcde", match result: success; substring matched: "c"; position: starts at 2, ends at 3.

    Example2: When pattern "bcd" matches string "abcde"，match result: success; substring matched: "bcd"; position: starts at 1, ends at 4.

---

1.2 Simple escaped characters

    Nonprinting characters which we know:

Expression	Matches
\r, \n	Carriage return, newline character
\t	Tabs
\\	Matches "\" itself

    Some punctuations are specially defined in regular expression. To match these characters in string, add "\" in pattern. For example: ^, $ has special definition, so we need to use "\^" and "\$" to match them.

Expression	Matches
\^	Matches "^" itself
\$	Matches "$" itself
\.	Matches dot(.) itself

    These escaped characters have the same effect as "common characters": to match a certain character.

    Example1: When pattern "\$d" matches string "abc$de", match result: success; substring matched: "$d"; position: starts at 3, ends at 5.

---

1.3 Expression matches anyone of many characters

    Some expressions can match anyone of many characters. For example: "\d" can match any number character. Each of these expressions can match only one character at one time, though they can match any character of a certain group of characters.

Expression	Matches
\d	Any digit character, any one of 0~9
\w	Any alpha, numeric, underline, any one of A~Z,a~z,0~9,_
\s	Any one of space, tab, newline, return, or newpage character
.	'.' matches any character except the newline character(\n)

    Example1: When pattern "\d\d" matches "abc123", match result: success; substring matched: "12"; position: starts at 3, ends at 5.

    Example2: When pattern "a.\d" matches "aaa100", match result: success; substring matched: "aa1"; position: starts at 1, ends at 4.

---

1.4 Custom expression matches anyone of many characters

    Expression uses square brackets [ ] to contain a series of characters, it can match anyone of them. Uses [^ ] to contain a series of characters, it can match anyone character except characters contained.

Expression	Matches
[ab5@]	Matches "a" or "b" or "5" or "@"
[^abc]	Matches any character except "a","b","c"
[f-k]	Any character among "f"~"k"
[^A-F0-3]	Any character except "A"~"F","0"~"3"

    Example1: When pattern "[bcd][bcd]" matches "abc123" , match result: success; substring matched: "bc"; position: starts at 1, ends at 3.

    Example2: When pattern "[^abc]" matches "abc123", match result: success; substring matched: "1"; position: starts at 3, ends at 4.

---

1.5 Special expression to quantify matching

    All expressions introduced before can match character only one time. If a expression is followed by a quantifier, it can matches more than one times.

    For example: we can use the pattern "[bcd]{2}" instead of "[bcd][bcd]".

Expression	Function
{n}	Match exactly n times, example: "\w{2}" equals "\w\w"; "a{5}" equals "aaaaa"
{m,n}	At least n but no more than m times: "ba{1,3}" matches "ba","baa","baaa"
{m,}	Match at least n times: "\w\d{2,}" matches "a12","_456","M12344"...
?	Match 1 or 0 times, equivalent to {0,1}: "a[cd]?" matches "a","ac","ad".
+	Match 1 or more times, equivalent to {1,}: "a+b" matches "ab","aab","aaab"...
*	Match 0 or more times, equivalent to {0,}: "\^*b" matches "b","^^^b"...

    Example1: When pattern "\d+\.?\d*" matches "It costs $12.5", match result: success; substring matched:"12.5"; position: starts at 10, ends at 14.

    Example2: When pattern "go{2,8}gle" matches "Ads by goooooogle", match result: success; substring matched: "goooooogle"; position: starts at 7, ends at 17.

---

1.6 Some special punctuations with abstract function

    Some punctuations in pattern have special function:

Expression	Function
^	Match the beginning of the string
$	Match the end of the string
\b	Match a word boundary

    More examples to help you to understand.

    Example1: When pattern "^aaa" matches "xxx aaa xxx", match result: failed. Because "^" must match the beginning of the string. It could match successfully on condition that "aaa" is at the beginning of the string, such as "aaa xxx xxx".

    Example2: When pattern "aaa$" matches "xxx aaa xxx", match result: failed. Bacause "$" must match the end of the string. It could match successfully on condition that "aaa" is at the end of the string, such as "xxx xxx aaa".

    Example3: When pattern ".\b." matches "@@@abc", match result: success; substring matched: "@a"; position: starts at 2, ends at 4.
    Further explanation: "\b" is similar to "^" and "$", matches no character itself, but it require a '\w' character at its one side, another not '\w' character at the other side.

    Example4: When pattern "\bend\b" matches "weekend,endfor,end", match result: success; substring matched: "end"; position: starts at 15, ends at 18.

    Some special punctuation can make effect on other sub-patterns:

Expression	Function
|	Alternation, matches either left side or right side
( )	(1). Let sub-patterns in it to be a whole part when it is quantified. (2). Match result of sub-patterns in it can be retrieved individually

    Example5: When pattern "Tom|Jack" matches string "I'm Tom, he is Jack", match result: success; substring matched: "Tom"; position: starts at 4, ends at 7. When match next, match result: success; substring matched: "Jack"; position: starts at 15, ends at 19.

    Example6: When pattern "(go\s*)+" matches "Let's go go go!", match result: success; substring matched: "go go go"; position: starts at 6, ends at 14.

    Example7: When pattern "￥(\d+\.?\d*)" matches "＄10.9,￥20.5", match result: success; substring matched: "￥20.5"; position: starts at 6, ends at 10. Match result of sub-patterns in "( )" is: "20.5".

---

2. Regular expression advanced syntax

2.1 Reluctant or greedy quantifiers

    There are serval method to quantify subpattern, such as: "{m,n}", "{m,}", "?", "*", "+". By default, a quantified subpattern is "greedy", that is, it will match as many times as possible (given a particular starting location) while still allowing the rest of the pattern to match. For example, to match "dxxxdxxxd":

Expression	Match result
(d)(\w+)	"\w+" matches all characters "xxxdxxxd" behind of "d"
(d)(\w+)(d)	"\w+" matches all characters "xxxdxxx" between the first "d" and the last "d". In order to let the whole pattern match success, "\w" has to give up the last "d", although it can match the last "d" too.

    Thus it can be seen that: when "\w+" matches, it will match as many characters as possible. In the second example, it does not match the last "d", but this is in order to let the whole pattern match successfully. Pattern with "*" or "{m,n}" will also match as many times as possible, pattern with "?" will match if possible. This type of matching is called "greedy matching". 。

    Reluctant Matching:

    To follow the quantifier with a "?", it can let the pattern to match the minimum number of times possible. This type of matching is called reluctant matching. In order to let the whole pattern match successfully, the reluctant pattern may match a few more times if it is required. For example, to match "dxxxdxxxd":

Expression	Match result
(d)(\w+?)	"\w+?" match as few times as possible, so "\w+?" matches only one "x"
(d)(\w+?)(d)	In order to let the whole pattern match successfully, "\w+?" has to match "xxx". So, match result is: "\w+?" matches "xxx"

    More examples:

    Example1: When pattern "<td>(.*)</td>" matches "<td><p>aa</p></td> <td><p>bb</p></td>", match result: success; substring matched: the whole "<td><p>aa</p></td> <td><p>bb</p></td>", "</td>" in the pattern matches the last "</td>" in the string.

    Example2: For comparison, when pattern "<td>(.*?)</td>" matches the string in example1, it matches "<td><p>aa</p></td>". When match next, the next "<td><p>bb</p></td>" can be matched.

---

2.2 Referring to matched substring \1, \2...

    During the process of matching, the match results of subpattern between parentheses "( )" are recorded for later use. When retrieving match results, those match result of subpattern can be retrieved individually, and this has been demonstrated many times in former examples. In practice, parentheses "( )" must be used to get what we want indeed after match, such as "<td>(.*?)</td>".

    In fact, those match result of subpattern between parentheses can be used not only after matching, but also during matching. The latter part of subpattern, can refer the match result of former subpattern. Usage: "\" plus a number to refer to the corresponding substring. "\1" refers to 1st pair of parentheses' match result, "\2" refers to 2nd pair of parentheses' match result.

    Examples:

    Example1: When pattern "('|")(.*?)(\1)" matches " 'Hello', "World" ", match result: success; substring matched: " 'Hello' "; when match next, substring matched: " "World" ".

    Example2: When pattern "(\w)\1{4,}" matches "aa bbbb abcdefg ccccc 111121111 999999999", match result: success; substring matched: "ccccc"; when match next, substring matched "999999999". This pattern require a character of "\w" to repeat at least 5 times. Pay attention to comparison with "\w{5,}".

    Example3: When pattern "<(\w+)\s*(\w+(=('|").*?\4)?\s*)*>.*?</\1>" matches "<td id='td1' style="bgcolor:white"></td>", match result: success. If both "<td>" and "</td>" are not "td", the match will fail.

---

2.3 Lookahead assertion; Lookbehind assertion

    In former chapters, I have introduced serval punctuations with special function: "^"，"$"，"\b". They all do not match any characters, but they all require certain conditions on their position. Now, this chapter will introduce more methods to add conditions on the gap between characters.

    Lookahead assertion: "(?=xxxxx)", "(?!xxxxx)"

    Format: "(?=xxxxx)", the condition which it add on the gap is that: string on the right side of the gap must be abe to match the subpattern "xxxxx" between the parentheses. It is just a condition, not a match operation, so there is no match result.

    Example1: When pattern "Windows (?=NT|XP)" matches "Windows 98, Windows NT, Windows 2000", it can match only "Windows " of "Windows NT", the other "Windows " could not be matched.

    Example2: When pattern "(\w)((?=\1\1\1)(\1))+" matches "aaa ffffff 999999999", it can match first 4 "f"s among the 6 "f"s, it can match first 7 "9"s among 9 "9"s.

    Format: "(?!xxxxx)", string on the right side of the gap must not be able to match the subpattern "xxxxx".

    Example3: When pattern "((?!\bstop\b).)+" matches "fdjka ljfdl stop fjdsla fdj", it will match from the beginning of string to the position of "stop". If there is no "stop" in the string, the pattern will match the whole string.

    Example4: When pattern "do(?!\w)" matches "done, do, dog", it can only match "do". Here, "(?!\w)" has the same effect as "\b".

    Lookbehind assertion: "(?<=xxxxx)", "(?<!xxxxx)"

    The concepts of "Lookbehind assertion" and "Lookahead assertion" are similar. "(?<=xxxxx)" and "(?<!xxxxx)" require the string on the left side of the gap to be able to match or to be not able to match the subpattern, not the right side. And they will not match any characters themselves too.

    Example5: When pattern "(?<=\d{4})\d+(?=\d{4})" matches "1234567890123456", it will match 8 numbers in the middle, except first 4 numbers and last 4 numbers. Because lookbehind assertion is not supported by JScript.RegExp, this example could not be demonstrated. There are many engines support lookbehind assertion, such as java.util.regex package in Java 1.4 or later, System.Text.RegularExpressions namespace in .NET platform, and DEELX Regexp Engine etc.

---

3. Other usually supported rules

    There are several usually supported rules which have not been mentioned.

3.1 In pattern, a character can be expressed as "\xXX" or "\uXXXX" ("X" is a hex number)

Format	Character range
\xXX	0 ~ 255, such as space can be "\x20"
\uXXXX	Any character can be expressed as "\u" plus 4 hex numbers, such as "\u4E2D"

3.2 While "\s", "\d", "\w", "\b" are specially defined, their uppercase letters have the opposite meaning

Pattern	Matches
\S	All characters except spaces
\D	All characters except numeric characters
\W	All characters except alpha, numeric, "_"
\B	Characters' gap which is not a word boundary

3.3 Specially defined characters table

Character	Description
^	Matches the beginning of the string. Use "\^" to match "^" itself
$	Matches the end of the string. Use "\$" to match "$" itself
( )	Grouping. Use "\(" and "\)" to match "(" and ")"
[ ]	Character class. Use "\[" and "\]" to match "[" and "]"
{ }	Define quantifiers. Use "\{" and "\}" to match "{" and "}"
.	Match any character except newline(\n). Use "\." to match "." itself
?	Let subpattern match 0 or 1 time. Use "\?" to match "?" itself
+	Let subpattern match at least 1 times. Use "\+" to match "+" itself
*	Let subpattern match any times. Use "\*" to match "*" itself
|	Alternation. Use "\|" to match "|" itself

3.4 If a subpattern is in "(?:xxxxx)", the match result is not recorded for later use.

    Example1: When pattern "(?:(\w)\1)+" matches "a bbccdd efg", the substring matched: "bbccdd". The match result of subpattern in "(?:)" is not recorded, so "\1" is used to refer to the match result of "(\w)".

3.5 Pattern attribute: Ignorecase，Singleline，Multiline，Global

Attribute	Description
Ignorecase	Do case-insensitive pattern matching. Default is case-sensitive.
Singleline	Treat string as single line. That is, change "." to match any character whatsoever, even a newline, which it normally would not match.
Multiline	Treat string as multiple lines. Default is that "^" and "$" match at only the very start① of the string and end④ of the string. If multiline, they match the start③ of any line and end② of any line within the string: ①xxxxxxxxx②\n ③xxxxxxxxx④
Global	Replace all matches if the pattern is used in replace operation.

---

4. Integrated prompt

4.1 If you want to know what else are implemented by advanced engines, you can refer to DEELX Syntax on this site.

4.2 If the pattern is required to match the whole string, not a part of string, we may use "^" and "$", such as: "^\d+$" require the whole string consist of digit characters.

4.3 If the pattern is required to match a whole word, not a part of word, we may use "\b" at the beginning and the end of the pattern, such as: use "\b(if|while|else|void|int……)\b" to match keywords in a program.

4.4 Do not let pattern match empty string "". Or you will get an empty substring matched, while the match operation returns success. For example: if we need a pattern to match "123"、"123."、"123.5"、".5", we should not use this pattern "\d*\.?\d*". Though there is nothing, we may still get a success. Proper pattern: "\d+\.?\d*|\.\d+".

4.5 Do not let a subpattern loop infinite times if the subpattern can match empty string.

4.6 Choose reluctant or greedy quantifier properly.

4.7 Only one side of "|" to match a certain character.

欢迎访问最专业的网吧论坛,无盘论坛,网吧经营,网咖管理,网吧专业论坛 https://bbs.txwb.com 关注天下网吧微信/下载天下网吧APP/天下网吧小程序，一起来超精彩