| 1 | <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> |
|---|
| 2 | |
|---|
| 3 | <html> |
|---|
| 4 | <head> |
|---|
| 5 | <meta http-equiv="Content-Language" content="en-us"> |
|---|
| 6 | <meta http-equiv="Content-Type" content="text/html; charset=us-ascii"> |
|---|
| 7 | <meta name="GENERATOR" content="Microsoft FrontPage 6.0"> |
|---|
| 8 | <meta name="ProgId" content="FrontPage.Editor.Document"> |
|---|
| 9 | |
|---|
| 10 | <title>Boost Tokenizer Overview</title> |
|---|
| 11 | </head> |
|---|
| 12 | |
|---|
| 13 | <body bgcolor="#FFFFFF" text="#000000" link="#0000EE" vlink="#551A8B" alink= |
|---|
| 14 | "#FF0000"> |
|---|
| 15 | <p><img src="../../boost.png" alt="C++ Boost" width="277" height= |
|---|
| 16 | "86"><br></p> |
|---|
| 17 | |
|---|
| 18 | <h1 align="center">Table Of Contents</h1> |
|---|
| 19 | |
|---|
| 20 | <p align="left"> </p> |
|---|
| 21 | |
|---|
| 22 | <h2 align="left"><a href="introduc.htm">Introduction</a></h2> |
|---|
| 23 | |
|---|
| 24 | <h2 align="left">Containers and Iterators</h2> |
|---|
| 25 | |
|---|
| 26 | <ul> |
|---|
| 27 | <li> |
|---|
| 28 | <h3 align="left"><a href="tokenizer.htm">tokenizer</a></h3> |
|---|
| 29 | </li> |
|---|
| 30 | |
|---|
| 31 | <li> |
|---|
| 32 | <h3 align="left"><a href="token_iterator.htm">token iterator</a></h3> |
|---|
| 33 | </li> |
|---|
| 34 | </ul> |
|---|
| 35 | |
|---|
| 36 | <h2><a href="tokenizerfunction.htm">TokenizerFunction Concept</a></h2> |
|---|
| 37 | |
|---|
| 38 | <h2>TokenizerFunction Models</h2> |
|---|
| 39 | |
|---|
| 40 | <ul> |
|---|
| 41 | <li> |
|---|
| 42 | <h3><a href="char_separator.htm">char_separator</a></h3> |
|---|
| 43 | </li> |
|---|
| 44 | |
|---|
| 45 | <li> |
|---|
| 46 | <h3><a href= |
|---|
| 47 | "escaped_list_separator.htm">escaped_list_separator</a></h3> |
|---|
| 48 | </li> |
|---|
| 49 | |
|---|
| 50 | <li> |
|---|
| 51 | <h3><a href="offset_separator.htm">offset_separator</a></h3> |
|---|
| 52 | </li> |
|---|
| 53 | |
|---|
| 54 | <li><font color="red">Deprecated:</font> <a href= |
|---|
| 55 | "char_delimiters_separator.htm">char_delimiters_separator</a></li> |
|---|
| 56 | </ul> |
|---|
| 57 | |
|---|
| 58 | <h2> </h2> |
|---|
| 59 | |
|---|
| 60 | <h2>Acknowledgements</h2> |
|---|
| 61 | |
|---|
| 62 | <p>I wish to thank the members of the boost mailing list, whose comments, |
|---|
| 63 | compliments, and criticisms during both the development and formal review |
|---|
| 64 | helped make the Tokenizer library what it is. I especially wish to thank |
|---|
| 65 | Aleksey Gurtovoy for the idea of using a pair of iterators to specify the |
|---|
| 66 | input, instead of a string. I also wish to thank Jeremy Siek for his idea |
|---|
| 67 | of providing a container interface for the token iterators and for |
|---|
| 68 | simplifying the template parameters for the TokenizerFunctions. He and |
|---|
| 69 | Daryle Walker also emphasized the need to separate interface and |
|---|
| 70 | implementation. Gary Powell sparked the idea of using the isspace and |
|---|
| 71 | ispunct as the defaults for char_delimiters_separator. Jeff Garland |
|---|
| 72 | provided ideas on how to change to order of the template parameters in |
|---|
| 73 | order to make tokenizer easier to declare. Thanks to Douglas Gregor who |
|---|
| 74 | served as review manager and provided many insights both on the boost list |
|---|
| 75 | and in e-mail on how to polish up the implementation and presentation of |
|---|
| 76 | Tokenizer. Finally, thanks to Beman Dawes who integrated the final version |
|---|
| 77 | into the boost distribution.</p> |
|---|
| 78 | <hr> |
|---|
| 79 | |
|---|
| 80 | <p><a href="http://validator.w3.org/check?uri=referer"><img border="0" src= |
|---|
| 81 | "http://www.w3.org/Icons/valid-html401" alt="Valid HTML 4.01 Transitional" |
|---|
| 82 | height="31" width="88"></a></p> |
|---|
| 83 | |
|---|
| 84 | <p>Revised |
|---|
| 85 | <!--webbot bot="Timestamp" s-type="EDITED" s-format="%d %B, %Y" startspan -->25 |
|---|
| 86 | December, 2006<!--webbot bot="Timestamp" endspan i-checksum="38518" --></p> |
|---|
| 87 | |
|---|
| 88 | <p><i>Copyright © 2000 Jeremy Siek<br> |
|---|
| 89 | Copyright © 2001 John R. Bandela</i></p> |
|---|
| 90 | |
|---|
| 91 | <p><i>Distributed under the Boost Software License, Version 1.0. (See |
|---|
| 92 | accompanying file <a href="../../LICENSE_1_0.txt">LICENSE_1_0.txt</a> or |
|---|
| 93 | copy at <a href= |
|---|
| 94 | "http://www.boost.org/LICENSE_1_0.txt">http://www.boost.org/LICENSE_1_0.txt</a>)</i></p> |
|---|
| 95 | </body> |
|---|
| 96 | </html> |
|---|
