Planet
navi homePPSaboutscreenshotsdownloaddevelopmentforum

source: downloads/boost_1_34_1/libs/tokenizer/char_delimiters_separator.htm @ 29

Last change on this file since 29 was 29, checked in by landauf, 16 years ago

updated boost from 1_33_1 to 1_34_1
File size: 5.1 KB
Line 
1<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
2
3<html>
4<head>
5  <meta http-equiv="Content-Language" content="en-us">
6  <meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
7  <meta name="GENERATOR" content="Microsoft FrontPage 6.0">
8  <meta name="ProgId" content="FrontPage.Editor.Document">
9
10  <title>Boost Char Delimiters Separator</title>
11</head>
12
13<body bgcolor="#FFFFFF" text="#000000" link="#0000EE" vlink="#551A8B" alink=
14"#FF0000">
15  <p><img src="../../boost.png" alt="C++ Boost" width="277" height=
16  "86"><br></p><font color="red">Note: This class is deprecated. Please use
17  <a href="char_separator.htm"><tt>char_separator</tt></a> instead.</font>
18
19  <h1 align="center">Char Delimiters Separator</h1>
20  <pre>
21template &lt;class Char, class Traits = std::char_traits&lt;Char&gt; &gt;
22class char_delimiters_separator{
23</pre>
24
25  <p>The char_delimiters_separator class is an implementation of the <a href=
26  "tokenizerfunction.htm">TokenizerFunction</a> concept that can be used to
27  break text up into tokens. It is the default TokenizerFunction for
28  tokenizer and token_iterator_generator. An example is below.</p>
29
30  <h2>Example</h2>
31  <pre>
32// simple_example_4.cpp
33#include&lt;iostream&gt;
34#include&lt;boost/tokenizer.hpp&gt;
35#include&lt;string&gt;
36
37int main(){
38   using namespace std;
39   using namespace boost;
40   string s = "This is,  a test";
41   tokenizer&lt;char_delimiters_separator&lt;char&gt; &gt; tok(s);
42   for(tokenizer&lt;char_delimiters_separator&lt;char&gt; &gt;::iterator beg=tok.begin(); beg!=tok.end();++beg){
43       cout &lt;&lt; *beg &lt;&lt; "\n";
44   }
45}
46</pre>
47
48  <h2>Construction and Usage</h2>
49
50  <p>There is one constructor of interest. It is as follows</p>
51  <pre>
52explicit char_delimiters_separator(bool return_delims = false,
53const Char* returnable = "",const Char* nonreturnable = "" )
54</pre>
55
56  <table border="1" summary="">
57    <tr>
58      <td>
59        <p align="center"><strong>Parameter</strong></p>
60      </td>
61
62      <td>
63        <p align="center"><strong>Description</strong></p>
64      </td>
65    </tr>
66
67    <tr>
68      <td>return_delims</td>
69
70      <td>Whether or not to return the delimiters that have been found. Note
71      that not all delimiters can be returned. See the other two parameters
72      for explanation.</td>
73    </tr>
74
75    <tr>
76      <td>returnable</td>
77
78      <td>This specifies the returnable delimiters. These are the delimiters
79      that can be returned as tokens when return_delims is true. Since these
80      are typically punctuation, if a 0 is provided as the argument, then the
81      returnable delmiters will be all characters Cfor which std::ispunct(C)
82      yields a true value. If an argument of "" is provided, then this is
83      taken to mean that there are noreturnable delimiters.</td>
84    </tr>
85
86    <tr>
87      <td>nonreturnable</td>
88
89      <td>This specifies the nonreturnable delimiters. These are delimiters
90      that cannot be returned as tokens. Since these are typically
91      whitespace, if 0 is specified as an argument, then the nonreturnable
92      delimiters will be all characters C for which std::isspace(C) yields a
93      true value. If an argument of "" is provided, then this is taken to
94      mean that there are no non-returnable delimiters.</td>
95    </tr>
96  </table>
97
98  <p>The reason there is a distinction between nonreturnable and returnable
99  delimiters is that some delimiters are just used to split up tokens and are
100  nothing more. Take for example the following string "b c +". Assume you are
101  writing a simple calculator to parse expression in post fix notation. While
102  both the space and the + separate tokens, you only only interested in the +
103  and not in the space. Indeed having the space returned as a token would
104  only complicate your code. In this case you would specify + as a
105  returnable, and space as a nonreturnable delimiter.</p>
106
107  <p>To use this class, pass an object of it anywhere a TokenizerFunction
108  object is required.</p>
109
110  <h3>Template Parameters</h3>
111
112  <table border="1" summary="">
113    <tr>
114      <th>Parameter</th>
115
116      <th>Description</th>
117    </tr>
118
119    <tr>
120      <td><tt>Char</tt></td>
121
122      <td>The type of the elements within a token, typically
123      <tt>char</tt>.</td>
124    </tr>
125
126    <tr>
127      <td>Traits</td>
128
129      <td>The traits class for Char, typically
130      std::char_traits&lt;Char&gt;</td>
131    </tr>
132  </table>
133
134  <h2>Model of</h2>
135
136  <p><a href="tokenizerfunction.htm">TokenizerFunction</a></p>
137
138  <p>&nbsp;</p>
139  <hr>
140
141  <p><a href="http://validator.w3.org/check?uri=referer"><img border="0" src=
142  "http://www.w3.org/Icons/valid-html401" alt="Valid HTML 4.01 Transitional"
143  height="31" width="88"></a></p>
144
145  <p>Revised
146  <!--webbot bot="Timestamp" s-type="EDITED" s-format="%d %B, %Y" startspan -->25
147  December, 2006<!--webbot bot="Timestamp" endspan i-checksum="38518" --></p>
148
149  <p><i>Copyright &copy; 2001 John R. Bandela</i></p>
150
151  <p><i>Distributed under the Boost Software License, Version 1.0. (See
152  accompanying file <a href="../../LICENSE_1_0.txt">LICENSE_1_0.txt</a> or
153  copy at <a href=
154  "http://www.boost.org/LICENSE_1_0.txt">http://www.boost.org/LICENSE_1_0.txt</a>)</i></p>
155</body>
156</html>
Note: See TracBrowser for help on using the repository browser.