Planet
navi homePPSaboutscreenshotsdownloaddevelopmentforum

source: downloads/boost_1_33_1/more/regression.html @ 12

Last change on this file since 12 was 12, checked in by landauf, 17 years ago

added boost
File size: 9.6 KB
RevLine 
[12]1<html>
2
3<head>
4<meta http-equiv="Content-Type"
5content="text/html; charset=iso-8859-1">
6<meta name="GENERATOR" content="Microsoft FrontPage 5.0">
7<title>Boost Regression Test User Documentation</title>
8</head>
9
10<body bgcolor="#FFFFFF" text="#000000">
11
12<table border="1" cellpadding="2" bgcolor="#007F7F">
13    <tr>
14        <td bgcolor="#FFFFFF"><img src="../boost.png"
15        alt="boost.png (6897 bytes)" width="277" height="86"></td>
16        <td><a href="../index.htm"><font color="#FFFFFF" size="4"
17        face="Arial,Helvetica">Home</font></a></td>
18        <td><a href="../libs/libraries.htm"><font color="#FFFFFF"
19        size="4" face="Arial,Helvetica">Libraries</font></a></td>
20        <td><a href="../people/people.htm"><font color="#FFFFFF"
21        size="4" face="Arial,Helvetica">People</font></a></td>
22        <td><a href="faq.htm"><font color="#FFFFFF" size="4"
23        face="Arial,Helvetica">FAQ</font></a></td>
24        <td><a href="index.htm"><font color="#FFFFFF" size="4"
25        face="Arial,Helvetica">More</font></a></td>
26    </tr>
27</table>
28
29<h1>Boost Regression Test User Documentation</h1>
30
31<p><a href="#Introduction">Introduction</a><br>
32<a href="#Preparation">Preparation</a><br>
33<a href="#Execution">Execution</a><br>
34<a href="#Adding_new_test">Adding a new test</a><br>
35<a href="#Requirements">Requirements</a><br>
36<a href="#History">History</a></p>
37
38<h2><a name="Introduction">Introduction</a></h2>
39
40<p>Boost's internal regression test suite produces the <a
41href="../status/compiler_status.html">compiler status tables</a>.
42</p>
43
44<p>Although not ordinarily run by Boost library users, it is
45documented here for the benefit of Boost developers, and for
46Boost users porting to a new platform.</p>
47
48<p>Version 3 of the Boost regression testing framework is based
49on <a href="../tools/build/index.html">Boost.Build</a>, and uses <i>bjam</i>
50to actually run the tests. Because Boost.Build does dependency
51analysis, only tests for which some dependency has change are
52rerun.</p>
53
54<p>The reporting of test results as HTML files is accomplished by
55separate <a href="../tools/regression/index.htm">regression
56reporting programs</a> which process the residue and log files
57from the <i>bjam</i> run.</p>
58
59<h2><a name="Preparation">Preparation</a></h2>
60
61<p>Install the following programs on your system, in some
62location suitable for program executables. Normally that location
63must be in a directory which is part of your search path for
64executables.</p>
65
66<ul>
67    <li><i>bjam</i> - Executables and sources are available; see <a
68        href="../tools/build/index.html">Boost.Build docs</a>.&nbsp;
69        Before worrying about regression tests, you might want to
70        verify your <i>bjam</i> installation works by
71        building the
72        boost-root/status/Jamfile libraries.<br>
73        &nbsp;</li>
74    <li><i>process_jam_log</i> and <i>compiler_status</i> -
75    <a href="../tools/regression/index.htm">Sources and Jamfile</a> are available
76        for these, too. </li>
77</ul>
78
79<h2><a name="Execution">Execution</a></h2>
80
81<p>A complete shell script, <a href="../tools/regression/run_tests.sh">
82run_test.sh</a>, is available for POSIX systems. For Windows, please read the
83following.</p>
84
85<p>These examples assume several environment variables have been
86set:</p>
87
88<ul>
89    <li>BOOST_ROOT is set to the directory where the unzipped
90        Boost distribution or CVS working copy resides.&nbsp; For
91        example:<br>
92        <br>
93        set BOOST_ROOT=/boost_1_29_0<br>
94        &nbsp;</li>
95    <li>TOOLS is set to the <a
96        href="../tools/build/index.html">compiler toolsets</a>
97        you wish to use. For example:<br>
98        <br>
99        set TOOLS=borland gcc intel-win32 metrowerks vc7<br>
100        &nbsp;</li>
101    <li>Any environment variables required for particular
102        compilers.&nbsp; You might wish to test first with one
103        compiler at a time to make sure each toolset is fully
104        operational.</li>
105</ul>
106
107<p>Just running bjam for a specific library can be run like this on Windows,
108using the Filesystem Library as an example:</p>
109
110<blockquote>
111  <pre>cd \boost-root\libs\filesystem\test
112bjam</pre>
113</blockquote>
114
115<p>A full set of tests and status tables can be run thusly
116on a Windows system:</p>
117
118<blockquote>
119    <pre>cd %BOOST_ROOT%\status
120bjam --dump-tests test &gt;bjam.log 2&gt;&amp;1
121start notepad bjam.log
122process_jam_log &lt;bjam.log
123compiler_status %BOOST_ROOT% cs-win32.html
124rem Specify links file, even though it will be overwritten, so report html includes links
125rem This works because the generated bookmark names are the same regardless of other settings
126compiler_status --ignore-pass --no-warn %BOOST_ROOT% cs-win32-fail.html cs-win32-links.html
127compiler_status --ignore-pass %BOOST_ROOT% cs-win32-warn-or-fail.html cs-win32-links.html
128compiler_status %BOOST_ROOT% cs-win32-full.html cs-win32-links.html</pre>
129</blockquote>
130
131<p>Modulo syntax adjustments, the same procedure should work on
132other operating systems (an example a <a href="../tools/regression/run_tests.sh">
133UNIX shell script</a> for this is provided, see the
134script itself for comments and directions).&nbsp; Rename the cs-win-xxx
135output files as appropriate.</p>
136
137<p>If you execute <i>compiler_status</i> without arguments, you
138can see the available options and tailor your own favorite report.</p>
139
140<p>If you want to run just a single test, specify it as the <i>bjam</i>
141target rather than &quot;test&quot;.&nbsp; For example, to debug
142configurations, it might be useful to just run the <i>config_info</i>
143test, with a switch to force even up-to-date programs be rebuilt:</p>
144
145<blockquote>
146    <pre>bjam -a --dump-tests config_info &gt;bjam.log 2&gt;&amp;1
147process_jam_log &lt;bjam.log
148compiler_status %BOOST_ROOT% cs-win32.html
149</pre>
150</blockquote>
151
152<h2><a name="Adding_new_test">Adding a new test</a></h2>
153
154<h3>The best way - the <a name="subinclude"> <i>subinclud</i>e</a> approach</h3>
155
156<p>The best way to add a test to the <a
157href="../status/Jamfile">boost-root/status/Jamfile</a> is usually to use the bjam&nbsp;
158<i>subinclude</i> feature, since it allows the same test specification to be run
159either as part of the overall Boost regression test or as a standalone test of
160only the library involved.</p>
161
162<p>The test is specified in a Jamfile, which is usually placed in the library's
163<i>test</i> subdirectory. See the <a href="../libs/filesystem/test/Jamfile">
164Filesystem Library's test Jamfile</a> for an example, including the boilerplate
165wrapped around the actual tests. By <a href="#Execution">executing bjam</a> 
166(with or without the status reporting programs) in the library's <i>test</i> 
167subdirectory, the library can be tested without the need to run the full Boost
168regression tests.</p>
169
170<p>Then by adding a <i>subinclude</i> statement to <a
171href="../status/Jamfile">boost-root/status/Jamfile</a>, the library's test
172Jamfile will become part of the main Boost regression tests.&nbsp; For example,
173the Filesystem Library's tests are included by this line in the main
174boost-root/status Jamfile:</p>
175
176<blockquote>
177  <pre>subinclude libs/filesystem/test ;</pre>
178</blockquote>
179
180<p>Don't forget that Jamfiles are white-space sensitive; delete the
181space before the semi-colon in the example above, and you will
182get an introduction to Jam error messages.</p>
183
184<h3>Another way - direct inclusion in Boost regression tests</h3>
185
186<p>Adding a test is as simple as adding a single line to the <a
187href="../status/Jamfile">boost-root/status/Jamfile</a>:</p>
188
189<blockquote>
190    <pre>run libs/mylib/test/mytest.cpp ;</pre>
191</blockquote>
192
193<p>For creating more complex tests and test-suites, use the
194<a href="#subinclude">subinclude approach</a>. Look at
195examples in the <a
196href="../status/Jamfile">boost-root/status/Jamfile</a>.&nbsp;
197The <i>bind</i> and <i>config</i> test-suites are simple
198examples, while the <i>regex</i> and <i>threads</i> test suites
199show more of the power of Jam based testing.</p>
200
201<h2><a name="Requirements">Requirements</a></h2>
202
203<p>The test suite has been designed to meet to the following
204requirements. </p>
205
206<ul>
207    <li>Doesn't depend on an external tool chain. This requirement
208        is met by supplying all tools in the regular <a
209        href="getting_started.html#Download">Boost distribution</a>, except for a
210        C++ compiler.</li>
211    <li>Tools are written in C++; it is the only language that
212        all Boost developers and users are comfortable with.</li>
213    <li>Supports tests which expect an error </li>
214    <li>Configuration is independent of the target platform or
215        compiler. </li>
216    <li>HTML output </li>
217</ul>
218
219<p>These requirements rule out any script-based approach such as
220dejagnu (requires Tcl and expect) or even shell scripts. </p>
221
222<h2><a name="History">History</a></h2>
223
224<p>The version 3 Regression Test system is based on
225<a href="../tools/build/index.html">Boost.Build</a>. The&nbsp; testing.jam and status/Jamfile foundation was
226contributed by Dave Abrahams. Rene Rivera contributed several pieces. The post-bjam processing programs
227were contributed by Beman Dawes.</p>
228
229<p>The version 2 regression.cpp test program was contributed by
230Jens Maurer, generalizing and improving an earlier version 1
231program by Beman Dawes.</p>
232
233<hr>
234
235<p>Revised <!--webbot bot="Timestamp" startspan s-type="EDITED"
236s-format="%d %B, %Y" -->08 January, 2004<!--webbot bot="Timestamp"
237i-checksum="38708" endspan --></p>
238
239<p>© Copyright <a href="../people/jens_maurer.htm">Jens
240Maurer</a> 2001<br>
241© Copyright <a href="../people/beman_dawes.html">Beman Dawes</a> 2001</p>
242<p> Use, modification, and distribution are subject to the Boost Software
243License, Version 1.0. (See accompanying file <a href="../LICENSE_1_0.txt">
244LICENSE_1_0.txt</a> or copy at <a href="http://www.boost.org/LICENSE_1_0.txt">
245www.boost.org/LICENSE_1_0.txt</a>)</p>
246</body>
247</html>
Note: See TracBrowser for help on using the repository browser.