Planet
navi homePPSaboutscreenshotsdownloaddevelopmentforum

source: downloads/boost_1_34_1/libs/python/doc/v2/feb2002.html @ 29

Last change on this file since 29 was 29, checked in by landauf, 17 years ago

updated boost from 1_33_1 to 1_34_1
File size: 14.1 KB
Line 
1<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
2    <meta http-equiv="Content-Type" content=
3    "text/html; charset=iso-8859-1">
4    <link rel="stylesheet" type="text/css" href="../boost.css">
5
6    <title>Boost.Python - February 2002 Progress Report</title>
7<style type="text/css">
8 :link { color: #0000ff }
9 :visited { color: #800080 }
10 p.c3 {font-style: italic}
11 h2.c2 {text-align: center}
12 h1.c1 {text-align: center}
13</style>
14
15    <table border="0" cellpadding="7" cellspacing="0" width=
16    "100%" summary="header">
17      <tr>
18        <td valign="top" width="300">
19          <h3><a href="../../../../index.htm"><img height="86" width="277" alt=
20          "C++ Boost" src="../../../../boost.png" border="0"></a></h3>
21
22        <td valign="top">
23          <h1 class="c1"><a href="../index.html">Boost.Python</a></h1>
24
25          <h2 class="c2">February 2002 Progress Report</h2>
26    </table>
27    <hr>
28
29    <h2>Contents</h2>
30
31    <dl class="index">
32      <dt><a href="#Python10">Python10 Conference Report</a>
33
34      <dt><a href="#progress">Boost.Python v2 Progress</a>
35
36      <dd>
37        <dl class="index">
38          <dt><a href="#documentation">Documentation</a>
39
40          <dt><a href="#conversion">Overhaul of
41          <code>to_python</code>/<code>from_python</code>
42          conversion mechanism</a>
43
44          <dt><a href="#miscellaneous">Miscellaneous</a>
45        </dl>
46    </dl>
47
48    <h2><a name="Python10">Python10 Conference Report</a></h2>
49    I spent the first week of February at the Python10 conference
50    in Alexandria, VA. I&#39;m including this experience report
51    for two reasons: firstly, it documents where my time was
52    used. Secondly, a public presence for Boost.Python and
53    interaction between the Python and C++ communities is
54    important to the future of Boost.Python, which in turn is
55    important to the Kull Project.
56
57    <p>Andy Koenig, of all people, was the keynote speaker of
58    this year&#39;s opening plenary session. He presented his
59    &quot;impressions of a polyglot outsider&quot;, which
60    studiously avoided any mention of C++ until the end of his
61    talk, when he was asked about standardization. I was
62    surprised to learn that the C++ community at large wanted a
63    few more years before beginning but when ANSI accepted
64    HP&#39;s request for a standard, the process was forced to
65    start: it was a matter of participating or having
66    standardization proceed without one&#39;s input. Andy managed
67    to highlight very effectively the balance of strengths in
68    Python, one of the most important being its support for
69    extension via libraries. In many ways that makes Python a
70    good analogue for C++ in the interpreted world
71
72    <p>There were several kind mentions of the Boost.Python
73    library from people who found it indispensable. I was
74    particularly happy that Karl MacMillan, Michael Droettboom,
75    and Ichiro Fujinaga from Johns Hopkins is using it to do OCR
76    on a vast library of music notation, since in a previous life
77    I was an author of music notation software. These guys are
78    also drawing on Ullrich Koethe&#39;s VIGRA library for image
79    manipulation (Ullrich has been a major contributor to
80    Boost.Python). They also have a system for writing the
81    Boost.Python wrapper code in C++ comments, which allows them
82    to keep all of the code in one place. I&#39;ve asked them to
83    send me some information on that.
84
85    <p>The development of Swig has been gaining momentum again
86    (the basic description at
87    www.boost.org/libs/python/doc/comparisons.html still
88    applies). The talk given about it by David Beazly was very
89    well-attended, and they appear to have quite a few users.
90    Swig&#39;s strengths (coverage of many langauages) and
91    weaknesses (incomplete C++ language support) haven&#39;t
92    changed, although the C++ support seems to have improved
93    considerably - they now claim to have a complete model of the
94    C++ type system. It seems to be mostly geared at wrapping
95    what Walter Landry calls &quot;C-Tran&quot;: C++ code which
96    traffics in built-in types with little use of abstraction.
97    I&#39;m not knocking that, either: I&#39;m sure a lot of that
98    code exists, so it&#39;s a valuable service. One feature Swig
99    has which I&#39;d like to steal is the ability to unwrap a
100    single Python argument into multiple C++ arguments, for
101    example, by converting a Python string into a pointer and
102    length. When his talk was over, David approached me about a
103    possible joint workshop on language binding, which sounds
104    like a fun idea to me.
105
106    <p>I spent some considerable time talking with Steven Knight,
107    the leader of the Scons build tool effort. We had a lot to
108    share with one another, and I gained a much better
109    appreciation for many of the Scons design decisions. Scons
110    seems to be concentrating on being the ultimate build system
111    substrate, and Steve seemed to think that we were on the
112    right track with our high-level design. We both hope that the
113    Boost.Build V2 high-level architecture can eventually be
114    ported to run on top of Scons.
115
116    <p>They also have a highly-refined and successful development
117    procedure which I&#39;d like to emulate for Boost.Build V2.
118    Among many other things they do, their source-control system
119    automatically ensures that when you check in a new test, it
120    is automatically run on the currently checked-in state of the
121    code, and is expected to fail -- a relatively obvious good
122    idea which I&#39;ve never heard before.
123
124    <p>Guido Van Rossum&#39;s &quot;State of the Python
125    Union&quot; address was full of questions for the community
126    about what should be done next, but the one idea Guido seemed
127    to stress was that core language stability and continuing
128    library development would be a good idea (sound familiar?) I
129    mentioned the Boost model as a counterpoint to the idea of
130    something like CPAN (the massive Perl library archives), and
131    it seemed to generate some significant interest. I&#39;ve
132    offered to work with anyone from the Python community who
133    wants to set up something like Boost.
134
135    <p>There was some discussion of &quot;string
136    interpolation&quot; (variable substitution in strings), and
137    Guido mentioned that he had some thoughts about the
138    strengths/weaknesses of Python&#39;s formatting interface. It
139    might be useful for those working on formatting for boost to
140    contact him and find out what he has to say.
141
142    <p>Ka-Ping Yee demoed a Mailman discussion thread weaver.
143    This tool weaves the various messages in a discussion thread
144    into a single document so you can follow the entire
145    conversation. Since we&#39;re looking very seriously at
146    moving Boost to Mailman, this could be a really useful thing
147    for us to have. If we do this, we&#39;ll move the yahoogroups
148    discussions into the mailman archive so old discussions can
149    be easily accessed in the same fashion.
150
151    <p>And, just because it&#39;s cool, though perhaps not
152    relevant: http://homepages.ulb.ac.be/~arigo/psyco/ is a
153    promising effort to accelerate the execution of Python code
154    to speeds approaching those of compiled languages. It
155    reminded me a lot of Todd Veldhuizen&#39;s research into
156    moving parts of C++ template compilation to runtime, only
157    coming from the opposite end of things.
158
159    <h2><a name="progress">Boost.Python v2 Progress</a></h2>
160    Here&#39;s what actually got accomplished.
161
162    <h3><a name="documentation">Documentation</a></h3>
163
164    <p>My first priority upon returning from Python10 was to get
165    some documentation in place. After wasting an unfortunate
166    amount of time looking at automatic documentation tools which
167    don&#39;t quite work, I settled down to use Bill Kempf&#39;s
168    HTML templates designed to be a boost standard. While they
169    are working well, it is highly labor-intensive.
170
171    <p>I decided to begin with the high-level reference material,
172    as opposed to tutorial, narrative, or nitty-gritty details of
173    the framework. It seemed more important to have a precise
174    description of the way the commonly-used components work than
175    to have examples in HTML (since we already have some test
176    modules), and since the low-level details are much
177    less-frequently needed by users it made sense for me to
178    simply respond to support requests for the time being.
179
180    <p>After completing approximately 60% of the high-level docs
181    (currently checked in to libs/python/doc/v2), I found myself
182    ready to start documenting the mechanisms for creating
183    to-/from-python converters. This caused a dilemma: I had
184    realized during the previous week that a much simpler,
185    more-efficient, and easier-to-use implementation was
186    possible, but I hadn&#39;t planned on implementing it right
187    away, since what was already in place worked adequately. I
188    had also received my first query on the C++-sig about how to
189    write such a converter
190
191    <p>Given the labor-intensive nature of documentation writing,
192    I decided it would be a bad idea to document the conversion
193    mechanism if I was just going to rewrite it. Often the best
194    impetus for simplifying a design is the realization that
195    understandably documenting its current state would be too
196    difficult, and this was no exception.
197
198    <h3><a name="conversion">Overhaul of
199    <code>to_python</code>/<code>from_python</code> conversion
200    mechanism</a></h3>
201
202    <p>There were two basic realizations involved here:
203
204    <ol>
205      <li><code>to_python</code> conversion could be a one-step
206      process, once an appropriate conversion function is found.
207      This allows elimination of the separate indirect
208      convertibility check
209
210      <li>There are basically two categories of from_python
211      conversions: those which lvalues stored within or held by
212      the Python object (essentially extractions), like what
213      happens when an instance of a C++ class exposed with class_
214      is used as the target of a wrapped member function), and
215      those in which a new rvalue gets created, as when a Python
216      Float is converted to a C++
217      <code>complex&lt;double&gt;</code> or a Python tuple is
218      converted to a C++ <code>std::vector&lt;&gt;</code>. From
219      the client side, there are two corresponding categories of
220      conversion: those which demand an lvalue conversion and
221      those which can accept an lvalue or an rvalue conversion.
222    </ol>
223    The latter realization allowed the following collapse, which
224    considerably simplified things:
225
226    <blockquote>
227      <table border="1" summary="Conversion protocol">
228        <tr>
229          <th>Target Type
230
231          <th>Eligible Converters
232
233        <tr>
234          <td><code>T</code> 
235
236          <td rowspan="5"><code>T</code> rvalue or lvalue
237
238        <tr>
239          <td><code>T const</code> 
240
241        <tr>
242          <td><code>T volatile</code> 
243
244        <tr>
245          <td><code>T const volatile</code> 
246
247        <tr>
248          <td><code>T const&amp;</code> 
249
250        <tr>
251          <td><code>T const*</code> 
252
253          <td rowspan="9"><code>T</code> lvalue
254
255        <tr>
256          <td><code>T volatile*</code> 
257
258        <tr>
259          <td><code>T const volatile*</code> 
260
261        <tr>
262          <td><code>T&amp;</code> 
263
264        <tr>
265          <td><code>T volatile&amp;</code> 
266
267        <tr>
268          <td><code>T const volatile&amp;</code> 
269
270        <tr>
271          <td><code>T* const&amp;</code> 
272
273        <tr>
274          <td><code>T const* const&amp;</code> 
275
276        <tr>
277          <td><code>T volatile*const&amp;</code> 
278
279        <tr>
280          <td><code>T const volatile*const&amp;</code> 
281      </table>
282    </blockquote>
283    This job included the following additional enhancements:
284
285    <ul>
286      <li>Elimination of virtual functions, which cause object
287      code bloat
288
289      <li>Registration of a single converter function for all
290      lvalue conversions, two for all rvalue conversions
291
292      <li>Killed lots of unneeded code
293
294      <li>Increased opacity of registry interface
295
296      <li>Eliminated all need for decorated runtime type
297      identifiers
298
299      <li>Updated test modules to reflect new interface
300
301      <li>Eliminated the need for users to worry about converter
302      lifetime issues Additional Builtin Conversion Enhancements
303
304      <li>Support for complex&lt;float&gt;,
305      complex&lt;double&gt;, and complex&lt;long double&gt;
306      conversions
307
308      <li>Support for bool conversions
309
310      <li>NULL pointers representable by None in Python
311
312      <li>Support for conversion of Python classic classes to
313      numeric types
314    </ul>
315
316    <h3><a name="miscellaneous">Miscellaneous</a></h3>
317    These don&#39;t fit easily under a large heading:
318
319    <ul>
320      <li>Support CallPolicies for class member functions
321
322      <li>from_python_data.hpp: revamped type alignment
323      metaprogram so that it&#39;s fast enough for KCC
324
325      <li>classfwd.hpp header forward-declares class_&lt;T&gt;
326
327      <li>indirect_traits.hpp:
328
329      <li>added is_pointer_to_reference
330
331      <li>fixed bugs
332
333      <li>Reduced recompilation dependencies
334
335      <li>msvc_typeinfo works around broken MS/Intel typeid()
336      implementation
337
338      <li>Many fixes and improvements to the type_traits library
339      in order to work around compiler bugs and suppress warnings
340
341      <li>Eliminated the need for explicit acquisition of
342      converter registrations
343
344      <li>Expanded constructor support to 6 arguments
345
346      <li>Implemented generalized pointer lifetime support
347
348      <li>Updated code generation for returning.hpp
349
350      <li>Tracked down and fixed cycle GC bugs
351
352      <li>Added comprehensive unit tests for destroy_reference,
353      pointer_type_id, select_from_python, complex&lt;T&gt;,
354      bool, and classic class instance conversions
355    </ul>
356
357    <p>Revised
358    <!--webbot bot="Timestamp" S-Type="EDITED" S-Format="%d %B, %Y" startspan -->
359  13 November, 2002
360  <!--webbot bot="Timestamp" endspan i-checksum="39359" -->
361
362
363  <p class="c3">&copy; Copyright <a href=
364  "../../../../people/dave_abrahams.htm">Dave Abrahams</a> 2002. Distributed
365  under the Boost Software License, Version 1.0. (See accompanying file
366  LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)</p>
367
Note: See TracBrowser for help on using the repository browser.