| 1 | <!-- Copyright David Abrahams 2006. Distributed under the Boost --> |
|---|
| 2 | <!-- Software License, Version 1.0. (See accompanying --> |
|---|
| 3 | <!-- file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) --> |
|---|
| 4 | <html> |
|---|
| 5 | <head> |
|---|
| 6 | <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> |
|---|
| 7 | <link rel="stylesheet" type="text/css" href="../boost.css"> |
|---|
| 8 | <title>Boost.Python - March 2002 Progress Report</title> |
|---|
| 9 | </head> |
|---|
| 10 | <body link="#0000ff" vlink="#800080"> |
|---|
| 11 | <table border="0" cellpadding="7" cellspacing="0" width="100%" summary= |
|---|
| 12 | "header"> |
|---|
| 13 | <tr> |
|---|
| 14 | <td valign="top" width="300"> |
|---|
| 15 | <h3><a href="../../../../index.htm"><img height="86" width="277" alt= |
|---|
| 16 | "C++ Boost" src="../../../../boost.png" border="0"></a></h3> |
|---|
| 17 | </td> |
|---|
| 18 | <td valign="top"> |
|---|
| 19 | <h1 align="center"><a href="../index.html">Boost.Python</a></h1> |
|---|
| 20 | <h2 align="center">March 2002 Progress Report</h2> |
|---|
| 21 | </td> |
|---|
| 22 | </tr> |
|---|
| 23 | </table> |
|---|
| 24 | <hr> |
|---|
| 25 | <h2>Contents</h2> |
|---|
| 26 | <dl class="index"> |
|---|
| 27 | <dt><a href="#accomplishments">Accomplishments</a></dt> |
|---|
| 28 | <dl class="index"> |
|---|
| 29 | <dt><a href="#calling_python">Calling Python from C++</a></dt> |
|---|
| 30 | <dt><a href="#virtual_functions">Virtual Functions</a></dt> |
|---|
| 31 | <dt><a href="#abstract_classes">Abstract Classes</a></dt> |
|---|
| 32 | <dt><a href="#implicit_conversions">C++ Implicit Conversions</a></dt> |
|---|
| 33 | <dt><a href="#data_members">C++ Data Members</a></dt> |
|---|
| 34 | <dt><a href="#miscellaneous">Miscellaneous</a></dt> |
|---|
| 35 | </dl> |
|---|
| 36 | |
|---|
| 37 | <dt><a href="#future">The Near future</a></dt> |
|---|
| 38 | |
|---|
| 39 | <dt><a href="#notes">Notes</a></dt> |
|---|
| 40 | |
|---|
| 41 | </dl> |
|---|
| 42 | |
|---|
| 43 | <h2><a name="accomplishments">Accomplishments</a></h2> |
|---|
| 44 | |
|---|
| 45 | March was mostly devoted to the reimplementation of features from |
|---|
| 46 | Boost.Python v1, and some new features. Re-examination of the features |
|---|
| 47 | from Boost.Python v1 allowed me to make significant improvements. |
|---|
| 48 | |
|---|
| 49 | <h3><a name="calling_python">Calling Python from C++</a></h3> |
|---|
| 50 | |
|---|
| 51 | The ability to call Python from C++ is crucial for virtual function |
|---|
| 52 | support. Implementing this feature well for V2 proved to be more |
|---|
| 53 | interesting than I expected. You can review most of the relevant |
|---|
| 54 | design decisions |
|---|
| 55 | <a href="callbacks.txt">here</a>. |
|---|
| 56 | |
|---|
| 57 | <p> |
|---|
| 58 | One point which <i>isn't</i> emphasized in that document is that there |
|---|
| 59 | are subtle differences in the way <code>from_python</code> conversions |
|---|
| 60 | work when used for C++ function arguments and Python function return |
|---|
| 61 | values. In particular, while <code>T const&</code> arguments may |
|---|
| 62 | invoke rvalue converters, a reference-to-const return value requires |
|---|
| 63 | an lvalue converter, since a temporary conversion result would leave |
|---|
| 64 | the returned reference dangling. |
|---|
| 65 | |
|---|
| 66 | <p>I'm not particularly pleased with the current callback interface, |
|---|
| 67 | since it usually results in constructs like: |
|---|
| 68 | <pre> |
|---|
| 69 | <u>return returning</u><X&>::call(f, obj); |
|---|
| 70 | </pre> |
|---|
| 71 | However, I think the following may be possible and I plan to investigate: |
|---|
| 72 | <pre> |
|---|
| 73 | return apply<X&>(f, obj); |
|---|
| 74 | </pre> |
|---|
| 75 | I'm open to suggestion for better names (and syntaxes)! |
|---|
| 76 | |
|---|
| 77 | <h3><a name="virtual_functions">Virtual Functions</a></h3> |
|---|
| 78 | |
|---|
| 79 | Once Python callbacks were implemented, it was just a short step to |
|---|
| 80 | implementing virtual functions. Python extension class exposing a C++ |
|---|
| 81 | class whose virtual functions are overridable in Python must actually |
|---|
| 82 | hold a C++ instance of a class <i>derived</i> from the one exposed to |
|---|
| 83 | Python. Needing some way for users to specify that class, I added an |
|---|
| 84 | optional template argument to <code>value_holder_generator</code> and |
|---|
| 85 | <code>pointer_holder_generator<></code> to specify the class |
|---|
| 86 | actually held. This move began to put pressure on the |
|---|
| 87 | <code>class_<></code> interface, since the need for the user to |
|---|
| 88 | produce complicated instantations of |
|---|
| 89 | <code>class_<></code> was increased: |
|---|
| 90 | |
|---|
| 91 | <pre> |
|---|
| 92 | class<Foo, bases<>, value_holder_generator<Foo_callback> >("Foo") |
|---|
| 93 | .def("hello", &Foo::hello) |
|---|
| 94 | ... |
|---|
| 95 | </pre> |
|---|
| 96 | |
|---|
| 97 | <h3><a name="abstract_classes">Abstract Classes</a></h3> |
|---|
| 98 | |
|---|
| 99 | Normally when a C++ class is exposed to Python, the library registers |
|---|
| 100 | a conversion function which allows users to wrap functions returning |
|---|
| 101 | values of that type. Naturally, these return values are temporaries, |
|---|
| 102 | so the conversion function must make a copy in some |
|---|
| 103 | dynamically-allocated storage (a "holder") which is managed |
|---|
| 104 | by the corresponding Python object. |
|---|
| 105 | |
|---|
| 106 | <p>Unfortunately, in the case of abstract classes (and other types |
|---|
| 107 | without a publicly-accessible copy constructor), instantiating this |
|---|
| 108 | conversion function causes a compilation error. In order to support |
|---|
| 109 | non-copyable classes, there had to be some way to prevent the library |
|---|
| 110 | from trying to instantiate the conversion function. The only practical |
|---|
| 111 | approach I could think of was to add an additional template parameter |
|---|
| 112 | to the <code>class_<></code> interface. When the number of |
|---|
| 113 | template parameters with useful defaults begins to grow, it is often |
|---|
| 114 | hard to choose an order which allows users to take advantage of the |
|---|
| 115 | defaults. |
|---|
| 116 | |
|---|
| 117 | <p> |
|---|
| 118 | |
|---|
| 119 | This was the straw that broke the |
|---|
| 120 | <code>class_<></code> interface's back and caused the redesign |
|---|
| 121 | whose outcome is detailed <a |
|---|
| 122 | href="http://mail.python.org/pipermail/c++-sig/2002-March/000892.html">here</a>. |
|---|
| 123 | The approach allows the user to supply the optional parameters in an |
|---|
| 124 | arbitrary order. It was inspired by the use of <a |
|---|
| 125 | href="../../../utility/iterator_adaptors.htm#named_tempalte_parameters">named |
|---|
| 126 | template parameters</a> in the <a |
|---|
| 127 | href="../../../utility/iterator_adaptors.htm">Boost Iterator Adaptor |
|---|
| 128 | Library</a>, though in this case it is possible to deduce the meaning |
|---|
| 129 | of the template parameters entirely from their type properties, |
|---|
| 130 | resulting in a simpler interface. Although the move from a |
|---|
| 131 | policy-based design to what resembles a configuration DSL usually |
|---|
| 132 | implies a loss of flexibility, in this case I think any costs are far |
|---|
| 133 | outweighed by the advantages. |
|---|
| 134 | |
|---|
| 135 | <p>Note: working around the limitations of the various compilers I'm |
|---|
| 136 | supporting was non-trivial, and resulted in a few messy implementation |
|---|
| 137 | details. It might be a good idea to switch to a more-straightforward |
|---|
| 138 | approach once Metrowerks CodeWarrior Pro8 is released. |
|---|
| 139 | |
|---|
| 140 | <h3><a name="implicit_conversions">C++ Implicit Conversions</a></h3> |
|---|
| 141 | |
|---|
| 142 | Support for C++ implicit conversion involves creating |
|---|
| 143 | <code>from_python</code> converters for a type <code>U</code> which in |
|---|
| 144 | turn use <code>from_python</code> converters registered for a type |
|---|
| 145 | <code>T</code> where there exists a implicit conversion from |
|---|
| 146 | <code>T</code> to <code>U</code>. The current implementation is |
|---|
| 147 | subject to two inefficiencies: |
|---|
| 148 | <ol> |
|---|
| 149 | |
|---|
| 150 | <li>Because an rvalue <code>from_python</code> converter produces two |
|---|
| 151 | pieces of data (a function and a <code>void*</code>) from its |
|---|
| 152 | <code>convertible()</code> function, we end up calling the function |
|---|
| 153 | for <code>T</code> twice: once when the converter is looked up in the |
|---|
| 154 | registry, and again when the conversion is actually performed. |
|---|
| 155 | |
|---|
| 156 | <li>A vector is used to mark the "visited" converters, preventing |
|---|
| 157 | infinite recursion as <code>T</code> to |
|---|
| 158 | <code>U</code> and <code>U</code> to <code>T</code> converters |
|---|
| 159 | continually search through one-another. |
|---|
| 160 | |
|---|
| 161 | </ol> |
|---|
| 162 | |
|---|
| 163 | I consider the former to be a minor issue. The second may or may not |
|---|
| 164 | prove to be computationally significant, but I believe that |
|---|
| 165 | architecturally, it points toward a need for more sophisticated |
|---|
| 166 | overload resolution. It may be that we want CLOS-style multimethod |
|---|
| 167 | dispatching along with C++ style rules that prevent more than one |
|---|
| 168 | implicit conversion per argument. |
|---|
| 169 | |
|---|
| 170 | <h3><a name="data_members">C++ Data Members</a></h3> |
|---|
| 171 | |
|---|
| 172 | To supply the ability to directly access data members, I was able to |
|---|
| 173 | hijack the new Python <a |
|---|
| 174 | href="http://www.python.org/2.2/descrintro.html#property">property</a> |
|---|
| 175 | type. I had hoped that I would also be able to re-use the work of <a |
|---|
| 176 | href="make_function.html">make_function</a> to create callable python |
|---|
| 177 | objects from C++ functions which access a data member of a given |
|---|
| 178 | class. C++ facilities for specifying data member pointer non-type |
|---|
| 179 | template arguments require the user to explicitly specify the type of |
|---|
| 180 | the data member and this under-utilized feature is also not |
|---|
| 181 | well-implemented on all compilers, so passing the member pointer as a |
|---|
| 182 | runtime value is the only practical approach. The upshot is that any |
|---|
| 183 | such entity would actually have to be a function <i>object</i>, and I |
|---|
| 184 | haven't implemented automatic wrapping of C++ callable function |
|---|
| 185 | objects yet, so there is less re-use in the implementation than I'd |
|---|
| 186 | like. I hope to implement callable object wrapping and refactor this |
|---|
| 187 | code one day. I also hope to implement static data member support, |
|---|
| 188 | for which Python's property will not be an appropriate descriptor. |
|---|
| 189 | |
|---|
| 190 | <h3><a name="miscellaneous">Miscellaneous</a></h3> |
|---|
| 191 | <ul> |
|---|
| 192 | <li>Moved <code>args<></code> and <code>bases<></code> from unnamed namespace to <code>boost::python</code> in their own header files. |
|---|
| 193 | <li>Convert <code>NULL</code> pointers returned from wrapped C++ functions to <code>None</code>. |
|---|
| 194 | <li>Improved some compile-time error checks. |
|---|
| 195 | <li>Eliminated <code>boost/python/detail/eval.hpp</code> in favor of |
|---|
| 196 | more-general <code>boost/mpl/apply.hpp</code>. |
|---|
| 197 | <li>General code cleanup and refactoring. |
|---|
| 198 | <li>Works with Microsoft Visual C++ 7.0 |
|---|
| 199 | <li>Warning suppression for many compilers |
|---|
| 200 | <li>Elegant interface design for exporting <code>enum</code> types. |
|---|
| 201 | </ul> |
|---|
| 202 | <hr> |
|---|
| 203 | |
|---|
| 204 | <h2><a name="future">The Near Future</a></h2> |
|---|
| 205 | |
|---|
| 206 | Before April 15th I plan to |
|---|
| 207 | <ol> |
|---|
| 208 | <li>Document all implemented features |
|---|
| 209 | <li>Implement a <code>CallPolicy</code> interface for constructors of wrapped |
|---|
| 210 | classes |
|---|
| 211 | <li>Implement conversions for <code>char</code> types. |
|---|
| 212 | <li>Implement automated code generation for all headers containing |
|---|
| 213 | families of overloaded functions to handle arbitrary arity. |
|---|
| 214 | </ol> |
|---|
| 215 | |
|---|
| 216 | I also hope to implement a mechanism for generating conversions |
|---|
| 217 | between arbitrary Python sequences and C++ containers, if time permits |
|---|
| 218 | (and others haven't already done it)! |
|---|
| 219 | |
|---|
| 220 | <h2><a name="notes">Notes</a></h2> |
|---|
| 221 | |
|---|
| 222 | The older version of KCC used by Kull is generating lots of warnings |
|---|
| 223 | about a construct I use to instantiate static members of various class |
|---|
| 224 | templates. I'm thinking of moving to an idiom which uses a function |
|---|
| 225 | template to suppress it, but worry about bloating the size of debug |
|---|
| 226 | builds. Since KCC users may be moving to GCC, I'm not sure that it's |
|---|
| 227 | worth doing anything about it. |
|---|
| 228 | |
|---|
| 229 | <p>Revised |
|---|
| 230 | <!--webbot bot="Timestamp" S-Type="EDITED" S-Format="%d %B, %Y" startspan --> |
|---|
| 231 | 13 November, 2002 |
|---|
| 232 | <!--webbot bot="Timestamp" endspan i-checksum="39359" --> |
|---|
| 233 | </p> |
|---|
| 234 | <p><i>© Copyright <a href="../../../../people/dave_abrahams.htm">Dave Abrahams</a> |
|---|
| 235 | 2002. </i></p> |
|---|
| 236 | </body> |
|---|
| 237 | </html> |
|---|
