Reduced memory consumption

PyCAM is – as the name implies – based on the scripting language Python. Python is a very efficient tool from the developer’s point of view: even complicated features can be implemented quickly and cleanly in a structured way. The downside of this ease of development can be a source of annoyance for users: number crunching applications are not as ressource efficient as their compiled counterparts. But today PyCAM gained a useful feature to fight its own ressource hunger.

The new PyCAM plugin “Memory Analyzer” can be used by developers to analyse the memory consumption and distribution of PyCAM’s objects. It uses “heapy” (a part of the Guppy project) as its source of information. The output is basically just a table of object types combined with their number of instances and their memory consumption.

The first analysis showed that each face of a 3D model requires around 10 different Python object instances (triangle, vertices, edges, plane, normal vector, …). This is not a surprise or a problem by itself. But due to Python’s dynamic approach to object oriented programming, this leads to around 5kB of memory usage for each single face. Obviously things like points or lines are usually not really dynamic objects with customized attribute lists and overridden member functions. In fact: this kind of objects acts just like a data container. This special kind of “static” instances can be announced to Python by using the magic “slots“ attribute in the class definition. The interested reader should take a look at the boring details in the Python reference manual.

Switching all relevant PyCAM classes to the static “slots” type was quite easy. The result is impressive: the memory consumption for a single face of a 3D model went down to around 20%. Thus each face now takes approximately 1kB of memory. This should increase the maximum size of usable models by a factor of 4 to 5 (when using the svn development repository).

Please not that this assumption is only valid for single core machines. Parallel processes will require additional memory. The detailed numbers are not really predictable, but your experiences and comments are always welcome!