Configurable parallel und distributed processing settings

Multiprocessing settings

PyCAM v0.4 started to support parallel and distributed processing. This basically means that the calculation of toolpaths will use all available CPU cores of your machine automatically. Additionally you can also use some commandline arguments to connect to remote worker servers. This leverages your local CPU resources to process even huge models with the help of remote PyCAM servers in a timely manner. With the new upcoming release v0.4.1, PyCAM will also allow you to configure these network connections via the preferences dialog of the GUI.

The following items are configurable:

Enable parallel processes
This setting is the general switch for enabling parallel processing (using all cores of your computer) and distributed processing (sharing processing resources over the network).

Number of processes:
This specifies the maximum number of parallel local processes. By default this equals the number of CPU cores in your machine. Increasing this number will usually reduce overall performance. A lower number can be useful if you want to reserve CPU cores for other tasks.

Connect/disconnect:
Use this switch to open a connection to a remote server or to start a local server (depending on the Remote server setting below). Initiating a connecting as well as taking it down takes some moments – please be patient.

Remote server / port:
Here you can specify the hostname or IP of a remote server. You can also change the TCP port (default: 1250) if necessary.
Leave the hostname field empty if you only want to run a local server. This allows other hosts to connect to your instance of PyCAM. There is no difference between you connecting to another host or the other way around.

Local port:
This is the local TCP port, that you want to open for remote PyCAM instances. Usually you should keep the default value (1250).

Password:
Running a PyCAM server is a risky operation. Malicious peers have arbitrary control over your computer (limited only by the permissions of the user account running PyCAM). So please use a strong passoword for restricting access to your server. Click at New to generate a new random password. The Show password checkbox is used for toggling the visibility of the content of the password field above.

The network settings are currently not stored permanently. This will probably be implemented in a later release of PyCAM.

Comments

Configurable parallel und distributed processing settings |

I’m extremely pleased to find this web site.

I wanted to thank you for ones time due to this fantastic
read!! I definitely loved every little bit
of it and i also have you book-marked to check out new stuff in your website.

Here is my web blog :: tham my vien so 1

Configurable parallel und distributed processing settings |

Please let me know if you’re looking for a writer for your weblog.
You have some really great posts and I believe I would be a good asset.

If you ever want to take some of the load off, I’d absolutely love to write some
material for your blog in exchange for a link back to mine.
Please blast me an email if interested. Thank you!

my weblog … firehost Testimonials

Configurable parallel und distributed processing settings |

Very good post! We are linking to this particularly great content on our website.
Keep up the good writing.

my web-site seo service Watford

Reply to comment | sense.fab

You have made some really good points there. I looked on the net for additional information about the issue and found most individuals will go along with your views on this website.

Here is my blog post Sarang Semut Irian

Correct

I agree. I’ve looked through the code in SVN. I found the threaded pieces and the server code. I see what you are saying.

I think the correct approach is an app in “server mode” is required to solve work from a pre-determined filesystem location (either local disk, S3, ftp, or whatever really). Expose an API of sorts when the app is in “server mode” that only allows limited functionality.

I imagine it something along the lines of the SETI@Home project. The “server mode” units download pieces of work…solve the tool paths and upload the solutions. If a “server mode” unit fails to return a tool path in a certain time…no problem. Another will be allowed to download and return the tool path.

I’m standing up a linux box up at the moment. So I’ve only had the opportunity to experiment with the windows standalone binary. I’ll begin attempting to break it apart shortly when I can play with the linux version and some AWS EC2 machines.

I will write you at your address as soon as I am set up. :)

Cheers,
Bob

Crypto

Would you be interested in an openSSL or mayhaps an ssh implementation of the networking “mechanics”?

I note your concern about arbitrary execution of code on remote pycam instances. A way to mitigate that trust is to confirm with a private/public key setup. The remote pycam instances are configured to trust or establish connections with signed certificates. I’m not sure yet if there is for need bi-directional establishment of connection…but if there isn’t this could be solution.

I like the remote parallel processing idea. I figure setup a pycam ami on amazon ec2…blow it out to a bunch of high cpu instances and let it churn on the work. For a cup of coffee an hour you’d have mad cpu power available to do the tedious work of creating tool paths.

I read somewhere else that you were talking about chrooted environments or VMs. I think the VM throw away approach is the right idea at the moment. Don’t need much disk/s3. Just enough for an OS and pycam. Maybe have the result set stored to s3 so that it can be collected by the client pycam and maybe this would be a solution for fixing the hanging remote server part of the protocol.

Anyway, cheers!
Bob Tribit

code execution

Hi Bob,

thanks for sharing your thoughts!

Tunnelling the connection (including the shared secret) would surely be a good start for improved security.

The current protocol is quite useful for a setup where the server trusts any connecting client. But if we are thinking about public servers that are open for anonymous users, then we have a problem. The PyCAM server will directly execute any code given by the client. This can include the removal or manipulation of all files writeable for the pycam user. It also includes spawning new processes and so on. Basically: everything that is possible with Python can be requested from the server.

Thus a trow-away virtual machine that is reset in regular intervals is the only safe way of running a public PyCAM server right now. Is there anything that could be improved in PyCAM for easing this kind of setup?

If you have any ideas regarding the implementation of alternative authentication methods (e.g. SSL certificates), just drop me a mail (devel [at] sumpfralle.de). Currently we are giving out svn write access for the PyCAM repository quite freely :)

cheers,
Lars

Post new comment

  • You may embed media from the following providers vimeo. Just add the URL of the media to your textarea in the place where you would like the media to appear, i.e. http://www.youtube.com/watch?v=pw0jmvdh. You can optionally modify the display of the media by adding comma-separated attributes in brackets with no space following the URL, such as http://www.youtube.com/watch?v=pw0jmvdh[width=640,height=320].
  • Web page addresses and e-mail addresses turn into links automatically.
  • You may use [inline:xx] tags to display uploaded files or images inline.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <img><h2><h3<h4>
  • You can use Textile markup to format text.
  • Lines and paragraphs break automatically.
  • Freelinking helps you easily create HTML links. Links take the form of [[indicator:target|Title]]. By default (no indicator): Link to a local node by title
  • You can link nodes to other nodes using the following syntax:
    [node:node_id,title="val2"]
  • Enter node links as [1234:text] where 1234 is a node number and text is what should be displayed or $ to display the node's title.
  • Use [toc list: ol; title: Table of Contents; minlevel: 2; maxlevel: 3; attachments: yes;] to insert a mediawiki style collapsible table of contents. All the arguments are optional.

More information about formatting options

By submitting this form, you accept the Mollom privacy policy.