AmazonEC2: Difference between revisions

From OpenSeesWiki
Jump to navigation Jump to search
No edit summary
No edit summary
Line 2: Line 2:
and for those in Industry get a limited amount of resources [http://aws.amazon.com/free/ free for the first 12 months].
and for those in Industry get a limited amount of resources [http://aws.amazon.com/free/ free for the first 12 months].


The following are the 6 commands you would issue from your terminal window to run the file Example.tcl using OpenSeesMP on a 100 node Amazon Cluster
The following are the 6 commands you would issue from a terminal/dos prompt while you are located in the directory containing the main script to run, Example.tcl, using OpenSeesMP on a 100 node Amazon Cluster and obtain the results that the application generates in the results folder back to the results folder in your current directory. Utilizing Amazon bidding (see Amazon Spot pricing) would allow you to run such a 100 node machine for as little as $1.03 an hour!
and obtain the results that the scipts created in the results folder back. Amazon bidding (see Amazon Spot pricing) could allow you to run such a 100 node machine for as little as $1.03 an hour!


<pre>
<pre>
starcluser start smallcluster
starcluser start smallcluster
starcluster put Example.tcl /root
starcluster put smallcluster ./ /root
starcluster sshmaster smallcluster
starcluster sshmaster smallcluster
mpiexec -n 100 OpenSeesMP Example.tcl
mpiexec -n 100 OpenSeesMP Example.tcl; exit
starcluster get /root/results /results
starcluster get smallcluster /root/results ./
starcluster terminate smallcluster
starcluster terminate smallcluster
</pre>
</pre>


The first command starts the cluster. The second command will copy the file Eample.tcl to the remote machine. The third command will cause you  to log on to your remote clusters 'master' node. The next command is the command you would issue on the remote machine to run a 100 processor job. The next command retrieves the contents of the /root/results directory. And lastly and an important command (assuming you don't want to use all your free allocation and go into bankruptcy) is the command to terminate the remote cluster.
The first command starts the cluster. The second command will copy the file Example.tcl to the remote machine. The third command will cause you  to log on to your remote clusters 'master' node. The fourth command is the command you would issue on the remote machine to run a 100 processor job. The fifth command retrieves the contents of the /root/results directory. And lastly and an important command (assuming you don't want to use all your free allocation or max out your credit card!) is the command to terminate the remote cluster.
 
The following [http://openees.berkeley.edu/OpenSees/code/SmallMP.zip zip file] contains an Example to try out.


The initial set-up to enable this requires that:
The initial set-up to enable this requires that:

Revision as of 15:51, 5 June 2015

OpenSees can run Sequentially and in Parallel on the resources provided by Amazon EC2. Amazon EC2 is the largest and most widely used cloud service provider. Once you are set up with Amazon and have installed some software from MIT, the running of the applications is trivial. For example, 6 commands from the terminal and you could be running a parallel application on a remote Amazon EC2 cluster of a few thousand nodes for a few dollars an hour. And it might not cost you anything, i.e. it could be FREE! Students and Educators can get free hours through the education program and and for those in Industry get a limited amount of resources free for the first 12 months.

The following are the 6 commands you would issue from a terminal/dos prompt while you are located in the directory containing the main script to run, Example.tcl, using OpenSeesMP on a 100 node Amazon Cluster and obtain the results that the application generates in the results folder back to the results folder in your current directory. Utilizing Amazon bidding (see Amazon Spot pricing) would allow you to run such a 100 node machine for as little as $1.03 an hour!

starcluser start smallcluster
starcluster put smallcluster ./ /root
starcluster sshmaster smallcluster
mpiexec -n 100 OpenSeesMP Example.tcl; exit
starcluster get smallcluster /root/results ./
starcluster terminate smallcluster

The first command starts the cluster. The second command will copy the file Example.tcl to the remote machine. The third command will cause you to log on to your remote clusters 'master' node. The fourth command is the command you would issue on the remote machine to run a 100 processor job. The fifth command retrieves the contents of the /root/results directory. And lastly and an important command (assuming you don't want to use all your free allocation or max out your credit card!) is the command to terminate the remote cluster.

The following zip file contains an Example to try out.

The initial set-up to enable this requires that:

  • You log in with a credit card and obtain an Amazon EC2 account using their detailed install instructions.
  • You correctly install the StarCluster Software from MIT on your computer.
  • Lastly you use the AMI that we have created for you in the StarCluster config file. The Amazon Ec2 AMI to specify is ami-e9ff1182 and is available in the US East Region.

The change to the StarCluster config file is simply to swap out their ami and replace with ours. The NODE_IMAGE_ID lines should read:

NODE_IMAGE_ID = ami-e9ff1182

NOTES:

  • The Amazon AMI is a 64bit Linux machine image.
  • As you get more familiar with Amazon you will start using their Amazon storage service, Amazon S3, to store data (think remote hard drives in the sky). The data stored can be mounted to appear as part of the file system. This will allow you to preserve the data remotely without downloading it and allow you to use other AMIs that contain other applications to process the data.
  • It is assumed you know how to run OpenSees and use the terminal window. But if you have got this far on the website you obviously can!