Cloud Burst Service

Update: This feature is deprecated in the latest version of Caedium.

Caedium running on Windows HPC Server 2008 R2 SP2 (HPC Server) can perform CFD simulations unbounded by on-premises hardware limitations thanks to the 'Burst to Azure' capability that uses the Windows Azure cloud service.

Cloud Burst Features

  • Secure HTTPS communications between HPC Server and the Azure service
  • Advanced job scheduling using the HPC Cluster Manager
  • Simple Azure Compute Instance changes
  • Simple scaling up or down

Assumptions

For a video run through of the Burst to Azure setup see "How to deploy Windows Azure Worker Nodes as part of a Windows HPC Cluster".

  1. You have set up a Windows Azure account.
  2. In Hosted Services, Storage Accounts & CDN with the same affinity group name create:
    • Hosted Service - you will need the following hosted service parameter to configure the Burst to Azure service in HPC Server:
      • Name
    • Storage Account - you will need the following storage account parameters to configure the Burst to Azure service in HPC Server and the Caedium service:
      • Name
      • Primary access key
  3. You have administrator access to HPC Server to run the Azure service setup commands.
  4. You have configured the necessary security certificate sharing between HPC Server and your Azure hosted service account.
  5. You have configured an Azure node template.
  6. You have downloaded, installed, and activated the Caedium RANS Flow or Professional add-ons.

Procedure

  1. Upload the solver distribution from Caedium to your Azure storage account using the command: "C:\Program Files\Caedium\modules\ofburst\OfCloudUpload.exe" -StorageName <your storage name here> -StorageKey <your storage primary access key here> -BlobId openflow.zip -Dir "C:\Program Files\Caedium\openflow"
  2. In a temporary directory create the Caedium Azure runtime package using the command: hpcpack create OfCloudBurst.zip "C:\Program Files\Caedium\modules\ofburst"
  3. Upload the Caedium Azure runtime package using the command: hpcpack upload OfCloudBurst.zip /scheduler:<your head node computer name> /nodetemplate:"<your azure node template name>" /relativepath:openflow
  4. In the HPC Server HPC Cluster Manager select Node Management and then in the Actions panel click Add Node...
    1. In the Add Node Wizard select Add Windows Azure nodes, make sure Select Deployment Method is selected and click Next.
    2. Set the new node parameters:
      • Windows Azure node template = <your azure node template name>
      • Number of Windows Azure nodes = e.g., 2
      • Size of Windows Azure nodes = e.g., Extra Large
    3. Click Next and then Finish.
  5. Select an Azure node in the Nodes List and then in the Actions panel click Start.
  6. The Start Windows Azure Nodes dialog will open - click Start.
  7. To monitor progress click the Provisioning Log tab.
  8. Once the Node State reaches Offline, select all of the Azure nodes in the Nodes List and then in the Actions panel click Bring Online.
  9. In Caedium select the File Toolbar and click the Preferences button . In the Properties Panel turn on the preference Physics->OpenFOAM->Diagnostics to see feedback on the cloud service.
  10. Set the preference OpenFOAM->Parallel to Cloud Burst.
  11. Set the following Parallel:Cloud Burst associated preferences:
    • Azure Storage Name = <your storage name here>
    • Azure Storage Name->Azure Storage Key = <your storage primary access key here>
    • Processes = e.g., 4 (Extra Large Compute Instance has 4 processors)
  12. Try running a small simulation to test the service and if it is successful proceed to run your actual simulations
  13. When you have completed your simulations, delete your deployment by selecting the Azure nodes in the Nodes List, then in the Actions panel click Stop.

Notes

  • Azure charges for storage, bandwidth, and CPU time usage.
  • Your billable CPU time starts as soon as you commit to deploying a service and continues as long as the service is deployed - whether it is running a simulation or not.
  • If there are failures in the service your simulation may not run correctly.
  • If you interrupt a running simulation, the simulation will continue to run on Azure.
  • You might find it useful to browse your Azure storage account like a file system using CloudBerry. You can then delete any failed or redundant job files.