Beruflich Dokumente
Kultur Dokumente
0 Tutorials
ANSYS, Inc.
Southpointe
2600 ANSYS Drive
Canonsburg, PA 15317
ansysinfo@ansys.com
http://www.ansys.com
(T) 724-746-3304
(F) 724-514-9494
Disclaimer Notice
THIS ANSYS SOFTWARE PRODUCT AND PROGRAM DOCUMENTATION INCLUDE TRADE SECRETS AND ARE CONFIDENTIAL AND PROPRIETARY PRODUCTS OF ANSYS, INC., ITS SUBSIDIARIES, OR LICENSORS. The software products
and documentation are furnished by ANSYS, Inc., its subsidiaries, or affiliates under a software license agreement
that contains provisions concerning non-disclosure, copying, length and nature of use, compliance with exporting
laws, warranties, disclaimers, limitations of liability, and remedies, and other provisions. The software products
and documentation may be used, disclosed, transferred, or copied only in accordance with the terms and conditions
of that software license agreement.
ANSYS, Inc. is certified to ISO 9001:2008.
Third-Party Software
See the legal information in the product help files for the complete Legal Notice for ANSYS proprietary software
and third-party software. If you are unable to access the Legal Notice, please contact ANSYS, Inc.
Published in the U.S.A.
Table of Contents
Configuring Remote Solve Manager (RSM) to Submit Jobs to a Microsoft HPC Cluster .................................... 1
1. Installing ANSYS Products in a Microsoft HPC Cluster ........................................................................... 2
2. Configuring RSM on the Cluster Head Node ......................................................................................... 2
3.Troubleshooting RSM Issues ............................................................................................................... 15
3.1. Gathering RSM Job Logs for Systems Support ............................................................................ 15
3.2. Issue:My Computer Disabled in RSM Manager ......................................................................... 16
3.3. Configuring Multiple Network Cards (NIC) ................................................................................. 17
3.4. Disabling IPv6 ........................................................................................................................... 18
3.5. Issue: Cannot resolve localhost .................................................................................................. 18
3.6. Disabling Microsoft User Account Control (UAC) ........................................................................ 19
3.7. Common Errors Found in RSM Job Log ...................................................................................... 19
Configuring Remote Solver Manager (RSM) to Submit Jobs to a Linux LSF, PBS, Torque with Moab, or UGE
(formerly SGE) Cluster ................................................................................................................................. 23
1. Installing ANSYS Products in a Linux Cluster ....................................................................................... 23
1.1. Exporting the /ansys_inc Directory ............................................................................................ 23
2. Configuring RSM on the Cluster Head Node ....................................................................................... 24
2.1. Creating an rsmadmins Group and User Accounts ..................................................................... 24
2.1.1. About the rsmadmin Account ........................................................................................... 25
2.2. Using the RSM Setup Wizard to Configure RSM .......................................................................... 25
2.2.1. Launching the RSM Setup Wizard ..................................................................................... 25
2.2.2. Specifying Machine and Cluster Information ..................................................................... 26
2.2.3. Adding a Compute Server ................................................................................................ 28
2.2.4. Adding a Queue ............................................................................................................... 34
2.2.5. Defining Accounts ............................................................................................................ 38
2.2.6. Testing the Connection .................................................................................................... 40
3. Starting Automatic Startup (Daemon) Services for Linux Red Hat or SuSE ............................................ 41
3.1. Verifying that Daemon Services are Started ............................................................................... 41
4. Troubleshooting RSM ........................................................................................................................ 41
4.1. Gathering RSM Job Logs for Systems Support ............................................................................ 41
4.2. Issue:My Computer Disabled in RSM Manager ......................................................................... 42
4.3. Configuring Multiple Network Cards (NIC) ................................................................................. 43
4.4. Disabling IPv6 ........................................................................................................................... 44
4.5. Cannot Resolve localhost .......................................................................................................... 44
4.6. Common Errors Found in RSM Job Log ...................................................................................... 44
4.6.1. Caught exception at user logon: A required privilege is not held by the client. ................... 45
4.6.2. Caught exception at user logon; logon failure: unknown user name or bad password. Account
password not provided. ............................................................................................................ 45
4.6.3. Connection Error/No connection could be made .............................................................. 45
4.6.4. Failed to create Script Task: Access to the path is denied. ............................................. 46
4.6.5. Caught exception from script: Failed to find the TCP port from TaskHost run. ..................... 46
4.6.6.The submission of the requested job has been cancelled because the Solve Manager .
seems not fully initialized. ......................................................................................................... 47
4.6.7. Failed to create working directory on execution nodes via node share/mount ................... 47
Submitting CFX, Fluent and Mechanical Jobs to a Linux or Microsoft HPC Cluster ......................................... 49
1. Configuring RSM on a Windows Client Machine Prior to Submitting Jobs to a Linux or Windows
Cluster .................................................................................................................................................. 49
2. Submitting a CFX Job from Workbench to a Linux or Windows Cluster ................................................ 51
3. Submitting a Fluent Job from Workbench to a Linux or Windows Cluster ............................................ 53
4. Submitting a Mechanical Job from Workbench to a Linux or Windows Cluster ..................................... 55
5. Troubleshooting Job Failures ............................................................................................................. 59
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
iii
iv
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Assumptions
These instructions assume the following:
You have installed and configured a Microsoft HPC Server, and the compute nodes can access the cluster
head node. If Microsoft HPC is not configured properly, contact Microsoft for support before you attempt
to install ANSYS applications.
You can access a Getting Started Guide for Windows HPC Server at the following locations:
For Windows HPC Server 2008: http://technet.microsoft.com/en-us/library/cc793950.aspx
For Windows HPC Server 2012 R2: http://msdn.microsoft.com/en-us/library/jj884144.aspx
You are a local administrator of the Microsoft HPC cluster and know how to share directories and map network
drives. If you do not know how to perform these tasks, contact your Systems Administrator for assistance.
You can also access help from the Start menu on your desktop.
You know the machine name of the head node on the Microsoft Server HPC cluster.
You are able to install and run ANSYS, Inc. products, including Licensing on Windows systems. For information
on installation and licensing, see the tutorials on the Downloads menu of the ANSYS Customer Portal.
If you have any problems with, or questions about the installation process, go to the Support page of
the ANSYS Customer Portal and submit a support request.
This tutorial is divided into the following sections:
1. Installing ANSYS Products in a Microsoft HPC Cluster
2. Configuring RSM on the Cluster Head Node
3.Troubleshooting RSM Issues
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Configuring Remote Solve Manager (RSM) to Submit Jobs to a Microsoft HPC Cluster
Install ANSYS Fluent, CFX or Mechanical on the head node ONLY of the Microsoft Server HPC cluster. The
default installation directories are as follows:
Fluent: C:\Program Files\ANSYS Inc\v160\fluent
CFX: C:\Program Files\ANSYS Inc\v160\CFX
Mechanical: C:\Program Files\ANSYS Inc\v160\ansys
It is not necessary to install the solver on the compute nodes.
2.
Ensure that the Microsoft HPC user account has Read & Execute permissions for this directory. Typically,
it is sufficient to add DOMAIN USERS to the list of users that have access to submit jobs to the compute
cluster.
When using the ANSYS installer to install a solver (Fluent, CFX, Mechanical, Polyflow), RSM and Workbench
will be installed also.
2.
2
Click Next. Complete the steps presented by the wizard, using the sections that follow as a guide.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
On the Machine Information screen, select Yes, I am setting up a head node of a cluster as Remote
Solve Manager.
2.
3.
To allow for auto-configuration of Workbench, leave Configure ANSYS Workbench when starting RSM
services checked.
4.
Click Next. If you opted to configure ANSYS Workbench when starting RSM services, the HPC Setup
Prerequisites page will prompt you to cache your password with HPC:
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Configuring Remote Solve Manager (RSM) to Submit Jobs to a Microsoft HPC Cluster
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
On the Define Compute Servers screen, select Yes to specify that you want to define a new Compute
Server, then click Next.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Configuring Remote Solve Manager (RSM) to Submit Jobs to a Microsoft HPC Cluster
2.
On the Select a Compute Server screen, select Define a New Compute Server, then click Next.
3.
On the Identify Machine screen, enter a Machine Name or IP Address for the server. This must be the
actual computer name or IP address of the head node. In this example well enter headnode.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Enter a Display Name for the server. This can be any name that makes sense for you. In this example well
enter MS Compute Server.
5.
On the Set Cluster Information screen, specify whether you want to run jobs from a network share or
from the local disk, then click Next. In this example well select Network Share.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Configuring Remote Solve Manager (RSM) to Submit Jobs to a Microsoft HPC Cluster
6.
On the next Set Cluster Information screen, enter the UNC path for your Shared Cluster Directory. This
is the directory that is shared out to all the cluster nodes from the head node. In this example well use
the shared Temp directory as the shared cluster directory, so well enter \\Headnote\Temp as our path.
Click Next.
7.
On the Job Submission Settings screen, specify the Maximum Number of Jobs that can run concurrently
on this Compute Server, then click Next.
8.
On the Save Compute Server Settings screen, select Yes, save all changes to save your Compute
Server settings, then click Next.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
9.
On the Set up Compute Server screen, specify whether you want to auto-configure Compute Server
directories. In this example well select Yes, automatically configure directories, then click Next.
10. On the Additional Compute Servers screen, specify whether you want to create or modify another
Compute Server. In this example well select No, then click Next.
Adding a Queue
1.
On the Define Queues screen, select Yes to define a new or modify and existing queue, then click Next.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Configuring Remote Solve Manager (RSM) to Submit Jobs to a Microsoft HPC Cluster
2.
On the Select Queue screen, specify whether you want to create a new queue or modify one already in
list. In this example well select Define a new Queue, then click Next.
3.
On the Queue Information screen, enter a Name for the queue. In this example well enter MS Compute
Cluster Queue. The Compute Server you added previously (MS Compute Server, in this example)
appears in the list of Compute Servers. Select its check box to assign it to the new queue. Click Next.
10
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
4.
On the Additional Queues screen, specify whether you want to define or modify another queue. In this
example well select No, then click Next.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
11
Configuring Remote Solve Manager (RSM) to Submit Jobs to a Microsoft HPC Cluster
Defining Accounts
1.
On the Define Accounts screen, specify whether or not you want to define new accounts or modify
passwords. In this example well select Yes, then click Next.
2.
On the Select Account screen, select an existing account to modify or specify that you want to define a
new account. In this example well select Define a new account, then click Next.
12
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
3.
On the Define Account screen, enter the Username and Password that you use to log into your Windows
machine, then confirm your password. Click Next.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
13
Configuring Remote Solve Manager (RSM) to Submit Jobs to a Microsoft HPC Cluster
4.
On the Define More Accounts screen, specify if you want to define more accounts. In this example well
select No, then click Next.
On the Test Compute Servers screen, click the Queues drop box and select the queue that you want to
test.
2.
14
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
If the test succeeds, the Test Status will be Finished. If the test fails, the Test Status will be Test
Failed. Check over the steps to make sure that you followed all steps correctly. You can also check
Troubleshooting RSM Issues (p. 15) for information on adding firewall ports and so on.
3.
Click Next
4.
2.
15
Configuring Remote Solve Manager (RSM) to Submit Jobs to a Microsoft HPC Cluster
3.
4.
Right-click on the log in the lower right pane and select Debug Messages.
5.
6.
Solution: Make sure that the RSM services on the manager machine (in other words, the head node)
were started as Administrator.
For Windows, you must either have Windows administrative privileges on the Solve Manager, have RSM
administrative privileges (as a member of the RSM Admins user group), or launch the RSM Admin by
right-clicking on it and selecting Run as administrator.
1.
Log in as Administrator.
2.
On the machine where RSM is set up, open a Command Prompt and change the directory (cd) to
C:\Program Files\Ansys Inc\v160\RSM\bin.
3.
16
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
2.
3.
4.
In the New Group dialog, enter RSM Admins as the Group Name and add members by clicking Add.
5.
In the Select Users, Computers, Service Accounts, or Groups dialog, type a user name in the editing
window and then click Check Names to search for a matching name in the current domain. When found,
the user name will be displayed in full syntax in the editing window.
6.
Make sure that you can ping all of the nodes that you want to use.
2.
Note
Take note of the IP address (10.2.10.32 in the above example). You will need this address in
the steps that follow.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
17
Configuring Remote Solve Manager (RSM) to Submit Jobs to a Microsoft HPC Cluster
1.
Go to the head node and navigate to C:\Program Files\Ansys Inc\v160\RSM\Config and locate the
Ans.Rsm.AppSettings.config file.
2.
3.
Locate the Global appSettings section. If your text editor can show line numbers this section
starts on line 3.
4.
5.
6.
Go to Control Panel > Administrative Tools and restart the services ANSYS JobManager Service V16.0
and ANSYS ScriptHost Service V16.0. To restart a service, right-click on it and select Restart.
2.
Click the Fix this problem link for the procedure that you want to run.
3.
Run the downloaded file and follow the steps in the wizard.
If running on Windows (not a server OS) and you see this issue check your C:\Windows\System32\drivers\etc\hosts file.
2.
Make sure that 127.0.0.1 is not commented out with a # sign. If it is, remove the # sign.
3.
18
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
rhino.acme.com
x.acme.com
# source server
# x client host
In Windows, select Control Panel > User Accounts > Change User Account Control settings.
2.
To turn off UAC, move the slider to the Never notify position, and then click OK.
Caught exception at user logon: A required privilege is not held by the client
Description: In the Windows Task Manager, on the Processes page, RSM is running as a user and not
as SYSTEM. This is incorrect. To submit jobs to another Windows machine the processes need to be
running as SYSTEM.
Solution: Start the RSM services manually:
1.
Log in as Administrator.
2.
On the machine where RSM is set up, open a Command Prompt and change the directory (cd) to
C:\Program Files\Ansys Inc\v160\RSM\bin.
3.
Caught exception at user logon; logon failure: unknown user name or bad password
/ Account Password not Provided
Description: The following error is reported in the RSM log file:
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
19
Configuring Remote Solve Manager (RSM) to Submit Jobs to a Microsoft HPC Cluster
Compute Server running as: DOMAIN\username
Caught exception at user logon: A required privilege is not held by the client.
Or
Account Password not Provided
In the Set Password dialog box, if your DOMAIN and username match the one shown simply press
Enter.
If the Windows client account is different from the HPC Windows account, you will need to set up an
alternate account. You can do this after you have cached your primary Windows account with RSM by
selecting Set Password again, but this time enabling the This is the alternate account check box before
entering the credentials for the HPC Windows account. You can also set up an alternate account from
the Accounts dialog box (right-click on My Computer in RSM and select Accounts).
Error: A connection attempt failed because the connected party did not properly respond after a period of time. Or,
No connection could be made because the target machine actively refused it.
If you have a local firewall turned on for the server and/or RSM Client machines, you will need to add
two ports to the Exceptions List for RSM:
Add port 8160 to Ans.Rsm.SHHost.exe
Add port 9160 to Ans.Rsm.JMHost.exe
If that is not the case, check to see if IPv6 is enabled and if it is, disable it. See Disabling IPv6 (p. 18)
for details.
You can disable it partly by going to your network properties for the NIC card and unchecking the IPv6
box, but you also have to further disable it in the registry.
Caught exception from script: Failed to find the TCP port from TaskHost run.
Solution 1: Restart the RSM services.
1.
20
Go to Control Panel > System and Security > Administrative Tools > Services.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Restart the ANSYS RSM JHost and ANSYS RSM ScriptHost services.
Solution 2: Check for firewalls. Refer to Dealing with Firewalls (p. 17).
The submission of the requested job has been cancelled because the Solve Manager
. seems not fully initialized.
Solution: This is a dual network card issue. For instructions see Configuring Multiple Network Cards
(NIC) (p. 17).
You may also want to check for multiple RSM admins of the same version running concurrently.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
21
22
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Assumptions
These instructions assume the following:
You have installed and configured the Linux job scheduler and the compute nodes can access the cluster
head node. If your cluster is not configured properly please contact your hardware vendor or a third party
consultant for assistance.
You have passwordless ssh set up between the head node and compute nodes. Consult an IT professional
for assistance with setting up passwordless ssh.
You know the machine name of the head node on the Linux cluster.
You are able to install and run ANSYS, Inc. products, including Licensing on Windows systems. For information
on installation and licensing, see the tutorials on the Downloads menu of the ANSYS Customer Portal.
If you have any problems with, or questions about the installation process, go to the Support page of
the ANSYS Customer Portal and submit a support request.
23
Configuring Remote Solver Manager (RSM) to Submit Jobs to a Linux LSF, PBS,Torque
with Moab, or UGE (formerly SGE) Cluster
ANSYS directory if the machine you are installing on does not have a DVD/USB drive or an internet
connection for downloading files and you need to share files with a machine that does have a DVD/USB
drive or internet connection.
1.
Export the /ansys_inc directory by adding the following line to the /etc/exports file:
/usr/ansys_inc
2.
The default behavior on Linux provides read-only access from all clients. To enable read/write permission
from all clients, use *(rw):
/usr/ansys_inc *(rw)
3.
Run: exportfs a
4.
5.
If you perform a network install where you want the clients to be able to modify the licensing configuration,
you need to consider the NFS write options for the exported file system as shown in the above examples.
You also need local permissions to the licensing directory (/shared_files/licensing/) if you want
to be able to create the install_licconfig.log that the license configuration produces.
6.
If you need to transfer the files from a Windows machine with a DVD drive to a Linux machine without
one, copy the DVD contents using a Samba mount or some other transfer method that is safe to use
between Windows and Linux.
7.
If sharing the ANSYS directory between Linux machines, you must use the same mount point for both
the client and server. For example, if you installed to a file server in a directory named /apps/ansys_inc
and you did not choose the symbolic link to /ansys_inc, then you must mount this directory on the
client machine using /apps/ansys_inc as the mount point. If you did choose the symbolic link to
/ansys_inc during installation on the file server, you must either use /ansys_inc as the mount point
on the client or you must create a symbolic link to /ansys_inc on the client machine. (The symbolic
link is created by default during installation if you installed as root.)
Log in as ROOT (this is required initially to start the RSM daemons) and manually create a group called
rsmadmins.
2.
Add users to the group who will is responsible for configuring RSM Admin.
24
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
If you started the daemons as ROOT, log out as root. A local rsmadmin account and rsmadmins group
are automatically created when the daemons are started as ROOT.
2.
If other users will be configuring RSM Admin, add their user names to the rsmadmins group (this also
requires ROOT permission). You can log out as root now.
Note
If you start the services with an rsmadmins non-root user account, the service will be run
by that account in non-daemon mode. Root user privileges are required for starting RSM
services as daemons. If you start RSM services as daemons, any non-daemon services will be
killed.
If you logged in as ROOT to start the daemons, log out as ROOT. Log in as the user account that was added
to the rsmadmins group.
2.
25
Configuring Remote Solver Manager (RSM) to Submit Jobs to a Linux LSF, PBS,Torque
with Moab, or UGE (formerly SGE) Cluster
2.
26
a.
b.
c.
Click Next.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
3.
4.
Click Next.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
27
Configuring Remote Solver Manager (RSM) to Submit Jobs to a Linux LSF, PBS,Torque
with Moab, or UGE (formerly SGE) Cluster
28
When asked if you want to define new or modify existing Compute Servers, select Yes.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
b.
2.
Click Next.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
29
Configuring Remote Solver Manager (RSM) to Submit Jobs to a Linux LSF, PBS,Torque
with Moab, or UGE (formerly SGE) Cluster
b.
3.
30
Click Next.
Type in a Machine Name or IP Address for the server. This must be the actual computer name or
IP address of the head node. In this example, we will use: headnode
b.
Type in the Display Name. This can be any name that makes sense for you.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
c.
4.
Click Next.
Specify whether you want to run jobs from a network share or from the local disk. In this example,
we will select Network Share.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
31
Configuring Remote Solver Manager (RSM) to Submit Jobs to a Linux LSF, PBS,Torque
with Moab, or UGE (formerly SGE) Cluster
b. Click Next.
5.
6.
Enter the local path for your Shared Cluster Directory. This is the directory that is shared out and
mounted to all the cluster nodes from the head node.
b.
Enter the name of the network share. In this example, we will use the shared temp directory
/Headnode/Temp.
c.
Click Next.
32
Specify the Maximum Number of Jobs that can run concurrently on this Compute Server.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
b.
7.
8.
Click Next.
b.
Click Next.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
33
Configuring Remote Solver Manager (RSM) to Submit Jobs to a Linux LSF, PBS,Torque
with Moab, or UGE (formerly SGE) Cluster
a. Specify whether you want to auto-configure Compute Server directories. In this example, we will
select Yes, automatically configure directories.
b.
9.
Click Next.
Specify whether you want to create or modify another Compute Server. In this example, we will select
No.
b.
Click Next.
34
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
b.
2.
Click Next.
Specify whether you want to create a new queue or modify one already in list. In this example, we
will select Define a new Queue.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
35
Configuring Remote Solver Manager (RSM) to Submit Jobs to a Linux LSF, PBS,Torque
with Moab, or UGE (formerly SGE) Cluster
b.
3.
36
Click Next.
Enter a Name for the queue. In this example, we will enter Linux Cluster Queue. For your
configuration you can enter the actual cluster queue name that will be used to run jobs.
b.
The Compute Server you added previously (Linux Cluster, in this example) appears in the list
of compute servers. Select its check box to assign it to the new queue.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
c.
4.
Click Next.
Specify whether you want to define or modify another queue. In this example, we will select No.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
37
Configuring Remote Solver Manager (RSM) to Submit Jobs to a Linux LSF, PBS,Torque
with Moab, or UGE (formerly SGE) Cluster
b. Click Next.
2.
b.
Click Next.
38
Select an existing account to modify or specify that you want to define a new account. In this example,
we will select Define a new account.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
b.
3.
Click Next.
Enter the Username that you use to log into your Linux machine.
b.
Enter and confirm the Password that you use to log into your Linux machine.
Note
If you are going to later run a job from Windows to this Linux cluster machine, you
may need to also create an alternate Linux account that is associated with your
primary Windows account. For details refer to the Resolution in the troubleshooting
topic, Caught exception at user logon; logon failure: unknown user name or bad
password. Account password not provided. (p. 45)
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
39
Configuring Remote Solver Manager (RSM) to Submit Jobs to a Linux LSF, PBS,Torque
with Moab, or UGE (formerly SGE) Cluster
c.
4.
Click Next.
Specify if you want to define more accounts. In this example, we will select No.
b.
Click Next.
In the Queues drop-down, select the queue that you want to test.
2.
3.
Click Next.
4.
40
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Troubleshooting RSM
2.
3.
4.
5.
6.
Examples
The two examples below show the command line used to configure the Manager and Compute Server
service daemons via either the rsmconfig script or the install_daemon script.
tools/linux#> ./rsmconfig -mgr -svr
tools/linux#> ./install_daemon -mgr -svr
Once the daemon service is installed, the RSM service will be started automatically without rebooting.
The next time when the machine is rebooted, the installed RSM service will be started automatically.
4. Troubleshooting RSM
Refer to the following topics should you encounter any issues with RSM.
4.1. Gathering RSM Job Logs for Systems Support
4.2. Issue:My Computer Disabled in RSM Manager
4.3. Configuring Multiple Network Cards (NIC)
4.4. Disabling IPv6
4.5. Cannot Resolve localhost
4.6. Common Errors Found in RSM Job Log
2.
41
Configuring Remote Solver Manager (RSM) to Submit Jobs to a Linux LSF, PBS,Torque
with Moab, or UGE (formerly SGE) Cluster
3. Select the failed RSM job in the job list view.
4.
Right-click the log in the lower right pane and choose Debug Messages.
5.
6.
2.
Open a terminal window and log in to the cluster head node that is running RSM.
3.
Type cd /ansys_inc/v160/RSM/Config/tools/linux
4.
5.
42
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Troubleshooting RSM
1.
2.
Note
Record the IP address (10.2.10.32 in the above example). You will need this address in the
steps that follow.
1.
2.
Locate the Global appSettings section. If your text editor can show line numbers this section
starts on line 3.
3.
4.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
43
Configuring Remote Solver Manager (RSM) to Submit Jobs to a Linux LSF, PBS,Torque
with Moab, or UGE (formerly SGE) Cluster
5. Restart the RSM Services: open a terminal window in the [RSMInstall]/Config/tools/linux
directory and run the following commands:
./rsmmanager restart
./rsmserver restart
Note
1. When the RSM services are installed and started as daemon services by ANSYS-provided service
scripts, an rsmadmins administrative user group is automatically created on the Solve Manager
machine. An rsmadmin user account is created in the new user group. This account has administrative, non-root privileges and can be used to perform RSM administrative and configuration
tasks via the wizard on Linux.
2. On Linux, to provide additional users with RSM administrative privileges, you must add them
to the rsmadmins user group.
rhino.acme.com
x.acme.com
# source server
# x client host
44
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Troubleshooting RSM
4.6.1. Caught exception at user logon: A required privilege is not held by the client.
Resolution: Start the RSM Services manually.
1.
2.
Open a terminal window and log in to the machine that is running RSM.
3.
Type : cd /ansys_inc/v160/RSM/Config/tools/linux
4.
5.
4.6.2. Caught exception at user logon; logon failure: unknown user name or bad
password. Account password not provided.
You see one of the following errors in the RSM log file:
Compute Server running as: username
Caught exception at user logon: A required privilege is not held by the client.
or
Compute Server running as: username
Account Password not Provided
Resolution:
Right-click on My Computer in RSM Admin it and choose Set Password as your password is not set.
For the Set Password dialog box, if your name matches the one shown, press Enter.
If jobs will be submitted from a Windows client, and that account is different from the Linux account,
you will need to set up an alternate account. You can do this after you have cached your primary Windows account with RSM on your Windows client by selecting Set Password again, but this time enabling
the This is the alternate account check box before entering the credentials for the Linux account. You
can also set up an alternate account from the Accounts dialog box (right-click on My Computer in
RSM and select Accounts). If running on Linux, you do not need to enter a DOMAIN, just your username
and password.
or
No connection could be made because the target machine actively refused it.
Resolution: If you have a local firewall turned on for the server and/or RSM Client machines, you will
need to add two ports to the Exceptions List for RSM, as follows:
1.
2.
45
Configuring Remote Solver Manager (RSM) to Submit Jobs to a Linux LSF, PBS,Torque
with Moab, or UGE (formerly SGE) Cluster
If you do not have a local firewall turned on, check to see if IPv6 is enabled; if it is, disable it.
4.6.5. Caught exception from script: Failed to find the TCP port from TaskHost run.
Resolution 1: Restart the RSM services.
On Linux you can stop the RSM services manually by running the appropriate service script with the
command line option stop. The examples below illustrate how to stop the RSM services manually:
./rsmmanager stop
./rsmserver stop
You can start the RSM services manually by running the appropriate service script with the command
line option start. The examples below illustrate how to start each of the RSM services manually:
./rsmmanager start
./rsmserver start
2.
Try flushing iptables. Consult your MAN pages for instructions on how to do this.
Resolution 3: Check the permissions on the RSM scratch directory.
Check the permissions on the RSM scratch directory and ensure that all users have write access to it.
46
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Troubleshooting RSM
4.6.6. The submission of the requested job has been cancelled because the Solve
Manager . seems not fully initialized.
Resolution:
This may be a dual NIC issue. Also check for multiple RSM admins of the same version running concurrently. See the section Multiple Network Interface Cards (NIC) Issues in the Remote Solve Manager (RSM)
documentation for instructions.
4.6.7. Failed to create working directory on execution nodes via node share/mount
This is probably a case of attempting to use RSH for scratch creation when the cluster is not set up for
it; use SSH instead.
In the RSM Properties, General tab, check the box Use SSH protocol for inter and intra-node communication (Linux only) and that should resolve the issue.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
47
48
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Install ANSYS, Inc. products on each Client machine that will be submitting RSM jobs the cluster.
2.
On the Client machine, open ANSYS Workbench (Start Menu All Programs ANSYS 16.0 Workbench 16.0).
3.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
49
Submitting CFX, Fluent and Mechanical Jobs to a Linux or Microsoft HPC Cluster
4.
50
a.
b.
c.
Enter the user name and password for your alternate account to log on to the remote manager or
compute server, then click OK. This launches the Alternate Account Settings dialog box:
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
d.
Select the manager or compute server that you want to apply the alternate account to, then click
Done.
Proceed to the sections that follow to learn how to send your job to your Linux or Windows cluster.
Open ANSYS Workbench (Start Menu All Programs ANSYS 16.0 Workbench 16.0).
2.
3.
In the CFX system, right-click the Solution cell and select Properties.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
51
Submitting CFX, Fluent and Mechanical Jobs to a Linux or Microsoft HPC Cluster
4.
5.
52
b.
For Solve Manager, type the name of the Manager that will be used. (If you do not know the name
of the Solve Manager, contact your System Administrator for this information.)
c.
For Queue, enter the name of the queue that will be used. (If you do not know the name of the
Queue, contact your System Administrator for this information.)
d.
For automatic downloading of progress information, verify that Download Progress Information
is set to Always Download.
e.
Leave the Download Progress Information at the default of 120 seconds (or a different value depending on how frequently you would like the solver to query RSM for output files in order to display
progress). Note that if the job finishes before the first interval is reached, you will not see progress
results until the end of the job.
f.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
6.
Open ANSYS Workbench (Start Menu All Programs ANSYS 16.0 Workbench 16.0).
2.
3.
In the Fluent system, right-click the Solution cell and select Properties.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
53
Submitting CFX, Fluent and Mechanical Jobs to a Linux or Microsoft HPC Cluster
4.
54
b.
c.
For Solve Manager, type the name of the Manager that will be used. (If you do not know the name
of the Solve Manager, contact your System Administrator for this information.)
d.
For Queue, enter the name of the queue that will be used. (If you do not know the name of the
Queue, contact your System Administrator for this information.)
e.
f.
Set the Progress Download Interval to the default of 120 seconds (or a different value depending
on how frequently you would like the solver to query RSM for output files in order to display progress).
Note that if the job finishes before the first download interval is reached, you will not see progress
results until the end of the job.
g.
h.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
5.
Open ANSYS Workbench (Start Menu All Programs ANSYS 16.0 Workbench 16.0).
2.
Add a Mechanical system and assign a geometry, establish all necessary loads, and so on.
3.
On the analysis system on the Project Schematic, double-click either the Model or the Setup cell to launch
Mechanical.
4.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
55
Submitting CFX, Fluent and Mechanical Jobs to a Linux or Microsoft HPC Cluster
5.
6.
7.
56
a.
b.
Click OK. The Rename Solve Process Settings dialog box closes.
Select the solve process setting you just specified from the list on the left.
b.
Under Computer Settings, enter the machine name of the Solve Manager. (If you do not know the
name of the Solve Manager, contact your System Administrator for this information.)
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
8.
9.
c.
For Queue, enter the name of the queue that will be used. (If you do not know the name of the
Queue, contact your System Administrator for this information.)
d.
Click Advanced.
b.
c.
In the Solve Process Settings dialog box, click OK. The dialog box closes and the solve process setup is
complete.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
57
Submitting CFX, Fluent and Mechanical Jobs to a Linux or Microsoft HPC Cluster
10. In Mechanical, finish setting up your analysis. When the model is set up and ready to solve, open/launch
Mechanical and select the Solve toolbar button drop-down arrow. You will see the solve process name
you just defined (in this example, Cluster). Select that process.
11. The solve commences. When the solution has completed, the Solution branch and the items underneath
it in the project tree will each have a down-arrow next to them.
12. Right-click Solution and select Get Results to bring the solution items to the local machine.
58
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Choose Start Menu All Programs ANSYS 16.0 Remote Solve Manager RSM 16.0, then
right-click the shortcut and choose Run as Administrator.
b.
From the Tools menu choose Options. In the Name field type the name of the Solve Manager. (If
you do not know the name of the Solve Manager, contact your System Administrator for this information.)
c.
2.
3.
Right-click on the log in the lower right pane and select Debug Messages.
4.
5.
59
60
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
61
2.1. Creating the RSM Compute Server for Custom Cluster Type Keyword
Perform the following steps on your Windows RSM Client machine to configure RSM to use a custom
client-side integrated cluster. In this section, we are adding a Custom Linux cluster as the Compute
Server which can have user-programmed inputs.
1. Underneath the local Manager (My Computer) in the RSM tree view, right-click the Compute Servers
folder and select Add.
62
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
If you do not have RSH enabled on the cluster, then check Use SSH protocol for intra node communication. This means that the remote scripts will use SSH to contact other machines in the cluster.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
63
4. On the File Management tab, set properties as shown below. For more information on file management
and directory handling, see Compute Server Properties Dialog: File Management Tab in the Remote Solve
Manager User's Guide.
a. For the Remote Shared Cluster Directory property, enter the path to your central cluster file-staging
directory. This should be a directory that the cluster execution nodes share and all have mounted so
that every execution node can access the input files once they are moved there.
The Shared Cluster Directory is typically located on the machine defined on the General tab.
However, in this example, the General tab specifies localhost. Since we have set up and are
modifying the remote Manager from the Client machine, the directory reference here will be to
the remote machine. The RSM job needs to find this shared directory on the remote machine.
In this example, /path/to/shared/cluster/directory is a network share that all of the
cluster nodes have mounted.
b. Select Transferred by an external mechanism (e.g. SSH).
64
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
5. Select the General tab again. Now we can set the location of the Working Directory, which is used to store
all of the client files before sending them to the remote machine.
For the Working Directory Location property, select Reuse Manager Storage. This will reuse the
RSM Manager's project storage directory as the Working Directory.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
65
Note
The queue Name will be presented to the cluster directly, so this queue name should
match the desired submission queue name exactly. A Compute Server can be placed in
more than one queue in RSM, so you can submit to any number of queues enabled on
the cluster in this way.
3. The Compute Server you added previously (Client Side Integration Example in this example) appears
under Assigned Servers. Select the check box next to it to assign the server to this queue.
66
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
3.2.1. Modifying the Job Configuration File for the New Cluster Type
As part of the setup, you must add an entry for your custom cluster keyword in the jobConfiguration.xml file, and reference the HPC commands file that is needed for this cluster job type.
1. Navigate to [ANSYS 16.0 Install]/RSM/Config/xml.
2. Open the jobConfiguration.xml file and add an entry for your custom cluster job type. The sample
entry below is for the CUS_CLIENT keyword that we established earlier, and points to the custom
hpc_commands_CUS_CLIENT.xml file. Use your own keyword and HPC commands file name where
appropriate.
<keyword name="CUS_CLIENT">
<jobCode name="GenericJobCode_base.xml"/>
<hpcCommands name="hpc_commands_CUS_CLIENT.xml"/>
</keyword>
3.2.2. Modifying the Custom HPC Commands File to Reference Custom Scripts
Below is the entire hpc_commands_CUS_CLIENT.xml file in its unmodified form.
<?xml version="1.0" encoding="utf-8"?>
<jobCommands version="3" name="Custom Cluster Commands">
<environment>
<env name="RSM_HPC_PARSE">LSF</env>
<env name="RSM_HPC_PARSE_MARKER">START</env> <!-- Find "START" line before parsing according to parse
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
67
In the HPC Commands file shown above, you have only two steps to finish:
1. Referring to the example below, replace all of the Generic and SSH references with _CUS_CLIENT references (or your specific keyword), as was done in Making a Copy of CIS Example Files from RSM Directories (p. 67) above.
2. Prepend a directory reference to each script file. The example below uses the
%RSM_HPC_SCRIPTS_DIRECTORY_LOCAL% variable, which is appropriate for custom client integrations
where you are using the RSM scripts directory location. This variable is set automatically by RSM.
<?xml version="1.0" encoding="utf-8"?>
<jobCommands version="3" name="Custom Cluster Commands">
68
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Note
If you want to use other types of code such as C++, that is acceptable if you simply place your
compiled (executable) code in the <app> </app> section, arguments are not required. For
Python, an interpreter is included in the ANSYS Workbench install, so that is what you see referenced. If you want to use Python you can simply replace <app> </app> with <pythonapp>
</pythonapp> as shown and enter the Python code file name.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
69
Important
The scripts submitGeneric.py and cancelGeneric.py that you have copied and renamed to submit_CUS_CLIENT.py and cancel_CUS_CLIENT.py actually contain fully
functional code. However, the code could be considered to be quite complex, and going
over it in detail is beyond the scope of this tutorial. These scripts are intended for more advanced programmers in customizing the code.
Here we have provided simpler, commented versions of these scripts with only basic functionality, so that the scripts may be more easily understood by newer programmers. We have
illustrated the inner workings of these scripts so that you can modify them or write your
own scripts based on your specific needs.
If you want to use the simpler scripts, you can simply replace the content in the original
scripts with the following examples for submit_CUS_CLIENT.py and cancel_CUS_CLIENT.py.
70
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
71
This script will only submit an LSF job, but you can choose to use the built in RSM parsing
to get the JOB ID for you, or you can use CUSTOM parsing, which just means you must find it yourself
in the output of the bsub command. Parsing must line up with the value of RSM_HPC_PARSE Set in
hpc_commands_Keyword.xml
When CUSTOM is set, its set for submit, cancel, transfer, etc. at the same time, so you must change
all the scripts to output the proper print statements for CUSTOM parsing when you change this mode.
Both ways are illustrated here, obviously if you have an LSF cluster, using the LSF parsing is the
easiest way. But if you are using a cluster that is only "Like" LSF, you will need to figure out
how to parse all the commands yourself.
_parseType = os.getenv("RSM_HPC_PARSE")
if (_parseType == "LSF"):
for line in _process.stdout:
print line
sys.exit(0)
elif (_parseType == "CUSTOM"):
# It is optional to print the 'START' output, usually this is only done if
# previous command output #could be confused by RSM with the intended output. Since we are
# using CUSTOM RSM_HPC_PARSE the output RSM needs is very specific and this line isnt really needed.
print('START')
_jobid = None
for line in _process.stdout:
print 'RSM_HPC_DEBUG='+line
if line.startswith('Job <'):
# See Below #6
_jobid = line.split('<')[1]
_jobid = _jobid.split('>')[0]
print 'RSM_HPC_JOBID=' + _jobid
if _jobid == None:
print 'RSM_HPC_ERROR=Job not submitted'
sys.exit(1)
sys.exit(0)
Note
This code references many RSM-set environment variables. For more information on what
environment variables are available and their contents, see Environment Variables Set by
RSM in the Remote Solve Manager User's Guide.
1. You can add any code you want to this section; code placed here will execute before the job is submitted.
Also, you can stop the job from submitting with some controls on the Submit command, if desired.
2. Basic LSF command line starting point; we will continuously append arguments to this line as necessary
to complete the command.
3. Most blocks are comprised of three parts: storing an environment variable to a local variable, testing to
ensure that a variable either isnt empty or contains a special value, and then appending some flag to the
command line based on the findings.
4. One of the final actions is to read the RSM_HPC_COMMAND variable and append it to the submission
command. This command is created by RSM and contains the command line to run the ClusterJobs
script which can complete the submission process. It creates the full command line for ANSYS by using
the controls file created by the individual add-ins. ANSYS suggests that you always use the
RSM_HPC_COMMAND to submit a job whenever possible because of the complexities of the ANSYS command
line for different solvers and on different platforms.
5. Popen finally runs the command we have been building. Then we wait for it to finish.
72
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
73
Note
This code references many RSM-set environment variables. For more, information on what
environment variables are available and their contents, see Environment Variables Set by
RSM in the Remote Solve Manager User's Guide.
1. You can add any code you want to this section; code placed here will happen before the job is cancelled.
Also, some code could be run at the end of the script just before sys.exit(0), if some extra precautions
are to be taken after the job has been cancelled through the scheduler.
2. Basic LSF command line starting point. You would type bkill <job ID> at the command line in order
to cancel a job in LSF. We will continuously append arguments to this line as necessary to complete the
command. In this case, its only the job number being added in block #4.
3. Most blocks are comprised of three parts: storing an environment variable to a local variable, testing to
ensure that a variable isnt empty, and then appending some flag to the command line (or stopping the
command if an error is found) based on the findings. This environment variable is set by RSM. A list of these
useful variables can be found in Custom Integration Environment Variables in the Remote Solve Manager
User's Guide.
4. Popen finally runs the command we have been building. Then we wait for it to finish.
5. Finally, we simply print out all of the output along with a line that says that the command has finished,
just so we know it has run properly through RSM. Unlike the Submit command, the Cancel command has
no output requirements, as shown in the Cancel Command section of the Remote Solve Manager User's
Guide.
74
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
3. When the test job completes, you can view job details in the RSM Progress Pane.
If the test runs successfully, continue to the next section.
4. If the test fails:
a. Check to see if any firewalls are turned on and blocking the connection between the two machines.
b. Make sure you can reach the machine(s) via the network.
c. Attempt to use plink.exe from the command prompt and connect to the remote machine this way.
i.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
75
76
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
77
Note
If you are using SSH, see Appendix B: Integrating Windows with Linux using SSH/SCP
in the Remote Solve Manager User's Guide for instructions.
78
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
3. Enter the Linux Cluster Name (or IP address) into the Name field and click Add.
4. Select both the local and new remote Manager and then click OK.
Check in the UI to verify that the new machine has shown up. The first time you connect to it,
it should prompt you to set a password (covered in step 5).
5. Cache your login on this machine to gain access to change the properties. Your system administrator
needs to have added your login to be in the rsmadmins group. You will be setting up the Manager service
on the Linux machine remotely from your Client machine to make it easier. If you get a credentials
error, review Before You Begin (p. 78) section and/or have your systems administrator set up the
cluster as described and add you to the rsmadmins group.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
79
Note
This tutorial will use Tester1rsm as the remote Manager in the examples. We will be
configuring this REMOTE Manager, not My Computer from now on.
2.2. Creating the RSM Compute Server for Custom Cluster Type Keyword
Perform the following steps on your Windows RSM Client machine to configure RSM to use a custom
server-side integrated cluster. In this section, we are adding a Custom Linux cluster as the Compute
Server that can have user-programmed inputs.
1. Underneath the remote Manager in the RSM tree view, right-click the Compute Servers folder and select
Add.
80
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
3. On the Cluster tab of the Compute Server Properties dialog box, set properties as follows:
a. Set the Cluster Type property. In this example, well select CUSTOM.
b. For the Custom Cluster Type property, enter a short, descriptive name. This is your keyword and will
need to be appended to some filenames later, so try to keep it simple. For this example we will use
SHEF01.
c. In this example, we use optional Job Submission Arguments to override the queue name and force
it to be all.q regardless of the queue Name created in the next section. This is not required, however;
it is shown here only as an example of the functionality. Often, this box is left blank so that any
number of queues can be setup for this Compute Server as shown in the next section. Refer to your
specific clusters documentation for the exact commands that can be used here.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
81
4. On the File Management tab of the Compute Server Properties dialog box, set properties as follows:
a. Before we look at the Shared Cluster Directory, we should decide a File Management method. For
this example, we will choose to run the job In the Shared Cluster Directory.
For more information on file management and directory handling, see Compute Server Properties
Dialog: File Management Tab in the Remote Solve Manager User's Guide.
b. For the Shared Cluster Directory property, enter the path to your central cluster file-staging directory.
This should be a directory that the cluster execution nodes share and all have mounted so that every
execution node can access the input files once they are moved there.
The Shared Cluster Directory is located on the machine defined on the General tab. In this
example, the General tab specifies localhost, and we have set up and are modifying the remote
Manager from the Client machine. So the directory reference will be to the remote machine. The
RSM job needs to find the shared directory there. In this example,
/path/to/shared/cluster/directory is a network share that all of the cluster nodes
have mounted.
82
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Note
The directories you enter here must match the directory names exactly (capitalization
carries over to Linux). If the directory names do not match exactly, the process will fail.
2. Under General in the Queue Properties dialog box, enter a Name for this queue. In this example, we will
use Custom_Queue.
3. The Compute Server you added previously (Tester1 Custom Cluster in this example) appears under
Assigned Servers. Select the check box next to it to assign the server to this queue.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
83
Note
If this is not your configuration as stated in Before You Begin (p. 78), then this scripting
method could fail. This method ensures that all users use the same scripts. A method for
applying different scripts for different groups is also allowed, but not covered in this tutorial
and is not the preferred method.
84
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
3.3.1. Modifying the Job Configuration File for the New Cluster Type
As part of the setup, you must add an entry for your custom cluster keyword in the jobConfiguration.xml file, and reference the HPC commands file that is needed for this cluster job type.
1. Navigate to [ANSYS 16.0 Install]/RSM/Config/xml.
2. Open the jobConfiguration.xml file and add an entry for your custom cluster job type. The sample
entry below is for the SHEF01 keyword that we established earlier, and points to the custom hpc_commands_SHEF01.xml file. Use your own keyword and HPC commands file name where appropriate.
<keyword name="SHEF01">
<jobCode name="GenericJobCode_base.xml"/>
<hpcCommands name="hpc_commands_SHEF01.xml"/>
</keyword>
3.3.2. Modifying the Custom HPC Commands File to Reference Custom Scripts
As part of the setup, you must edit the cluster-specific HPC Commands file provided as part of the RSM
installation. A reference example of an unmodified HPC Commands file will be followed by instructions
on how to modify it and an example of the completed HPC Commands file.
Note
Commands files for different cluster types are sometimes very different, so this may not
look like yours if you have started from LSF or PBS scripts, but you should still find sections named similarly even if the actual commands are different than SGE/UGE as shown.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
85
86
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
In the HPC Commands file shown above, you need to do two things:
1. Replace all of the Submit command, between <primaryCommand name =submit> and
</primaryCommand>, with the new (much shorter) code reference to the
%RSM_HPC_SCRIPTS_DIRECTORY%/CustomSubmissionCode.py as shown below.
2. Replace all of the Cancel command, between <primaryCommand name =cancel> and
</primaryCommand>, with the new code reference to the %RSM_HPC_SCRIPTS_DIRECTORY%/CustomCancelCode.py as shown below. Modifications are in bold text.
Note
Replacing the references to this code here means that when RSM needs to Submit a
job or Cancel a job, it will now use this new code to do so. Changes made to these
scripts/code will be immediately implemented into RSM.
<?xml version="1.0" encoding="utf-8"?>
<jobCommands version="3" name="Custom Cluster Commands">
<environment>
<env name="RSM_HPC_PARSE">SGE</env>
<env name="RSM_HPC_JOBNAME">RSM</env>
<env name="RSM_HPC_PARSE_MARKER">START</env>
</environment>
<submit>
<precommands>
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
87
88
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Note
If you want to use other types of code such as C++, that is acceptable if you simply place your
compiled (executable) code in the <app> </app> section, arguments are not required. For
Python, an interpreter is included in the ANSYS Workbench install, so that is what you see referenced. If you want to use Python you can simply replace <app> </app> with <pythonapp>
</pythonapp> as shown and enter the Python code file name.
Any custom code that you want to provide as part of the customization should also be located
in the [RSMInstall]\RSM\Config\scripts directory corresponding to the remote (Manager machine)
installation. Alternatively, you must enter a full path to the script along with the name.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
89
90
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
Note
This code references many RSM-set environment variables. For more, information on what
environment variables are available and their contents, see Environment Variables Set by
RSM in the Remote Solve Manager User's Guide.
1. You can add any code you want to this section, code placed here will execute before the job is submitted.
Also, you can stop the job from submitting with some controls on the Submit command, if desired.
2. Basic SGE command line starting point. We will continuously append arguments to this line as necessary
to complete the command.
3. Most blocks are comprised of three parts: storing an environment variable to a local variable, testing to
ensure that a variable either isnt empty or contains a special value, and then appending some flag to the
command line based on the findings.
4. One of the final actions is to read the RSM_HPC_COMMAND variable and append it to the submission
command. This command is created by RSM and contains the command line to run the ClusterJobs
script that can complete the submission process. It creates the full command line for ANSYS by using the
controls file created by the individual add-ins. ANSYS suggests that you always use the RSM_HPC_COMMAND
to submit a job whenever possible because of the complexities of the ANSYS command line for different
solvers and on different platforms.
5. Popen finally runs the command we have been building. Then we wait for it to finish.
6. Finally, print any output that came from it so RSM can interpret it and obtain the job #.
Since this script is a Submit script, there are many options for qsub command. However, it is much
simpler to create a custom script for the Cancel command, although it contains the same basic parts.
This process is addressed in the next section.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
91
Note
This code references many RSM-set environment variables. For more information on what
environment variables are available and their contents, see Environment Variables Set by
RSM in the Remote Solve Manager User's Guide.
1. You can add any code you want to this section; code placed here will happen before the job is cancelled.
Also, some code could be run at the end of the script just before sys.exit(0), if some extra precautions
are to be taken after the job has been cancelled through the scheduler.
2. Basic SGE command line starting point: qdel is what you would type at the command line in order to
cancel a job in SGE. We will continuously append arguments to this line as necessary to complete the
command.
92
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
3. When the test job completes, you can view job details in the RSM Progress Pane.
If the test runs successfully, continue to the next section.
4. If the test fails:
a. Check to see if any firewalls are turned on and blocking the connection between the two machines.
b. Make sure you can reach the machine(s) via the network.
c. Add RSM ports to the firewall as needed. If you have a local firewall turned on (Compute Server and
RSM Client machines), you will need to add the following two ports the Exceptions List for RSM:
Add port 8160 to firewall exceptions for Ans.Rsm.SHHost.exe.
Add port 9160 to firewall exceptions for Ans.Rsm.JMHost.exe.
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.
93
94
ANSYS Release 16.0 - SAS IP, Inc. All rights reserved. - Contains proprietary and confidential information of ANSYS, Inc. and its subsidiaries and affiliates.