Beruflich Dokumente
Kultur Dokumente
SANsymphony-V
September 2015
DataCore cannot guarantee that following these best practices will result in a perfect
solution – there are simply too many separate (and disparate) components that make up a
complete SANsymphony-V installation, most of which are out of DataCore Software’s
control – but these guidelines will significantly increase the likelihood of a more secure,
stable and high-performing SANsymphony-V installation.
It does not replace DataCore technical training nor does it remove any need to use a
DataCore Authorized Training Partner and assumes fundamental knowledge of Microsoft
Windows and the SANsymphony-V suite of products.
Also see
DataCore Training
http://www.datacore.com/Support/Training.aspx
Also see
End of Life Notifications
http://datacore.custhelp.com/app/answers/detail/a_id/1329
Keep it simple
Avoid complexity at all levels.
Overly complicated environments may appear to cover all possible situations, but the more
complex the design, the more likely unforeseen errors will occur and can make the system
difficult to maintain and support especially when adding to it in future.
Knowledge
Document and share information
Document the environment properly; keeping it up-to-date and accessible. Establish
'shared knowledge' between at least two people who have been trained and are familiar
with all areas of the infrastructure.
Memory
Amount of memory
The minimum amount of memory for a DataCore Server is 8GB. The recommended amount
of memory for a DataCore Server depends entirely on the size and complexity of the
SANsymphony-V configuration. Use the ‘DataCore Server Memory Considerations’
document available from the Support Website;
http://datacore.custhelp.com/app/answers/detail/a_id/1543 to calculate the memory
requirement for the type, size and complexity of the SANsymphony-V configuration, always
allow for plans of future growth.
Memory type
To avoid any data integrity issues while I/O is being handled by the DataCore Server’s own
Cache, ECC Memory Modules should be used.
If applicable, enable the server hardware’s Command per Clock (CPC) setting in the server
BIOS (sometimes known as CPC Mask or CPC Override).
Also see
Recommended BIOS settings on a DataCore Server
http://datacore.custhelp.com/app/answers/detail/a_id/1467
CPU
NB. Unless specified, a ‘CPU’ in this section can refer to a single physical core, a single hyper-
threaded core or a single ‘virtual’ CPU (e.g. when running a DataCore Server inside a Virtual
Machine).
Processor type
Any x64 processor (except for Intel’s Itanium family) is supported for use in a DataCore
Server but use ‘server-class’ processors rather than those intended for ‘workstations’.
Processor speed
DataCore Software has not found any significant performance differences between the CPU
manufacturers for processors with similar architectures and frequencies when running
SANsymphony-V.
Faster (higher frequency) CPUs are always preferred over slower CPUs, as they can process
more instructions per second and so we recommend using faster cores over more (slower)
cores. Additional CPU Sockets may be necessary to use all of the available PCIe/Memory-
Sockets on the server’s motherboard. Please consult your server hardware vendor.
SANsymphony-V constantly polls all Fibre Channel and iSCSI ports so as to handle I/O
requests immediately, as they arrive (in the case of Front-end or Mirror ports) or when
they need to be sent on (in the case of Mirror or Back-end). Waiting for CPUs to ‘wake’ from
one of these low-power states interferes with this polling process and significantly adds to
overall I/O latency. Wherever possible, disable all ‘low-power’ settings on all CPUs and
ensure that they are always running at the maximum possible power and speed. Please
refer to the server vendor’s own documentation on how to adjust this setting.
Also see
‘Recommended BIOS settings on a DataCore Server’ from the Support Website:
http://datacore.custhelp.com/app/answers/detail/a_id/1467
1ISCSI connections generally have a much larger overhead, when compared to fibre channel ports, due to the
extra work required encapsulating and de-encapsulating of SCSI data to and from their IP packets.
2This does not mean that SANsymphony-V can only manage a maximum of 10 Fibre Channel or iSCSI ports
but that more CPUs over and above 10 will not necessarily provide any more significant performance gains
whereas using more than 10 ports will always result in increased throughput.
There will always be contention from other vCPUs on other Virtual Machines (i.e. that are
being used for Hosts) and even if the Hypervisor could ‘dedicate’ physical CPUs just for the
use of the DataCore Server’s virtual machine it would almost certainly come at a cost for
those same Host virtual machines, which would be restricted to the remaining CPUs on the
Hypervisor for all their workloads.
Even if the same number of vCPUs were to be created to match those of an equivalent
physical DataCore Server, there is still no guarantee that all of these vCPUs would be used
at the same rate and throughput as a physical DataCore Server would with physical CPUs.
We therefore recommend, for a DataCore Server running inside a virtual machine, to use
between 4 and 6 vCPUs depending on workload and the number of additional features
being used.
Also see
Installing a DataCore Server within a Virtual Machine
http://datacore.custhelp.com/app/answers/detail/a_id/1155
Power
Use redundant and uninterruptable power supplies (UPS).
Also see
UPS Support
http://www.datacore.com/SSV-Webhelp/DataCore_Servers_and_Server_Groups.htm
RAID and Storage Array controllers used to manage physical disks in Disk Pools need to be
able to handle I/O from multiple Hosts connected to the DataCore Server; so for high
performance/low latency hardware, use independent buses or backplanes designed for
‘heavy’ workloads (‘workstation’ hardware is not usually designed for such workloads).
Also see
Storage Array Guidelines for DataCore Servers
http://datacore.custhelp.com/app/answers/detail/a_id/1302
There is however a significant redundancy implication when using single adapters with
multiple ports on them as most hardware failures usually end up affecting all ports rather
than just one. Using adapters that have a smaller number of ports on them minimizes the
risk of multiple failures happening at the same time.
Use independent switches for redundant fabrics; this also prevents ‘network-loops’ making
spanning tree protocols obsolete and simplifies the overall iSCSI implementation.
1This assumes that there are always an adequate number of ‘PCIe Lanes’ available in the PCI Slot being used for the
adapter. Please refer to your server hardware vendor’s own documentation for this.
ISCSI connections
Multi-ported adaptors
Adapters used for iSCSI connections are usually available in either single, dual or quad-port
configurations and often there is little difference in performance capabilities when, for
example, comparing two single-ported adapters with one dual-port adaptor or two dual-
port adaptors with a one quad-port adapter1.
There is however a significant redundancy implication when using single adapters with
multiple ports on them as most hardware failures usually end up affecting all ports rather
than just one. Using adapters that have a smaller number of ports on them minimizes the
risk of multiple failures happening at the same time.
Fundamentally, SCSI load-balancing and failover functions are managed by Multipath I/O
protocols1; TCP/IP uses a completely different set of protocols for its own load-balancing
and failover functions. When SCSI commands, managed by Multipath I/O protocols but
‘carried’ by TCP/IP protocols are combined (i.e. iSCSI), then interaction between the two
protocols for the same function can lead to unexpected disconnections or even complete
connection loss.
It is not recommended to use NIC teaming with iSCSI connections as it adds more
complexity without any real gain in performance; and although teaming iSCSI Targets (i.e.
Front-end or Mirror ports) would increase the available bandwidth to that target, it still
only allows a single target I/O queue rather than, for example, two, separate NICs which
would allow two, independent target queues with the same overall bandwidth. None of the
Spanning Tree Protocols (STP, RSTP and MSTP) on networks for iSCSI connections are
recommended either as they may cause unnecessary interruptions to I/O; for example,
other, unrelated devices generating unexpected network-topology causing STP to re-route
iSCSI commands inappropriately or even blocking them completely from their intended
target.
Also see:
For a list of adapters that can be used for Fibre Channel or iSCSI connections in a DataCore
Server see ‘Qualified Hardware Components’
http://datacore.custhelp.com/app/answers/detail/a_id/1529 from the Support Website.
1Mirrored Virtual Disks that are configured to use multiple iSCSI Mirror paths on the DataCore Server are, by default,
auto-configured to be managed by Microsoft’s MPIO using the ‘Round Robin with Subset’ Policy.
The controller node is also responsible for the management and propagation of any
configuration changes made in the SANsymphony-V Management Console regardless of
which DataCore Server’s configuration is being modified, and makes sure that all other
DataCore Servers in the Server Group always have the most recent and up-to-date changes.
The ‘election’ of which DataCore Server is to become the controller node is decided by the
SANsymphony-V software automatically and whenever;
The decision on which DataCore Server becomes the controller node is decided
automatically between all the Servers in the Group and cannot be manually configured.
It is also important to understand that the controller node does not manage any Host,
Mirror or Back-end I/O (i.e. in-band connections) for other DataCore Servers in the Server
Group. In-band I/O is handled by each DataCore Server independently of the other Servers
in the Server Group, regardless if it is the elected controller or not. Nor does it send or
receive Replication data configured for another DataCore Server in the same Server Group,
although it will manage all Replication configuration changes and Replication status
updates regardless if it is the Source Replication Server or not.
This means that all DataCore Servers in the same Server Group must have a routable
TCP/IP connection to each other so that if the controller node ‘moves’ to a different server,
then the new controller node must also be able connect to all of the remaining DataCore
Servers in the group1.
This means that even if the workstation is on a separate network segment from the
DataCore Servers (e.g. in a different vLAN) it must still be able to send and receive TCP/IP
traffic to and from all the DataCore Servers in that vLAN.
Network connections for specific SANsymphony-V functions
SANsymphony-V’s Management Console, the VMware vCenter Integration component
Replication and Performance Recording function (when using a remote SQL Server) all use
their own separate TCP/IP session. To avoid unnecessary network congestion and delay as
well as losing more than one of these functions at once should any problems occur with one
or more network interfaces, we recommend using a separate network connection for each
function.
We also recommend that each NIC that is teamed is in its own separate network and that
‘failover’ mode is used rather than ‘load balancing’ as there is no specific performance
requirement for Inter-node communication as the TCP/IP and using ‘fail over’ mode means
that configuring and managing the network connections and switches is simpler. It also
makes troubleshooting any future connection problems easier as well.
1‘Re-election’ of the controller node takes place if the node is shutdown or if it becomes unavailable on the
network to the rest of the Server Group for any reason.
CPU
Use ‘server class’ processors.
Always enable Hyper-Threading (if available).
Always disable ‘Turbo Boost’ (applies to Intel CPUs only).
Disable all CPU ‘low-power’ modes (also known as ‘C states’).
Power
Use redundant power supplies.
Use an uninterruptable power supply (UPS).
Also see ‘Storage Array Guidelines for DataCore Servers’ from the Support
Website: http://datacore.custhelp.com/app/answers/detail/a_id/1302
ISCSI connections
Use many adapters with a smaller numbers of ports on them as opposed to fewer
adapters with larger numbers of ports.
Use faster, separate network adaptors instead of NIC teaming.
Do not use NIC teaming or STP protocols with iSCSI connections. Use more,
individual network connections (with Multipath I/O software) to manage
redundancy.
Use independent network switches for redundant iSCSI networks.
The default size of the page file created by Windows is determined by the amount of Physical
Memory installed in the DataCore Server and the type of memory dump that is configured.
The SANsymphony-V installer will automatically change the memory dump type – usually
Automatic memory dump – to Kernel Memory Dump (see the section ‘Startup and
Recovery/System Failure’ above) to make sure that if any crash analysis is required, the
correct type of dump file will be generated by the Windows Operating System.
This may mean that a DataCore Server that has a relatively small boot disk and significantly
‘large’ amounts of physical memory installed results in a page file that fills (or nearly fills) the
boot disk after the installation of SANsymphony-V. In this case, it is still recommended to keep
the Kernel Memory Dump setting but manually enter a custom value for the page file size as
large as practically possible by unchecking the ‘Automatically manage paging file size for all
drives’ option.
Power Options
Select the High Performance power plan under Control Panel\Hardware\Power Options.
Examples include:
SANsymphony-V Management Console Tasks that use a Scheduled Time trigger.
Continuous Data Protection’s retention time settings.
Time-sensitive licenses (e.g. that contain fixed, expiration dates either for evaluation
or migration purposes).
It will also help stop misleading ‘Configuration Conflict’ warnings between DataCore
Servers in the same Server Group (e.g. after a planned DataCore Server reboot).
It is also recommended to synchronize all host’s system clocks as well as any SAN or
Network switch hardware clocks (if applicable) with the DataCore Servers. This can be
especially helpful when using DataCore’s VSS on a host (where backup and restore
operations may take place); but generally to help with any troubleshooting where a host’s
own system logs need to be checked against those of a DataCore Server. Many ‘SAN events’
often occur over very short periods of time (e.g. Fibre Channel or ISCSI disconnect and
reconnection issues between Hosts and DataCore Servers).
Also see
SANsymphony-V’s minimum requirements
http://www.datacore.com/products/technical-information/SANsymphony-V-
Prerequisites.aspx
1 This also includes all Microsoft Windows’ ‘Cumulative Updates’ and ‘Update Rollups’.
Never upgrade ‘in-place’ to a newer version of Windows operating system, for example
upgrading from Windows 2008 to Windows 2012 or upgrading from Windows 2012 to
Windows 2012R2; even if the newer version is considered qualified by DataCore the
upgrade will stop the existing SANsymphony-V installation from running. Instead of an in-
place upgrade the DataCore Server’s operating system must be installed ‘as new’.
R2 versions of a particular Windows Operating System also need to be qualified for use on
a DataCore Server. Any ‘R2’ versions of Windows that have passed qualification for a
specific version of SANsymphony-V will be listed in both the SANsymphony-V Software
release notes and the SANsymphony-V minimum requirements page.
Also see
Please refer to ‘Reinstalling the DataCore Server's Windows Operating System’ from
the Support Website: http://datacore.custhelp.com/app/answers/detail/a_id/1537 which
gives instruction on how to manage this, while at the same time retaining the existing
SANsymphony-V configuration.
The SANsymphony-V Software release notes are available either as a separate download or
come bundled with the SANsymphony-V software. For the latest SANsymphony-V release
notes see: ‘DataCore Software Downloads’ from the Support Website:
http://datacore.custhelp.com/app/answers/detail/a_id/1419
Third-party software
It is recommended not to install third-party software on a DataCore Server. SANsymphony-
V requires significant amounts of system memory as well as CPU processing; it will also
prevent certain system devices (e.g. Disk devices) from being accessed by other software
components that may be installed on the DataCore Server which may lead to unexpected
errors from those other software components.
The purpose of the DataCore Server should not be forgotten and trying to run the DataCore
Server as a Domain Controller or as a Mail Server/Relay for example, as well as
SANsymphony-V, must not be done as this will affect the overall performance and stability
of the DataCore Server.
DataCore recognize that ‘certain types’ of third-party software are required to be able to
integrate the DataCore Server onto the user’s network. These include:
In these few cases, and as long as these applications or agents do not need exclusive access
to components that SANsymphony-V needs to function correctly (i.e. Disk, Fibre Channel or
iSCSI devices), then it is possible to run these alongside SANsymphony-V.
Always consult the third-party software vendor for any additional memory requirements
their products may require and refer to the ‘Known Issues - Third-party Hardware and
Software’ document for any potential problems with certain types of third-party software
that have already been found to cause issues or need additional configuration.
DataCore Support may ask for third-party products to be removed in order to assist with
Troubleshooting.
Also see
Known Issues - Third-party Hardware and Software
http://datacore.custhelp.com/app/answers/detail/a_id/1277
ISCSI connections
For a list of adapters that can be used for iSCSI connections in a DataCore Server see
‘Qualified Hardware Components’
http://datacore.custhelp.com/app/answers/detail/a_id/1529 from the Support Website.
Also see
Known Issues - Third-party Hardware and Software
http://datacore.custhelp.com/app/answers/detail/a_id/1277
This includes:
The Connection Interface’s default setting (‘All’) means that SANsymphony-V will use any
available network interface on the DataCore Server for its host name resolution, this is
determined by the Windows operating system and how it has been configured and
connected to the existing network.
It is possible to change this setting, and choose an explicit network interface (i.e. IP
Address) to use for host name resolution instead, but this requires that the appropriate
network connections and routing tables have been set up correctly and are in place.
SANsymphony-V will not automatically retry other network connections if it cannot resolve
to a hostname using an explicit interface.
We recommend leaving the setting to ‘All’ and use the appropriate ‘Hosts’ file or DNS
settings to control host name resolution.
DataCore do recommend however using Host Name resolution over just using IP addresses
as it is easier to manage any IP address changes that might occur, planned or unexpected,
by being able to simply update any ‘Hosts’ file or DNS entries instead of ‘reconfiguring’ a
Replication group or remote SQL server connection for Performance Recording (i.e.
manually disconnecting and reconnecting), which is disruptive.
When using a ‘Hosts’ file, do not add any entries for the local DataCore Server but only for
the ‘remote’ DataCore Servers and do not add multiple, different entries for the same
server (e.g. each entry has a different IP address and/or server name for the same server)
as this will cause problems when trying to (re)establish network connections.
Replication Groups
Please see the section TCP/IP connections between Replication Groups on page 28 for
Replication-specific TCP/IP configuration information.
Third-party Software
It is recommended not to install third-party software on a DataCore Server.
ISCSI connections
See ‘ISCSI with NIC teaming/link aggregation/Spanning Tree Protocols
(STP/RSTP/MSTP)’ on page 9.
Therefore, the disk device that holds the Replication buffer should be able to manage at
least 2x the write throughput for all replicated Virtual Disks combined together.
If the disk device used to hold the Replication buffer is too slow it may not be able empty
fast enough (so as to be able to accommodate new Replication data). This will result in a
full buffer and an overall increase in the replication time lag (or latency) on the Replication
Source DataCore Server.
A full Replication buffer will prevent future Replication checkpoint markers from being
created until there is enough available space in the buffer and in extreme cases may also
affect overall Host performance for any Virtual Disks served to it that are being replicated.
Using a dedicated storage controller for the physical disk(s) used to create the Windows
disk device where the buffer is be located will give the best possible throughput for the
replication process. Do not use the DataCore Server’s own boot disk so as to not cause
contentions for space and disk access.
It is technically possible to ‘loopback’ a Virtual Disk to the DataCore Server as a local SCSI
disk device to then use as the Replication buffer’s location. This is not recommended as
apart from the extra storage capacity that this will require, there may be unexpected
behavior when the SANsymphony-V software is ‘stopped’ (e.g. for maintenance) as the
Virtual Disk being used would suddenly no longer be available to the Replication process,
potentially corrupting the replication data that was being flushed while the SANsymphony-
V Software was stopping.
Creating a mirror from the Virtual Disk being ‘loop-backed’ may be considered a possible
solution to this but in the case where the mirrored Virtual Disk used for the Replication
buffer also has to handle a synchronous mirrored Virtual Disk resynchronization (e.g. after
an unexpected shutdown of the DataCore mirror partner) the additional reads and writes
used by the mirror synchronizing process as well as not using DataCore Server’s own write
caching (while the mirror is not healthy) will significantly reduce the overall speed of the
Replication buffer so this configuration is not recommended either.
The amount of write bytes that are sent to all Virtual Disks configured for replication
The speed of the Windows disk device that the buffer is using
The speed of the Replication Network Link (see the next section) to the Replication
Group
Situations when the Replication Link is ‘down’, and where the replication process will
continue to create and store replication data in the buffer until the link is re-established,
needs to be considered too. For example, plan for an ‘acceptable’ amount of network down-
time the Replication Group (e.g. 24 hours) and knowing (even approximately) how much
replication data could be generated in that time would allow for an appropriate sizing to
prevent the Replication ‘In log’ state.
Planning for future growth of the amount of replication data must also be considered.
Create GPT type Windows disk devices and using Dynamic Disks will give the most
flexibility in that it should be trivial to expand an existing NTFS partition used for the
location of an existing Replication buffer if required.
Be aware that determining the optimum size of the buffer for a particular configuration is
not always trivial and may take a few attempts before it is known.
All Replication configuration changes & updates made via the SANsymphony-V
Management Console. This includes Virtual Disk states; all Replication performance
metrics (e.g. transfer speeds and the number of files left to transfer). This TCP/IP
traffic is always sent to and from the ‘controller nodes’ of both the Source and
Destination Replication Groups1.
The Replication data between the Source and Destination Replication Groups. This
TCP/IP traffic always sent from the DataCore Server selected when the Virtual Disk
was configured for Replication on the Source Server Group regardless which
DataCore Server is the ‘controller node’.
In both cases, the DataCore Server’s own Connection Interface setting is still used.
This means that if the ‘controller node’ is not the same DataCore Server that is configured
for a particular Virtual Disk’s Replication, then the two different TCP/IP traffic streams (i.e.
Configuration changes & updates and Replication data) will be split between two different
DataCore Servers on the Source with each DataCore Server using their own Connection
Interface setting.
As the elected ‘controller node’ can potentially be any DataCore Server in the same Server
Group it is very important to make sure that all DataCore Servers in the same Local
Replication Group can route all TCP/IP traffic to all DataCore Servers in the Remote
Replication Group and vice versa.
Also see
Windows Security Settings Disclosure
http://www.datacore.com/SSV-Webhelp/windows_security_settings_disclosure.htm
Replication Operations
http://www.datacore.com/SSV-Webhelp/Replication_Operations.htm
1 See the section ‘TCP/IP Network Topology – Understanding the concept of a Server Group’s ‘controller node’ on page 11
for more explanation.
WAN/LAN optimization
The replication process does not have any specific WAN or LAN optimization capabilities
but can be used alongside any third-party solutions to help improve the overall replication
transfer rates between the local and remote DataCore Servers.
See the section ‘Replication Transfer Priority’ from the online help for more information:
http://www.datacore.com/SSV-Webhelp/Replication_Operations.htm
A simple, comparison test should be made after a reasonable period of time by disabling
compression temporarily and observing what (if any) differences there are in transfer rates
or replication time lags.
See the section ‘Enabling/disabling data compression during data transfer’ from the
online help for more information: http://www.datacore.com/SSV-
Webhelp/Configuring_Server_Groups_for_Replication.htm
Any third-party, network-based compression tool can be used to replace or add additional
compression functionality between the links used to transfer the replication data between
the local and remote DataCore Servers, again comparative testing is advised.
A Host Operating System’s page (or swap) file can also generate ‘large’ amount of extra,
unneeded replication data - which will not be useful after it has been replicated to the
remote DataCore Server.
Use separate Virtual Disks if these operations are not required to be replicated.
Some third-party backup tools may ‘write’ to any file that they have just backed up (for
example to set the ‘archive bit’ on a file it has backed up) and this too can potentially
generate extra amounts of replication data.
Encryption
The replication process does not encrypt any of the data sent to the remote DataCore
Server, but third-party encryption tools can be used to secure the links used for the
transfer of the replication data between the DataCore Servers. DataCore do not have any
recommendations for encryption other than to mention that the extra time required for the
encryption/decryption process of the replication data might add to the overall replication
time lag. Comparative testing is advised.
Anti-Virus software
The data created and stored in the replication buffer cannot be used to ‘infect’ the DataCore
Server’s own Operating System. Using Anti-Virus software to check the replication data is
therefore unnecessary and will just increase the overall replication transfer time as the
files are scanned delaying their sending and removal from the buffer as well as adding to
the number of reads to the buffer disk.
Also see
Replication - How it works
http://datacore.custhelp.com/app/answers/detail/a_id/1477
And
http://www.datacore.com/SSV-Webhelp/Replication.htm
Previous Changes
August 2015
Initial Publication containing the sections:
High level design objectives
Hardware configuration recommendations for the DataCore Server
Software configuration recommendations for the DataCore Server
DataCore, the DataCore logo and SANsymphony are trademarks of DataCore Software
Corporation. Other DataCore product or service names or logos referenced herein are
trademarks of DataCore Software Corporation. All other products, services and company
names mentioned herein may be trademarks of their respective owners.