HP Vertica 7.1.x New Features

New Features Guide
HP Vertica Analytic Database

Software Version: 7.1.x
Document Release Date: 7/21/2016
Legal Notices
Warranty
The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be
construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.
The information contained herein is subject to change without notice.
Restricted Rights Legend

Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer
Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial
license.
Copyright Notice
Copyright 2006 - 2015 Hewlett-Packard Development Company, L.P.
Trademark Notices
Adobe is a trademark of Adobe Systems Incorporated.
Microsoft and Windows are U.S. registered trademarks of Microsoft Corporation.
UNIX is a registered trademark of The Open Group.
Page 2 of 61
Contents
Contents
New Features and Changes in HP Vertica 7.1 SP2 (7.1.2)
New HP Vertica Place Functionality
New HP Vertica Pulse Functionality
Case-Sensitive Sentiment Analysis
Concurrent User Defined Dictionaries
Dictionary And Mapping Labels
New HP Vertica Pulse Functions
OLE DB Driver Added for SSAS
Row Counts for External Tables
10
Flex Tables Load and Query
10
New Parameters for fjsonparser
10
New Parameter for fdelimitedparser and mapdelimitedextractor Function
10
Improvements to GROUP BY Aggregation
11
New Hadoop and Storage Location Features
11
Using MC to Import HP Vertica in a Hadoop Environment
12
New Installer for ODBCDriver on Mac OSX
12
MC Manager and Associate Roles
12
Licensing Changes
13
Documentation Updates
13
14
Client Authentication in Catalog
14
15
Secure Password File
15
Rsync Service Account
16
Page 3 of 61
New Features Guide

Contents
New System Table
16
New Configuration Parameter Options for SET, CLEAR, and SHOW
16
Command-line Distribution of Configuration Files
16
New scrutinize Functionality
17
New Options
17
Enhanced collection capabilities
17
Export of License Audit Results to CSV
17
Projections Quick Stat forManagement Console
18
JDBC Routable Query (Key/Value) API Improvements
18
Pulse Multilingual Support
18
New Features and Changes in HP Vertica 7.1.0
19
Live Aggregate Projections and Projections with Expressions
19
New Live Aggregate Projections Capability
19
New Top-K Projections Capability
20
New Capability to Use Expressions in Projection Definitions
21
Security Model
21
New and Modified Client Authentication Methods
22
Client Authentication Information Now Stored in Database Catalog
23
SHA-512 Now Available for Hash Authentication
24
Configuring Client Authentication with Key/Value Pairs
25
SSL Changes
25
New Parameters Hold the Contents of Your SSL Certificate and Key Files
Setting the New SSL Parameters
25
26
Client Certificate Authentication
27
DSN Changes for ODBC
27
Vertica Configuration File (vertica.conf) Changes
28
SDK Enhancements
29
HP Vertica Analytic Database (7.1.x)
Page 4 of 61
New Features Guide

Contents
Change to SDK API Binary Compatibility Policy
29
Allowed Number of Arguments for UDxs Increased
29
Ability to Specify Dependency Paths for UDx Libraries
29
30
Changes to Flex Data Functions
31
Improved Performance Using PARTITION BESTOption
32
Data Management
32
COPY Statement Changes
32
Logging Projection Data
32
New Capability to Swap Partitions Atomically
33
New Column Encoding Compression Options
34
Usability and Administration
34
Active Standby Nodes
34
Management API
35
New Configuration Parameter Storage and Functionality
36
Dynamic Resource Pool Routing
37
New Configuration Parameters
37
Access Policy
38
System Tables
39
New and Changed System Tables
39
New in V_CATALOGSchema
39
Management Console
40
New Overview Page in Management Console
41
Monitoring RunningQueries in Management Console
41
Resource Pool Monitoring and Configuration in MC
41
MC Explain Plan and Profiling Views
42
Database Designer Enhancements in Management Console
42
Page 5 of 61
New Features Guide

Contents
Spread Retransmit Rate in Management Console
43
Clock Skew in Management Console
43
Table Treemap Enhancements in Management Console
43
SQL Enhancements and Changes
43
New REBALANCE_TABLE Function
43
New SQLROLLUP Function
45
How ROLLUPWorks
45
ROLLUPGrouping Functions
45
New Window Partition Clause Options
46
SQL For New Access Policy
46
Performance
47
WITH Clause Materialization Option
47
Improved Performance for Simple SELECT Queries
47
Concurrent Small Queries
48
HP Vertica and Hadoop Integration Updates
48
Storage Location for HDFS
48
Changes to the HP Vertica Connector for HDFS
48
Changes to the HCatalog Connector
49
Text Search
49
Scrutinize
49
IPv6 Client Support
50
Python 3 Support
50
Routable Connection API Support for Numeric Data Type
50
HP Vertica Place
51
HP Vertica Pulse
51
Programmer's Guide Separated Into Individual Guides
53
53
Page 6 of 61
New Features Guide

Contents
Documentation Set Available in a Single PDF File
53
New Location for HPVertica on Amazon Web Services Guide
53
Deprecated Functionality in This Release
54
Retired Functionality
55
We appreciate your feedback!
61
Page 7 of 61
New Features Guide

New Features and Changes in HP Vertica 7.1

SP2 (7.1.2)
Read the topics in this section for information about new and changed functionality in HP Vertica
7.1 SP2 (7.1.2).
New HP Vertica Place Functionality

The following function has been added to HP Vertica Place:
l
STV_Refresh_Index Appends newly added or updated polygons to an existing spatial index.
STV_LineStringPoint Retrieves the vertices of a linestring or multilinestring.
New HP Vertica Pulse Functionality

l

By default, Pulse ignores upper and lower case when performing sentiment analysis. 'ERROR'
produces the same results as 'error'. In version 7.1 SP2 and later, you can specify a case setting for
a word by using the $Case parameter when you add that word to a dictionary. For example, to
identify Apple rather than apple, you would add the following:
=> insert into pulse.white_list_en values('$Case(Apple)');
=> commit;

In version 7.1 SP2 and later, users can apply dictionaries on a per-user basis. Any number of Pulse
users can concurrently apply different sets of dictionaries without conflicts and without disrupting
Page 8 of 61
New Features Guide

other users' sessions. Each user can have one dictionary of each type loaded at any given time. If a
user does not specify a dictionary of a given type, Pulse uses the default dictionary for that type.

You can apply a label to any user defined dictionary or mapping when you load that object. Labels
enable to you perform sentiment analysis against a predetermined set of dictionaries and mappings
without having to specify a list of dictionaries. For example, you might have a set of dictionaries
labeled "music" and a set labeled "movies." The default user dictionaries automatically have a label
of "default."
A single dictionary or mapping can have multiple labels. For example, you might label a white list of
artists as both "painters" and "renaissance." You could load the dictionary by loading either label. A
label can only apply to one dictionary of each type. For example, you cannot have two stop words
dictionaries that share the same label. If you apply a label to multiple dictionaries of the same type,
the most recently applied label prevails.
You can view the labels associated with your current dictionaries using the
GetAllLoadedDictionaries() function. You can view the label associated with your current mapping
using the GetLoadedMapping() function.

HP Vertica includes the following new functions:
l
GetAllDictionarySetLabels()
GetAllLoadedDictionaries()
UnloadLabeledDictionary()
UnloadLabeledDictionarySet()
UnloadLabeledMapping()
OLE DB Driver Added for SSAS

This release adds the OLE DB driver to the HP Vertica Connectivity Pack for Microsoft. The
Connectivity Pack offers both 32- and 64-bit versions of the driver. Use the OLE DB driver only with
the Microsoft component, SQL Server Analysis Services (SSAS).
For information on the OLE DB properties and using the driver with SSAS click here to refer to the
guide, Connectivity Pack for Microsoft.
Page 9 of 61
New Features Guide

Download the latest HP Vertica Connectivity Pack for Microsoft from my.vertica.com (logon
required).
Row Counts for External Tables

HP Vertica 7.1.2 provides the capability to calculate the exact number of rows in an external table.
The Optimizer uses this count to optimize for queries that access external tables.
In particular, if an external table is part of a join, the Optimizer can now identify the smaller table to
be used as the inner input to the join. Identifying the inner input results in better query performance.
You can use the following functions to analyze and drop the row count statistic for external tables:
l
ANALYZE_EXTERNAL_ROW_COUNT
DROP_EXTERNAL_ROW_COUNT

This section presents changes and additions to the HP Vertica flex tables.
New Parameters for fjsonparser

This release includes new parameters for the fjsonparser:
l
start_point_occurrence
key_separator
suppress_nonalphanumeric_key_chars
For a description of these parameters, see the Flex Tables Guide, FJSONPARSER. Using the new
parameters is presented in Loading JSONData.
New Parameter for fdelimitedparser and

mapdelimitedextractor Function
HP Vertica 7.1.2 has a new parameter for the flex parser fdelimitedparser, and the
mapdelimitedextractor function.
The new parameter is treat_empty_val_as_null. The default value is true.
Page 10 of 61
New Features Guide

When loading delimited data, the fdelimitedparser (or mapdelimitedextractor function) now
includes a NULL for any row with an empty value, rather than an empty string (''). To continue using
empty strings when loading delimited data keys without a value, as in previous releases, specify
treat_empty_val_as_null=false.
See FDELIMITEDPARSER and MAPDELIMITEDEXTRACTOR in the Flex Tables Guide.
Improvements to GROUP BY Aggregation

HP Vertica 7.1.2 includes several major improvements to the GROUPBYaggregation
functionality:
l
The new CUBE aggregation performs the aggregation for all permutations of CUBE
expressions.
The new GROUPING SETS aggregation allows you to specify exactly which groupings of
aggregations you need.
You can now use multiple ROLLUP, CUBE, or GROUPING SETSaggregations in a single
GROUPBYclause.
For a description of how these aggregations integrate with multi-level aggregations, see
Aggregating GROUP BY Results.
New Hadoop and Storage Location Features

This release adds the following Hadoop-related features:
l
Kerberos integration for Hadoop. See Using Kerberos-Enabled Hadoop. This has been tested
with Hortonworks (HDP 2.2), Cloudera (CDH5.2 and 4.7) and MapR (4.0.1).
Direct reading of ORC files, a native Hadoop format, for improved efficiency. See Reading ORC
Files Directly.
Sharing of HCatalog configuration information. The hcatUtil script now copies HCatalog
configuration parameters from Hadoop so they do not need to be explicitly specified to
CREATEHCATALOGSCHEMA. See Configuring HP Vertica for HCatalog.
This release also adds improvements to the management of storage locations. Changes can now
be enforced immediately instead of waiting for the next automatic update, which simplifies the
process of retiring a storage location. Storage-location operations can also now be applied to all
Page 11 of 61
New Features Guide

nodes with a single command. See Managing Storage Locations and the corresponding metafunctions in the SQL Reference Manual.
Using MC to Import HP Vertica in a Hadoop

Environment
As of HP Vertica 7.1.2, you can use Management Console to connect to and monitor an HP Vertica
database that resides in an Apache Hadoop environment.
l
Connect to an Apache Ambari server through the Connect using an Ambari server to import
Vertica within a Hadoop environment button under the Provision section of Management
Console.
Monitor and update your Hadoop environment from the Management Console Existing
Infrastructure page.
For detailed instructions, see Using MC to Import and Monitor Vertica in a Hadoop Environment.
New Installer for ODBCDriver on Mac OSX

This release adds a Mac OSX installer for the HP Vertica ODBC driver on Mac OS X. The installer
for the ODBC driver on Mac OS X is packaged as a .pkg file. You can run the installer as a regular
Mac OS X installer or silently. This driver is compatible with both 32-bit and 64-bit applications.
For more information on installing the ODBC driver on Mac OS X, see Installing the ODBC Driver
on Mac OS X.
Download the latest HP Vertica ODBC driver on Mac OS X from my.vertica.com (logon required).
MC Manager and Associate Roles

This release adds new roles to Management Console for HP Vertica on Demand users.
The Manager role allows users to:
l
Monitor databases on which the user has privileges

View the database overview and activity pages
Page 12 of 61
New Features Guide

Monitor the node state
View messages and mark them read or unread
View database settings.
Anything database-related beyond that scope depends on the mapped database user's privileges
granted on the database through GRANT statements.
The Associate role allows full privileges to monitor activity and messages on databases managed
by MC. Associate users may have other database privileges, depending on the database user
account to which they are mapped. These privileges can include modifying settings, installing
licenses, and viewing the database designer.
See About MC Users and About MC Privileges and Roles.
Licensing Changes
HP Vertica no longer counts each delimiter character as a 1-byte value against your license data
limit. Runnning a license audit after upgrading results in less total data than the previous version.
Documentation on partially sorted GROUPBY was erroneously included in the documentation and
has been removed. This functionality is currently not included in the product.
Page 13 of 61
New Features Guide

New Features and Changes in HP Vertica 7.1

SP1 (7.1.1)
Read the topics in this section for information about new and changed functionality in HP Vertica
7.1 SP1 (7.1.1).
Client Authentication in Catalog

Client authentication information is now stored in the database catalog. Authentication information
is no longer stored in the individual vertica.conf files on each node. Any authentication
information that still exists in vertica.conf is obsolete. You can no longer use Administration
Tools to define client authentication.
When you upgrade to HP Vertica 7.1.1, the client authentication records in vertica.conf are
automatically converted and stored in the catalog on every node. There are a few steps that you
must take to fully configure authentication.
For details, see Upgrading Client Authentication in HP Vertica 7.1.1.
Storing client authentication information in the catalog has several benefits:
l
All client authentication records are stored in the same location using a consistent format.
Any changes to client authentication when a node goes down are automatically recovered when
the node restarts.
You use SQLcommands to:

n
Create and manage authentication information.
Grant and revoke authentication methods for individual users or for user roles.
Assign a default authentication method. A user who has not been granted an authentication
method is authenticated with the default method.
Temporarily grant or revoke authentication methods for specific users and user roles or for all
users and user roles.
For detailed information, see How Client Authentication Works.
Page 14 of 61
New Features Guide


This release includes several changes and additions to Flex Tables. For more details about each of
the changes and new features, see the Flex Tables Guide.
New fdelimitedpairparser Parser

This release includes fdelimitedpairparser, a simplified version of the fcefparser. Use the
fdelimitedpairparser to parse key-value pairs.
Updated fcefparser Parser

The fcefparser now fully supports log files that are compliant with the HPArcSight Common
Event Format (CEF) log files. This release includes the following changes and additions:
l
Changes default separator from a comma (,) to an empty string ('')
Support for CEFprefix sections
Follows HPArcSight CEFrules for escape characters, which are:

n \n
n
\\
New line \.)
New MAPLOOKUP() buffer_size Parameter

The maplookup()function supports a new, optional buffer-size parameter. Use this parameter to
specify the maximum byte size of column values processed by maplookup().
Secure Password File

HP Vertica 7.1 SP1 now stores saved passwords in a secure file. The vbr.py backup configuration
utility creates a user-named file that contains the database password as well as the password to
the rsync service account. Only the dbadmin, or members of that user's Linux permission group,
can view the contents of the file. If you alter the file permissions from their default values (xx0), HP
Page 15 of 61
New Features Guide

Vertica backup or restore actions fail with an error message. If you do not save any passwords,
running vbr.py still creates the password file, but leaves the file empty.
Rsync Service Account

HP Vertica 7.1 SP1 now has the ability to run rsync under an authenticated service account. If you
choose to save the password for the rsync account, HP Vertica stores it in the secure password
configuration file.
New System Table

The HP Vertica 7.1.1 release includes one new system table in the V_MONITORschema,
PARTITION_COLUMNS. This table shows, for each projection of each partitioned table, the disk space
used by each column on each node.
For detailed information, see the SQL Reference Manual.
New Configuration Parameter Options for SET,

CLEAR, and SHOW
HP Vertica 7.1 SP1 introduces a new way of setting, clearing, and viewing configuration
parameters.
You can now set and clear some parameter values at the database, node, or session level using
ALTER statements.
Additionally, the new SHOW SESSION, SHOWCURRENT, SHOW NODE, and SHOW
DATABASE statements allow you to view configuration parameter settings at different levels.
This version also introduces the new SESSION_PARAMETERS system table, which provides
information about configuration parameters at the session level.
For more information, see Setting and Clearing Configuration Parameters.
Command-line Distribution of Configuration Files

You can now distribute HP Vertica configuration files from the command line, as follows:
$ admintools -t distribute_config_files
Page 16 of 61
New Features Guide

This option provides the same functionality currently available through the Administration Tools
interface.
New scrutinize Functionality

For detailed information, see the Administration Guide, Collecting Diagnostics (scrutinize
Command).
New Options
The following options are new to the scrutinize diagnostics tool:
--diag-dump
Collects HP Vertica queries, system tables, and Data Collector tables. This
option replicates the behavior of the superseded collect_diag_dump tool.
--diagnostics
Collects log file data and runs commands against HP Vertica and its host
system. This option replicates the behavior of the superseded diagnostics
utility.
--log-limit limit
Limits how much data is collected from HP Vertica logs, where limit
specifies in gigabytes how much log data to collect, starting from the most
recent log entry.
--tmpdir=path
Specifies a temporary directory to use on all nodes. Use this option to ensure
that the temporary directory has enough free space so scrutinize can
complete execution.
Enhanced collection capabilities

scrutinize now performs the following tasks:
l
Collects cluster hardware configuration from systems where the lshw utilty is installed.
Collects Kerberos configuration data if it is installed on the cluster.
Export of License Audit Results to CSV

You can use the admintools license_audit tool to audit a database and export the results to a
CSV file.
Page 17 of 61
New Features Guide

Projections Quick Stat forManagement Console

In HP Vertica 7.1, Management Console introduces a new Quick Stat for the Overview Page. This
projections Quick Stat displays the total projection count and number of unsegmented and unsafe
projections for the database's schema with the largest number of projections.
For information about the Overview Page inManagement Console, see Viewing the Overview
Page.
JDBC Routable Query (Key/Value) API

Improvements
In HP Vertica 7.1 SP1 the JDBC Key/Value API has been renamed to the JDBC Routable Query
API. The API introduces a new class called VerticaRoutableExecutor, which allows you to query
single nodes using SQL syntax instead and allows for JOIN and GROUP BY operations. For details
see Routing JDBC Queries Directly to a Single Node.
Pulse Multilingual Support

HP Vertica Pulse 7.1 SP1 now supports analyzing Spanish text. For details see Multilingual Pulse.
Page 18 of 61
New Features Guide

New Features and Changes in HP Vertica

7.1.0
This guide describes new features and changes that were added to the Vertica Analytic Database
7.1.0 release.
For a list of known and resolved issues, see the HP Vertica 7.1.x Release Notes, available at
http://www.vertica.com/documentation
Live Aggregate Projections and Projections with

Expressions
HP Vertica 7.1 provides three new kinds of projections that aggregate data from large and
frequently updated tables. You query aggregated data from a live aggregate projection instead of
querying an anchor table and then aggregating the results. This approach eliminates resourceintensive computations because HP Vertica performs the calculations when it loads the data, not
each time you query the projection.
New Live Aggregate Projections Capability

HP Vertica 7.1 introduces live aggregate projections. A live aggregate projection contains column
values that have been aggregated from columns in its anchor table.
Querying data from a live aggregate projection is usually faster than querying data from an anchor
table and then aggregating it. Because the data is already aggregated, when you query the live
aggregate projection, reaggregation is not necessary. You receive the same results that you get
when querying from the anchor table.
When you create a live aggregate projection for a table and load data into the table, HP Vertica
aggregates the data from the anchor table and loads it into the live aggregate projection. On
subsequent loads, HP Vertica updates both the anchor table and the live aggregate projection.
When working with live aggregate projections, keep in mind:
l
When you create a live aggregate projection, you can no longer delete, update, or merge data in
the anchor table or the anchor projection. You can only add data to them.
Page 19 of 61
New Features Guide

A Top-K projection is a specific type of live aggregate projection.
The capability to create live aggregate projections is enabled by default.
For details, see:

l
Retrieving Data from Frequently Updated Tables
Live Aggregate Projections
Top-K Projections
New Top-K Projections Capability

HP Vertica 7.1introduces Top-Kprojections. Top-Kprojections are a type of live aggregate
projection, also new in HP Vertica 7.1. Top-Kprojections improve the performance of queries that
retrieve the top k rows per partition of selected rows. These queries are called Top-K queries.
For example, the following Top-K query retrieves the four most recent readings from a gas meter,
partitioned by meter_id and metric:
=> SELECT meter_id, metric, reading_time, reading_value
FROM readings
LIMIT 4 OVER (PARTITION BY meter_id, metric ORDER BY reading_time DESC);
For optimal performance of Top-K queries, create a Top-Kprojection that aggregates the data in the
table for fast access. Querying the preaggregated data directly from the Top-K projection is usually
faster than querying the data from the anchor table and then calculating the top k rows.
When you create a Top-K projection, keep in mind:
l
After you create a Top-K projection for a table, you can no longer delete, update, or merge data in
the anchor table or anchor projection. You can only add data to them.
The capability to create Top-K projections is enabled by default.
For details, see:

l
Retrieving Data from Frequently Updated Tables
Top-K Projections
Page 20 of 61
New Features Guide

New Capability to Use Expressions in Projection

Definitions
In HP Vertica 7.1, you can create projections that use expressions in the column definitions. The
expression calculates values based on data in the anchor table and stores it in the projection. When
you load data into an anchor table that has a projection that uses expressions, HP Vertica
calculates the values using the data from that table. HP Vertica then inserts the calculated data into
the new projection.
Projections with expressions behave the same as normal projections, except you cannot perform
any merge operations on the anchor table.
A projection that uses expressions is not a live aggregate projection unless it aggregates the data in
one or more columns, If it does not aggregate data, you can continue to delete and update data in
the anchor table but you cannot perform any merge operations in the anchor table. If the projection
with expressions does aggregate data, it behaves like Live Aggregate Projections.
Using expressions when defining projections allows you to sort the data based on the calculated
results of an expression.
For details, see:
l
Retrieving Aggregated Data from Tables
Projections with Expressions
Security Model
HP Vertica 7.1 introduces these security features:
l
Client Authentication Information Now Stored in Database Catalog
SSL Changes
Page 21 of 61
New Features Guide


In HP Vertica 7.1, there are several changes to the client authentication methods that the database
supports:
l
GSS-API(Generic Security Services API)allows you to authenticate clients to Kerberos v5.
Note: For details about GSS-API implementation, see RFC2744 and RFC4121.
Hash authentication is new; it replaces MD5 authentication. It allows you to store passwords
using the MD5 algorithm and the more secure SHA-512 algorithm. You configure the hash
authentication and then define user-level and system-level parameters to specify which
algorithm to use.
TLS(Transport Layer Security) authentication now uses OpenSSL 0.9.8za.
You can use TLSfor any authentication method:

n
The HOST NO TLS syntax replaces hostnossl
The HOST TLSsyntax replaces hostssl.
You can now self-authenticate a client. To do this:

n
Use an SSL client certificate whose Common Name (CN) field contains the user name for the
target database.
Create an authentication method that uses an auth_type of tls with the authentication
method HOST TLS.
For information, see Configuring TLSAuthentication.
There are several new parameters for configuring LDAP. For details, see LDAPParameters.
The following client authentication methods have no changes:

l
Password. Password authentication sends passwords over the network in clear text. HP
Vertica recommends that you use hash authentication with the SHA512 algorithm.
Ident. The Ident protocol authenticates a database user with their system user name.
Page 22 of 61
New Features Guide

Trust. When you grant trust authentication to a user, they can always log in without a password.
Reject. When you grant reject authentication to a user, they can never log in.
For more information, see Client Authentication.
Client Authentication Information Now Stored in

Database Catalog
In HP Vertica 7.1, client authentication information is now stored in the database catalog.
Authentication information is no longer stored in the individual vertica.conf files on each node.
Any authentication information that still exists in vertica.conf is obsolete. You can no longer use
Administration Tools to define client authentication.
When you upgrade to HP Vertica7.1, the vertica.conf file still contains all the client
authentication information, but your database does not use it. No client authentication is configured
for your database, and anyone can connect to the database.
You need to reconfigure all client authentication as described in Upgrading Client Authentication
Records from vertica.conf to the Catalog. Once you have reconfigured this information, you should
delete the client authentication records from the vertica.conf file.
This feature has several benefits:
l
All client authentication records are stored in the same location using a consistent format.
Any changes to client authentication when a node goes down are automatically recovered when
the node restarts.
You use SQLcommands to:

n
Create and manage authentication information.
Grant and revoke authentication methods for individual users or for user roles.
Assign a default authentication method. A user who has not been granted an authentication
method is authenticated with the default method.
Temporarily grant or revoke authentication methods for specific users and user roles or for all
users and user roles.
Page 23 of 61
New Features Guide


HP Vertica 7.1 provides the capability to use the Secure Hash Algorithmspecifically, SHA-512
for password encryption.
When you upgrade to HP Vertica 7.1, the MD5 encryption is the default hash algorithm. The SHA512 algorithm provides more secure hashing than MD5.
You configure the hash algorithm in one of two ways:
l
At the system level, set the SecurityAlgorithm configuration parameter. Valid values are:
n
'NONE'
'MD5'
'SHA512'
This setting applies to all users, unless the DBADMINhas set the user-level parameter to 'MD5'
or 'SHA512'. In that case, the user-level value overrides the system-level value.
l
At the user level, use ALTER USERto set the Security_Algorithm user parameter. Valid values
are the same as for the system-level parameter:
n
'NONE'
'MD5'
'SHA512'
If the value of the user-level parameter Security_Algorithm is 'NONE', HP Vertica uses the
algorithm specified in the system-level parameter, SecurityAlgorithm. If both parameters are
'NONE', HP Vertica uses the MD5 algorithm.
Important: Before you upgrade, read about the issues you might encounter in Upgrade
Considerations for Hash Authentication.
See Also
l
Hash Authentication Parameters
How to Configure Hash Authentication.
Page 24 of 61
New Features Guide


In HP Vertica 7.1, you specify client authentication parameters as key/value pairs, as follows:
parameter_name = parameter_value
You use the new ALTERAUTHENTICATIONstatement to set and change LDAPand Ident
parameters.The following examples set parameters using key/value pairs:
=> ALTER AUTHENTICATION ident1 SET system_users='root';
=> ALTER AUTHENTICATION Ldap1 SET host='ldap://172.16.65.177',
binddn_prefix='cn=', binddn_suffix=',dc=qa_domain,dc=com';
For more information, see

l
ALTERAUTHENTICATION
Configuring LDAPAuthentication
Configuring Ident Authentication
SSL Changes
HP Vertica 7.1.0 introduces these changes to SSL:
l
New Parameters for Holding SSL Certificates and Key Files
DSNChanges for ODBC
New Parameters Hold the Contents of Your SSL

Certificate and Key Files
HP Vertica 7.1 adds three security configuration parameters to hold the contents of the certificate
and key files:
l
SSLPrivateKeySet this parameter to the contents of your servers private key (server.key)
file. Note that only the value of this parameter is visible to dbadmin users.
SSLCertificateSet this parameter to contents of your servers SSL certificate (server.crt)
Page 25 of 61
New Features Guide

file.
l
SSLCASet this parameter to the contents of SSL certificate authority (root.crt) file if using
mutual authentication.
These new parameters simplify distribution of your server certificate file (server.crt) and private
key (server.key) to server hosts in your cluster. BeforeHP Vertica7.1, the server certificate and
private key had to be copied to the HP Vertica catalog directory of each server host in your cluster.
Note: The SSLPrivateKey and SSLCertificate parameters are automatically set during the 7.1
upgrade, if EnableSSL=1 before the upgrade.
For a listing and description of these new configuration parameters and all other security
parameters, refer to the Administrator's Guide, Security Parameters.
Setting the New SSL Parameters

After you create your certificate and key files, set these parameters from the command line using
the statement:
ALTER DATABASE
If you use ALTERDATABASE to set the new parameters, you must include the actual contents of
the file, not the filename.
ALTER DATABASE exampleDB SET SSLPrivateKey ='<contents of server.key file>';
For more information on using this approach, see the SQL Reference Manual topic, ALTER
DATABASE.
Alternatively, use Admintools to point to and distribute the files as in previous releases; Admintools
sets the value of the parameters for you. For more information on using Admintools to distribute
SSL certificates and certifications, refer to the Administrator's Guide, Redistributing Configuration
Files to Nodes.
If you are upgrading from a previous version of HP Vertica and have already distributed your
certificate and key files, you must still use one of these approaches to prepare your parameters
for upgrade.
Common Questions
The following table provides answers to common questions on setting the new parameters.
Page 26 of 61
New Features Guide

Question
Response
Does the database have to be
No. The database does not have to be started. In
started to use Admintools to
Admintools, point to the locations of the certificate and
distribute certificate and key files?
key files, and Admintools sets the new parameters for

you.
If you are upgrading to HP
Yes. HP Vertica looks for the content of the certificate and
Vertica7.1 and have already
key files within the parameters. You must use Admintools
distributed your certificate and key
once again to point to the location of the certificate and
files, would you still need to use
key files so that the parameters are set for the upgrade.
Admintools to point to the location
Alternatively you can use ALTERDATABASE to set the
of the certificate and key files?
parameters.
If you use ALTERDATABASE to
Include the actual contents of the file when you set a
set the new parameters, do you use
parameter with the statement ALTERDATABASE.
the filename or the actual contents

of the file?
ALTER DATABASE exampledb SET SSLPrivateKey

= '<contents of server.key file>';

Database users can now self-authenticate by using an SSL client certificate whose Common
Name (CN) field contains their user name for the target database. To enable user selfauthentication, you set the CN to the database user name for the target database when you create
the client key. For information on creating certificates and keys, refer to the Administrator's Guide,
Generating SSL Certificates and Keys.
After you set the CN to the database user name, you can create an authentication method using the
CREATE AUTHENTICATION statement together with the tls authentication method. You then
associate the method with the user using the GRANT AUTHENTICATION statement.
DSN Changes for ODBC

When you use an ODBC driver, HP Vertica 7.1 adds new DSN parameters for certification and key
file locations. You can use these parameters, SSLCertFile and SSLKeyFile, to point to the
locations of the client's public certificate file and the client's private key file, respectively. For a
listing of these and all the DSN parameters, refer to the Connecting to HP Vertica Guide section,
DSN Parameters.
For information on setting up a DSN, refer to the section, Setting Up a DSN.
Page 27 of 61
New Features Guide

For Windows ODBC, HP Vertica 7.1 also adds connection string properties within the ODBC Data
Source Administrator GUI, including SSL cert file and SSL key file fields under the Client Setting
tab.
Vertica Configuration File (vertica.conf) Changes

HP Vertica 7.1 introduces some changes to the vertica.conf files.
Previously, the vertica.conf files stored configuration parameter information. As of 7.1,
configuration parameter values are stored in the database catalog. By storing the values in the
database catalog, the state of configuration parameters remains consistent over all nodes, even if a
node was down when a parameter value changed.
This feature also eliminates the need for hand-editing node vertica.conf files because node-level
changes can be made in the catalog using SET_CONFIG_PARAMETER. In the past, you needed to log
on to each node and hand-edit its vertica.conf file to specify node-level parameter values, which
is risky and tedious.
Important: Hewlett-Packard now encourages the use of the SET and CLEAR parameters in
the ALTER NODE, ALTER DATABASE, and ALTER SESSION statements to set and clear
configuration parameters. See Setting and Clearing Configuration Parameters for more
information.
Specifically, client authentication information is now stored in the catalog, not in the vertica.conf
files. When you upgrade to HP Vertica7.1, the vertica.conf file still contains the client
authentication information defined by the ClientAuthentication configuration parameter, but your
database does not use it. No client authentication is configured for your database, and anyone can
connect to the database.
For instructions on configuring client authentication after you upgrade to HP Vertica7.1, see
Upgrading Client Authentication Records from vertica.conf to the Catalog.
Page 28 of 61
New Features Guide

For more information about configuration parameters, see Configuration Parameters in the
Administrator's Guide.
SDK Enhancements
HPVertica 7.1.0 introduces these SDK Enhancement features:
l
Change to SDK APIBinary Compatibility Policy
Increase in Allowed Number of Arguments for UDxs
Ability to Specify Dependency Path for UDx Libraries
Change to SDK API Binary Compatibility Policy

HPmay now make changes to the SDKAPIin the first service pack for each major release of the
HP Vertica Analytics Platform. This change allows HPto fix any issues found after the release of a
new version of the HP Vertica server. Without this policy change, HPwould not be able to resolve
issues that require a modification of the SDKAPI until the next major software release.
SDKAPIchanges require that UDx libraries be recompiled in order to work with the new version of
the API. See UDx Library Compatibility with New Server Versions in the Extending HPVertica
Guide for more information.
Allowed Number of Arguments for UDxs Increased

The maximum number of arguments that a UDx can accept has increased from 32 to 500.
Ability to Specify Dependency Paths for UDx Libraries

In previous versions of HP Vertica, you had two options to handle libraries on which your UDx
depended:
l
Bundle the library by statically linking it into your UDx library.
Manually copy the library to every node in your cluster to a location where the UDx could find it.
For example, copy a Java library to each node in your cluster. Then you update each node's
CLASSPATH to include the library's directory.
In HP Vertica Version 7.1.x, you can specify one or more directories containing support library files
using the the new DEPENDSkeyword in the CREATELIBRARYstatement. HP Vertica handles
these support libraries the same way it treats the UDx library itself. It copies the libraries to each
Page 29 of 61
New Features Guide

node in the cluster in a location where the UDx library can access them. For Java UDx libraries,
you can use this new feature as a way to set a Java classpath for each UDx library.
For more information, see Handling Java UDx Dependencies, Compiling Your C++ UDx, and
CREATE LIBRARY.

This release includes several changes and additions to Flex Tables. For more details about each of
the changes and new features, see the Flex Tables Guide.
Polystructured Data Support

This release adds polystructured data support for nested arrays and arbitrarily-deep map data. To
support polystructured data, several additions and changes to flex table functionality have been
made in the following areas:
l
Usability improvements in flex parsers and functions (new parameters)
New functions to extract semi-structured data from the database directly into a Vmap, without
first having to write such data to disk and load it with COPY and the flex parsers
Polystructured data query support by pass-through arguments in the MAPITEMS function
New Polystructured Data Extractor Functions

Three new scalar functions support polystructured data, returning a single row of map data from
existing database content. Each extractor function name includes the type of data it parses:
l
MAPDELIMITEDEXTRACTOR
MAPJSONEXTRACTOR
MAPREGEXEXTRACTOR
Each function processes a specific type of data, using identical arguments to the corresponding
flex table parser (FDELIMITEDPARSER, FJSONPARSER, and FREGEXPARSER). Use the
extractor functions to extract semi-structured data from database contents (such as an existing
table column or data returned from an expression), or entered directly as a string. The function
returns a single VMAProw that you can use in a flex or regular table column.
For more information, see the the Flex ExtractorFunctions Reference in the Flex Tables Guide.
Page 30 of 61
New Features Guide

New Regular Expressions Parser

This release includes a new Flex table parser for regular expressions.
For more information, see the FREGEXPARSER reference page.
New Options for FJSONPARSER

The FJSONPARSER has these new options:
l
start_point
reject_on_empty_key
omit_empty_keys
reject_on_materialized_type_error
Each of these options is described in the FJSONPARSER reference page.
New Options for FDELIMITEDPARSER

The FDELIMITEDPARSER has these new options:
l
reject_on_empty_key
omit_empty_key
reject_on_materialized_type_error
Each of these options is described in the FJSONPARSER reference page.
Changes to Flex Data Functions

The following Flex data functions now ignore empty keys:
l
BUILD_FLEXTABLE_VIEW
COMPUTE_FLEXTABLE_KEYS_AND_BUILD_VIEW
MATERIALIZE_FLEXTABLE_COLUMNS
Page 31 of 61
New Features Guide

Updates to MAPITEMS() Function

The MAPITEMS()flex map function now supports pass-through arguments. For more information,
see the MAPITEMS reference page.
Improved Performance Using PARTITION BEST Option

For any flex map functions that require the use of an OVER()clause, HP Vertica recommends the
PARTITION BEST option. This option results in multi-threaded operations across all available nodes,
which can significantly improve performance when processing large amounts of raw data in flex
tables.
All of the flex table documentation examples now include the PARTITION BESTusage in the OVER()
clauses.
Data Management
HPVertica 7.1.0 introduces these Data Management features:
l
COPYStatement Changes
Log Projection Data in Database Designer
Swap Partitions Automatically
Column Encoding Compression Options
COPY Statement Changes

In this HP Vertica release, using COPY to bulk load data has these changes:
l
Apportioned load support. The COPYstatement already supported multi-threaded bulk loading
on a single node (parallel load). Now, load also occurs across multiple nodes to increase
performance.
LZO format support. COPYnow accepts LZO files, as it does other formats, such as BZIP and
GZIP files. For more information, see COPY in the SQL Reference Manual.
Logging Projection Data

HP Vertica 7.1 allows you to log information about the projections that theOptimizer recommends.
Database Designer considers these projections when creating a design. HP Vertica stores this
Page 32 of 61
New Features Guide

information in two Data Collector (DC)tables:

l
DC_DESIGN_PROJECTION_CANDIDATES
DC_DESIGN_QUERY_PROJECTION_CANDIDATES
When you enable logging and start Database Designer, the Optimizer proposes a set of ideal
projections based on the options that you specify. The logs contain information about:
l
Whether or not the projections are actually created when the design is deployed.
How projections are optimized.
Whether the projections are created with and without the ideal criteria that the Optimizer
identified.
If you do not deploy the design immediately, review the logs to determine if you want to change the
design and proposed projections. If Database Designer deployed the design, you can still manually
create some of the projections that Database Designer did not create in the deployed design.
By default, logging the Database Designer design data is disabled. To enable it, turn on the
configuration parameter DBDLogInternalDesignProcess:
=> ALTER DATABASE exampledb SET DBDLogInternalDesignProcess = '1';
For more information, see Logging Projection Data for .
New Capability to Swap Partitions Atomically

HP Vertica 7.1 allows you to swap the partitions of two tables that have identical definitions using
the SWAP_PARTITIONS_BETWEEN_TABLESfunction. Use this feature if you are loading new
or updated data into a table on a regular basis.
You can swap partitions only between tables that have the same:
l
Column definitions
Segmentation
Partitioning expression
Number of projections
Projection sort order
Page 33 of 61
New Features Guide

Swapping partitions is an atomic operation. The swapping operation moves the data to a new
partition and then swaps the newly partition with the partition in the target table while maintaining
the integrity of the data. If any task in the swap operation fails, the entire operation fails.
For more information, see:
l
Swapping Partitions
Tutorial for Swapping Partitions
SWAP_PARTITIONS_BETWEEN_TABLES
New Column Encoding Compression Options

This release of HP Vertica adds two new table column encoding options:BZIP_COMP and GZIP_
COMP. These options let you use bzip2 and gzip compression on table columns. They result in
high compression, but also require more CPUtime to process, especially when loading data. See
Encoding-Type in the SQL Reference Manual for more information.
Usability and Administration

HPVertica 7.1.0 introduces these Usability and Administration features:
l
Management API
New Configuration Parameter Storage and Functionality

An active standby node is a specialized type of HP Vertica node. An active standby node exists as
a backup node, ready to replace a failed node. Unlike standard HP Vertica nodes, an active standby
node does not perform computations or contain data. If a standard(permanent) node fails, an active
standby node can replace the failed node, after the failed node exceeds the failover time limit. When
it take the place of a failed node, the active standby node contains all of the projections and
performs all of the calculations of the replaced node.
To deploy active standby nodes automatically, you must first configure the
FailoverToStandbyAfter parameter. If possible, HP Vertica selects a standby node from the
Page 34 of 61
New Features Guide

same fault group as the failed node. Otherwise, HP Vertica randomly selects an available active
standby node.
If you are an administrator, you can manually replace a failed node using the ALTER NODE
command. You can specify a particular standby node to replace a failed node, or you can allow HP
Vertica to choose a node. As with automatic node replacement, HP Vertica defaults to a standby
node from the same fault group as the failed node. If the fault group has no available standby nodes,
HP Vertica selects any available active standby node.
Management API
HP Vertica 7.1 introduces the Management API. The Management API is a REST API that can be
used to manage HP Vertica databases with scripts or applications that understand REST and
JSON.
The Management API provides a subset of the database administration tools found in the
Management Console. Features include the ability to:
l
View host and network details for each node
View details on database version, structure, and state
View and update licenses
Start and stop databases on the entire cluster or individual nodes
View details about available database backups, start a new database backup, and restore
backups from specific archives
View and set database configuration parameters
Add hosts to the database, remove hosts from the database, and replace hosts with standby
hosts
Create and delete databases
View a list of running jobs
Run rebalance or workload analyzer on a database
Create and manage event listeners (also known as webhooks)
For complete details see About the Management API.
Page 35 of 61
New Features Guide

New Configuration Parameter Storage and

Functionality
HP Vertica 7.1 introduces a new way of storing and setting configuration parameters. Configuration
parameter values are now stored in the database catalog, rather than in individual vertica.conf
files on each node, and can be set at the database or node level using SET_CONFIG_PARAMETER.
Because parameter values were stored in individual vertica.conf files, and the SET_CONFIG_
PARAMETER statement only acted on up nodes, it was possible that a down node would have
inconsistent parameter values when it returned to the cluster. Additionally, there was no way to tell
which vertica.conf file had the most up-to-date information.
By storing the values in the database catalog, the state of configuration parameters remains
consistent over all nodes. When a down node returns to the cluster, it retrieves the latest catalog
file and thus adopts the most up-to-date parameter values. The catalog storage feature also
assures that all nodes agree on what all other nodes' values are.
This new feature allows node-level parameter configuration though a new SET_CONFIG_PARAMETER
argument called node_name:
SET_CONFIG_PARAMETER (parameter_name, value, node_name)
This eliminates the need for hand-editing node vertica.conf files. In the past, you needed to log
on to each node and hand-edit its vertica.conf file to specify node-level parameter values. This is
discouraged because it is risky and tedious. The new feature allows you to set any one node's
parameter values from any other node using SET_CONFIG_PARAMETER.
Important: Hewlett-Packard now encourages the use of the SET and CLEAR parameters in
the ALTER NODE, ALTER DATABASE, and ALTER SESSION statements to set and clear
configuration parameters. See Setting and Clearing Configuration Parameters for more
information.
This version also introduces new columns to the CONFIGURATION_PARAMETERS system table:
l
catalog_valuevalue currently stored in the catalog.
database_valuevalue set at the database level. Shows default value if no value is set at the
database level.
sourcelevel at which the catalog_value is set.
Page 36 of 61
New Features Guide

is_mismatchtrue if current_value and catalog_value are different.
groupsdisplays the group(s) to which the parameter belongs (e.g. security parameter).
For more information, see Setting and Clearing Configuration Parameters.

This release introduces two new features related to creating or altering a resource pool. The new
features allow you to dynamically reroute queries to a secondary resource pool, and designate a
pool to hold queued queries until a later time.
CASCADE TO Parameter
With the new CREATE RESOURCE POOl and ALTER RESOURCE POOL CASCADE TO parameter, you can
indicate a secondary resource pool to which queries can cascade. When a query reaches the initial
pool's RUNTIMECAP, the query cascades to your defined secondary pool where it executes.
PRIORITY HOLDOption
The new HOLD value option for the PRIORITY parameter sets a pool's priority to -999 so queries in
that pool queue until QUEUETIMEOUTis reached. You can set PRIORITY HOLD on a secondary pool if
you want a query that exceeds the initial pool's RUNTIMECAP to abort and stay on hold until a later
time.
For more information, see the RESOURCE_POOLS system table in the SQL Reference Manual
and Defining Secondary Resource Pools in the Administration Guide.

This section lists new configuration parameters in HP Vertica 7.1. See Configuration Parameters in
the Administrator's Guide for details.
HP Vertica 7.1 adds the following configuration parameters:
l
DBDLogInternalDesignProcessWhen you enable DBDLogInternalDesignProcess,

Database Designer logs data about the projections that theOptimizer proposes. These
projections are created when Database Designer deploys the design.
Page 37 of 61
New Features Guide

EnableGroupByProjectionsWhen you set EnableGroupByProjections to '1', you can

create live aggregate projections. EnableGroupByProjections is enabled by default. For more
information, see Live Aggregate Projections.
EnableTopKProjections HP Vertica 7.1 adds the EnableTopKProjections configuration

parameter. When you set EnableTopKProjections to '1', you can create Top-K projections that
allow you to retrieve Top-K data quickly. EnableTopKProjections is enabled by default. For more
information, see Top-K Projections.
EnableExprsInProjectionsWhen you set EnableExprsInProjections to '1', you can create

projections that use expressions to calculate column values. EnableExprsInProjections is
enabled by default. For more information, see Projections with Expressions.
SSLCLAYou set the SSL certificate authority by providing the contents of the certificate
authority root.crt file. For more information, see Security Parameters.
SSLCertificateYou set the SSL certificate by providing the contents of the server.crt file.
For more information, see Security Parameters.
SSLPrivateKeyYou set the server's private key by providing the contents of the server.key
file. Note that only the value of this parameter is visible to dbadmin users. For more information,
see Security Parameters.
SecurityAlgorithmUse this parameter to set the algorithm for the function that hash
authentication uses:MD5 or SHA512 .
Access Policy
With HPVertica you can now control access to data in a table at the column level. The new access
policy feature provides the capability to control user access at the table column level adding a
further level of security. Implementing this capability across your HP Vertica database can prevent
unauthorized users from retrieving sensitive information.
Implementing secure access to your HP Vertica table columns is achieved by creating user, roles,
and privileges. Once such required privileges exist, you choose which entities to associate with
table columns. Only user and roles granted relevant access can view and operate on data that the
column contains.
For more information see the following in the Administrator's Guide:
Page 38 of 61
New Features Guide

Creating Secure Access Policies
Using Access Policies
Limitations on Creating Access Policies with Projections
Schema and Table Privileges
Database-Level Column Security
Table-Level Column Security
See Also
SQLFor New Access Policy
System Tables
This section lists the new and modified system tables in this release. See the SQL Reference
Manual for details.
New and Changed System Tables

This section describes any new or changed system tables in both V_MONITORand V_CATALOG
schemas for HP Vertica 7.1.
Modified in V_MONITORSchema
TheLOGIN_FAILUREStable has one new column:
l
CLIENT_AUTHENTICATION_NAME
The SESSIONStable has the following new columns:

l
CLIENT_AUTHENTICATION_NAME
CLIENT_AUTHENTICATION
JVM_MEMORY_KB
New in V_CATALOGSchema
The following system tables were added in this release:
Page 39 of 61
New Features Guide

CLIENT_AUTH
CLIENT_AUTH_PARAMS
PASSWORD_AUDITOR
USER_CLIENT_AUTH
Modified in V_CATALOGSchema
The LICENSE_AUDITS system table has one new column:
l
LICENSE_NAME.
ThePROJECTION_COLUMNS system table has the following new columns:

l
IS_EXPRESSION
IS_AGGREGATE
COLUMN_EXPRESSION
The PROJECTIONS system table has the following new columns:

l
SEGMENT_EXPRESSION
HAS_EXPRESSIONS
IS_AGGREGATE
AGGREGATE_TYPE
TheTABLES system table has the following new columns:

l
HAS_AGGREGATE_PROJECTION
Management Console
This section lists the new functionality HP Vertica Analytics Platform introduced to Management
Console.
Page 40 of 61
New Features Guide

New Overview Page in Management Console

HP Vertica 7.1 introduces a redesigned Overview page in Management Console. This page offers a
more comprehensive view of your database. Quickly navigate the Overview page with three new
tabs: Status Summary, Systems Health, and Query Synopsis, which each contain charts that
summarize the state of your system. The QuickStats widgets on the right of the page give you at-aglance information, from node health and current queries to disk health and license status.
Monitoring Running Queries in Management Console

HP Vertica 7.1 introduces a new chart on the Activity page that monitors running queries through
Management Console.
The Query Monitoring page displays running queries, queued queries, successfully run queries, and
failed queries. You can profile a query or cancel a running query from this page. Charts at the
bottom of the page display aggregate query usage by node or user.
For more information about the Query Monitoring page, see Monitoring Running Queries in the
Administrator's Guide.
Resource Pool Monitoring and Configuration in MC

HP Vertica 7.1 introduces enhancements for monitoring and configuring resource pools. These new
features help you manage workloads by providing resource pool configuration options and visual
representations of resource usage through Management Console.
The Management Console Activity page now displays graphs that show:
Page 41 of 61
New Features Guide

Average historical resource usage by pool across all nodes.
Total resource pool memory usage by node.
Average query queue time and execution time per pool across all nodes.
Total number of resource requests that were rejected on a specified pool across all nodes.
From the MC Configuration page, you can now change certain pool parameters.
MC Explain Plan and Profiling Views

The HP Vertica 7.1 release introduces new features in Management Console that enhance
EXPLAIN plan and profile viewing. These new features allow you to analyze and monitor multiple
query attributes in a flexible and assessable visual environment.
With these new features, you can:
l
Change query input and output settings
Specify the query you want to profile by transaction ID and statement ID
View counter information at the node and operator level
Monitor the progress of the query visually in the form of a tree
Run profile analysis
Database Designer Enhancements in Management

Console
HP Vertica 7.1 introduces enhancements to Database Designer in Management Console. These
new features improve usability and provide Database Designer functionality that was not previously
available in Management Console.
These features appear in Database Designer in Management Console:
l
The option to turn on column correlation detection for a design appears in the General tab, or in
the Options screen when using the Database Designer Wizard.
Page 42 of 61
New Features Guide

When selecting queries to include in a design, the queries view in the Queries tab can be sorted
by most recent, most frequent, and longest running.
Icons now indicate the optimization status of design queries (optimized, partially optimized, or
unoptimized) in the Queries tab.
For more information about Database Designer in Management Console, see Using Management
Console to Create a Design in the Administrator's Guide.
Spread Retransmit Rate in Management Console

HP Vertica 7.1 introduces spread retransmit rate monitoring in Management Console. You can now
enable a spread retransmit rate check in Management Console Settings, and set an alert for spikes
in the spread retransmit rate that exceed a customizable threshold.
Clock Skew in Management Console

HP Vertica 7.1 introduces clock skew monitoring in Management Console. A new widget on the
Overview page alerts you to clock skew among nodes, which can interfere with Management
Console's ability to accurately monitor database activity.
Table Treemap Enhancements in Management Console

HP Vertica 7.1 includes the addition of a projections summary to the Table Treemap page in
Management Console's Activity page. The new projections summary displays the number of total
projections; skewed projections; unsafe projections; unsegmented projections; unused projections;
and storage containers for projections.
For more information on the Table Treemap chart in Management Console, see Monitoring Table
Utilization.
SQL Enhancements and Changes

This section contains the SQL enhancements made in Vertica Analytic Database 7.0.
New REBALANCE_TABLE Function

Synchronously rebalances data in the specified table.
A rebalance operation performs the following tasks:
Page 43 of 61
New Features Guide

Distributes data based on:

n
user-defined fault groups, if specified,
large cluster automatic fault groups
Redistributes the database projections' data across all nodes
When to Rebalance
Rebalancing is useful (or necessary) after you:
l
Mark one or more nodes as ephemeral in preparation of removing them from the cluster
Add one or more nodes to the cluster so that HP Vertica can populate the empty nodes with data
Change the scaling factor of an elastic cluster, which determines the number of storage
containers used to store a projection across the database
Set the control node size or realign control nodes on a large cluster layout
Add nodes to or remove nodes from a fault group
Syntax
REBALANCE_TABLE('schema.table_name')
Parameters
[schema]
[Optional] Specifies the name of the schema that contains the table that you want to
rebalance. If you do not specify a schema-name, the statement checks for matching
tables in the first schema listed in the current search_path.
table_name
Specifies the table that you want to rebalance.
Privileges
Must be a superuser.
Example
The following command shows how to rebalance data on the specified table.
Page 44 of 61
New Features Guide

=> SELECT REBALANCE_TABLE('online_sales.online_sales_fact');

REBALANCE_TABLE
------------------REBALANCED
(1 row)
New SQL ROLLUP Function

HP Vertica 7.1 adds the GROUPBY clause Online Analytic Processing (OLAP)
ROLLUPextension. The ROLLUP extension, and its accompanying grouping functions, let you
perform multi-level aggregates in a single query.
How ROLLUPWorks
ROLLUP performs aggregate operations on multiple levels in a hierarchy, creating sub-totals. HP
Vertica7.1 then aggregates (or rolls up) these subtotals from the most detailed level to a grand total.
ROLLUP requires an ordered list of grouping expressions used as arguments.
To aggregate data, ROLLUP performs these operations:
l
Calculates the standard aggregate values in the GROUP BY clause.
Creates progressively higher-level subtotals by moving from right to left through the grouping
columns, starting with the lowest level.
Continues this process until all the grouping columns have been incorporated and aggregations
computed. It then reports a grand total.
Using ROLLUP equates to using a UNIONALL with a set of GROUPBYqueries. With ROLLUP,
you specify only the required groupings in the GROUPBYclause. This approach allows more
efficient analysis and improved database performance.
ROLLUPGrouping Functions
ROLLUP supports the following grouping functions:
l
GROUPING()
GROUPING_ID()
GROUP_ID()
Page 45 of 61
New Features Guide

These grouping functions identify the group to which each row belongs and enable sorting subtotal
rows and filtering results .
See Also
l
GROUP BY Clause
GROUPING()
GROUP_ID ()
GROUPING_ID ()
New Window Partition Clause Options

HP Vertica7.1 adds a new option to the window_partition_clause named PARTITION BEST and
renames PARTITION AUTO to PARTITION NODES. Previously, you could use PARTITION AUTO within
the window partition clause of an OVER() statement to add parallelism while processing for
improved performance.
HP Vertica7.1, PARTITION BESTprovides increased performance for multi-threaded queries across
multiple nodes. PARTITION AUTO is renamed to PARTITION NODES, and provides increased
performance for single-threaded queries across multiple nodes. The term PARTITION AUTO is
deprecated in this release.
For complete details see window_partition_clause.
SQL For New Access Policy

Use the following new SQLstatements to manage the new access policy. The access policy
restricts access to sensitive information in a column to only those users authorized to view that
data.
l
CREATE ACCESSPOLICY - creates an access policy for a specific column(s) in a table.
ALTER ACCESS POLICY - specify actions that can be performed on an access policy.
DROP ACCESS POLICY - delete an access policy.
Page 46 of 61
New Features Guide

Performance
In this release, HP Vertica introduces numerous infrastructure improvements to increase
performance across the entire platform.
WITH Clause Materialization Option

HP Vertica 7.1.x provides an additional WITH clause evaluation method that materializes the
WITHclause subquery results into a temporary table.
Currently, HP Vertica uses inline expansion to implement all WITH clauses, evaluating them each
time they are referenced in the main query. The new functionality evaluates WITH clause
subqueries once and stores the results in a local temporary table. The optimizer then references the
temporary tables as often as necessary while processing the main query. The table is dropped
when the primary query completes.
By default, HP Vertica continues to use the inline expansion method. To enable the new
functionality, use add_vertica_option:
SELECT add_vertica_options('OPT', 'ENABLE_WITH_CLAUSE_MATERIALIZATION');
To disable the materialization method, use clr_vertica_options:

SELECT clr_vertica_options('OPT', 'ENABLE_WITH_CLAUSE_MATERIALIZATION');
Note: You must be a superuser to enable the materialization method.

For more information, see WITH Clauses in SELECT in the Administrators Guide, and WITH
Clause in the SQL Reference Manual.
Improved Performance for Simple SELECT Queries

HP Vertica 7.1 now provides better performance for simple SELECT queries. If HP Vertica detects
that the query can be answered by a single node, then the initiating node routes the query plan
directly to that node instead of running a distributed query plan across all nodes in the cluster. This
results in increased performance for simple SELECT queries.
Page 47 of 61
New Features Guide

Concurrent Small Queries

HP Vertica7.1 improves performance for workloads consisting of many short queries. Such queries
will have reduced lock/mutex contention, so will perform better and utilize modern CPUs more
effectively.
HP Vertica and Hadoop Integration Updates

This section explains new and updated features for integrating HP Vertica and Apache Hadoop.
Storage Location for HDFS

HP Vertica Version 7.1.x expands its ability to integrate with Apache Hadoop by introducing the HP
Vertica Storage Location for HDFS. This feature lets you create a storage location on HDFS. Once
created, HP Vertica can store ROS containers on HDFS the same way it stores them on native
Linux filesystems.
You can use the Storage Location for HDFS feature to free storage space on your HP Vertica
cluster by moving less-queried data to HDFS. The data remains available for queries, although at
the cost of slower performance.
HDFS is slower than native Linux filesystems. Use HDFS-based storage locations to store data
only when you can afford to have queries on that data perform slower than queries on data stored
natively. Because of the slower performance of HDFS, you must never use HDFS-based storage
locations to store temporary files.
See Storage Location for HDFS in the Hadoop Integration Guide for more information.
Changes to the HP Vertica Connector for HDFS

The HP Vertica Connector for HDFShas increased performance and better fault tolerance in this
release:
l
The connector now determines if the file it is retrieving from HDFSis split across Hadoop nodes.
If it is split, then the connector connects to the Hadoop nodes storing each piece of the file and
retrieves and reassembles the pieces itself. Previously, the Hadoop nodes would reassemble
the file before sending it to the connector. Having the connector directly access the pieces and
reassemble the file removes the delay caused by having the file's contents transferred between
nodes in the Hadoop cluster before being sent to HP Vertica.
Page 48 of 61
New Features Guide

When the connecter encounters an error while retrieving data from HDFS, it now automatically
retries the request. Previously, errors would often result in the query failing.
The connector now monitors the data transfer speed from the Hadoop cluster. If any connection
slows below a threshold (1 MB per second by default), the connector breaks it and tries to
retrieve the data from another node in the cluster. You can manually set the lowest allowable
data transfer rate. See Connector for HDFS Troubleshooting Tips for more information.
Changes to the HCatalog Connector

You can use the HCatalog Connector with all supported versions of Hadoop and Hive. To support
this use, HP Vertica requires some additional, one-time configuration. For details, see Configuring
Vertica for HCatalog.
Text Search
HP Vertica 7.1 introduces functionality to allow you to perform a text search on a field within a table.
The text search feature allows you to quickly search the contents of a single CHAR, VARCHAR or
LONGVARCHAR field within a table to locate a specific keyword. This feature is best used on
columns that will be repeatedly queried regarding their contents. After the text index is created,
DML operations will be slightly slower on the source table. This is a result of syncing the text index
and source table. Regular queries on the source table will not be affected.
This functionality includes the addition of two new SQL statements, CREATE TEXT INDEX and
DROP TEXT INDEX.
For more information and a detailed example, see Text Search Overview.
Scrutinize
The Scrutinize command will now accept more options to provide users with a custom diagnostic
report to best suit their needs.
The new options are:
l
--version
--no-containers
--no-active-queries
--with-active-queries
Page 49 of 61
New Features Guide

--begin
--end
For more information, see Collecting Diagnostics (scrutinize Command).
IPv6 Client Support

HP Vertica7.1 introduces support for IPv6 client connections, supported on these platforms:
l
Linux
Windows
Macintosh
See the for specifics on these platforms.

HP Vertica clients running on IPv6 networks can now connect to HP Vertica servers that have an
IPv6 interface. No server configuration is necessary to enable this feature as HP Vertica listens for
client connections on all network interfaces available on the server.
Note that IPV6 is not supported on the following client platforms in this release:
l
Solaris
AIX
HP-UX
The Connecting to HP Vertica Guide contains details for configuring clients to connect to an IPv6
interface on an HP Vertica server.
Python 3 Support
HP Vertica 7.1 introduces support for Python 3 and newer versions of PyODBC. The specific
versions supported are detailed in the Supported Platforms guide. See Perl and Python
Requirements.
Routable Connection API Support for Numeric Data

Type
HP Vertica 7.1 introduces support for the numeric data type in the Routable Connection API.
Page 50 of 61
New Features Guide

HP Vertica Place
HP Vertica 7.1 includes HP Vertica Place, an analytics package that provides users with the ability
to manipulate geospatial data. With 7.1.x, HP Vertica Place is now generally available. Previously,
HP Vertica Place was only available as part of the HP Vertica Innovations program.
You can download HP Vertica Place from my.vertica.com/downloads.
HP Vertica Place Features:
l
Over 50 Geospatial SQL Functions.
100% accuracy and faster spatial index creation.
Improved performance of spatial joins with points and polygons.
HP Vertica Place will be available on all supported HP Vertica platforms after 7.1.
For more information, see Place: An Overview.

Note: The HP Vertica Place version must be the same as the version of the HP Vertica server.
HP Vertica Pulse
HP Vertica 7.1 includes support for HP Vertica Pulse. HP Vertica Pulse is an add-on package that
provides a suite of functions that allow you to analyze and extract the sentiment from English
language text directly from your HP Vertica database.
HP Vertica Pulse features include:
l
Attribute based sentiment scoring - Pulse scores the sentiment of attributes in a sentence.
Attributes are generally nouns and are automatically discovered by Pulse. Pulse typically scores
sentiment from a range of -1 (negative sentiment) to +1 (positive sentiment). A sentiment of 0 is
considered neutral. Scoring individual attributes in a sentence instead of scoring the sentence as
a whole provides a more granular analysis for the text. For example, consider the sentence "The
quick brown fox jumped over the lazy dog." It would be difficult to score the sentiment on the
sentence as a whole, but if you score on the attributes of fox and dog, you could say the
sentiment on the fox was positive (the fox is quick), and the sentiment on the dog is negative
(the dog is lazy).
Page 51 of 61
New Features Guide

Tuning to your domain - Pulse provides functionality to recognize attributes that are specific
to your domain. For example, you can add the name of your product or company to a 'white_list'
so that it is discovered by Pulse.
Tuning of how sentiment is scored - Pulse includes user-dictionaries of words that are used
to help score sentiment. You can alter these user-dictionaries to fine tune the way your text is
analyzed.
Filtering of attributes you are not interested in - Pulse supports a special 'stop words' userdictionary to indicate attributes that should not be analyzed. Alternately, you can choose to
score sentiment only on attributes defined in your white_list.
Synonym mappings - Pulse provides customizable mappings so that you can map synonyms
to a base word, and then normalize the analysis for the synonyms to the base word. For
example, you can map Hewlett Packard to HP.
HP Vertica Pulse requires that Java and the HP Vertica Java Support Package are installed on all
nodes in the HP Vertica cluster.
Note: The HP Vertica Pulse version must be the same as the version of the HP Vertica server.
For more information, see About the Package.
Page 52 of 61
New Features Guide

HP Vertica 7.1.x introduces a number of changes to the standard documentation set.
Programmer's Guide Separated Into Individual

Guides
The Programmer'sGuide has been broken into several smaller guides, to better target separate
audiences. These new guides are:
l
Analyzing Data explains how to write queries in HP Vertica SQL.
Connecting to HP Vertica explains how to use the vsql client and the HP Verticaclient libraries.
Extending HP Vertica explains how to use SQLmacros and the SDK to add analytic features to
HP Vertica.
Documentation Set Available in a Single PDF File

The vertica documentation set is now available in its entirety in a single PDF file.
Using the new PDF gives you all HP Vertica documents in one file while working offline. You can
search across all guides and all cross-book hyperlinks are now live. (Separate PDF docs do not
support live links to other guides.)
The documentation set continues to be available online, in HTML. You can also access all guides in
individual PDF files as well.
New Location for HPVertica on Amazon Web

Services Guide
The HPVertica on Amazon Web Services Guide has moved to a new location:
http://www.vertica.com/resources-for-technology-partner-integrations/
The new location will allow us more flexibility when updating this documentation. (HPreleases
HPVertica AMIs on a slightly different schedule than our software release schedule.)
Page 53 of 61
New Features Guide


In version 7.1, the following HP Vertica functionality has been deprecated.
l
The Geospatial Package SQL functions.
The ADD_LOCATION() function, which has been replaced by the

CREATELOCATIONstatement.
The JavaClassPathForUDx configuration parameter. Instead of setting this parameter, you now
set a class path on each UDx library. Set this per-library class path using the new
DEPENDSkeyword of the CREATE LIBRARYand ALTER LIBRARYstatements.
The verticaConfig vbr.py configuration option has been deprecated.
See Also
For information about the meaning of obsolete and deprecated functionality, see Retired
Functionality.
Page 54 of 61
New Features Guide

This section describes the two phases HP follows to retire HP Vertica functionality.
Deprecated. HP Vertica declares a feature deprecated in a major or minor release. Deprecated
features remain in the product and are functional. However, users accessing the feature receive
messages informing them that the feature will be removed in the following major or minor release.
Features are identified as deprecated in HP Vertica release notes and in feature documentation; all
documentation remains accessible. HP announces feature deprecation only in major and minor
releases.
Removed. HP removes a feature in the major or minor release immediately following the
deprecation announcement. Users can no longer access the functionality. The feature removal is
announced in HP Vertica release notes. All feature documentation is removed.
The following functionality has been deprecated, or removed in past and present versions:
Page 55 of 61
New Features Guide

Functionality
Deprecated Removed
Component Version
Version
Geospatial Package SQL Functions
Server
BB_WITHIN
BEARING
CHORD_TO_ARC
DWITHIN
ECEF_CHORD
ECEF_x
ECEF_y
ECEF_z
ISLEFT
KM2MILES
LAT_WITHIN
LL_WITHIN
LLD_WITHIN
LON_WITHIN
MILES2KM
RADIUS_LON
RADIUS_M
RADIUS_N
RADIUS_R
RADIUS_Ra
RADIUS_Rc
7.1
7.2
Page 56 of 61
New Features Guide

Functionality
l
RADIUS_Rv
RADIUS_SI
RAYCROSSING
WGS84_a
WGS84_b
WGS84_e2
WGS84_f
WGS84_if
WGS84_r1
Deprecated Removed
Component Version
Version
verticaConfig vbr.py configuration option
Server
7.1
JavaClassPathForUDx configuration parameter
Server
7.1
ADD_LOCATION()
Server
7.1
bwlimit
Server
7.1
EXECUTION_ENGINE_PROFILES counters file
Server
7.0
MERGE_PARTITIONS()
Server
7.0
Administration Tools option check_spread
Server,
7.0
handles, memory allocated, and memory reserved
clients
krb5 client authentication method
All clients
7.0
Pload Library
Server
7.0
USESINGLETARGET
Server
7.0
scope parameter of CLEAR_PROFILING
Server
6.1
7.1
Page 57 of 61
New Features Guide

Functionality
Deprecated Removed
Component Version
Version
IMPLEMENT_TEMP_DESIGN()
Server,
6.1
clients
USER_TRANSFORMSuser table
Server
6.0
UPDATEprivileges on sequences
Server
6.0
Query Repository, which includes:
Server
6.0
Server
6.0
SYS_DBA.QUERY_REPO table
Functions:
l
CLEAR_QUERY_REPOSITORY()
SAVE_QUERY_REPOSITORY()
Configuration parameters:
l
CleanQueryRepoInterval
QueryRepoMemoryLimit
QueryRepoRetentionTime
QueryRepositoryEnabled
SaveQueryRepoInterval
QueryRepoSchemaName
QueryRepoTableName
See Notes section below table.

RESOURCE_ACQUISITIONS_HISTORY system
table
Volatility and NULL behavior parameters of CREATE
Server
6.1
Server
6.0
FUNCTION
Ganglia on Red Hat 4
Page 58 of 61
New Features Guide

Functionality
Deprecated Removed
Component Version
Version
copy_vertica_database.sh
Server
restore.sh
Server
backup.sh
Server
LCOPY (see Note section below table)
Server,
clientsw
4.1 (Client)
5.1 (Server)
5.1
(Client)
MergeOutPolicySizeList
Server
4.1
5.0
EnableStrataBasedMrgOutPolicy
Server
4.1
5.0
ReportParamSuccess
All clients
4.1
5.0
BatchAutoComplete
All clients
4.1
5.0
use35CopyParameters
ODBC,
4.1
5.0
JDBC
clients
getNumAcceptedRows
getNumRejectedRows
ODBC,
5.0
JDBC
clients
MANAGED load (server keyword and related client
Server,
5.0
parameter)
clients
EpochAdvancementMode
Server
4.1
5.0
VT_ tables
Server
4.1
5.0
RefreshHistoryDuration
Server
4.1
5.0
Notes
l
While the HPVertica Geospatial package has been deprecated, it has been replaced by
HPVertica Place. This analytics package is available on my.vertica.com/downloads.
LCOPY: Supported by the 5.1 server to maintain backwards compatibility with the 4.1 client
drivers.
Page 59 of 61
New Features Guide

Query Repository: You can still monitor query workloads with the following system tables:
n
QUERY_PROFILES
SESSION_PROFILES
EXECUTION_ENGINE_PROFILES
In addition, HP Vertica Version 6.0 introduced new robust, stable workload-related system table:
l
QUERY_REQUESTS
QUERY_EVENTS
RESOURCE_ACQUISITIONS
The RESOURCE_ACQUISITIONS system table captures historical information.
Use the Kerberos gss method for client authentication, instead of krb5. See Configuring
Kerberos Authentication.
Page 60 of 61
We appreciate your feedback!

If you have comments about this document, you can contact the documentation team by email. If
an email client is configured on this system, click the link above and an email window opens with
the following information in the subject line:
Feedback on New Features Guide (Vertica Analytic Database 7.1.x)
Just add your feedback to the email and click send.
If no email client is available, copy the information above to a new message in a web mail client,
and send your feedback to vertica-docfeedback@hp.com.
Page 61 of 61

HP Vertica 7.1.x New Features

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

HP Vertica 7.1.x New Features

Hochgeladen von

Copyright:

Verfügbare Formate

New Features Guide

HP Vertica Analytic Database

Document Release Date: 7/21/2016

Restricted Rights Legend

HP Vertica Analytic Database

New Features and Changes in HP Vertica 7.1 SP2 (7.1.2)

New HP Vertica Place Functionality

New HP Vertica Pulse Functionality

Case-Sensitive Sentiment Analysis

Concurrent User Defined Dictionaries

Dictionary And Mapping Labels

New HP Vertica Pulse Functions

OLE DB Driver Added for SSAS

Row Counts for External Tables

Flex Tables Load and Query

New Parameters for fjsonparser

New Parameter for fdelimitedparser and mapdelimitedextractor Function

Improvements to GROUP BY Aggregation

New Hadoop and Storage Location Features

Using MC to Import HP Vertica in a Hadoop Environment

New Installer for ODBCDriver on Mac OSX

MC Manager and Associate Roles

New Features and Changes in HP Vertica 7.1 SP1 (7.1.1)

Client Authentication in Catalog

Flex Tables Load and Query

Secure Password File

Rsync Service Account

HP Vertica Analytic Database

New Features Guide

New System Table

New Configuration Parameter Options for SET, CLEAR, and SHOW

Command-line Distribution of Configuration Files

New scrutinize Functionality

Enhanced collection capabilities

Export of License Audit Results to CSV

Projections Quick Stat forManagement Console

JDBC Routable Query (Key/Value) API Improvements

Pulse Multilingual Support

New Features and Changes in HP Vertica 7.1.0

Live Aggregate Projections and Projections with Expressions

New Live Aggregate Projections Capability

New Top-K Projections Capability

New Capability to Use Expressions in Projection Definitions

New and Modified Client Authentication Methods

Client Authentication Information Now Stored in Database Catalog

SHA-512 Now Available for Hash Authentication

Configuring Client Authentication with Key/Value Pairs

Client Certificate Authentication

DSN Changes for ODBC

Vertica Configuration File (vertica.conf) Changes

HP Vertica Analytic Database (7.1.x)

New Features Guide

Change to SDK API Binary Compatibility Policy

Allowed Number of Arguments for UDxs Increased

Ability to Specify Dependency Paths for UDx Libraries

Flex Tables Load and Query

Changes to Flex Data Functions

Improved Performance Using PARTITION BESTOption

COPY Statement Changes

Logging Projection Data

New Capability to Swap Partitions Atomically

New Column Encoding Compression Options

Usability and Administration

Active Standby Nodes