Beruflich Dokumente
Kultur Dokumente
Table of Contents
Preface .................................................................................................................................................... ix 1. Introduction to Couchbase Server .............................................................................................................. 1 1.1. Couchbase Server Basics ............................................................................................................... 1 1.2. Couchbase Server and NoSQL ........................................................................................................ 2 1.3. Architecture and Concepts ............................................................................................................. 2 1.3.1. Nodes and Clusters ............................................................................................................ 3 1.3.2. Cluster Manager ................................................................................................................ 3 1.3.3. Memory Quotas ................................................................................................................. 3 1.3.4. Buckets ............................................................................................................................ 4 1.3.5. vBuckets .......................................................................................................................... 6 1.3.6. Disk Storage ..................................................................................................................... 6 1.3.7. Failover ............................................................................................................................ 7 1.3.8. Client Interface .................................................................................................................. 7 1.3.9. Administration Tools .......................................................................................................... 8 1.3.10. Statistics and Monitoring ................................................................................................... 8 2. Getting Started ..................................................................................................................................... 10 2.1. Preparation ................................................................................................................................ 10 2.1.1. Supported Platforms ......................................................................................................... 10 2.1.2. Hardware Requirements .................................................................................................... 10 2.1.3. Storage Requirements ....................................................................................................... 10 2.1.4. Web Browser (for administration) ....................................................................................... 11 2.1.5. Network Ports ................................................................................................................. 11 2.2. Installing Couchbase Server .......................................................................................................... 11 2.2.1. Red Hat Linux Installation ................................................................................................. 11 2.2.2. Ubuntu Linux Installation .................................................................................................. 12 2.2.3. Microsoft Windows Installation .......................................................................................... 13 2.2.4. Mac OS X Installation ...................................................................................................... 17 2.3. Upgrading to Couchbase Server 1.8 ............................................................................................... 18 2.3.1. Manually Controlled Upgrade Options ................................................................................. 19 2.3.2. Using cbupgrade Manually ............................................................................................... 20 2.4. Setting up Couchbase Server ........................................................................................................ 21 2.5. Testing Couchbase Server ............................................................................................................ 29 2.6. Next Steps ................................................................................................................................. 30 3. Administration Basics ............................................................................................................................ 31 3.1. Working with Couchbase data files ................................................................................................ 31 3.2. Startup and Shutdown of Couchbase Server ..................................................................................... 32 3.2.1. Startup and Shutdown on Linux .......................................................................................... 32 3.2.2. Startup and Shutdown on Windows ..................................................................................... 32 3.2.3. Startup and Shutdown on Mac OS X ................................................................................... 32 4. Best Practices ....................................................................................................................................... 35 4.1. Cluster Design Considerations ...................................................................................................... 35 4.2. Sizing Guidelines ........................................................................................................................ 35 4.2.1. RAM Sizing .................................................................................................................... 36 4.2.2. Disk Throughput and Sizing ............................................................................................... 38 4.2.3. Network Bandwidth .......................................................................................................... 38 4.2.4. Data Safety ..................................................................................................................... 39 4.3. Deployment Considerations .......................................................................................................... 40 4.4. Ongoing Monitoring and Maintenance ............................................................................................ 41 4.5. Couchbase Behind a 2nd Firewall .................................................................................................. 42 4.6. Using Couchbase in the Cloud ...................................................................................................... 42 4.6.1. Local Storage .................................................................................................................. 43
iii
5.
6.
7.
8.
4.6.2. IP addresses .................................................................................................................... 4.6.3. Security groups/firewall settings .......................................................................................... 4.6.4. Swap Space ..................................................................................................................... 4.7. Deployment Strategies ................................................................................................................. 4.7.1. Using Couchbase Embedded Proxy ..................................................................................... 4.7.2. Standalone Proxy ............................................................................................................. 4.7.3. Using a vBucket aware Client ............................................................................................ Administration Tasks ............................................................................................................................. 5.1. Failover with Couchbase .............................................................................................................. 5.1.1. Automatic Failover ........................................................................................................... 5.1.2. Automated failover considerations ....................................................................................... 5.1.3. Monitored failover ............................................................................................................ 5.2. Backup and Restore with Couchbase .............................................................................................. 5.2.1. Backup ........................................................................................................................... 5.2.2. Restoring a Backup .......................................................................................................... 5.3. Expanding and Shrinking your Cluster (Rebalancing) ........................................................................ 5.3.1. Growing your cluster ........................................................................................................ 5.3.2. Shrinking your cluster ....................................................................................................... 5.3.3. Rebalancing .................................................................................................................... Administration Web Console ................................................................................................................... 6.1. Cluster Overview ........................................................................................................................ 6.1.1. Cluster ........................................................................................................................... 6.1.2. Buckets .......................................................................................................................... 6.1.3. Servers ........................................................................................................................... 6.2. Web Console Statistics ................................................................................................................ 6.3. Data Buckets ............................................................................................................................. 6.3.1. Creating and Editing Data Buckets ...................................................................................... 6.3.2. Bucket Information ........................................................................................................... 6.3.3. Individual Bucket Monitoring ............................................................................................. 6.4. Server Nodes ............................................................................................................................. 6.5. Couchbase Server States .............................................................................................................. 6.6. Settings ..................................................................................................................................... 6.6.1. Update Notification Settings ............................................................................................... 6.6.2. Auto-Failover Settings ...................................................................................................... 6.6.3. Alerts ............................................................................................................................. 6.7. Log .......................................................................................................................................... 6.7.1. Diagnostic Report ............................................................................................................. 6.8. Update Notifications .................................................................................................................... 6.8.1. Notifications .................................................................................................................... 6.8.2. Viewing Available Updates ................................................................................................ 6.9. Warnings and Alerts ................................................................................................................... Couchbase Command-line Interface .......................................................................................................... 7.1. couchbase-cli Tool ..................................................................................................................... 7.2. cbstats Tool .............................................................................................................................. 7.3. cbflushctl Tool .......................................................................................................................... 7.4. tap.py Tool ............................................................................................................................... 7.5. cbvbucketctl Tool ...................................................................................................................... 7.6. cbvbuckettool Tool .................................................................................................................... 7.7. cbdbmaint Tool ......................................................................................................................... 7.8. cbcollect_info Tool ..................................................................................................................... Couchbase Management REST API ......................................................................................................... 8.1. Key Concepts ............................................................................................................................ 8.1.1. Resources ....................................................................................................................... 8.1.2. HTTP Request Headers .....................................................................................................
43 44 45 45 45 45 46 47 47 47 48 48 49 49 50 52 52 52 53 54 54 55 56 56 57 57 58 61 61 68 71 72 72 73 73 75 76 77 77 77 77 79 79 82 82 83 83 83 84 84 85 85 85 86
iv
8.1.3. HTTP Status Codes .......................................................................................................... 86 8.2. Operations ................................................................................................................................. 87 9. Developing with Couchbase .................................................................................................................. 105 9.1. Use Cases ................................................................................................................................ 105 9.1.1. Session store .................................................................................................................. 105 9.1.2. Social gaming ................................................................................................................ 105 9.1.3. Ad, offer and content targeting ......................................................................................... 105 9.1.4. Real-time logging and alerting .......................................................................................... 105 9.2. Best practices ........................................................................................................................... 106 9.2.1. What should I store in an object ........................................................................................ 106 9.2.2. How should I store an object? .......................................................................................... 106 9.2.3. Objects that refer to other objects ...................................................................................... 106 9.2.4. Nested Items .................................................................................................................. 106 9.2.5. Simple Lists .................................................................................................................. 107 9.2.6. Multi-GET .................................................................................................................... 108 9.2.7. Locking ........................................................................................................................ 108 9.2.8. Data partitioning with buckets .......................................................................................... 109 9.3. Couchbase for Memcached Users ................................................................................................ 109 10. Developing Couchbase Clients ............................................................................................................. 110 10.1. REST/JSON ........................................................................................................................... 110 10.1.1. Parsing the JSON .......................................................................................................... 111 10.1.2. Encoding the vBucketId into each request ......................................................................... 112 10.1.3. Client Libraries and Rebalancing ..................................................................................... 112 10.1.4. Fast Forward Map ......................................................................................................... 113 10.1.5. Redundancy & Availability ............................................................................................. 113 10.2. SASL Authentication Example .................................................................................................. 113 10.2.1. List Mechanisms ........................................................................................................... 114 10.2.2. Authentication request ................................................................................................... 115 11. Couchbase Architecture ...................................................................................................................... 117 11.1. Cluster Design ........................................................................................................................ 117 11.2. Persistence Design ................................................................................................................... 117 11.3. Component Overview ............................................................................................................... 117 11.4. Disk Storage (Growing Data Sets Beyond Memory) ...................................................................... 118 11.4.1. Design ........................................................................................................................ 118 11.4.2. Consequences of Memory faster than Disk ........................................................................ 119 11.5. Couchbase APIs ...................................................................................................................... 121 11.5.1. memcached protocol ..................................................................................................... 121 11.5.2. Additional Protocol Operations ........................................................................................ 124 11.6. Buckets ................................................................................................................................. 125 11.7. vBuckets ................................................................................................................................ 127 11.7.1. Couchbase Document ID-vBucket-Server Mapping Illustrated ............................................... 127 11.7.2. vBuckets in a world of memcached clients ....................................................................... 128 12. Monitoring Couchbase ........................................................................................................................ 130 12.1. Port numbers and accessing different buckets ............................................................................... 130 12.2. Monitoring startup (warmup) ..................................................................................................... 130 12.3. Disk Write Queue ................................................................................................................... 131 12.3.1. Handling Reads ............................................................................................................ 131 12.3.2. Monitoring the Disk Write Queue .................................................................................... 132 12.4. Couchbase Server Statistics ....................................................................................................... 132 12.4.1. REST Interface Statistics ................................................................................................ 132 12.4.2. Couchbase Server Node Statistics .................................................................................... 132 12.5. Couchbase Server Moxi Statistics ............................................................................................... 133 13. Troubleshooting ................................................................................................................................. 135 13.1. General Tips ........................................................................................................................... 135
13.2. Responding to Specific Errors ................................................................................................... 13.3. Log File Entries ...................................................................................................................... 13.4. Linux Logs ............................................................................................................................ 13.5. Windows Logs ........................................................................................................................ 13.6. Common Errors ...................................................................................................................... A. FAQs ................................................................................................................................................ B. Uninstalling Couchbase Server .............................................................................................................. B.1. Uninstalling on a RedHat Linux System ....................................................................................... B.2. Uninstalling on an Debian/Ubuntu Linux System ........................................................................... B.3. Uninstalling on a Windows System ............................................................................................. B.4. Uninstalling on a Mac OS X System ........................................................................................... C. Release Notes ..................................................................................................................................... C.1. Release Notes for Couchbase Server 1.8.0 GA (23 January 2012) ...................................................... D. Licenses ............................................................................................................................................ D.1. Documentation License ............................................................................................................. D.2. Couchbase, Inc. Community Edition License Agreement ................................................................. D.3. Couchbase, Inc. Enterprise License Agreement: Free Edition ............................................................
135 136 136 136 136 138 140 140 140 140 140 141 141 144 144 144 145
vi
List of Figures
1.1. Multi-tenancy ...................................................................................................................................... 5 2.1. Windows Installation Welcome Screen ................................................................................................... 13 2.2. Windows Installation Location Screen .................................................................................................... 14 2.3. Windows Installation Ready Screen ....................................................................................................... 15 2.4. Windows Installation Progress Screen .................................................................................................... 16 2.5. Windows Installation Completion Screen ................................................................................................ 17 2.6. Couchbase Server Setup ....................................................................................................................... 22 2.7. Couchbase Server Setup Step 1 (New Cluster) ......................................................................................... 23 2.8. Couchbase Server Setup Step 1 (Existing Cluster) .................................................................................... 24 2.9. Couchbase Server Setup Step 2 ............................................................................................................. 25 2.10. Couchbase Server Setup Step 3 ........................................................................................................... 26 2.11. Couchbase Server Setup Step 4 ........................................................................................................... 28 2.12. Couchbase Server Setup Completed ..................................................................................................... 29 3.1. Couchbase Server on Mac OS X Menubar Item ....................................................................................... 33 4.1. Deployment Strategy: Using the Embedded Proxy .................................................................................... 45 4.2. Deployment Strategy: Standalone Proxy ................................................................................................. 46 4.3. Deployment Strategy: Using a vBucket Aware Client ................................................................................ 46 6.1. Couchbase Web Console - Cluster Overview ........................................................................................... 55 6.2. Couchbase Web Console - Data Buckets Overview ................................................................................... 58 6.3. Couchbase Web Console - Create Bucket ............................................................................................... 59 6.4. Couchbase Web Console - Bucket Information ........................................................................................ 61 6.5. Couchbase Web Console - Summary Statistics ......................................................................................... 62 6.6. Couchbase Web Console - vBucket Resources statistics ............................................................................. 63 6.7. Couchbase Web Console - Disk Queue Statistics ...................................................................................... 65 6.8. Couchbase Web Console - TAP Queue Statistics ...................................................................................... 66 6.9. Couchbase Web Console - Memcached Statistics ..................................................................................... 67 6.10. Couchbase Web Console - Server Nodes ............................................................................................... 69 6.11. Couchbase Web Console- Data Bucket/Server view ................................................................................ 70 6.12. Couchbase Web Console - Server specific view ..................................................................................... 71 6.13. Down Status .................................................................................................................................... 72 6.14. Pend Status ...................................................................................................................................... 72 6.15. Web Console Settings Update Notifications ....................................................................................... 73 6.16. Web Console Settings Auto-Failover ................................................................................................ 73 6.17. Web Console Settings Alerts .......................................................................................................... 75 6.18. Web Console Log Viewer .................................................................................................................. 76 11.1. Bucket Configuration ....................................................................................................................... 119 11.2. Background Fetch Workflow ............................................................................................................. 120 11.3. Couchbase Buckets .......................................................................................................................... 126
vii
List of Tables
1.1. Bucket Types ...................................................................................................................................... 4 1.2. Couchbase Bucket Capabilities ............................................................................................................... 5 2.1. Network Ports used by Couchbase Server ............................................................................................... 11 4.1. Input Variables ................................................................................................................................... 36 4.2. Constants .......................................................................................................................................... 37 4.3. Input Variables ................................................................................................................................... 37 4.4. Constants .......................................................................................................................................... 37 4.5. Variable Calculations .......................................................................................................................... 38 7.1. Couchbase Server Command-line Tools .................................................................................................. 79 7.2. couchbase Tool Commands ................................................................................................................. 80 7.3. Standard couchbase Options ................................................................................................................ 80 8.1. Supported Request Headers .................................................................................................................. 86 8.2. HTTP Status Codes ............................................................................................................................. 86 8.3. Controller Functions ............................................................................................................................ 94 8.4. Node Attributes .................................................................................................................................. 94 8.5. Cluster Joining Arguments .................................................................................................................. 103 8.6. Additional Arguments ........................................................................................................................ 103 12.1. Stats .............................................................................................................................................. 131 13.1. Responses to Specific Errors ............................................................................................................. 135
viii
Preface
This manual documents the Couchbase Server database system, for the 1.8 release series. For differences between individual version within this release series, see the version-specific notes throughout the manual. Couchbase Server 1.8 is a re-branded release of Membase Server 1.7 with some additional fixes and changes. See Appendix C, Release Notes for more details of the precise changes. Couchbase Server 1.8 retains all the same functionality and capabilities of Membase Server 1.7, including: Memcached compatible Clone to grow with auto-sharding Zero-downtime maintenance Production-ready management and monitoring Reliable storage architecture Data replication with auto-failover Professional SDKs for wide variety of languages For more help on specific areas of using Couchbase Server 1.8, see: Chapter 2, Getting Started Chapter 4, Best Practices Section 5.2, Backup and Restore with Couchbase Section 5.1, Failover with Couchbase Section 4.6, Using Couchbase in the Cloud Chapter 6, Administration Web Console Section 4.2, Sizing Guidelines Chapter 11, Couchbase Architecture Appendix A, FAQs
ix
In tandem with the elastic nature of Couchbase Server, a Couchbase Cluster also takes advantage of the clustered architecture to support high availability. All nodes in a cluster are identical, and the cluster automatically creates replicas of information across the cluster. If a node fails, the stored data will be available on another node in the cluster. memcached Compatibility memcached is an memory-based caching application that uses the notion of a document store to save important data that are required by applications directly in RAM. Because the information is stored entirely in RAM, the latency for storing and retrieving information is very low. As a caching solution in it's right,memcached is used by a wide range of companies, including Google, Facebook, YouTube, FarmVille, Twitter and Wikipedia to help speed up their web-application performance by acting as a storage location for objects retrieved at comparative expense from a traditional SQL database. Couchbase Server supports the same client protocol used by memcached for creating, retrieving, updating and deleting information in the database. This enables Couchbase Server to be a drop-in replacement for memcached, and this means that applications already employing memcached can take advantage of the other functionality within Couchbase Server, such as clustered and elastic distribution.
in the cluster. To keep throughput high and latency low, Couchbase Server will always keep metadata about all items in memory. When configuring a Couchbase Server, a memory quota is set. Couchbase Server will automatically migrate items from memory to disk when the configured memory quota is reached. If those items are later accessed, they will be moved back into system memory. For efficiency purposes, these operations are performed on a regular basis in the background. At the moment, there is no ability define a quota for the on-disk persistent storage. It is up to the administrator to appropriately monitor the disk utilization and take action (either deleting data from Couchbase or adding capacity by upgrading the nodes themselves or adding more nodes). Couchbase Server monitors and reports on statistics for managing disk and memory. As with any multi-tier cache, if the working set of data is greater than the available amount of the bucket RAM quota (the first level of caching), performance will drop due to disk access latencies being higher and disk throughput being lower than RAM latencies and throughput. Acceptable performance of the system is application dependent. Statistics should be monitored in case tuning adjustments are required. Server Quotas Each server node has a memory quota that defines the amount of system memory that is available to that server node on the host system. The first node in a cluster sets a memory quota that is subsequently inherited by all servers joining the cluster. The maximum memory quota set on the first server node must be less than or equal to 80% of the total physical RAM on that node. A server cannot join a cluster if it has less physical RAM than 1.25x the RAM quota (the same maximum allocation of 80% of physical RAM to the cluster). If a server that was a standalone cluster joins another cluster, the memory quota is inherited from the cluster to which the node is added. Server nodes do not have disk quotas. System administrators are responsible for monitoring free disk space on individual server nodes. Each server node in a cluster has its own storage path - the location on disk where data will be stored. Storage paths do not need to be uniform across all server nodes in a cluster. If a server that was a standalone cluster joins another cluster, the storage path for that server remains unchanged. Bucket Quotas Memory quota allocation is also controlled on a bucket-by-bucket basis. A fixed amount of memory per node is allocated for use by a bucket. Adding or removing nodes will change the size of the bucket.
1.3.4. Buckets
Couchbase Server provides data management services using named buckets. These are isolated virtual containers for data. A bucket is a logical grouping of physical resources within a cluster of Couchbase Servers. They can be used by multiple client applications across a cluster. Buckets provide a secure mechanism for organizing, managing, and analyzing data storage resources. Couchbase Server provides the two core types of buckets that can be created and managed. Couchbase Server collects and reports on run-time statistics by bucket type. Table 1.1. Bucket Types Bucket Type Couchbase Description Provides highly-available and dynamically reconfigurable distributed data storage, providing persistence and replication services. Couchbase buckets are 100% protocol compatible with, and built in the spirit of, the memcached open source, distributed document cache. Provides a directly-addressed, distributed (scale-out), inmemory, document cache. Memcached buckets are de-
Memcached
Bucket Type
Description signed to be used alongside relational database technology caching frequently-used data, thereby reducing the number of queries a database server must perform for web servers delivering a web application.
The different bucket types support different capabilities. Couchbase-type buckets provide a highly-available and dynamically reconfigurable distributed data store. Couchbase-type buckets survive node failures and allow cluster reconfiguration while continuing to service requests. Couchbase-type buckets provide the following core capabilities. Table 1.2. Couchbase Bucket Capabilities Capability Persistence Description Data objects can be persisted asynchronously to hard-disk resources from memory to provide protection from server restarts or minor failures. Persistence properties are set at the bucket level. A configurable number of replica servers can receive copies of all data objects in the Couchbase-type bucket. If the host machine fails, a replica server can be promoted to be the host server, providing continuous (HA) cluster operations via fail-over. Replication is set at the bucket level. Rebalancing enables load distribution across resources and dynamic addition or removal of buckets and servers in the cluster.
Replication
Rebalancing
For more information on the bucket types, their configuration and accessibility, see Section 11.6, Buckets. Couchbase Server leverages the memcached storage engine interface and the Couchbase Bucket Engine to enable isolated buckets that support multi-tenancy. Figure 1.1. Multi-tenancy
Smart clients discover changes in the cluster using the Couchbase Management REST API. Buckets can be used to isolate individual applications to provide multi-tenancy, or to isolate data types in the cache to enhance performance and visibili-
ty. Couchbase Server allows you to configure different ports to access different buckets, and gives you the option to access isolated buckets using either the binary protocol with SASL authentication, or the ASCII protocol with no authentication Couchbase Server allows you to use and mix different types of buckets (Couchbase and Memcached) as appropriate in your environment. Buckets of different types still share the same resource pool and cluster resources. Quotas for RAM and disk usage are configurable per bucket so that resource usage can be managed across the cluster. Quotas can be modified on a running cluster so that administrators can reallocate resources as usage patterns or priorities change over time.
1.3.5. vBuckets
A vBucket is defined as the owner of a subset of the key space of a Couchbase cluster. These vBuckets are used to allow information to be distributed effectively across the cluster. The vBucket system is used both for distributing data, and for supporting replicas (copies of bucket data) on more than one node. Note vBuckets are not a user-accessible component, but they are a critical component of Couchbase Server and are vital to the availability support and the elastic nature. Every document ID belongs to a vBucket. A mapping function is used to calculate the vBucket in which a given document belongs. In Couchbase Server, that mapping function is a hashing function that takes a document ID as input and outputs a vBucket identifier. Once the vBucket identifier has been computed, a table is consulted to lookup the server that "hosts" that vBucket. The table contains one row per vBucket, pairing the vBucket to its hosting server. A server appearing in this table can be (and usually is) responsible for multiple vBuckets. The hashing function used by Couchbase Server to map document IDs to vBuckets is configurable - both the hashing algorithm and the output space (i.e. the total number of vBuckets output by the function). Naturally, if the number of vBuckets in the output space of the hash function is changed, then the table which maps vBuckets to Servers must be resized.
ments stored in the database and this information is held in RAM. This means that the server can always return a 'document ID not found' response for an invalid document ID, while waiting (and returning) the data for an item either in RAM (in which case it is returned immediately), or after the item has been read from disk (after a delay, or until a timeout has been reached). The process of moving information to and from disk is asynchronous. Data is evicted to disk from memory in the background while the server continues to service active requests. Eviction requests are written to an eviction queue to be written to disk in the background. During sequences of high writes to the database, clients will be notified that the server is temporarily out of memory until enough items have been evicted from memory to disk. Similarly, when the server identifies an item that needs to be loaded from disk because it is not in active memory, the process is handled by a background process that processes the load queue and reads the information back disk and into memory. The client is made to wait until the data has been loaded back into memory before the information is returned. The asynchronous nature and use of queues in this way enables reads and writes to be handled at a very fast rate, while removing the typical load and performance spikes that would otherwise cause a server running Couchbase Server to produce erratic performance.
1.3.7. Failover
Information is distributed around a cluster using a series of replicas. For Couchbase Buckets you can configure the number of replicas(complete copies of the data stored in the bucket) that should be kept within the Couchbase Cluster, and the number of replicas should be less than the number of Couchbase Servers configured in your cluster. In the event of a failure in a server (either due to transient failure, or for administrative purposes), you can use a technique called failover to indicate that a node within the Couchbase Cluster is no longer available, and that the replica vBuckets for the server are enabled. The Failover process contacts each server that was acting as a replica and updates the internal table that maps client requests for documents to an available server. Failover can be performed manually, or you can use the built-in automatic failover that reacts after a preset time when a node within the cluster becomes unavailable. For more information, see Section 5.1, Failover with Couchbase.
Update To update information in the database, you must use the memcached protocol interface. The memcached protocol includes functions to directly update the entire contents, and also to perform simple operations, such as appending information to the existing record, or incrementing and decrementing integer values. Delete To delete information from Couchbase Server you need to use the memcached protocol which includes an explicit delete command to remove a document from the server. However, Couchbase Server also allows information to be stored in the database with an expiry value. The expiry value states when a document should be automatically deleted from the database, and can either be specified as a relative time (for example, in 60 seconds), or absolute time (31st December 2012, 12:00pm). The methods of creating, updating and retrieving information are critical to the way you work with storing data in Couchbase Server.
The statistics are divided into a number of groups, allowing you to identify different states and performance information within your cluster: By Node Node statistics show CPU, RAM and I/O numbers on each of the servers and across your cluster as a whole. This information can be used to help identify performance and loading issues on a single server. By vBucket The vBucket statistics show the usage and performance numbers for the vBuckets used to store information in the cluster. These numbers are useful to determine whether you need to reconfigure your buckets or add servers to improve performance. By Disk Queues These statistics monitor the queues used to read and write information to disk and between replicas. This information can be helpful in determining whether you should expand your cluster to reduce disk load. By TAP Queues The TAP interface is used to monitor changes and updates to the database. TAP is used internally by Couchbase to provide replication between Couchbase nodes, but can also be used by clients for change notifications. In nearly all cases the statistics can be viewed both on a whole of cluster basis, so that you can monitor the overall RAM or disk usage for a given bucket, or an individual server basis so that you can identify issues within a single machine.
2.1. Preparation
Warning Heterogeneous or mixed deployments (deployments with both Linux and Windows server nodes) are not supported at this time. It is recommended that when deploying to multiple systems, that system be running the same operating system. When running Couchbase Server your system should meet or exceed the following system requirements.
10
Getting Started
Disk space to match your physical RAM requirements for persistence of information
11
Getting Started
To install, use the rpm command-line tool with the RPM package that you downloaded. You must be logged in as root (Superuser) to complete the installation:
root-shell> rpm --install couchbase-server version.rpm
Where version is the version number of the downloaded package. Once the rpm command has been executed, the Couchbase Server starts automatically, and is configured to automatically start during boot under the 2, 3, 4, and 5 runlevels. Refer to the RedHat RPM documentation for more information about installing packages using RPM. Once installation has completed, the installation process will display a message similar to that below:
Starting Couchbase server: [ OK ] You have successfully installed Couchbase Server. Please browse to http://hostname:8091/ to configure your server. Please refer to http://couchbase.com/support for additional resources. Please note that you have to update your firewall configuration to allow connections to the following ports: 11211, 11210, 4369, 8091 and from 21100 to 21199. By using this software you agree to the End User License Agreement. See /opt/couchbase/LICENSE.txt.
Once installed, you can use the RedHat chkconfig command to manage the Couchbase Server service, including checking the current status and creating the links to enable and disable automatic startup. Refer to the RedHat documentation for instructions. To continue installation you must open a web browser and access the web administration interface. See Section 2.4, Setting up Couchbase Server.
Where version is the version number of the downloaded package. Once the rpm command has been executed, the Couchbase Server starts automatically, and is configured to automatically start during boot under the 2, 3, 4, and 5 runlevels. Refer to the Ubuntu documentation for more information about installing packages using the Debian package manager. Once installation has completed, the installation process will display a message similar to that below:
Selecting previously deselected package couchbase-server. (Reading database ... 218698 files and directories currently installed.) Unpacking couchbase-server (from couchbase-server-community_x86_64_beta.deb) ... Setting up couchbase-server (2-0~basestar) ... * Started Couchbase server You have successfully installed Couchbase Server. Please browse to http://tellurium-internal:8091/ to configure your server. Please refer to http://couchbase.com for additional resources. Please note that you have to update your firewall configuration to allow connections to the following ports: 11211, 11210, 4369, 8091 and from 21100 to 21199. By using this software you agree to the End User License Agreement. See /opt/couchbase/LICENSE.txt.
12
Getting Started
Once installed, you can use the service command to manage the Couchbase Server service, including checking the current status. Refer to the Ubuntu documentation for instructions. To continue installation you must open a web browser and access the web administration interface. See Section 2.4, Setting up Couchbase Server.
Click Next to start the installation. You will be prompted with the Installation Location screen. You can change the location where the Couchbase Server application is located. Note that this does not configure the location of where the persis-
13
Getting Started
tent data will be stored, only the location of the application itself. To select the install location, click the Browse button to select the folder. Click Next to continue the installation. Figure 2.2. Windows Installation Location Screen
Configuration has now been completed. You will be prompted to confirm that you want to continue installation. Click Next to confirm the installation and start the installation process.
14
Getting Started
The install will copy over the necessary files to the system. During the installation process, the installer will also check to ensure that the default administration port is not already in use by another application. If the default port is unavailable, the installer will prompt for a different port to be used for administration of the Couchbase Server.
15
Getting Started
Once the installation process has been completed, you will be prompted with the completion screen. This indicates that the installation has been completed and your Couchbase Server is ready to be setup and configured. When you click Finish, the installer will quit and automatically open a web browser with the Couchbase Server setup window.
16
Getting Started
To continue installation you should follow the server setup instructions. See Section 2.4, Setting up Couchbase Server.
You will be prompted with the installation choices as outlined above, but the installation process will not actually be completed. Instead, a file with your option choices will be recorded in the file C:\Windows\setup.iss. To perform an installation using a previously recorded setup file, copy the setup.iss file into the same directory as the installer executable. Run the installer from the command-line, this time using the/s option.
C:\Downloads> couchbase_server_version.exe /s
You can repeat this process on multiple machines by copying the executable package and the setup.iss file to each machine.
17
Getting Started
1. Download the Mac OS X Zip file. 2. Double-click the downloaded Zip installation file to extract the contents. This will create a single file, the Couchbase.app application. 3. Drag and Drop the Couchbase.app to your chosen installation folder, such as the system Applications folder. Once the application has been copied to your chosen location, you can double-click on the application to start it. The application itself has no user interface. Instead, the Couchbase application icon will appear in the menubar on the right-hand side. If there is no active configuration for Couchbase, then the Couchbase Web Console will be opened and you will be asked to complete the Couchbase Server setup process. See Section 2.4, Setting up Couchbase Server for more details. The Couchbase application runs as a background application. Clicking on the menubar gives you a list of operations that can be performed. For more information, see Section 3.2.3, Startup and Shutdown on Mac OS X.
1.7.0 Windows
C:\Program Files\Membase\Server\Config\config.dat
Important If you have multiple version subdirectories in your /etc/opt/membase directory, you must first cleanup the directory so only the last, most recent version remains. 5. Linux Upgrade Process
18
Getting Started
Linux package managers will prevent couchbase-server from being installed when there's already a membase-server installed. Red Hat / CentOS a. Uninstall the existing membase-server package this will keep the user's db data and copies of their configuration. b. Install Couchbase Server 1.8 with special environment variable flags, which force an upgrade. The special env var is INSTALL_UPGRADE_CONFIG_DIR.
INSTALL_UPGRADE_CONFIG_DIR=/opt/membase/var/lib/membase/config rpm -i \ couchbase-server-edition_and_arch_1.8.0.rpm
Ubuntu a. Uninstall the existing membase-server package this will keep the user's db data and copies of their configuration. b. Install Couchbase Server 1.8 with special environment variable flags, which forces an upgrade. The special env var is INSTALL_UPGRADE_CONFIG_DIR
INSTALL_UPGRADE_CONFIG_DIR=/opt/membase/var/lib/membase/config dpkg -i \ couchbase-server-edition_and_arch_1.8.0.deb
Windows Upgrade Process Windows installer will upgrade your current membase server installation to Couchbase Server in the same location where Membase Server was already installed. If you have installed membase server in the default location, C: \Program Files\Membase\Server, Couchbase Server installer will copy the new files to the same location. Once the upgrade process is completed you will see Couchbase Server icon in the Desktop and under Start->Programs replacing membase server. After every node has been upgraded and restarted, and you can follow the steps below to monitor its progress of "warming up" Turn your application back on.
19
Getting Started
INSTALL_DONT_AUTO_UPGRADE When set to '1', the rpm/dpkg scripts will not automatically invoke the cbupgrade script that's included in Couchbase Server 1.8.0, allowing you to manually invoke cbupgrade later. This may be useful in case you need to perform more debugging. This should be used with the INSTALL_DONT_START_SERVER=1 and INSTALL_UPGRADE_CONFIG_DIR= PATH environment variables. Example flag usage for RedHat / CentOS:
INSTALL_DONT_START_SERVER=1 INSTALL_DONT_AUTO_UPGRADE=1 \ INSTALL_UPGRADE_CONFIG_DIR=/opt/membase/var/lib/membase/config \ rpm -i couchbase-server-community_x86_64_1.8.0.rpm
For Ubuntu
INSTALL_DONT_START_SERVER=1 INSTALL_DONT_AUTO_UPGRADE=1 \ INSTALL_UPGRADE_CONFIG_DIR=/opt/membase/var/lib/membase/config \ dpkg -i couchbase-server-community_x86_64_1.8.0.deb
Example output when using flags, first uninstalling the existing Membase Server 1.7.x:
[root@localhost ~]# rpm -e membase-server Stopping membase-server[ OK ] warning: /opt/membase/var/lib/membase/config/config.dat saved as /opt/membase/var /lib/membase/config/config.dat.rpmsave [root@localhost ~]# INSTALL_DONT_START_SERVER=1 INSTALL_DONT_AUTO_UPGRADE=1 INSTALL_UPGRADE_CONFIG_DIR=/opt/membase/var/lib/membase/config rpm -i couchbase-server-community_x86_64_1.8.0r-55-g80f24f2.rpm Upgrading couchbase-server ... /opt/couchbase/bin/cbupgrade -c /opt/membase/var/lib/membase/config -a yes Skipping cbupgrade due to INSTALL_DONT_AUTO_UPGRADE ... Skipping server start due to INSTALL_DONT_START_SERVER ... You have successfully installed Couchbase Server. Please browse to http://localhost.localdomain:8091/ to configure your server. Please refer to http://couchbase.com for additional resources. Please note that you have to update your firewall configuration to allow connections to the following ports: 11211, 11210, 4369, 8091 and from 21100 to 21299. By using this software you agree to the End User License Agreement. See /opt/couchbase/LICENSE.txt. [root@localhost ~]#
The cbupgrade program can be run using the -n flag, which tells cbupgrade to not modify any files, but just describe the changes it would make. For example:
20
Getting Started
[root@localhost ~]# /opt/couchbase/bin/cbupgrade -c /opt/membase/var/lib/membase/config -nDry-run mode: no actual upgrade changes will be made. Upgrading your Couchbase Server to 1.8.0r-55-g80f24f2. The upgrade process might take awhile. Analysing... Previous config.dat file is /opt/membase/var/lib/membase/config/config.dat.rpmsave Target node: ns_1@127.0.0.1 Membase/Couchbase should not be running. Please use: /etc/init.d/couchbase-server stop or: /etc/init.d/membase-server stop Is the Membase/Couchbase server already stopped? [yes|no] yes Database dir: /opt/membase/var/lib/membase/data Is that the expected database directory to upgrade? [yes|no] yes Buckets to upgrade: default Are those the expected buckets to upgrade? [yes|no] yes Checking disk space available for buckets in directory: /opt/membase/var/lib/membase/data Free disk bucket space wanted: 0.0 Free disk bucket space available: 177790963712 Free disk space factor: 2.0 Ok. Analysis complete. Proceed with config & data upgrade steps? [yes|no] yes SKIPPED (dry-run): Copying /opt/membase/var/lib/membase/config/config.dat.rpmsave SKIPPED (dry-run): cp /opt/membase/var/lib/membase/config/config.dat.rpmsave /opt/couchbase/var /lib/couchbase/config/config.dat Ensuring bucket data directories. SKIPPED (dry-run): Ensuring bucket data directory: /opt/membase/var/lib/membase/data/default-data SKIPPED (dry-run): mkdir -p /opt/membase/var/lib/membase/data/default-data SKIPPED (dry-run): Ensuring dbdir owner/group: /opt/membase/var/lib/membase/data SKIPPED (dry-run): chown -R couchbase:couchbase /opt/membase/var/lib/membase/data SKIPPED (dry-run): Ensuring dbdir owner/group: /opt/membase/var/lib/membase/data SKIPPED (dry-run): chown -R couchbase:couchbase /opt/membase/var/lib/membase/data Upgrading buckets. Skipping already converted bucket: /opt/membase/var/lib/membase/data/default-data Skipping already converted bucket: /opt/membase/var/lib/membase/data/test0-data Done.
21
Getting Started
Click the SETUP button to start the setup process. 2. First, you must set the disk storage and cluster configuration.
22
Getting Started
Configure Disk Storage The Configure Disk Storage option specifies the location of the persistent (on-disk) storage used by Couchbase Server. The setting affects only this server and sets the directory where all the data will be stored on disk. Join Cluster/Start New Cluster The Configure Server Memory section sets the amount of physical RAM that will be allocated by Couchbase Server for storage.
23
Getting Started
If you are creating a new cluster, you specify the memory that will be allocated on each node within your Couchbase cluster. You must specify a value that will be supported on all the nodes in your cluster as this is a global setting. If you want to join an existing cluster, select the radio button. This will change the display and prompt the IP address of an existing node, and the username and password of an administrator with rights to access the cluster. Figure 2.8. Couchbase Server Setup Step 1 (Existing Cluster)
Click Next to continue the installation process. 3. You must specify the default bucket for this server.
24
Getting Started
The options are: Bucket Type Specifies the type of the bucket to be created, either Memcached or Couchbase. See Section 1.3.4, Buckets for more information. The remainder of the options differ based on your selection. When selecting the Couchbase bucket type: Memory Size This option specifies the amount of available RAM configured on this server which should be allocated to the default bucket. Replication For Couchbase buckets you can enable replication to support multiple replicas of the default bucket across the servers within the cluster. You can configure up to three replicas. Each replica receives copies of all the documents that are managed by the bucket. If the host machine for a bucket fails, a replica can be promoted to take its place, providing continuous (high-availability) cluster operations in spite of machine failure.
25
Getting Started
You can disable replication by setting the number of replica copies to zero (0). This will configure the default bucket as local-only and therefore only accessible on this machine. When selecting the Memcached bucket type: Memory Size The bucket is configured with a per-node amount of memory. Total bucket memory will change as nodes are added/removed. For more information, see Section 1.3.3, Memory Quotas. Click Next to continue the setup process. 4. You can optionally enable the notification system within the Couchbase Web Console. Figure 2.10. Couchbase Server Setup Step 3
If you select the Update Notifications option, the Web Console will communicate with Couchbase servers to confirm the version number of your Couchbase installation. During this process, the client submits the following information to the Couchbase Server:
26
Getting Started
The current version of your Couchbase Server installation. When a new version of Couchbase Server becomes available, you will be provided with notification of the new version and information on where you can download the new version. Basic information about the size and configuration of your Couchbase cluster. This information will be used to help us prioritize our development efforts. Note The process occurs within the browser accessing the web console, not within the server itself, and no further configuration or internet access is required on the server to enable this functionality. Providing the client accessing the Couchbase Server console has internet access, the information can be communicated to the Couchbase Servers. The update notification process the information anonymously, and the data cannot be tracked. The information is only used to provide you with update notification and to provide information that will help us improve the future development process for Couchbase Server and related products. Enterprise Edition You can also register your product from within the setup process. On Enterprise Editions of Couchbase Server you can optionally select that Couchbase, Inc. plants a tree as a thank you for completing the registration process. For more information on the tree planting process, see Mokugift where you can find out about the tree planting sponsership programme, supported by the United Nations Encironment Programme. To have your tree planted, please fill in your email address, name and company details, and tick the Yes, please plant my tree! checkbox. Community Edition Supplying your email address will add you to the Couchbase community mailing list, which will provide you with news and update information about Couchbase and related products. You can unsubscribe from the mailing list at any time using the unsubscribe link provided in each email communication. Click Next to continue the setup process. 5. The final step in the setup process is to configure the username and password for the administrator of the server. If you are create a new cluster then this information will be used to authenticate each new server into the cluster. The same credentials are also used when using the Couchbase Management REST API. Enter a username and password. The password must be at least six characters in length.
27
Getting Started
Click Next to continue the complete the process. Once the setup process has been completed, you will be presented with the Couchbase Web Console showing the Cluster Overview page.
28
Getting Started
29
Getting Started
Make sure it's responding (stats is a great way to check basic health):
stats STAT delete_misses 0 STAT ep_io_num_write 0 STAT rejected_conns 0 ... STAT time 1286678223 ... STAT curr_items_tot 0 ... STAT threads 4 STAT pid 23871 ... END
Disconnect:
quit Connection closed by foreign host. shell>
30
Windows:
C:\Program Files\Couchbase\Server\data\ns_1
This path can be changed for each node at setup either via the Web UI setup wizard, using theREST API or using the Couchbase CLI: Linux:
/opt/couchbase/bin/membase node-init -c <node_IP>:8091 --node-init-data-path=<new_path> -u <user> -p <password>
Windows:
C:\Program Files\Couchbase\Server\bin\membase node-init -c <node_IP>:8091 --node-init-data-path=<new_path> -u <user> -p <password>
Once a node or cluster has already been setup and storing data, you can still change the path but it is not an "online" operation: 1. Change the path on a running node either via the REST API or using the Couchbase CLI (commands above). This change won't actually take effect until the node is restarted. 31
Administration Basics
2. Shut the service down 3. Copy all the data files from their original location into the new location 4. Start the service again and monitor the "warmup" of the data.
Start and Stop scripts are also provided in the standard Couchbase Server installation in the bin directory. To start the server using this script:
shell> C:\Program Files\Couchbase\Server\bin\service_start.bat
32
Administration Basics
The individual menu options perform the following actions: About Couchbase Opens a standard About dialog containing the licensing and version information for the Couchbase Server installed. Open Admin Console Opens the Web Administration Console in your configured default browser. Visit Support Forum Opens the Couchbase Server support forum within your default browser at the Couchbase website where you can ask questions to other users and Couchbase developers. Check for Updates Checks for updated versions of Couchbase Server. This checks the currently installed version against the latest version available at Couchbase and offers to download and install the new version. If a new version is available, you will be presented with a dialog containing information about the new release. If a new version is available, you can choose to skip the update, notify the existence of the update at a later date, or to automatically update the software to the new version. If you choose the last option, the latest available version of Couchbase Server will be downloaded to your machine, and you will be prompted to allow the installation to take place. Installation will shut down your existing Couchbase Server process, install the update, and then restart the service once the installation has been completed. Once the installation has been completed you will be asked whether you want to automatically update Couchbase Server in the future. Note Using the update service also sends anonymous usage data to Couchbase on the current version and cluster used in your organization. This information is used to improve our service offerings. You can also enable automated updates by selecting the Automatically download and install updates in the future checkbox.
33
Administration Basics
Launch Admin Console at Start If this menu item is checked, then the Web Console for administrating Couchbase Server will be opened whenever the Couchbase Server is started. Selecting the menu item will toggle the selection. Automatically Start at Login If this menu item is checked, then Couchbase Server will be automatically started when the Mac OS X machine starts. Selecting the menu item will toggle the selection. Quit Couchbase Selecting this menu option will shut down your running Couchbase Server, and close the menubar interface. To restart, you must open the Couchbase Server application from the installation folder.
34
35
Best Practices
Disk throughput and sizing Network bandwidth Data distribution and safety Each of these factors can be the determining factor for sizing, although due to the in-memory nature of Couchbase Server, RAM is normally the most important factor. How you choose your primary factor will depend on the data set and information that you are storing: If you have a very small data set that gets a very high load, you'll need to size more off of network bandwidth than RAM. If you have a very high write rate, you'll need more nodes to support the disk throughput of persisting all that data (and likely more RAM to buffer the incoming writes). Even with a very small dataset, under low load, you may still want 3 nodes for proper distribution and safety. With Couchbase Server, you can increase the capacity of your cluster (RAM, Disk, CPU or network) by increasing the number of nodes within your cluster, since each limit will be increased linearly as the cluster size is increased.
36
Best Practices
Description The percentage of your data you want in memory. How much RAM can be assigned to Couchbase
The following are the items that are used in calculating memory required and are assumed to be constants. Table 4.2. Constants Constant Meta data per document (metadata_per_document) SSD or Spinning headroom
a
Description This is the space that Couchbase needs to keep metadata per document. It is 120 bytes. All the documents and their metadata need to live in memory at all times SSDs give better I/O performance. Typically 25% (0.25) for SSD and 30% (0.30) for spinning (traditional) hard disks as SSD are faster than spinning disks. By default it is set at 70% of memory allocated to the node
The headroom is the additional overhead required by the cluster to store metadata about the information being stored. This requires approximately 25-30% more space than the raw RAM requirements for your dataset.
This is a rough guideline to size your cluster: Variable no_of_copies total_metadata total_dataset working_set Cluster RAM quota required number of nodes
a
Calculation 1 + number_of_replicas
a
(documents_num) * (metadata_per_document + ID_size) * (no_of_copies) (documentss_num) * (value_size) * (no_of_copies) total_dataset * (working_set_percentage) (total_metadata + working_set) * (1 + headroom) / (high_water_mark) Cluster RAM quota required / per_node_ram_quota
Note You will need at least the number of replicas + 1 nodes irrespective of your data size. Example sizing calculation Table 4.3. Input Variables Input Variable documentss_num ID_size value_size number_of_replicas working_set_percentage Table 4.4. Constants Constants Type of Storage value SSD value 1000,000 100 10000 1 20%
37
Best Practices
Constants overhead_percentage metadata_per_document high_water_mark Table 4.5. Variable Calculations Variable no_of_copies total_metadata total_dataset working_set Cluster RAM quota required
a
Calculation =2a = 100,0000 * (100 + 120) * (2) = 440,000,000 = 100,0000 * (10000) * (2) = 20,000,000,000 = 20,000,000,000 * (0.2) = 4,000,000,000 = (440,000,000 + 4000,000,000) * (1+0.25)/(0.7) = 7928,000,000
For example, if you have 8GB machines and you want to use 6 GB for Couchbase:
number of nodes = Cluster RAM quota required/per_node_ram_quota = 7.9 GB/6GB = 1.3 or 2 nodes
Note RAM quota You will not be able to allocate all your machine RAM to the per_node_ram_quota as there maybe other programs running on your machine.
38
Best Practices
In general you can calculate your network bandwidth requirements using the formula:
Bandwidth = (operations per second * item size) + overhead for rebalancing
4.2.4.2. Replication
Couchbase Server allows you to configure up to 3 replicas (creating 4 copies of the dataset). In the event of a failure, you can only "failover" (either manually or automatically) as many nodes as you have replicas. For example: In a 5 node cluster with one replica, if one node goes down, you can fail it over. If a second node goes down, you no longer have enough replica copies to fail over to and will have to go through a slower process to recover. In a 5 node cluster with 2 replicas, if one node goes down, you can fail it over. If a second node goes down, you can fail it over as well. Should a 3rd one go down, you now no longer have replicas to fail over. Note After a node goes down and is failed over, you should attempt to replace that node as soon as possible and rebalance. The rebalance is what will recreate the replica copies (if you still have enough nodes to do so). As a rule of thumb, it is recommended that you configure: 1 replica for up to 5 nodes 1 or 2 replicas for 5 to 10 nodes 1, 2 or 3 replicas for over 10 nodes While there many be variations to this, there are definitely diminishing returns from having more replicas in smaller clusters.
39
Best Practices
Best Practices
Some firewall or proxy software will drop TCP connections which are idle for a certain amount of time (e.g., 20 minutes). If the software does not allow changing that timeout, send a command from the client periodically to keep the connection alive.
41
Best Practices
It is a best practice to continue to monitor the system until you are confident that replication has completed. There are essentially two stages to replication: Backfilling - This is the first stage of replication and involves reading all data for a given active vBucket and sending it to the server that is responsible for the replica. This can put increased load on the disk subsystem as well as network bandwidth but is not designed to impact any client activity. You can monitor the progress of this task by watching for ongoing TAP disk fetches and/or watching 'stats tap': '/opt/couchbase/bin/ep_engine/management/stats <couchbase_node>:11210 tap | grep backfill' will return a list of TAP backfill processes and whether they are still running (true) or done (false). When all have completed, you should see the Total Item count ( curr_items_tot _as opposed to the active item count, _curr_items) be equal to the number of active items times the replica count. If you are continuously adding data to the system, these values may not line up exactly at a given instant in time, but it should be clear from an order-of-magnitude sense whether your items are properly replicated. Until this is completed, you should avoid using the "failover" functionality since that may result in loss of the data that has not been replicated yet. Draining - After the backfill process is completed, all nodes that had replicas materialized on them will need to also persist those items to disk. It is important to continue monitoring the disk write queue and memory usage of
42
Best Practices
We've also authored an AMI for use within EC2 independent of RightScale. When using these, you will have to handle the specific complexities yourself. You can find this AMI by searching for 'couchbase' in Amazon's EC2 portal. Some considerations to take into account when deploying within the cloud are: Local storage being ephemeral IP addresses of a server changing from runtime to runtime Security groups/firewall settings Swap Space
4.6.2. IP addresses
The second issue is a bit trickier and requires configuring Couchbase to use a DNS entry instead of an IP address. By default, Couchbase Servers use their IP address as a unique identifier. If the IP changes, an individual node will not be able to identify its own configuration and other nodes that it was clustered to will not be able to access it. In order for a node to identify itself via a DNS name rather than and IP address, the following instructions must be followed. Note that this configuration is automatically handled by the RightScale server template. A few points to keep in mind when setting this up: Make sure that this hostname always resolves to the IP address of the node that it is on. This can be accomplished by using a dynamic DNS service such as DNSMadeEasy which will allow you to automatically update the hostname when an underlying IP address changes. It is best to make sure that the IP address registered with the hostname is the internal address for the node (rather than the external one provided by Amazon) so that other nodes and application machines can contact it Warning The below steps will completely wipe any data and configuration from the node, so it is best to start with a fresh Couchbase install. If you already have a running cluster, you can easily rebalance a node out of the cluster, make the change and then rebalance it back into the cluster. Nodes with IP's and hostnames can exist in the same cluster. For Linux: 1. Install the Couchbase software 2. Execute:
sudo /etc/init.d/couchbase-server stop
3. Edit the script located at /opt/couchbase/bin/couchbase-server Add a '\' to the end of the last line
43
Best Practices
Add a new last line that reads:-name ns_1@<hostname>. Where hostname is either a DNS name or an IP address that you want this server to listen on (the 'ns_1@' prefix is mandatory). For example:
-detached \ -run ns_bootstrap -- \ -ns_server config_path "\"/opt/couchbase/etc/couchbase/static_config\"" \ -name ns_1@couchbase1.company.com
6. See the node correctly identifying itself as the hostname in the GUI under the Manage Servers page (you will be taken back to the setup wizard since the configuration was cleared out, but after completing the wizard the node will be named properly). For Windows: 1. Install the Couchbase Server software 2. Stop the service by running:
C:\Program Files\Couchbase\Server\bin\service_stop.bat
4. Edit the script located at C:\Program Files\Couchbase\Server\bin\service_register.bat: On the 7th line it says:set NS_NAME=ns_1@%IP_ADDR% Replace%IP_ADDR%with the hostname/IP address that you want to use/ 5. Register the service by running the modified script: C:\Program Files\Couchbase\Server\bin\service_register.bat 6. Delete the files located under:C:\Program Files\Couchbase \Server\var\lib\couchbase\mnesia. 7. Start the service by running:
C:\Program Files\Couchbase\Server\bin\service_start.bat
8. See the node correctly identifying itself as the hostname in the GUI under the Manage Servers page (you will be taken back to the setup wizard since the configuration was cleared out, but after completing the wizard the node will be named properly).
44
Best Practices
45
Best Practices
The memcached OTC client is configured to have just one server in its server list (localhost), so all operations are forwarded to localhost:11211 a port serviced by the proxy. The proxy hashes the document ID to a vBucket, looks up the host server in the vBucket table, and then sends the operation to the appropriate couchbase server on port 11210. Figure 4.2. Deployment Strategy: Standalone Proxy
46
47
Administration Tasks
48
Administration Tasks
nents that are outside the scope of Couchbase visibility. For example, by observing that a network switch is flaking and that there is a dependency on that switch by the Couchbase cluster, the management system may determine that failing the Couchbase nodes will not help the situation. If, however, everything around Couchbase and across the various nodes is healthy and that it does indeed look like a single node problem, and that the aggregate traffic can support loading the remaining nodes with all traffic, then the management system may fail the system over. Couchbase fully supports this model through its REST interface.
The files are stored by default in /var/opt/couchbase/<version>/data/ns_1. This path can be changed on a per-node basis when initially setting up the node: Chapter 2, Getting Started When backing up or restoring a bucket, all files must be included. Warning Make sure that any restoration of files also sets the proper ownership of those files to the couchbase user
5.2.1. Backup
To take a backup of a running Couchbase node (note, this needs to be performed on all buckets and on all nodes of a cluster to take a backup of that cluster): Use the mbbackup script to backup data:
mbbackup [bucket_path_name] [dest_dir_path].
Make sure that there is enough disk space to take the backup. The mbbackup script will also perform a vacuum of the database files to defragment them which provides faster startup times. Depending on the amount of data, this script can take an extended amount of time to run. It is a best practice to make sure that your connection to the server running the script is not broken. Linux
shell> mbbackup /var/opt/couchbase/1.6.4/data/ns_1/default /backups/2010-12-22/
Backup the configuration file located at: /etc/opt/couchbase/<version>/ns_1/config.dat Windows Open a command line. Click on Start ->All Programs-> Accessories-> Command Prompt. Start powershell by typing powershell and press Enter key. Type:
shell> set-executionpolicy remotesigned
49
Administration Tasks
50
Administration Tasks
3. Place the copies of the original files into the data directory on all the new nodes. Warning You do not have to "match" up the nodes one-for-one. However, ensure that each set of original data files gets placed onto one and only one node of the new cluster Note Please ensure that you retain file ownership properties for those files which you placed on the destination node. 4. Start couchbase-server on the new nodes 5. Create a bucket with the same name and SASL configuration on the new nodes. 6. After the bucket creation, each node will start loading items from the data files into memory. 7. The cluster will be in a balanced state after warm up. 8. Do not start a rebalance process while nodes are still warming up. 9. If any nodes go down during the warmup, it is a best practice to restart all nodes together.
Depending on the amount of data, this script can take an extended amount of time to run. It is a best practice to make sure that your connection to the server running the script is not broken, or that you are using something to let the script run in the background (i.e. screen) Linux:
/opt/couchbase/bin/mbrestore -a default default-0.mb default-1.mb default-2.mb default-3.mb
51
Administration Tasks
In order to correctly restore you must put all of the database backup file names as command arguments Windows:
cd "C:/Program Files/Couchbase/Server/bin/" .\mbrestore -a "C:/backup/2010-12-22/default" "C:/backup/2010-12-22/default-0.mb" \ "C:/backup/2010-12-22/default-1.mb" "C:/backup/2010-12-22/default-2.mb" "C:/backup/2010-12-22/default-3.mb"
In order to correctly restore you must put all of the database backup file names as command arguments.
52
Administration Tasks
It is recommended that you remove a node rather than fail it over. When a node fails and is not coming back to the cluster, the failover functionality will promote its replica vBuckets become active immediately. If a healthy node is failed over, there might be some data loss for the replication data that was in flight during that operation. Using the remove functionality will ensure that all data is properly replicated and continuously available.
5.3.3. Rebalancing
Once you decide to add or remove nodes to your Couchbase cluster, there are a few things to take into consideration: If you're planning on adding and/or removing multiple nodes in a short period of time, it is best to add them all at once and then kick-off the rebalancing operation rather than rebalance after each addition. This will reduce the overall load placed on the system as well as the amount of data that needs to be moved. Choose a quiet time for adding nodes. While the rebalancing operation is meant to be performed online, it is not a "free" operation and will undoubtedly put increased load on the system as a whole in the form of disk IO, network bandwidth, CPU resources and RAM usage. It is our recommended best practice to do any "voluntary" (i.e., not to resolve a failure) rebalancing operation during a period of low usage of the system. Obviously with today's 24x7 web applications there may not be a period of complete inactivity but it is up to the administrator to understand the impact that a rebalancing operation may have on the cluster and application. Memory required for rebalancing. The rebalancing operation requires moving large amounts of data around the cluster. The more RAM that is available will allow the operating system to cache more disk access which will allow it to perform the rebalancing operation much faster. If there is not enough memory in your cluster the rebalancing maybe quite slow. It is recommended that you don't wait for your cluster to reach full capacity before adding capacity and rebalancing.
53
54
6.1.1. Cluster
The Cluster section provides information on the RAM and disk usage information for your cluster. For the RAM information you are provided with a graphical representation of your RAM situation, including: Total in Cluster Total RAM configured within the cluster. This is the total amount of memory configured for all the servers within the cluster. Total Allocated The amount of RAM allocated to data buckets within your cluster.
55
Unallocated The amount of RAM not allocated to data buckets within your cluster. In Use The amount of memory across all buckets that is actually in use (i.e. data is actively being stored). Unused The amount of memory that is unused (available) for storing data. The Disk Overview section provides similar summary information for disk storage space across your cluster. Total Cluster Storage Total amount of disk storage available across your entire cluster for storing data. Usable Free Space The amount of usable space for storing information on disk. This figure shows the amount of space available on the configured path after non-Couchbase files have been taken into account. Other Data The quantity of disk space in use by data other than Couchbase information. In Use The amount of disk space being used to actively store information on disk. Free The free space available for storing objects on disk.
6.1.2. Buckets
The Buckets section provides two graphs showing the Operations per second and Disk fetches per second. The Operations per second provides information on the level of activity on the cluster in terms of storing or retrieving objects from the data store. The Disk fetches per second indicates how frequently Couchbase is having to go to disk to retrieve information instead of using the information stored in RAM.
6.1.3. Servers
The Servers section indicates overall server information for the cluster: Active Servers is the number of active servers within the current cluster configuration. Servers Failed Over is the number of servers that have failed over due to an issue that should be investigated. Servers Down shows the number of servers that are down and not-contactable. Servers Pending Rebalance shows the number of servers that are currently waiting to be rebalanced after joining a cluster or being reactivated after failover. 56
For each bucket, the following information is provided in each column: Bucket type is indicated by the displayed icon. A couch icon indicates a Membase bucket. The M icon indicates a memcached bucket. Bucket name is the given name for the bucket. Clicking on the bucket name takes you to the individual bucket statistics page. For more information, see Section 6.3.3, Individual Bucket Monitoring. RAM Usage/Quota shows the amount of RAM used (for active objects) against the configure bucket size. Disk Usage shows the amount of disk space in use for active object data storage. Item Count indicates the number of objects stored in the bucket. Ops/sec shows the number of operations per second for this data bucket. Disk Fetches/sec shows the number of operations required to fetch items from disk. Clicking the Information button opens the basic bucket information summary. For more information, see Section 6.3.2, Bucket Information. To create a new data bucket, click the Create New Data Bucket. See Section 6.3.1, Creating and Editing Data Buckets for details on creating new data buckets.
58
Bucket Name The bucket name. The bucket name can only contain characters in range A-Z, a-z, 0-9 as well as underscore, period, dash and percent symbols. Bucket Type Specifies the type of the bucket to be created, either Memcached or Membase. Access Control The access control configures the port your clients will use to communicate with the data bucket, and whether the bucket requires a password.
59
To use the TCP standard port (11211), the first bucket you create can use this port without requiring SASL authentication. For each subsequent bucket, you must specify the password to be used for SASL authentication, and client communication must be made using the binary protocol. To use a dedicated port, select the dedicate port radio button and enter the port number you want to use. Using a dedicated port supports both the text and binary client protocols, and does not require authentication. Memory Size This option specifies the amount of available RAM configured on this server which should be allocated to the default bucket. Note that the allocation is the amount of memory that will be allocated for this bucket on each node, not the total size of the bucket across all nodes. Replication For Membase buckets you can enable replication to support multiple replicas of the default bucket across the servers within the cluster. You can configure up to three replicas. Each replica receives copies of all the documents that are managed by the bucket. If the host machine for a bucket fails, a replica can be promoted to take its place, providing continuous (high-availability) cluster operations in spite of machine failure. You can disable replication by setting the number of replica copies to zero (0). This will configure the default bucket as local-only and therefore only accessible on this machine. Once you selected the options for the new bucket, you can click Create button to create and activate the button within your cluster. You can cancel the bucket creation using the Cancel button.
60
You can edit the bucket information by clicking the Edit button within the bucket information display.
61
The summary section provides a quick overview of the cluster activity. For more information, see Section 6.3.3.1, Bucket Monitoring Summary Statistics. vBucket Resources This section provides detailed information on the vBucket resources across the cluster, including the active, replica and pending operations. For more information, see Section 6.3.3.2, Bucket Monitoring vBucket Resources. Disk Queues Disk queues show the activity on the backend disk storage used for persistence within a data bucket. The information displayed shows the active, replica and pending activity. For more information, see Section 6.3.3.3, Bucket Monitoring Disk Queues. TAP Queues The TAP queues section provides information on the activity within the TAP queues across replication, rebalancing and client activity. For more information, see Section 6.3.3.4, Bucket Monitoring TAP Queues. Top Keys This shows a list of the top 10 most actively used keys within the selected data bucket. For Memcached bucket types, the Memcached static summary is provided. See Section 6.3.3.5, Bucket Monitoring Memcached Buckets.
The following graph types are available: ops per second The total number of operations per second on this bucket. cache miss %
62
Percentage of reads per second to this bucket which required a read from disk rather than RAM. creates per second Number of new items created in this bucket per second. updates per second Number of existing items updated in this bucket per second. disk reads Number of reads per second from disk for this bucket. back-offs per second Number of 'back-offs' sent per second to clients due to out of memory situations for this bucket.
63
The Active column displays the information for vBuckets within the Active state. The Replica column displays the statistics for vBuckets within the Replica state (i.e. currently being replicated). The Pending columns shows statistics for vBuckets in the Pending state, i.e. while data is being exchanged during rebalancing. These states are shared across all the following statistics. For example, the graph news items per sec within the Active state column displays the number of new items per second created within the vBuckets that are in the active state. The individual statistics, one for each state, shown are: active vBuckets The number of vBuckets within the specified state. active items Number of items within the vBucket of the specified state. % resident items Percentage of items within the vBuckets of the specified state that are resident (in RAM). new items per second Number of new items created in vBuckets within the specified state. Note that new items per second is not valid for the Pending state. ejections per second Number of items ejected per second within the vBuckets of the specified state. user data in RAM Size of user data within vBuckets of the specified state that are resident in RAM. metadata in RAM Size of item metadata within the vBuckets of the specified state that are resident in RAM.
64
The Active column displays the information for the Disk Queues within the Active state. The Replica column displays the statistics for the Disk Queues within the Replica state (i.e. currently being replicated). The Pending columns shows statistics for the disk Queues in the Pending state, i.e. while data is being exchanged during rebalancing. These states are shared across all the following statistics. For example, the graph fill rate within the Replica state column displays the number of items being put into the replica disk queue for the selected bucket. The displayed statistics are: items The number of items waiting to be written to disk for this bucket for this state. fill rate The number of items per second being added to the disk queue for the corresponding state. drain rate Number of items actually written to disk from the disk queue for the corresponding state. average age The average age of items (in seconds) within the disk queue for the specified state.
65
The statistics in this section are detailed below: # tap senders Number of TAP queues in this bucket for internal (replica), rebalancing or client connections. # items Number of items in the corresponding TAP queue for this bucket. fill rate Number of items per second being put into the corresponding TAP queue for this bucket. drain rate Number of items per second being sent over the corresponding TAP queue connections to this bucket. back-off rate Number of back-offs per second received when sending data through the corresponding TAP connection to this bucket. # backfill remaining Number of items in the backfill queue for the corresponding TAP connection for this bucket. # remaining on disk Number of items still on disk that need to be loaded to service the TAP connection to this bucket.
66
The Memcached statistics are: Operations per sec. Total operations per second serviced by this bucket Hit Ratio % Percentage of get requests served with data from this bucket Memory bytes used Total amount of RAM used by this bucket Items count Number of items stored in this bucket RAM evictions per sec. Number of items per second evicted from this bucket Sets per sec. Number of set operations serviced by this bucket Gets per sec. Number of get operations serviced by this bucket 67
Net. bytes TX per sec Number of bytes per second sent from this bucket Net. bytes RX per sec. Number of bytes per second sent into this bucket Get hits per sec. Number of get operations per second for data that this bucket contains Delete hits per sec. Number of delete operations per second for data that this bucket contains Incr hits per sec. Number of increment operations per second for data that this bucket contains Decr hits per sec. Number of decrement operations per second for data that this bucket contains Delete misses per sec. Number of delete operations per second for data that this bucket does not contain Decr misses per sec. Number of decr operations per second for data that this bucket does not contain Get Misses per sec. Number of get operations per second for data that this bucket does not contain Incr misses per sec. Number of increment operations per second for data that this bucket does not contain CAS hits per sec. Number of CAS operations per second for data that this bucket contains CAS badval per sec. Number of CAS operations per second using an incorrect CAS ID for data that this bucket contains CAS misses per sec. Number of CAS operations per second for data that this bucket does not contain
The Server Nodes monitoring overview shows summary data for the Swap Usage, RAM Usage, CPU Usage and Active Items across all the nodes in your cluster. Figure 6.10. Couchbase Web Console - Server Nodes
Clicking the server name provides server node specific information, including the IP address, OS, Couchbase version and Memory and Disk allocation information. Selecting a server from the list shows a server-specific version of the Bucket Monitoring overview, showing a combination of the server-specific performance information, and the overall statistic information for the bucket across all nodes.
69
The graphs specific to the server are: swap usage Amount of swap space in use on this server. free memory
70
Amount of RAM available on this server. CPU utilization Percentage of CPU utilized across all cores on the selected server. connection count Number of connects to this server of all types for client, proxy, TAP requests and internal statistics. You can select an individual bucket and server to view a statistic for using the popup selections for the server and bucket, and clicking on the mini-graph for a given statistic. Figure 6.12. Couchbase Web Console - Server specific view
For more information on the data bucket statistics, see Section 6.3, Data Buckets.
71
Host is down, not replicating data between nodes and not servicing requests from clients. Figure 6.13. Down Status
Pend Host is up and currently filling RAM with data, but is not servicing requests from clients. Client access will be supported once the RAM has been pre-filled with information. Figure 6.14. Pend Status
You can monitor the current server status using both the Manage: Server Nodes and Monitor: Server Nodes screens within the Web Console.
6.6. Settings
The Settings interface sets the global settings for your Couchbase Server instance.
72
For more information on how Update Notifications work, see Section 6.8, Update Notifications.
6.6.3. Alerts
You can enable email alerts to be raised when a significant error occurs on your Couchbase Server cluster. The email alert system works by sending email directly to a configured SMTP server. Each alert email is send to the list of configured email recipients. The available settings are: Enable email alerts
73
If checked, email alerts will be raised on the specific error enabled within the Available Alerts section of the configuration. Host The hostname for the SMTP server that will be used to send the email. Port The TCP/IP port to be used to communicate with the SMTP server. The default is the standard SMTP port 25. Username For email servers that require a username and password to send email, the username for authentication. Password For email servers that require a username and password to send email, the password for authentication. Sender email The email address from which the email will be identified as being sent from. This email address should be one that is valid as a sender address for the SMTP server that you specify. Recipients A list of the recipients of each alert message. You can specify more than one recipient by separating each address by a space, comma or semicolon. Available alerts You can enable individual alert messages that can be sent by using the series of checkboxes. The supported alerts are: Node was auto-failovered The sending node has been auto-failovered. Maximum number of auto-failovered nodes was reached The auto-failover system will stop auto-failover when the maximum number of spare nodes available has been reached. Node wasn't auto-failovered as other nodes are down at the same time Auto-failover does not take place if there are no spare nodes within the current cluster. Node wasn't auto-failovered as the cluster was too small (less than 3 nodes) You cannot support auto-failover with less than 3 nodes.
74
6.7. Log
The Log section of the website allows you to view the built-in event log for Couchbase Server so that you can identify activity and errors within your Couchbase cluster.
75
The Couchbase Web Console displays the following information about each logged event. Name Event Module Server Node Time Description Event that triggered the alert. Code Software module in which the event occurred and an internal alert code associated with the event. Server Node that raised the log entry. Time and date when the event occurred.
76
Generate Diagnostic Report (http://hostname:8091/diag). The Couchbase Web Console opens a new browser window and downloads the text of the diagnostic report.
6.8.1. Notifications
If an update notification is available, the counter within the button display within the Couchbase Console will be displayed with the number of available updates.
77
If the IP address of a Couchbase Server in your cluster changes, you will be warned that the address is no longer available. You should check the IP address on the server, and update your clients or server configuration. OOM (Hard) Indicates if the bucket memory on a node is entirely used for metadata. OOM (Temporary) Indicates when a bucket on a node is temporarily unable to accept new items and is now waiting for additional items to be ejected to disk to free up active memory. Commit Failure Indicates that writing data to disk for a specific bucket has failed. Metadata Overhead Indicates that a bucket is now using more than 50% of the allocated RAM for storing metadata and keys, reducing the amount of RAM available for data values. Disk Usage Indicates that the available disk space used for persistent storage has reached at least 90% of capacity.
78
This tool provides access to various management operations for Couchbase Server clusters, nodes and buckets. The basic usage format is:
couchbase-cli COMMAND CLUSTER [OPTIONS]
Where: COMMAND is a command from Table 7.2, couchbase Tool Commands CLUSTER is a cluster specification. You can use either:
--cluster=HOST[:PORT]
Table 7.2. couchbase Tool Commands Command server-list server-info server-add server-readd rebalance rebalance-stop rebalance-status failover cluster-init node-init bucket-list bucket-create bucket-edit bucket-delete bucket-flush help Table 7.3. Standard couchbase Options Option --user=USERNAME,-u USERNAME --password=PASSWORD,-p PASSWORD --output=KIND,-o KIND Command Description Administrator username for accessing the cluster Administrator password for accessing the cluster Output kind, either json for JSON format, or standard for the native format for the command. Output debug information. server-add,server-readd, re- Server to be added balance server-add,server-readd, re- Admin username for the server to be balance added server-add,server-readd, re- Admin password for the server to be balance added rebalance failover cluster-init cluster-init The server to be removed Server to failover New admin username New admin password Description List all servers in a cluster Show details on one server Add one or more servers to the cluster Re-add a server that was failed over Start a cluster rebalancing Stop current cluster rebalancing Show status of current cluster rebalancing Failover one or more servers Set the username,password and port of the cluster Set node specific parameters List all buckets in a cluster Add a new bucket to the cluster Modify an existing bucket Delete an existing bucket Flush a given bucket Show longer usage/help and examples
80
Option --cluster-init-port=PORT
Command cluster-init
Description New cluster port (must be at least 6 characters) New RAM quota Bucket to act on Memcached or Memhbase Supports ASCII protocol and is authless Standard port, exclusive with bucket-port RAM quota in MB Replication count
bucket*
Server information
shell> couchbase-cli server-info -c 192.168.0.1:8091
81
Delete a bucket:
shell> couchbase-cli bucket-delete -c 192.168.0.1:8091 --bucket=test_bucket
A tool for obtaining the couchbase node statistics. The general format for usage is:
shell> cbstats HOST COMMAND [options] [username password]
Where HOST is the hostname and port (HOSTNAME[:PORT]) combination for a Couchbase bucket, and username and password are the authentication details. COMMAND(and[options]) are one of:
all dispatcher hash reset tap timings vkey keyname vbid
Example: The stats output can be used with other command-line tools to sort and filter the data.
watch --diff "cbstats \ ip-10-12-19-81:11210 all | egrep 'item|mem|flusher|ep_queue|bg|eje|resi|warm'"
Usage:
82
Available params: min_data_age - minimum data age before flushing data queue_age_cap - maximum queue age before flushing data max_txn_size - maximum number of items in a flusher transaction bg_fetch_delay - delay before executing a bg fetch (test feature) max_size - max memory used by the server mem_high_wat - high water mark mem_low_wat - low water mark
Usage:
/opt/couchbase/bin/ep_engine/management/tap.py host:port
Usage:
mbvbucketctl mbvbucketctl mbvbucketctl mbvbucketctl Usage: host:port list [username password] or host:port rm vbid [username password] or host:port set vbid vbstate [username password]
The cbvbuckettool expects a vBucketServerMap JSON mapfile, and will print the vBucketId and servers each key should live on. You may use '-' instead for the filename to specify stdin. Examples:
shell> cbvbuckettool file.json some_key another_key
Or
shell> curl http://host:8091/pools/default/buckets/default | cbvbuckettool - some_key another_key
83
Usage:
cbdbmaint [--vacuum] [--backupto=<dest_dir>] [--port=11210]
/opt/couchbase/bin/cbcollect_info
84
8.1.1. Resources
There are a number of different resources within the Couchbase Server and these resources can optionally have one or more controllers, which have RESTful endpoints in the representation of the item one would control. Pools/Clusters A pool (or cluster) is a collection of physical resources that are grouped together and provide services and a management interface. A single, default pool exists for every deployment of Couchbase Server. A Node (server node) is a member of a pool. Couchbase Server collects run-time statistics for pools, maintaining an overall pool-level data view of counters and periodic metrics of the overall system. The Couchbase Management REST API can be used to retrieve historic statistics for pools. Server Nodes A Server Node is a physical or virtual machine running Couchbase Server as a member of a pool.
85
Data Buckets A data bucket is a logical grouping of resources within a pool. In support of pool and resource management, a bucket provides unconstrained namespaces to users to define whatever bucket name makes sense to the user. It also allows the same key in the same application to be available in multiple places. Couchbase Server collects run-time statistics for buckets, maintaining an overall bucket-level data view of counters and periodic metrics of the overall system. Buckets are categorized by storage type. Buckets can implement different kinds of behaviors, such as cache only or persisted (either synchronous or asynchronous) document stores.
Comma-delimited list of me- Indicates to the server what media type(s) this dia types or media type pat- client is prepared to accept. terns. Basic plus username and password (per RFC 2617). Body Length (in bytes) Identifies the authorized user making this request. Describes the size of the message body.
Authorization Content-Length
No, unless secured Yes, on requests that contain a message body. Yes, on requests that contain a message body. All requests No
Content-Type
Content type
Describes the representation and syntax of the request message body. Required to allow support of multiple origin hosts at a single IP address. Declares the specification version of the YYYYY API that this client was programmed against.
Host
Origin hostname
X-YYYYYString Client-Specification-Version
201 Created
202 Accepted
86
HTTP Status
Description dication of the request's current status, and either a pointer to a status monitor or some estimate of when the user can expect the request to be fulfilled. The server fulfilled the request, but does not need to return a response message body. The request could not be processed because it contains missing or invalid information (such as validation error on an input field, a missing required value, and so on). The authentication credentials included with this request are missing or invalid. The server recognized your credentials, but you do not possess authorization to perform this request. The request specified a URI of a resource that does not exist. The HTTP verb specified in the request (DELETE, GET, HEAD, POST, PUT) is not supported for this request URI. The resource identified by this request is not capable of generating a representation corresponding to one of the media types in the Accept header of the request. A creation or update request could not be completed, because it would cause a conflict in the current state of the resources supported by the server (for example, an attempt to create a new resource with a unique identifier already assigned to some existing resource). The server encountered an unexpected condition which prevented it from fulfilling the request. The server does not (currently) support the functionality required to fulfill the request. The server is currently unable to handle the request due to temporary overloading or maintenance of the server.
204 No Content 400 Bad Request 401 Unauthorized 403 Forbidden 404 Not Found 405 Method Not Allowed 406 Not Acceptable 409 Conflict
500 Internal Server Error 501 Not Implemented 503 Service Unavailable
8.2. Operations
This section describes Couchbase Management REST API operations. Launching the Couchbase Web Console Although not strictly part of the REST API, the Couchbase Web Console does use many of the REST API endpoints for the management and information gathering within the console itself. The Couchbase Web Console loads and runs in a common Web Browser user agent. It consists of HTML pages, images, CSS, and JavaScript. For a list of supported browsers, see System Requirements. For the Couchbase Web Console, a separate UI hierarchy is served from each node of the system (though asking for the root "/" would likely return a redirect to the user agent). To launch the Couchbase Web Console, point your browser to the appropriate host and port:
GET https://node.in.pool.com/
Bootstrapping Clients must bootstrap to gain an entry point into the RESTful service. Bootstrapping requires looking for pools, as shown in the request blow. The URI space, in Couchbase's implementation may appear to have very specific URI and in some ways may even appear as RPC or some other architectural style using HTTP operations and semantics. That is only an artifact of the URIs Couchbase chose. Clients are advised to be properly RESTful and will not expect to receive any handling instructions resource descriptions or presume any conventions on URI structure for resources represented. Also note that the hierarchies shown here can allow reuse of agent handling of representations, since they are similar for different parts of the hierarchy. Request
87
GET /pools Host: node.in.your.pool.com Authorization: Basic xxxxxxxxxxxxxxxxxxx Accept: application/json X-memcachekv-Store-Client-Specification-Version: 0.1
Response
HTTP/1.1 200 OK Content-Type: application/json Content-Length: nnn
{ "implementationVersion": "253", "pools" : [ { "name": "Default Pool", "uri": "/pools/default", }, { "name": "Couchbase kvcaching pool name", "uri": "/pools/anotherpool", }, { "name": "A pool elsewhere", "uri": "https://a.node.in.another.pool.com:4321/pools/default" } ] "uri": "https://node.in.your.pool.com/pools", "specificationVersion": [ "0.1" ] }
Only one pool per group of systems will be known and it will likely be defaulted to a name. POSTing back a changed pool will return a 403. As can be seen, the "build" number of the implementation is apparent in the implementation_version, the specifications supported are apparent in the specification_version. While this node can only be a member of one pool, there is flexibility which allows for any given node to be aware of other pools. The Client-Specification-Version is optional in the request, but advised. It allows for implementations to adjust to adjust representation and state transitions to the client, if backward compatibility is desirable. Provisioning a Node Before a node can be used in a cluster, a few things may need to be configured. Specifically, if creating a new cluster, the memory quota per node for that cluster must be set. Whether the node is joining an existing cluster or starting a new cluster, it's storage path must be configured. Either creating a new cluster or adding a node to a cluster is referred to as provisioning, and has several steps required. After bootstrapping the following will need to be accomplished. Configure the node's disk path. Configure the cluster's memory quota. This is optional. It will inherit the memory quota if the node is to be joined to a cluster and it will default to 80% of physical memory if not specified. The next step depends on whether a new cluster is being created or an existing cluster will be joined. If a new cluster is to be created, it will need to be "secured" by providing a username and password for the administrator. If the node is to be joined to another cluster, then it will need the location and credentials to the REST interface of that cluster. Configuring the disk path for a node
88
Node resources can be configured through a controller on the node. The primary resource to be configured is the path on the node for persisting files. This must be configured prior to creating a new cluster or configuring the node into an existing cluster. Request
POST /nodes/self/controller/settings HTTP/1.1 Host: node.in.your.cluster:8091 Content-Type: application/x-www-form-urlencoded; charset=UTF-8 Authorization: Basic YWRtaW46YWRtaW4= Content-Length: xx path=/var/tmp/test
Response
HTTP/1.1 200 OK Content-Type: application/json Content-Length: 0
Example
shell> curl -d path=/var/tmp/test http://localhost:8091/nodes/self/controller/settings
Response
HTTP/1.1 200 OK Content-Type: application/json Content-Length: 0
Example
shell> curl -d memoryQuota=400 http://localhost:8091/pools/default
Setting a node's username and password While this can be done at any time for a cluster, it is the last step in provisioning a node into being a new cluster. The response will indicate the new base URI if the parameters are accepted. Clients will want to re-bootstrap based on this response. Request Response
HTTP/1.1 200 OK Content-Type: application/json Server: Couchbase Server 1.6.0 Pragma: no-cache Date: Mon, 09 Aug 2010 18:50:00
Example
curl -d username=Administrator -d password=letmein -d port=8091 http://localhost:8091/settings/web
Note Note that even if it is not to be changed, the port number must be specified. Global settings Setting whether statistics should be sent to the outside or not It's a global setting for all clusters. You need to be authenticated to change this value. Request
89
Response 200 OK 400 Bad Reqeust, The value of "sendStats" must be true or false. 401 Unauthorized Example
curl -i -u Administrator:letmein -d sendStats=true http://localhost:8091/settings/stats
Getting whether statistics should be sent to the outside or not It's a global setting for all clusters. You need to be authenticated to read this value. Request
GET /settings/stats HTTP/1.1 Host: node.in.your.pool.com Authorization: Basic YWRtaW46YWRtaW4= Accept: */*
Response
HTTP/1.1 200 OK Content-Type: application/json Content-Length: nnn
Example
curl -u Administrator:letmein http://localhost:8091/settings/stats
It's a global setting for all clusters. You need to be authenticated to change this value. Possible parameters are: enabled (true|false) (required): whether to enable or disable auto-failover timeout (integer biger (or equal) than 30) (required; optional when enabled=false): The number of seconds a node must be down before it is automatically failovered Request
Response 200 OK 400 Bad Request, The value of "enabled" must be true or false. 400 Bad Request, The value of "timeout" must be a positive integer bigger or equal to 30. 401 Unauthorized Example
curl -i -u Administrator:letmein -d enabled=true&timeout=60http://localhost:8091/settings/autoFailover
It's a global setting for all clusters. You need to be authenticated to read this value. Request Response
GET /settings/autoFailover HTTP/1.1 Host: node.in.your.pool.com Authorization: Basic YWRtaW46YWRtaW4= Accept: */* HTTP/1.1 200 OK Content-Type: application/json Content-Length: nnn { "enabled": true, "timeout": 60, "count": 0 }
Example
curl -u Administrator:letmein http://localhost:8091/settings/autoFailover
90
It's a global setting for all clusters. You need to be authenticated to change this value. Reset the number of nodes that where automatically failovered to zero. No parameters Request
It's a global setting for all clusters. You need to be authenticated to change this value. You will receive an email when certain events happen (currently only events cause by auto-failover are supported). Possible parameters are: enabled (true|false) (required): whether to enable or disable email notifications sender (string) (optional, default: couchbase@localhost): The sender address of the email recipients (string) (required): A comma separated list of recipients of the of the alert emails. emailHost (string) (optional, default: localhost): Host address of the SMTP server emailPort (integer) (optional, default: 25): Port of the SMTP server emailEncrypt (true|false) (optional, default: false): Whether you'd like to use TLS or not emailUser (string) (optional, default: ""): Username for the SMTP server emailPass (string) (optional, default: ""): Password for the SMTP server alerts (string) (optional, default: auto_failover_node, auto_failover_maximum_reached, auto_failover_other_nodes_down, auto_failover_cluster_too_small): Comma separated list of alerts that should cause an email to be sent. Possible values are: auto_failover_node, auto_failover_maximum_reached, auto_failover_other_nodes_down, auto_failover_cluster_too_small. Request
POST /settings/alerts HTTP/1.1 Host: node.in.your.cluster:8091 Content-Type: application/x-www-form-urlencoded Authorization:
Response 200 OK 400 Bad Request: JSON object ({"errors": {"key": "error"}}) with errors. Possible errors are: alerts: alerts contained invalid keys. Valid keys are: [list_of_keys]. email_encrypt: emailEncrypt must be either true or false. email_port: emailPort must be a positive integer less than 65536. enabled: enabled must be either true or false.
91
recipients: recipients must be a comma separated list of valid email addresses. sender: sender must be a valid email address. general: No valid parameters given. 401 Unauthorized Example
It's a global setting for all clusters. You need to be authenticated to read this value. Request
GET /settings/alerts HTTP/1.1 Host: node.in.your.pool.com Authorization: Basic YWRtaW46YWRtaW4= Accept: */*
Response Example
HTTP/1.1 200 OK Content-Type: application/json Content-Length: nnn { "alerts": [ "auto_failover_node", "auto_failover_maximum_ curl -u Administrator:letmein http://localhost:8091/settings/alerts
It's a global setting for all clusters. You need to be authenticated to change this value. Sends a test email with the current configuration (see mail alerts section) Takes the same parameters as /settings/alerts and additionally: subject (string) (required): The subject of the test email body (string) (required): The body of the test email Request
Response 200 OK 400 Bad Request: Unknown macro: {"error"} 401 Unauthorized Example
GET /pools/default Host: node.in.your.pool.com Authorization: Basic xxxxxxxxxxxxxxxxxxx Accept: application/json X-memcachekv-
Response
92
"hostname":"10.0.1.20",
"ram":{ "total":2032558080, "used":1641816064 }, "hdd":{ "total":239315349504.0, "used": 229742735523.0 } }, "buckets":{ "uri":"/pools/default/buckets" }, "controllers":{ "ejectNode":{ > "uri":"/pools/default/controller/ejectNode" }, "addNode":{
"uri":"/controller/addNode" }, "rebalance":{
"uri":"/controller/rebalance" }, "failover":{
"uri":"/controller/failOver" }, "reAddNode":{
"uri":"/controller/reAddNode" }, "stopRebalance":{
"uri":"/controller/stopRebalance" } }, "rebalanceProgress":{
93
"uri":"/pools/default/rebalanceProgress" }, "balanced": true,
At the highest level, a pool describes a cluster (as mentioned above). This cluster exposes a number of properties which define attributes of the cluster and "controllers" which allow users to make certain requests of the cluster. Note that since buckets could be renamed and there is no way to determine what the default bucket for a pool is, the system will attempt to connect non-SASL, non-proxied to a bucket clients to a bucket named "default". If it does not exist, the connection will be dropped. Clients MUST NOT rely on the node list here to create their "server list" for when connecting. They MUST instead issue an HTTP get call to the bucket to get the node list for that specific bucket. The controllers, all of which accept parameters as x-www-form-urlencoded, for this list perform the following functions: Table 8.3. Controller Functions Function ejectNode addNode Description Eject a node from the cluster. Required parameter: "otpNode", the node to be ejected. Add a node to this cluster. Required parameters: "hostname", "user" (which is the admin user for the node), and "password". Rebalance the existing cluster. This controller requires both "knownNodes" and "ejectedNodes". This allows a client to state the existing known nodes and which nodes should be removed from the cluster in a single operation. To ensure no cluster state changes have occurred since a client last got a list of nodes, both the known nodes and the node to be ejected must be supplied. If the list does not match the set of nodes, the request will fail with an HTTP 400 indicating a mismatch. Note rebalance progress is available via the rebalanceProgress uri. Failover the vBuckets from a given node to the nodes which have replicas of data for those vBuckets. The "otpNode" parameter is required and specifies the node to be failed over. The "otpNode" parameter is required and specifies the node to be re-added. Stop any rebalance operation currently running. This takes no parameters.
rebalance
failover
reAddNode stopRebalance
The list of nodes will list each node in the cluster. It will, additionally, list some attributes of the nodes. Table 8.4. Node Attributes Attribute memoryTotal Description The total amount of memory available to Couchbase, allocated and free. May or may not be equal to the amount of memory configured in the system. The amount of memory available to be allocated. This is equal to the memoryTotal, subtracting out all memory allocated as reported by the host operating system. The amount of memory reserved for use by Couchbase across all buckets on this node. This value does not include
memoryFree
mcdMemoryReserved
94
Attribute
Description some overhead for managing items in the node or handling replication or other TAP streams. The amount of memory actually used by all buckets on this node.
mcdMemoryAllocated
List Buckets and Bucket Operations Clients to the system can choose to use either the proxy path or the direct path. If they use the direct path, they will not be insulated from most reconfiguration changes to the bucket. This means they will need to either poll the bucket's URI or connect to the streamingUri to receive updates when the bucket configuration changes. This happens, for instance, when nodes are added, removed, or may fall into an unhealthy state. Request
GET /pools/default/buckets Host: node.in.your.pool.com Authorization: Basic xxxxxxxxxxxxxxxxxxx Accept: application/json X-memcachekv-Store-Client-Specification-Version: 0.1
Response
95
HTTP/1.1 200 OK Server: Couchbase Server 1.6.0 Pragma: no-cache Date: Wed, 03 Nov 2010 18:12:19 GMT Content-Type: application/json Content-Length: nnn Cache-Control: no-cache no-store max-age=0 [ { "name": "default", "bucketType": "couchbase", "authType": "sasl", "saslPassword": "", "proxyPort": 0, "uri": "/pools/default/buckets/default", "streamingUri": "/pools/default/bucketsStreaming/default", "flushCacheUri": "/pools/default/buckets/default/controller/doFlush", "nodes": [ { "uptime": "784657", "memoryTotal": 8453197824.0, "memoryFree": 1191157760, "mcdMemoryReserved": 6449, "mcdMemoryAllocated": 6449, "clusterMembership": "active", "status": "unhealthy", "hostname": "10.1.15.148:8091", "version": "1.6.0", "os": "windows", "ports": { "proxy": 11211, "direct": 11210 } } ], "stats": { "uri": "/pools/default/buckets/default/stats" }, "nodeLocator": "vbucket", "vBucketServerMap": { "hashAlgorithm": "CRC", "numReplicas": 1, "serverList": [ "192.168.1.2:11210" ], "vBucketMap": [ [ 0, -1 ], [ 0, -1 ], [ 0, -1 ], [ 0, -1 ], [ 0, -1 ], [ 0, -1 ]] }, "replicaNumber": 1, "quota": { "ram": 104857600, "rawRAM": 104857600 }, "basicStats": { "quotaPercentUsed": 24.360397338867188, "opsPerSec": 0, "diskFetches": 0, "itemCount": 0, "diskUsed": 0, "memUsed": 25543728 } }, { "name": "test-application", "bucketType": "memcached", "authType": "sasl", "saslPassword": "", "proxyPort": 0, "uri": "/pools/default/buckets/test-application", "streamingUri": "/pools/default/bucketsStreaming/test-application", "flushCacheUri": "/pools/default/buckets/test-application/controller/doFlush", "nodes": [ { "uptime": "784657", "memoryTotal": 8453197824.0, "memoryFree": 1191157760, "mcdMemoryReserved": 6449, "mcdMemoryAllocated": 6449, "clusterMembership": "active", "status": "healthy", "hostname": "192.168.1.2:8091", 96 "version": "1.6.0", "os": "windows", "ports": { "proxy": 11211,
Named Bucket and Bucket Streaming URI The individual bucket request is exactly the same as what would be obtained from the item in the array for the entire buckets list above. The streamingUri is exactly the same except it streams HTTP chunks using chunked encoding. A response of "\n\n\n\n" delimits chunks. This will likely be converted to a "zero chunk" in a future release of this API, and thus the behavior of the streamingUri should be considered evolving. Request
GET /pools/default/buckets/default Host: node.in.your.pool.com Authorization: Basic xxxxxxxxxxxxxxxxxxx Accept: application/json X-memcachekv-Store-Client-Specification-Version: 0.1
Response
97
HTTP/1.1 200 OK Content-Type: application/json Content-Length: nnn { "name": "default", "bucketType": "couchbase", "authType": "sasl", "saslPassword": "", "proxyPort": 0, "uri": "/pools/default/buckets/default", "streamingUri": "/pools/default/bucketsStreaming/default", "flushCacheUri": "/pools/default/buckets/default/controller/doFlush", "nodes": [ { "uptime": "308", "memoryTotal": 3940818944.0, "memoryFree": 1608724480, "mcdMemoryReserved": 3006, "mcdMemoryAllocated": 3006, "replication": 1.0, "clusterMembership": "active", "status": "healthy", "hostname": "172.25.0.2:8091", "clusterCompatibility": 1, "version": "1.6.4r_107_g49a149d", "os": "i486-pc-linux-gnu", "ports": { "proxy": 11211, "direct": 11210 } }, { "uptime": "308", "memoryTotal": 3940818944.0, "memoryFree": 1608724480, "mcdMemoryReserved": 3006, "mcdMemoryAllocated": 3006, "replication": 1.0, "clusterMembership": "active", "status": "healthy", "hostname": "172.25.0.3:8091", "clusterCompatibility": 1, "version": "1.6.4r_107_g49a149d", "os": "i486-pc-linux-gnu", "ports": { "proxy": 11211, "direct": 11210 } }, { "uptime": "308", "memoryTotal": 3940818944.0, "memoryFree": 1608597504, "mcdMemoryReserved": 3006, "mcdMemoryAllocated": 3006, "replication": 1.0, "clusterMembership": "active", "status": "healthy", "hostname": "172.25.0.4:8091", "clusterCompatibility": 1, "version": "1.6.4r_107_g49a149d", "os": "i486-pc-linux-gnu", "ports": { "proxy": 11211, "direct": 11210 } } ], "stats": { "uri": "/pools/default/buckets/default/stats" }, "nodeLocator": "vbucket", "vBucketServerMap": { "hashAlgorithm": "CRC", "numReplicas": 1, "serverList": [ "172.25.0.2:11210", "172.25.0.3:11210", "172.25.0.4:11210" ], 98 "vBucketMap": [ [1,0], [2,0], [1,2],
Flushing a Bucket Warning This operation is data destructive.The service makes no attempt to double check with the user. It simply moves forward. Clients applications using this are advised to double check with the end user before sending such a request. Note As of couchbase 1.6 bucket flushing via REST API is not supported. Flushing via memcached protocol works. The bucket details provide a bucket URI at which a simple request can be made to flush the bucket. Request
Any parameters are accepted. Since the URI was defined by the bucket details, neither the URI nor the parameters control what is actually done by the service. The simple requirement is for a POST with an appropriate Authorization header, if the system is secured. Response
204 No Content - if the flush is successful 404 Not Found - if the URI is invalid or does not correspond to a bucket the syste
Deleting a Bucket Warning This operation is data destructive.The service makes no attempt to double check with the user. It simply moves forward. Clients applications using this are advised to double check with the end user before sending such a request. Request
Response Getting Statistics Statistics can be gathered via the REST interface at the bucket level. Stats URL should be taken from stats.uri property of bucket response. By default it returns stats samples for last minute and hot keys. Via query parameters another zoom level can be requested. zoom - stats zoom level (minute | hour | day | week | month | year) resampleForUI - pass 1 if you need 60 samples haveTStamp - request sending only samples newer than given timestamp Request Response
GET /pools/default/stats Host: node.in.your.pool.com Authorization: Basic xxxxxxxxxxxxxxxxxxx Accept: application/json X-memca
99
HTTP/1.1 200 OK Content-Type: application/json Content-Length: nnn { "op": { "samples": { "hit_ratio": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_cache_miss_rate": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_resident_items_rate": [ 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0 ], "ep_replica_resident_items_rate": [ 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0 ], "bytes_read": [ 151, 130, 72, 158, 120, 223, 103, 127, 127, 127, 103, 130, 72, 158, 72, 220, 103, 175, 130, 127, 103, 127, 120, 158, 72, 271, 103, 124, 130, 127, 103, 130, 72, 155, 72, 223, 103, 124, 178, 127, 103, 130, 72, 206, 69, 223, 103, 175, 130, 127, 103, 130, 72, 155, 72, 223, 103, 127, 130 ], "bytes_written": [ 36776, 40699, 15212, 41705, 15260, 41326, 36728, 20360, 40658, 20388, 36728, 40699, 15212, 41705, 15212, 41283, 36764, 20408, 40700, 20369, 36728, 40656, 15274, 41705, 15212, 41374, 36728, 20338, 40700, 20388, 36728, 40699, 15212, 41662, 15212, 41366, 36728, 20338, 40748, 20388, 36728, 40699, 15212, 41753, 15195, 41326, 36728, 20408, 40739, 20369, 36728, 40699, 15212, 41662, 15212, 41366, 36728, 20360, 40700 ], "cas_badval": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "cas_hits": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "cas_misses": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "cmd_get": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "cmd_set": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "curr_connections": [ 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139 ], "curr_items": [ 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125 ], "curr_items_tot": [ 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250 ], "decr_hits": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "decr_misses": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], 100 "delete_hits": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "delete_misses": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Managing Bucket Resources A new bucket may be created with a POST command to the URI to the buckets defined URI for the pool. This can be used to create either a couchbase or a memcached type bucket. The bucket name cannot have a leading underscore. When creating a bucket, an authType parameter must be specified: If the authType is "none". then a proxyPort number MUST be specified. If the authType is "sasl". then a "saslPassword" parameter MAY be optionally specified. In release 1.6.0, any SASL auth based access must go through the proxy which is fixed to port 11211. The ramQuotaMB attribute allows the user to specify how much memory (in megabytes) will be allocated on each node for the bucket. In the case of memcached buckets, going beyond the ramQuotaMB will cause an item to be evicted on a mostly-LRU basis. The type of item evicted may not be the exact LRU due to object size, whether or not it is currently being referenced or other situations. In the case of couchbase buckets, the system may return temporary failures if the ramQuotaMB is reached. The system will try to keep 25% of the available ramQuotaMB free for new items by ejecting old items from occupying memory. In the event these items are later requested, they will be retrieved from disk. Creating a memcached Bucket Request
Response
202: bucket will be created asynchronously with the location header returned.
No URI to check for the completion task is available, but most bucket creations complete within a few seconds. Example
Creating a Couchbase Bucket In addition to the aforementioned parameters, a replicaNumber parameter specifies the number of replicas. Request
Response Example
202: bucket will be created.
Response
HTTP/1.1 200 OK Content-Type: application/com.couchbase.store+json Content-Length: nnn { "name" : "Another bucket" "bucketRule
101
Clients MUST use the nodes list from the bucket, not the pool to indicate which are the appropriate nodes to connect to. Modifying Bucket Properties Buckets may be modified by POSTing the same kind of parameters used to create the bucket to the bucket's URI. Since an omitted parameter can be equivalent to not setting it in some cases, it is recommended that clients get existing parameters, make modifications where necessary, and then POST to the URI. The name of a bucket cannot be changed. Increasing the Memory Quota for a Bucket Increasing a bucket's ramQuotaMB from the current level. Note, the system will not let you decrease the ramQuotaMB for a couchbase bucket type and memcached bucket types will be flushed when the ramQuotaMB is changed. Warning As of 1.6.0, there are some known issues with changing the ramQuotaMB for memcached bucket types. Request (curl)
curl -d ramQuotaMB=25 -d authType=none -d proxyPort=11215 http://localhost:8080/pools/default/buckets/newbucket
Response The response will be 202, indicating the quota will be changed asynchronously throughout the servers in the cluster.
HTTP/1.1 202 OK Server: Couchbase Server 1.6.0 Pragma: no-cache Date: Wed, 29 Sep 2010 20:01:37 GMT Content-Length: 0 Cache-Co
Changing Bucket Authentication Changing a bucket from port based authentication to SASL authentication can be done with:
curl -d ramQuotaMB=130 -d authType=sasl -d saslPassword=letmein http://localhost:8080/pools/default/buckets/acache
Cluster and Pool Operations Creating a new pool is not currently supported. A new pool may be supported in future releases. At that time a pool can be created by posting to the /pools URI. Request
POST /pools/mynewpool name=mynewpool
Response
201: pool was created and valid URIs for referencing it returned 403: user not authorized (or no users are authorized because
Warning For this release, this will always return 405 Method Not Allowed. Joining a Cluster Clusters (a.k.a. pools) cannot be merged if they are made of multiple nodes. However, a single node can be asked to join a cluster. It will need several parameters to be able to negotiate a join to the cluster. Request
POST /node/controller/doJoinCluster Host: target.node.to.do.join.from:8091 Authorization: Basic xxxxxxxxxxxx Accept: */* Conte
102
The following arguments are required: Table 8.5. Cluster Joining Arguments Argument clusterMemberHostIp clusterMemberPort Description Hostname or IP address to a member of the cluster the node receiving this POST will be joining Port number for the RESTful interface to the system
If the server has been "secured" via the Couchbase Web Console, the following are also required Table 8.6. Additional Arguments Argument user password Response Description Administration user Password associated with the Administration user
200 OK with Location header pointing to pool details of pool just joined - successful join 400 Bad Request - missing parameter
Example (curl)
Ejecting a Node from a Cluster In situations where a node is down either temporarily or permanently, we may need to eject it from the cluster. It may also be important to eject a node from another node participating in the same cluster. Request
POST /controller/ejectNode Host: altnernate.node.in.cluster:8091 Authorization: Basic xxxxxxxxxxxx Accept: */* Content-Length:
Response
200 OK - node ejected 400 Error, the node to be ejected does not exist 401 Unauthorized - Credentials were not supplied and ar
Example (curl)
curl -d otpNode=ns_1@192.168.0.107 http://192.168.0.106:8091/controller/ejectNode
System Logs System modules log various messages, which are available via this interface. These log messages are optionally categorized by the module. Both a generic list of recent log entries and recent log entries for a particular category are available. A GET without specifying a category returns all categories. Returned types may be "info" "crit" or "warn". Accessing logs requires administrator credentials if the system is secured. Request
Response
201: bucket was created and valid URIs returned HTTP/1.1 200 OK Content-Type: application/json Content-Length: nnn [{"cat":"in
103
Clients might want to add entries to the service's central logger. These entries would typically be responses to exceptional like difficulty handling a server response. For instance, the Web UI uses this functionality to log client error conditions for later diagnosis. Request
POST /logClientError Host: node.in.your.pool.com Authorization: Basic xxxxxxxxxxxxxxxxxxx Accept: application/json X-memcachek
Response
200 - OK
104
105
"harrypotter"), Couchbase can be used as the database for fast topic to subscriber matching, allowing your application to quickly answer, "who is interested in event X?"
106
If the above is stored in Couchbase under document ID "user-1234", you can then know the interests for that user by doing a simple GET for user-1234 and decoding the JSON response.
Such lists can be easily constructed by using Couchbase's APPEND or PREPEND operations, where you append/prepend values that look like "user-XXXXX,". Note that the list is delimited by commas, but that can be any character you choose.
So, after the client library retrieves that list and does some post-processing, the effective, actual list of interested subscribers is user-1234 and user-987. Care must be taken to count correctly, in case user-222 decides to add themselves again to the list (and her clicks are faster than whatever logic your application has to prevent duplicate clicks):
user-222,|user-1234,user-222,user-987,user-222
A similar encoding scheme would use '+' or '-' delimiter characters to the same effect, where the client sends an APPEND of "+ID" to add an entry to a list, and an APPEND of "-ID" to remove an entry from a list. The client application would still perform post-processing on the list response, tracking appropriate list entry counts. In this and other encodings, we must take care not to use the delimiter characters that were chosen:
+1234+222+987-222
Yet another variation on this would be store deleted items to a separate paired list. So your application might have two lists for a topic, such as a "follow-X" and "unfollow-X".
107
A client could multi-GET on topic-X.a and topic-X.b, and the combined result would contain the full list. To mutate the list, the client would look at the "pointer" item of topic-X.active, and know to APPEND values to topic-X.a. A randomly self-chosen client may choose to garbage-collect the active list when it sees the list length is large enough, by writing a compressed version of topic-X.a into topic-X.b (note: XXX) and by flipping the topic-X.active item to point to "b". New clients will start APPEND'ing values to topic-X.b. Old, concurrent clients might still be APPEND'ing values to the old active item of topic-X.a, so other randomly self-selected clients can choose to help continue to compress topic-X.a into topic-X.b so that topic-X.a will be empty and ready for the next flip. An alternative to a separate "topic-X.active" pointer item would be instead to PREPEND a tombstone marker value onto the front of the inactivated list item. For example, if '^' was the tombstone marker character, all concurrent clients would be able to see in that a certain list should not be appended to:
topic-X.a => +1234+222+987-222 topic-X.b => ^+1234
There are concurrency holes in this "active flipping" scheme, such as if there's a client process failure at the step noted above at "XXX", so for periods of time there might be duplicates or reappearing list items. In general, the idea is that independent clients try to make progress towards an eventually stabilized state. Please consider your application use cases as to whether temporary inconsistencies are survivable.
The "topic-X" item just lists pointers to items that have the actual lists. In this approach, you could have randomly self-selected clients decide to add new topic sub-lists (topic-X.N) and APPEND'ing updated info to the "index" item (topic-X). Other randomly self-chosen clients could attempt to compress topic sub-lists that are old.
9.2.6. Multi-GET
Once your client application has a list of document IDs, the highest performance approach to retrieve the actual items is to use a multi-GET request. Doing so allows for concurrent retrieval of items across your Couchbase cluster. This will perform better than a serial loop that tries to GET for each item individually and sequentially.
9.2.7. Locking
Advisory locks can be useful to control access to scarce resources. For example, retrieving information from backend or remote systems might be slow and consume a lot of resources. Instead of letting any client access the backend system and
108
potentially overwhelm the backend system with high concurrent client requests, you could create an advisory lock to allow only one client at a time access the backend. Advisory locks in Couchbase or Memcached can be created by setting expiration times on a named data item and by using the 'add' and 'delete' commands to access that named item. The 'add' or 'delete' commands are atomic, so you can be know that only one client will become the advisory lock owner. The first client that tries to ADD a named lock item (with an expiration timeout) will succeed. Other clients will see error responses to an ADD command on that named lock item, so they can know that some other client is owning the named lock item. When the current lock owner is finished owning the lock, it can send an explicit DELETE command on the named lock item to free the lock. If the lock owning client crashes, the lock will automatically become available to the next client that polls for the lock (using 'add') after the expiration timeout.
109
10.1. REST/JSON
The mapping between the vBucket designation and the client interface operates through a request to the Couchbase Server via client-initiated REST/JSON requests. The REST/JSON URLs should be provided to the client library as some initial configuration parameter by the user's application. The client application should bootstrap the REST/JSON information by "chasing URLs" from a standard base, starting URL. For more information on REST/JSON "bootstrapping" and chasing URLs, please follow the links at Chapter 8, Couchbase Management REST API. Eventually, after following the bootstrapping sequence, your client library will have a REST/JSON URL that looks like:
http://HOST:PORT/pools/default/bucketsStreaming/BUCKET_NAME
For example:
http://couchbase1:8080/pools/default/bucketsStreaming/default
The REST/JSON URLs might be under HTTP Basic Auth authentication control, so the client application may also have to provide (optional) user/password information to the your client library so that the proper HTTP/REST request can be made. The REST/JSON URLs are "streaming", in that the couchbase REST server doesn't close the HTTP REST connection after responding with one vBucket-to-server map. Instead, couchbase keeps the connection open, and continues to stream new maps to your client library when there are cluster changes (new server nodes being added, removed, and/or vBuckets getting reassigned to different servers). In the couchbase streaming approach, new vBucket-to-server map JSON messages are delimited by 4 newlines ("\n\n\n\n") characters. The above section describes what we call "per-bucket" REST/JSON URLs. That is, each port-based bucket has a streaming REST/JSON URL of the form:
http://HOST:PORT/pools/default/bucketsStreaming/BUCKET_NAME
110
There is another kind of REST/JSON URL, which describes all SASL-authenticated buckets. This SASL REST/JSON URL has the form of:
http://HOST:PORT/pools/default/saslBucketsStreaming
Sample output:
{"buckets":[ {"name":"default", "nodeLocator":"vbucket", "saslPassword":"", "nodes":[ {"clusterMembership":"active","status":"healthy","hostname":"10.1.4.11:8091", "version":"1.6.1rc1","os":"x86_64-unknown-linux-gnu", "ports":{"proxy":11211,"direct":11210}}, {"clusterMembership":"active","status":"healthy","hostname":"10.1.4.12:8091", "version":"1.6.1pre_21_g5aa2027","os":"x86_64-unknown-linux-gnu", "ports":{"proxy":11211,"direct":11210}}], "vBucketServerMap":{ "hashAlgorithm":"CRC","numReplicas":1, "serverList":["10.1.4.11:11210","10.1.4.12:11210"], "vBucketMap":[[0,-1],[1,-1],...,[0,-1],[0,-1]]}} ] }
One main difference between the SASL REST/JSON response versus the per-bucket REST/JSON response is that the SASL REST/JSON response can describe more than one bucket. In the SASL REST/JSON response, these multiple buckets would be found under the "buckets" array.
The vBucketMap is zero-based indexed by vBucketId. So, if you have a vBucket whose vBucketId is 4, you'd look up vBucketMap[4]. The entries in the vBucketMap are arrays of integers, where each integer is a zero-based index into the serverList array. The 0th entry in this array describes the primary server for a vBucket. Here's how you read this stuff, based on the above config: The vBucket with vBucketId of 0 has a configuration of vBucketMap[0], or[0, 1]. So vBucket 0's primary server is at serverList[0], or 10.1.4.11:11210. While vBucket 0's first replica server is at serverList[1], which is 10.1.4.12:11210. The vBucket with vBucketId of 1 has a configuration of vBucketMap[1], or[1, 0]. So vBucket 1's primary server is at serverList[1], or 10.1.4.12:11210. And vBucket 1's first replica is at serverList[0], or 10.1.4.11:11210. This structure and information repeats for every configured vBucket. If you see a -1 value, it means that there is no server yet at that position. That is, you might see:
111
"vBucketMap":[[0,-1],[0,-1],[0,-1],[0,-1],:]
Sometimes early before the system has been completely configured, you might see variations of:
"serverList":[], "vBucketMap":[]
112
Eventually, one server will respond with success, and your client library has then independently discovered the new, correct owner of that vBucketId. Your client library should record that knowledge in its vBucket-server-map(s) for use in future operations time. An implementation of this can be seen in the libvBucket API vbucket_found_incorrect_master(). The following shows a swim-lane diagram of how moxi interacts with libvBucket during NOT_MY_VBUCKET errors libvbucket_notmyvbucket.pdf . At the end of the Rebalance, the couchbase cluster will notify streaming REST/JSON clients, finally, with a new vBucket-to-server map. This can be handled by your client library like any other vBucket-to-server map update message. However, in the meantime, your client library didn't require granular map updates during the Rebalancing, but found the correct vBucket owners on its own.
113
28| 4e ('N') |
Header breakdown Field (offset) (value) Magic (0): 0x81 (PROTOCOL_BINARY_RES) Opcode (1): 0x20 (sasl list mechs) Key length (2-3): 0x0000 (0) Extra length (4): 0x00 Data type (5): 0x00 Status (6-7): 0x0000 (SUCCESS) Total body (8-11): 0x00000005 (5) Opaque (12-15): 0x00000000 (0) CAS (16-23): 0x0000000000000000 (0) Mechanisms (24-28): PLAIN
Please note that the server may support a different set of mechanisms. The list of mechanisms is a space-separated list of SASL mechanism names (e.g. "PLAIN CRAM-MD5 GSSAPI").
114
If the server accepts this username/password combination, it may return one of two status codes: Success or "Authentication Continuation". Success means that you're done
Byte/ 0 | 1 | 2 | 3 | / | | | | |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| +---------------+---------------+---------------+---------------+ 0| 81 | 21 ('!') | 00 | 00 | +---------------+---------------+---------------+---------------+ 4| 00 | 00 | 00 | 00 | +---------------+---------------+---------------+---------------+ 8| 00 | 00 | 00 | 0d | +---------------+---------------+---------------+---------------+ 12| 00 | 00 | 00 | 00 | +---------------+---------------+---------------+---------------+ 16| 00 | 00 | 00 | 00 | +---------------+---------------+---------------+---------------+ 20| 00 | 00 | 00 | 00 | +---------------+---------------+---------------+---------------+ 24| 41 ('A') | 75 ('u') | 74 ('t') | 68 ('h') | +---------------+---------------+---------------+---------------+ 28| 65 ('e') | 6e ('n') | 74 ('t') | 69 ('i') | +---------------+---------------+---------------+---------------+ 32| 63 ('c') | 61 ('a') | 74 ('t') | 65 ('e') | +---------------+---------------+---------------+---------------+ 36| 64 ('d') |
115
Header breakdown Field (offset) (value) Magic (0): 0x81 (PROTOCOL_BINARY_RES) Opcode (1): 0x21 (sasl auth) Key length (2-3): 0x0000 (0) Extra length (4): 0x00 Data type (5): 0x00 Status (6-7): 0x0000 (SUCCESS) Total body (8-11): 0x0000000d (13) Opaque (12-15): 0x00000000 (0) CAS (16-23): 0x0000000000000000 (0) Info (24-36): Authenticated
116
117
Couchbase Architecture
memcached Though couchbase is different than memcached, it does leverage the core of memcached. The core includes networking and protocol handling. The bulk of couchbase is implemented in two components: couchbase engine (ep-engine) This is loaded through the memcached core and the bucket_engine. This core component provides persistence in an asynchronous fashion and implements the TAP protocol. bucket engine The bucket engine provides a way of loading instances of engines under a single memcached process. This is how couchbase provides multitenancy. vbucketmigrator Effectively a TAP client, based on how ns_server starts one or more vBucketmigrator processes, data is either replicated or transferred from one node to another. Moxi A memcached proxy, moxi "speaks" vBucket hashing (implemented in libvbucket) and can talk to the REST interface to get cluster state and configuration, ensuring that clients are always routed to the appropriate place for a given vBucket. Across multiple cloud instances, VMs or physical servers, all of these components come together to become a couchbase cluster.
11.4.1. Design
Each instance of ep-engine in a given node will have a certain memory quota associated with it. This memory quota is sometimes referred to as >the amount of cache memory. That amount of memory will always store the index to the entire working set. By doing so, we ensure most items are quickly fetched and checks for the existence of items is always fast.
118
Couchbase Architecture
In addition to the quota, there are two watermarks the engine will use to determine when it is necessary to start freeing up available memory. These are mem_low_wat and mem_high_wat. As the system is loaded with data, eventually the mem_low_wat is passed. At this time, no action is taken. This is the "goal" the system will move toward when migrating items to disk. As data continues to load, it will evenutally reach mem_high_wat. At this point a background job is scheduled to ensure items are migrated to disk and the memory is then available for other couchbase items. This job will run until measured memory reaches mem_low_wat. If the rate of incoming items is faster than the migration of items to disk, the system may return errors indicating there is not enough space. This will continue until there is available memory.
119
Couchbase Architecture
hash table. In the case of an "ejected" record, the value will be missing, effectively pointed to NULL. This is useful for larger objects, but not particularly efficient for small objects. This is being addressed in future versions. When fetching a value, we will first look in the hash table. If we don't find it, we don't have it. MISS. If we do have it and it's resident, we return it. HIT. If we have it and it's not resident, we schedule a background fetch and let the dispatcher pull the object from the DB and reattach it to the stored value in memory. The connection is then placed into a blocking state so the client will wait until the item has returned from slower storage. The background fetch happens at some point in the future via an asynchronous job dispatcher. Figure 11.2. Background Fetch Workflow
When the job runs, the item is returned from disk and then the in-memory item is pulled and iff it is still not resident, will have the value set with the result of the disk fetch.* Once the process is complete, whether the item was reattached from the disk value or not, the connection is reawakened so the core server will replay the request from the beginning. It's possible (though very unlikely) for another eject to occur before this process runs in which case the entire fetch process will begin again. The client has no particular action to take after the get request until the server is able to satisfy it. Note An item may be resident after a background fetch either in the case of another background fetch for the same document ID having completed prior to this one or another client has modified the value since we looked in memory. In either case, we assume the disk value is older and will discard it.
120
Couchbase Architecture
Future storage mechanisms may allow for concurrent execution under different conditions and will indicate this by reporting their level of concurrency differently. The concurrentDB engine parameter allows the user to disable concurrent DB access even when the DB reports it's possible. The possible concurrency levels are reported via the ep_store_max_concurrency, ep_store_max_readers and, ep_store_max_readwrite stats. The dispatcher stats will show the read-only dispatcher when it's available.
121
Couchbase Architecture
CMD_STOP_PERSISTENCE CMD_START_PERSISTENCE CMD_SET_FLUSH_PARAM CMD_SET_VBUCKET CMD_GET_VBUCKET CMD_DEL_VBUCKET CMD_START_REPLICATION CMD_STOP_REPLICATION CMD_SET_TAP_PARAM CMD_EVICT_KEY
11.5.1.3.2. Example
The following example tests the server listening on port 11211 on the local host (in this example I've got the stock memcached server running there)
122
Couchbase Architecture
trond@opensolaris> memcapable ascii quit [pass] ascii version [pass] ascii verbosity [pass] ascii set [pass] ascii set noreply [pass] ascii get [pass] ascii gets [pass] ascii mget [pass] ascii flush [pass] ascii flush noreply [pass] ascii add [pass] ascii add noreply [pass] ascii replace [pass] ascii replace noreply [pass] ascii CAS [pass] ascii CAS noreply [pass] ascii delete [pass] ascii delete noreply [pass] ascii incr [pass] ascii incr noreply [pass] ascii decr [pass] ascii decr noreply [pass] ascii append [pass] ascii append noreply [pass] ascii prepend [pass] ascii prepend noreply [pass] ascii stat [pass] binary noop [pass] binary quit [pass] binary quitq [pass] binary set [pass] binary setq [pass] binary flush [pass] binary flushq [pass] binary add [pass] binary addq [pass] binary replace [pass] binary replaceq [pass] binary delete [pass] binary deleteq [pass] binary get [pass] binary getq [pass] binary getk [pass] binary getkq [pass] binary incr [pass] binary incrq [pass] binary decr [pass] binary decrq [pass] binary version [pass] binary append [pass] binary appendq [pass] binary prepend [pass] binary prependq [pass] binary stat [pass] All tests passed
The following example runs the test suite, but prompts the user before each test
trond@opensolaris> memcapable -P ascii quit Press <return> when you are ready? ascii quit [pass] ascii version Press <return> when you are ready? quit
123
Couchbase Architecture
11.5.2.2. getl
getl/unl: Get an item with a lock that has a timeout. It can then be unlocked with either a CAS operation or with an explicit unlock command. Note Note, this documentation and functionality are still under development and may not necessarily match what you see here. Through an extension, Couchbase has a feature allowing individual clients to be sure they have exclusive access to an item for a period of time. This is not yet supported functionality, and should be considered experimental. Thegetlcommand takes two arguments a document ID and an expiration time. If no expiration time is set then the default expiration is 15 seconds.getl also has a maximum expiration time of 29 seconds so any expiration time greater than 29 seconds is automatically changed to 29 seconds. Below are four use cases that describe how getl works. In these use cases we use the set operation as an example of an operation that is blocked when a getl is applied to a document ID. Note however that the operations delete, incr, etc. also will be blocked by getl.
124
Couchbase Architecture
Perform a get on the document ID to verify the document has its new value Attempt to set the document with a new value Perform a get on the document ID to verify the document has the new value 3. getl fails against a locked document ID Set a document ID Perform a getl on the document ID with a timeout of 15 seconds Perform a getl without a timeout Should get "LOCK_ERROR" 4. getl honors an old timeout Set a document ID Perform a getl on the document ID with a timeout of 3600 seconds (Will be set to 29 seconds by Couchbase) Perform a getl on the document ID with a timeout of 5 seconds Wait 5 seconds Perform a set on the document ID with a new value Set should fail with "EXISTS" Wait 24 seconds Perform a set on the document ID with a new value Set should pass Perform a get on the document ID to verify the document has the new value
11.6. Buckets
Buckets are used to compartmentalize data within Couchbase Server and are also used as the basic mechanism used to replicate and duplicate information (if supported). Couchbase Server supports two different bucket types. These are: memcached Buckets The memcached buckets are designed to fully support the core memcached protocol as an in-memory caching solution. The support and functionality is therefore limited to the same functionality as within a standalone memcached implementation. The main features are: Item size is limited to 1 Mbyte. Persistence is not supported. Replication is not supported; data is available only on one node. 125
Couchbase Architecture
Statistics are limited to those directly related to the in-memory nature of the data. Statistics related to persistence, disk I/O and replication/rebalancing are not available. Client setup should use ketama consistent hashing memcached buckets do not use vBuckets, so there is no rebalancing. Couchbase Buckets Couchbase buckets support the full range of Couchbase-specific functionality, including balancing, persistence and replication. The main features are: Item size is limited to 20 Mbyte. Persistence, including data sets larger than the allocated memory size. Replication and rebalancing are fully supported. Full suite of statistics supported. In addition to these overall bucket differences, there are also security and network port differences that enable you to configure and structure the connectivity to the different bucket types differently. Figure 11.3. Couchbase Buckets
126
Couchbase Architecture
There are three bucket interface types that can be be configured: The default Bucket The default bucket is a Couchbase bucket that always resides on port 11211 and is a non-SASL authenticating bucket. When Couchbase is first install this bucket is automatically set up during installation. This bucket may be removed after installation and may also be re-added later, but when re-adding a bucket named "default", the bucket must be place on port 11211 and must be a non-SASL authenticating bucket. A bucket not named default may not reside on port 11211 if it is a non-SASL bucket. The default bucket may be reached with a vBucket aware smart client, an ASCII client or a binary client that doesn't use SASL authentication. Non-SASL Buckets Non-SASL buckets may be placed on any available port with the exception of port 11211 if the bucket is not named "default". Only one Non-SASL bucket may placed on any individual port. These buckets may be reached with a vBucket aware smart client, an ASCII client or a binary client that doesn't use SASL authentication SASL Buckets SASL authenticating Couchbase buckets may only be placed on port 11211 and each bucket is differentiated by its name and password. SASL bucket may not be placed on any other port beside 11211. These buckets can be reached with either a vBucket aware smart client or a binary client that has SASL support. These buckets cannot be reached with ASCII clients.
11.7. vBuckets
Warning For simplicity, in this section we completely ignore Couchbase Server multi-tenancy (or what we have historically called a "bucket," which represents a "virtual couchbase instance" inside a single couchbase cluster). The bucket and vBucket concepts are not to be confused - they are not related. For purposes of this section, a bucket can simply be viewed as synonymous with "a couchbase cluster." A vBucket is defined as the "owner" of a subset of the key space of a Couchbase Server cluster. Every document ID "belongs" to a vBucket. A mapping function is used to calculate the vBucket in which a given document ID belongs. In couchbase, that mapping function is a hashing function that takes a document ID as input and outputs a vBucket identifier. Once the vBucket identifier has been computed, a table is consulted to lookup the server that "hosts" that vBucket. The table contains one row per vBucket, pairing the vBucket to its hosting server. A server appearing in this table can be (and usually is) responsible for multiple vBuckets. The hashing function used by couchbase to map document IDs to vBuckets is configurable - both the hashing algorithm and the output space (i.e. the total number of vBuckets output by the function). Naturally, if the number of vBuckets in the output space of the hash function is changed, then the table which maps vBuckets to Servers must be resized.
127
Couchbase Architecture
After some period of time, there is a need to add a server to the cluster (e.g. to sustain performance in the face of growing application use). Administrator adds Server D to the cluster and the vBucket Map is updated as follows [note: the vBucket-Server map is updated by an internal Couchbase algorithm and that _updated table is transmitted by Couchbase to all cluster participants - servers and proxies] After the addition, a client once again wants to look up (get) the value of document ID . Because the hashing algorithm in this case has not changed 1 Hash(ID) = vB8 as before. The client examines the vBucket-server mapping table and determines Server D now owns vB8. The get operation is sent to Server D.
128
Couchbase Architecture
vices, such as connection pooling. The diagram below shows the flow with a standalone proxy installed on the application server. The memcached client is configured to have just one server in its server list (localhost), so all operations are forwarded to localhost:11211- a port serviced by the proxy. The proxy hashes the key to a vBucket, looks up the host server in the vBucket table, and then sends the operation to the appropriate couchbase server (Server A in this case) on port 11210.
129
curr_items: curr_items_tot: ep_warmed_up: ep_warmup: ep_warmup_dups: ep_warmup_oom: ep_warmup_thread: ep_warmup_time: And when it is complete:
130
Monitoring Couchbase
ep_warmup: ep_warmup_dups: ep_warmup_oom: ep_warmup_thread: ep_warmup_time Table 12.1. Stats Stat curr_items curr_items_tot
Description The number of items currently active on this node. During warmup, this will be 0 until complete The total number of items this node knows about (active and replica). During warmup, this will be increasing and should match ep_warmed_up The number of items retrieved from disk. During warmup, this should be increasing. The number of duplicate items found on disk. Ideally should be 0, but a few is not a problem How many times the warmup process received an Out of Memory response from the server while loading data into RAM The status of the warmup thread. Can be either running or complete How long the warmup thread was running for. During warmup this number should be increasing, when complete it will tell you how long the process took
ep_warmup_thread ep_warmup_time
131
Monitoring Couchbase
If a request comes in for an item that is on disk, it will preempt the writing process in order to serve this read.
The most commonly needed statistics are surfaced through the Web Console and have descriptions there and in the associated documentation. Software developers and system administrators wanting lower level information have it available through the stats interface.
132
Monitoring Couchbase
There are seven commands available through the stats interface: stats(referred to as 'all') dispatcher hash tap timings vkey reset
The first entry, dispatcher, monitors the process responsible for disk access. The second entry is a non-IO (non disk) dispatcher. There may also be a ro_dispatcher dispatcher present if the engine is allowing concurrent reads and writes. When a task is actually running on a given dispatcher, the "runtime" tells you how long the current task has been running. Newer versions will show you a log of recently run dispatcher jobs so you can see what's been happening.
133
Monitoring Couchbase
$ echo "stats proxy" | nc localhost 11211 STAT basic:version 1.6.0 STAT basic:nthreads 5 ... STAT proxy_main:conf_type dynamic STAT proxy_main:behavior:cycle 0 STAT proxy_main:behavior:downstream_max 4 STAT proxy_main:behavior:downstream_conn_max 0 STAT proxy_main:behavior:downstream_weight 0 ... STAT proxy_main:stats:stat_configs 1 STAT proxy_main:stats:stat_config_fails 0 STAT proxy_main:stats:stat_proxy_starts 2 STAT proxy_main:stats:stat_proxy_start_fails 0 STAT proxy_main:stats:stat_proxy_existings 0 STAT proxy_main:stats:stat_proxy_shutdowns 0 STAT 11211:default:info:port 11211 STAT 11211:default:info:name default ... STAT 11211:default:behavior:downstream_protocol 8 STAT 11211:default:behavior:downstream_timeout 0 STAT 11211:default:behavior:wait_queue_timeout 0 STAT 11211:default:behavior:time_stats 0 STAT 11211:default:behavior:connect_max_errors 0 STAT 11211:default:behavior:connect_retry_interval 0 STAT 11211:default:behavior:front_cache_max 200 STAT 11211:default:behavior:front_cache_lifespan 0 STAT 11211:default:behavior:front_cache_spec STAT 11211:default:behavior:front_cache_unspec STAT 11211:default:behavior:key_stats_max 4000 STAT 11211:default:behavior:key_stats_lifespan 0 STAT 11211:default:behavior:key_stats_spec STAT 11211:default:behavior:key_stats_unspec STAT 11211:default:behavior:optimize_set STAT 11211:default:behavior:usr default ... STAT 11211:default:pstd_stats:num_upstream 1 STAT 11211:default:pstd_stats:tot_upstream 2 STAT 11211:default:pstd_stats:num_downstream_conn 1 STAT 11211:default:pstd_stats:tot_downstream_conn 1 STAT 11211:default:pstd_stats:tot_downstream_conn_acquired 1 STAT 11211:default:pstd_stats:tot_downstream_conn_released 1 STAT 11211:default:pstd_stats:tot_downstream_released 2 STAT 11211:default:pstd_stats:tot_downstream_reserved 1 STAT 11211:default:pstd_stats:tot_downstream_reserved_time 0 STAT 11211:default:pstd_stats:max_downstream_reserved_time 0 STAT 11211:default:pstd_stats:tot_downstream_freed 0 STAT 11211:default:pstd_stats:tot_downstream_quit_server 0 STAT 11211:default:pstd_stats:tot_downstream_max_reached 0 STAT 11211:default:pstd_stats:tot_downstream_create_failed 0 STAT 11211:default:pstd_stats:tot_downstream_connect 1 STAT 11211:default:pstd_stats:tot_downstream_connect_failed 0 STAT 11211:default:pstd_stats:tot_downstream_connect_timeout 0 STAT 11211:default:pstd_stats:tot_downstream_connect_interval 0 STAT 11211:default:pstd_stats:tot_downstream_connect_max_reached 0 ... END
134
The primary source for run-time logging information is the Couchbase Server Web Console. Run-time logs are automatically set up and started during the installation process. However, the Couchbase Server gives you access to lower-level
135
Troubleshooting
logging details if needed for diagnostic and troubleshooting purposes. Log files are stored in a binary format in the logs directory under the Couchbase installation directory. You must use browse_logs to extract the log contents from the binary format to a text file.
Look for ns_log in the output to see anything that would have been shown via the Couchbase Server Web Console.
You can redirect the output of this batch file to a text file that you can save for later viewing or email to Couchbase Technical Support. The following example redirects output to a text file named nslogs.txt.
shell> cbbrowse_logs > /tmp/nslogs.txt
136
Troubleshooting
If you are having problems starting Couchbase Server on Linux for the first time, there are two very common causes of this that are actually quite related. When the /etc/init.d/couchbase-server script runs, it tries to set the file descriptor limit and core file size limit:
shell> ulimit -n 10240 ulimit -c unlimited
Depending on the defaults of your system, this may or may not be allowed. If Couchbase Server is failing to start, you can look through the logs (see Section 13.3, Log File Entries) and pick out one or both of these messages:
ns_log: logging ns_port_server:0:Port server memcached on node 'ns_1@127.0.0.1' exited with status 71. Restarting. Messages: failed to set rlimit for open files. Try running as root or requesting smaller maxconns value.
The resolution to these is to edit the /etc/security/limits.conf file and add these entries:
couchbase hard nofile 10240 couchbase hard core unlimited
137
Appendix A. FAQs
What kind of client do I use with couchbase? couchbase is compatible with existing memcached clients. If you have a memcached client already, you can just point it at couchbase. Regular testing is done with spymemcached(the Java client), libmemcached and fauna (Ruby client). See the Client Libraries page What is a "vbucket"? An overview from Dustin Sallings is presented here: memcached vBuckets What is a TAP stream? A TAP stream is a when a client requests a stream of item updates from the server. That is, as other clients are requesting item mutations (for example, SET's and DELETE's), a TAP stream client can "wire-tap" the server to receive a stream of item change notifications. When a TAP stream client starts its connection, it may also optionally request a stream of all items stored in the server, even if no other clients are making any item changes. On the TAP stream connection setup options, a TAP stream client may request to receive just current items stored in the server (all items until "now"), or all item changes from now onwards into in the future, or both. Trond Norbye's written a blog post about the TAP interface. See Blog Entry. What ports does couchbase Server need to run on? The following TCP ports should be available: 8091 GUI and REST interface 11211 Server-side Moxi port for standard memcached client access 11210 native couchbase data port 21100 to 21199 inclusive for dynamic cluster communication What hardware and platforms does couchbase Server support? Couchbase Server supports Red Hat (and CentOS) versions 5 starting with update 2, Ubuntu 9 and and Windows Server 2008 (other versions have been shown to work but are not being specifically tested). There are both 32-bit and 64-bit versions available. Community support for Mac OS X is available. Future releases will provide support for additional platforms. How can I get couchbase on (this other OS)? The couchbase source code is quite portable and is known to have been built on several other UNIX and Linux based OSs. See Consolidated sources. Can I query couchbase by something other than the key name? Not directly. It's possible to build these kinds of solutions atop TAP. For instance, via Cascading it is possible to stream out the data, process it with Cascading, then create indexes in Elastic Search. What is the maximum item size in couchbase? The default item size for couchbase buckets is 20 MBytes. The default item size for memcached buckets is 1 MByte. Both are tunable, though not through the system console.
138
FAQs
How do I change the disk path? Use the couchbase command-line tool:
shell> couchbase node-init -c cluster_IP:8091 -u \ username-p password--node-init-data-path=/tmp
Why are some clients getting different results than others for the same requests? This should never happen in a correctly-configured couchbase cluster, since couchbase ensures a consistent view of all data in a cluster. However, if some clients can't reach all the nodes in a cluster (due to firewall or routing rules, for example), it is possible for the same key to end up on more than one cluster node, resulting in inconsistent duplication. Always ensure that all cluster nodes are reachable from every smart client or client-side moxi host.
139
Refer to the RedHat RPM documentation for more information about uninstalling packages using RPM. You may need to delete the data files associated with your installation. The default installation location is /opt. If you selected an alternative location for your data files, you will need to separately delete each data directory from your system.
Refer to the Ubuntu documentation for more information about uninstalling packages using dpkg. You may need to delete the data files associated with your installation. The default installation location is /opt. If you selected an alternative location for your data files, you will need to separately delete each data directory from your system.
140
C.1. Release Notes for Couchbase Server 1.8.0 GA (23 January 2012)
Couchbase Server 1.8 is the updated and rebranded release of Membase Server. Important In line with the rebranding and name changes, there are some significant changes to the following areas of this release: Directory Changes Couchbase Server is now installed into a couchbase directory, for example on Linux the default installation directory is /opt/couchbase instead of /opt/membase. During an upgrade, the location of your data files will not be modified. Command Changes The name of many of the core commands provided with Couchbase Server have been renamed, with the existing scripts deprecated. For example, the backup command mbbackup in Membase Server is now called cbbackup. See the full release note entry below for more detailed information. New Features and Feature Changes in 1.8.0 The membase bucket type has been deprecated. The couchbase bucket type should be used instead to select the balanced, persistent and replicated buckets supported by a Couchbase cluster. For more information, see Section 11.6, Buckets. Tags: deprecation Internet Explorer 7 as a supported browser for the Couchbase Web Admin Console is now deprecated, and support will be removed entirely in a future release. Tags: deprecation Allow disk write queue upper limit to be modified at runtime. Issues: MB-4418 The SYNC protocol command has been deprecated. Applications and solutions should no longer use the SYNC command. The command will be removed entirely in a future version. Tags: deprecation Ubuntu 9.10 is no longer a supported platform. Tags: removal The following command-line tools have been deprecated and replaced with the corresponding tool as noted in the table below.
141
Release Notes
Deprecated Tool membase mbadm-online-restore mbadm-online-update mbadm-tap-registration mbbackup-incremental mbbackup-merge-incremental mbbackup mbbrowse_logs mbcollect_info mbdbconvert mbdbmaint mbdbupgrade mbdumpconfig.escript mbenable_core_dumps.sh mbflushctl mbrestore mbstats mbupgrade mbvbucketctl
Replacement Tool couchbase-cli cbadm-online-restore cbadm-online-update cbadm-tap-registration cbbackup-incremental cbbackup-merge-incremental cbbackup cbbrowse_logs cbcollect_info cbdbconvert cbdbmaint cbdbupgrade cbdumpconfig.escript cbenable_core_dumps.sh cbflushctl cbrestore cbstats cbupgrade cbvbucketctl
Using a deprecated tool will result in a warning message that the tool is deprecated and will no longer be supported. For more information, see Chapter 7, Couchbase Command-line Interface. Tags: deprecation Fixes in 1.8.0 Rebalancing process might hang when adding or removing nodes in a large cluster where client load is running. Issues: MB-4517, MB-4490 Selecting the master node during rolling upgrade where there are different membase/couchbase servers in the cluster. Issues: MB-4592 The ep_num_not_my_vbuckets stat doesn't reset. Issues: MB-3852 Decreased memory usage by coalesce checkpoints in the replica side if item persistence is slower. Server will maintain only two replica checkpoints for each vbucket. Issues: MB-4578 Installation of Membase on Amazon Linux would fail.
142
Release Notes
Issues: MB-3419 Unable to create a default bucket using membase-cli. Issues: MB-4453 Fixed a case where there are two replicas where replication cursor would get stuck and slave node didn't replicate the data into the other node in the replication chain. Issues: MB-4461 Installer dependencies on RHEL 5.4 have been updated. Issues: MB-4561
143
Appendix D. Licenses
This documentation and associated software is subject to the following licenses.
As used in this Agreement, "Software" means the object code version of the applicable elastic data management server software provided by Couchbase, Inc.
144
Licenses
2. Support. Couchbase, Inc. will provide Licensee with access to, and use of, the Couchbase, Inc. support forum available at the following URL: http://forums.membase.org. Couchbase, Inc. may, at its discretion, modify, suspend or terminate support at any time upon notice to Licensee. 3. Warranty Disclaimer and Limitation of Liability. THE SOFTWARE IS PROVIDED "AS IS," WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL COUCHBASE INC. OR THE AUTHORS OR COPYRIGHT HOLDERS IN THE SOFTWARE BE LIABLE FOR ANY CLAIM, DAMAGES (INCLUDING, WITHOUT LIMITATION, DIRECT, INDIRECT OR CONSEQUENTIAL DAMAGES) OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
145
Licenses
4. Support. Couchbase Inc. will provide Licensee with: (a) periodic Software updates to correct known bugs and errors to the extent Couchbase Inc. incorporates such corrections into the free edition version of the Software; and (b) access to, and use of, the Couchbase Inc. support forum available at the following URL: http://forums.membase.org. Licensee must have Licensed Servers at the same level of Support Services for all instances in a production deployment running the Software. Licensee must also have Licensed Servers at the same level of Support Services for all instances in a development and test environment running the Software, although these Support Services may be at a different level than the production Licensed Servers. Couchbase Inc. may, at its discretion, modify, suspend or terminate support at any time upon notice to Licensee. 5. Records Retention and Audit. Licensee shall maintain complete and accurate records to permit Couchbase Inc. to verify the number of Licensed Servers used by Licensee hereunder. Upon Couchbase Inc.'s written request, Licensee shall: (a) provide Couchbase Inc. with such records within ten (10) days; and (b) will furnish Couchbase Inc. with a certification signed by an officer of Licensee verifying that the Software is being used pursuant to the terms of this Agreement. Upon at least thirty (30) days prior written notice, Couchbase Inc. may audit Licensee's use of the Software to ensure that Licensee is in compliance with the terms of this Agreement. Any such audit will be conducted during regular business hours at Licensee's facilities and will not unreasonably interfere with Licensee's business activities. Licensee will provide Couchbase Inc. with access to the relevant Licensee records and facilities. If an audit reveals that Licensee has used the Software in excess of the authorized Licensed Servers, then (i) Couchbase Inc. will invoice Licensee, and Licensee will promptly pay Couchbase Inc., the applicable licensing fees for such excessive use of the Software, which fees will be based on Couchbase Inc.'s price list in effect at the time the audit is completed; and (ii) Licensee will pay Couchbase Inc.'s reasonable costs of conducting the audit. 6. Confidentiality. Licensee and Couchbase Inc. will maintain the confidentiality of Confidential Information. The receiving party of any Confidential Information of the other party agrees not to use such Confidential Information for any purpose except as necessary to fulfill its obligations and exercise its rights under this Agreement. The receiving party shall protect the secrecy of and prevent disclosure and unauthorized use of the disclosing party's Confidential Information using the same degree of care that it takes to protect its own confidential information and in no event shall use less than reasonable care. The terms of this Confidentiality section shall survive termination of this Agreement. Upon termination or expiration of this Agreement, the receiving party will, at the disclosing party's option, promptly return or destroy (and provide written certification of such destruction) the disclosing party's Confidential Information. 7. Disclaimer of Warranty. THE SOFTWARE AND ANY SERVICES PROVIDED HEREUNDER ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. COUCHBASE INC. DOES NOT WARRANT THAT THE SOFTWARE OR THE SERVICES PROVIDED HEREUNDER WILL MEET LICENSEE'S REQUIREMENTS, THAT THE SOFTWARE WILL OPERATE IN THE COMBINATIONS LICENSEE MAY SELECT FOR USE, THAT THE OPERATION OF THE SOFTWARE WILL BE ERROR-FREE OR UNINTERRUPTED OR THAT ALL SOFTWARE ERRORS WILL BE CORRECTED. COUCHBASE INC. HEREBY DISCLAIMS ALL WARRANTIES, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, TITLE, AND ANY WARRANTIES ARISING OUT OF COURSE OF DEALING, USAGE OR TRADE. 8. Agreement Term and Termination. The term of this Agreement shall begin on the Effective Date and will continue until terminated by the parties. Licensee may terminate this Agreement for any reason, or for no reason, by providing at least ten (10) days prior written notice to Couchbase Inc. Couchbase Inc. may terminate this Agreement if Licensee materially breaches its obligations hereunder and, where such breach is curable, such breach remains uncured for ten (10) days following written notice of the breach. Upon termination of this Agreement, Licensee will, at Couchbase Inc.'s option, promptly return or destroy (and provide written certification of such destruction) the applicable Software and all copies and portions thereof, in all forms and types of media. The following sections will survive termination or expiration of this Agreement: Sections 2, 3, 6, 7, 8, 9, 10 and 11. 9. Limitation of Liability. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT WILL COUCHBASE INC. OR ITS LICENSORS BE LIABLE TO LICENSEE OR TO ANY THIRD PARTY FOR ANY INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL OR EXEMPLARY DAMAGES OR FOR THE COST OF PROCURING SUBSTITUTE PRODUCTS OR SERVICES ARISING OUT OF OR IN ANY WAY RELATING TO OR IN CONNECTION WITH THIS AGREEMENT OR THE USE OF OR INABILITY TO USE
146
Licenses
THE SOFTWARE OR DOCUMENTATION OR THE SERVICES PROVIDED BY COUCHBASE INC. HEREUNDER INCLUDING, WITHOUT LIMITATION, DAMAGES OR OTHER LOSSES FOR LOSS OF USE, LOSS OF BUSINESS, LOSS OF GOODWILL, WORK STOPPAGE, LOST PROFITS, LOSS OF DATA, COMPUTER FAILURE OR ANY AND ALL OTHER COMMERCIAL DAMAGES OR LOSSES EVEN IF ADVISED OF THE POSSIBILITY THEREOF AND REGARDLESS OF THE LEGAL OR EQUITABLE THEORY (CONTRACT, TORT OR OTHERWISE) UPON WHICH THE CLAIM IS BASED. IN NO EVENT WILL COUCHBASE INC.'S OR ITS LICENSORS' AGGREGATE LIABILITY TO LICENSEE, FROM ALL CAUSES OF ACTION AND UNDER ALL THEORIES OF LIABILITY, EXCEED ONE THOUSAND DOLLARS (US $1,000). The parties expressly acknowledge and agree that Couchbase Inc. has set its prices and entered into this Agreement in reliance upon the limitations of liability specified herein, which allocate the risk between Couchbase Inc. and Licensee and form a basis of the bargain between the parties. 10.General. Couchbase Inc. shall not be liable for any delay or failure in performance due to causes beyond its reasonable control. Neither party will, without the other party's prior written consent, make any news release, public announcement, denial or confirmation of this Agreement, its value, or its terms and conditions, or in any manner advertise or publish the fact of this Agreement. Notwithstanding the above, Couchbase Inc. may use Licensee's name and logo, consistent with Licensee's trademark policies, on customer lists so long as such use in no way promotes either endorsement or approval of Couchbase Inc. or any Couchbase Inc. products or services. Licensee may not assign this Agreement, in whole or in part, by operation of law or otherwise, without Couchbase Inc.'s prior written consent. Any attempt to assign this Agreement, without such consent, will be null and of no effect. Subject to the foregoing, this Agreement will bind and inure to the benefit of each party's successors and permitted assigns. If for any reason a court of competent jurisdiction finds any provision of this Agreement invalid or unenforceable, that provision of the Agreement will be enforced to the maximum extent permissible and the other provisions of this Agreement will remain in full force and effect. The failure by either party to enforce any provision of this Agreement will not constitute a waiver of future enforcement of that or any other provision. All waivers must be in writing and signed by both parties. All notices permitted or required under this Agreement shall be in writing and shall be delivered in person, by confirmed facsimile, overnight courier service or mailed by first class, registered or certified mail, postage prepaid, to the address of the party specified above or such other address as either party may specify in writing. Such notice shall be deemed to have been given upon receipt. This Agreement shall be governed by the laws of the State of California, U.S.A., excluding its conflicts of law rules. The parties expressly agree that the UN Convention for the International Sale of Goods (CISG) will not apply. Any legal action or proceeding arising under this Agreement will be brought exclusively in the federal or state courts located in the Northern District of California and the parties hereby irrevocably consent to the personal jurisdiction and venue therein. Any amendment or modification to the Agreement must be in writing signed by both parties. This Agreement constitutes the entire agreement and supersedes all prior or contemporaneous oral or written agreements regarding the subject matter hereof. To the extent there is a conflict between this Agreement and the terms of any "shrinkwrap" or "clickwrap" license included in any package, media, or electronic version of Couchbase Inc.furnished software, the terms and conditions of this Agreement will control. Each of the parties has caused this Agreement to be executed by its duly authorized representatives as of the Effective Date. Except as expressly set forth in this Agreement, the exercise by either party of any of its remedies under this Agreement will be without prejudice to its other remedies under this Agreement or otherwise. The parties to this Agreement are independent contractors and this Agreement will not establish any relationship of partnership, joint venture, employment, franchise, or agency between the parties. Neither party will have the power to bind the other or incur obligations on the other's behalf without the other's prior written consent. 11.Definitions. Capitalized terms used herein shall have the following definitions: "Confidential Information" means any proprietary information received by the other party during, or prior to entering into, this Agreement that a party should know is confidential or proprietary based on the circumstances surrounding the disclosure including, without limitation, the Software and any non-public technical and business information. Confidential Information does not include information that (a) is or becomes generally known to the public through no fault of or breach of this Agreement by the receiving party; (b) is rightfully known by the receiving party at the time of disclosure without an obligation of confidentiality; (c) is independently developed by the receiving party without use of the disclosing party's Confidential Information; or (d) the receiving party rightfully obtains from a third party without restriction on use or disclosure. "Documentation" means any technical user guides or manuals provided by Couchbase Inc. related to the Software. "Licensed Server" means an instance of the Software running on one (1) operating system. Each operating system instance may be
147
Licenses
running directly on physical hardware, in a virtual machine, or on a cloud server. "Couchbase" means Couchbase, Inc. "Couchbase Website" means www.couchbase.com. "Software" means the object code version of the applicable elastic data management server software provided by Couchbase Inc. and ordered by Licensee during the ordering process on the Couchbase Website. If you have any questions regarding this Agreement, please contact sales@couchbase.com.
148