Everand Logo

Welcome to Everand!

  • Discover millions of ebooks, audiobooks, and so much more with a free trial

    Only $11.99/month after trial. Cancel anytime.

    Implementing Cloud Storage with OpenStack Swift
    Implementing Cloud Storage with OpenStack Swift
    Implementing Cloud Storage with OpenStack Swift
    Ebook301 pages1 hour

    Implementing Cloud Storage with OpenStack Swift

    By Amar Kapadia, Kris Rajana and Sreedhar Varma

    ()

    Read preview

    About this ebook

    This tutorialbased book has a stepbystep approach for each topic, ensuring it is thoroughly covered and easy to follow. If you are an IT architect or administrator who wants to enter the world of cloud storage using OpenStack Swift, then this book is ideal for you. Whether your job is to build, manage, or use OpenStack Swift, this book is an excellent way to move your career ahead.
    LanguageEnglish
    PublisherPackt Publishing
    Release dateMay 17, 2014
    ISBN9781782168065
    Implementing Cloud Storage with OpenStack Swift

    Related to Implementing Cloud Storage with OpenStack Swift

    Related ebooks

    Programming For You

    View More
    View More

    Related podcast episodes

    Related articles

    • Article

      The State Of Linux Security

      Apr 7, 2020

      The State Of Linux Security
      1 min read

    • Article

      Exploding Firewalls

      Feb 11, 2020

      Exploding Firewalls
      8 min read

    • Article

      Client-free Remote Desktop Access For All

      Sep 24, 2019

      Guacamole is a open-source software project consisting of a client and server component that work together to enable remote connectivity to server environments from a web browser. In fact, this open-source project goes back as far as 2010, having spe

      9 min read

    • Article

      QEMU and KVM

      Mar 10, 2020

      QEMU and KVM
      4 min read

    • Article

      Ubiquiti Networks UniFiU6-LR

      Jan 6, 2022

      Ubiquiti Networks UniFiU6-LR
      3 min read

    • Article

      KAFKA Build Utilities With The Kafka Server

      Jul 2, 2019

      Nowadays, quite a few data architectures involve both a database and Apache Kafka, which is a distributed streaming platform and the subject of this tutorial. You can also find Kafka described as a publish-subscribe message system, which is a fancy w

      7 min read

    • Article

      Basic Concepts

      Jul 2, 2019

      A messaging system such as Kafka enables you to send messages between processes, applications and servers. Applications connect to Kafka to send or get data. Strictly speaking, a Kafka ‘topic’ is a unit of storage in Kafka: data in Kafka is stored in

      1 min read

    • Article

      Monitor Systems And Docker Deployments

      Jun 30, 2020

      Welcome to Netdata, software for distributed real-time performance and health monitoring of UNIX machines. Don’t you dare turn that page! A key advantage of Netdata is that it collects all of its metrics without introducing too much load on to the Li

      8 min read

    • Article

      How To Develop A RESTful Client In Go

      Nov 16, 2021

      Mihalis Tsoukalos is a systems engineer and technical writer. He’s the author of Go Systems Programming and Mastering Go. You can reach him at @mactsouk. The subject of this month’s tutorial is RESTful services. In particular, you’re going to learn h

      9 min read

    • Article

      Metrics & Visuals In Go

      Nov 17, 2020

      Mihalis Tsoukalos is a DataOps engineer and a technical writer. He’s the author of Go Systems Programming and Mastering Go, 2nd edition. The subject of this tutorial is two-fold. First, it’s about creating a Go application that exports metrics to P

      7 min read

    • Article

      How We Tested…

      Oct 19, 2021

      We wrote our content using Markdown, creating five sites. The content was all placed in a local directory, though the files are all synchronised to Git repositories. Static site generators are actually fairly simple collections of scripts that take y

      1 min read

    • Article

      Network Monitoring Software

      Dec 10, 2020

      Network Monitoring Software
      3 min read

    • Article

      Also Consider…

      Nov 17, 2020

      There are quite a few pen-testing distros out there and we’ve already covered the ones that are actively maintained in this Roundup. A better bet instead would be to look for pen-testing tools inside security-focused distros that might also include t

      1 min read

    • Article

      How Netflix’s OTT Architecture Functions?

      May 1, 2022

      With so many OTT platforms in the market today, Netflix has managed to capture a majority of the audience on a global scale. Netflix has become the go-to source of so much entertainment for consumers in less than 20 years. It can even be said that Ne

      4 min read

    • Article

      Poisoning The Well

      Jan 11, 2022

      Poisoning The Well
      4 min read

    • Article

      Newsdesk

      Nov 14, 2023

      Newsdesk
      8 min read

    • Article

      Personal Cloud Servers

      Sep 19, 2023

      Personal Cloud Servers
      1 min read

    • Article

      The Verdict GUI Backup Tools

      Jan 11, 2022

      The Verdict GUI Backup Tools
      2 min read

    • Article

      Nourishment For The Soul

      Sep 20, 2022

      Nourishment For The Soul
      9 min read

    • Article

      Enterprise Soaring Success

      Aug 27, 2019

      Enterprise Soaring Success
      7 min read

    • Article

      How We Tested…

      Jan 12, 2021

      You’ll find these applications in the software repositories of most desktop distributions, even if the featured version is not the latest. Some programs provide Snap packages, and others provide installable binaries for RPM- and DEB-based distributio

      1 min read

    Related categories

    Reviews for Implementing Cloud Storage with OpenStack Swift

    0 ratings

    0 ratings0 reviews

    What did you think?

    Tap to rate

    Review must be at least 10 words

      Book preview

      Implementing Cloud Storage with OpenStack Swift - Amar Kapadia

      Table of Contents

      Implementing Cloud Storage with OpenStack Swift

      Credits

      Foreword

      About the Authors

      About the Reviewers

      www.PacktPub.com

      Support files, eBooks, discount offers and more

      Why Subscribe?

      Free Access for Packt account holders

      Preface

      What this book covers

      What you need for this book

      Who this book is for

      Conventions

      Reader feedback

      Customer support

      Downloading the example code

      Errata

      Piracy

      Questions

      1. Cloud Storage: Why Can't I be like Google?

      Elements of cloud storage

      Reduced TCO

      Unlimited scalability

      Elastic

      On-demand

      Universal access

      Multitenanancy

      Use cases

      Application impact

      Cloud gateways

      Object storage

      OpenStack Swift

      Summary

      2. OpenStack Swift Architecture

      The logical organization of objects

      The Swift implementation

      Key architectural principles

      Physical data organization

      Data path software servers

      A day in the life of a create operation

      A day in the life of a read operation

      A day in the life of an update operation

      A day in the life of a delete operation

      Postprocessing software components

      Replication

      Updaters

      Auditors

      Other processes

      Inline middleware options

      Auth

      Logging

      Other modules

      Additional features

      Large object support

      Metadata

      Multirange support

      CORS

      Server-side copies

      Cluster health

      Summary

      3. Installing OpenStack Swift

      Hardware planning

      Server setup and network configuration

      Preinstallation steps

      Downloading and installing Swift

      Setting up storage server nodes

      Installing services

      Formatting and mounting hard disks

      RSYNC and RSYNCD

      Setting up the proxy server node

      The ring setup

      Starting services on all storage nodes

      Multiregion support

      The Keystone service

      Installing MySQL

      Installing Keystone

      Summary

      4. Using Swift

      Installing the clients

      Creating a token using authentication

      Displaying metadata information for an account, container, or object

      Using the Swift Client CLI

      Using cURL

      Using the REST API

      Listing containers

      Using the Swift Client CLI

      Using cURL

      Listing objects in a container

      Using the Swift Client CLI

      Using cURL

      Using the REST API

      Updating the metadata for a container

      Using the Swift Client CLI

      Using the REST API

      Environment variables

      Pseudo-hierarchical directories

      Container ACLs

      Transferring large objects

      Amazon S3 API compatibility

      Accessing Swift using S3 commands

      Accessing Swift using client libraries

      Java

      Python

      Ruby

      Summary

      5. Managing Swift

      Routine management

      Swift cluster monitoring

      Swift Recon

      Swift Informant

      Swift dispersion tools

      StatsD

      Swift metrics

      Logging using rsyslog

      Failure management

      Detecting drive failure

      Handling drive failure

      Handling node failure

      Proxy server failure

      Zone and region failure

      Capacity planning

      Adding new drives

      Adding new storage and proxy servers

      Migrations

      Summary

      6. Choosing the Right Hardware

      The hardware list

      The hardware selection criteria

      Step 1 – choosing the storage server configuration

      Step 2 – determining the region and zone configuration

      Step 3 – choosing the account and container server configuration

      Step 4 – choosing the proxy server configuration

      Step 5 – choosing the network hardware

      Step 6 – choosing the ratios of various server types

      Step 7 – choosing additional networking equipment

      Step 8 – choosing a cloud gateway

      Additional selection criteria

      The vendor selection strategy

      Branded hardware

      Commodity hardware

      Summary

      7. Tuning Your Swift Installation

      Performance benchmarking

      Hardware tuning

      Software tuning

      The ring considerations

      Data path software tuning

      Postprocessing software tuning

      Additional tuning parameters

      Summary

      8. Additional Resources

      Use cases

      Service providers

      Web 2.0

      Enterprises

      Operating systems used for OpenStack implementations

      Virtualization used for OpenStack implementations

      Provisioning and distribution tools

      Monitoring and graphing tools

      Additional information

      Summary

      A. Advanced Features

      Commands

      List

      Examples

      Stat

      Examples

      Post

      Examples

      Upload

      Examples

      Download

      Examples

      Delete

      Examples

      Index

      Implementing Cloud Storage with OpenStack Swift

      Implementing Cloud Storage with OpenStack Swift

      Copyright © 2014 Packt Publishing

      All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

      Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

      Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

      First published: May 2014

      Production Reference: 1090514

      Published by Packt Publishing Ltd.

      Livery Place

      35 Livery Street

      Birmingham B3 2PB, UK.

      ISBN 978-1-78216-805-8

      www.packtpub.com

      Cover Image by Seenivasan Kumaravel (<kseenivasan@hotmail.com>)

      Credits

      Authors

      Amar Kapadia

      Sreedhar Varma

      Kris Rajana

      Reviewers

      Juan J. Martínez

      Sriram Subramanian

      Alex Yang

      Commissioning Editor

      Kartikey Pandey

      Acquisition Editor

      Harsha Bharwani

      Content Development Editor

      Priyanka S

      Technical Editor

      Faisal Siddiqui

      Copy Editors

      Janbal Dharmaraj

      Sayanee Mukherjee

      Aditya Nair

      Alfida Paiva

      Project Coordinator

      Puja Shukla

      Proofreaders

      Maria Gould

      Ameesha Green

      Paul Hindle

      Indexer

      Mariammal Chettiyar

      Graphics

      Ronak Dhruv

      Abhinash Sahu

      Production Coordinator

      Alwin Roy

      Cover Work

      Alwin Roy

      Foreword

      I have worked with Amar in the OpenStack San Francisco Bay Area user group and the Entertainment Technology Council cloud effort over the past year. Amar is part of the larger Seagate and Evault effort to transform a manufacturer and product commodity vendor. He has been working with Swift for around 3 years and has deep understanding of what makes it tick.

      The authors, like myself, have been lured into the great experiment that is OpenStack and it has changed our careers for the better. Seagate, EVault, and Vedams are working to provide higher-level services like key value store disks and API implementations that provide novel solutions for software defined infrastructure problems. The authors have produced an excellent operational guide that will benefit anyone interested in understanding Swift.

      Object storage predates the implementations of Swift and S3. It originated in the universities and spread to Internet based companies such as Yahoo and Google. Internet companies require vast amounts of eventually consistent data. As the business of search changed the way the technology industry thought about services, more uses for object stores were found. Swift was publicly released about a year after Rackspace started working on the CloudFiles replacement in August 2009. The development was born out of a tight group that blended development and operations expertise. Rackspace needed massively scalable storage that they had control over the implementation and the code base.

      We are very fortunate that at the time Swift was being released to the world as a new open source project in the summer of 2010, NASA engineers were finishing up their rewrite of the virtual server software Eucalyptus. Nova, as the NASA project became known, had an engineering effort that was so similar to Swift, that both teams were stunned. NASA engineer, Joshua McKenty, noted, We were using the same tools. We had made the same language decisions. We got the two development teams together — none of whom had ever met each other — and we both said: 'Wow, you just wrote the code that we were going to write.' - http://www.wired.com/2012/04/openstack-histor/.

      It was more than just luck that the two teams were developing similar code in a similar fashion. Similar minds came to similar conclusions. I first met Joshua McKenty, Jesse Andrews, and Vishvananda Ishaya, in May 2010. We were all at the MSST storage conference in Incline Village, NV. They were debating over the few nights available to us of what storage to use for their project. I provided some backdrop for Yahoo's storage options. Many drinks later and a few days, it seemed that they were no closer to deciding between the choices available at the time. Just a month later, Rackspace and NASA were to begin down the road of making history.

      Swift is an open source private object store for companies seeking to be part of the open source software defined infrastructure movement. Storage APIs breed innovative new ways to develop and operate. Lifting the restrictions of POSIX interfaces has been cathartic. This remote storage model breaks down, however, when you factor in latency and the network cost of repatriating your data. As John Dickenson states, Storage is key. It always grows. It is incredibly sticky. It is very hard to move around. - https://www.youtube.com/watch?v=Dd7wmJCDh4w.

      Swift fills this gap of local, simple object storage. It is open source, eventually consistent, supports ACLs, large objects, failure domains, and both Swift and S3 APIs. Using simple, inexpensive servers it drives the cost down below many other vendor backed solutions. While listing off features and direct benefits is a fun exercise, the hidden benefits of using Swift are the most important. Once you start down the path of using Swift and other OpenStack projects, you are on your way to automating your infrastructure.

      To properly operate distributed computing software like Swift; you will need to embrace automating your infrastructure using DevOps techniques. DevOps simply means your operations engineers must have development abilities. This is not a new idea, but making it

      Enjoying the preview?
      Page 1 of 1