Automating Development Environments With Vagrant and Puppet TH PDF

Your Situation
This could be quite varied, you could be:

A solo developer looking for a fast/easy way to have a local dev environment that resembles your
production environment (say you develop on OS X, but are deploying to an infrastructure running
some distribution of Linux. As a bonus, there is also an easy way to deploy to Amazons EC2 if you
have a solid setup locally.
1.
A member of a team, where everyone has their own development style and want to avoid the
headaches of cross-platform support.
2.
Someone who normally sets up servers in a third-party hosting environment, but you want to test
your deployment without paying a bunch of money in wasted servers (this is where I am!).
3.
What is vagrant?
Vagrant (http://vagrantup.com/) is essentially a wrapper around a variety of virtual machine
providers. If you have ever used Make to build a piece of software, it is kind of like that except with
virtual machines. It provides a single command that uniformly creates, provisions, destroys, and
connects to machines. You can use many different VM providers, but I will be using VirtualBox because
it is free and easy to use. Usually it is a pain in the butt to create a virtual machine, install the operating
system, etc. Vagrant makes it super easy, and there are lots of premade boxes for you to use (more on
this later).
What is Puppet?
Puppet (http://puppetlabs.com) is an infrastructure automation tool and we are going to use it to take
the hard work out of setting up our systems. We can do this because lots of people have put a ton of
effort into writing modules that we can use. This wont be a tutorial on Puppet, but it will go over the
basics so that you can use modules that other people have written.
But l don't like/use Puppet!
That is fine. Thankfully Vagrant is flexible in its provisioners and you can read more about the
alternatives (http://docs.vagrantup.com/v2/provisioning/index.html). For simplicity I am just going
Automating Development Environments with Vagrant and... http://blog.kloudless.com/2013/07/01/automating-develo...
2 of 17 11/03/2013 10:25 PM
to cover Bash and Puppet since that is what I am familiar with. The overall process should be the same if
you decide to use Chef (http://www.opscode.com/chef/) or Ansible
(http://www.ansibleworks.com/), but because I dont know much about them, I wont discuss them
further.
Getting Started
lnstallation
Vagrant is a breeze to install: you can read over their installation instructions
(http://docs.vagrantup.com/v2/installation/index.html). Essentially, download the package that is
relevant for your platform of choice and install it in the way you would normally install a package.
You will also need to install your virtual machine provider, in my case VirtualBox
(https://www.virtualbox.org/).
First vM
Once it is installed, you will want to create a new project directory and initialize it.
$ mkd1r ~1varanl
$ cd ~1varanl
$ varanl 1n1l
This will create a new directory and get it ready with a file called Vagrantfile
(http://docs.vagrantup.com/v2/vagrantfile/index.html) which will contain all the information that
Vagrant needs to manage your dev environments.
In that file there will be a bunch of comments about the different things you can put in that file. For now
we just care about configuring a base box (http://docs.vagrantup.com/v2/boxes.html) (which is just
a virtual machine image) and some other machine properties. I ended up with a file that looked
something like this:
3 of 17 11/03/2013 10:25 PM
# -*- mode. ruby -*-
# v1. sel il=ruby .
varanl.coni1ure{"2"} do ]coni1]
# Every varanl v1rlual env requ1res a box lo bu1ld oii oi
coni1.vm.box = "puppellabs-prec1se64"
coni1.vm.box_url = "hllp.11puppel-varanl-boxes.puppellabs.com1ubunlu-serv
coni1.vm.hoslname = "developmenl.kloudless.vm"
coni1.vm.nelWork .pr1vale_nelWork, 1p. "192.168.33.10"
coni1.vm.nelWork .iorWarded_porl, uesl. 80, hosl. 8080
# v1rlualBox Spec1i1c Cuslom1zal1on
coni1.vm.prov1der .v1rlualbox do ]vb]
# use vBoxManae lo cuslom1ze lhe vM. Ior example lo chane memory.
vb.cuslom1ze |"mod1iyvm", .1d, "--memory", "1024"]
end
end
Here are the things that we configured so far:
coni1.vm.box: The name of the box that you are bringing up. 1.
coni1.vm.box_url: The location where Vagrant can look to download the box if you dont
already have a copy on your machine.The box that I chose is provided by Puppet Labs and doesnt
have any pre-installed provisioning software. This will be installed later as part of the bootstrapping
process so that it can always be up to date.
2.
coni1.vm.hoslname: The hostname of the VM. 3.
coni1.vm.nelWork .pr1vale_nelWork: The private IP address that the VM will have on the
private VM network.
4.
coni1.vm.nelWork .iorWarded_porl: The port labelled guest on the guest VM will be
accessible on the port labelled host on your machine.
5.
VM memory: I gave my VM 1GB because it seemed like it would be enough (Note: this is provider
dependent).
6.
Once you have the file in place you can create and provision your VM from the same directory:
$ varanl up
4 of 17 11/03/2013 10:25 PM
After that command finishes running, you will have a VM ready for you to connect and start messing
around with. You can access the machine via SSH using:
$ varanl ssh
You will now have a shell on your virtual machine as the varanl user. The user has passwordless
sudo access on the machine, it full fledged Ubuntu 12.04 LTS VM and you can do whatever you want!
This is great and all, but we want to make things more automated, so you will want to exit your SSH
session and get rid of the VM with:
$ varanl deslroy
Automating All the Things
As was talked about earlier, the nice thing about Vagrant is that it is really easy to offload the
configuration to an automated tool, in this case Puppet. Since our VM is pretty bare bones, there is some
extra work that we want to do to prepare it.
Bootstrapping
The box that I chose doesnt by default come with puppet installed on it, this was a deliberate choice to
make sure that I could use the same version I am using in production without having to change the box
all the time. As such we need to do a little extra work. Preparing the machine to be puppeted is
relatively straightforward and we are going to take advantage of the fact that you can use multiple
provisioners on a single machine. In order to use the shell provisioner we add the following lines to our
Vagrantfile before the final end:
5 of 17 11/03/2013 10:25 PM
# Enable shell prov1s1on1n lo boolslrap puppel
coni1.vm.prov1s1on .shell, .palh => "boolslrap.sh"
Then we create a file called boolslrap.sh in the same folder as our Vagrantfile that contains the
following:
#!1usr1b1n1env bash
sel -e
1i | "$EuT0" -ne "0" ] , lhen
echo "Scr1pl musl be run as rool." >&2
ex1l 1
i1
1i Wh1ch puppel > 1dev1null , lhen
echo "Puppel 1s already 1nslalled"
ex1l 0
i1
echo "Tnslall1n Puppel repo ior ubunlu 12.04 LTS"
Wel -q0 1lmp1puppellabs-release-prec1se.deb \
hllps.11apl.puppellabs.com1puppellabs-release-prec1se.deb
dpk -1 1lmp1puppellabs-release-prec1se.deb
rm 1lmp1puppellabs-release-prec1se.deb
apl1lude updale
#apl1lude uprade -y
echo Tnslall1n puppel
apl1lude 1nslall -y puppel
echo "Puppel 1nslalled!"
One important thing to notice is that the script is idempotent (meaning that it can be run multiple times
without having any bad effects), this is important because we can run the provisioners without creating
a new machine. Your virtual machine will now be ready to be controlled via Puppet!
6 of 17 11/03/2013 10:25 PM
Adding Puppet
Since we dont want to install Puppet on our host machine, bring up the VM and connect to it. Puppet is
already installed. We can do all of our initial Puppet configuration directly within the vm. By default,
the folder containing the Vagrantfile is shared on the virtual machine in the path 1varanl and this
will make a good location to store our puppet configurations. Once in that directory you will want to
create a skeleton of your puppet dir:
$ mkd1r -p puppel1{man1iesls,modules)
$ louch man1iesls1s1le.pp
From here we will want to install some puppet modules that will make setting up a basic LAMP server
easy (for now we will just keep everything on the same box). There are lots of different ways to install
puppet modules, but the most straightforward is using the way that is built into puppet:
$ puppel module 1nslall -1 1varanl1puppel1modules \
puppellabs1apache
$ puppel module 1nslall -1 1varanl1puppel1modules \
puppellabs1mysql
Now we will want to actually write a puppet manifest , so we will want to create /vagrant/puppet
/manifests/site.pp with the following:
7 of 17 11/03/2013 10:25 PM
node `developmenl.kloudless.vm` { # |1]
class { `mysql..server`. # |2]
coni1_hash => { `rool_passWord` => `herpderpderp` ),
)
1nclude mysql..php # |3]
# Coni1ur1n apache
1nclude apache # |4]
1nclude apache..mod..php
apache..vhosl { $..iqdn. # |5]
porl => `80`,
docrool => `1var1WWW1lesl`,
requ1re => I1le|`1var1WWW1lesl`],
)
# Sell1n up lhe documenl rool
i1le { |`1var1WWW`, `1var1WWW1lesl`] . # |6]
ensure => d1reclory,
)
i1le { `1var1WWW1lesl11ndex.php` . # |7]
conlenl => `>?php echo \`>p<Rello World!>1p<\` ?<`,
)
# "Real1ze" lhe i1reWall rule
I1reWall <] ]> # |8]
)
This describes how the server gets configured, the basic function is setting your vm with apache, mysql,
and php along with a test page. Here are some more details about the different parts:
The node definition is how we collect configuration for the machine, the label
development.kloudless.vm matches the hostname that we configured the vagrant box with.
1.
This statement uses the mysql class to install the mysql server and sets up the admin password for
the db as herpderpderp.
2.
That is how the php bindings for mysql get installed 3.
Those two lines install the apache server package and mod_php respectively 4.
We want an apache vhost where we can access our basic test application. 5.
This actually creates the directories where the vhost content will live 6.
That is our application! Right now it doesnt actually use the database, but it is a good example, we
can define the contents of the file inline relatively easily this way. We will replace this later.
7.
This is a way of realizing virtual resources which in this case configures your machines firewall 8.
8 of 17 11/03/2013 10:25 PM
rules.
Now that we have the puppet configuration ready, we need to have Vagrant use it. This can be done
easily by adding the following lines to your Vagrantfile after the shell provisioner lines:
# Enable prov1s1on1n W1lh Puppel sland alone.
coni1.vm.prov1s1on .puppel do ]puppel]
puppel.man1iesls_palh = "puppel1man1iesls"
puppel.man1iesl_i1le = "s1le.pp"
puppel.module_palh = "puppel1modules"
puppel.opl1ons = "--verbose --debu"
end
This just tells vagrant where to find the modules we installed and the manifest we wrote. So now you
are ready to varanl up. Once your box is built, you should be able to visit http://localhost:8080
(http://localhost:8080) and see the output of your test page.
With a few small adjustments, you could use this to develop a full blown php app. The adjustments you
would probably want to make are as follows:
Remove the index.php file block from the sites.pp file 1.
Change the webroot value in the vhost code block to be /var/www/app 2.
Configure a shared folder that apache can point to as the webroot. For example, if you have a folder
called app that contained your application and it is in the same directory as your Vagrantfile, you
would add the following line to the Vagrantfile:
coni1.vm.synced_iolder "app", "1var1WWW1app"
3.
Once those changes are made, you should be able to just run vagrant provision to update the settings
and you can start dumping your project files into the app directory and you should see those changes on
the vm.
Learning More
If you wanted a LAMP server to develop a php application, you dont really need to go any further.
Odds are, however, that you want to do something more than just this. If you want to run a more
complicated application you can do this pretty easily if you can find a module to do it. The specific
details of how you use a module depends on what it is, but the documentation is usually ok. If you want
to more seriously manage your dev box with puppet, you should do some more reading so you can use
the different modules people have written more easily. Here is some recommended reading :
9 of 17 11/03/2013 10:25 PM
The Learning Puppet Series (http://docs.puppetlabs.com/learning/index.html) 1.
Example42 Tutorials (http://www.example42.com/?q=Example42PuppetTutorials). They also
have a bunch of good modules you can use
2.
Puppet 3 reference manual (http://docs.puppetlabs.com/puppet/3/reference/) 3.
Hiera
If you are a more experienced puppet user, you might be familiar with Hiera and want to use it with the
modules/classes that you have written. It is pretty easy to do. First, change the value of puppet.options
in your Vagrantfile to verbose debug hiera_config /vagrant/puppet/hiera.yaml. Now you need to
populate that file with your hiera configuration, depending on how you are using it you might end up
with something like this:
---
.h1erarchy.
- common
.backends.
- yaml
- puppel
.yaml.
.dalad1r.
1varanl1puppel1h1eradala
Now you will want to make a directory called h1eradala in the puppet directory. From there you can
put all of your hiera variables in a file called common.yaml.
Multiple Servers
One server is great, but if you have a real production infrastructure, it most likely doesnt consist of a
single machine. Vagrant is pretty nice in that it inherently supports it, you just need to do some
modifications to your Vagrantfile. If we wanted to have our database server be separate from our web
server. Here is our new Vagrantfile:
10 of 17 11/03/2013 10:25 PM
# -*- mode. ruby -*-
# v1. sel il=ruby .
varanl.coni1ure{"2"} do ]coni1]
# All varanl coni1ural1on 1s done here. The mosl common coni1ural1on
# opl1ons are documenled and commenled beloW. Ior a complele reierence,
# please see lhe onl1ne documenlal1on al varanlup.com.
# Every varanl v1rlual env1ronmenl requ1res a box lo bu1ld oii oi.
coni1.vm.box = "puppellabs-prec1se64"
coni1.vm.box_url = "hllp.11puppel-varanl-boxes.puppellabs.com1ubunlu-serv
coni1.vm.dei1ne .Web do ]WWW] # |1]
WWW.vm.hoslname = "dev-WWW.kloudless.vm"
WWW.vm.nelWork .pr1vale_nelWork, 1p. "192.168.33.10"
WWW.vm.nelWork .iorWarded_porl, uesl. 80, hosl. 8080
end
coni1.vm.dei1ne .db do ]db] # |2]
db.vm.hoslname = "db.kloudless.vm"
db.vm.nelWork .pr1vale_nelWork, 1p. "192.168.33.11"
end
# v1rlualBox Spec1i1c Cuslom1zal1on
coni1.vm.prov1der .v1rlualbox do ]vb]
# use vBoxManae lo cuslom1ze lhe vM. Ior example lo chane memory.
vb.cuslom1ze |"mod1iyvm", .1d, "--memory", "512"]
end
# v1eW lhe documenlal1on ior lhe prov1der you`re us1n ior more
# 1niormal1on on ava1lable opl1ons.
# Enable shell prov1s1on1n lo boolslrap puppel
coni1.vm.prov1s1on .shell, .palh => "boolslrap.sh"
# Enable prov1s1on1n W1lh Puppel sland alone.
coni1.vm.prov1s1on .puppel do ]puppel]
puppel.man1iesls_palh = "puppel1man1iesls"
puppel.man1iesl_i1le = "s1le.pp"
puppel.module_palh = "puppel1modules"
puppel.opl1ons = "--verbose --debu"
end
end
11 of 17 11/03/2013 10:25 PM
The primary change are the blocks labelled [1] and [2], those are just adding the host specific
configurations for the network settings, so that they can be dealt with separately and talk to each other.
Now that there are two boxes vagrant can refer to, for example if you wanted to just bring up the
webserver you would do vagrant up web. The name you refer it to is the key that is the argument to
coni1.vm.dei1ne.
In order to have puppet provision both of the servers, we also need to modify the site.pp file so we take
into account the fact that we have two nodes. This is pretty straightforward and ends up essentially
splitting the single node declaration into two, resulting in:
node `dev-WWW.kloudless.vm` {
# Coni1ur1n apache
1nclude apache
apache..vhosl { $..iqdn.
porl => `80`,
)
i1le { |`1var1WWW`, `1var1WWW1lesl`] .
)
i1le { `1var1WWW1lesl11ndex.php` .
)
I1reWall <] ]>
)
node `db.kloudless.vm` {
class { `mysql..server`.
coni1_hash => { `rool_passWord` => `herpderpderp` ),
)
1nclude mysql..php
I1reWall <] ]>
)
12 of 17 11/03/2013 10:25 PM
Essentially all that happened, is that we split the original node definitions into two separate ones. Once
you have these manifests in place, running vagrant up brings up both virtual machines in sequence.
Once they are up, they can communicate over the private network via the configured ip addresses. The
hostnames you configure cant get resolved (it wouldnt be too hard to put the ips and hostnames in
each servers 1elc1hosls file through puppet, but that isnt too relevant here). Now this is more like
something you would see in your actual infrastructure.
Moving to the Cloud
Once you have a real application developed and configured you probably want it to be accessible to
everyone, so why not push it out to Amazons EC2! This can be done easily through the AWS provider
add on to Vagrant. This will basically be a different Vagrantfile that you will use specifically to push to
EC2. In order to replicate the configuration we had locally, you first need to install the plugin:
$ varanl plu1n 1nslall varanl-aWs
Once you have the plugin installed, we are going to take advice from the plugins docs to get started
quickly using a dummy box. All this means is that our configuration will be explicit within the
declaration of the box. To register the dummy box, we do the following:
$ varanl box add dummy hllps.111lhub.com1m1lchellh1varanl-aWs1raW1masler1d
Then we need to modify our existing Vagrantfile to create and provision the box, since it is going to be
pretty different, we can create the different configuration blocks:
13 of 17 11/03/2013 10:25 PM
# Be1n lhe AWS Prov1der Coni1ural1on.
coni1.vm.prov1der .aWs do ]aWs,overr1de]
aWs.access_key_1d = "Y0uR KEY"
aWs.secrel_access_key = "Y0uR SECRET KEY"
aWs.keypa1r_name = "KEYPATR NAME"
aWs.re1on = "us-Wesl-2"
aWs.am1 = "am1-ii68i8ci" # ubunlu 12.04LTS 1n us-Wesl-2"
overr1de.ssh.username = "ubunlu"
overr1de.ssh.pr1vale_key_palh = "PATR T0 Y0uR PRTvATE KEY"
end
# Th1s box W1ll be brouhl up 1n EC2
coni1.vm.dei1ne .Web_aWs do ]Web]
Web.vm.box = "dummy"
Web.vm.hoslname = "WWW.kloudless.aWs" # 0ummy hoslname
end
These blocks can just be added into your Vagrantfile before the final end. This assumes that you already
have an account and a key-pair set up, so you will need to substitute your credentials into the proper
place. I have chosen the Ubuntu 12.04LTS AMI because it is easy to use and us-west-2 because it is
pretty close to where I am located (it is in Oregon). Now here is the somewhat tricky bit, because of the
way that EC2 works, you wont really know the hostname of the machine before you bring it up and the
network configuration options of Vagrant dont support setting the hostname. There are a couple ways
around this:
Nodeless Puppet (https://github.com/jordansissel/puppet-examples/tree/master/nodeless-
puppet/): A pretty novel approach that is fact driven. It is interesting and probably what I
would recommend if you are going to use this in a real production environment.
1.
Just bring it up and do provisioning afterwards: This is clunky, but easy and what I will do for
this blog post.
2.
So you will bring up your vm with varanl up Web_aWs. The shell provisioning will go ahead just
fine, but the puppet provisioning will fail. Ths is ok, we just need another puppet node definition.
Basically I am going to just copy the node definition on dev-www (if you wanted to bring up the
database server in EC2 it would be the same kind of process):
14 of 17 11/03/2013 10:25 PM
node `TRE NEW R0STNAME` {
# Coni1ur1n apache
1nclude apache
apache..vhosl { $..iqdn.
porl => `80`,
)
i1le { |`1var1WWW`, `1var1WWW1lesl`] .
)
i1le { `1var1WWW1lesl11ndex.php` .
)
I1reWall <] ]>
)
You can replace TRE NEW R0STNAME with the short domain name that the node thinks it has, in my
case it was 1p-10-251-32-195. Now you can actually provision your vm with vagrant provision
web_aws. In order to actually view the test page, you will need to have your security groups set up
properly, but if you just want to check you can varanl ssh Web_aWs and then curl
hllp.11localhosl {hllp.11localhosl} and see that it works.
Extra Notes
Some things work differently when you are using the AWS provider and the main one you will notice is
the shared directories. These get syncd with rsync every time you run vagrant up, vagrant reload, or
vagrant provision. You can also build a bunch more aws configuration into your box definition, so you
dont have to specify it by hand.
Since it is in EC2 you can take advantage of tags and user data. In our production puppet environment,
we have an enc that decides what classes to give an instance based on its tags, but that requires a puppet
master, which I didnt really talk about (right now, vagrant just uses puppet apply).
15 of 17 11/03/2013 10:25 PM
Conclusion
Hopefully I have given you a good taste of how easy it is to put these two tools together to make your
development and deployment a lot easier! Puppet is a really great tool and I highly recommend learning
more so you can take full advantage of this work flow. I am still working to fully utilize all these great
tools and it would be great to hear about other peoples experiences.
16 thoughts on Automating Development Environments
with vagrant and Puppet"
One would add hostnames/IP addresses to /etc/hosts, not resolv.conf.
Oh lol, yeah good catch. Fixed.
1.
Well written introduction.
One thing to add. Ive done a similar Puppet preinstall script ( https://gist.github.com/dol/5776169
). The difference is that the script allows to specify a certain version to be installed. Even if
Puppetlabs ships a new version and removes some deprecated stuff, your Puppet code will work as
expected.
That is pretty nice, I explicitly didnt want to worry about versions for the sake of simplicity and
because I have been trying to keep my manifests up to date. I always get the feeling that it would
be pretty easy to get comfy in a version and that makes it hard to get out
2.
Nice article. Im slightly confused about why youve added a bootstrap to add Puppet. Isnt Puppet
added by Vagrant automagically?
I added a bootstrap script because the versions of puppet that are usually installed on vagrant
boxes tend to be out of date, so I opted to use a clean box and do some extra bootstrapping on my
own to make it more consistent with what I use on my production servers. I also added an
explanation in the bootstrapping section to avoid further confusion
Thanks, I thought as much but wanted to confirm.
3.
Howdy! This post couldnt be written any better! Reading through this post reminds me of my
previous room mate! He always kept talking about this. I will forward this write-up to him. Pretty
sure he will have a good read. Thank you for sharing!
4.
Pingback: vagrant puppet runliferun 5.
Pingback: Links for June 22nd through July 8th 6.
16 of 17 11/03/2013 10:25 PM
Pingback: Bookmarks for July 1st | Chriss Digital Detritus 7.
Pingback: Lamp server A perfect choice for your web host | Hosting Home 8.
Great post, detailed and well written. One small remark, the apache module uses mom_module =>
worker as default; to override it I had to use:
class { apache:
mpm_module => prefork,
}
in site.pp
Nice catch, that will probably come in handy for people who use non-threadsafe libraries.
9.
The bootstrap.sh file threw errors out to me. So I modified it like this to work for me:
wget -q https://apt.puppetlabs.com/puppetlabs-release-precise.deb -O /tmp/puppetlabs-release-
precise.deb
10.
Absolutely awesome! Many thanks for doing this. 11.
Follow
Follow The Kloudless Blog | Thoughts on the company, technology,
design, and the future of cloud"
Powered by WordPress.com
17 of 17 11/03/2013 10:25 PM

Automating Development Environments With Vagrant and Puppet TH PDF

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Automating Development Environments With Vagrant and Puppet TH PDF

Hochgeladen von

Copyright:

Verfügbare Formate

Your Situation

This could be quite varied, you could be:

Das könnte Ihnen auch gefallen