However, it feels rather “un-Puppet-like” to SSH into a server directly, make Puppet manifest changes, and then run the Puppet agent in order to make configuration changes to a server; it is not very efficient.
The Solution: Capistrano
We felt it would be more effective to have a single point from which we could modify manifests, and then run commands to make the necessary configuration changes on a server, while not actually needing to run a Puppet master.
To achieve this, we’ve started using Capistrano to bootstrap Puppet, upload manifests, and run the Puppet agent in standalone mode. For a while now, we’ve used Capistrano to deploy applications to servers. We’ve found it incredibly convenient to have both application deployment information and server configuration information (Puppet manifests) in the same place — allowing us to readily provision and configure a server, deploy an application, and document the whole setup in one source control repository.
Overview
At a high level, once we have an application that needs to be deployed, we:
- Add Capistrano to the source code repository, if not already in use.
- Add Puppet manifests and modules to the source code repository.
- Configure Capistrano to bootstrap Puppet, upload Puppet manifests and modules, and run Puppet.
- Run Puppet via Capistrano to provision and configure the servers.
- Configure Capistrano to deploy the application, making use of Capistrano multi-stage if we need to handle multiple environments or servers.
- Run Capistrano to deploy and manage the application.
Implementation
In practice, we write Puppet manifests and modules incrementally — writing some Puppet code, running the Puppet agent to verify it produces expected results, and repeating the process. After we have finished writing the Puppet manifests and modules, we do a full test run on a clean system to verify that our code behaves correctly from bootstrapping to final configuration implementation.
After we have the Puppet code finalized, we use a similar process on the Capistrano recipes to verify that deployment completes correctly. Often we will complete this process on our local machines, making use of virtual machines that we can snapshot and revert as needed to facilitate the code development process.
The remainder of the post gives some specific examples and code snippets which show how we configure Capistrano to bootstrap and run puppet, how we bootstrap RVM and Puppet, and how we tie it all together.
Repository Structure
For a generic Rails application, our source control repository may look something like this (emphasis on directories used by Capistrano and Puppet):
??? Capfile
??? Gemfile
??? Gemfile.lock
??? README
??? Rakefile
??? app
??? bin
??? config
? ??? deploy
? ? ??? demo.rb
? ? ??? production.rb
? ? ??? sandbox.rb
? ??? deploy.rb
| ??? ...
??? config.ru
??? db
??? doc
??? features
??? lib
??? log
??? public
??? puppet
? ??? bootstrap.sh
? ??? manifests
? ? ??? site.pp
? ??? modules
? ? ??? module1
? ? ??? module2
? ? ??? module3
? ? ??? ...
??? script
??? spec
??? tasks
??? vendor
Capistrano Modifications for Puppet
Note: We presume that the given user has sudo privileges on the system already. The “bootstrap” namespace involves bootstrapping Puppet, which includes installing RVM. The “puppet” namespace involves running puppet after bootstrapping.
require "rvm/capistrano"
...
namespace :bootstrap do
task :default do
# Specific RVM string for managing Puppet; may or may not match the RVM string for the application
set :user, "ubuntu"
# Set the default_shell to "bash" so that we don't use the RVM shell which isn't installed yet...
set :default_shell, "bash"
# We tar up the puppet directory from the current directory -- the puppet directory within the source code repository
system("tar czf 'puppet.tgz' puppet/")
upload("puppet.tgz","/home/#{user}",:via => :scp)
# Untar the puppet directory, and place at /etc/puppet -- the default location for manifests/modules
run("tar xzf puppet.tgz")
try_sudo("rm -rf /etc/puppet")
try_sudo("mv /home/#{user}/puppet/ /etc/puppet")
# Bootstrap RVM/Puppet!
try_sudo("bash /etc/puppet/bootstrap.sh")
end
end
namespace :puppet do
task :default do
# Specific RVM string for managing Puppet; may or may not match the RVM string for the application
set :rvm_ruby_string, '1.9.3-p125'
set :rvm_type, :system
set :user, "ubuntu"
# We tar up the puppet directory from the current directory -- the puppet directory within the source code repository
system("tar czf 'puppet.tgz' puppet/")
upload("puppet.tgz","/home/#{user}",:via => :scp)
# Untar the puppet directory, and place at /etc/puppet -- the default location for manifests/modules
run("tar xzf puppet.tgz")
try_sudo("rm -rf /etc/puppet")
try_sudo("mv puppet/ /etc/puppet")
# Run RVM/Puppet!
run("rvmsudo -p '#{sudo_prompt}' puppet apply /etc/puppet/manifests/site.pp")
end
end
RVM/Puppet Bootstrap
Note: This particular shell script is written for Ubuntu-based systems. It may also work for other Debian-based systems. It shouldn’t be hard to tweak for Redhat/Fedora-based systems. We install Ruby 1.9.3-p125 to manage Puppet. This may or may not be the version of Ruby needed for the application that will be deployed. That can be specified elsewhere in Capistrano, in the individual stage files as :rvm_ruby_string.
#!/bin/bash # Update our package manager... sudo apt-get update # Install dependencies for RVM and Ruby... sudo apt-get install -y build-essential libxslt1-dev libxml2-dev libreadline-dev zlib1g-dev libssl-dev curl git-core # Get and install RVM curl -L https://get.rvm.io | sudo bash -s stable # Source rvm.sh so we have access to RVM in this shell source /etc/profile.d/rvm.sh # Install Ruby 1.9.3-125 rvmsudo rvm install 1.9.3-p125 rvmsudo rvm alias create default 1.9.3-p125 source /etc/profile.d/rvm.sh # Update rubygems, and pull down facter and then puppet... rvmsudo rvm 1.9.3-p125 do gem update --system rvmsudo rvm 1.9.3-p125 do gem install facter --no-ri --no-rdoc rvmsudo rvm 1.9.3-p125 do gem install puppet --no-ri --no-rdoc rvmsudo rvm 1.9.3-p125 do gem install libshadow --no-ri --no-rdoc # Create necessary Puppet directories... sudo mkdir -p /etc/puppet /var/lib /var/log /var/run
Making it All Work
Now that we have all of our Puppet manifests and modules in one repository with our application code and Capistrano tying everything together, we can setup our server and get our application running:
- cap bootstrap
- cap puppet
- cap deploy:setup
- cap deploy
And if we had stages setup for separate servers with Capistrano multi-stage, we could easily setup our server and application on each stage:
- cap [stage] bootstrap
- cap [stage] puppet
- cap [stage] deploy:setup
- cap [stage] deploy
Reference: Standalone Puppet with Capistrano from our JCG partner Justin Kulesza at the Atomic Spin blog.
