Skip to content

Automating Virtual Machine creation using VirtualBox, Vagrant and Puppet


Introduction

In this tutorial, I’ll be going through the process of creating Virtual Machine using VirtualBox, Vagrant and Puppet. Lets first understand what these mean and then we can go through the manual process of creating Virtual Machines and then automating everything using Vagrant and Puppet.

VirtualBox

VirtualBox is a general-purpose full virtualizer for x86 hardware, targeted at server, desktop and embedded use.

site: https://www.virtualbox.org/

Documentation: https://www.virtualbox.org/manual/

Commands: https://www.virtualbox.org/manual/ch08.html#idp5692800

vm-vista-running

Vagrant

Vagrant is a tool for building complete development environments. With an easy-to-use workflow and focus on automation, Vagrant lowers development environment setup time, increases development/production parity, and makes the “works on my machine” excuse a relic of the past.

Vagrant creates and manages the Virtual Machine based on VirtualBox. Internally it uses VirtualBox commands for the management.

Vagrant supports several provisioners using which we can install and configure any software on the Virtual Machine. Some of the provisioners are:

The most popular one is Puppet which we will be using it to provision our Virtual Machine.

Site: http://www.vagrantup.com

Documentation: http://docs.vagrantup.com/v1/docs/

Commands: http://docs.vagrantup.com/v1/docs/commands.html

Free Vagrant Boxes: http://www.vagrantbox.es/

Puppet

Puppet is IT automation software that helps system administrators manage infrastructure throughout its lifecycle, from provisioning and configuration to patch management and compliance. Using Puppet, you can easily automate repetitive tasks, quickly deploy critical applications, and proactively manage change, scaling from 10s of servers to 1000s, on-premise or in the cloud.

Puppet is available as both open source and commercial software. You can see the differences here and decide which is right for your organization.

Puppet can be configured to work in Standalone and Master Slave modes.

Site: https://puppetlabs.com/puppet/what-is-puppet/

Syntax: http://www.puppetcookbook.com/

Automating the Virtual Machine Creation Process

Assumption: Hope you have already installed VirtualBox and Vagrant on your Host Machine(machine on which you are going to create the Virtual Machine).

Note that the Virtual Machine is called Guest Machine because its running on your Host Machine.

1. Pick your base box from http://www.vagrantbox.es/

For our example, I picked http://developer.nrel.gov/downloads/vagrant-boxes/CentOS-6.3-x86_64-v20130101.box  which is ‘CentOS 6.3 x86_64 Minimal (VirtualBox Guest Additions 4.2.6, Chef 10.16.4, Puppet 3.0.2)’.

These free boxes are configured with all the essential softwares e.g. Guest Additions, Sudo access, SSH, Puppet etc and also have under account vagrant/vagrant created.

These boxes have root/admin and vagrant/vagrant accounts already created.

2. In case you do not want to create a Box based on the public boxes then you can do below:

2.1.       Create Virtual Machine on VirtualBox. Lets say you named it as myDevBox.

2.2.       Follow steps @ http://docs.vagrantup.com/v1/docs/base_boxes.html

Make sure you install necessary softwares e.g. Puppet, SSH, Guest Additions, Ruby & RubyGems. Also create below user account:

Hostname: vagrant-[os-name], e.g. vagrant-debian-lenny
Domain: vagrantup.com
Root Password: vagrant
Main account login: vagrant
Main account password: vagrant

Make sure you also create root/admin account.

We will be following step 2.1 for our example.

3. Enable the logging. It helps to debug the issue if any.

export VAGRANT_LOG=info (or set VAGRANT_LOG=info in case you are running on windows)

4. Adding box to our environment

This will start downloading the .box file (these files are generally in GB).

vagrant box add myDevBox http://developer.nrel.gov/downloads/vagrant-boxes/CentOS-6.3-x86_64-v20130101.box

This process also creates a file named ‘Vagrantfile’ which holds the configuration for the Virtual Machine.

5. Initialize the VirtualBox with the box we added above

vagrant init myDevBox

6. Start the VirtualMachine

vagrant up

7. Logging into the VirtualMachine

7.1.       Using SSH via vagrant

vagrant ssh

7.2.       Using SSH via port forwarding

Port forwarding can be done using VirtualBox GUI or running command from http://www.virtualbox.org/manual/ch06.html#natforward

ssh -p 2222 root@127.0.0.1

7.3.       Start Virtual Machine from VirtualBox GUI directly and work inside the Guest Machine itself

8. Once you logged into the Virtual Machine, you can work on it. Install additional softwares, configure it based on your requirements.

9. Exporting/Packaging the current working Virtual Machine and distributing it to other developers

vagrant package –include README.txt –vagrantfile Vagrantfile –base MyDevBox –output MyDevBox.box

Note: –include README.txt :  means you could attach additional files to the .box file. This along with Vagrantfile will be created once someone starts utilizing it (step #4).

The box will be created with MyDevBox.box name which you can distribute to others

Automating Essential Software Installation and Configuration Process

Anything we do at Step#8 could be automated using Puppet. All you need to do is put the puppet configuration inside Vagrantfile and run “vagrant up” or only “vagrant provision”.

So far in above section we followed the commands to download box, port forwarding, directory sharing etc. All these can be done by configuring the Vagrantfile itself.

Below is the sample Vagrantfile :


# Author: HashFold

# description: created VM and provisions basic tools

Vagrant::Config.run do |config|

config.vm.box = "MyDevBox"

config.vm.box_url = "http://developer.nrel.gov/downloads/vagrant-boxes/CentOS-6.3-x86_64-v20130101.box "

config.vm.host_name = "com.mycompany.virtualmachines.centos.project1"

#--- common ports and share folders ---

#ssh -p 2222 root@127.0.0.1

config.vm.forward_port 22, 2222, :adapter => 1

#config.vm.network :hostonly, "33.33.33.10", :adapter => 1

#--- Directory sharing it shares /myshare (inside Guest Machine) to ~/myshare (on Host Machine, note that ~ mapped to corresponding user’s home directory)

config.vm.share_folder "my-data-share", "/myshare", File.expand_path('~/myshare'), :create => true

#sharing current folder makes it OS independent mount

config.vm.share_folder "v-root", "/vagrant", File.expand_path('~/vagrant'), :create => true

#--- common ports for lamp system ---

#accounts: root/admin, vagrant/vagrant

#mysql passwd is blank. (Or “admin”)

#mysql client port

config.vm.forward_port 3306, 3306, :adapter => 1

#httpd/apache @ http://127.0.0.1:8080/

config.vm.forward_port 80, 8080

#--- common ports for oracle xe ---

#ORACLE XE: system/manager@http://localhost:8081/

config.vm.forward_port 1521, 1521, :adapter => 1

config.vm.forward_port 8081, 8081, :adapter => 1

#--- Puppet configuration ---

#--- suppose puppet_store directory is in the current folder ---

config.vm.provision :puppet do |puppet|

puppet.module_path = "puppet_store/modules"

puppet.manifests_path = "puppet_store/manifests"

puppet.manifest_file  = "mybox.pp"

end

#--- virtual machine configurational parameters

#get list of VMBox commands @http://www.virtualbox.org/manual/ch08.html#vboxmanage-modifyvm

Vagrant::Config.run do |config|

config.vm.customize [

"modifyvm", :id,

"--cpus", 2,

"--memory", 2048,

"--name", "MyDevBox"

]

end

end

Lets look inside of puppet_store and see how the files are laid out:

$ find puppet_store


puppet_store

puppet_store/manifests

<b>puppet_store/manifests/</b><b>mybox</b><b>.pp</b><b>  --main driver for puppet</b>

puppet_store/modules

puppet_store/modules/commontools

puppet_store/modules/commontools/files

puppet_store/modules/commontools/files/epel_ex.repo

puppet_store/modules/commontools/files/populate_shared_data.sh

puppet_store/modules/commontools/files/puppet.repo

puppet_store/modules/commontools/files/ruby.repo

puppet_store/modules/commontools/files/rubyworks.repo

puppet_store/modules/commontools/manifests

puppet_store/modules/commontools/manifests/init.pp <b>– 2nd main driver (called from mybox.pp)</b>

puppet_store/modules/lamp

puppet_store/modules/lamp/files

puppet_store/modules/lamp/files/apache2

puppet_store/modules/lamp/files/apache2/apache2.CentOS.conf

puppet_store/modules/lamp/files/apache2/apache2.conf

puppet_store/modules/lamp/files/apache2/apache2.Ubuntu.conf

puppet_store/modules/lamp/files/mysql

puppet_store/modules/lamp/files/mysql/my.CentOS.cnf

puppet_store/modules/lamp/files/mysql/my.cnf

puppet_store/modules/lamp/files/mysql/my.Ubuntu.cnf

puppet_store/modules/lamp/files/mysql/mysql_create_user.sh

puppet_store/modules/lamp/files/mysql/mysql_create_user.sql

puppet_store/modules/lamp/files/mysql/mysql_import_data.sh

puppet_store/modules/lamp/files/sysconfig

puppet_store/modules/lamp/files/sysconfig/httpd.CentOS

puppet_store/modules/lamp/files/sysconfig/httpd.Ubuntu

puppet_store/modules/lamp/files/www

puppet_store/modules/lamp/files/www/index.php

puppet_store/modules/lamp/manifests

puppet_store/modules/lamp/manifests/init.pp<b> – 3rd main driver for LAMP system installtion</b>

puppet_store/modules/oracle

puppet_store/modules/oracle/files

puppet_store/modules/oracle/files/oracle-shm

puppet_store/modules/oracle/files/oracle-xe-11.2.0-1.0.x86_64.rpm.zip

puppet_store/modules/oracle/files/oracle_create_user.sh

puppet_store/modules/oracle/files/oracle_create_user.sql

puppet_store/modules/oracle/files/oracle_create_user_data_import.sh

puppet_store/modules/oracle/files/oracle_db.dmp.zip

puppet_store/modules/oracle/files/oracle_path.sh

puppet_store/modules/oracle/files/xe-sysctl.conf

puppet_store/modules/oracle/files/xe.rsp

puppet_store/modules/oracle/manifests

puppet_store/modules/oracle/manifests/init.pp <b>– 4th main driver for LAMP system installtion</b>

1st Level Puppet file:

The main driver for puppet is puppet_store/manifests/mybox.pp  :


# Project: Project1

# Author: HashFold

# description: main driver script for Puppet

# read more puppet syntax @ http://www.puppetcookbook.com/

#make sure the puppet is installed on the .box otherwise rest of the puppet provisioning will not function

group { "puppet":

ensure => "present",

}

Exec { path => [ "/bin/", "/sbin/" , "/usr/bin/", "/usr/sbin/" ] }

#below is to fix "yum.Errors.RepoError: database disk image is malformed" error

exec {

"common_tool_set_yum_clean":

alias => "common_tool_set_yum_clean",

command => "sudo yum -y clean  dbcache",

path => '/usr/bin:/bin:/usr/sbin:/sbin',

cwd => "/tmp",

user => root;

}

File { owner => 0, group => 0, mode => 0644 }

file { '/etc/motd':

content => "Welcome to My Developer Box!

Managed by HashFold team.

contact Admin at My Company.com for any issues.\n"

}

node default {

include commontools::install

include lamp

include oracle::server

include oracle::xe

}

2nd Level Puppet file:

puppet_store/modules/commontools/manifests/init.pp – 2nd main driver (called from mybox.pp)


# Project: Project1

# Author: HashFold

# description: runs common commands

class commontools::install {

file {

"/etc/yum.repos.d/epel_ex.repo":

source => "puppet:///modules/commontools/epel_ex.repo";

"/etc/yum.repos.d/puppet.repo":

source => "puppet:///modules/commontools/puppet.repo";

"/etc/yum.repos.d/ruby.repo":

source => "puppet:///modules/commontools/ruby.repo";

"/etc/yum.repos.d/rubyworks.repo":

source => "puppet:///modules/commontools/rubyworks.repo";

"/tmp/populate_shared_data.sh":

mode => 0755,

source => "puppet:///modules/commontools/populate_shared_data.sh";

}

exec { 'populate_shared_data':

command => "/usr/bin/sudo /tmp/populate_shared_data.sh",

require => [File["/tmp/populate_shared_data.sh"]];

}

}
<pre>

In populate_shared_data.sh, you could put any commands which you would want to run at the time of Virtual Machine setup by vagrant (vagrant provision).
One example could be to run below command to download mysql installer file:
sudo wget http://localhost/mysql-workbench-gpl-5.2.42-win32.msi (this is not for Centos however this will be visible under user’s shared folder so they can click and install these free softwares.)


Some trouble shooting tips


1.     Provisioner error:

</pre>
←[0;36mnotice: /Stage[main]/Oracle::Server/Exec[sudo yum -y install libaio bc flex]/returns: executed successfully←[0m

←[0;36mnotice: /Stage[main]/Oracle::Xe/Exec[unzip xe]/returns: executed successfully←[0m

←[1;35merr: /Stage[main]/Oracle::Xe/Package[oracle-xe]/ensure: change from absent to latest failed: Could not update: Execution of '/bin/rpm -i --oldpackage /tmp/Disk1/oracle-xe-11.2.0-1.0.x86_64.rpm' returned 1:

This system does not meet the minimum requirements for swap space.  Based on

the amount of physical memory available on the system, Oracle Database 11g

Express Edition requires <b>2004 MB</b> of swap space. This system has <b>703 MB</b>

of swap space.  Configure more swap space on the system and retry the installation.

error: %pre(oracle-xe-11.2.0-1.0.x86_64) scriptlet failed, exit status 1

error:   install: %pre scriptlet failed (2), skipping oracle-xe-11.2.0-1.0

Solution:

Log into the Virtual Machine and run below commands:


swapoff /dev/mapper/VolGroup00-LogVol01

dd if=/dev/zero of=/home/swapfile bs=1024 count=2097152

mkswap /home/swapfile

swapon /home/swapfile

swapon -a

cp /etc/fstab /etc/fstab.backup_`date +%N`

'/home/swapfile swap swap defaults 0 0' >> /etc/fstab

swapon –s



2 Comments

  1. Raul wrote:

    Nice work

    Could you publish the puppet modules you are using for Oracle XE ?

    Wednesday, May 15, 2013 at 9:35 am | Permalink
  2. Praveendra Singh wrote:

    I could however I don’t have them now with me.
    anyway its too late response from me and I hope you could have figured it out by now.
    In case you are still searching for it or for other modules, try to look at github.com. I found one from there and tweaked it little bit.

    good luck Raul!

    Monday, August 12, 2013 at 11:45 pm | Permalink

Post a Comment

Your email is never published nor shared. Required fields are marked *
*
*