Launching EC2 instances - Amazon Web Services (AWS)

I hope this article will help some you who is still under the impression that AWS allows you to launch any no of instances on any region instantly.

I had the same misunderstanding till today, and learnt that you need to request them before hand to increase your quota on no of instances you can launch within a region.

First, you should read this page. It allows you to get an idea about all soft limitations on AWS services( please note that all these ec2 limitations can be revised apon your request)

http://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html#limits_ec2 

You can also read about it from the EC2 FAQ page as well.

http://aws.amazon.com/ec2/faqs/#How_many_instances_can_I_run_in_Amazon_EC2

Hope AWS team will understand the value of the lost time due to this manual step involved in provisioning certain types of servers for the first time.

Error Msg:
You have requested more instances (1) than your current instance limit of 0 allows for the specified instance type. Please visit http://aws.amazon.com/contact-us/ec2-request to request an adjustment to this limit.

Few other great resources I found that helps to decide which AMI to select for a given instance type:

1. Ubuntu EC2 image locator - http://cloud-images.ubuntu.com/locator/ec2/
2. AWS Linux instance type matrix - http://aws.amazon.com/amazon-linux-ami/instance-type-matrix/

google-api python access through refresh token

I couldn't find a single python example on the web that explain how to get an access token to connect to google APIs using a pre-created refresh token.

Note that this is a sample code I wrote so it is far from being production ready. Don't forget to add values to REFRESH_TOKEN, CLIENT_ID, CLIENT_SECRET, REDIRECT_URI and OAUTH_SCOPE variables based on your app settings.

feel free to comment on the code below, Specially on my novice python coding skills.


#!/usr/bin/python

import httplib2
import pprint

from apiclient.discovery import build
from apiclient.http import MediaFileUpload
from oauth2client.client import OAuth2WebServerFlow
from oauth2client.client import OAuth2Credentials
from oauth2client import GOOGLE_AUTH_URI
from oauth2client import GOOGLE_REVOKE_URI
from oauth2client import GOOGLE_TOKEN_URI

#Refresh token
REFRESH_TOKEN = ""

# Copy your credentials from the console
CLIENT_ID = ''
CLIENT_SECRET = ''

## Check https://developers.google.com/drive/scopes for all available scopes
OAUTH_SCOPE = ['https://www.googleapis.com/auth/drive','http://localhost:5001/storage/getGDriveAuth']

## Redirect URI for installed apps
REDIRECT_URI = ''


credentials = OAuth2Credentials(None, CLIENT_ID,
                               CLIENT_SECRET, REFRESH_TOKEN, None,
                               GOOGLE_TOKEN_URI, None,
                               revoke_uri=GOOGLE_REVOKE_URI,
                               id_token=None,
                               token_response=None)

# Create an httplib2.Http object and authorize it with our credentials
http = httplib2.Http()
http = credentials.authorize(http)

drive_service = build('drive', 'v2', http=http)
file = drive_service.files().list().execute()

pprint.pprint(file)
exit();

fixing "npm install jshint" error on #chef #nodejs

Error output:
npm ERR! Error: No compatible version found: date-now@'^0.1.4'
npm ERR! Valid install targets:
npm ERR! ["0.1.0","0.1.1","0.1.3","0.1.2","0.1.4","1.0.0"]

If you get the above error while installing/updating your test servers with #chef, you will need to update the version of npm that is installed on your machine. (ref: https://github.com/npm/npm/issues/5298)

I use the nodejs chef cookbook (http://community.opscode.com/cookbooks/nodejs) to install npm, so updating the npm install version variable to 1.4.19 worked like a charm.

File: /cookbooks/nodejs/attributes/default.rb
Change: default['nodejs']['npm'] = '1.4.19'

While reading through the source I noticed that this cookbook isn't using the install script provided by npmjs site(https://www.npmjs.org/doc/README.html) for installing npm. sounds like an opportunity for any #chef lover to contribute and update the cookbook.

Upgrading to PHPUNIT 4.0 #CIUNIT #chef

#notetomyself

The phpunit test suite failed to execute on the newly built VM (configuration was done automatically via #chef), and that made me look into whats wrong with the test suite.

[CIUnit] PHP Error: Warning - require_once(PHPUnit/Autoload.php): failed to open stream: No such file or directory File Path: CIUnit/bootstrap_phpunit.php (line: 260)

PHP Fatal error:  require_once(): Failed opening required 'PHPUnit/Autoload.php' (include_path='.:/usr/share/php:/usr/share/pear') in /var/www/picsyn/application/third_party/CIUnit/bootstrap_phpunit.php on line 260

the first test was to check whether there were any changes to unit test files, but few minutes later found out that it wasn't the case.

Then I did a comparison of the phpunit versions I had installed before and now. first lead of suspect is that there have been a major version change. reading more on the changeset I learnt that phpunit installation via PEAR will be dis-continued after this year, so the most logical step to take is to update the installation process to support PHAR or Composer.

I'm going to take the PHAR approach this time.

here are the steps I took to fix the issue.

1. Download the latest phpunit cookbook from (http://community.opscode.com/cookbooks/phpunit/)
or alternatively
cd path_to_cookbooks_folder
knife cookbook site download phpunit
tar -xf phpunit_cookbook_tar_file

2. change config to mention phar install (cookbooks/phpunit/attributes/default.rb)
default[:phpunit][:install_method] = 'phar'

3. change config of the install dir (cookbooks/phpunit/attributes/phar.rb)
default[:phpunit][:install_dir] = '/usr/local/bin'

4. I had to make the following hack to ensure 'phpunit' command works from the CLI
In cookbooks/phpunit/recipes/phar.rb change remote_file section as below.
remote_file "#{phpunit_dir}/phpunit" do
  source node[:phpunit][:phar_url]
  mode 0755
end

5. On the PHPUNIT bootstrap file remove all mentions of the include of PHPUnit/Autoload.php as it isn't needed anymore. eg. there were two places in bootstrap_phpunit.php file that is used by my test project.

require_once ('PHPUnit/Autoload.php');

Thats all, and the following command works again without any modifications to how it should be called.

phpunit -c php/phpunit.xml models/AllModelsTest.php

How to setup an apache web server with PHP using vagrant and chef-solo

Note to self.

I will change this post in the future to cover the entire stack installation. but for now lets get this apache server running. and also in the next version i will write about how to do this much easily using librarian.

Steps followed:

1. create the project folder

2. create the Vagrantfile inside it using the command 'vagrant init'.

3. update your Vagrantfile to use the correct base box, share the correct folders and configure the port mapping

Vagrant.configure("2") do |config|

  config.vm.box = "ubuntu_precise64"

  config.vm.provider :virtualbox do |vb|

    vb.customize ["setextradata", :id, "VBoxInternal2/SharedFoldersEnableSymlinksCreate/v-root", "1"]

    vb.customize ["modifyvm", :id, "--memory", "2048"]

  end

  config.vm.network :forwarded_port, guest: 80, host: 6001

  config.vm.synced_folder "prototypev2/", "/var/www/prototypev2", :extra => "dmode=777,fmode=777"

  

  config.vm.synced_folder "chefrepo/", "/var/chefrepo", :extra => "dmode=777,fmode=777"

  config.vm.provision :shell, :inline => <<-eot font="">

    apt-get install -y build-essential

    gem install chef --version 11.4.0 --no-rdoc --no-ri --conservative

  EOT

end

4. inside the chefrepo folder make sure you have the solo.rb file, config.json file and a folder for cookbooks

5. solo.rb should read like:

file_cache_path "/var/chefrepo"

cookbook_path "/var/chefrepo/cookbooks"

6. execute 'vagrant up' from host machine and login to the box using 'vagrant ssh''

7. cd /var/chefrepo/cookbooks

8. download php and apache cookbooks using knife utility:

knife cookbook site download php

tar -xvf php-1.3.14.tar.gz

knife cookbook site download apache2

tar -xvf apache2-1.9.0.tar.gz

rm *.gz

8.1 download apt cookbook, this is needed as the first thing on the runlist so it will always execute a 'apt-get update' before the installation process begins

knife cookbook site download apt

tar -xvf apt-2.3.8.tar.gz

9. create a new cookbook for installing and configuring web server

sudo knife cookbook create appserver -o /var/chefrepo/cookbooks

10. update metadata.rb inside appserver folder, don't forget to include the dependencies

depends "apache2"

depends "php"

11. download all dependencies mentioned on php and apache cookbooks (you can get them by looking at metadata.rb of each cookbook)

build-essential

xml

mysql

yum-epel

windows

iis

iptables

logrotate

pacman

chef_handler

yum

openssl

12. create a template for apache site config in appserver/templates/default/ folder named apache_site_config.erb

        ServerAdmin ops@example.com

        DocumentRoot /var/www/prototypev2/

       

                Options Indexes FollowSymLinks MultiViews

                AllowOverride All

                Order allow,deny

                allow from all

       

        ErrorLog /var/log/apache2/error.log

        LogLevel warn

        CustomLog /var/log/apache2/access.log combined

        ServerSignature On

13 update cookbooks/appserver/recipes/default.rb as below:

include_recipe "apache2"

include_recipe "php"

include_recipe "apache2::mod_php5"

apache_site "default" do

  enable false

end

execute "change permission web" do

  command "chmod -R 0777 /var/www/prototypev2 "

  action :run

end

web_app 'prototypev2' do

  template 'apache_site_conf.erb'

end

14. update the runlist variable of config.json which lets chef know the recipes it need to run

{

  "run_list": ["recipe[apt]","recipe[appserver]"]

}

15. run chef solo

cd /var/chefrepo

sudo chef-solo -c solo.rb -j config.json