Setting up Ruby on Rails with Puma and Nginx on AWS EC2

This is an old blog post (Jan 2018) on setting up a blog using Ruby on Rails, Puma, Nginx, AWS EC2, Capistrano

Setting this all up on AWS took quite a bit of work and research. I already had an EC2 instance and Route 53 set up for my Wordpress website, but there's a lot more possible with AWS. I set the extra up when I migrated to Ruby on Rails, including getting an SSL certificate and setting up https for this website.

SSL Certificate

Getting an SSL certificate wasn't hard at all, AWS has a certificate manager that made it really easy. All you have to do is request a certificate, then validate using either email or DNS validation. They then issue you one, although it didn't seem like the same kind I saw on tutorials, you basically have to use AWS to implement with it.

Next, I had to figure out how to deploy the certificate to my Ruby on Rails site, and ended up deciding to use an elastic load balancer, and auto-scaling group.

ELB and Auto-Scaling<

There were a lot of tutorials available to set up the load balancers and scaling groups, but there weren't a lot specifically to implement SSL and only have one instance (still not trying to pay for AWS, other than my domain maintenance fee).

First, I just set up some easy stuff, the load balancer didn't have to be connected to anything to start with. I chose an application load balancer, then set up a listener pointed from my domain name to a target group (called it rails-target), that didn't contain anything to start with, as I wanted to set up my auto-scaling group first. That wasn't needed though, as I could've pointed it at my existing instance to start with.

The auto-scaling group wasn't too hard either, as I didn't have to change anything from the multi-instance tutorials, except for leaving the max number of instances at one group. I set up a launch configuration that used an AMI of my instance for the auto-scaling group, so it would already have a deployed version of the rails app available when it started a new instance.

Finally, I changed my DNS configuration in Route 53 to point to the load balancer, and it all worked after the records propagated. The only thing I had issues with was setting up http to redirect to the https site, which I still haven't finished with yet. The security groups and listeners in the load balancer should be able to accept both https and http connections, but then the nginx backend of my app should redirect the traffic to the https version if it's connected through http.

That's basically all it took to set up the SSL though, really not hard at all, and the auto-scaling group will start up a new instance automatically if the current one goes down. The load balancer will also end up balancing heavy traffic loads to different instances, if I ever decide to pay for several.

Ruby on Rails Deployment

In the future, I'll explain how this was all put together in Ruby on Rails, but the deployment was much harder due to the lack of explicit explanations of how to deploy. Many have the basics of it, using Capistrano with Ruby on Rails with every possible combinations of backends, passenger, nginx, unicorn, etc., whatever they choose to use. However, a lot of them only have the explanation of 'install your backend gems, install the actual applications on an AWS, or whichever, instance, then use cap production deploy to deploy to your server'. This is meant to be a better explanation of the process, and of the parts that are most likely to fail with a cookie-cutter setup.

Pre-Capistrano Deploy

First, you need to set up everything in order to setup the basics for the Capistrano deploy. This assumes that you have a basic working Rails application, in that you can run it with 'rails server' and find it ready to deploy to a server for full production deployment.

Shell Scripts for Setup on Deployment

One of the main issues that I had was the incorrect or ill-explained ways to set up basic configuration files for Puma and Nginx after deploying the actual application to the server. This assumes that you're using a Puma-Nginx stack, which was the easiest to finally figure out for me after a good amount of trial and error with several different stacks. After finally getting the deployment fully setup, I created a couple of shell scripts to configure Puma, Nginx, and the Rails application to streamline the process for me in the end, which makes it far easier to pull it all off (until I had it all setup and created AMIs of the EBS volume so I could start a new one and deploy new code to it when needed). Assuming you know how to install dependencies on Linux using apt-get, this begins at the configuration part of Puma and Nginx.

Setting Up Nginx and Puma

Here are some basic commands to easily configure each of these.

Nginx:

sudo rm /etc/nginx/sites-enabled/default
sudo ln -nfs /home/deploy/apps/YOUR_APP_NAME/current/config/nginx.conf /etc/nginx/sites-enabled/YOUR_APP_NAME
sudo ln -nfs /home/deploy/apps/YOUR_APP_NAME/current/config/nginx.conf /etc/nginx/sites-available/YOUR_APP_NAME

Puma:

mkdir -p ~/apps/YOUR_APP_NAME/shared/pids ~/apps/YOUR_APP_NAME/shared/tmp/sockets ~/apps/YOUR_APP_NAME/shared/log

These will setup your Nginx and Puma directories to link the correct conf file to sites in Nginx, and create the directories for sockets and pids for Puma in your app shared directory. It also creates the directories for logs within the shared directory in the file, following here are the configuration files for each of the applications:

config/nginx.conf:

upstream puma {
  server   unix:///home/USERNAME/apps/YOUR_APP_NAME/shared/tmp/sockets/YOUR_APP_NAME-puma.sock;
}

server {
  listen 80 default_server deferred;
  server_name YOUR_IP;

  root /home/USERNAME/apps/YOUR_APP_NAME/current/public;
  access_log /home/USERNAME/apps/YOUR_APP_NAME/current/log/nginx.access.log;
  error_log /home/USERNAME/apps/YOUR_APP_NAME/current/log/nginx.error.log info;

  location ^~ /assets/ {
    gzip_static on;
    expires max;
    add_header Cache-Control public;
  }

  try_files $uri/index.html $uri @puma;
  location @puma {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;

    proxy_pass http://puma;
  }

  error_page 500 502 503 504 /500.html;
  client_max_body_size 10M;
  keepalive_timeout 30;
}

config/puma.rb:

# Change to match your CPU core count
workers 2

# Min and Max threads per worker
threads 1, 6

app_dir = "/home/USERNAME/apps/YOUR_APP_NAME"
shared_dir = "#{app_dir}/shared"

# Default to production
rails_env = ENV['RAILS_ENV'] || "production"
environment rails_env

# Set up socket location
bind "unix://#{shared_dir}/tmp/sockets/YOUR_APP_NAME-puma.sock"

# Logging
stdout_redirect "#{shared_dir}/log/puma.stdout.log", "#{shared_dir}/log/puma.stderr.log", true

# Set master PID and state locations
pidfile "#{shared_dir}/pids/puma.pid"
state_path "#{shared_dir}/pids/puma.state"
activate_control_app

on_worker_boot do
  require "active_record"
  ActiveRecord::Base.connection.disconnect! rescue ActiveRecord::ConnectionNotEstablished
  ActiveRecord::Base.establish_connection(YAML.load_file("#{app_dir}/current/config/database.yml")[rails_env])
end

plugin :tmp_restart

In these, you must replace USERNAME with the username of your app user (I prefer deploy because it's rather obvious), and YOUR_APP_NAME with the name of the application/folder. The scripts above link the actual nginx configuration files to the nginx.conf within your config file, and setup the directories for the puma pids and socket. These should all be run after Capistrano deploy command, which will be following.

Capistrano Deployment

You should setup the nginx.conf and puma.rb files in APP/config and the shell scripts before deploying with Capistrano, so that that after it deploys from GitHub, you can use the shell scripts to finish the setup quickly. Other than these shell scripts and configuration files, there are several other things to setup.

Secrets Files

For me, the secret files, that should never be put on Github, in raw form at least, were config/secrets.yml, config/database.yml, and config/initializers/devise.rb. This is because the raw forms would contain the secret keys for each of them in production, which is a terrible idea as it allows easy decryption of passwords, etc., if anyone can manage to access the hashed form of the passwords. In the end, there are several ways to protect them to allow not having to recreate the files everytime that you push a new deployment there. You can set environmental variables in /etc/env, or in ~/.profile with export statements, in order to always have the secrets in the environment. With these, you can put your secret files on Github, and have it still impossible to find the actual secrets because your files pull them from the environment, rather than having them in plaintext within a file. Here are some examples:

config/database.yml:

default: &default
  adapter: postgresql
  encoding: unicode
  # For details on connection pooling, see Rails configuration guide
  # http://guides.rubyonrails.org/configuring.html#database-pooling
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

production:
  <<: *default
  database: YOUR_DB_NAME
  username: USERNAME
  password: <%= ENV['YOUR_DB_PASSWORD'] %>

config/secrets.yml:

production:
  secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>

config/initializers/devise.rb:

config.secret_key = ENV["SECRET_KEY_BASE"] if Rails.env == 'production'

All of these will pull the actual variables from the environment; for example, you put "export SECRET_KEY_BASE="YOUR_SECRET_KEY_BASE"" into your ~/.profile, and it will always be one of your environment variables after starting up with that user, and will always pull the secret key base from the environment, rather than pulling it from secrets.yml, or another file. This allows you to have a secrets.yml always available within the actual app folder, and not publicize your production secret key base. Hopefully, you already know how to set up your postgresql or other database, because I'm not trying to explain that much. There are lots of tutorials, and rails should do it for you anyways when you initially created the app.

Capistrano Deployment

This is one of the easiest parts if everything's set up correctly. First, add 'capistrano' and 'capistrano-ext' to your Gemfile if you haven't already, then install them. Next, navigate to the root directory and run the command Capify .. This adds a Capfile and a template for deployment to your app. Then, you have to add some stuff to the deploy file, located at config/deploy.rb:

# Change these

set :repo_url,        'YOUR_GITHUB_REPOSITORY'
set :application,     'YOUR_APP_NAME'
set :user,            'USERNAME'
set :puma_threads,    [4, 16]
set :puma_workers,    0

#don't change
set :pty,             true
set :use_sudo,        false
set :stage,           :production
set :deploy_via,      :remote_cache
set :deploy_to,       "/home/#{fetch(:user)}/apps/#{fetch(:application)}"
set :puma_bind,       "unix://#{shared_path}/tmp/sockets/#{fetch(:application)}-puma.sock"
set :puma_state,      "#{shared_path}/tmp/pids/puma.state"
set :puma_pid,        "#{shared_path}/tmp/pids/puma.pid"
set :puma_access_log, "#{release_path}/log/puma.error.log"
set :puma_error_log,  "#{release_path}/log/puma.access.log"
set :ssh_options,     { forward_agent: true, user: fetch(:user), keys: %w(~/.ssh/id_rsa.pub) }
set :puma_preload_app, true
set :puma_worker_timeout, nil
set :puma_init_active_record, true  # Change to false when not using ActiveRecord

namespace :puma do
  desc 'Create Directories for Puma Pids and Socket'
  task :make_dirs do
    on roles(:app) do
      execute "mkdir #{shared_path}/tmp/sockets -p"
      execute "mkdir #{shared_path}/tmp/pids -p"
    end
  end

  before :start, :make_dirs
end

namespace :deploy do
  desc "Make sure local git is in sync with remote."
  task :check_revision do
    on roles(:app) do
      unless git rev-parse HEAD == git rev-parse origin/master
        puts "WARNING: HEAD is not the same as origin/master"
        puts "Run git push to sync changes."
        exit
      end
    end
  end

  desc 'Initial Deploy'
  task :initial do
    on roles(:app) do
      before 'deploy:restart', 'puma:start'
      invoke 'deploy'
    end
  end

  desc 'Restart application'
  task :restart do
    on roles(:app), in: :sequence, wait: 5 do
      invoke 'puma:restart'
    end
  end

  before :starting,     :check_revision
  after  :finishing,    :compile_assets
  after  :finishing,    :cleanup
  after  :finishing,    :restart
end<

Change the settings at the beginning to your git repository, app name, and the username on your server. Don't change anything else unless you know what you're doing. It will set up all the directories and pull from git to set it all up. In my case, there was an issue with compiling the asset pipeline on the server, so I did one other thing before deploying. In the root directory of your app, run rake assets:precompile, and commit these precompiled assets to git, and push them. This means you won't have to compile them server-side. If you've got it all set up correctly, run cap deploy in the root directory. Assuming you have SSH set up, it will ask for your password, then deploy all of to the server.

Final Setup

Finally, some simple tasks; assuming that it's all set up correctly, you only need two commands to finish setting it all up.

sudo service nginx start (or restart)

bundle exec puma -b unix:///home/USERNAME/apps/YOUR_APP_NAME/shared/tmp/sockets/YOUR_APP_NAME-puma.sock &

That should be all you need to finish it off. In my next post, I'll expand on the AWS instance that I used, as well as the load balancer, domain setup, auto-scaling group, and enabling SSL.