Saving Uploaded Images in the Database

Work progresses on Picutive, my simple web app to help busy professionals create passport and visa photos.  It's a Rails 3 app that I have been working on for the last six months, usually during the early mornings or late evenings - kids...  I started uploading images using carrierwave and saving them to disk.  It bothered me that I had to store the images in the public directory, thus making them accessible to anyone who could guess the path and file name.  Many websites do this (facebook.com for example) by trying to make the file names hard to guess.  This seems like a terrible concept as it only stops the laziest of hackers.  This system also resulted in a proliferation of files being saved; one for each version of a user's image and one for each print example that needed to be displayed.  I could just keep generating them on the fly and then erasing them, but this would make the site less responsive.  Finally, I need separate back-up solutions for the database and image files. As this project is really a hobby, I don't want to sign up for more long-term work than absolutely required. In other words, I want to minimize my time-debt.

I recently decided to change and store uploaded images in the database.  With  this solution, I only need one back-up process.  I can keep only the optimized base image in the database. If I need to format it for a certain ID type or generate a print then I can do that on the  fly, but never save it to disk. Instead, I will use caching at the server level to avoid having to recreate it.  Yes, this is still writing to disk, but I don't have to manage it and it is a solution that provides low response times to the user.  Finally, the images are not accessible on the public files, so access is only possible after the user has been authorized.  These changes weren't too difficult to implement, although there were a few things to figure out, which is the main topic of this post.

The first part is to upload an image and store it in the database. There are actually lots of tutorials on the web that all say basically the same thing, so nothing new here.

Above is the form partial that will allow a user to upload an image. There is one file_field, which actually has two and  a half components associated with it. A filename, a content_type and its associated data. Below is the corresponding image model.  It validates the content_type.

//view
<%= form_for(@image, :html => {:multipart => true}) do |f| %>
  <div class="field">
    <%= f.label :image_file %>
    <%= f.file_field :uploaded_image %>
  </div>
  <div class="actions">
    <%= f.submit %>
  </div>
<% end %> 

Above is the form partial that will allow a user to upload an image. There is one file_field, which actually has two and  a half components associated with it. A filename, a content_type and its associated data. Below is the corresponding image model.  It validates the content_type. The uploaded_image=method will assign the file_field values to the model. The model has corresponding fields which are stored in the database, shown in the migration file.

//model
class Image < ActiveRecord::Base
  validates_format_of :content_type, :with => /^image/, :message => "file must be an image type"

  def uploaded_image=(image_field)
    self.name =         base_part_of(image_field.original_filename)
    self.content_type = image_field.content_type.chomp
    self.data = image_field.read
  end
  
  def base_part_of(filename)
    File.basename(filename).gsub(/[^\w._-]/,'')
  end
end 

//migration
class CreateImages < ActiveRecord::Migration
  def self.up
    create_table :images do |t|
      t.string :name
      t.string :content_type
      t.binary :data, :limit => 1.megabyte
      t.timestamps
    end
  end

  def self.down
    drop_table :images
  end
end

Sweet, now I can store uploaded images in the database.  What if I want to show them on my website?  Well, that's too hard, but it is a little bit more complicated than just showing a file. I need to create a custom controller action to serve the image.  The advantage of this is that Rails caching can be set up to cache controller requests, so each image served from the controller can be cached. You see where this is going don't you...

The send_data method can encode the image data stored in the image model so that it can be displayed in the web page. The disposition parameter tells the browser to display the image in the page as opposed to just linking to it for download.  The route also needs to be added to routes.rb

 
//controller
class ImagesController < ApplicationController 
  def img_url
    @image = Image.find(params[:id])
    send_data(@image.data, :filename => @image.name, :type => @image.content_type,:disposition => "inline")
  end  
end

//routes.rb
  resources :images do
    member do
      get 'img_url'
    end
  end

Ok, that's it for the basics. But I needed to do a bit more than the basics. I want to create a lower resolution version of the image with RMagic, send that to Face.com for face detection and then crop the original, again with RMagic all before doing any saves to the hard drive.

Let's start with RMagick first. This is actually quite easy.  Just use the  from_blob method to create an RMagick Image from your file_field's data. Now use RMagick methods to reduce the resolution of the image to make it suitable for sending to the Face.com api.

I am using rociiu's great face gem. (He fixed a bug for me very quickly while I was working this out - thanks!).  Unfortunately, I could not figure out how to  use the binary data either from the RMagick image file or the raw field data with his gem, which relies on RestClient to post the request. To get around this, I ended creating a RAM drive and using that to simulate saving the file to disk. This makes RestClient happy, but doesn't require saving anything to disk. The RAM drive I created is really small (20 MB) so I have to be careful to keep it clean.

Once the results are back (yes, I know I should do this in a separate process, but that must come later...)I can manipulate the original image based on the new data and update the database. 

class Image < ActiveRecord::Base
  validates_format_of :content_type, :with => /^image/, :message => "file must be an image type"
  
  require 'RMagick'
  
  def uploaded_image=(image_field)
    self.name =         base_part_of(image_field.original_filename)
    self.content_type = image_field.content_type.chomp
    raw = image_field.read
    image = Magick::Image.from_blob(raw).first
    image.resize_to_fit!(200)
    
    #write to ramdisk /tmp
    tmp_name = "face_#{self.id}_#{self.name}"
    path = "/home/jon/tmp/#{tmp_name}"
    image.write path
    image.delete  #don't need this one any more
    
    face_key = FACE_KEY
    face_secret = FACE_SECRET
    client = Face.get_client(:api_key => face_key, :api_secret => face_secret)
    face_params  = client.faces_detect( :file => File.new(path, "rb"))
    File.delete(path) #delete the temp file from RAM drive
    
    #use RMagick to manipule the oringal file based on the face detection relults
    image = Magick::Image.from_blob(raw).first
    image.do_something_with_RMagick
    self.data =  image.to_blob
  end
  
  def base_part_of(filename)
    File.basename(filename).gsub(/[^\w._-]/,'')
  end
end 

Ok, so now the image has been uploaded, manipulated and stored in the database with only one write to the hard drive. We can serve the image as it is or even do further processing if needed for a specific request with the customer image controller action.  Setting up the URL for the image action  so that each unique image or version of an image has a unique URL will allow us to cache that image so that we don't have to hit the  database again to serve it.

Testing this process is made quite easy thanks to the helper method fixture_file_upload.  I am using this in unit testing, so I need to include ActionDispatch::TestProcess.  Be sure to check the documentation for you version of Rails since the location of this method has moved in 3.0.8 and my need a different include statement for earlier versions.

//test_helper.rb
ENV["RAILS_ENV"] = "test"
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
include ActionDispatch::TestProcess


class ActiveSupport::TestCase
  
  def make_image(data, b, user=nil)
    base ||= "portraites/" #works with fixture_file_upload
    f = data['file']
    path = "test_image.png" #this file is in /test/fixtures
    im = image.new(:photo => fixture_file_upload(path, 'image/png'))
    
    pic = Magick::Image.from_blob(im.data).first
    assert_equal 300, pic.rows, "Number of rows is false for #{f}"
    assert_equal 300, pic.columns, "Number of columns is false for #{f}"
    return im
  end
  
end 

The final problem to solve was attaching the image stored in the database to an email.  I'm happy to report that this was relatively painless.

//rails 3.0.x mailer
class Mailman < ActionMailer::Base
  default :from => "jon@picutive.com"
  
  def email_print(image, user)
    @image, @user = image, user
    attachments["#{@image.doc.name}.jpg"] = @image.data #File.read(@image._path) for files on disk
    receiver = @user.email
    mail(:to => receiver, :subject => "Picutive: #{@print.doc.name}")
  end
  
end

That's all I have now. I hope this might help someone else as it took me a while to put it all together.