How Sequel and Sinatra Solve Ruby’s API Problem

Introduction

In recent years, the number of JavaScript single page application frameworks and mobile applications has increased substantially. This imposes a correspondingly increased demand for server-side APIs. With Ruby on Rails being one of the today’s most popular web development frameworks, it is a natural choice among many developers for creating back-end API applications.

Yet while the Ruby on Rails architectural paradigm makes it quite easy to create back-end API applications, using Rails only for the API is overkill. In fact, it’s overkill to the point that even the Rails team has recognized this and has therefore introduced a new API-only mode in version 5. With this new feature in Ruby on Rails, creating API-only applications in Rails became an even easier and more viable option.

But there are other options too. The most notable are two very mature and powerful gems, which in combination provide powerful tools for creating server-side APIs. They are Sinatra and Sequel.

Both of these gems have a very rich feature set: Sinatra serves as the domain specific language (DSL) for web applications, and Sequel serves as the object-relational mapping (ORM) layer. So, let’s take a brief look at each of them.

Ruby API on a diet: introducing Sequel and Sinatra.

Sinatra

Sinatra is Rack-based web application framework. The Rack is a well known Ruby web server interface. It is used by many frameworks, like Ruby on Rails, for example, and supports lot of web servers, like WEBrick, Thin, or Puma. Sinatra provides a minimal interface for writing web applications in Ruby, and one of its most compelling features is support for middleware components. These components lie between the application and the web server, and can monitor and manipulate requests and responses.

For utilizing this Rack feature, Sinatra defines internal DSL for creating web applications. Its philosophy is very simple: Routes are represented by HTTP methods, followed by a route matching a pattern. A Ruby block within which request is processed and the response is formed.

get '/' do
  'Hello from sinatra'
end

The route matching pattern can also include a named parameter. When route block is executed, a parameter value is passed to the block through the params variable.

get '/players/:sport_id' do
  # Parameter value accessible through params[:sport_id]
end

Matching patterns can use splat operator * which makes parameter values available through params[:splat].

get '/players/*/:year' do
  # /players/performances/2016
  # Parameters - params['splat'] -> ['performances'], params[:year] -> 2016
end

This is not the end of Sinatra’s possibilities related to route matching. It can use more complex matching logic through regular expressions, as well as custom matchers.

Sinatra understands all of the standard HTTP verbs needed for creating a REST API: Get, Post, Put, Patch, Delete, and Options. Route priorities are determined by the order in which they are defined, and the first route that matches a request is the one that serves that request.

Sinatra applications can be written in two ways; using classical or modular style. The main difference between them is that, with the classical style, we can have only one Sinatra application per Ruby process. Other differences are minor enough that, in most cases, they can be ignored, and the default settings can be used.

Classical Approach

Implementing classical application is straightforward. We just have to load Sinatra and implement route handlers:

require 'sinatra'

get '/' do
  'Hello from Sinatra'
end

By saving this code to demo_api_classic.rb file, we can start the application directly by executing the following command:

ruby demo_api_classic.rb

However, if the application is to be deployed with Rack handlers, like Passenger, it is better to start it with the Rack configuration config.ru file.

require './demo_api_classic'

run Sinatra::Application

With the config.ru file in place, the application is started with the following command:

rackup config.ru

Modular Approach

Modular Sinatra applications are created by subclassing either Sinatra::Base or Sinatra::Application:

require 'sinatra'

class DemoApi < Sinatra::Application
  # Application code

  run! if app_file == $0
end

The statement beginning with run! is used for starting the application directly, with ruby demo_api.rb, just as with the classical application. On the other hand, if the application is to be deployed with Rack, the handlers content of rackup.ru must be:

require './demo_api'

run DemoApi

Sequel

Sequel is the second tool in this set. In contrast to ActiveRecord, which is part of the Ruby on Rails, Sequel’s dependencies are very small. At the same time, it is quite feature rich and can be used for all kinds of database manipulation tasks. With its simple domain specific language, Sequel relieves the developer from all the problems with maintaining connections, constructing SQL queries, fetching data from (and sending data back to) the database.

For example, establishing a connection with the database is very simple:

DB = Sequel.connect(adapter: :postgres, database: 'my_db', host: 'localhost', user: 'db_user')

The connect method returns a database object, in this case, Sequel::Postgres::Database, which can be further used to execute raw SQL.

DB['select count(*) from players']

Alternatively, to create a new dataset object:

DB[:players]

Both of these statements create a dataset object, which is a basic Sequel entity.

One of the most important Sequel dataset features is that it does not execute queries immediately. This makes it possible to store datasets for later use and, in most cases, to chain them.

users = DB[:players].where(sport: 'tennis')

So, if a dataset does not hit the database immediately, the question is, when does it? Sequel executes SQL on the database when so-called “executable methods” are used. These methods are, to name a few, all, each,map, first, and last.

Sequel is extensible, and its extensibility is a result of a fundamental architectural decision to build a small core complemented with a plugin system. Features are easily added through plugins which are, actually, Ruby modules. The most important plugin is the Model plugin. It is an empty plugin which does not define any class or instance methods by itself. Instead, it includes other plugins (submodules) which define a class, instance or model dataset methods. The Model plugin enables the use of Sequel as the object-relational-mapping (ORM) tool and is often referred to as the “base plugin”.

class Player < Sequel::Model
end

The Sequel model automatically parses the database schema and sets up all necessary accessor methods for all columns. It assumes that table name is plural and is an underscored version of the model name. In case there is a need to work with databases that do not follow this naming convention, the table name can be explicitly set when the model is defined.

class Player < Sequel::Model(:player)
end

So, we now have everything we need to start building the back-end API.

Building the API

Like what you're reading?

Get the latest updates first.

No spam. Just great engineering and design posts.

• 
• 
• 

Code Structure

Contrary to Rails, Sinatra does not impose any project structure. However, since it is always a good practice to organize code for easier maintenance and development, we’ll do it here too, with the following directory structure:

project root
   |-config
   |-helpers
   |-models
   |-routes

The application configuration will be loaded from the YAML configuration file for the current environment with:

Sinatra::Application.config_file File.join(File.dirname(__FILE__),
                                           'config',
                                           "#{Sinatra::Application.settings.environment}_config.yml")

By default, Sinatra::Applicationsettings.environment value is development, and it is changed by setting the RACK_ENV environment variable.

Furthermore, our application must load all files from the other three directories. We can do that easily by running:

%w{helpers models routes}.each {|dir| Dir.glob("#{dir}/*.rb", &method(:require))}

At first glance, this way of loading might look convenient. However, with this one line of the code, we cannot easily skip files, because it will load all the files from the directories in the array. That’s why we will use a more efficient single-file load approach, which assumes that in each folder we have a manifest file init.rb, which loads all other files from the directory. Also, we will add a target directory to the Ruby load path:

%w{helpers models routes}.each do |dir|
  $LOAD_PATH << File.expand_path('.', File.join(File.dirname(__FILE__), dir))
  require File.join(dir, 'init')
end

This approach requires a bit more work because we have to maintain require statements in each init.rb file but, in return, we get more control, and we can easily leave one or more files out by removing them from the manifest init.rb file in the target directory.

API Authentication

The first thing we need in each API is authentication. We will implement it as a helper module. Complete authentication logic will be in the helpers/authentication.rb file.

require 'multi_json'

module Sinatra
  module Authentication
    def authenticate!
      client_id = request['client_id']
      client_secret = request['client_secret']

      # Authenticate client here

      halt 401, MultiJson.dump({message: "You are not authorized to access this resource"}) unless authenticated?
    end

    def current_client
      @current_client
    end

    def authenticated?
      !current_client.nil?
    end
  end

  helpers Authentication
end

All we have to do now is to load this file by adding a require statement in the helper manifest file (helpers/init.rb) and to call the authenticate! method in Sinatra’s before hook which will be executed before processing any request.

before do
  authenticate!
end

Database

Next, we have to prepare our database for the application. There are many ways to prepare the database, but since we are using Sequel, it is natural to do it by using migrators. Sequel comes with two migrator types - integer and timestamp based. Each one has its advantages and drawbacks. In our example, we decided to use the Sequel’s timestamp migrator, which requires migration files to be prefixed with a timestamp. The timestamp migrator is very flexible and can accept various timestamp formats, but we will only use the one that consists of year, month, day, hour, minute, and second. Here are our two migration files:

# db/migrations/20160710094000_sports.rb
Sequel.migration do
  change do
    create_table(:sports) do
      primary_key :id
      String :name, :null => false
    end
  end
end

# db/migrations/20160710094100_players.rb
Sequel.migration do
  change do
    create_table(:players) do
      primary_key :id
      String :name, :null => false
      foreign_key :sport_id, :sports
    end
  end
end

We are now ready to create a database with all the tables.

bundle exec sequel -m db/migrations sqlite://db/development.sqlite3

Finally, we have the model files sport.rb and player.rb in the models directory.

# models/sport.rb
class Sport < Sequel::Model
  one_to_many :players

  def to_api
    {
      id: id,
      name: name
    }
  end
end


# models/player.rb
class Player < Sequel::Model
  many_to_one :sport

  def to_api
    {
      id: id,
      name: name,
      sport_id: sport_id
    }
  end
end

Here we are employing a Sequel way of defining model relationships, where the Sport object has many players and Player can have only one sport. Also, each model defines its to_api method, which returns a hash with attributes that need to be serialized. This is a general approach we can use for various formats. However, if we will only use a JSON format in our API, we could use Ruby’s to_json with only argument to restrict serialization to required attributes, i.e. player.to_json(only: [:id, :name, :sport_i]). Of course, we could also define a BaseModel that inherits from Sequel::Model and defines a default to_api method, from which inherit all models could then inherit.

Now, we can start implementing the actual API endpoints.

API Endpoints

We will keep the definition of all endpoints in files within the routes directory. Since we are using manifest files for loading files, we will group routes by resources (i.e., keep all sports related routes in sports.rb file, all players routes in routes.rb, and so on).

# routes/sports.rb
class DemoApi < Sinatra::Application
  get "/sports/?" do
    MultiJson.dump(Sport.all.map { |s| s.to_api })
  end

  get "/sports/:id" do
    sport = Sport.where(id: params[:id]).first
    MultiJson.dump(sport ? sport.to_api : {})
  end

  get "/sports/:id/players/?" do
    sport = Sport.where(id: params[:id]).first
    MultiJson.dump(sport ? sport.players.map { |p| p.to_api } : [])
  end
end

# routes/players.rb
class DemoApi < Sinatra::Application
  get "/players/?" do
    MultiJson.dump(Player.all.map { |p| s.to_api })
  end

  get "/players/:id/?" do
    player = Player.where(id: params[:id]).first
    MultiJson.dump(player ? player.to_api : {})
  end
end

Nested routes, like the one for getting all the players within one sport /sports/:id/players, can either be defined by placing them together with other routes, or by creating a separate resource file that will contain only the nested routes.

With designated routes, the application is now ready to accept requests:

curl -i -XGET 'http://localhost:9292/sports?client_id=<client_id>&client_secret=<client_secret>'

Note that as required by the application’s authentication system defined in helpers/authentication.rb file, we are passing credentials directly in request parameters.

Conclusion

The principles demonstrated in this simple example application apply to any API back-end application. It is not based on the model-view-controller (MVC) architecture, yet it keeps a clear separation of responsibilities in a similar way; complete business logic is kept in model files while handling requests is done in Sinatra’s routes methods. Contrary to MVC architecture, where views are used to render responses, this application does that at the same place where it handles requests - in the routes methods. With new helper files, the application can be easily extended to send pagination or, if needed, request limits information back to the user in response headers.

In the end, we’ve built a complete API with a very simple toolset and without losing any functionality. The limited number of dependencies helps ensure that the application loads and starts much faster, and has a much smaller memory footprint than a Rails based one would have. So, next time you start work on a new API in Ruby, consider using Sinatra and Sequel since they are very powerful tools for such a use case.

This article was written by Boško Ivanišević, a Toptal freelance developer.

The Six Commandments of Good Code: Write Code that Stands the Test of Time

Humans have only been grappling with the art and science of computer programming for roughly half a century. Compared to most arts and sciences, computer science is in many ways still just a toddler, walking into walls, tripping over its own feet, and occasionally throwing food across the table. As a consequence of its relative youth, I don’t believe we have a consensus yet on what a proper definition of “good code” is, as that definition continues to evolve. Some will say “good code” is code with 100% test coverage. Others will say it’s super fast and has a killer performance and will run acceptably on 10 year old hardware. While these are all laudable goals for software developers, however I venture to throw another target into the mix: maintainability. Specifically, “good code” is code that is easily and readily maintainable by an organization (not just by its author!) and will live for longer than just the sprint it was written in. The following are some things I’ve discovered in my career as an engineer at big companies and small, in the USA and abroad, that seem to correlate with maintainable, “good” software.

Never settle for code that just "works." Write superior code.

Commandment #1: Treat Your Code the Way You Want Other’s Code to Treat You

I’m far from the first person to write that the primary audience for your code is not the compiler/computer, but whomever next has to read, understand, maintain, and enhance the code (which will not necessarily be you 6 months from now). Any engineer worth their pay can produce code that “works”; what distinguishes a superb engineer is that they can write maintainable code efficiently that supports a business long term, and have the skill to solve problems simply and in a clear and maintainable way.

In any programming language, it is possible to write good code or bad code. Assuming we judge a programming language by how well it facilitates writing good code (it should at least be one of the top criteria, anyway), any programming language can be “good” or “bad” depending on how it is used (or abused).

An example of a language that by many is considered ‘clean’ and readable is Python. The language itself enforces some level of white space discipline and the built in APIs are plentiful and fairly consistent. That said, it’s possible to create unspeakable monsters. For example, one can define a class and define/redefine/undefine any and every method on that class during runtime (often referred to as monkey patching). This technique naturally leads to at best an inconsistent API and at worst an impossible to debug monster. One might naively think,”sure, but nobody does that!” Unfortunately they do, and it doesn’t take long browsing pypi before you run into substantial (and popular!) libraries that (ab)use monkey patching extensively as the core of their APIs. I recently used a networking library whose entire API changes depending on the network state of an object. Imagine, for example, calling client.connect() and sometimes getting a MethodDoesNotExist error instead of HostNotFound or NetworkUnavailable.

Commandment #2: Good Code Is Easily Read and Understood, in Part and in Whole

Good code is easily read and understood, in part and in whole, by others (as well as by the author in the future, trying to avoid the “Did I really write that?” syndrome).

By “in part” I mean that, if I open up some module or function in the code, I should be able to understand what it does without having to also read the entire rest of the codebase. It should be as intuitive and self-documenting as possible.

Code that constantly references minute details that affect behavior from other (seemingly irrelevant) portions of the codebase is like reading a book where you have to reference the footnotes or an appendix at the end of every sentence. You’d never get through the first page!

Some other thoughts on “local” readability:

• Well encapsulated code tends to be more readable, separating concerns at every level.
• Names matter. Activate Thinking Fast and Slow’ssystem 2 way in which the brain forms thoughts and put some actual, careful thought into variable and method names. The few extra seconds can pay significant dividends. A well-named variable can make the code much more intuitive, whereas a poorly-named variable can lead to headfakes and confusion.
• Cleverness is the enemy. When using fancy techniques, paradigms, or operations (such as list comprehensions or ternary operators), be careful to use them in a way that makes your code more readable, not just shorter.
• Consistency is a good thing. Consistency in style, both in terms of how you place braces but also in terms of operations, improves readability greatly.
• Separation of concerns. A given project manages an innumerable number of locally important assumptions at various points in the codebase. Expose each part of the codebase to as few of those concerns as possible. Say you had a people management system where a person object may sometimes have a null last name. To somebody writing code in a page that displays person objects, that could be really awkward! And unless you maintain a handbook of “Awkward and non obvious assumptions our codebase has” (I know I don’t) your display page programmer is not going to know last names can be null and is probably going to write code with a null pointer exception in it when the last name-being null case shows up. Instead handle these cases with well thought out APIs and contracts that different pieces of your codebase use to interact with each other.

Commandment #3: Good Code Has a Well Thought-out Layout and Architecture to Make Managing State Obvious

State is the enemy. Why? Because it is the single most complex part of any application and needs to be dealt with very deliberately and thoughtfully. Common problems include database inconsistencies, partial UI updates where new data isn’t reflected everywhere, out of order operations, or just mind numbingly complex code with if statements and branches everywhere leading to difficult to read and even harder to maintain code. Putting state on a pedestal to be treated with great care, and being extremely consistent and deliberate with regard to how state is accessed and modified, dramatically simplifies your codebase. Some languages (Haskell for example) enforce this at a programmatic and syntactic level. You’d be amazed how much the clarity of your codebase can improve if you have libraries of pure functions that access no external state, and then a small surface area of stateful code which references the outside pure functionality.

Commandment #4: Good Code Doesn’t Reinvent the Wheel, it Stands on the Shoulders of Giants

Before potentially reinventing a wheel, think about how common the problem is you’re trying to solve or the function is you’re trying to perform. Somebody may have already implemented a solution you can leverage. Take the time to think about and research any such options, if appropriate and available.

That said, a completely reasonable counter-argument is that dependencies don’t come for “free” without any downside. By using a 3rd party or open source library that adds some interesting functionality, you are making the commitment to, and becoming dependent upon, that library. That’s a big commitment; if it’s a giant library and you only need a small bit of functionality do you really want the burden of updating the whole library if you upgrade, for example, to Python 3.x? And moreover, if you encounter a bug or want to enhance the functionality, you’re either dependent on the author (or vendor) to supply the fix or enhancement, or, if it’s open source, find yourself in the position of exploring a (potentially substantial) codebase you’re completely unfamiliar with trying to fix or modify an obscure bit of functionality.

Certainly the more well used the code you’re dependent upon is, the less likely you’ll have to invest time yourself into maintenance. The bottom line is that it’s worthwhile for you to do your own research and make your own evaluation of whether or not to include outside technology and how much maintenance that particular technology will add to your stack.

Below are some of the more common examples of things you should probably not be reinventing in the modern age in your project (unless these ARE your projects).

Databases

Figure out which of CAP you need for your project, then chose the database with the right properties. Database doesn’t just mean MySQL anymore, you can chose from:

• “Traditional” Schema’ed SQL: Postgres / MySQL / MariaDB / MemSQL / Amazon RDS, etc.
• Key Value Stores: Redis / Memcache / Riak
• NoSQL: MongoDB/Cassandra
• Hosted DBs: AWS RDS / DynamoDB / AppEngine Datastore
• Heavy lifting: Amazon MR / Hadoop (Hive/Pig) / Cloudera / Google Big Query
• Crazy stuff: Erlang’s Mnesia, iOS’s Core Data

Data Abstraction Layers

You should, in most circumstances, not be writing raw queries to whatever database you happen to chose to use. More likely than not, there exists a library to sit in between the DB and your application code, separating the concerns of managing concurrent database sessions and details of the schema from your main code. At the very least, you should never have raw queries or SQL inline in the middle of your application code. Rather, wrap it in a function and centralize all the functions in a file called something really obvious (e.g., “queries.py”). A line like users = load_users(), for example, is infinitely easier to read than users = db.query(“SELECT username, foo, bar from users LIMIT 10 ORDER BY ID”). This type of centralization also makes it much easier to have consistent style in your queries, and limits the number of places to go to change the queries should the schema change.

• 
• 
• 

Other Common Libraries and Tools to Consider Leveraging

• Queuing or Pub/Sub Services. Take your pick of AMQP providers, ZeroMQ, RabbitMQ, Amazon SQS
• Storage. Amazon S3, Google Cloud Storage
• Monitoring: Graphite/Hosted Graphite, AWS Cloud Watch, New Relic
• Log Collection / Aggregation. Loggly, Splunk

Auto Scaling

• Auto Scaling. Heroku, AWS Beanstalk, AppEngine, AWS Opsworks, Digital Ocean

Commandment #5: Don’t Cross the Streams!

There are many good models for programming design, pub/sub, actors, MVC etc. Choose whichever you like best, and stick to it. Different kinds of logic dealing with different kinds of data should be physically isolated in the codebase (again, this separation of concerns concept and reducing cognitive load on the future-reader). The code which updates your UI should be physically distinct from the code that calculates what goes into the UI, for example.

Commandment #6: When Possible, Let the Computer Do the Work

If the compiler can catch logical errors in your code and prevent either bad behavior, bugs, or outright crashes, we absolutely should take advantage of that. Of course, some languages have compilers that make this easier than others. Haskell, for example, has a famously strict compiler that results in programmers spending most of their effort just getting code to compile. Once it compiles though, “it just works”. For those of you who’ve either never written in a strongly typed functional language this may seem ridiculous or impossible, but don’t take my word for it. Seriously, click on some of these links, it’s absolutely possible to live in a world without runtime errors. And it really is that magical.

Admittedly, not every language has a compiler or a syntax that lends itself to much (or in some cases any!) compile-time checking. For those that don’t, take a few minutes to research what optional strictness checks you can enable in your project and evaluate if they make sense for you. A short, non-comprehensive list of some common ones I’ve used lately for languages with lenient runtimes include:

• Python: pylint, pyflakes, warnings, warnings in emacs
• Ruby: warnings
• Javascript: jslint

Conclusion

This is by no means an exhaustive or the perfect list of commandments for producing “good” (i.e., easily maintainable) code. That said, if every codebase I ever had to pick up in the future followed even half of the concepts in this list, I will have many fewer gray hairs and might even be able to add an extra 5 years on the end of my life. And I’ll certainly find work more enjoyable and less stressful.

This article was written by Zachary Goldberg, a Toptal freelance developer.