Wednesday, August 10, 2016

Social Network APIs: The Internet’s Portal to the Real World

Social network APIs have come a long way since Google released the first version of its YouTube API in May 2008 and Facebook released the first version of the Graph API in April 2010. Today, these APIs give you the opportunity to query social network platforms for posts, users, channels, and demographic data. They even let you create your own service or find out more about your user base.
In this article, we will examine the ways we can utilize some of the popular social network APIs:
  • Facebook (Graph and Marketing API)
  • Instagram
  • Twitter
  • YouTube
  • Pinterest
We will also discuss their limitations, and explore some of the useful tools that are available for use with these APIs. Later in the article, we will also take a look at how to integrate these APIs in any Rails application.
Social Network APIs: The Internet’s Portal to the Real WorldUse social network APIs to get to know your users better than they know themselves.
I will focus on one social network API at a time and explain its capabilities, limitations and available tools. There will be a matrix with the different APIs and their properties for better comparison later in this article.
In order to use the APIs you will first need to setup an app that creates queries on behalf of your application with OAuth based requests. Users will authenticate against your app and you can then access their data with the resulting user access-token.

FACEBOOK

The now outdated FQL (Facebook Query Language) used to be a SQL-like query language that could be used to access all data from Facebook.
Facebook released the first version of its Graph API in April 2010. The most recent version at the time of writing this article is 2.6 which was introduced on April 12, 2016. It is a low level HTTP-based API that can be used to query data, create posts, and even create automated ad campaigns.

Tools

The Graph API Explorer is the most commonly used tool when working with the Facebook API. It lets you execute Graph API queries in the browser and examine the results: You can use one of your app’s access tokens or create one on the fly with selected scopes.

Capabilities

The Graph API is a REST-based API that lets you create, update, and delete objects per HTTP request on certain nodes.

Access Token

To run queries against the Graph API, you need an access token that is obtained as soon as a user successfully authorizes in your app. The access token should be stored by your application.

Scopes

Scopes determine which actions can be performed on behalf of a user. The application asks for certain scopes when a user authorizes in an app. The publish_actions scope, for example, lets an app publish posts on behalf of a user. The email scope lets the app read the user’s email. A full overview over all scopes is listed in the official documentation.
Certain scopes like the publish_actions or ads_management require a review by Facebook prior to the release of the app.

Examples

To demonstrate how the Graph API works, I will show you how to read, create, update, and delete posts with the API.
To get your own posts, you can execute the GET query /me/posts. The result will be a JSON string with a list of posts, including their message, created_time, and id.
To get more data about your posts, you can extend the query with fields as query parameters. For example, the query me/posts?fields=reactions, picture will give you the post’s picture and reactions.
To create a post, you can simply send a POST action against the edge feed, e.g. me/feed, with parameters such as message: hello world. The Graph API will return a JSON object with the ID of your created post. You can then view the post at the address http://facebook.com/[post_id].
To update a post, you can send a POST request to the post’s node with the fields to be updated as parameters; e.g., /[post_id] and params like Message: lorem ipsum. A success indicator with a value of true or false will be returned.
To delete a post, you can simply make a DELETE request to the node with the post ID (e.g., /[post_id]). The return value will be a JSON object with a success value of true or false.
A full overview over all nodes and actions is available in the Graph API Reference.

Marketing API

The marketing API deserves special mention because it is a powerful tool to manage Facebook ads and get ad insights through your application.
It works the same way as other Graph API methods. However, you need the ads_management scope in order to get access to the user’s ads. Facebook also needs to review your app before you can publish it.

Testing

Once you create your app, it is in development mode and automatically visible in your app dashboard (i.e., https://developers.facebook.com/apps/).
In development mode, only admins, devs, and testers have access to your app. You can add testers and admins in the roles section of your app dashboard.

Review Process

When adding certain permissions,Facebook needs to review your app before you can publish it. The review process is defined by this set of guidelines.
In order to submit certain items for review, you can simply add them in the App Review section of your app dashboard. Facebook will then guide you through the review process and you will be alerted once your app is approved.

Limitations and Workarounds

Rate Limits

An app can make 200 calls per hour per user in aggregate. If you reach that limit, your API calls will result in error.

Searching for Posts on Facebook

Facebook restricts searching for posts and tags on Facebook through the Graph API and FQL. However, you can use the Google Search API to search for public Facebook posts and then use the post-id in the URL to retrieve more information about specific posts through the Graph API.

Getting Custom Audience Data

Audience Insights on Facebook is a powerful research tool to learn more about a particular audience based on interests, demographics, or a other attrributes (e.g., a collection of email addresses).
However, I have not found a way to automatically create audience insights through the ad API. Let us know in the comments if you have any creative ideas or suggestions for this.

INSTAGRAM

The Instagram API was first released in April 2014 and allows you to build apps that analyze user posts and help users to manage their own posts.

Tools

Since the API console by Instagram is deprecated at the time of this article, I recommend using Apigee for testing purposes in your browser.

Capabilities

The Instagram API is a REST-based API. All of its endpoints are described in their official documentation.

Access Token

To run queries against the Instagram API, you need an access token that is obtained as soon as a user authorizes in your app. In order for a user to receive an access token, he or she must be directed to your app’s authorization URL. The server will then redirect the user after authorizing your app and you will then be able to read the token.

Scopes

Your app can ask for different permissions. For instance, “basic” limits you to reading a user’s profile info and media. “public_content” lets you read any public profile and media on behalf of a user.

Examples

To demonstrate how the Instagram API works, I will go through some examples based on the media endpoint https://api.instagram.com/v1/media/popular.
This endpoint returns the currently popular media from Instagram if passed an access token as a parameter. The result will be a JSON array of posts containing, for each, its media ID, a link to its image, likes, comments, the user that posted it, and some other attributes.
You can use apigee to play around and find out more about the API endpoints and their parameters.

Testing

Every new app created on the Instagram platform starts in sandbox mode. This is a fully functional environment that allows you to test publicly available API endpoints before you submit your app for review.
To test your app, simply create a staging version and run all queries through that version instead of the live version that got through the review.

Review Process

Apps in sandbox mode can use any API endpoint, but are restricted to a limited number of users and media. It’s a great mechanism for developing and testing an app.
To go live and access all Instagram content, you will need to submit your application for review. Once reviewed, you will only be able to request the scopes for users for which your app was approved.

Limitations and workarounds

Demographic Analysis

At the time of writing this article, there is no way to get information about a public users’ age, gender, or interests, because Instagram does not provide you with that information.
In order to get demographics data about followers or a list of Instagram users, you would need to iterate over all of them and try to determine their age and gender or interests based on their followers or the information provided in their bio.
A good big data solution for this problem might be a valuable service to some companies.

Rate Limits

All rate limits on the Instagram platform are controlled by access token on a sliding 1-hour window. Live apps have higher rate limits than apps in Sandbox Mode. The global rate limit for a live app is currently 5,000 calls per hour.

TWITTER

The Twitter API was first released in September 2006. It is a public REST API that provides read and write access to Twitter data. Authentication is performed using OAuth. Responses are in JSON format.

Tools

Twitter has an API console tool powered by apigee that can be used to test requests in the browser.

Capabilities

The REST API lets you get a user’s tweets, followers, and followed people. You can also search for hashtags in other tweets.

Access Token

Twitter lets you create apps that users can authenticate against in return of an access token. The authentication model is OAuth.

Scopes

There are only two permissions that have to be set on the app’s setting page: Read only and Read and Write. The latter lets you create tweets and perform other post actions on behalf of a user.

Examples

To demonstrate the usage of the Twitter API I will retrieve the authorized user’s tweets. The result is a JSON array with the tweet’s images, favorites, retweets, urls, creation date, and other attributes. Use Apigee to play around and find out more about the API endpoints and their parameters.

Testing and Review Process

There is currently no review process or test mode available for the Twitter API.

Limitations and Workarounds

Demographic Analysis

There is currently no easy way to get demographic data from someone’s Twitter followers. The brute force approach would be to browse through each follower and try to get the data through their bio and linked social network accounts.
You can then make further assumptions based on the collected follower data through data analysis. Another way to get more insights is through Twitter’s paid enterprise API platform GNIP. Among other things, it lets you create audiences and get information about those through the API. The API is currently in BETA.

Rate Limits

Twitter has rate limits on a per-user basis and on a 15 minute basis. If your application has multiple tokens, you can simply alternate tokens for public operations in order to avoid reaching the limit.

YOUTUBE

The YouTube Data API was first introduced in January 2013. It lets you add YouTube features to your application, search for content, and analyze a YouTube channel’s demographics. It is an OAuth, token-based REST API that returns JSON responses.

Tools

The API Explorer lets you test unauthorized and authorized requests. You can run requests from your browser against the provided endpoints.

Capabilities

Among other things, you can work with activities, chats, live broadcasts, playlists, channels, videos, and subscriptions. Most of the endpoints require you to authorize with a YouTube account.

Access Token

The YouTube Data API supports the OAuth 2.0 protocol for authorizing access to private user data. Once a user has been authorized in your application, they will be redirected to your application where the access token should be saved.
In order to use OAuth 2.0 authorization, you first need to obtain authorization credentials in the Google developer console.

Scopes

The YouTube Data API currently supports the following scopes:
  • Force SSL - Manage your youtube account but only over an SSL connection.
  • Default - Manage your YouTube account. This scope is functionally identical to the youtube.force-ssl scope but does not require an SSL connection.
  • Read Only - View your YouTube account.
  • Upload - Upload YouTube videos and manage your YouTube videos.
  • Partner Channel Audit - Retrieve information that Multichannel Networks use as criteria to accept or reject a channel in their network.

Examples

As an example of usage of the Youtube Data API, the following request queries for videos with “coding” in their title and description:
https://www.googleapis.com/youtube/v3/search?part=snippet&q=coding&key={YOUR_API_KEY}
The result is a JSON object containing the title, description, videoId, and channelId. You can use the latter to find out more about the channel.
The part parameter is required for any API request that returns a certain resource. The parameter identifies resource properties that should be included in an API response. For example, a video resource has the following parts: snippet, contentDetails, fileDetails, player, processingDetails, recordingDetails, statistics, status, suggestions, topicDetails.
All other parameters, except the API key, differ from call to call. Read more about it in the API reference guide.

PINTEREST

The Pinterest API was initially released in April 2015. It is a RESTful API that provides access to a user’s Pinterest data, such as their boards, pins, followers and more. The Pinterest API uses OAuth and allows both read and write permissions when interacting with a user’s content.

Tools

Like others, Pinterest provides an API Explorer to test their endpoints and run queries against them. You can have a look at all their tools here.

Capabilities

The Pinterest REST API allows you to create pins, boards and query Pinterest data with OAuth.

Access Token

Pinterest uses OAuth 2.0 to authenticate requests between your app and your users. All requests must be made over HTTPS.

Scopes

Scopes determine what an app can do on behalf of a user. Pinterest uses the following scopes:
  • none (must know the identifier): Use GET method on a user’s profile, board and Pin details, and the Pins on a board.
  • read_public: Use GET method on a user’s Pins, boards and likes.
  • write_public: Use PATCH, POST, and DELETE methods on a user’s Pins and boards.
  • read_relationships: Use GET method on a user’s follows and followers (on boards, users and interests).
  • write_relationships: Use PATCH, POST, and DELETE methods on a user’s follows and followers (on boards, users and interests).

Examples

To demonstrate the use of the Pinterest API, I will demonstrate how to read the user’s latest pins:
https://api.pinterest.com/v1/me/pins/?access_token={your_token}&fields=id,link,note,url,counts,board,created_atwill return a user’s pins with their id, link, note, url, likes, and repins.

Testing and Review Process

Apps are initially in development mode and must be submitted for review before they are released in production mode.

Limitations and workarounds

Demographic Analysis

There is no common way to get demographics data from a board. However,you can try to get a board’s followers and information about them from their bio and links to other social network accounts. A big data solution over the user’s common connections would also be a possibility.

Search for Pins

There is currently no way to search for pins with certain tags or keywords through the API. You can bypass that limitation by using the Google Custom Search API to search for results on Pinterest pins only and gather the pin ID through the URL. The ID can then be used to get information about the pin through the API.

Rate Limits

Each app (with a unique app ID) is allowed 1,000 calls per endpoint per hour for each unique user token.
Every API response returns a header that gives you an update about rate limits. X-Ratelimit-Limit is the rate limit for that specific request, and X-Ratelimit-Remaining is the number of requests you have left in the 60-minute window.
If you exceed your rate limit for a given endpoint, you’ll get a 429 “Too many requests” error code.

COMPARISON OF SOCIAL NETWORK APIS

VersionOAuthFormatDemographics
Facebookv2.6
Initial Release: April 2010
OAuth 2REST requests with JSON responsesSupported
Instagramv1
Initial Release: April 2014
OAuth 2REST requests with JSON responsesNot supported
Twitterv1.1
Initial Release: September 2006
OAuth 1REST requests with JSON responsesOnly supported with GNIP
YouTubev3
Initial Release: January 2013
OAuth 2REST requests with JSON responsesSupported
Pinterestv1
Initial Release: April 2015
OAuth 2REST requests with JSON responsesNot Supported

DEMO APPLICATION WITH DEVISE

Integrating these APIs in your new or existing applications, thanks to a plethora of social network API packages and libraries, is easier than ever. Most modern platforms and frameworks have time-tested third-party libraries that even unify the authentication aspect all these APIs into a single library with neat plugin architecture.
For this article, we will take a look at how Devise, a Ruby gem, does this ever so elegantly for Rails applications. Devise is a flexible authentication library based on Warden that implements authentication, registration, login, and data storage for multiple login providers. If you are more of a front-end guy and want to check something similar out for AngularJS, take a look at this article.
Devise, like most libraries of this class, doesn’t come built-in with support for any of the above mentioned social network APIs. Support for each of these social network API is provided through additional gems. The following gems are available for Rails authentication that cover the 5 providers discussed in this article:
gem 'omniauth-facebook'
gem 'omniauth-pinterest'
gem 'omniauth-twitter'
gem 'omniauth-google-oauth2'
gem 'omniauth-instagram'
One of the best things about Rails is that there are many plugins written by the open source community. These are distributed as gems. Listed under a central configuration file, these gems are managed by Bundler.
Since these only provide authentication, registration, login, and storage for each of those providers, we will also need to get the following gems for the actual API clients:
gem 'twitter' # https://github.com/sferik/twitter
gem 'instagram' # https://github.com/facebookarchive/instagram-ruby-gem
gem 'koala' # (Facebook API) https://github.com/arsduo/koala
gem 'google-api-client' # (YouTube API), https://github.com/google/google-api-ruby-client
gem 'pinterest-api' # https://github.com/realadeel/pinterest-api

Omniauth and Authentication

In order for a user to authorize your app with your provider, you can simply provide a link with the following path:
omniauth_authorize_path('user', :facebook)
omniauth_authorize_path('user', :instagram)
...
In order to react on the callback after authenticating a user you can define a OmniauthCallbacksController with the scopes as functions like:
class AuthenticationsController < Devise::OmniauthCallbacksController
  def facebook
    if request.env["omniauth.auth"]
   ...
    end
  end
end
That is the place to add a new Authentication model with the token and data into your application:
authentication = where(provider: omniauth.provider, user_id: user.id)
  .first_or_create do |auth|

    auth.user = user
    auth.uid = omniauth.uid

    auth.secret = omniauth.credentials.secret
    auth.token =  omniauth.credentials.token
    ...
end

Making API Calls

Here is an example of how to use Koala to query the Facebook API. The rest of the providers work more or less similarly and are documented in the gem’s README.
This is how you get your user data using Koala:
authentication = user.authentication_for_provider(:facebook)
token = authentication.token
api = Koala::Facebook::API.new(token)
results = api.get_object("me")
You can then use the JSON result returned by the API. Source code of this demo application is available on GitHub.

WRAP UP

Social network APIs provide you with a powerful tool to query the large data set of social networks and collect big data for your application. You can build a service on top of these APIs or use them to enhance your own application and user insights.
Rails and the available gems make it easy to integrate these APIs into your rails app and query the interfaces with an abstraction layer between your app and the API.
This article was written by Behsaad Ramez, a Toptal Ruby developer.

4 comments:

Suseela said...


Awesome article. It is so detailed and well formatted that i enjoyed reading it as well as get some new information too.


SAP HR training in Chennai

Arkon Web Solutions said...

Wonderful Blog, It is really helpful for users. thanks for sharing
Arkon Web Solution, the best digital marketing agency usa performs with 100% client satisfaction.

ankit said...

I’m not that much of an internet reader to be honest but your blogs really nice, keep it up! I’ll go ahead and bookmark your website to come back down the road sap mm training hyderabad

markjack said...

Happy to found this blog. I have some facts related to this blog and I would like to share with all its readers. Definitely it is going to help everyone and aware people with some more knowledgeable points.Discord Javascript Error