Methods Part II: RubyGems

Hydraulics on Fifth Avenue. (Photo by Dan Nguyen)

Don't worry if the previous chapter on methods was tough to understand. You still know enough to start using the methods by other programmers to start doing some powerful operations.

Installing Gems With RubyGems

This lesson requires that you have an active Internet connection (in case you're reading an offline copy of this.)

Note: My installation guide can't always keep up with the changes or variations. An excellent guide that has worked for me and others is Moncef Belyamani's exhaustively detailed and illustrated guide.

RubyGems is a handy system that Ruby developers use to distribute code to each other. To install the Ruby on Rails framework, for example, we simply go to the command line and type:

gem install rails

RubyGems is how we get to use all the powerful code libraries contributed by the many developers out there. When we want to use someone else's code for our programs, we typically say "let's install their gem."

Using RubyGems

If you got this far with the installation steps I described, then RubyGems should already be installed.

Go to your command prompt and type in: gem -v

You should get a version number, something like: 1.5.1

If that didn't work, see if the manual will help. Otherwise, we're ready to install some gems.

The basic procedure to install gems is to go to the command prompt and type in:

gem install the_gem_name_here

On a Mac, depending on how you installed Ruby, you might need to add sudo in front of gem. You shouldn't have to if you installed using Homebrew and/or RVM. So try gem install the_gem_name without the sudo first.

Refer to the excellent RubyGems tutorial for more information.

Return to chapter outline

Fetch data from the Internet with the `Rest-Client` gem

The first gem we'll try out is rest-client. This is how you install it:

gem install "rest-client"

rest-client (official instructions here) provides a variety of methods to simplify the sending and retrieval of data such as webpages. The simplest method is get, which for our basic purposes, does about the same thing that the open method provided by open-uri (as we saw at the end of the previous chapter). Here's how to fetch the Wikipedia homepage:


require "rubygems"
require "rest-client"
res = RestClient.get("http://en.wikipedia.org/wiki")
puts res.code
#=> 200

puts res.body
#=> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 
#=> "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
#=> <html lang="en" dir="ltr" class="client-nojs" xmlns="http://www.w3.org/1999/xhtml">
#=> <head> ...

Here's a breakdown of the code:

require "rubygems": The keyword require is how we bring in code libraries for our program to use. To access installed gems, we first require "rubygems"
require "rest-client": Here we require the gem we want to use, "rest-client". And just like that, we can start using the RestClient class.
RestClient.get: The get method of the RestClient class simply retrieves a webpage from the address you pass it using the HTTP GET protocol. This is basically what your web browser does every time you give it a standard website address.
code: The RestClient.get method returns an Object that contains a HTTP status code. You've probably heard of a 404 error, the status code indicating that a webpage is not at the specified URL. A 200 indicates a successful GET request.
body: The RestClient response object also has a body method that contains the raw HTML or other content from the request.

Return to chapter outline

Application Programming Interfaces (APIs)

If you've ever had to look up directions, you've probably used Google's mapping services, which lets you navigate dynamic maps with your web browser. For us programmers, Google offers programmatic ways to draw maps and find addresses through its APIs.

API stands for Application Programming Interface, a fancy term for the formalized rules your program has to follow if it wants to communicate with another program. In this case, the other program is Google's mapping service.

Practice APIs with Static Maps

For those of you totally unfamiliar with APIs, let's do a little visual practice with Static Maps API. This is how it works:

You provide a URL which contains a description of the map you want.
How do you know how to write this description? Only by reading Google's documentation for the Static Maps API

After perusing the instructions, you learn that Google wants a URL in this format:

http://maps.google.com/maps/api/staticmap?sensor=false&size=550x300&markers=902+Broadway+New+York,NY

Visit that URL with your browser to see the map Google draws for you.

You should see something like this:

Parameters for Google Static Maps

http://maps.google.com/maps/api/staticmap

This is the endpoint of the API, the base address of the service. Think of it as the location of the remote script that will be reading your input and responding back. In this case, if you just visit the endpoint http://maps.google.com/maps/api/staticmap without any parameters, the script won't serve anything to you.

?

This is a delimiter that separates the endpoint from the parameters that you use to specify the details of your request, such as what address you want to find.

some_key=some_value

The parameters you pass in will be in key-value pairs. The key is the name of the property, such as size. The equals sign = is how you specify the value (e.g. 300x200) for the key.

How do you know what the keys of an API are? Read the documentation. There's no special science to it besides following the instructions.

&

Each parameter key-value pair is delimited by an ampersand

The following are the basic parameters for the static maps API:

sensor=false: This can either be true or false and indicates whether or not the browser is keeping track of the user's geo-location. For our purposes, we leave it to false.
size=550x300: This specifies the widthxheight dimensions of the map image that we would like.
markers=902+Broadway+New+York,NY: This specifies the address in which we want the map image to focus around. The + sign denotes a space character in the URL (this is true across all Web addresses, not just this particular API).

Here I modify the parameters slightly to get this image:

http://maps.google.com/maps/api/staticmap?sensor=false&size=300x300&markers=3100+Broadway+New+York,NY

Exercise: Test out the Static Maps API

Using the Google Statics Map API, what are the HTTP calls needed to generate the following maps?

400 pixels wide and 250 pixels high
At a zoom level of 4
Includes a green marker at 100 Wall Street, NY and a blue marker at 100 Main St. Queens, NY; has a zoom level of 9; and is 500x350

Solution

Return to chapter outline

Google's Geocoding API

Let's write a program to get latitude and longitude coordinates for any given address. For this we'll use the Google Geocoding API.

Test out the API. Open a web browser window and go to the following address:

http://maps.googleapis.com/maps/api/geocode/xml?address=1400+Broadway,+New+York,+NY&sensor=false

You should see something like this:

Taking the parameters from our HTTP request, Google responds with a mountain of geographical information about the address "1400 Broadway, New York, NY" in the XML format.

If you're completely unfamiliar with XML – think of it as a markup language similar to HTML, but for more general use cases and applications.

Instead of having tags for just paragraphs, links, images, and other webpage elements, you can define XML tags for any kind of data. In Google's XML response, you can see how the <latitude> tag, wrapped in between the <location> tags, contains the latitude coordinates for the address

Return to chapter outline

Crack XML with the Crack Gem

The crack gem turns XML into an easier-to-handle data structure that's called a Hash. I go more into detail about hashes in the collections chapter.

For now, try to follow the patterns and symbols, even if you don't explicitly understand them yet.

Here's how to access the latitude and longitude of Google's response with methods from the crack gem.


# res contains the response as retrieved through RestClient.get                     
parsed_res = Crack::XML.parse(res)

status = parsed_res["GeocodeResponse"]["status"]
lat = parsed_res["GeocodeResponse"]["result"]["geometry"]["location"]["lat"]
lng = parsed_res["GeocodeResponse"]["result"]["geometry"]["location"]["lng"]

Can you see how the strings "GeocodeResponse" and "result" correspond to the XML tags <GeocodeResponse> and <result>?

Note the use of the dot operator: parse is a method that belongs to the object Crack::XML

Essentially, the use of square brackets allow us to "name" the XML elements that we're trying to access. Below is an excerpt of Google's XML response. I've bolded the parts affected by the Crack::XML.parse method calls:

   <?xml version="1.0"?>
   <GeocodeResponse>
      <status>OK</status>
      <result>
      
      ...
      
      <address_component>
         <long_name>10018</long_name>
         <short_name>10018</short_name>
         <type>postal_code</type>
      </address_component>
      <geometry>
      <geometry>
         <location>
            <lat>40.7530520</lat>
            <lng>-73.9870970</lng>
         </location>
         <location_type>ROOFTOP</location_type>
         <viewport>
            <southwest>
      ...
      </result>
   </GeocodeResponse>

As you'll learn in the Collections chapter, the use of square brackets and strings allow us to name the XML elements that we're interested in:


parsed_res = Crack::XML.parse(res)      
puts parsed_res["GeocodeResponse"]

This parses the <location> element. Adding another set of square brackets allow us to target an element nested within <GeocodeResponse>:


puts parsed_res["GeocodeResponse"]["status"]

Exercise: Write a method to read the coordinates

Use what you know about methods and their structure to create a method that takes in an address string as an argument and uses RestClient and Crack methods to retrieve the latitude and longitude from Google's Geocoding API. Return the coordinates as a string.

You'll need to use one more method that I haven't covered yet: The URI module has an encode method that converts any invalid characters in a URL to their proper URL-safe entities. For example, whitespace isn't allowed in properly-formatted URLs. So URI.encode will convert white spaces to %20:


str = "http://mysite.com?q=This string needs to be encoded!"      
puts URI.encode(str) 
#=> "http://mysite.com?q=This%20string%20needs%20to%20be%20encoded!"

Don't worry about memorizing what characters need to be converted; URI.encode will take care of those details for us.

Solution


require 'rubygems'
require 'rest-client'
require 'crack'

def get_coordinates_from_address(addr)
   base_google_url = "http://maps.googleapis.com/maps/api/geocode/xml?sensor=false&address="
   res = RestClient.get(URI.encode("#{base_google_url}#{addr}"))
   parsed_res = Crack::XML.parse(res)
   lat = parsed_res["GeocodeResponse"]["result"]["geometry"]["location"]["lat"]
   lng = parsed_res["GeocodeResponse"]["result"]["geometry"]["location"]["lng"]
   
   return "#{lat}, #{lng}"
end

No new concepts here, just wrapping up a series of method calls so that any user using this method just has to write a single line to turn an address into mappable coordinates:


latlng = get_coordinates_from_address("1 Times Square, NYC")
puts latlng   
#=> 40.7589600, -73.9851950

I jumped into how to use RestClient and Crack just to acquaint you with some interesting methods. Count on using them in many other projects.

You're not expected to understand all of the syntax and concepts introduced here. Some of it, such as the Hash object, we'll cover very soon. Some of the under-the-hood code within the gems you may never get around to studying.

But this is how useful programming is achievable by beginners. As long as you have a problem you want to solve – and you know what tools are available to you – you can leave the implementation details to those who are better qualified to handle them.

Hopefully, some day, you'll be someone whose code gets reused. But baby steps first.

Installing Gems With RubyGems

Using RubyGems

Fetch data from the Internet with the Rest-Client gem

Application Programming Interfaces (APIs)

Practice APIs with Static Maps

Parameters for Google Static Maps

Exercise: Test out the Static Maps API

Solution

Google's Geocoding API

Crack XML with the Crack Gem

Exercise: Write a method to read the coordinates

Solution

Fetch data from the Internet with the `Rest-Client` gem