Talking to Erlang

In a previous post, Scalable Web Apps: Erlang + Python, I talked broadly about using HTTP to make external applications talk to an Erlang cluster over the internet or network. The following code is an example of a MochiWeb server set up to receive HTTP requests with embedded JSON. HTTP is basically used as a nice wrapper around whatever data is being sent and data is wrapped-up in a serialized form called JSON. The following goes in the generated MochiWeb skeleton code file called project_name_web.erl:

%% @author Luke Hoersten <Luke@Hoersten.org>
%% @copyright 2008 Luke Hoersten.

%% @doc Web server for example.

-module(example_web).
-author('Luke Hoersten <Luke@Hoersten.org>').

-export([start/1, stop/0, loop/2]).

%% External API
start(Options) ->
    {DocRoot, Options1} = get_option(docroot, Options),
    Loop = fun (Req) -> ?MODULE:loop(Req, DocRoot) end,
    mochiweb_http:start([{name, ?MODULE}, {loop, Loop} | Options1]).

stop() ->
    mochiweb_http:stop(?MODULE).

loop(Req, _) ->
    io:format("~nReceived:~n~p~n", [Req]),
    case {Req:get(method), Req:get_header_value("Content-Type")} of
        {'POST', "application/jsonrequest"} ->
            do_stuff(mochijson2:decode(Req:recv_body())),
            Req:ok("text/plain",
                   [{"User-Agent", "Erlang/example/0.1"}],
                   <<"Processing Request">>)
        _ ->
            Req:not_found()
    end.

%% Internal API
do_stuff(Data) ->
    io:format("~nData: ~p~n", [Data]),
    Nodes = [{'erlang@pod5-1', 4}, {'erlang@pod5-2', 4},
             {'erlang@pod5-3', 4}, {'erlang@pod5-4', 4},
             {'erlang@pod5-5', 4}],
    plists:foreach(fun stuff:do_stuff/1, Data, {nodes, Nodes}).

get_option(Option, Options) ->
    {proplists:get_value(Option, Options), proplists:delete(Option, Options)}.

How it Works

The server is basically a loop which uses MochiWeb to pull HTTP requests out of a TCP/IP socket. If the HTTP header is a JSON HTTP POST, then the JSON is deserialized and applied to the function do_stuff. do_stuff then parallelizes the data over a cluster of Erlang nodes using the plists library for processing which can talk to each other with message passing as needed. The “OK” response is sent back to the application which sent the HTTP request, after the parallelized processing completes, without doing any exception handling.

For reference, here is the example Python from the other end of the connection:

def send_to_erlang(data):
    url = "http://erlang.nodes.tld:8000/"
    body = json.dumps(data)
    headers = {'Content-Type': 'application/jsonrequest',
               'User-Agent'  : 'Python/Project/0.1'}
    urlopen(Request(url, body, headers))

It may seem simple, and it is simple because of Erlang and MochiWeb, but before seeing how this all fits together, it can seem mysterious. Please feel free to comment with questions and I’ll try to clarify as much as possible. Thanks to the MochiWeb mailing list for helping me when I was starting out.

    None Found
  • mark

    luke,
    check this out, a somewhat similar but more constrained way of exposing erlang methods/ functions

    http://www.lshift.net/blog/2007/02/17/json-and-…

  • https://www.humani.st Luke Hoersten

    Good find. Just to clarify for the readers, JSON-RPC is a special specification for calling functions remotely. This example is just a simple HTTP body with JSON. Designed with the explicit intent of simplicity. More functionality would come from adding different URLs like REST. Kevin Smith has actually just completed a screencast that should describe this RESTful setup in more detail. I’ll add a link when it becomes available.

  • daniel palacio

    Did you became to important to write a blog ????????????? how are thing's ???? I have to go to chicago sometime and visit you.

  • http://twitter.com/lukevenedger Luke Venediger

    Hi,

    You're missing a ; after <<”Processing Request”>>).

    Thanks,
    Luke.

  • http://blog.therestfulway.com/ Justin Sheehy

    Luke,

    This is a nice little example, and Mochiweb is indeed a great HTTP driver for Erlang. If you want a layer atop Mochiweb that adds easy and RESTful extensibility to your services, you might want to check out Webmachine:

    http://bitbucket.org/justin/webmachine

    Cheers,

    Justin

  • http://twitter.com/lukevenedger Luke Venediger

    Also, Req:ok() takes a single tuple, not three arguments. So, this:
    Req:ok(“text/plain”,
    [{"User-Agent", "Erlang/example/0.1"}],
    <<”Processing Request”>>)

    Should be this:
    Req:ok( { “text/plain”,
    [{"User-Agent", "Erlang/example/0.1"}],
    <<”Processing Request”>>
    } )

    This had me stumped for hours!

  • https://www.humani.st Luke Hoersten

    Thanks for the fixes. I actually did run this before posting it so they must have changed the interface… as for the missing semicolon, perhaps a bad paste?

  • https://www.humani.st Luke Hoersten

    Looks cool. Thanks.

  • https://www.humani.st Luke Hoersten

    Thanks for the fixes. I actually did run this before posting it so they must have changed the interface… as for the missing semicolon, perhaps a bad paste?

  • https://www.humani.st Luke Hoersten

    Looks cool. Thanks.