In this blogpost we are going to show you how to use Mongoose to file upload to Amazon S3 service. In essence, this is an example on how to make a RESTful service that uses another RESTful service. As always, the full source code of this example is available on Github. All you need to do is to clone the repo and type “make” in the example directory.

How the example works

In order to try this example, you need to make an S3 bucket to store your files. First, login to Amazon IAM console, at get security credentials from there:

  • Access Key ID
  • Secret Access Key ID

Then, create an Amazon S3 bucket.

The next step is to build an example. This is simple, just type “make” in the example directory. When the example is built, it produces a binary executable file which can be started from the shell:

$ ./restful_server_s3
Usage: . -a access_key_id -s s_secret_access_key [-p port] [-D hexdump_file]
$ ./restful_server_s3 -p 8080 -a XXX -s XXX
Starting RESTful server on port 8080

Point your browser to it, you’ll see this:

Screen_Shot_2016-06-08_at_13.30.55.png

You can choose a file to upload, press the button and a file will get placed in Amazon S3 bucket.

The code explained

I’ll skip through the usual boilerplate of parsing command line arguments and creating a listening connection, and start straight from the event handler function. Our service is a usual web server, which has an “/upload” endpoint that takes submitted form data and stores files on Amazon S3. So, the event handler treats “/upload” specially. For all other URLs it acts as a simple static file server:

static void ev_handler(struct mg_connection *nc, int ev, void *ev_data) {
struct http_message *hm = (struct http_message ) ev_data;
switch (ev) {
case MG_EV_HTTP_REQUEST:
if (mg_vcmp(&hm->uri, "/upload") == 0) {
handle_api_call(nc, hm); / Handle RESTful call /
} else {
mg_serve_http(nc, hm, s_http_server_opts); / Serve static files */
}
break;
case MG_EV_CLOSE:
unlink_conns(nc);
break;
default:
break;
}
}

Note the MG_EV_CLOSE snippet - unlink_conns() call. I’ll get to that later.

What should the upload handler do?

  • Parse a form input and get uploaded file data.
  • If there is any parse error, respond with an error.
  • If everything is fine, create a connection to the S3 service and send file there, using specific S3 API.

All this is done by the handle_api_call() function:

static void handle_api_call(struct mg_connection *nc, struct http_message hm) {
char file_name[100], file_data[100], host[100], bucket[100];
/ Get form variables /
mg_get_http_var(&hm->body, "file_name", file_name, sizeof(file_name));
mg_get_http_var(&hm->body, "file_data", file_data, sizeof(file_data));
mg_get_http_var(&hm->body, "host", host, sizeof(host));
mg_get_http_var(&hm->body, "bucket", bucket, sizeof(bucket));
/ Send headers /
mg_printf(nc, "%s", "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n");
/ Send body */
if (file_name[0] == '\0' || file_data[0] == '\0' || bucket[0] == '\0') {
send_error_result(nc, "bad input");
} else {
send_s3_request(nc, file_name, file_data, host, bucket);
}
}

The interesting part is the send_s3_request() function. It should:

  1. Create an outgoing HTTP connection to the S3 web service
  2. Prepare an S3 authentication HTTP header for that outgoing connection
  3. Send a request to the S3 service
  4. Interlink a connection from the user and outgoing connection to the S3 service. When we get a reply from S3 service, we’ll be able to forward the reply to the user.

All this is done in the send_s3_request() function, the most interesting part is how the connections are interlinked:

static void link_conns(struct mg_connection *c1, struct mg_connection *c2) {
c1->user_data = c2;
c2->user_data = c1;
}

So if we take any connection, we can form the following conclusions:

  • If a connection is inbound, then it’s a connection from the user
  • If a connection is outbound, it’s a connection to S3
  • If the user_data is not NULL, then it points to the corresponding inbound (or outbound) connection.

The logic of the S3 connection handler is now clear. When we receive a reply, we send an error status to the user connection. Then we unlink both connections. Also, we unlink connections in the server event handler when either of the connections terminates:

static void s3_handler(struct mg_connection *nc, int ev, void *ev_data) {
struct http_message *hm = (struct http_message *) ev_data;
struct mg_connection *nc2 = (struct mg_connection *) nc->user_data;

switch (ev) {
case MG_EV_HTTP_REPLY:
if (nc2 != NULL) {
mg_printf_http_chunk(nc2, "Error: %.*s", (int) hm->message.len,
hm->message.p);
mg_send_http_chunk(nc2, "", 0);
}
unlink_conns(nc);
nc->flags |= MG_F_SEND_AND_CLOSE;
break;
case MG_EV_CLOSE:
unlink_conns(nc);
break;
default:
break;
}
}

In a nutshell, this is how it works:

  • Accept inbound connection, parse it and create an outbound connection to the backing RESTful service.
  • Interlink both connections in order to know how to forward data between them.
  • When an outbound connection receives a reply, forward it to the inbound connection, and let the outbound connection terminate.
  • If either of the connections terminate, unlink them from each other.

Enjoy implementing this example, and if you haven't already done so, you can download Mongoose here.

To contact: send us a message or ask on the developer forum.