Always Move Fast

I realize I haven’t written anything new here lately. I’ve been working on a bunch of pretty cool stuff that I really want to share and get in the open. Hopefully I’ll be writing more about the stuff I’ve been doing.

For database work, I’ve been using Jay McCarthy’s sqlite.plt library. If you’ve used this library at all, you’ll quickly realize that it returns results similar to this:

The common way to parse these records is by using match-let from the match.ss library. Using match-let every time you have to parse a result set can be a pain in the ass. I ended up writing a function that takes a result set like above and transforms it into a list of associative lists. So the above is transformed into:

Now instead of having to use match-let every time, all you have to do is pull the attributes you want by using assoc on each associative list. Since the result above is a list of associative lists, you’d have to iterate over the list and then use assoc. Assuming we only got one result back, it could look like this:

If I wanted to get the username attribute from this associative list, I could do

Not bad, eh? So, what’s the code to transform my records into a list of associative lists?

Granted, this code is kind of funny to look at. Let’s start at the end where I’m returning either res or (car res). If the length of the cdr of the vector list is 1, meaning the list of vectors is (#(”id” “username” “password”) #(”1″ “david” “batman”)) and its cdr is (#(”1″ “david” “batman”)), return the car of the result. If there are multiple records in the cdr of the vector list, return the result as is. The results in either case are associative lists. When the length of the cdr is 1, the resulting list will only have one associative list in it. When the length of the cdr is greater than 1, the resulting list will have multiple associative lists in it that you have to iterate over. Returning the car of the multiple associative lists will give you only the first associative list when you really want all of them.

I hope that makes at least a little sense.

The actual mapping of keys with values should hopefully be pretty straightforward. The inner map:

Returns an associative list where (car list-of-vecs) is #(”id” “username” “password”) and v is a result element in the form of #(”1″ “david” “batman”). The outer map passes that v element from each of the cdr of list-of-vecs, which could be (#(”1″ “david” “batman”) #(”2″ “foo” “bar”)) or similar. Also remember that functions you pass to map don’t necessarily have to take one argument. Like cons, you have to pass two arguments, so mapping cons to data requires two arguments. In this case, the two arguments are (vector->list (car list-of-vecs)) and (vector->list v).

I think that’s about it for this function. If anything is unclear or you have a better solution definitely feel free to leave me a comment.

send/suspend/dispatch

send/suspend/dispatch takes a function as its argument just like send/suspend. The difference is that this page-generating function takes another function as its argument, conventionally called embed/url. embed/url is used to embed continuations into a URL. To generate continuation URLs with embed/url, you pass it a function that takes the HTTP request object as its argument. When that continuation URL is requested, either by clicking a link or posting a form, the continuation is resumed with the HTTP request object being passed to it.

This article’s example will be a simple counter that lets users increment or decrement their counter by clicking the appropriate links. Each link will use embed/url to dispatch to separate functions in the servlet. The two increment functions are really simple. They take the current value of the counter and return its new result.

Most of the work is done in the show-counter function. It shows the current value of the counter and generates links for incrementing and decrementing the counter. I also added a little styling to the links that just separates them with a little whitespace.

This code should be pretty self-explanatory after my little introduction to send/suspend/dispatch at the top of the article. And if you’ve followed along with my previous posts you should know what all the supporting pieces of code do.

Everytime the increment and decrement functions are called, show-counter returns control back to the start function of the servlet. This only allows us to increment or decrement once, so we have to loop each time show-counter returns.

When show-counter returns, it returns the new result of the counter. The loop then calls show-counter again with the new counter value.

This is a pretty good introduction to send/suspend/dispatch I think and it is a building block for bigger and greater things. The code for this article can be found here.

I’m trying to decide what I want to write about next. I have a pretty cool dispatching server that lets programmers create “pretty” URLs similar to Ruby on Rails’ controller/action dispatching. I may write about how to do that and provide a little proof-of-concept web application. I’ve also been meaning to write about keyword tracking in scheme. Keyword tracking is a pretty hot topic in the world of internet marketing. I’ll come up with something to write about. Until next time, hack away at the code and learn some stuff.

send/suspend

Using continuation-based web servers is a very powerful way to develop web applications. Continuations can essentially remove problems associated with back buttons and multiple tabs/windows in web browsers. The basic idea behind web server continuations is that a continuation reference id is passed in the URL of a web request and then the continuation with that id is executed on the server. This allows web developers the ability to maintain state without using sessions. You could even think of the continuation URLs as a basic session id that’s directly tied to the execution of server-side code.

The continuation ids aren’t true session variables. They simply maintain the state of current execution. If you were to use two browser windows and point them at the same continuation-based website, each browser window would have its own unique execution path. On a normal website using cookies to maintain state, both of your browser windows would share data (such as a user id). Using continuations to maintain state allow both of your browser sessions to be completely independent of eachother. Keep in mind that there’s nothing stopping you from using cookies with continuation-based web servers like PLT Web Server.

If you followed along with my previous posts here and here, you should have scheme installed and ready to go. In your /usr/plt/collects/web-server/default-web-root/servlets/examples/ directory there are a bunch of scheme servlet examples. One of the more famous examples used for introducing people to continuation web programming is the add.ss servlet in the examples directory.

I’m including that add.ss example in this article’s source code so all you have to do to test it out is require “main.ss” in your mzscheme prompt and visit http://localhost:3000/servlets/add.ss. When you’re submitting numbers to the servlet, take notice of the strange ids in the URL that look something like “2*1*34452322″. The first number is the servlet instance id, the second number is the continuation id, and the third number is a randomly-generated id that helps prevent users from guessing continuation URLs.

All the code for this article can be found under svn: http://alwaysmovefast.com/public_svn/article_src/two.

Like the simple.ss example, add.ss has a start function where the initial request is passed. When a request for add.ss comes into the server, the start function is executed. Before the start function returns, it calls (request-number “first”), followed by (request-number “second”), and then adds the result of those two function calls.

When (request-number “first”) is called, it runs the build-request-page function inside of a send/suspend call. send/suspend captures the current continuation and binds it to a URL (k-url) that’s passed to the function returned by (build-request-page which-number). Once the lambda function is called with k-url, it’s expected to generate an HTTP response. That response is the HTML form you see on the page.

When you use send/suspend with the continuation-bound URL (k-url), you can use that in your links and form actions. That’s exactly what build-request-page is doing here:

The responses generated by (start) and (build-request-page) are inside a quasi-quote (`). This means that these functions return code as data that will be evaluated later. When you quasi-quote an expression, you’re able to unquote expressions and variables such as k-url and which-number. By unquoting k-url and which-number, we’re telling the interpreter to evaluate them immediately.

When you enter the first number in the form and post it to the continuation URL, the continuation is invoked and the request is returned from send/suspend. The call to send/suspend is wrapped in a request-bindings call. request-bindings takes an HTTP request and returns the variable bindings. The variable bindings can then be extracted. In this case, the code uses extract-binding/single to request a single variable from the request bindings. The text field of the form is named “number”, so in extract-binding/single, we pass ‘number as the binding to be extracted from the request-bindings call. Once the number value is extracted, we have to convert it from a string to an integer by using string->number. The number is then returned as the last evaluated expression of request-number.

After (request-number “first”) returns with an integer, (request-number “second”) is called and follows the same path as (request-number “first”). When (request-number “second”) returns, both values are added together and then converted from an integer to a string in the start function.

The start function then returns its HTTP response with the sum of the two numbers.

There is a major limitation when using send/suspend: you can’t easily generate multiple links and forms from one continuation URL unless you’re content with using URL query strings. This means that all your requests will essentially go to one function and then you will have to dispatch based on the query string values. It’s definitely not very elegant. One of the solutions to this problem is by using send/suspend/dispatch, which I’ll be covering in my next blog post.

Although send/suspend is somewhat limited, you should still hack around with it and create some cool things to get used to it.

Building the Server

Web application development in Scheme is something I’ve just recently gotten into. It’s been a challenging and exciting journey and I’m happy to begin sharing what I’ve learned. I figured I would start with writing a custom web server built from PLT Web Server that serves files and does some basic servlet dispatching.

If you don’t yet have PLT Scheme installed, read my post on Installing PLT Scheme from Source.

PLT Web Server technology allows developers to essentially create their own web servers in scheme. I’m going to start with a basic web server that’s built off of the /usr/plt/collects/web-server/run.ss web server. That will allow us to evolve the code over the course of these articles as we get into some more advanced techniques.

All of the code for this article can be found here: http://alwaysmovefast.com/public_svn/article_src/one/

Go ahead and download it so you can follow along easier and actually run the software on your machine. To run the server:

1. cd into the directory
2. mzscheme
3. (require “main.ss”)

The server should be started now so point your web browser to http://localhost:3000/
and you should see the index.html file.

At this point, I’m not going to go in-depth with what the server is actually doing. For now I’m only going to be talking about building servlets so you shouldn’t have to worry about the server for now.

Go ahead and point your browser at http://localhost:3000/servlets/simple.ss now. The code for this servlet is pretty short:

All you really need to know about this module is that its start function is called when the web server receives a request for it and the request is passed to the start function. The start function then passes back a response to the browser as html.

As a little exercise you could implement your own servlets in the servlets/ directory so you can get a feel for scheme web development.

I’m going to be writing a bunch of follow-up posts on different aspects of scheme web development. The next article is going to be on continuation-based web development. For reference, I’ll be talking about send/suspend and send/suspend/dispatch, so feel free to read up a little on those beforehand.

Installing PLT Scheme

To install for Unix-based machines:
1. cd /usr/src
2. curl -O http://download.plt-scheme.org/bundles/372/plt/plt-372-src-unix.tgz
3. tar xzf plt-372-src-unix.tgz
4. cd plt-372/src
5. ./configure
6. make
7. make install

One thing to note at this point is that the installation is in-place, meaning it will
install everything to /usr/src/plt-372/.

So if you want to install in a separate directory, I normally just mv the plt-372
directory to /usr/plt.

8. cd /usr/src
9. mv plt-372 /usr/plt

Now you need to make sure the binaries are executable from your path.

10. vim ~/.bash_profile

Add /usr/plt/bin to your path by doing something like export PATH=/usr/plt/bin:$PATH

11. source ~/.bash_profile

Type mzscheme at the console. If it drops you into the mzscheme interface then you’re golden. Exit mzscheme by doing a Ctrl+D command.