Saturday, December 23, 2006

Priorities for a New Year

Priorities for a New Year

Sorting out projects in the fire form projects on fire

Happy Holidays to everyone! I hope you are all able to spend some time with your families. and have that singular joy of watching a kid's face as he opens a present that knocks their socks off. If that doesn't put a smile on your face, then I afraid nothing will. When I was a kid it was all about whether the next package under the tree had my name on it or not. I'm afraid its come full circle now. My recommendation to those without children is to get in good with friends that do so you can give their kids toys at the holidays. Nothing beats it.

It's about this time of year, when I've been stuffed with too many pieces of turkey, pumpkin pie and egg nog and had my fill of the ubiquitous work related social gatherings that I'm obliged to attend that I begin to sort through what I've been doing for this last year and start to prioritize what I'd like to get done this coming year. I suspect this is pretty much the natural order of things. Some people call them New Years resolutions, but I find that giving a goal that title condemns it to the fate of exercise and weight loss programs, which inevitably fail. So I keep it less dramatic and simply give some time to considering my options, reflecting on my past efforts and coming up with a strategy for the coming year.

With regard to Scheme I'm actually quite pleased with how things have come along this last year. To understand the progress that I've made you need to know a little about my background. I have a BS and MS in computer science and an MD specializing in Radiology. I'm retired now, and at the beginning of 2006 had only a minimal exposure to Lisp as a graduate student. My passion for Scheme came directly out of a massive read-a-thon of books, I had, primarily on XML technologies. Along the way I figured I needed to learn some of the more popular languages of the day, since I had programmed almost exclusively in C an C++ both as a student and as an independent computer consultant, which is how I supported myself before entering medicine. So I read books on Pearl and PHP, and then, as luck would have it, I choose to read about Scheme before Python. Well I never did get to Python. A small, unassuming book titled, The Scheme Programming Language (TSPL), 3rd ed., by Kent Dybvig really captured my attention. I almost put it down after the first chapter because it seemed to be basically Lisp the way I remembered it from school, with car and cdr and the lot. But as I continued with it I realized I was up against a worthy adversary, with the introduction of syntactic extensions to the language and this funny concept of call-with-current-continuation. Now I don't suggest that TSPL is the right choice for ever student of Scheme to start with, as it is prone to introducing concepts in examples before they've been discussed in the text. But for my taste it was just what the doctor ordered. The exercises often appear trivial until attempted, and then you learn that Dybvig expects that previous exercises will have been completed as he progress through the text. It should also be pointed out that he places the answers to many of the exercises in the back of the book. However, as I learned, simply having the answer was typically not enough to continue, I really had to understand how the answer was derived to understand discussions later in the text; but I suspect this is true with any introductory text you would settle on to learn Scheme.

Being used to mature IDE's in the C++ world, I was disappointed at my early attempts to find a reasonable Scheme system to program in; they all seemed to be about twenty years behind the times with regard to debugging tools and development environments. That's until I found PLT's DrScheme, which really can't be beat for anyone attempting to learn the language. Coming in second on my list was Dybvig's own Petite-Chez Scheme, which sports an excellent debugger (in many ways better than DrScheme's until recently), but lacked the integration with the source, so that one was relegated to checking line numbers in error messages against line numbers in your editor to try and figure out where things had gone wrong. Today, I am aware of several different approaches, all utilizing Emacs (a long-toothed unix editor with a Lisp scripting language built in) that allow the kind of integration I was looking for, and had I been aware of them at the time things might have turned out differently, but from where I sat at the time DrScheme had no competition. Till this day I occasionally surf around looking at other implementations of Scheme to see if I would now be better off with a different flavor, but DrScheme is very much a living implementation, with updates coming out nightly, if you are so inclined. So I can still say to anyone that wants to build applications in Scheme that DrScheme is your best bet for a development environment, and if you ever need the speed of a compiled version the PLT group always has mzscheme, which is the command line implementation of Scheme upon which DrScheme is built, and which produces very fast executable compiled binaries. And if you really must have the ultimate speed provided by xyz's super Scheme compiler, then you can always develop it in DrScheme using the strict R5RS language, restricting yourself to purely portable libraries, and then compile them on what ever implementation you like.

Now that you've learned a little bit about how I came to Scheme, and heard my song and dance about the virtues of DrScheme it's time to return to my priorities for the upcoming year. The first project I undertook after doing my homework in Scheme, because of my intense interest in XML technology, was the development of a native XML database written entirely in Scheme that sported both an SQL and XQuery front-end. That project has become a very large undertaking, turning into several sub-system projects along the way, one of which I've posted about previously, that being Active Comments. The projects name is xmlServer, and my thoughts on its future are that I'm likely to take several of the subsystems, polish them off as systems in their own right, and place them on PlaneT. They include a structured symbol table, a multithreaded messaging system and a Red-Black Tree data structure that have all been written in DrScheme and all very useful subsystems. As for xmlServer as a whole, I have an XQuery syntax subsystem, and a multi-threaded database process subsystem to complete before it will show signs of life. It is already quite capable of storing and retrieving data in native XML format using a pseudo-SQL syntax from a single database. Truth be told however, it was a learning project, and the world has quite a sufficient number of database options already, so this project as a whole will remain on the back burner, as I have more interesting projects to pursue.

Next on the list is building a Scheme implementation from the ground up. I've seen few different models of methodology, everything from break out the c compiler and start writing to compiling a certain core of functionality from an existing implementation and building the rest as syntactic extensions to the language. My purpose, I suppose, needs some explanation given my song of praise I gave to DrScheme earlier. I'm not out to create a Scheme implementation that will necessarily attract anyone's attention as far as using it for developing applications. What I'm after is the deep understanding of Scheme that I'm quite certain will follow from having to make the engineering trade-off decisions between performance, standards compliance and features. I fully expect that this will be a multi-person year project, so this initial phase of deciding on methodology could take a few months and I wouldn't be surprised. This project is definitely off the back burner but is nowhere near the flame yet.

Then we get back to Active Comments, who's epilogue I wrote about a couple of posts back. The fact is this is a high priority item for me to complete, even if not in the fashion I had originally envisioned. The project entails writing a tool, as they are called in DrScheme speak, that becomes part of the DrScheme IDE. This particular tool would allow for a user specified syntax, with Active Comments being the default syntax, to govern their comments. The user would be free to completely ignore the tool by either not installing it or simply not employing it to check the syntax of their comments. They would also be free to use their own syntax if they had something else in mind regarding the manner in which they wanted to document source files. For those who decided to buy into the entire idea, the tool would come complete with a functioning syntax that can trivially be transformed into an XML document where the comments are the XML entities and attributes and the source code is simply preformatted text, in the sense of an XHTML document. This project is very much a front burner project, and has been delayed somewhat as the new unit system on which tools are built in DrScheme was just now overhauled, so it didn't make sense to get started on a system using an underlying library I knew was about to fundamentally change. But now that the change is in place I will begin work on this project with ernest.

Finally, another top priority of mine is my ongoing dialogue on Scheme From the Florida Keys, and especially my tour of the Delimited Continuation Operators that I've been writing about. If your ever out at my web site, I keep a bibliography there of journal articles related to the topic. I think once you read a few of these articles you'll realize how significant the operators are to DrScheme. They also supply a wealth of very interesting topics for blog posts. It is quite impressive the number of ways people have come up with to utilize these new tools. I'm going to be selecting just those examples that most clearly demonstrate the advantage of using delimited over non-delimited continuations. From a purely Turing machine perspective, you can get the job done with either, but there are times when its not so clear how you would get it done without delimited continuations.

Putting the tour to the side, the blog will be my outlet for my public progress notes as I take on all of the projects I've talked about, and I'm sure others I haven't thought of yet. Programming is, by its nature, an intensely introspective process. The blog serves the purpose of exposing those introspective thought processes to other peers, so it will remain a front burner item for me for the foreseeable future.

--kyle