July 05, 2009

"Take this REPL, brother, and may it serve you well."

Clojure_300x300.png

I have recently started playing with Clojure. Like ANTLR it also has the trinity of technology, documentation, and tools. On the technology side Clojure is as Lisp dialect that runs on top of the Java Virtual Machine. For me, it has three outstanding features. These are superb Java integration, its concurrency features especially software transaction memory, and the lazy sequence library.

Clojure has very good Java integration. Calling Java code is easy and direct. Classes in the java.lang package are imported automatically. Classes in other packages can be imported using the (import ...) form. Java objects can be created using the (new classname...) form or the (classname. ...) macro. Calling methods is just as easy.

Clojure includes both an interpreter and a compiler. The compiler compiles Clojure code to Java byte code in standard Java class files, so Clojure code can be called from Java code easily. In addition Clojure provides a (proxy ...) macro, which creates a proxy class, of a particular type, and sets the implementation for some or all of the methods of the interface. This is useful when calling Java code that need callback classes, such as, for example a SAX ContentHandler. For Java code that need Runnable or Callable objects, the (proxy ...) macro is not even needed, as all Clojure functions automatically implement both these interfaces.

Clojure also includes functions to generate primitive Java types and arrays and to work with Java primitive arrays.

The second outstanding feature is concurrency support. In addition to all the Java concurrency features, Clojure has two extra features to enable ease development of programs which need concurrency. These features are Agents and Software Transaction Memory. By default data is immutable in Clojure: once a data binding is set, it normally cannot be changed. Refs are an exception to this. They can be changed but only in the context of a transaction in the Software Transactional Memory (STM). STM is the in memory reference analogue of database table transactions. Agents are another exception. Updates to agent values are done by sending asynchronous messages using the (send ...) or (send-off ...) forms. The update message consists of an update function that is applied to the agent's data. The updates are are queued and handled atomically in a separate thread.

The last killer feature for me is the lazy sequence library. Clojure includes a library that allows for lazily evaluated sequences to be defined. This allows for infinite sequences to be defined: the elements in the sequence are only actually generated as they are needed, so memory is not wasted. If it turns out that only a limited number of elemnts are actually used, this can save quite a bit on processing time, too. All the main sequence processing functions, such as (map ...), etc., return lazy sequences, so lazy sequences end up being quite powerful.

Documentation is available at the REPL via the (doc ..) and (find-doc ...) forms. There is a fantastic book, published by the Pragramatic Programmers, called Programming Clojure. In this book the author, Stuart Halloway, gives a solid introduction to all of the features of Clojure. There are chapters devoted to macros, Java integration, Clojure's concurrency features, Clojure's sequence library, and the multimethod feature. All of these are accompanied by informative examples and each chapter ends with the application of the leasons to an extended example: a build DSL called Lancet. The book also contains a chapter on funtional programming, which deals with all the features that help with programming in a functional style, such as lazy sequences, partial function evaluation using the (partial ...) form, and the forms to enable efficient recursion. The book ends with a chapter called "Clojure in the Wild", which deals with real world issues with using Clojure, such as connecting to database, running unit tests, and building web applications with Clojure.

The final member of the trinity is tools support. A Clojure mode has been developed for Emacs, and Clojure support has been added to SWANK and SLIME. The three of these provide a very nice development environment for Clojure, with syntax highlighting, indenting, and an embedded and integrated REPL, for trying stuff out. All the SLIME key-bindings seem to do what is expected, though, I haven't used a lot of them yet as I am still getting used to the environment. However, documentation look up and the different evaluation options seem to work a treat. There are even a few humorous touches: the title of this post is one of the messages SLIME gives you when you change to a buffer to SLIME mode with M-x slime. All in all, the whole experience has taken me one step further along the path to The Dark Side regular Emacs usage.

The whole thing was very easy to set up: I just followed the instructions at this web page, and was up and running in about 5 minutes. A post on Bill Clementson's blog was also useful, though, it would probably be more useful for an experienced Lisper.

Technorati Tags:

Posted by JohnC at 11:18 PM

January 18, 2009

Dual Booting OpenSolaris and Ubuntu 8.10

Recently, I bought a new laptop. There are a number of tools and technologies that I am interested in that are included with either Ubuntu Linux or OpenSolaris. I decided that I would set up my new laptop to dual boot OpenSolaris and Ubuntu Linux 8.10. Both operating systems use the GRUB bootloader so in theory it should be easy to get them to work together. In practice it was not so easy. I am writing this post just in case other people have the same issues I had.

The problem with setting up the two operating systems to dual boot is that although they both use GRUB as their bootloader, their versions of GRUB have both been modified to handle special circumstances for the OS in question. The GRUB versions are like feuding cousins: they have the same name, the same origin, and they look pretty similar, but they are different enough that the just don't get along.

It turns out that the version of GRUB for each operating system doesn't understand the default filesystem that the other installs on. Linux doesn't support ZFS, so, plainly, its GRUB wouldn't either. The problem with OpenSolaris' GRUB was a little more obscure as, in theory it should have supported ext3. I found a blog posting, by Peter Buckingham, entitled GRUB, GRUB, why do you haunt me so?, which explained the problem. In Peter's words:

At this point I did some digging Google is your friend. So it turns out that the problem is a little while back the size of the inode for ext3 was increased to 256 bytes to make it more forward compatible with ext4. My old Ubuntu and OpenSolaris setup never hit this because the filesystems were set up with the older sized inodes.

Peter fixed his problem by patching the OpenSolaris GRUB, so eventually Peter's fix will be committed into OpenSolaris and the issue will go away. However that was a little too involved for me. My solution to the impasse was to create an ext2 boot partition for Ubuntu and have the OpenSolaris version of GRUB chainload that. The steps I used to do that are as follows:

  1. Partition the disk into 4 partions. For me the partition fdisk -lu gives back the following:

    
Device Boot      Start         End      Blocks   Id  System
/dev/sda1              63      208844      104391   83  Linux
/dev/sda2          208845   461001239   230396197+  83  Linux
/dev/sda3   *   461001240   923512589   231255675   bf  Solaris
/dev/sda4       923512590   976768064    26627737+  82  Linux swap

    The first is for an ext2 partion to hold Linux's /boot. The second partition is for the rest of Ubuntu Linux, on ext3, the third for OpenSolaris, and the last one is for Linux swap.

  2. Install Ubuntu Linux 8.10, with the first partition mounted as /boot and the second partition mounted as /.

  3. Install GRUB into the start of the /boot partition. This can be done from the GRUB command line. For my setup, I typed the following at the GRUB command line:

    root (hd0,0)
setup (hdo,0)

    Alternatively, you can get the Ubuntu installer to do this for you in the Advanced Setup dialog box, accessed from the partitions page in the setup wizard.

  4. Install OpenSolaris.

  5. Add an entry to the OpenSolaris menu.lst file, located in /rpool/boot/grub, to chainload the Ubuntu GRUB. For me the entry looked like this:

    title Ubuntu Linux 8.10
root (hd0,0)
chainloader (hd0,0)+1

That should leave you with a system that dual boots OpenSolaris and Ubuntu Linux 8.10.

The following links were useful to me when I was trying to figure out GRUB:

Technorati Tags:

Posted by JohnC at 09:38 PM

July 08, 2007

Martin Fowler test-drives Parser Generators

I have been catching up on my blog reading this weekend. One of the blogs I have caught up on is Martin Fowler's. Recently, i.e., since February, he has being trying out parser generator tools and he has written a series of posts on his experiences.

Technorati Tags:

Posted by JohnC at 08:14 PM

June 12, 2007

The ANTLR Trinity

There are 3 components to a really useful software development technology: innovative features, clear and comprehensive documentation, and solid tools. The recent release of ANTLR v3.0 is a perfect example of this. This parser generator tool has all 3 components and each component is done superbly.

ANTLR is a parser generator tool that is capable of targeting multiple output languages. Out of the box, it will generate Java, Python, C, C#, or Ruby code. Other target languages are possible if the code generators are written. Amongst its cool features are:

The Definitive ANTLR Guide, available from the Pragmatic Programmer website, is the main documentation for ANTLR v3.0. The book is well named. It clearly and concisely describes the features of ANTLR v3.0 and how to use them. The book is divided into 3 sections: an introduction to ANTLR and language translation, a reference section for the ANTLR syntax, and a section on how to write predicated LL(*) grammars. I have read through the first section and skimmed parts of the reference and this has enabled me to put together a basic parser/recognizer for Scheme. It really is that straight forward.

In addition to The Definitive ANTLR Guide there is a really good Wiki with tutorials and FAQs at http://www.antlr.org/wiki/display/ANTLR3/ANTLR+3+Wiki+Home. Lastly, the distribution comes with great sample grammars. There is a Java grammar and a Python grammar in there and these put ANTLR through its paces and are worth referring to.

ANTLRWorks is the IDE for building ANTLR grammars. It is a standalone editor, written in Java Swing, that provides the standard features that you would expect from an IDE, such as syntax highlighting and error detection. In addition it has a few handy features that you would not expect. Firstly, there is a syntax diagram pane in the GUI. Selecting a grammar rule in editor pane causes a syntax diagram for the rule to be displayed in the syntax diagram pane. The graphic below is an example.

Syntax Diagram

This is very useful for documenting grammars as the diagrams can be saved as graphics (JPEG, PNG, EPS, etc.). There is also a debugger and an interpreter. The interpreter will interpret the grammar and draw a graphical representation of the parse tree for an input string based on the grammar. If the parse fails it still draws the diagram for what was recognized and puts a node in the tree, at the point were the parse failed, to represent the type of error detected. The parse tree graphics can also be saved and are a real help with visualizing how the rules fit together and what choices the parser is taking.

Technorati Tags:

Posted by JohnC at 10:15 PM

May 02, 2007

Peter Seibel has started blogging...

Peter Seibel, author of Practical Common Lisp, has started blogging [atom feed]. The first post was on April 10th, but already there a few thought provoking posts up there. In particular, Software estimation considered harmful?, really made me think. In summary, the article states that software development estimates are a mechanism for communicating with others about a project to enable them to co-ordinate with the development team. Using estimates for this form of communication is not optimal in many cases and the article explains why. It is well worth reading.

Posted by JohnC at 10:04 PM

April 23, 2007

James Clark is blogging

XML guru, James Clark, has started a blog.

Posted by JohnC at 10:33 AM

The Viking Laws

Last year, I had a week long vacation in Iceland. While there, I picked up a t-shirt with the viking laws printed on the front. These laws were guidelines for conducting a viking expedition. The laws themselves are:

  1. Be brave and aggressive.

  2. Be prepared.

  3. Be a good merchant.

  4. Keep the camp in order.

These laws are pretty relevant to software development and the software business in general. In my experience, problems during software development projects stem from one of these guidelines not being followed. It is particularly dangerous to ignore Choose one chief, Find good battle comrades, or Don't make promises that you can't keep. Yet again, advice on software development has come from an apparently unrelated source.

Posted by JohnC at 10:33 AM

January 24, 2007

Quick bits of programming language news...

Dtrace for Javascript: There are details of how to set up Dtrace to trace Javascript code at Brendan Gregg's blog on blogs.sun.com. The bare bones is that he is providing a modified version of the Spider Monkey Javascript engine with Dtrace probes coded in. Installing the shared library into the appropriate location on a machine running Solaris will give Dtrace visibility into Javascript code running in Firefox. AJAX bugs have nowhere to hide now.

A new draft of the R6RS for Scheme: Revision 5.92 Recommended Specification for Scheme is up on site, www.r6rs.org. It has been split into two PDFs, one for the core language and one for the libraries. The total page count for the two files is about 150 pages.

Posted by JohnC at 11:14 PM | Comments (0)

Fortress

The researchers at SUN Labs have recently released a beta version of their new programming language, Fortress. It is available here under a BSD license. It is designed to tackle problems in the high performance computing arena and looks rather interesting.

The positives are:

The negatives are:

Other interesting things:

Posted by JohnC at 12:54 AM | Comments (0)

January 14, 2007

Standard Deviation

If you find yourself needing to migrate an application from one of the major RDBMS packages to another, as I have recently, then this webpage is invaluable. It compares how MySQL, MS SQL, DB2, Oracle, and PostgreSQL compare to the SQL92 standard. The page links to relevant documentation for each RDBMS, for each feature compared, which is really handy.

Posted by JohnC at 09:21 PM | Comments (0)

November 14, 2006

Object Oriented Design Principles

I came across a good set of design principles for object oriented software. The details are in PDF files linked off the page and each of these are well worth the read. I found the package cohesion paper, opened by the second bunch of links, especially informative.

Posted by JohnC at 01:31 PM | Comments (0)

November 12, 2006

Python Ireland meeting

Python logo

I have been meaning to attend a Python Ireland meeting for quite a while. Last Wednesday I finally got around to it and I was treated to two excellent talks. The first was by Sean McGrath on Jython. This gave an introduction to Jython, with some examples of where it could be useful in a Java environment. The SWT GUI example was particularly good in showing how Jython could compact GUI code for Java applications. Sean also filled in a lot of background around Python on the JVM and Microsoft's CLR and this lead to a lively discussion.

Mick Twomey gave the second talk, briefly, as the Jython discussions took up most of the session, and described the Web Server Gateway Interface (WSGI). This is a rather clever API, developed to modularize Python web frameworks. It essentially applies the decorator pattern to code running in the web framework. For example, if you have a web application that you want to have protected with a password, you could use an authentication module with the WSGI to add that to your application. In a sense, the authentication module is applied as a decorator for the application code. Mick also gave an example of gzip compression. WSGI sounds pretty useful.

After the talks the attendees took a short walk to The Pembroke for a beer and more conversation. This could become a regular occurrence for me.

Posted by JohnC at 11:24 PM | Comments (0)

June 22, 2006

Two good articles on LISP

I came across two good articles on The Nature of LISP and Functional Programming on www.defmacro.org. Both articles are excellent. They are very clear and informative and explain LISP and functional programming concepts in terms familiar to Java or C programmers.

Posted by JohnC at 09:53 PM

June 10, 2006

Scheming

I have started learning Scheme for fun and (hopefully) profit. I have been learning from a number of sources. Firstly, I have after learning the basics, I wrote another program to calculate the digits of π. The code is here (pretty printed). The algorithm I used is the same as the one I used for the python program and the two programs look very similar. I haven't yet learned how to code up closures in Scheme, so the method of calculating the Fibonacci terms to use is different, but apart from that they are pretty much the same.

Secondly, I found a few good online resources:

I use Kawa and SISC as scheme interpreters. Both of these run on top of the Java VM and are easy to use and fit in well on UNIX/Linux systems. Running on top of the Java VM opens up some possibilities such as using the Java VM Dtrace probes to observe Scheme programs. I am not sure how well this will work, but it is something I will be investigating.

Why am I learning Scheme? Well, there are a few reasons. I really admire Paul Graham's essays and his essays on LISP and strong advocacy piqued my interest in the language. I also came across an essay describing design patterns supported by LISP. Many of the patterns I use regularly in Java programs, such as the Visitor pattern, Command pattern, and Iterator pattern, are not needed in LISP: they are built in at the language level. Because of this, I wanted to learn more and, although I have just started, I am sure this is going to make me a better programmer.

Posted by JohnC at 07:57 PM

June 08, 2006

Readable code

I had an, "Aha!", moment yesterday, regarding source code readability. I followed a link from Tim Bray's blog, to an analysis by Josh Bloch of a bug in the Java binary search code. In summary, the bug was caused by an overflow in an integer calculation. While reading the explanation of the problem and its solution I realized I had read something like it before and, after a few minutes thinking, I realized it was in the code of Metafont. I had read this in Donald Knuth's book, Literate Programming, a few months earlier. In particular, it was from section 108 of Metafont and the text in question was:

Notice that the computation specifies (p - q) + p instead of (p + p) - q because the latter could overflow. Let us hope that optimizing compilers do not miss this point; a special variable be_careful is used to emphasize the order of computation.

Now when I was reading this I was not studying the code: I was just idly reading it. However, the insight into variable overflow stayed with me. Perhaps it was because the program was written in the literate programming style and, therefore, was explicitly written to be as readable as possible, that I remembered. If so, reading literate programs is a subtly powerful technique for programmer training.

Posted by JohnC at 11:54 PM

February 13, 2006

Know the Ways of All Occupations

While reading an essay by Donald Knuth, in his book, Literate Programming, I came across this sentence:

I have felt for a long time that talent for programming consists largely of the ability to switch readily from the microscopic to the macroscopic view of things, i.e., to change levels of abstraction fluently.

This is spookily similar to the advice given by Miyamoto Musashi in his book, The Book of Five Rings:

In the Way of the Martial Arts, you must consider Head of a Rat, Neck of a Bull constantly and repeatedly. No matter how engrossed in the details you are you should suddenly show great heart and exchange the great and the small.

It has been a while since I had considered another piece of advice given by Musashi, to know the ways of all occupations. If programming and swordsmanship can share principles then it is advice I should follow more diligently.

Posted by JohnC at 10:10 PM

September 11, 2005

Hello world. Here is pi. ...or was that Py?

Programming languages are the fundamental tools in a programmer's toolbox. Without them programming computers is next to impossible, requiring the use of machine code. The work I am doing now is mostly in PHP, Perl, and occasionally, Java. I have written a lot of Java code in the past, and Java is my preferred programming language. However, I bump into and ex-colleague, Mick, every few months and, when I do, he badgers me about giving Python a try. This weekend I did.

After the almost mandatory hello world, I decided to write a program to calculate the digits of π. I did something similar to this earlier in the year, when I flirted with Lisp, and I found it useful. It teaches you a lot about how a programming language handles numbers and how easy it is to implement numerical algorithms in that language.

What did I learn? Well, I learned that I like Python. The code (pretty printed here) came to under 70 lines and manipulating numbers was not cumbersome. The numeric operators worked fine with the huge numbers involved, apart from the division operator, /, which returned its results as floats and lost a lot of precision as a result. Fortunately, the integer division operator, //, did what I wanted, so this wasn't a problem. Once I discovered that, writing the program was pretty painless.

The generator feature of Python is really cool. A generator is a function that allows a programmer to lazy instantiate items in a sequence. When called it returns an object that has the iterator interface and calling the next() method on that object gets the next item in the sequence. I used a generator to calculate every second number in the Fibonacci sequence. The same effect could be achieved by creating a class. However, generators are concise, and feel more elegant.

I also discovered that the Python Cookbook is really good. Mick recommended it, so I bought it. I learned lots about generators from Chapter 19 and I expect I will be referring a fair bit to the chapters on XML processing, accessing databases, and string processing, as these activities make up the bulk of my coding tasks at work. I am looking forward to reading Chapter 20, on metaclasses and decorators.

All in all, from this weekend's dabbling, I am impressed with Python. Java might have some competition for my interest. It is a good job the two languages work well together.

Technorati Tags:

Posted by JohnC at 01:34 AM | Comments (0)

September 02, 2005

A Book On Career Development

Scanning through the postings on blogs.sun.com, I came across a review, for My Job Went To India (And All I Got Was This Lousy Book), on Frederic Jean's weblog. Fred's review fired up my interest, so I took a look at the book's webpage, and after browsing the sample chapters, I bought the PDF and dead tree combo. I downloaded the PDF this evening. The physical book will be shipped to me at the end of September.

The book claims to be about what programmers should do to protect themselves from off-shoring. It is so much more than that: it is a very clear and concise book about career development for software professionals. Every page advice that would be useful even if your job wasn't in danger of being sent to India. In fact, programmers in Bangalore would get as much from the book as someone in the USA or Western Europe. There is advice on how to choose which technologies to learn. There is advice on how to develop programming skills. There is advice on how to market yourself. There is even advice on how to manage an off shore team (surprisingly). All the advice is insightful and delivered clearly and briefly. If you are a programmer then My Job Went To India (And All I Got Was This Lousy Book) deserves to be on your bookshelf (when the physical book is actually ready, that is).

Posted by JohnC at 11:15 PM | Comments (0)