I have recently started playing with Clojure. Like ANTLR it also has the trinity of technology, documentation, and tools. On the technology side Clojure is as Lisp dialect that runs on top of the Java Virtual Machine. For me, it has three outstanding features. These are superb Java integration, its concurrency features especially software transaction memory, and the lazy sequence library.

Clojure has very good Java integration. Calling Java code is easy and direct. Classes in the java.lang package are imported automatically. Classes in other packages can be imported using the (import ...) form. Java objects can be created using the (new classname...) form or the (classname. ...) macro. Calling methods is just as easy.

Clojure includes both an interpreter and a compiler. The compiler compiles Clojure code to Java byte code in standard Java class files, so Clojure code can be called from Java code easily. In addition Clojure provides a (proxy ...) macro, which creates a proxy class, of a particular type, and sets the implementation for some or all of the methods of the interface. This is useful when calling Java code that need callback classes, such as, for example a SAX ContentHandler. For Java code that need Runnable or Callable objects, the (proxy ...) macro is not even needed, as all Clojure functions automatically implement both these interfaces.

Clojure also includes functions to generate primitive Java types and arrays and to work with Java primitive arrays.

The second outstanding feature is concurrency support. In addition to all the Java concurrency features, Clojure has two extra features to enable ease development of programs which need concurrency. These features are Agents and Software Transaction Memory. By default data is immutable in Clojure: once a data binding is set, it normally cannot be changed. Refs are an exception to this. They can be changed but only in the context of a transaction in the Software Transactional Memory (STM). STM is the in memory reference analogue of database table transactions. Agents are another exception. Updates to agent values are done by sending asynchronous messages using the (send ...) or (send-off ...) forms. The update message consists of an update function that is applied to the agent's data. The updates are are queued and handled atomically in a separate thread.

The last killer feature for me is the lazy sequence library. Clojure includes a library that allows for lazily evaluated sequences to be defined. This allows for infinite sequences to be defined: the elements in the sequence are only actually generated as they are needed, so memory is not wasted. If it turns out that only a limited number of elemnts are actually used, this can save quite a bit on processing time, too. All the main sequence processing functions, such as (map ...), etc., return lazy sequences, so lazy sequences end up being quite powerful.

Documentation is available at the REPL via the (doc ..) and (find-doc ...) forms. There is a fantastic book, published by the Pragramatic Programmers, called Programming Clojure. In this book the author, Stuart Halloway, gives a solid introduction to all of the features of Clojure. There are chapters devoted to macros, Java integration, Clojure's concurrency features, Clojure's sequence library, and the multimethod feature. All of these are accompanied by informative examples and each chapter ends with the application of the leasons to an extended example: a build DSL called Lancet. The book also contains a chapter on funtional programming, which deals with all the features that help with programming in a functional style, such as lazy sequences, partial function evaluation using the (partial ...) form, and the forms to enable efficient recursion. The book ends with a chapter called "Clojure in the Wild", which deals with real world issues with using Clojure, such as connecting to database, running unit tests, and building web applications with Clojure.

The final member of the trinity is tools support. A Clojure mode has been developed for Emacs, and Clojure support has been added to SWANK and SLIME. The three of these provide a very nice development environment for Clojure, with syntax highlighting, indenting, and an embedded and integrated REPL, for trying stuff out. All the SLIME key-bindings seem to do what is expected, though, I haven't used a lot of them yet as I am still getting used to the environment. However, documentation look up and the different evaluation options seem to work a treat. There are even a few humorous touches: the title of this post is one of the messages SLIME gives you when you change to a buffer to SLIME mode with M-x slime. All in all, the whole experience has taken me one step further along the path to The Dark Side regular Emacs usage.

The whole thing was very easy to set up: I just followed the instructions at this web page, and was up and running in about 5 minutes. A post on Bill Clementson's blog was also useful, though, it would probably be more useful for an experienced Lisper.

Technorati Tags: programming clojure

The problem with setting up the two operating systems to dual boot is that although they both use GRUB as their bootloader, their versions of GRUB have both been modified to handle special circumstances for the OS in question. The GRUB versions are like feuding cousins: they have the same name, the same origin, and they look pretty similar, but they are different enough that the just don't get along.

It turns out that the version of GRUB for each operating system doesn't understand the default filesystem that the other installs on. Linux doesn't support ZFS, so, plainly, its GRUB wouldn't either. The problem with OpenSolaris' GRUB was a little more obscure as, in theory it should have supported ext3. I found a blog posting, by Peter Buckingham, entitled GRUB, GRUB, why do you haunt me so?, which explained the problem. In Peter's words:

At this point I did some digging Google is your friend. So it turns out that the problem is a little while back the size of the inode for ext3 was increased to 256 bytes to make it more forward compatible with ext4. My old Ubuntu and OpenSolaris setup never hit this because the filesystems were set up with the older sized inodes.

Peter fixed his problem by patching the OpenSolaris GRUB, so eventually Peter's fix will be committed into OpenSolaris and the issue will go away. However that was a little too involved for me. My solution to the impasse was to create an ext2 boot partition for Ubuntu and have the OpenSolaris version of GRUB chainload that. The steps I used to do that are as follows:

1. Partition the disk into 4 partions. For me the partition fdisk -lu gives back the following:

   
   Device Boot      Start         End      Blocks   Id  System
   /dev/sda1              63      208844      104391   83  Linux
   /dev/sda2          208845   461001239   230396197+  83  Linux
   /dev/sda3   *   461001240   923512589   231255675   bf  Solaris
   /dev/sda4       923512590   976768064    26627737+  82  Linux swap
               
   

   The first is for an ext2 partion to hold Linux's /boot. The second partition is for the rest of Ubuntu Linux, on ext3, the third for OpenSolaris, and the last one is for Linux swap.

2. Install Ubuntu Linux 8.10, with the first partition mounted as /boot and the second partition mounted as /.

3. Install GRUB into the start of the /boot partition. This can be done from the GRUB command line. For my setup, I typed the following at the GRUB command line:

   root (hd0,0)
   setup (hdo,0)
   

   Alternatively, you can get the Ubuntu installer to do this for you in the Advanced Setup dialog box, accessed from the partitions page in the setup wizard.

4. Install OpenSolaris.

5. Add an entry to the OpenSolaris menu.lst file, located in /rpool/boot/grub, to chainload the Ubuntu GRUB. For me the entry looked like this:

   title Ubuntu Linux 8.10
   root (hd0,0)
   chainloader (hd0,0)+1
   

That should leave you with a system that dual boots OpenSolaris and Ubuntu Linux 8.10.

The following links were useful to me when I was trying to figure out GRUB:

• A GRUB reference site
• Making a separate boot partition
• GRUB bootloader - Full tutorial
• How To Multi-Boot Operating Systems

Technorati Tags: Ubuntu OpenSolaris GRUB multiboot

Vinum Regum, Rex Vinorum

Louis XIV

A few years ago I had the good fortune to spend a 10 days in Budapest. While there I developed a taste for Hungarian wine, especially Egri Bikáver, and Tokaji, the wine that Louis XIV loved so much. These wines used to be very difficult to come by in Ireland. Fortunately, this is no longer the case, as a wine shop specializing in Hungarian wines has opened in Dublin. The shop is called Essence of Wine and is located in the Moore Street Mall. The staff there are passionate about wine, very knowledgeable and helpful. I was there a few weekends ago I spent about half an hour chatting about wine and Budapest with Laszlo, one of the guys that runs the place. Over the course of the conversation he recommended 2 wines from the Gal Tibor winery, which I bought, and which were shared with friends last night. They were superb. So if you happen to be shopping in Dublin for a good bottle of wine, I'd recommend there.

Technorati Tags: wine Hungary

• HelloSablecc
• HelloAntlr
• FlexibleAntlrGeneration
• HelloCup
• HelloRacc

Technorati Tags: programming antlr sablecc parsers racc cup

ANTLR is a parser generator tool that is capable of targeting multiple output languages. Out of the box, it will generate Java, Python, C, C#, or Ruby code. Other target languages are possible if the code generators are written. Amongst its cool features are:

• LL(*) parsing: This is an extension to the normal, top down with lookahead, LL(k) parsing technique. It allows for more powerful parsers than those possible with LL(k). Not something I have needed yet, but I can see it being useful in the future.

• Semantic and syntactic predicates: These are tests that can be embedded in grammar rules to turn certain choices in rules on and off based on a boolean test. These allow parsers to be built that simulate recognizers for context dependent grammars and this makes the automated parser generation applicable to a lot more problems. Again, this is something I haven't used yet, but will be in the future.

• Memoized backtracking: This is a performance improving feature for grammars that use a lot of lookahead. If a parser fails at a choice near the end of the rule it has to go back to the start of the rule and start matching again. The memoization of intermediate matches for a rule speeds this up.

• Unicode support: parsers built with ANTLR recognize Unicode input. ANTLR grammar files themselves do not recognize Unicode characters yet, but Unicode characters can be specified as escape sequences.

• Hierarchical lexers: Most parser generator tools define tokens by means of a regular expression like language. For these tools lexer rules are independent of each other and cannot refer to each other. ANTLR is different: it allows lexer rules to reference other lexer sub-rules. It also allows recursive lexer rules. Now, this is a very useful feature and, for me, tidies up lexer rules a lot.

• Abstract Syntax Tree (AST) generation features: ANTLR has a powerful AST generation feature. When generating parsers I prefer to generate a parse tree or AST and pass that onto a second stage, rather than embedding actions/code in the grammar. This allows the a parser to be used as a module in numerous tools. ANTLR's AST building support is superb and really facilitates that type of design.

• Tree grammars: This is a feature of ANTLR that allows for a parser to match tree structures such as ASTs and trigger an action based on this. I used to use JavaCC and what I normally did was use it to generate a parse tree and then use the visitor pattern to process that parse tree. If the action at a particular node depended on the structure of the tree, it was up to me to track that in the Java code in the visitor class. ANTLR's tree grammars really simplify situations like that and I am really looking forward to trying that out.

The Definitive ANTLR Guide, available from the Pragmatic Programmer website, is the main documentation for ANTLR v3.0. The book is well named. It clearly and concisely describes the features of ANTLR v3.0 and how to use them. The book is divided into 3 sections: an introduction to ANTLR and language translation, a reference section for the ANTLR syntax, and a section on how to write predicated LL(*) grammars. I have read through the first section and skimmed parts of the reference and this has enabled me to put together a basic parser/recognizer for Scheme. It really is that straight forward.

In addition to The Definitive ANTLR Guide there is a really good Wiki with tutorials and FAQs at http://www.antlr.org/wiki/display/ANTLR3/ANTLR+3+Wiki+Home. Lastly, the distribution comes with great sample grammars. There is a Java grammar and a Python grammar in there and these put ANTLR through its paces and are worth referring to.

ANTLRWorks is the IDE for building ANTLR grammars. It is a standalone editor, written in Java Swing, that provides the standard features that you would expect from an IDE, such as syntax highlighting and error detection. In addition it has a few handy features that you would not expect. Firstly, there is a syntax diagram pane in the GUI. Selecting a grammar rule in editor pane causes a syntax diagram for the rule to be displayed in the syntax diagram pane. The graphic below is an example.

This is very useful for documenting grammars as the diagrams can be saved as graphics (JPEG, PNG, EPS, etc.). There is also a debugger and an interpreter. The interpreter will interpret the grammar and draw a graphical representation of the parse tree for an input string based on the grammar. If the parse fails it still draws the diagram for what was recognized and puts a node in the tree, at the point were the parse failed, to represent the type of error detected. The parse tree graphics can also be saved and are a real help with visualizing how the rules fit together and what choices the parser is taking.

Technorati Tags: programming antlr

1. Be brave and aggressive.

   • Be direct.
   • Grab all opportunities.
   • Using varying methods of attack.
   • Be versatile and agile.
   • Attack one target at a time.
   • Don't plan everything in detail.
   • Use top quality weapons.

2. Be prepared.

   • Keep weapons in good condition.
   • Keep in shape.
   • Find good battle comrades.
   • Agree on important points.
   • Choose one chief.

3. Be a good merchant.

   • Find out what the market needs.
   • Don't make promises that you can't keep.
   • Don't demand overpayment.
   • Arrange things so that you can return.

4. Keep the camp in order.

   • Keep things tidy and organized.
   • Arrange enjoyable activities which strengthen the group.
   • Make sure everyone does useful work.
   • Consult all members of the group for advice.

These laws are pretty relevant to software development and the software business in general. In my experience, problems during software development projects stem from one of these guidelines not being followed. It is particularly dangerous to ignore Choose one chief, Find good battle comrades, or Don't make promises that you can't keep. Yet again, advice on software development has come from an apparently unrelated source.

A new draft of the R6RS for Scheme: Revision 5.92 Recommended Specification for Scheme is up on site, www.r6rs.org. It has been split into two PDFs, one for the core language and one for the libraries. The total page count for the two files is about 150 pages.

The positives are:

• The language is terse. There is very little boiler plate code needed to get up and running in Fortress. Hello world is:

  export Executable
  run(args:String...) = print "Hello, World!\n"

• Fortress is designed to have a syntax that resembles mathematical notation. For example, the absolute value function appears like this in Fortress: |someVar|. This makes the code really brief and also partly familiar to mathematically inclined readers. Fortress is a good way along to achieving Fortran's original goal of formula translation.

• Operators are functions and can be defined for classes, overloaded, etc. In addition to normal operators, it is possible to define bracket operators, e.g., the absolute value operator shown above.

• Multi-paradigm programming is supported, particularly for OO and functional styles.

• There is strong support for parallel/multi-processor computing. I have not played with this, but the demos include a parallel matrix multiplication example that demonstrates this.

• Fortress has support for comprehensions, similar to those in Python.

The negatives are:

• The language is not fully implemented yet, so some features are not working, e.g., atomic modification of multiple variables is not fully implemented yet.

• Libraries are extremely sparse at the moment. There isn't even a maths library yet.

Other interesting things:

• Fortress targets the JVM initially. This means that there is a possibility of calling Fortress programs from Java programs via the interfaces defined by JSR 223.

• Fortress makes extensive use of Unicode and this use isn't superficial: many of the mathematical operators and functions can be specified by the Unicode characters used to display them in mathematical papers. For example, the floor function on a variable x can be written as ⎣x⎦ . In fact the Fortress compiler and interpreter deal exclusively with Unicode. While Fortress programs can be written completely in ASCII there is a conversion stage that translates ASCII longhand into equivalent Unicode characters before the lexer and parser get at the code.

The staff there are pretty cool and know their stuff, so I am learning lots. A few of them have blogs, notably (in alphabetical order), Bill de hÓra, Vicky Lee, and Sean McGrath. Anyhow, so far so good. I need to check with people here what is considered proprietary information, but hopefully I will be able to blog about some of the things I have learned over the past few week.