CLClojure: An Experiment Port of Clojure to Common Lisp

综合技术 2019-01-13 阅读原文

CLClojure: An Experiment Port of Clojure to Common Lisp


Porting Clojure seems to be the thing to do these days. After the clojurescript
compiler came out, people had a really good blueprint for hosting clojure in other
languages. The fact that the clojurescript compiler is defined, largely, in terms
of protocols, makes it a bit closer to the “clojure in clojure” goal. As a result,
you should be able to bootstrap the language (as they did with JavaScript) with
a minimum of primitives. Most of the language is defined in Clojure, so voila.

We currently have Clojure ports targetting Scheme, C, C-via-Scheme, Python, etc.
I’ve found Common Lisp to be a particularly slick environment with some cool tools
and a fairly active community. I think it would be interesting to port the
ideas from Clojure to Common Lisp, ultimately implementing a compliant Clojure in
Common Lisp.


Bridge the gap between the cool stuff in clojure and Common Lisp.

Implement really useful bits of clojure in portable common lisp, and provide them as stand-alone libraries.

This includes lazy sequences, the generic sequence abstraction,
and the fundamental persistent hash-array-mapped-trie data structures in clojure:

  • persistent vectors
  • persistent maps
  • persistent sets.

Extend the generic sequence abstraction and other idioms to Common Lisp’s built-in mutable structures.

Common Lisp already has a sequence library, but I think Clojure’s is more general and can be trivially extended to new types.
Common Lisp’s irrational locking down of SEQUENCE is a hurdle here. The HYPERSPEC will never be updated in my lifetime 🙂
So generic functions are the current way to bridge this stuff.

  • Protocols are really nice, as are Clojure’s arbitrary dispatch multimethods.
  • Data literals are also highly useful. I think Clojure nailed the choice of literals, so providing reader macros for these guys would be very nice.

Possibly wrap a Common Lisp STM implementation, or cheat and use something like lparallel or just delegate to clojure.core.async (like clojurescript).

Bootstrap a fully functional Clojure onto Common Lisp.


So far, this project continues to be a great way to learn about both CL and Clojure, to include
implementation hurdles, support for PLT stuff, long-standing warts from decisions made ~25 years or more ago,
and work-arounds. Nothing is insurmountable so far, which is pretty cool.


Began work porting Clojure’s persistent data structures from Java about years ago, while

simultaneously learning Common Lisp (and by necessity Java :/ ).

  • Got a working persistent vector implementation, with compatible clojure literals about a year ago.
  • Started working on persistent hash maps around November 2012.
  • Built a temporary implementation of clojure-style protocols in Common Lisp ~ Dec 2012.
  • Pulled the bits into an actual ASDF system and pushed everything to the Github August 2013.
  • Implemented a baby clojure evaluator that __should__ bridge the lisp1/lisp2 gap between clojure and the Common Lisp host. Unsure if this is going to work out in the long term, but eh.
  • It’s real trivial at the moment.
  • Working on library code in my spare time, still need to port the persistent map.

Revisited 2017

  • implemented some rudimentary stuff
  • vector reader macros not fleshed out; work fine at the REPL,
    but failed to return a vector (instead returning a form to create the vector).
    Trivial but important oversight.
  • Still hijacking the global read-table. Pulled in named-readtables to help,
    but haven’t integrated.
  • Working on variadic functions, explored funcallable classes, refined protocols, deftype.
  • cleaned up the build process (more work to be done here)

Revisiting 2018

  • got reader macros matured (vector literals produced properly now),
  • got protocol definitions and implementations respecting vectors,
    albeit in a hacky way. We still allow lists for args….
  • working on deftype, then getting the bootstrap from CLJS (core protocols and
    functions) to compile.
  • Working on leveraging named-readtables for better delineation of clojure source,
    to include file-level (somewhat racket like).
  • Also intend to leverage named-readtable
    to get case-senstive reader (via :modern), and to enable CL interop trivially
    with a macro allows CL reader inside body (vice versa, for inline clojure if
    it makes sense).
  • refining the approach after reading a lot, looking at some other sources of
    inspiration (including early proto-clojure->java compiler from Hickey in CL)
  • Basic def, fn works. Protocols work. Metadata works mostly. Deftype
  • got let, albeit without destructuring. Still useful for bootstrapping!
  • Initial implementation of reify working, wrapped deftype in a version
    compatible with cl:deftype


A couple of big hurdles:

Lisp1 vs Lisp2.

  • rely on macros (like def) to unify symbol and function namespaces,
    leveraging CL’s environment.
  • The longer route would be writing a custom eval, compiler, etc.
    Doesn’t look necessary at the moment.

Lexical Scope vs. Top Level

So, Common Lisp as a lisp-2 has multiple namespaces,
foremost of which are function and symbol. We already have
the top-level (that is, null lexical environment) symbol
and function namespaces unified by using setf for symbol-function
to make it identical to symbol value….but…..

  • Lexical environments don’t work that way!

    • symbol-function and symbol-value only work on non-lexical symbols.
    • Initial hack was to unify the namespaces by traversing
      the vars in the let bindings and unifying prior to continued binding.

      • Had a false-positive success since the symbol modifications were
        actually pointing to non-lexical scoped symbols (stuff from prior
        REPL interactions).
  • The fix for this is to use a combination of let* and labels, which CAN
    unify the symbol-value and symbol-function namespaces for lexical vars..

    • Defined a macro, unified-let*, that does this for us:

      • We parse the bindings, detecting if any symbol points to a literal
      • We ensure keyaccess objects are compiled during macroexpansion,
        which creates the plumbing for keyword access if it didn’t
        already exist

        • and we create function definitions for the vars that point to
          keywords, where the function bindings invoke the funcallable
          keyword accessor directly.
      • We need to apply this as an implicit body for fn forms as well..

Persistent structures.

  • If we get defprotocol, deftype, etc. up and running,
    these are already defined in CLJS core.
  • For bootstrapping, using original port of Persistent Vector
    from JVM clojure, also creating a dumb wrapper on top.

    • Need to add meta field to struct, also atomic locks at
      some point (unless cljs provides this….)


  • Already implemented as generic functions.
  • Explore alternative implementations if generic functions aren’t
    speedy enough (doubtful, but haven’t profiled anything).
  • protocol definitions need to be namespace/package qualified.

    • Looks like they are already, at the package level.
  • Need better error messaging on missing protocols

    • TODO: get-protocol should signal.


  • shadows CL deftype.
  • current implementation follows defprotcol, appears to work for
    non-varidiac protocol impls.
  • Look at walking method impl bodies to eliminated unnecesary
    slot bindings (currently we generally bind all non-shadowed


  • Initial ideas for multiple dispatch following clojure’s implementation.


  • Symbols and collections (anything that supports IMeta) can have
    a persistent map of metadata in clojure.
  • Used to communicate a lot of information, including type hinting,
    source code, documentation, etc.
  • Current expectation is to leverage property lists for symbols,
    and unify with generic functions behind a protocol.


  • CL has packages. Initial hack would be to leverage packages
    as an anlogue to ns.
  • Longer term solution is implement own ns system via objects.

    • Rather leverage CL where possible.
  • Currently implementation of def exports by default.

    • Looking at introspection possibilites for more
      easily tagging meta, arglists. custom function
      objects (experimental) are looking good for this.
    • Should we maintain a parallel collection of

      • Based on def and derived forms, we ought to
        be able to hook in and register stuff.
      • Allows the reflection and introspection
        we care about / use in clojure.
  • Need to translate between require, refer, import (clojure)
    and import (cl).


CL macros use , and ,@ in place of ~ and ~@ in Clojure.

We’ll need to either cook the common lisp reader, or build a separate clojure
reader that will perform the appropriate replacements.

  • Looks like a simple tweak to the ` reader macro will suffice,
    we’ll just flip the symbols as appropriate.
  • TODO: quasitquoting in clojure (let []) inside of macros is
    not splicing, need to revisit quoted-children.

    • ex `[,’a] => [,’A] (incorrect)

@ is a literal for #’deref in clojure, is whitespace in clojure.

  • Similar, we’ll flip the read-table.

[] denote vectors

  • already have a reader macro in reader.lisp
  • There’s an incorrect read going on, [x 2] will
    currently read when it should throw since x
    is not defined.

    • TODO: revisite quoted-children for vectors and
      the reader fn bracket-reader.
    • If we’re not reading a quoted
      vec, we need to eval args to persistent-vector ctor.

{} denote maps

  • already have a reader macro in pmap.lisp

#{} denote sets

  • Easy enough to add a dispatch in reader.lisp

^ denotes metadata for the reader

  • Trivial to implement as a reader macro, BUT, we need to get
    metadata working generally.

c vs. #c for chars

  • Added initial support for clj->cl, need to define
    printable representation for chars as well.
  • Current holdup is defining a print-method for
    STANDARD-CHAR, which claims the class doesn’t exist.

    • Looking at print-escape
      and friends to see if
      we can hack this. We may need our own printer,
      outside of the provided REPL.

reading/printing booleans

  • Similar issues as characters. PAIP has a good
    chapter on this for similar issues with Scheme.

.- field access

  • wrap to calls for CLOS slot

  • detect / in symbol name, coerce to
    qualified internal symbol

(.someMethod object)

  • .hello is a valid function name in CL…
  • Reader macro for .?

    • need to incorporate . form ala: (. obj someMethod)


is used as a delimiter for package symbols in CL.
  • need to parse the symbol name and dispatch….
  • ::blah -> :BLAH for now…CL reader eliminates
    redundant colon.
  • Should be simple to modify the reader macro for :

(aget obj field) from cljs…

  • keep? This doesn’t work the same in clj jvm…

Keyword access

Keywords are applicable in Clojure. That means they show
up in the function position in forms. This won’t
jive in CL directly.

  • Possible option: reader macro for lists, detect
    presence of keyword in function call positition,
    if not quoted.
  • Replace keyword with a funcallable thing that
    has a print-method looking like a keyword?
  • Or try to hack eval (dubious).
  • custom read / eval / print….

Looks like we can just leverage he function namespace
to get around this….keywords are “just”

  • (defun :a (m) (gethash :a m))
  • (defun (setf :A) (new-value m)
    (setf (gethash :A m)
  • (defparameter ht (make-hash-table))
  • (setf (:A ht) 3)

So the workaround is:

  • Need a reader macro lists.
  • If we see a keyword in the function position,

    • we define a function for the keyword via defun.
    • define a setf expander that provides hashmap
      access (alternately, something that’s not mutating).

Looks like that may work inside the existing ecosystem….
Significant progress / experimentation in
clclojure.keyworfuncs. However, it’s looking like,
to get “full” access (even with some tricky pseudo-keyword
class, macros, and the above suff), we’re probably better
off hacking eval / compile.


This may be a bit tricky, although there are a limited number of clojure forms.

Seq library.

This shouldn’t be too hard. I already have a lazy list lib prototype as well as
generic functions for the basic ops. I think I’ll try to use the protocols
defined in the clojurescript version as much possible, rather than baking in a
lot of the seq abstraction in the host language like clojure does.

  • For simple bootstrapping, this if fine, but we already get all of this
    with the CLJS core implementation. So, get the minimums there and
    gain everything else….


CL strings are mutable (character arrays), clj/cljs are not…

  • Can we inherit from string to create an immutable type that
    outlaws setf?
  • I think most string operations (ala the sequence-based ones)
    return copies (which are mutable).

    • We can prevent setf in clojure mode, providing a pure API
      for string manipulation…

      • As well as impure….hmm


Leverage CL-PPRCE

  • check for reader macro collisions….

Compilation / Analysis

Currently, we project everything to (portable) CL, and let
the host do the dirty work for compilation/analysis.

  • Future, port

    • Maybe use that for some of our efforts…
  • If we have enough ported, look at using to help as well.


Currently just clone the repository. Assuming you have ASDF setup properly,
you should be able to evaluate (require ‘clclojure) at your Lisp REPL and it’ll
compile (if you’re in the project dir in lisp).

Alternately, you can compile and load the .asd file, then use quicklisp
to load it from the repl. (ql:quickload :clclojure)

You currently get persistent vectors, literals, defprotocol, extend-protocol.

You can mess with the currently limited clojure evaluator in the
:clclojure.base package, although I’m moving in a different direction now
that I think I can explot CL better.

You can see rudimentary examples and usage in the :clclojure.example
package (example.lisp). TODO: shift to named-readtables to keep
clclojure from hijacking your session. Right now, we take over the
REPL…..user beware!

I’m a bit new to building Common Lisp projects, so the packaging will likely
change as I learn. At some point, if anything useful comes out of this
experiment, I may see if it can get pushed to quicklisp.


Eclipse Public License, just like Clojure.