I have started reading LuaJIT sources. I like the fact that the source code is compact and it is reasonable to print and read a whole file (or read it an iPad with iOctocat
The parts I am reading now are theprofiler, thedumper, and thetrace assembler. I have a basic mental model of tracing JITs from Thomas Schilling's thesis
I have a few interests here:
- I would like to have a stronger mental model of the data structures involved. How is the Intermediate Representation stored in memory? Is it ephemeral or persistent? How much cross-referencing information is available between the representations: could you generate an interleaved listing of the IR and the machine code for example? (source too?) I am accustomed to knowing these kind of details from other languages like Forth, Lisp, and Smalltalk, but I haven't dug down to that level of LuaJIT yet.
- Does the assembler really assemble backwards from the last IR instruction? (If so then does each IR instruction assembler emit the machine code backwards too?)
- How can I always have a visceral feeling for how my code is executing on the CPU? Currently it takes me quite a bit of manual legwork to analyze program behavior: dump traces to a file, profile to see which traces are relevant, stare at the traces to see which code they are related too, and so on. I would love to have this much more streamlined e.g. for the profiler to automatically show me an interleaved IR/machinecode dump of all traces using >= 5% CPU with annotations on the hotspots. This is the kind of thing that is quite transparent in
when programming in C.
- I would like to have a better feeling for what makes LuaJIT happy, what makes it sad, and what makes it unpredictable. I want to really see in the generated code what are the consequences of things like unpredictable branches within loops. I am sure that I could adapt my programming style to be better suited to the compiler but this has to be driven by a better understanding of the compiler rather than following "do this, don't do that" lists of program optimization rules.
Generally I am very enthusiastic about LuaJIT. I do see it as a technology in the tradition of Lisp, Forth, and Smalltalk: one that is intellectually rewarding to study and use. I look forward to spending a lot more time with it.