What is a tracing JIT?
Although tracing JITs are a complex technology, the core concept is about optimizing execution of the hot paths in a program. The emphasis is specifically on hot paths that return to the start of a path which sounds very much like a loop. However, the traditional definition of a programming loop is only a subset of these hot paths. The broader definition includes code that spans methods and possibly even modules. This broader definition of a loop is what’s called a trace.
Why use traces?
By JIT compiling the traces, code that is executed frequently runs faster. Code that isn’t included in traces isn’t compiled. The claim is untraced code would not benefit as greatly from JIT compilation since it is not executed as frequently. The basic trade-off is the time needed to JIT compile and cache the resulting code versus the time needed to execute the code.
When does tracing happen?
How is a tracing JIT different from other JITs?
Run-time profiling for optimization and specific CPU and memory optimizations are done with most JITs to adapt the application for best performance. However, non-tracing JIT implementations (Sun’s Java VM and Microsoft’s CLR) do this at the method level. When code is determined to be “hot,” the entire method containing that code is JIT compiled and cached. Tracing JITs do this at the trace level and may consist of a portion of a method (just a specific loop), multiple methods and even multiple methods from different modules.
Who is using tracing JITs?
A Python interpreter called PyPy recently switched from being based on partial evaluation to a tracing JIT. Microsoft is researching and prototyping a tracing JIT compiler (TJIT) for the common intermediate language (CIL) called SPUR.
Where can I find out more?
This post only scratches the surface. There’s plenty more about tracing JITs!