# Moonjit Extensions and API
Moonjit is fully upwards-compatible with Lua 5.1. It supports all [standard Lua
library functions](http://www.lua.org/manual/5.1/manual.html#5) and the full
set of [Lua/C API functions](http://www.lua.org/manual/5.1/manual.html#3).
Moonjit is also fully ABI-compatible to Lua 5.1 at the linker/dynamic loader
level. This means you can compile a C module against the standard Lua headers
and load the same shared library from either Lua or moonjit.
Moonjit extends the standard Lua VM with new functionality and adds several
extension modules. These extensions are intended to be fully compatible with
moonjit in the `v2.1.*` series of releases and mostly compatible in `v2.2.*` and
beyond.
## Extensions Modules
Moonjit comes with several built-in extension modules:
### `bit.*` — Bitwise operations
Moonjit supports all bitwise operations as defined by [Lua
BitOp](https://bitop.luajit.org/):
```
bit.tobit bit.tohex bit.bnot bit.band bit.bor bit.bxor
bit.lshift bit.rshift bit.arshift bit.rol bit.ror bit.bswap
```
This module is a moonjit built-in — you don't need to download or install Lua
BitOp. The Lua BitOp site has full documentation for all [Lua BitOp API
functions](http://bitop.luajit.org/api.html). The FFI adds support for 64 bit
bitwise operations, using the same API functions.
Please make sure to require the module before using any of its functions:
```
local bit = require("bit")
```
An already installed Lua BitOp module is ignored by moonjit. This way you can
use bit operations from both Lua and moonjit on a shared installation.
### `ffi.*` — FFI library
The [FFI library](ffi/ffi.md) allows calling external C functions and the
use of C data structures from pure Lua code.
### `jit.*` — JIT compiler control
The functions in this module [control the behavior of the JIT compiler engine](jit.md).
### C API extensions
moonjit adds some [extra functions to the Lua/C API](c_api.md).
### Enhanced Standard Library Functions
#### xpcall(f, err [,args...]) passes arguments
Unlike the standard implementation in Lua 5.1, `xpcall()` passes any arguments
after the error function to the function which is called in a protected
context.
#### loadfile() etc. handle UTF-8 source code
Non-ASCII characters are handled transparently by the Lua source code parser.
This allows the use of UTF-8 characters in identifiers and strings. A UTF-8 BOM
is skipped at the start of the source code.
#### tostring() etc. canonicalize NaN and ±Inf
All number-to-string conversions consistently convert non-finite numbers to the
same strings on all platforms. NaN results in "nan", positive infinity results
in "inf" and negative infinity results in "-inf".
#### tonumber() etc. use builtin string to number conversion
All string-to-number conversions consistently convert integer and
floating-point inputs in decimal and hexadecimal on all platforms. `strtod()` is
not used anymore, which avoids numerous problems with poor C library
implementations. The builtin conversion function provides full precision
according to the IEEE-754 standard, it works independently of the current
locale and it supports hex floating-point numbers (e.g. 0x1.5p-3).
<a name="string_dump"></a>
#### string.dump(f [,strip]) generates portable bytecode
An extra argument has been added to `string.dump()`. If set to true, stripped
bytecode without debug information is generated. This speeds up later bytecode
loading and reduces memory usage. See also the [-b command line
option](moonjit.md#b_opt).
The generated bytecode is portable and can be loaded on any architecture that
moonjit supports, independent of word size or endianess. However the bytecode
compatibility versions must match. Bytecode stays compatible for dot releases
(x.y.0 → x.y.1), but may change with major or minor releases (2.0 → 2.1) or
between any beta release. Foreign bytecode (e.g. from Lua 5.1) is incompatible
and cannot be loaded.
Note: `LJ_GC64` mode requires a different frame layout, which implies a
different, incompatible bytecode format for ports that use this mode (e.g.
ARM64 or MIPS64) or when explicitly enabled for x64. This may be rectified in
the future.
#### table.new(narray, nhash) allocates a pre-sized table
An extra library function table.new() can be made available via
`require("table.new")`. This creates a pre-sized table, just like the C API
equivalent `lua_createtable()`. This is useful for big tables if the final
table size is known and automatic table resizing is too expensive.
#### table.clear(tab) clears a table
An extra library function table.clear() can be made available via
`require("table.clear")`. This clears all keys and values from a table, but
preserves the allocated array/hash sizes. This is useful when a table, which is
linked from multiple places, needs to be cleared and/or when recycling a table
for use by the same context. This avoids managing backlinks, saves an
allocation and the overhead of incremental array/hash part growth.
Please note this function is meant for very specific situations. In most cases
it's better to replace the (usually single) link with a new table and let the
GC do its work.
#### Enhanced PRNG for math.random()
moonjit uses a Tausworthe PRNG with period 2^223 to implement math.random() and
math.randomseed(). The quality of the PRNG results is much superior compared to
the standard Lua implementation which uses the platform-specific ANSI rand().
The PRNG generates the same sequences from the same seeds on all platforms and
makes use of all bits in the seed argument. `math.random()` without arguments
generates 52 pseudo-random bits for every call. The result is uniformly
distributed between 0.0 and 1.0. It's correctly scaled up and rounded for
`math.random(n [,m])` to preserve uniformity.
#### `io.*` functions handle 64 bit file offsets
The file I/O functions in the standard `io.*` library handle 64 bit file
offsets. In particular this means it's possible to open files larger than 2
Gigabytes and to reposition or obtain the current file position for offsets
beyond 2 GB (`fp:seek()` method).
#### `debug.*` functions identify metamethods
`debug.getinfo()` and `lua_getinfo()` also return information about invoked
metamethods. The namewhat field is set to "metamethod" and the name field has
the name of the corresponding metamethod (e.g. "\_\_index").
### Fully Resumable VM
The moonjit VM is fully resumable. This means you can yield from a coroutine
even across contexts, where this would not possible with the standard Lua 5.1
VM: e.g. you can yield across `pcall()` and `xpcall()`, across iterators and
across metamethods.
### Extensions from Lua 5.2
moonjit supports some language and library extensions from Lua 5.2. Features
that are unlikely to break existing code are unconditionally enabled:
* `goto` and `::labels::`.
* Hex escapes `\x3F` and `\*` escape in strings.
* `load(string|reader [, chunkname [,mode [,env]]])`.
* `loadstring()` is an alias for `load()`.
* `loadfile(filename [,mode [,env]])`.
* `math.log(x [,base])`.
* `string.rep(s, n [,sep])`.
* `string.format()`: `%q` reversible. `%s` checks `__tostring`. `%a` and `%A` added.
* String matching pattern `%g` added.
* `io.read("*L")`.
* `io.lines()` and `file:lines()` process `io.read()` options.
* `os.exit(status|true|false [,close])`.
* `package.searchpath(name, path [, sep [, rep]])`.
* `package.loadlib(name, "*")`.
* `debug.getinfo()` returns nparams and isvararg for option `u`.
* `debug.getlocal()` accepts function instead of level.
* `debug.getlocal()` and `debug.setlocal()` accept negative indexes for varargs.
* `debug.getupvalue()` and `debug.setupvalue()` handle C functions.
* `debug.upvalueid()` and `debug.upvaluejoin()`.
* Lua/C API extensions: `lua_version()` `lua_upvalueid()` `lua_upvaluejoin()`
`lua_loadx()` `lua_copy()` `lua_tonumberx()` `lua_tointegerx()` `lua_len()`
`lua_rawlen()` `lua_absindex()` `lua_pushglobal()`
`luaL_fileresult()` `luaL_execresult()` `luaL_loadfilex()`
`luaL_loadbufferx()` `luaL_traceback()` `luaL_setfuncs()` `luaL_pushmodule()`
`luaL_newlibtable()` `luaL_newlib()` `luaL_testudata()` `luaL_setmetatable()`
`luaL_requiref()` `luaL_len()` `luaL_getsubtable()` `luaL_pushresultsize()`
`luaL_tolstring()`
* Command line option `-E`.
* Command line checks `__tostring` for errors.
* String matching patterns may contain `\0` as a regular character.
Other features are only enabled, if moonjit is built with
`-DLUAJIT_ENABLE_LUA52COMPAT`:
* `goto` is a keyword and not a valid variable name anymore.
* `break` can be placed anywhere. Empty statements (;;) are allowed.
* `__lt`, `__le` are invoked for mixed types.
* `__len` for tables. `rawlen()` library function.
* `pairs()` and `ipairs()` check for `__pairs` and `__ipairs`.
* `coroutine.running()` returns two results.
* `table.pack()` and `table.unpack()` (same as `unpack()`).
* `io.write()` and `file:write()` return file handle instead of true.
* `os.execute()` and `pipe:close()` return detailed exit status.
* `debug.setmetatable()` returns object.
* `debug.getuservalue()` and `debug.setuservalue()`.
* Remove `math.mod()`, `string.gfind()`.
* package.searchers.
* module() returns the module table.
**Note**: this provides only partial compatibility with Lua 5.2 at the language
and Lua library level. moonjit is API+ABI-compatible with Lua 5.1, which
prevents implementing features that would otherwise break the Lua/C API and ABI
(e.g. `_ENV`).
### Extensions from Lua 5.3
Moonjit supports some extensions from Lua 5.3:
* Unicode escape `'\u{XX...}'` embeds the UTF-8 encoding in string literals.
* The argument table arg can be read (and modified) by `LUA_INIT` and `-e` chunks.
* `io.read()` and `file:read()` accept formats with or without a leading `*`.
* `assert()` accepts any type of error object.
* `string.pack(fmt, v1, v2, ···)`, `string.packsize(fmt)`, `string.unpack(fmt, s [, pos])`
* `table.move(a1, f, e, t [,a2])`.
* `coroutine.isyieldable()`.
* `math.maxinteger`, `max.mininteger`, `math.tointeger(x)`, `math.type(x)`, `math.ult(m, n)`
**Note:** moonjit uses the same numeric type model as Lua 5.1 which is incompatible
with Lua 5.3. As a result, these functions work only in the range [-2^53,
2^53]. `math.maxinteger` and `math.mininteger` thus give the limits of this
range.
* `utf8.char(...)`, `utf8.charpattern`, `utf8.codepoints(s [, i [, j]])`, `utf8.codes(s)`,
`utf8.len(s [, i [, j]])`, `utf8.offset(s, n [, i])`
* Lua/C API extensions: `lua_isyieldable()`, `luaopen_utf8()`
### Extensions from OpenResty luajit2
The following extensions were incorporated from luajit2, a LuaJIT fork
maintained by the OpenResty project:
#### thread.exdata
**syntax:** *exdata = th_exdata(data?)*
This API allows for embedding user data into a thread (`lua_State`).
The retrieved `exdata` value on the Lua land is represented as a cdata object
of the ctype `void*`.
As of this version, retrieving the `exdata` (i.e. `th_exdata()` without any
argument) can be JIT compiled.
Usage:
```lua
local th_exdata = require "thread.exdata"
th_exdata(0xdeadbeefLL) -- set the exdata of the current Lua thread
local exdata = th_exdata() -- fetch the exdata of the current Lua thread
```
Also available are the following public C API functions for manipulating
`exdata` on the C land:
```C
void lua_setexdata(lua_State *L, void *exdata);
void *lua_getexdata(lua_State *L);
```
The `exdata` pointer is initialized to `NULL` when the main thread is created.
Any child Lua thread will inherit its parent's `exdata`, but still can override
it.
**Note:** This API will not be available if moonjit is compiled with
`-DLUAJIT_DISABLE_FFI`.
**Note bis:** This API is used internally by the OpenResty core, and it is
strongly discouraged to use it yourself in the context of OpenResty.
#### jit.prngstate
**syntax:** *state = jit.prngstate(state?)*
Returns (and optionally sets) the current PRNG state (a Lua number) currently
used by the JIT compiler.
When the `state` argument is non-nil, it is expected to be a number, and will
override the current PRNG state.
Usage:
```lua
local state = jit.prngstate()
local newstate = jit.prngstate(123456)
```
**Note:** This API has no effect if moonjit is compiled with
`-DLUAJIT_DISABLE_JIT`, and will return `0`.
#### -bl flag for jit.dump
The bytecode option `l` was updated to display the constant tables of each Lua
prototype.
For example, `luajit -bl a.lua'` now produces bytecode dumps like below:
```
-- BYTECODE -- a.lua:0-48
KGC 0 "print"
KGC 1 "hi"
KGC 2 table
KGC 3 a.lua:17
KN 1 1000000
KN 2 1.390671161567e-309
...
```
#### bytecode option L to display lua source line numbers
The bytecode option `L` was added to display Lua sources line numbers.
For example, `luajit -bL -e 'print(1)'` now produces bytecode dumps like below:
```
-- BYTECODE -- "print(1)":0-1
0001 [1] GGET 0 0 ; "print"
0002 [1] KSHORT 1 1
0003 [1] CALL 0 1 2
0004 [1] RET0 0 1
```
The `[N]` column corresponds to the Lua source line number. For example, `[1]`
means "the first source line".
#### Trace logging for debugging the JIT compiler
Internal memory-buffer-based trace entry/exit/start-recording event logging,
mainly for debugging bugs in the JIT compiler. it requires `-DLUA_USE_TRACE_LOGS`
when building moonjit.
<a name="exceptions"></a>
### C++ Exception Interoperability
moonjit has built-in support for interoperating with C++ exceptions. The
available range of features depends on the target platform and the toolchain
used to compile moonjit:
POSIX/x64, DWARF2 unwinding | GCC 4.3+ | Full
Other platforms, DWARF2 unwinding | GCC | Limited
Windows/x64 | MSVC or WinSDK | Full
Windows/x86 | Any | No
Other platforms | Other compilers | No
**Full** interoperability means:
* C++ exceptions can be caught on the Lua side with `pcall()`, `lua_pcall()`
etc.
* C++ exceptions will be converted to the generic Lua error "C++ exception",
unless you use the [C call wrapper](c_api.md#mode_wrapcfunc) feature.
* It's safe to throw C++ exceptions across non-protected Lua frames on the C
stack. The contents of the C++ exception object pass through unmodified.
* Lua errors can be caught on the C++ side with `catch(...)`. The corresponding
Lua error message can be retrieved from the Lua stack.
* Throwing Lua errors across C++ frames is safe. C++ destructors will be
called.
**Limited** interoperability means:
* C++ exceptions can be caught on the Lua side with `pcall()`, `lua_pcall()`
etc.
* C++ exceptions will be converted to the generic Lua error "C++ exception",
unless you use the [C call wrapper](c_api.md#mode_wrapcfunc) feature.
* C++ exceptions will be caught by non-protected Lua frames and are rethrown as
a generic Lua error. The C++ exception object will be destroyed.
* Lua errors cannot be caught on the C++ side.
* Throwing Lua errors across C++ frames will not call C++ destructors.
**No** interoperability means:
* It's not safe to throw C++ exceptions across Lua frames.
* C++ exceptions cannot be caught on the Lua side.
* Lua errors cannot be caught on the C++ side.
* Throwing Lua errors across C++ frames will not call C++ destructors.