The Lua API is a tool that allows users to extend the game with their own code pieces without recompiling the application. If you read the Lua reference manual, you probably see that not all things that logically may be scripted are actually scriptable yet. So, maybe you want to join the development team and write some more functions, classes and signals for game elements. If you read this, you are supposed to have some basic knowledge of C and Lua programming; if so, writing the necessary code probably won't be much difficult for you.
Working with Tolua
To connect Lua to C, Freeciv developers use tolua, a bit abandoned but well known thing to do so. The 5.2 version of the tool can be found in dependencies/tolua-5.2 subdirectory of the source tree. Tolua uses package files (*.pkg) to describe comprehensively the C-Lua interfaces; when, called by the make shell command, the tool works, it produces corresponding C code that is then compiled into the application (the header files to be generated should be listed in #include directives wherever their functionality is applied). This code, when it is called, manipulates the builtin Lua environment and adds to it the functions and variables that are the scripting API.
Files and compilation
The code embedded with tolua mainly goes in package files scripting_dir/tolua_something.pkg where scripting_dir is a subdirectory where scripting-related sources for given application reside and something is your application that may describe a specific Lua API table, or a specific part of it, or whatever. The C side of the API functions is stored in the same directory in api_something_morespecific.[ch] source files that follow regular C syntax; some more C files may lay nearby to assist them and to define frontend of scripting for the other code. For example, client-specific API lives in client/luascript, there you can find tolua_client.pkg that imports functions from api_client_base.h defined in api_client_base.c, and also you can find there script_client.[ch] that encapsulates tasks like initializing the client's Lua state, do a script file, invoke a signal etc. In the Makefile.am in the directory you can see what and how tolua will produce:
tolua_client_gen.c tolua_client_gen.h: tolua_client.pkg $(TOLUA) -n client -o $(srcdir)/tolua_client_gen.c \ -H $(srcdir)/tolua_client_gen.h $(srcdir)/tolua_client.pkg
So, to actually provide to the script-writing user what you have made with tolua on something, you put corresponding lines on Makefile.* (if you have added a new file) and list tolua_something_gen.h in an #include directive where you initialize the state (it provides int tolua_something_open (lua_State* tolua_S) to put the functions into S).
Contents of a .pkg file
Strings that start in $, with certain exceptions, will go to the generated C/C++ file as they are; this is mainly used to inlude your API functions and the types used in their signatures:
$#include "fc_types.h"$#include "api_something_morespecific.h"
Mainly, you want to access the game objects, that are C structs. They can be managed by Lua as "userdata" type that represents arbitrary binary data; for the code to understand what could be done to the object, the userdata are tagged with a metatable that describes the fields and methods. More specifically, tolua uses light userdata that are merely pointers to the actual objects stored in Lua state structures with some metadata attached. The Direction type that is merely enum direction8 is also stored by pointer to a malloc()ed integer that is freed by Lua garbage collector. Note, the struct does not need to be programmed in object-oriented way in the compiling language, but it's natural to use it so in the scripts. Of course, the functional programming elements are also used where they are needed (but you are supposed to not contaminate _G with items better belonging to more specific tables).
In the .pkg file, you list only those fields of the struct that you want to give to the user:
/* somestruct.h */
struct somestruct {
int user_int_data;
void *server_secret_block;
}
/* luascript_types.h */
typedef struct somestruct Somestruct;
/* tolua_something.pkg */
Somestruct {
int user_int_data;
}
A table Somestruct will become a metatable for all pointers to somestructs passed to Lua code, its members starting from letters will become the objects' methods. Note: writing access would be granted intristically by Somestruct[".set"] subtable but it is deleted by a mechanism invoked by tolua_common_z.pkg that is appended after all other packages to all instances.
The fuctions are attached to the tables of methods or any other tables usually within a module syntax:
module Somestruct {
api_something_somestruct_somefun
@ somefun (lua_State *L, Somestruct *self, int param = 0);
api_something_somestruct_user_int_data_set
@ setint (lua_State *L, Somestruct *self, int new_value);
}
If the table does not exist, it will be created, if it exists, it will be extended, so you can break a module even between different packages (e.g. Unit API has common part and parts specific to the client or the server). Modules can be included into each other (resulting in subtables). The function prototype must be basically strictly the same as in the header file but may be extended by a Lua pseudonym (after @) and parameter default values (they are a tolua feature not dependent on C++; only a group of last parameters may have them).
Also, the package may contain embedded Lua code lines in $[ ... $]. They are executed during the API loads immediately after anything before them influents the state. For example, all standard classes are extended with Lua-written method (class):exists() that just returns true, but for Unexistent it returns false; there is a mechanism that replaces the metatable of Lua userdata referring to an expired game object on Unexistent (but it works now only on the server.)
Enums may be redeclared in packages with basic C syntax (plus @ pseudonyms) resulting in a table with named constants.
Some trivia about function prototypes
- Standard types are mapped to Lua types: bool to boolean, int and double to number, const char* to string. Pointers to declared structs are mapped to userdata, void* to any userdata. The C functions that you write are wrapped into a code that makes necessary conversion and Lua state manipulation.
- Arrays map to tables. Their dimension must be either constant or calculated from other arguments.
- NULL in any pointer corresponds to Lua nil.
- If the parameters don't have default values, the interpreter will complain for filling not all of them in a call.
- You can declare several Lua functions with the same Lua name and different prototypes, they will be selected at runtime in reversed declaration order, what fits the first. When prototypes are different in string and some other type of an arguement, the const char* variant must be declared first.
- The reason is that most types have some string representation and will be interpreted as string if it is the first tested option, and that string will be usually not at all what you would like there; also, nil is considered a valid string value for tolua. Another lurking polymorphism trap is that strings that are conversible to integer or real number (like "-01", or "01e9") will be easily taken as corresponding numerical value, that you sometimes also don't want (people can give a name like this to a city - yes, they really do it!) The solution may sometimes be using a lua_Object parameter and testing its kind yourself with lua_type(L, v) == LUA_TNUMBER, tolua_isstring(L, v) etc.
- Normally, your function returns zero (void) or one value. If you want to return more values, you use pointers as parameters:
void api_something_two_vals_ret @ twov (int *a = 0, int *b = 0); $[ local a, b = twov() $]
lua_State *type parameter is not requested from the calling Lua code but refers to the Lua state it is running on.lua_Valuetype for an argument or the function allows you to use arbitrary Lua value. In fact, it is just an integer marking Lua stack position.
Freeciv code conventions and recipes
The code of Freeciv applications that have builtin Lua contains several tolua packages, to which the API is divided for programming logic and convenience. The API of common game concepts, naturally, reside in the folder common/scriptcore. When the scripting environment is initialized, firstly tolua_common_a is applied to the Lua state, then come other common packages, then the specific packages of the instance, and finally the API is tuned (that means, cleaned from all that is less idiot-proof) by tolua_common_z. See how it is done for the server in server/scripting/script_server.c:
//...
/* dependencies/lua */
#include "lua.h"
#include "lualib.h"
/* dependencies/tolua */
#include "tolua.h"
//...
/* common/scriptcore */
#include "api_game_specenum.h" // Autogenerated E table
#include "luascript.h"
#include "luascript_signal.h"
#include "luascript_func.h"
#include "tolua_common_a_gen.h" // *_gen files generated by tolua
#include "tolua_common_z_gen.h"
#include "tolua_game_gen.h"
#include "tolua_signal_gen.h"
//...
/* server/scripting */
#include "tolua_server_gen.h"
//...
/***********************************************************************//**
Lua virtual machine states.
***************************************************************************/
static struct fc_lua *fcl_main = NULL;// The global Freeciv server Lua state
static struct fc_lua *fcl_unsafe = NULL;
//...
/***********************************************************************//**
Initialize the scripting state.
***************************************************************************/
bool script_server_init(void)
{
//...
fcl_main = luascript_new(NULL, TRUE);
//...
tolua_common_a_open(fcl_main->state);
api_specenum_open(fcl_main->state);
tolua_game_open(fcl_main->state);
tolua_signal_open(fcl_main->state);
tolua_server_open(fcl_main->state);
tolua_common_z_open(fcl_main->state);
// Now, the API for the server is ready,
// we populate it with some things defined out of tolua
script_server_code_init();
script_server_vars_init();
luascript_signal_init(fcl_main);
script_server_signals_create();
luascript_func_init(fcl_main);
script_server_functions_define();
//...
}
Another useful thing that tolua_common_z.pkg inline code does is turning into (readonly) property of any standard class each function defined to the moment in its ClassName.properties subtable:
/* api_something_foo.c*/
int api_something_somestruct_index (lua_State *L, Somestruct *self)
{
LUASCRIPT_CHECK_STATE(L, -1);
LUASCRIPT_CHECK_SELF(L, self, -1);
return self->zero_based_index + 1;
}
/* tolua_something_foo.pkg */
module Somestruct {
module properties {
int api_something_somestruct_index @ index (lua_State *L, Somestruct *self);
}
}
Now you can use (Somestruct).index in Lua code that will start at 1 if internal somestruct.zero_based_index starts at 0.
Also, at the end of all API initialization the table private is deleted from _G. You are supposed to put to it and its subtables the functionality you need but don't want to give to the scriptwriters directly (e.g. C list iterators). You'd better relocate the functions from there to local variables soon after their definition for safety and performance.
Another Lua-written tools include _G serializer freeciv_state_dump() (server records its value into savegames, in 2.6 only scalar variables and values gettable by find.a_type(...?, id) are saved) and const metatabled table that only accepts new values, not allowing to change existing ones, and makes an unmodifiable interfaces for subtables put into it.
Basic style
As it was said before, scripting-related files should be put into special subdirectory of an instance directory in the source tree. Package files are named tolua_something.pkg format. The main need in dividing API between packages is when they must be loaded on separate instances, or in different time.
The C/C++ functions provided to the scripts with tolua should lay in special header files (and corresponding definition files) named api_something_morespecific.[ch] (something should if posdible correspond to the package name, morespecific is methods for bundles of object-oriented interface). Each function name from the file should also start from api_something_, for methods then goes the class name in lowercase, and a proper function name; the Lua pseudonym must be the most comprehensive name that not necessary is a part of C api_... function name (but it is good if it is the same as a function doing the same thing in C/C++ code).
Usually, when in C a function or constant name is logically divided with underscores, you should make modules to divide it with dots in Lua (e.g. log.normal() instead of log_normal()).
Checks
Note that one who writes a script is not obliged to be an omniscent genius. Neither you are. Any of your API functions may be used in a wrong time, in a wrong place and with wrong parameters. You should check if they are now, and if so, raise an error via luaL_error() function. No Lua error is supposed to halt the application.
In the API functions you should always check if the state is ready and the patameters are valid. So, in spite of that it is not mandatory for tolua, you should always start your function prototype with a lua_State* parameter. You should include luascript.h file for macros that provide you some checking utilities, here are the most common:
LUASCRIPT_CHECK_STATE(L, retval)
If L is NULL, prints an error message and returns retval from the function (if the function is not void, you have to provide some matching type value here).
LUASCRIPT_CHECK_ARG(L, check, narg, msg, retval)
If check is logically false, raises a scripting error reporting "bad argument #narg to '<func>': msg".
LUASCRIPT_CHECK_ARG_NIL(L, value, narg, type, retval)
If value is NULL, reports an error "got 'nil', '" #type "' expected" in narg-th argument.
LUASCRIPT_CHECK_SELF(L, value, retval)
If value is NULL, says that you have not provided self for the method.
Debugging tolua packages
Now, about the probably darkest side of tolua, at least how it appears to be in Freeciv-2.6.0.
- You don't know where you have missed the @£=}[●>ed line end semicolon. The parsing error message reports lines in tolua itself, not your pkg.
- make does not require tolua to do its work. If you have compiled your project before and there are some tolua_*_gen.[ch], gcc will link it; you can only scroll compilation log to see if tolua has ever failed in process.
- tolua code itself is darn hard to ynderstand, being a conglomerate of endless Lua chunks wrapped into C. If you can easily get through it, you hardly need this manual.
- tolua performs little to no syntax checking in $[...$] Lua blocks. They will be converted to unreadable byte arrays and will be executed only when the application interprets them. The error output will say you that your Lua has broken on the line 5 but won't say line 5 of which of the blocks it was.
- I don't know any editors that parse tolua*.pkg files correctly.
Automatic enum import
Module E, containing constants for message types, is autogenerated from event_type specenum defined in common/events.h.
Adding a game object type
Additionally to putting corresponding struct and module in a package, you have to mention a new standard type specially here:
- add the name to
local api_typestable in tolua_common_z.pkg (to finalize the class API properly); - put a
typedef struct somestruct Somestructline into common/scriptcore/luascript_types.h to get rid of "struct" qualifier in tolua-related code. For list links, you'd better use "const" modifier in the typedef. - In the same file, create an item in
api_typesspecenum (mainly needed for defining signals):
/*replace 99 with appropriate index*/ #define SPECENUM_VALUE99 API_TYPE_SOMESTRUCT #define SPECENUM_VALUE99NAME "Somestruct"
- If the object you provide to Lua can possibly be cleared from the memory while the Lua state continues to be used, userdata referring to it must be reset to
Unexistentto avoid unhandled segmentation fault error. For the server, the functionscript_server_remove_exported_object()is called from the destructors for player, unit and city data; for the client, the problem is not yet handled, see HRM#79990. - Check how your type is saved by
_freeciv_state_dump()in tolua_common_a.pkg. By default, user object that provides "id" property is saved as"find." .. string.lower(tolua.type(v)) .. "(" .. v.id .. ")", one that does not provide it is saved in the same way without any parameter. So, you probably want to definefind.somestruct()method and/or modify the saving function.
Defining signals
Signals are event handlers that on specific occasions call script-defined functions. They are supposed to exist for all Lua instances but in 2.6 they are developed only on server; there are though project forks that define them for the client, just things doable with them like client-side autoattack may be considered cheatery by other players... However, Freeciv is not about stopping the progress because somebody else does not like it.
So, to define a signal, you write on certain point in your code:
luascript_signal_create(fcl_main, "my_signal", /*2,*/ API_TYPE_INT, API_TYPE_SOMESTRUCT)
that means that a function accepting 2 arguements (the number is autocalculated since v.2.6), the first shall be an integer number and the second shall be a Somestruct userdata. The point where you put it should lay in a special function in your main scripting C file in your application scripting directory:
- for the server, it's script_server_signals_create() in server/scripting/script_server.c;
- for the client, it's script_client_signal_create() in client/luascript/script_client.c.
The functions return a pointer to the created signal's deprecation string that may be used in deprecate_signal().
Creating a signal allows you to bind Lua functions to it, but it then should be emitted somewhere (that the only client signal, "new_tech", never does up to v.3.0). To emit a signal, you put into the code point active at the desired moment a call of
script_application_signal_emit(fcl_main, some_int, &some_spmestruct)
that will call the Lua functions bound to the signal in their reverse definition order until either the list ends or one of them returns true. Now, an important moment:
|
Any portion of a scripted code run at any time in the game may change anything, up to everything! |
So, you can well expect that some scenario author will need a "unit_moved" callback that once turns all lakes into tundra, throws all players into a civil war (if they have no cities, gives them a pair built on ocean tiles) and replaces all military units onto workers on the opposite side of the world. If a script can't do some changes (except 100% game-breaking or dangerous stuff), remember that you are (we hope!) not the last one who extends its capability and make a code that won't easily break if it gets possible.
If you don't want that ever happen, write a checking function that will stop the incorrect script and print an error message to the console. But you probably often won't.
So, after emitting a signal, check that any previously calculated assumptions are still valid: the objects used by the C function still exist (see HRM#824815 for handling the parameters inside signal emission), the diplomatic state does not now block the attack etc.