Any C++ <-> Python programmers interested in contribut

[Dave]

I think it'd be great if we had the ability to write AIs in Python. This would allow scenario designers to code their own AI specific to scenarios they are writing, and would open a whole new world of opportunities with respect to what can be done in a scenario.

Currently there are many great scenario ideas which can't be done well simply because the AI isn't well suited to them. With the ability to customize the AI in Python, this would all change.

So, I'm wondering, are there any skilled programmers who would be interested in implementing this? That is, in writing a Wesnoth AI that takes a Python script in its arguments, and implements itself by calling the Python script. It would, of course, use the Python restricted execution model to stop the script from doing something nasty to the user's machine (we don't want to create the possibility of 'Wesnoth scenario viruses' through the campaign server!)

I'd suggest using Boost::Python, though other libraries (perhaps directly using the Python C API) would also be possible.

Obviously anyone who writes this would have to be a fairly skilled C++ programmer. This would be a great way to become involved in Wesnoth development.

Anyone interested?

David

[ILikeProgramming]

This would be a great idea because I'll have the cahnce to write AI's in Python
Though this might be medium-difficulty to implement, how will the AI run on a machine without the python interpreter?

[lwa]

This would be a great idea because I'll have the cahnce to write AI's in Python
Though this might be medium-difficulty to implement, how will the AI run on a machine without the python interpreter?

I think the python library has to be linked with the game, as is currently SDL.

The game will give access to some of its internal variables and functions to python and interpret a python function in a script when the AI (or anything else) is triggered.

[allefant]

Or just read here: http://www.boost.org/libs/python/doc/

[Ryo]

(note: i'm not a Wesnoth developer, though i skimmed through the code. I do have some experience with making a Python interface to a game)

I'd say the first step is defining precisely what functions, objects, structures, ... exactly Python scripts should be able to access.
Then how scripts will access that (objects probably: Wesnoth.Unit? Wesnoth.Square? and so on).
Then it's just a matter of wrapping all, and binding with Python (sounds simple, doesn't it? :p).

Things to consider:
* when will Python be called? Every step? In a background thread to compute nicely?
* what about the script passing invalid arguments to code? Should everything be hardened so that script can't possibly crash the core? Or should script developers be excepted to be nice & don't mess too much?
* what about using Python for other things besides AI (change terrain, insert units? That would be redundant with WML though)
* what about performance? Not trolling, just wondering
* will Python then be mandatory? Should everything be linked statistically (is that possible?), or distribute Python libraries, but then you have to decide which to distribute

[Xan]

Why python?
Why not Lua?

IMHO, Lua is much easier to program in.

http://lua-users.org/wiki/LuaVersusPython

[allefant]

I guess, AI interfaces for multiple languages would be possible without problems - simply have source folders like: ai/python, ai/lua, ai/perl. Then compile your wesnoth.exe with support for whatever you want to have, and AIs could then use scripts written in a supported language.

[lwa]

I guess, AI interfaces for multiple languages would be possible without problems - simply have source folders like: ai/python, ai/lua, ai/perl. Then compile your wesnoth.exe with support for whatever you want to have, and AIs could then use scripts written in a supported language.

But it will increase the requirements to build wesnoth.

[Corwin]

I guess, AI interfaces for multiple languages would be possible without problems - simply have source folders like: ai/python, ai/lua, ai/perl. Then compile your wesnoth.exe with support for whatever you want to have, and AIs could then use scripts written in a supported language.

I agree that it would be easy to have AIs in any language as long as someone added that language's support to the game. That is the hard part.

[Dave]

I'd say the first step is defining precisely what functions, objects, structures, ... exactly Python scripts should be able to access.
Then how scripts will access that (objects probably: Wesnoth.Unit? Wesnoth.Square? and so on).

Yes, I agree. In fact, working out all this is a large part of the problem.

When coding an AI in C++, the coder derives from ai_interface, and overrides play_turn() with their code to control the AI. Then ai_interface provides some functions to allow the AI to move units, attack enemies, recruit units, get information about the game state, and so forth. All these functions would be exposed to Python. All this is in ai_interface.hpp.

This is mostly straightforward except for the function get_info() which returns an object of type ai_interface::info which has a wealth of references to objects which contain the state of the game. These objects include:

- display, which I don't think has to be exposed to Python. This is used by C++ AIs mostly for debugging purposes. We might want to allow Python scripts to write messages to the display for debugging purposes also, but they would only need very limited access.
- gamemap, defined in map.hpp, which contains the definition of the map. Definitely needs to be exposed to Python.
- game_data, defined in unit_types.hpp, which contains definitions of the available unit types, attacks, races, and so forth. Needs to be exposed to Python.
- unit_map, a map<gamemap::location,unit>. gamemap::location is defined in map.hpp and unit is defined in unit.hpp. This contains all the units currently in the game. Will need to be exposed.
- vector<team>, a listing of the teams (i.e. sides) in the game. Defined in team.hpp. Will need to be exposed.
- gamestatus, the current turn and time of day, etc. Defined in gamestatus.hpp. Will need to be exposed.
- turn_info, does not need to be exposed.

Additionally, useful utility functions such as those found in pathutils.hpp should be exposed.

Also, it will likely turn out that certain functionality will be used by many Python scripts and is fairly performance-intensive. Additional 'helper' modules that could be written in Python but are slow could be implemented in C++ and made available. That's something we'd do once the basic proof-of-concept is working though.

We will also likely want to implement some

We can start off very simple though, and only expose the base functionality in ai_interface.hpp. This will essentially allow someone to code an AI in Python, but it will be playing completely 'in the dark', without any knowledge of where units are and so forth (and so admittedly will make it rather difficult for it to make any legal moves at all ). However once we have this 'base' down, adding more functionality will probably be pretty easy.

* when will Python be called? Every step? In a background thread to compute nicely?

Well, currently an AI is derived from ai_interface, and every time it's the AI's turn, play_turn() gets called. I would imagine that we would have python_ai which is defined from ai_interface, and in its play_turn() it would call the Python script to control the turn.

It'd just have to sit there, remaining interface responsive while the Python script runs. To keep the interface responsive, it might be best to make the Python script run in its own thread. Whenever the Python script calls a function to do something, such as move a unit, it'd probably pass a message to the interface thread telling it what's happening. The interface thread would call the ai_interface function to implement the move, ai_interface would return a result which would be passed back to the Python thread, which would continue.

If you can find a way to do it all in one thread, I'd also be open to that.

* what about the script passing invalid arguments to code? Should everything be hardened so that script can't possibly crash the core? Or should script developers be excepted to be nice & don't mess too much?

Scripts shouldn't be allowed to crash Wesnoth. All results and arguments from scripts should be checked.

* what about using Python for other things besides AI (change terrain, insert units? That would be redundant with WML though)

It might be useful for random map generation.

For other things, sure, it might be useful for some people, but I think alot less useful than for AI. I think we should get it working for one thing and then we can perhaps discuss its use for other things.

* what about performance? Not trolling, just wondering

We'd try to move most performance-intensive tasks into C++ code and provide interfaces.

However, if someone is going to want to write "The Deep Blue of Wesnoth" they will probably have to do it in C++. What Python provides is a way for scenario designers to make the AI do things very specific to their scenario.

* will Python then be mandatory? Should everything be linked statistically (is that possible?), or distribute Python libraries, but then you have to decide which to distribute

I think Python should be mandatory, but linked dynamically.

David

[gauss]

So, I'm wondering, are there any skilled programmers who would be interested in implementing this? That is, in writing a Wesnoth AI that takes a Python script in its arguments, and implements itself by calling the Python script. It would, of course, use the Python restricted execution model to stop the script from doing something nasty to the user's machine (we don't want to create the possibility of 'Wesnoth scenario viruses' through the campaign server!)

Python's restricted execution modules were disabled in version 2.3 and higher, because of security holes created by new style classes. There were other problems before 2.3 as well. However, Zope has a RestrictedPython module that is supposed be secure.

Python is easy to embed, and the C API is well documented. The original poster mentioned Boost, but SWIG might be worth a try.

[Ryo]

Python's restricted execution modules were disabled in version 2.3 and higher, because of security holes created by new style classes. There were other problems before 2.3 as well. However, Zope has a RestrictedPython module that is supposed be secure.

Python is easy to embed, and the C API is well documented. The original poster mentioned Boost, but SWIG might be worth a try.

Interesting to know about the restriction.

Thought about SWIG, yeah. I'll need to play around with Boost & SWIG to see if it's worth using them or not
Note that Boost is not thread-safe, something that may be a bother.

[bison]

Scripting AI's in Python is a great idea. Is there already something like a Python wrapper around elementary game functions (calling a server, initialize map, initialize seed, get legal moves, etc...) It could also be used for staging thousands of contests between differently configured AI's in an evolutive setting, as well as for statistical analysis of maps or game strategies. I have some experience in Python, C++, general and game AI's, ranking and tournament systems, and would like to contribute where i can.

The thing we would need to create first would be a Python wrapper for all relevant game functions and/or a text communication protocol for AI's and online games for easy interface

During the testing phase of different AI strategies and parameters, guidelines for the developement of AI developement tools could evolve into a well-thought out project.

Where can i find the relevant information?

[Ryo]

So far there is no wrapper or anything.

Haven't yet had the time to play with things

Hasn't been much discussion on that yet, either - best place to check is either the irc channel, or the dev mailing list. I wanted to tweak a few things before sending suggestions & questions & such.

Note that I volunteered to help on that too, so we can certainly cooperate

[ryn]

Why Python or Lua? Why not a generic I/O interface?

ie, the game can run and open a pipe to any executable and then communicate with it, allowing AIs in any language - even an interpreted language can easily be used (to clarify the use of 'executable') with a batch file or shell script. We'd just need to specify a protocol and create a class to encapsulate the actions of such generic AIs (pretending that it, in fact, is executing moves while actually running the external AI).

The pros in taking such action is that we leave it up to individual programmers to create libraries to wrap the IO protocol in a simple manner, and the game has (potentially) infinite AIs in Python, Lua, LISP, APL, J, BASIC, Plankalkul, and any other programming language Wesnoth fans know. Also, we don't need to deal with adding, learning, or deciding on using other libraries. The only con I can think of is that individual programmers actually need to create such functions to encapsulate the protocol on their side. Of course, because of the number of AI functions accessible to the programmer, that might be a problem.