Proposed radical WML changes

[Viliam]

I have looked at event WML wiki page, and my impression was that the number of events is growing, which can lead to chaos unless some system is designed. There are a few irregularities, and some questions I was unable to answer by reading the documentation. I think it would be good to make the system more regular and intuitive, and that's why this topic is opened.

I would like to discuss not only how the system works now, but why it works so, and whether some re-designing could lead to a more elegant solution, which in long term would be easier to maintain for coders, and easier to use for scenario developers.


Here are the things I consider irregular or unobvious:


Why is "attack" and "attack_end", but "advance" and "post_advance"?

Why are two events for "attacker_hits" and "defender_hits", but only one event for "die"?

Why first turn has "start", but the following turns have "turn 2", "turn 3", etc? (By the way, is "turn 2" really equivalent to "start" or rather to "prestart"?)

In which order do the following events happen at the beginning of the second turn: "turn 2", "new turn", "side turn"?

If my unit captures a village and sees a new enemy unit in the same move, which event comes first: "capture" or "sighted"?

If my unit kills enemy leader, so it wins scenario, and it advances, in which order do the following events happen: "advance", "enemies defeated", "victory"?

[scott]

I have looked at event WML wiki page, and my impression was that the number of events is growing, which can lead to chaos unless some system is designed. There are a few irregularities, and some questions I was unable to answer by reading the documentation. I think it would be good to make the system more regular and intuitive, and that's why this topic is opened.

I would like to discuss not only how the system works now, but why it works so, and whether some re-designing could lead to a more elegant solution, which in long term would be easier to maintain for coders, and easier to use for scenario developers.


Here are the things I consider irregular or unobvious:


Why is "attack" and "attack_end", but "advance" and "post_advance"?

Why are two events for "attacker_hits" and "defender_hits", but only one event for "die"?

Why first turn has "start", but the following turns have "turn 2", "turn 3", etc? (By the way, is "turn 2" really equivalent to "start" or rather to "prestart"?)

In which order do the following events happen at the beginning of the second turn: "turn 2", "new turn", "side turn"?

If my unit captures a village and sees a new enemy unit in the same move, which event comes first: "capture" or "sighted"?

If my unit kills enemy leader, so it wins scenario, and it advances, in which order do the following events happen: "advance", "enemies defeated", "victory"?

This is some good thinking. To answer your question about order, I think "equivalent-time" events are determined by the order they appear in the scenario WML. That has been my experience so far. I would need to do some more experimenting with the victory event to be sure though.

[Viliam]

Here are my ideas to refactor the event system. Some things are changed, some are only renamed to make a naming consistent...

The event triggers can be roughly divided to two groups.
- "Game phase triggers" are when the game is moving to the next phase... next side, next turn, or next scenario.
- "Unit action triggers" are when a unit does something.

When events describe a start and end of something, they should be named consistently, e.g. "start X" and "end X". Such events should always be well nested, that is "start A, start B, end B, end A", but never "start A, start B, end A, end B".


These should be game phase events: "start scenario", "start turn", "start side", "end side", "end turn", "end scenario".

The "start scenario" is a former "prestart"; the "end scenario" is former "victory" and "defeat" (they can be distinguished by some WML attribute, e.g. "result=victory" and "result=defeat"). These are the first and last events in scenario.

The "start turn" and "end turn" replace the former "start", "new turn" and "time over" events (with a WML attribute "turn=1" or "turn=last").

The "start side" and "end side" replace the former "side turn" and "ai turn" (WML attribute "controller=ai", "controller=human"). Any other events (that is: unit-triggered events) can only happen between "start side" and "end side".

[Viliam]

This is some good thinking. To answer your question about order, I think "equivalent-time" events are determined by the order they appear in the scenario WML.

Thanks. Yes, this is a good way to design it... "equivalent-time" events should happen in order they are defined in script... unless there is a good design reason for deciding otherwise.


Example:

Capturing a village and seeing an enemy are equivalent events; both happen in the same move, and generally do not influence each other. So their order should be determined by order in script.

(More thinking... if the first of those events -- capturing a village, seeing the enemy -- teleports my unit away or kills it, then the second event probably should not happen. Because if it did happen, who would be the triggering unit?)


Example:

The "begin turn" should always happen before the "begin side" of the first side, because by design 'sides' happen inside of 'turns'.

[Viliam]

During those years WML has evolved from a simple "config file language" to a Turing complete event-driven programming language. As a config file language (and most of WML code is still doing this) it is a very elegant solution. As a programming language, I think some re-designing could be useful... and unfortunately it would come at a cost remaking maybe all existing scenarios. Let's not talk yet about whether such cost is justified. Let's just discuss what are the tasks which WML has to do now, and which will probably be requested in future; and what would be the elegant solution to them.

Things done by current WML can be divided to two groups: "definitions" and "actions". Definitions are lists of properties of some object. Actions are sequences of commands which must be executed in a given order. These two groups have somewhat different characteristics, which should be recognized (the current trend seems like implementing actions as definitions and ignoring the difference as much as possible).


Definitions describe things of some given type. In Wesnoth there are many types of things -- unit types, units, scenarios, campaigns, terrains, times of day, text domains, help items, fonts, hot keys... -- but it is a limited predefined set. Each set has its own fixed list of properties. You cannot (in WML, without touching the C++ code) create a new property or a new thing type.

The order of properties in not important... it does not matter whether e.g. Bat has "hitpoints 24 and movement 9" or "movement 9 and hitpoints 24". The order is only important in items of array-like properties, for example the order of death frames. Properties can be structured, but the tree structure of definition has a pre-known shape and limited depth (because e.g. unit type cannot recursively contain another unit type) -- the only exceptions are scenarios which include events (because inside event tag depth is unlimited). It would be relatively easy to make an generic editor of definitions, and it could be very intuitive and user-friendly.

The relations between definitions of different types are incosistent. Sometimes one definition references another by identifier -- a campaign contains identifier of first scenario, a unit type contains identifier of race and movement type. A reference to text domain is implemented by preprocessor. Sometimes one definition includes another, usually per macro -- a race includes traits, a scenario includes times of day. Scenario sides are included in a scenario directly, because they exist only in that scenario. I think that instead of abusing macros to include a frequently used definition, the WML language should reference it. This would reduce the memory consumed by expanded macros. (However, I have not measured the memory used by macros, so I am not sure whether it is a significant burden on computer resources or not.)


Actions are commands of a Turing-complete procedural language, inside of events. The order of commands is very important. They can include each other recursively (e.g. tag "while" can include another tag "while") and thus in theory can go to unlimited depth. The 'meaning' of actions is not only based on the values of their properties, but rather on the whole structure -- editing actions feels like programming, not like just filling values in a given template -- no graphic editor can remove from author the burden of designing an algorithm.

In my opinion, the especially clumsy part of WML are the expressions. It seems like the language tries to avoid them whenever possible (a few tags and attributes were designed for this purpose; e.g. "filter"), but cannot remove them completely, and it creates a few inconsistencies. For example to program something like "if X + Y > Z then display HELLO WORLD" you probably have to compute the expression to a temporary variable, use the "if" tag, and then clear the temporary variable; just because you cannot use a generic expression inside of "if". (Maybe I am wrong in this example... if yes, then please imagine a bit more difficult expression.)

I think we should acknowledge that WML need expressions, and support them conveniently. Separate actions from expressions clearly. The first steps toward this are tags "or", "and", "not", which evaluate to boolean values. Let's support other expressions in similar way, like this:

Code: Select all

[if]
  [test]
    [is_greater]
      [plus]
        [value]
          value=$x
        [/value]
        [value]
          value=$y
        [/value]
      [/plus]
      [value]
        value=$z
      [/value]
    [/is_greater]
  [/test]
  [then]
    [message]
      message= _"HELLO WORLD"
    [/message]
  [/then]
[/if]

In this example tags "is_greater" and "plus" return values. First they evaluate in given order all their subtags, then they perform a given operation and return a value ("plus" adds the values and returns the sum; "is_greater" returns true if the first value is greater than the second one). Attribute "value" returns the value. To make the language regular, any tag which contains values ("test", "is_greater", "plus", "and", "or", "not",...) can contain any tag which returns value ("value", "is_greater", "plus", "and", "or", "not",...). The tag "value" is long, but if we want more values in an expression. we cannot use attributes for this (a tag cannot have two attributes with the same name), so probably this one situation could be a macro... see this; the expression is short and legible, yet the system allows construction of complicated expressions:

Code: Select all

[if]
  [test]
    [is_greater]
      [plus]
        {VALUE $x}
        {VALUE $y}
      [/plus]
      {VALUE $z}
    [/is_greater]
  [/test]
  [then]
    [message]
      message= _"HELLO WORLD"
    [/message]
  [/then]
[/if]

[Viliam]

Currently event has three parts:
- an event type, such as "prestart", which says when program checks the events of this type
- an event trigger, such as "filter", which is compared with some circumstances provided by program... if it does not match, nothing happens... if it does match, the event is "triggered", and we go to:
- an event action, that is a few commands which happen after the event is triggered. If attribute "first_time_only" is set to true, the event cannot be triggered again; if it is set to false, the event is triggered repeatedly.

This is unnecessarily irregular, because what is and what is not a part of event "trigger" is an arbitrary decision. Things that are part of trigger have to be supported by special tags like "filter", "filter_second", "special_filter", "special_filter_second", and they contribute to feature creep. Things that are not part of trigger, must be implemented in a different way. Two ways to do the same thing; see the following examples:


1) When your White Mage steps on a magic rune (some specific location), the magic gate opens.

Code: Select all

[event]
  name=moveto
  first_time_only=yes
  [filter]
    side=1
    type=White Mage
    x,y=50,50
  [/filter]
  [then]
    # ...open the magic gate...
  [/then]
[/event]

2) When both your White Mages stand on two magic runes (some specific locations) at the same time, the magic gate opens.

Now this is more complicated. The action obviously happens in a "moveto" event, but the problem is that we cannot check in trigger whether the second mage is on his place too. So we have to trigger the event repeatedly... and we must check manually whether the gate was already open (via setting the variable, or checking for the magic gate on the map).

Code: Select all

[event]
  name=moveto
  first_time_only=no
  [if]
    [have_unit]
      side=1
      type=White Mage
      x,y=50,50
    [/have_unit]
    [have_unit]
      side=1
      type=White Mage
      x,y=70,70
    [/have_unit]
    # ...if the magic gate was not opened yet...
    [then]
      # ...open the magic gate...
    [/then]
  [/if]
[/event]

We see that the trigger mechanism does not work in non-trivial situations, and there are two different ways to make events. Let's unify it. Why do we need triggers anyway?

First, triggers cooperate with attrribute "first_time_only". This can be solved e.g. by command "clear_event" in event action; when it is executed, the current event is un-scheduled.

Second, via "filter" and other tags we receive event parameters. The tag "filter" is very nice, because it allows to compare a lot of values in a very few lines of script. However the primary unit and secondary unit should be rather passed by temporary variables ($primary, $secondary -- or should the names reflect the role, e.g. $moved, $recruited, $advanced, $attacker, $defender, $killed...?), and the "filter" tag should have an attribute to specify which unit it compares.

So these are the examples:

1)

Code: Select all

[event]
  name=moveto
  [if]
    [filter]
      what=$moved
      side=1
      type=White Mage
      x,y=50,50
    [/filter]
    [then]
      # ...open the magic gate...
      [clear_event]
      [/clear_event]
    [/then]
  [/if]
[/event]

2)

Code: Select all

[event]
  name=moveto
  [if]
    [have_unit]
      side=1
      type=White Mage
      x,y=50,50
    [/have_unit]
    [have_unit]
      side=1
      type=White Mage
      x,y=70,70
    [/have_unit]
    [then]
      # ...open the magic gate...
      [clear_event]
      [/clear_event]
    [/then]
  [/if]
[/event]

Actually, when I started writing this, I thought that the advantages of the second version will be more obvious. Now... well, if someone sees them, please write. I see only minor advantages... having only "filter" tag instead of "filter" and "filter_second"; replacing the unreliable concept of triggering and "first_time_only" attribute with reliable "clear_event" tag.

[Viliam]

Well, now I am going to post perhaps the most controversial suggestion to WML language... and then I will wait for your feedback and look forward to a discussion. Assuming that if no one would want to discuss these suggestions, no one is probably going to implement them either.


Currently WML variables can contain the following types of values:
- strings
- arrays
- associative arrays (sometimes called "hashes")
And also arrays of arrays, etc. There are no special "integer" or "boolean" variables; all are implemented as strings. Instead of declaring that "this variable contains a number" we just treat it as a number. That's why a "variable" tag has different attributes "equals" and "numerical_equals"... for example string "6" is not equal to "06", but it is numerically equal.

I think it would in a long run really simplify things if WML also had objects. Well, maybe "handles" or "references" is a better name (but it does not sound so cool).


There are many WML commands, whose purpose is to read and change some values of game objects. For example we want to know how many hitpoints a specific unit has, and then we want to reduce it to half. Or we want to know the location of a unit, and then we make the unit poisoned. Units are used most often, but we also sometimes want to know how many gold a given side has, etc. Speaking in an OO language, we want to read and write properties of objects. And how do we do it now?

By specific commands. Command "capture_village" changes the owner of a village. Commands "allow_recruit" and "disallow_recruit" modify list of units a side can recruit. Command "gold" gives gold to a side. Commands "hide_unit" and "unhide_unit" set visibility of a unit. Commands "unit_overlay" and "remove_unit_overlay" set an overlay image on a unit. Command "role" gives unit a role by which the unit can be found later. Command "terrain" changes a terrain of the hex. Command "modify_side" changes a few properties of a side.

Converting objects to arrays so that we can use them in WML. We "store_unit" to an associative array, read information from that array, eventually modify that array and "unstore_unit". We also can "store_starting_location", "store_locations" and "store_gold".

Avoiding to deal with objects directly, and filtering them instead. Commands "have_unit", "filter" check for units with given properties and return information if they exist.


WML should support putting a game object to a variable. What does it mean? When I put for example a unit into variable $u, the WML interpreter (a) must remember that variable $u has an object type "unit", and (b) each unit in game must have some unique identifier, and the variable $u must be connected with an identifier of that unit. (So technically a variable containing object is a "type + identifier" pair; though such technicallity should be hidden from script author.) So program is able to recognize that $u points to that unit... and this connection must be serializable; it must survive saving and loading a game.

For script author these variables look like associative arrays. (But they are not!) We can ask how much is "$u.hitpoints", or assign to "$u.hitpoints" a different value. Reading or writing a property of object invokes some C++ method to handle it.

This would make a few WML tags obsolete, it would open a way to add even more scripting functionality without making WML more difficult, and it would not be necessary to export and import objects to arrays. Compare this:

a) Converting to arrays: (UtBS-style damage)

Code: Select all

[store_unit]
  [filter]
    x,y=50,50
  [/filter]
  variable=u
[/store_unit]
[if]
  [variable]
    name=u.hitpoints
    greater_than=5
  [/variable]
  [then]
    [set_variable]
      name=u.hitpoints
      add=-5
    [/set_variable]
  [/then]
  [else]
    [set_variable]
      name=u.hitpoints
      value=1
    [/set_variable]
  [/else]
[/if]
[unstore_unit]
  variable=u
[/unstore_unit]
[clear_variable]
  name=u
[clear_variable]

b) Using objects:

Code: Select all

[get_unit]
  [filter]
    x,y=50,50
  [/filter]
  variable=u
[/get_unit]
[if]
  [variable]
    name=u.hitpoints
    greater_than=5
  [/variable]
  [then]
    [set_variable]
      name=u.hitpoints
      add=-5
    [/set_variable]
  [/then]
  [else]
    [set_variable]
      name=u.hitpoints
      value=1
    [/set_variable]
  [/else]
[/if]
[clear_variable]
  name=u
[clear_variable]

Actually, there is almost no difference.

Well, somehow my examples are not very convincing today... At least imagine the following: the properties of objects can be other objects. For example object "unit" can have property "location", which is hex where the unit stands. The object "location" has properties "x", "y", "terrain" and also "unit"... the last one is the unit which stands there, or null. Object "unit" also has property "unittype", which has "advanceto" etc. By having such unified system, script author could explore many different properties -- now it could be implemented by new WML commands like "store_unittype", "store_movetype", etc. Of course the object properties would have to be implemented anyway... but they would not be added to WML command namespace.

OK, I really wait for your opinions, hoping that someone will understand my desire for a more "regular" system, and perhaps push these ideas further.

[scott]

Using objects you could skip the "get variable" step.

Code: Select all

[set_variable]
game.campaign.scenario.units.Nym.hitpoints
add=-5
[/set_variable]

Or, make a sun damage method:

Code: Select all

[unit]

.... normal unit keys ...

[method]
id=sun_damage
     [set_variable]
     $me.hitpoints
     add=-5
     [/set_variable]
[/method]
[/unit]

In the scenario:

Code: Select all

[execute_method]
game.campaign.scenario.units.Nym.sun_damage
[/execute_method]

[theCAS]

Since WML is becoming a programming language we might use a more compact and orthodox syntax.

Example, ability get gold for every unit killed based on level:

Code: Select all

[EVENT type='onDie']
  [IF]killer.weapon.ability=='CorpseRobber'[/IF]
  [THEN]killer.side.gold+=killed.level[/THEN]
  [DOEVENT /]
[/EVENT]

Note the use of attributes.
DOEVENT means that the unit is actually killed, this way we can emulate onBeforeEvent and onAfterEvent.

Another example, Stone of Resurrection, if a unit is killed its hitpoints are restored to a half once

Code: Select all

[EVENT type='onDie']
  [IF]killed.StoneOfResurrection==1[/IF]
  [THEN]
    killed.hitpoints=killed.maxhitpoints/2
    killed.StoneOfResurrection=NULL
  [/THEN]
[/EVENT]

Note the absence of DOEVENT: the unit is not killed.
Variables can be assigned to objects as StoneOfResurrection and can be deleted assigning NULL.

[Dacyn]

Note the absence of DOEVENT: the unit is not killed.

Don't you need

Code: Select all

[event type='onDie']
  [if]killed.StoneOfResurrection==1[/if]
  [then]
    killed.hitpoints=killed.maxhitpoints/2
    killed.StoneOfResurrection=NULL
  [/then]
  [else]
  [do_event /]
  [/else]
[/event]

then?
(BTW, why does [/if] occur before [then]?)

[turin]

Since WML is becoming a programming language we might use a more compact and orthodox syntax.

I wouldn't particularly have a problem with writing scenarios in a more compact language, but if we do that we might as well go ahead and use python. The purpose of using our own language is ease-of-use - we don't have to redefine everything, and it is easy to learn. What you suggest sounds like we would have to rewrite every single scenario, in which case I ask, why not just use an already-existing language?

And if there is a reason, we should still call it WML1.0 and WML2.0 - and have preprocessors for both of them. So that we don't have to rewrite everything.

[theCAS]

Don't you need

Code: Select all

[event type='onDie']
  [if]killed.StoneOfResurrection==1[/if]
  [then]
    killed.hitpoints=killed.maxhitpoints/2
    killed.StoneOfResurrection=NULL
  [/then]
  [else]
  [do_event /]
  [/else]
[/event]

then?
(BTW, why does [/if] occur before [then]?)

Of course, you are right, my mistake(s).

[theCAS]

What you suggest sounds like we would have to rewrite every single scenario, in which case I ask, why not just use an already-existing language?

Well, I've always supported the use of an existing language instead of WML but there isn't a general consensus.

If we decide to do that we could simply redefine event functions in scenario configuration and create an intuitive object model.

For example:

Code: Select all

Events:
function onTurn(side,dayTime)
function onAttack(attacker,defender)
function onLevelUp(unit)
function onStrike(attacker,attacked,isHit)
function onDie(killer,killed)
function onMove(unit,from,to)
function onCapture(unit,village)
function onRecruit(unit)
function onRecall(unit)
function onScenarioEnd(isVictory)

Objects:
side.name
side.units[array(unit)]
side.recruitable[array(unit)]
side.recallable[array(unit)]
side.decesead[array(unit)]
side.gold
side.villages[array(village)]

unit.name
unit.level
unit.side[side]
unit.abilities[Array(ability)]
unit.allignment
unit.traits[array(trait)]
unit.canRecruit
unit.mustSurvive
unit.type
unit.moveType
unit.experience
unit.maxExperience
unit.hitpoints
unit.maxHitpoints
unit.weapons[array(weapon)]
unit.hex[hex]
unit.moves
unit.maxMoves
unit.isPoisoned
unit.isStoned
unit.isSlowed

weapon.name
weapon.type
weapon.specials[array]
weapon.damage
weapon.attacks

village.name
village.goldPerTurn
village.side[side]
village.hex[hex]

hex.label
hex.x
hex.y
hex.type
hex.dayTime[dayTime]

scenario.name
scenario.sides[side]
scenario.dayTimeCicle[array(dayTime)]
scenario.turn
scenario.maxTurns

dayTime.name
dayTime.modifier

Throw in some globals, an object to control AI and you can probably write every campaign possible.

How is that for simple? 10 events, 7 objects, all with self-explaining names and properties.

[tsr]

I would love this.

I can even say that I would gladly help out with this project, allthough I cant read C++ code or python code (but I am somewhat of a coder, so it should be fairly easy to catch up ).

(One important thing though is to keep the WML (possibly adjusted to conform to the above python(?)-code as to allow non-programmers (and people that don't want/are afraid of learning a programming language)

/tsr

[toms]

IMO WML should _not_ become a programming language. It was originally made to be able to write scenarios in a simple way.
Sometimes WML is not enough though, that's why I suggested this.
Just my 2¢.