Allow more new api in stable releases

Discussion among members of the development team.

Moderator: Forum Moderators

gfgtdf
Developer
Posts: 1432
Joined: February 10th, 2013, 2:25 pm

Allow more new api in stable releases

Post by gfgtdf »

Currently we have a rule not to allow new api in stable releases (1.16).

This however has some major problems:

During development releases we simply don't know what people (umc authors) might need, so if we want to add something we have two options:

1) Add it and risk not only doing for for nothing, or even worse than that creating a new maintance burden forever for a feature that might never be used at all.

2) Don't add it and risk missing it later

Both options are really bad for us, and probably even more for umc authors whop then have to wait for yeas for a really simple feature that could make their campaign idea work and would probably give up on writing their addons out of frustration.


This would require adding a functionality to store multiple versions of addons on the addon server so that players with older wesnoth versions could then download the older version that doesn't require new features.




I really see no disadvantages. New api features esepcially of the simple of this type are usually much less likely to introduce bugs to the code than other patches, furthermore players with old version could still be able to play the old version of the addons, and if they want to download newer versions i think it's not asked too much to also download the new wesnoth version. Lastly the idea that old wesnoth versions can download addosn written for newer stable version (within a stable release) is also already just wrong, we constanty fix bugs that woudl make a lot um umc campaigsn unplayable.


So i'm making this thread to see if anyone has serious concerns about this.

Also as i said in the other thread, this might have a quite huge impact on what features, will be added in 1.15 in particular if we do this it will mean we will delay certain features until we have a usecase and if we don't we will probably put feature in without knowing whether someone actually needs them.
Scenario with Robots SP scenario (1.11/1.12), allows you to build your units with components, PYR No preperation turn 1.12 mp-mod that allows you to select your units immideately after the game begins.
User avatar
Pentarctagon
Project Manager
Posts: 5527
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: Allow more new api in stable releases

Post by Pentarctagon »

The main concerns I have are:
  • While there's no data on SP, for MP there's still about 1/3 of the player base that's not using a distribution method that supports automatic updates (SourceForge and Linux distros, essentially), so moving to a model that could cause problems for 1/3 of our MP players sounds not ideal.
  • This would also make us much more reliant on Steam, since the largest part of the player base is on Windows and on Windows the only distribution method that supports updating without redownloading the entire game is Steam.
  • I think there should be some attempt made to ask UMC creators which they'd prefer, though I also don't have a really good idea of how to do this besides a forum poll and then linking to it in the 1.14 MP instance motd.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
User avatar
Elvish_Hunter
Posts: 1575
Joined: September 4th, 2009, 2:39 pm
Location: Lintanir Forest...

Re: Allow more new api in stable releases

Post by Elvish_Hunter »

Pentarctagon wrote: July 18th, 2020, 6:38 pm While there's no data on SP, for MP there's still about 1/3 of the player base that's not using a distribution method that supports automatic updates (SourceForge and Linux distros, essentially), so moving to a model that could cause problems for 1/3 of our MP players sounds not ideal.
At least Linux users can still use Flatpaks or compile a clone of the Github repository. It's a pity that the unofficial PPA is dead...
Pentarctagon wrote: July 18th, 2020, 6:38 pm This would also make us much more reliant on Steam, since the largest part of the player base is on Windows and on Windows the only distribution method that supports updating without redownloading the entire game is Steam.
The main problem for Windows users without Steam is that they're forced to re-download the whole game each time we release an update. Wouldn't it be possible to also create smaller update-only installers?
The idea here is to create a smaller installer, or updater, which contains only the files that have been changed from the first version of the current stable release (1.14.0). This updater should try to run the wesnoth --version command, get the currently installed version and change only the necessary files to perform the update; since music and image files don't usually change and won't be included, this should make the updater quite small. Is it technically possible?
Pentarctagon wrote: July 18th, 2020, 6:38 pm I think there should be some attempt made to ask UMC creators which they'd prefer, though I also don't have a really good idea of how to do this besides a forum poll and then linking to it in the 1.14 MP instance motd.
A forum poll sounds good.
As a UMC maintainer (who should really restart working on his add-ons...), I don't have a strong preference either way, given that if I need to use a new feature I can always use an #ifver directive to block a campaign from running on versions that don't support it. But I can easily imagine users of an older version of Wesnoth being notified of an update of my add-on, downloading it and suddenly finding themselves blocked due to the #ifver directive... That'd be an unpleasant experience, unless the add-ons server is modified to keep old versions of add-ons and allow downgrading.
Current maintainer of these add-ons, all on 1.16:
The Sojournings of Grog, Children of Dragons, A Rough Life, Wesnoth Lua Pack, The White Troll (co-author)
User avatar
Pentarctagon
Project Manager
Posts: 5527
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: Allow more new api in stable releases

Post by Pentarctagon »

Elvish_Hunter wrote: July 18th, 2020, 9:45 pm
Pentarctagon wrote: July 18th, 2020, 6:38 pm While there's no data on SP, for MP there's still about 1/3 of the player base that's not using a distribution method that supports automatic updates (SourceForge and Linux distros, essentially), so moving to a model that could cause problems for 1/3 of our MP players sounds not ideal.
At least Linux users can still use Flatpaks or compile a clone of the Github repository. It's a pity that the unofficial PPA is dead...
Compiling wesnoth themselves is not a viable option for most users though, even on linux, so flatpak is definitely preferred if the user just hates Steam/Valve.
Elvish_Hunter wrote: July 18th, 2020, 9:45 pm
Pentarctagon wrote: July 18th, 2020, 6:38 pm This would also make us much more reliant on Steam, since the largest part of the player base is on Windows and on Windows the only distribution method that supports updating without redownloading the entire game is Steam.
The main problem for Windows users without Steam is that they're forced to re-download the whole game each time we release an update. Wouldn't it be possible to also create smaller update-only installers?
The idea here is to create a smaller installer, or updater, which contains only the files that have been changed from the first version of the current stable release (1.14.0). This updater should try to run the wesnoth --version command, get the currently installed version and change only the necessary files to perform the update; since music and image files don't usually change and won't be included, this should make the updater quite small. Is it technically possible?
I'm sure it's technically possible, but IMO it'd be a step backwards in the sense that it'd be one more custom thing that'd need to be maintained and updated ourselves even though most other distribution methods do handle the updating process already.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
User avatar
doofus-01
Art Director
Posts: 4122
Joined: January 6th, 2008, 9:27 pm
Location: USA

Re: Allow more new api in stable releases

Post by doofus-01 »

As someone who was introduced to Wesnoth on a non-Ubuntu Linux distribution (so google-searches for fixing/upgrading were often not totally helpful), also not directly connected to the internet at the time, I'd be violently opposed to this. However, that was a long time ago, and I have no idea how many people would be in a similar boat these days. I do understand the dilemma gfgtdf is addressing.

That said, let's keep in mind we aren't competing with modern games, and we shouldn't think that what's easily measured is necessarily our only audience.
gfgtdf wrote: July 18th, 2020, 5:41 pm Also as i said in the other thread, this might have a quite huge impact on what features, will be added in 1.15 in particular if we do this it will mean we will delay certain features until we have a usecase and if we don't we will probably put feature in without knowing whether someone actually needs them.
What sort of features are you envisioning? Could most things be addressed by Lua script "official add-ons", that are checked at some point without having to specifically start the add-on manager?
BfW 1.12 supported, but active development only for BfW 1.13/1.14: Bad Moon Rising | Trinity | Archaic Era |
| Abandoned: Tales of the Setting Sun
GitHub link for these projects
User avatar
Pentarctagon
Project Manager
Posts: 5527
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: Allow more new api in stable releases

Post by Pentarctagon »

Non-Ubuntu distros would be more likely to support Flatpak out of the box, though offline installing would still be an issue (I'm not sure how regular linux installation methods would address that either).
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
User avatar
doofus-01
Art Director
Posts: 4122
Joined: January 6th, 2008, 9:27 pm
Location: USA

Re: Allow more new api in stable releases

Post by doofus-01 »

Pentarctagon wrote: July 19th, 2020, 3:41 am Non-Ubuntu distros would be more likely to support Flatpak out of the box, though offline installing would still be an issue (I'm not sure how regular linux installation methods would address that either).
Sometimes I could download RPMs to a thumbdrive, but eventually I had to download source tarballs. Either way, upgrading was a PITA and was a concerted effort, often over multiple days.
BfW 1.12 supported, but active development only for BfW 1.13/1.14: Bad Moon Rising | Trinity | Archaic Era |
| Abandoned: Tales of the Setting Sun
GitHub link for these projects
gfgtdf
Developer
Posts: 1432
Joined: February 10th, 2013, 2:25 pm

Re: Allow more new api in stable releases

Post by gfgtdf »

Pentarctagon wrote: July 18th, 2020, 6:38 pm The main concerns I have are:
  • While there's no data on SP, for MP there's still about 1/3 of the player base that's not using a distribution method that supports automatic updates (SourceForge and Linux distros, essentially), so moving to a model that could cause problems for 1/3 of our MP players sounds not ideal.
  • This would also make us much more reliant on Steam, since the largest part of the player base is on Windows and on Windows the only distribution method that supports updating without redownloading the entire game is Steam.
What problems would it exactly cause for them ?

Addons that don't need new featutes will still work on older wesnoth version, and addons that use the new features probably wouldn't exist on stable in the first place. For addons that added here new features later you people could still use the older version. Furthermore addon authors do care about who can use their addons, they can decide whether and when they will use these, some will probably decide not to use them, some will wait a few month until the new version is deployed enough, and some people will use then right away. And i think this is totally fine to give addon authors this decision (whether and when to use new features which version to support).

Furthermore it's not like we will make feature releases on a monthly basis, in particular considering how few feature requests we have recently.
doofus-01 wrote: July 19th, 2020, 3:27 am As someone who was introduced to Wesnoth on a non-Ubuntu Linux distribution (so google-searches for fixing/upgrading were often not totally helpful), also not directly connected to the internet at the time, I'd be violently opposed to this. However, that was a long time ago, and I have no idea how many people would be in a similar boat these days. I do understand the dilemma gfgtdf is addressing.

That said, let's keep in mind we aren't competing with modern games, and we shouldn't think that what's easily measured is necessarily our only audience.
Same reasoning as above, I think you are greatly overestimating the problems it would cause for people with older wesnoth versions.
doofus-01 wrote: July 19th, 2020, 3:27 am What sort of features are you envisioning? Could most things be addressed by Lua script "official add-ons", that are checked at some point without having to specifically start the add-on manager?
There are quite some features where its really hard to decide on their final form without someone actually requireing them, for example:
https://github.com/wesnoth/wesnoth/pull/4856

But most type of feature i had i'm mind are for example adding a new widget for wesnoth.show_dialog or basicially whatever pop up that add-on authors might need.
Pentarctagon wrote: July 19th, 2020, 3:41 am Non-Ubuntu distros would be more likely to support Flatpak out of the box, though offline installing would still be an issue (I'm not sure how regular linux installation methods would address that either).


I'm not really sure what you mean by 'offline installing' but shouldn't this be totally irrelvant for people without internet connection since they won't be able to install addons anyways ?
Scenario with Robots SP scenario (1.11/1.12), allows you to build your units with components, PYR No preperation turn 1.12 mp-mod that allows you to select your units immideately after the game begins.
User avatar
Pentarctagon
Project Manager
Posts: 5527
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: Allow more new api in stable releases

Post by Pentarctagon »

gfgtdf wrote: July 19th, 2020, 2:30 pm
Pentarctagon wrote: July 18th, 2020, 6:38 pm The main concerns I have are:
  • While there's no data on SP, for MP there's still about 1/3 of the player base that's not using a distribution method that supports automatic updates (SourceForge and Linux distros, essentially), so moving to a model that could cause problems for 1/3 of our MP players sounds not ideal.
  • This would also make us much more reliant on Steam, since the largest part of the player base is on Windows and on Windows the only distribution method that supports updating without redownloading the entire game is Steam.
What problems would it exactly cause for them ?

Addons that don't need new featutes will still work on older wesnoth version, and addons that use the new features probably wouldn't exist on stable in the first place. For addons that added here new features later you people could still use the older version. Furthermore addon authors do care about who can use their addons, they can decide whether and when they will use these, some will probably decide not to use them, some will wait a few month until the new version is deployed enough, and some people will use then right away. And i think this is totally fine to give addon authors this decision (whether and when to use new features which version to support).

Furthermore it's not like we will make feature releases on a monthly basis, in particular considering how few feature requests we have recently.
For the part "Addons that don't need new featutes will still work on older wesnoth version, and addons that use the new features probably wouldn't exist on stable in the first place." - is that what you meant to write? If add-ons that use newly added features wouldn't exist on stable in the first place, then there wouldn't be a purpose to allowing adding new API features to stable, would there?

Though I suppose ultimately it does come back to getting some sense of whether UMC authors would use new features if they were added to stable. If the majority of UMC authors would find it useful, then that's a strong argument for allowing it, whereas if the opposite is true then everything else is moot - breaking forwardbackward compatibility isn't a fair trade-off if UMC authors wouldn't actually use the new features added to stable.
gfgtdf wrote: July 19th, 2020, 2:30 pm
Pentarctagon wrote: July 19th, 2020, 3:41 am Non-Ubuntu distros would be more likely to support Flatpak out of the box, though offline installing would still be an issue (I'm not sure how regular linux installation methods would address that either).
I'm not really sure what you mean by 'offline installing' but shouldn't this be totally irrelvant for people without internet connection since they won't be able to install addons anyways ?
If they did offline installation of the base game, it stands to reason that they could also do offline installation of add-ons as well. Though I suppose in that case they'd also have the previous versions available to downgrade to.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
gfgtdf
Developer
Posts: 1432
Joined: February 10th, 2013, 2:25 pm

Re: Allow more new api in stable releases

Post by gfgtdf »

Pentarctagon wrote: July 19th, 2020, 4:45 pm For the part "Addons that don't need new featutes will still work on older wesnoth version, and addons that use the new features probably wouldn't exist on stable in the first place." - is that what you meant to write? If add-ons that use newly added features wouldn't exist on stable in the first place, then there wouldn't be a purpose to allowing adding new API features to stable, would there?

What? There sure would be purpose: To be able to put them on stable.

From my own experience as addon author. I often would write addons for dev version (scenario with robots (1.11), pyr.no-perperation.turn (1.13), and now the 1.15 version of WCII) , simply because there was no way to do what i wanted on stable. This however means that only small part of the player could play it and by the time the new stable come out and when it was available to the majority of players i already lost interest on the project also partly due to the lack of feedback.


If we would be less strict about new features (for no reason) things might have went differently.
Pentarctagon wrote: July 19th, 2020, 4:45 pm Though I suppose ultimately it does come back to getting some sense of whether UMC authors would use new features if they were added to stable. If the majority of UMC authors would find it useful, then that's a strong argument for allowing it, whereas if the opposite is true then everything else is moot - breaking forward compatibility isn't a fair trade-off if UMC authors wouldn't actually use the new features added to stable.

Again: WHAT?

Exactly the opposite is true: add-ons who don't use new features would still work on older wesnoth versions as before. If noone uses the new features than nothing would change to the current situation since all addons would work in older wesnoth version the same way as the do now. THese features donb't break compatibility by just existing, you have to actually use them to make your code non-working on older wesnoth versions.

And i'm at this point really expecting only a few few people to be using new features since our api is in some playes already quite complete already. As i said before one example could be support for a new widget wensoth.show_dialog for someone who really needs it. Or a way to query the 'value' of an abiltiy if someone wants to implement an custom ability with a value.


To make it clear: noone ever suggested something that will make the majority of addons that are written using for example wesnoth 1.16.5 will no longer work with wesnoth 1.16.4 or earlier.
Scenario with Robots SP scenario (1.11/1.12), allows you to build your units with components, PYR No preperation turn 1.12 mp-mod that allows you to select your units immideately after the game begins.
User avatar
Pentarctagon
Project Manager
Posts: 5527
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: Allow more new api in stable releases

Post by Pentarctagon »

I guess I'm not really understanding what you're meaning by at the same time saying that it should be allowed because it will be useful but also won't cause many problems because not many people will use the new features. Those sound contradictory to me.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
gfgtdf
Developer
Posts: 1432
Joined: February 10th, 2013, 2:25 pm

Re: Allow more new api in stable releases

Post by gfgtdf »

Pentarctagon wrote: July 19th, 2020, 5:30 pm I guess I'm not really understanding what you're meaning by at the same time saying that it should be allowed because it will be useful but also won't cause many problems because not many people will use the new features. Those sound contradictory to me.

No it's not, it useful for people to use it (probaly few people) (and for us who design the api), while causing no problems to people who don't use it.

I'll say it again: addons that don't use the new api features work on all version of a stable releases, just as before. (except for bugs in older stable versions, also: just as before)
Scenario with Robots SP scenario (1.11/1.12), allows you to build your units with components, PYR No preperation turn 1.12 mp-mod that allows you to select your units immideately after the game begins.
User avatar
Pentarctagon
Project Manager
Posts: 5527
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: Allow more new api in stable releases

Post by Pentarctagon »

Thinking about it, it is also true that if allowing new APIs does end up causing more problems than expected, or entirely unexpected problems, then we can simply stop adding new APIs to stable at that point. ie: if new APIs are added between 1.16.0 and 1.16.4, and for whatever reason it causes problems, then we can just not add more new APIs with 1.16.5+.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
User avatar
Pentarctagon
Project Manager
Posts: 5527
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: Allow more new api in stable releases

Post by Pentarctagon »

I think I'm actually alright with this, in that case.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
User avatar
doofus-01
Art Director
Posts: 4122
Joined: January 6th, 2008, 9:27 pm
Location: USA

Re: Allow more new api in stable releases

Post by doofus-01 »

gfgtdf wrote: July 19th, 2020, 5:21 pm rom my own experience as addon author. I often would write addons for dev version (scenario with robots (1.11), pyr.no-perperation.turn (1.13), and now the 1.15 version of WCII) , simply because there was no way to do what i wanted on stable. This however means that only small part of the player could play it and by the time the new stable come out and when it was available to the majority of players i already lost interest on the project also partly due to the lack of feedback.
Yeah, for whatever problems this proposal might cause, it does address the fact that development versions can be a bit obscure and uninteresting for too many people.
BfW 1.12 supported, but active development only for BfW 1.13/1.14: Bad Moon Rising | Trinity | Archaic Era |
| Abandoned: Tales of the Setting Sun
GitHub link for these projects
Post Reply