very slow wesnoth_addon_manager
Moderator: Forum Moderators
Forum rules
Before reporting issues in this section, you must read the following topic:
Before reporting issues in this section, you must read the following topic:
Re: very slow wesnoth_addon_manager
Looks like this was not updated to python 3 yet. Replace
file(name, "w").write(mythread.data)
with with open(name, "wb") as f: f.write(mythread.data)
"If gameplay requires it, they can be made to live on Venus." -- scott
Re: very slow wesnoth_addon_manager
Thank you Soliton, that fixed --raw_download.
I tried doing the unpacking with --unpack and it threw a similar error.
I tried doing the unpacking with --unpack and it threw a similar error.
Code: Select all
$ ../tools/wesnoth_addon_manager -U Wild_Frontiers -c add-ons
Opening socket to add-ons.wesnoth.org:15014 for 1.14.x
Connected as 42.
Traceback (most recent call last):
File "../tools/wesnoth_addon_manager", line 291, in <module>
data = file(args.unpack).read()
NameError: name 'file' is not defined
Closing socket.
- Elvish_Hunter
- Posts: 1575
- Joined: September 4th, 2009, 2:39 pm
- Location: Lintanir Forest...
Re: very slow wesnoth_addon_manager
Good catch. I just pushed a fix for these errors both in master (c72ea20) and 1.14 (fab37ca979a1c9625eace6e25b9cb60bddf70a36).
Current maintainer of these add-ons, all on 1.16:
The Sojournings of Grog, Children of Dragons, A Rough Life, Wesnoth Lua Pack, The White Troll (co-author)
The Sojournings of Grog, Children of Dragons, A Rough Life, Wesnoth Lua Pack, The White Troll (co-author)
Re: very slow wesnoth_addon_manager
Actually, upon further investigation, I think that the problem is that the parser uses
bytes
objects everywhere, which causes the parse time to increase quadratically with respect to add-on size. Using bytearray
instead would probably improve performance for large add-ons.Re: very slow wesnoth_addon_manager
Is it as simple as find/replace bytes into bytearray or is it more than that?gnombat wrote: ↑April 13th, 2021, 1:38 am Actually, upon further investigation, I think that the problem is that the parser usesbytes
objects everywhere, which causes the parse time to increase quadratically with respect to add-on size. Usingbytearray
instead would probably improve performance for large add-ons.
Because I can try doing that and report back the results if it is so.
- Elvish_Hunter
- Posts: 1575
- Joined: September 4th, 2009, 2:39 pm
- Location: Lintanir Forest...
Re: very slow wesnoth_addon_manager
There's more. For example, to create an empty bytearray you can't just use
Other than that, all methods and operators supported by bytes seem to be supported by bytearrays as well, so you can give it a try and let us know.
b""
, but you have to use bytearray()
(and there are other differences when constructing such objects). You can create a bytearray from bytes with bytearray(b"My sequence of bytes")
. Bytearrays also support mutable object methods, like the ones available for lists, which bytes object don't support. Some functions expect bytes objects as arguments, so you'll have to cast bytearrays to bytes with bytes(my_bytearray)
.Other than that, all methods and operators supported by bytes seem to be supported by bytearrays as well, so you can give it a try and let us know.
Current maintainer of these add-ons, all on 1.16:
The Sojournings of Grog, Children of Dragons, A Rough Life, Wesnoth Lua Pack, The White Troll (co-author)
The Sojournings of Grog, Children of Dragons, A Rough Life, Wesnoth Lua Pack, The White Troll (co-author)
Re: very slow wesnoth_addon_manager
I'm not very familiar with programming in Python, so I'm not sure exactly what needs to be changed, but the basic idea is to do something like this:
Note that this patch is not quite correct, because
Code: Select all
--- data/tools/wesnoth/wmlparser3.py
+++ data/tools/wesnoth/wmlparser3.py
@@ -448,7 +448,7 @@ class Parser:
self.in_arrows = False
self.temp_string += line[:arrows]
self.temp_string_node = StringNode(self.temp_string)
- self.temp_string = b""
+ self.temp_string = bytearray()
self.temp_key_nodes[self.commas].value.append(
self.temp_string_node)
self.in_arrows = False
@@ -477,7 +477,7 @@ class Parser:
if self.translatable:
self.temp_string_node.textdomain = self.textdomain
self.translatable = False
- self.temp_string = b""
+ self.temp_string = bytearray()
if not self.temp_key_nodes:
raise WMLError(self, "Unexpected string value.")
@@ -587,7 +587,7 @@ class Parser:
for subsegment in segment.split(b",", maxsplit):
self.temp_string += subsegment.strip()
self.temp_string_node = StringNode(self.temp_string)
- self.temp_string = b""
+ self.temp_string = bytearray()
self.temp_key_nodes[self.commas].value.append(
self.temp_string_node)
if self.commas < n - 1:
@@ -605,7 +605,7 @@ class Parser:
"""
# parsing state
- self.temp_string = b""
+ self.temp_string = bytearray()
self.temp_string_node = None
self.commas = 0
self.temp_key_nodes = []
bytearray
is not the same type as bytes
(in particular it means that the type signature for the StringNode
class is not right any more). However, it does appear to work at least with the wesnoth_addon_manager
program.