while generating the classes: use all cpu cores/threads

[michabbb]

hi,
it´s more a question, maybe i am doing something wrong.
when u have to deal with ebay, you need time, a lot of time 😖

http://developer.ebay.com/webservices/latest/ebaysvc.wsdl

that WSDL is that huge, my dev env takes 50minutes to generate all the classes. ~2600 files.
my dev host has 8 threads, but - using the docker version - I can see that only one thread is at 100%.
so, maybe it´s a PHP thing, maybe it´s a docker thing, maybe both - I don´t know.

is there, right now, anything I can do to speed up the code generation process ?

thanks a lot,
micha

[mikaelcom]

50 minutes is indeed very long as the WSDL is very large. It contains many elements that are parsed by the WSDL parsers. As you mentionned, 2600 files is a lot and their content is determined based on the Soap Structs/Functions then from the WSDL elements as the native SoapClient class is limited.

It could be worth disabling the WSDL parsers that would require to compare the generated packages with and without the WSDL parsers to see the differences. It's currently not possible as the WSDL parsers are used to consolidate the structs and services classes.

In addition, PHP is limited to one thread.

[michabbb]

is there a chance u see a way to use multi-threading in your generator process ?
i mean, PHP may be limited if u work with a single process, but still you can do things in parallel with PHP,
but I don´t know if that is possible within your project.

[mikaelcom]

I mean the process is sequential as each task depends on the previous one except the files generation which is not the most expensive.

Do you have any multi-threading code sample?

[michabbb]

as you know, the php world offers a view methods using threads. the important question is: how to u parse the WSDL ? from top to bottom or by method and then find all objects, that are involved in that method ? in case 2nd... it could maybe an option to take look at every call separately and create the needed objects/classes/files in a separate thread by call. i am aware, that many objects are used in more than one call, so before creating the object/class/file you would have to check if that file doesn´t already exist. i mean... it´s just an idea.... in case of this monster ebay wsdl, it could have a huge impact.. because 50minutes on a "gaming-notebook" is terrible... 🙈 but sure... ebay´s wsdl is terrible, no question 😞

[mikaelcom]

In addition, it's one isolated terrible case, so I sincerely won't spend time on this in the meantime. As soon as you generated it, you should not need to generated it again often, am I wrong?

[michabbb]

ebay doesn´t change it´s WSDL that often, so in general you are right. but I wanted to mention this whole "threading" thing so also other developers my have a huge advantage in case there is chance to speed things up. ebay may be a "exceptional case" here, but there are other companies out there (for sure) with large WSDL files, too. so whenever you have the feeling that you are doing a major update in this project, it would be nice if you thought of these problems and there might be an opportunity to do things in parallel.

[mikaelcom]

duely noted in #233 😉