Author Topic: What language is faster execute than QB64 ?  (Read 10231 times)

0 Members and 1 Guest are viewing this topic.

Offline jack

  • Seasoned Forum Regular
  • Posts: 408
    • View Profile
Re: What language is faster execute than QB64 ?
« Reply #30 on: March 25, 2019, 10:14:11 pm »
c_compiler\bin\g++ -O2 -s -Wfatal-errors -w -Wall qbx.cpp -lws2_32 -lwinspool parts\core\os\win\src.a -lopengl32 -lglu32 -lwinmm -lgdi32 -mwindows -static-libgcc -static-libstdc++ -D GLEW_STATIC -D FREEGLUT_STATIC -lksguid -lole32 -lwinmm -ldxguid -o ..\..\

Offline MLambert

  • Forum Regular
  • Posts: 115
    • View Profile
Re: What language is faster execute than QB64 ?
« Reply #31 on: March 25, 2019, 11:53:58 pm »
Thank you Jack

Offline MLambert

  • Forum Regular
  • Posts: 115
    • View Profile
Re: What language is faster execute than QB64 ?
« Reply #32 on: April 06, 2019, 07:11:38 pm »
Thank you ... I will investigate this.

Mike

Offline Raven_Singularity

  • Forum Regular
  • Posts: 158
    • View Profile
Re: What language is faster execute than QB64 ?
« Reply #33 on: April 08, 2019, 04:24:24 pm »
If you have 20 cores to work with, your bottleneck will be using your CPU.

If you already have single-core code worked out in QB64, I'd suggest you try to find a way to parallellise it, so it can be run with a command line parameter, then run 30-40 instances at once to max out your CPU load to full.  If you're processing directories, that's a simple way to pass the parameter to your QB64 app.  You can use the command line tool xargs to specify how many of the directories to pass to your QB64 app at once, it's handy for quick parallellisation tasks.

What is your code actually doing?  Something that can be broken down into a bunch of pieces, or does later code rely on earlier code?

I'm not sure if QB64 supports calling an app in the background, but it should be doable by adding a "&" after the command to place it as a background job on GNU+Linux, possibly doing it as ( mycommand & ) to force it into a subshell.  If you're on Windows, I think the "START" command can launch an app in the background, so it detaches from your app.  Then you just need your app to monitor for the sub-processes to finish.  An easy way to do this is to use filenames like SUB_00001.TMP while it is processing a data set, and when it completes rename it to SUB_00001.TXT or similar.  Then your app just waits for .TXT files to appear, then does something with the results.

Offline codeguy

  • Forum Regular
  • Posts: 174
    • View Profile
Re: What language is faster execute than QB64 ?
« Reply #34 on: April 10, 2019, 02:35:49 am »
You can use SHELL _DON'TWAIT to launch multiple simultaneous instances. Been there.

Offline Raven_Singularity

  • Forum Regular
  • Posts: 158
    • View Profile
Re: What language is faster execute than QB64 ?
« Reply #35 on: April 10, 2019, 01:00:08 pm »
You can use SHELL _DON'TWAIT to launch multiple simultaneous instances. Been there.

Oh, nice, QB64 added that by itself.  Hopefully that command is fully cross-platform compatible.

Offline MLambert

  • Forum Regular
  • Posts: 115
    • View Profile
Re: What language is faster execute than QB64 ?
« Reply #36 on: April 23, 2019, 09:18:39 pm »
Thank you for the input. I have started to run parallel processes and this definitely does work faster.

Mike

Offline Raven_Singularity

  • Forum Regular
  • Posts: 158
    • View Profile
Re: What language is faster execute than QB64 ?
« Reply #37 on: April 24, 2019, 10:38:39 am »
What sort of time decrease/performance gain are you seeing?

Just to be clear, to max out a 20 core system you will need significantly more than 20 concurrent tasks.  I recommend starting at twice as many tasks as cores.  This is because each task will be doing things that make it wait (such as reading disk, waiting for database, reading directory entries, loading DLLs, etc.), which will reduce CPU usage below 100% of 1 core per task.  When paralellising tasks on my 4 core system, I usually needed 6-12 simultaneous tasks to maintain 100% usage of all cores.  Don't be afraid to try 100 or more concurrent tasks, just track the total processing time to see if it improves or slows down when adding tasks.

Offline MLambert

  • Forum Regular
  • Posts: 115
    • View Profile
Re: What language is faster execute than QB64 ?
« Reply #38 on: April 26, 2019, 08:21:59 am »
Hi Raven,

Thks for the input. I have broken the program into 41 different processes. Initially it took 16 hours to process 100,000,000 + records.

Introducing and extra 20 processes reduced it to 7 hours  and this is bearable for the time being. I am converting the code to use MySQL and re-writing parts to change the methodology from a 'reload' every time I add transactions to an 'update' situation.

On preliminary testing this has reduced it down to less than 60 seconds. It is not the introduction of MySQL but the concept of update and not reload or rebuild the database.

I will have to reduce this even further to be realtime.

This thread in the forum did help me.

Thks to all.

Mike