Speed Comparison Tool

[SMcNeill]

A quickly expandable routine to test how fast our operators are against each other, for those cases when we want to optimize speed for our programs.  (Such as the EllipseFill topic STx posted.)

Code: QB64: [Select]

1. _TITLE "Expandable Speed Comparison Tool"
2.  
3. SCREEN _NEWIMAGE(800, 600, 32) 'nice size screen to display results
4.  
5. CONST LIMIT = 1 'time allowed for tests (in seconds)
6. RANDOMIZE TIMER
7.  
8.  
9. 'An expandable list of things to test
10. DATA "+","-","^","*","/","\","INT"
11. DATA "SQR","^.5"
12. DATA "x + x","x * 2"
13. DATA "DONE"
14.  
15. TYPE Results
16.     Int AS _FLOAT 'to store times with integers being tested
17.     Float AS _FLOAT 'to store times with floats being tested
18.     Index AS LONG 'to store the index of the operation, in case we want to sort the results at a later date as we generate more data
19. END TYPE
20.  
21. DIM TestString(1000) AS STRING 'to hold the string
22. DIM Times(1000) AS Results 'to hold the number of times we can run in a LIMIT
23.  
24. DO UNTIL t$ = "DONE" 'Read in the test results
25.     READ t$
26.     IF t$ <> "DONE" THEN count## = count## + 1: TestString(count##) = t$
27. LOOP
28.  
29. PRINT "Running tests.  Wait"; count## * 2; " seconds..."
30.  
31. counter## = 0
32. DO
33.     counter## = counter## + 1
34.     DEFLNG A-Z
35.  
36.     t# = TIMER
37.     DO UNTIL TIMER > t# + LIMIT
38.         junk = RND * 1000000 + 1 'just some random junk to test
39.         junk1 = RND * 1000000 + 1 'add one so we never get division by 0 glitches
40.         SELECT EVERYCASE TestString(counter##) 'We use everystring here so we don't skew results with fast exit CASE comparisons
41.             CASE "+": junk = junk + junk1
42.             CASE "-": junk = junk - junk1
43.             CASE "^": junk = junk ^ junk1
44.             CASE "*": junk = junk * junk1
45.             CASE "/": junk = junk / junk1
46.             CASE "\": junk = junk \ junk1
47.             CASE "INT": junk = INT(junk)
48.             CASE "SQR": junk = SQR(junk)
49.             CASE "^.5": junk = junk ^ 0.5
50.             CASE "x + x": junk = junk + junk
51.             CASE "x * 2": junk = junk * 2
52.         END SELECT
53.         Times(counter##).Int = Times(counter##).Int + 1
54.     LOOP
55.  
56.  
57.     _DEFINE A-Z AS _FLOAT
58.     t# = TIMER
59.  
60.     'Simply copy/paste a duplicate of the SELECT CASE from above to here for _FLOAT comparison speeds
61.  
62.     DO UNTIL TIMER > t# + LIMIT
63.         junk = RND * 1000000 + 1 'just some random junk to test
64.         junk1 = RND * 1000000 + 1 'add one so we never get division by 0 glitches
65.         SELECT EVERYCASE TestString(counter##) 'We use everystring here so we don't skew results with fast exit CASE comparisons
66.             CASE "+": junk = junk + junk1
67.             CASE "-": junk = junk - junk1
68.             CASE "^": junk = junk ^ junk1
69.             CASE "*": junk = junk * junk1
70.             CASE "/": junk = junk / junk1
71.             CASE "\": junk = junk \ junk1
72.             CASE "INT": junk = INT(junk)
73.             CASE "SQR": junk = SQR(junk)
74.             CASE "^.5": junk = junk ^ 0.5
75.             CASE "x + x": junk = junk + junk
76.             CASE "x * 2": junk = junk * 2
77.         END SELECT
78.         Times(counter##).Float = Times(counter##).Float + 1
79.     LOOP
80. LOOP UNTIL counter## = count##
81.  
82.  
83. FOR i = 1 TO count##
84.     PRINT "Testing speed for: "; TestString(i),
85.     PRINT "INTEGERS:"; Times(i).Int,
86.     PRINT "FLOATS:"; Times(i).Float
87. NEXT

Oddly enough, some of the things which I've always taken for granted now seem to be much different performace-wise than they used to be.  (For a quick example, run this under GL and then run it under the old SDL version, if you have it, and watch how drastically the values change.)

In SDL, addition is about 1.5x as fast as multiplication. (3.9 vs 2.5).  In the latest GL version of QB64, the two run at comparable speeds, with multiplication being a little faster than addition on my PC.

x + x no longer seems to be any more optimized than x * 2 is.  (The speed difference between the two is negligible, at less than 1% difference on my tests.)

x * x, however, still shows a remarkable improvement over x ^ 2.  (1.8M vs 0.5M)


Perhaps swapping out the old compiler to the newer compiler which is packaged with QB64 has changed the old rules which I remember so firmly for optimization; I'm not certain.  What *is* certain, just from these simple tests, is that I need to start spending more time and testing things more thoroughly to maximize speeds than I used to, for routines which require it.

Also notice that in some cases, speed depends on what type of data you're working with..

Integer Division is faster than Division when dealing with INTEGERS.  (Who would've guessed that?!)
Division is faster than Integer Division when dealing with _FLOATS.

(Of course, if you add in an additional INT() command to make the number an integer, then / probably isn't as fast as \ with _FLOATS...)


General rule to remember:  Higher numbers relate to faster operations.  (After all, it represents the number of times we can perform the operation in a minute.)


Play around with it.  Test it with SDL, GL, 32 and 64-bit versions of QB64.  See how the values compare on your own machines.   Add more test conditions to it and then share them here so we all can make use of them. 

And, most importantly, if you see something wrong with the logic/flow which you think makes the tests invalid, report that issue/concern here.  Personally, I think it'd be nice to have a quick little tool to use to see what would be the best operators to use when optimizing for speed as much as possible.  ;)

[_vince]

A lot of the operator speed differences only apply to old 16 bit processors on which QB45 was intended to run on which is why you see those conventions on old recycled code.  Modern processors will execute all operators from bitwise AND to shift to division at the same speed for a given data type.  QB64 doesn't seem to prioritize speed when translating to C++ but most modern C++ compilers will optimize the obvious operator substitutions in the final executable. It is generally thought of as bad practice to micro-optimize at the expense of code readability. Other than improving your algorithm and logical optimizations, it is almost impossible to optimize down to the cycle level these days since executed code will be heavily pipelined and cached on modern processors and you will see different results on different machines which may even vary due to time of day and dozen other factors such as cached data/code, processor load, multitasking, etc. Beyond an order of magnitude difference, it is worth ignoring slight time differences when doing time comparisons