Author Topic: Floating point math - a banality (sometimes sometimes doubts come back...)  (Read 7046 times)

0 Members and 1 Guest are viewing this topic.

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
The first out of tune note in many years...

Yes ... I always consult the manual (for decades, I was not born yesterday) and I also search among the answers of other users. I know that certain issues are necessarily trivial but I had never noticed this behavior of the variables. I mainly dedicated myself to data management and interfaces for their entry, and little to calculation. Now, coincidentally, I need some calculations and this thing comes up.

I insist: 1.49999... is different from 1.5 and I would like to decide when to round and how (I know how to do it)
In single precision I could understand. I don't understand becauset rounds itself into _FLOAT and _DOUBLE

I'm certainly missing something extremely simple because I won't be the first to notice this.
Can someone please tell me where am I wrong?

Code: QB64: [Select]
  1. a#=1234567.49999999
  2. ? a#
  3.  
  4. a#=1234567.499999999
  5. ? a#
  6.  
  7. a##=1234567.49999999
  8. ? a##
  9.  
  10. a##=1234567.499999999
  11. ? a##
  12.  
  13. ? STR$(a##)
  14.  
  15.  

If the system rounds by itself I can't do it and it creates some problems with the results



I am asking the question in another way: I need to treat the number 1.4999999999999999999
not like 1.5 but just for what it is 1.4 and a long series of "9"
I thought I had to do it with double precision numbers but obviously I was wrong.

Any suggestions?








15 digits is the maximum number you get to see no matter where decimal goes:
Code: QB64: [Select]
  1. a# = 1
  2. FOR i = 1 TO 17
  3.     bs$ = "." + STRING$(i, "9")
  4.     b# = VAL(bs$)
  5.     PRINT "bs string, b Double, a Double + b Double: "; bs$, b#, a# + b#
  6.     FOR j = 1 TO i
  7.         PRINT "Round a + b to "; j; "places: "; RoundDblX2DP$(a# + b#, j)
  8.     NEXT
  9.     PRINT: PRINT "Press any for next..."
  10.     SLEEP
  11.     CLS
  12.  
  13. FUNCTION RoundDblX2DP$ (x AS DOUBLE, DP AS INTEGER)
  14.     ' OK x <> .555...   .55 OK
  15.     'for DP = number of decimal places
  16.     DIM test$, p AS INTEGER, r AS DOUBLE
  17.     r = 5 * 10 ^ (-1 * (DP + 1))
  18.     'PRINT r      'that's it!
  19.     test$ = _TRIM$(STR$(INT((x + r) * 10 ^ DP) / 10 ^ DP))
  20.     p = INSTR(test$, ".")
  21.     IF p = 0 AND DP <> 0 THEN test$ = test$ + "." + STRING$(DP, "0")
  22.     IF p AND DP = 0 THEN test$ = LEFT$(test$, LEN(test$) - 1)
  23.     RoundDblX2DP$ = test$
  24.  
  25.  

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
Dang the freaking extra decimals popup again at 7 or 8 and then disappear:
Code: QB64: [Select]
  1. a# = 1
  2. FOR i = 1 TO 17
  3.     bs$ = "." + STRING$(i, "9")
  4.     b# = VAL(bs$)
  5.     PRINT "bs string, b Double, a Double + b Double: "; bs$, b#, b#
  6.     FOR j = 1 TO i
  7.         PRINT "Round a + b to "; j; "places: "; RoundDblX2DP$(b#, j)
  8.     NEXT
  9.     PRINT: PRINT "Press any for next..."
  10.     SLEEP
  11.     CLS
  12.  
  13. FUNCTION RoundDblX2DP$ (x AS DOUBLE, DP AS INTEGER)
  14.     ' OK x <> .555...   .55 OK
  15.     'for DP = number of decimal places
  16.     DIM test$, p AS INTEGER, r AS DOUBLE
  17.     r = 5 * 10 ^ (-1 * (DP + 1))
  18.     'PRINT r      'that's it!
  19.     test$ = _TRIM$(STR$(INT((x + r) * 10 ^ DP) / 10 ^ DP))
  20.     p = INSTR(test$, ".")
  21.     IF p = 0 AND DP <> 0 THEN test$ = test$ + "." + STRING$(DP, "0")
  22.     IF p AND DP = 0 THEN test$ = LEFT$(test$, LEN(test$) - 1)
  23.     RoundDblX2DP$ = test$
  24.  


It's at 7 decimal places here. Must be a letter E or D popping into the STR$(Double).

Update: no it's not the E or D thing, guess I need to trim the tail
« Last Edit: August 09, 2020, 04:23:08 pm by bplus »

Offline krovit

  • Forum Regular
  • Posts: 179
    • View Profile
Code: QB64: [Select]
  1. x1# = 10051.49999999999999999999999999999 ' :(
  2.  
  3. PRINT "Rounded to 5 places: "; RoundDblX2DP$(x1#, 5)
  4. PRINT "Rounded to 2 places: "; RoundDblX2DP$(x1#, 2)
  5. PRINT "Rounded to 0 places: "; RoundDblX2DP$(x1#, 0)
  6. ?
  7. PRINT "Rounded to Tens: "; RoundDblX2DP$(x1#, -1) 'just out of curiousity is this 0  in the 1's place ? yes!
  8. PRINT "Rounded to 100's: "; RoundDblX2DP$(x1#, -2)
  9.  
  10. FUNCTION RoundDblX2DP$ (x AS DOUBLE, DP AS INTEGER)
  11.     ' OK x <> .555...   .55 OK
  12.     'for DP = number of decimal places
  13.     DIM test$, p AS INTEGER, r AS DOUBLE
  14.     r = 5 * 10 ^ (-1 * (DP + 1))
  15.     'PRINT r      'that's it!
  16.     test$ = _TRIM$(STR$(INT((x + r) * 10 ^ DP) / 10 ^ DP))
  17.     p = INSTR(test$, ".")
  18.     IF p = 0 AND DP <> 0 THEN test$ = test$ + "." + STRING$(DP, "0")
  19.     IF p AND DP = 0 THEN test$ = LEFT$(test$, LEN(test$) - 1)
  20.     RoundDblX2DP$ = test$
  21.  
  22.  

The site reported by Fellippe probably tells us that the problem is without solution.
In some cases this is a big problem! For example if the discriminant is 1.49 (or 1.4999999) rather than 1.5 or 1.51.
See how the first three cases give an erroneous laughter.

In the end, a conversion into a string as it is would be enough, without automatic rounding (even sMcNeill has provided a starting point to operate in this sense, however impractical).
If the number could be turned into a literal string, then everything would be possible.
But maybe that's too much to ask, I don't know... I think only Fillippe could do something about it at the source code level.

but it's a shame ... I don't know the extent of this problem on some of my applications, but I think the impact is strong

Nothing is easy, especially when it appears simple (and nothing could be as dangerous as trying to do good to others)

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
Code: QB64: [Select]
  1. x1# = 10051.49999999999999999999999999999 ' :(
  2.  
  3. PRINT "Rounded to 5 places: "; RoundDblX2DP$(x1#, 5)
  4. PRINT "Rounded to 2 places: "; RoundDblX2DP$(x1#, 2)
  5. PRINT "Rounded to 0 places: "; RoundDblX2DP$(x1#, 0)
  6. ?
  7. PRINT "Rounded to Tens: "; RoundDblX2DP$(x1#, -1) 'just out of curiousity is this 0  in the 1's place ? yes!
  8. PRINT "Rounded to 100's: "; RoundDblX2DP$(x1#, -2)
  9.  
  10. FUNCTION RoundDblX2DP$ (x AS DOUBLE, DP AS INTEGER)
  11.     ' OK x <> .555...   .55 OK
  12.     'for DP = number of decimal places
  13.     DIM test$, p AS INTEGER, r AS DOUBLE
  14.     r = 5 * 10 ^ (-1 * (DP + 1))
  15.     'PRINT r      'that's it!
  16.     test$ = _TRIM$(STR$(INT((x + r) * 10 ^ DP) / 10 ^ DP))
  17.     p = INSTR(test$, ".")
  18.     IF p = 0 AND DP <> 0 THEN test$ = test$ + "." + STRING$(DP, "0")
  19.     IF p AND DP = 0 THEN test$ = LEFT$(test$, LEN(test$) - 1)
  20.     RoundDblX2DP$ = test$
  21.  
  22.  

The site reported by Fellippe probably tells us that the problem is without solution.
In some cases this is a big problem! For example if the discriminant is 1.49 (or 1.4999999) rather than 1.5 or 1.51.
See how the first three cases give an erroneous laughter.

In the end, a conversion into a string as it is would be enough, without automatic rounding (even sMcNeill has provided a starting point to operate in this sense, however impractical).
If the number could be turned into a literal string, then everything would be possible.
But maybe that's too much to ask, I don't know... I think only Fillippe could do something about it at the source code level.

but it's a shame ... I don't know the extent of this problem on some of my applications, but I think the impact is strong

You misunderstand how to use this function which only controls the display of numbers not the floating point math behind the scenes.

Only the first 14 to 16 digits are accurate in floating point math, the rest is always garbage.
So when you list more digits than a floating math memory address can hold of course it is going to chop off what it can't hold, it is generous that it does round at chop off point.

As applied to function, you asked for Rounded and for rounding to take place you need a cutoff of decimal places that is what DP parameter is for. Not any DP will work because of the 14-16 digit limit with Floating math.

If you test 15 digits or less the function rounds properly.

If you test more than 14-16 digits the floating point math will cut the number off at 14-16 digits (in base 10) because that is all the memory slot can hold.

Repeat: you only get 14-16 digits that will be treated in Floating point math.

As Steve said if you need higher precision than 14-16 digits then you have to write your own floating point math subs.

BTW I fixed the function to remove the extra decimal garbage creeping in, but again it will only work for 14-16 digits max, so the DP is limited to 14-16 minus the number of digits in the integer part of number.
Code: QB64: [Select]
  1. '_title "RoundDblX2DP$ test sub" b+ 2020-08-09
  2. ' 2020-08-09 revise function name as _TITLE
  3. ' trim excess tail if any and don't add decimal and 0's if not any
  4.  
  5. x1# = 10051.555555555555555555555555555 'Great!
  6. x1# = .3333333333333 'Great!
  7. x1# = .888888888888 'Great
  8. x1# = 5.55555555555 ' Fabulous!
  9. x1# = .444444444444 ' Marvelous!
  10. x1# = 73.737373737373 'Perfecto!
  11. x1# = .666666666666666 'good again!
  12. x1# = .54545454545 'Fine
  13. x1# = .4999999999 ' 10 places OK kept all the 9's
  14. x1# = .4555555 ' just lovely
  15. x1# = .5 'fine too though expecting 0's
  16. x1# = .55 'OK
  17. x1# = .655 'OK
  18. x1# = .6555 'OK
  19. x1# = .65555 'ok
  20. x1# = .655555 'ok
  21. x1# = 1.4999999999999990 'copied from forum
  22.  
  23. x1# = .555 'here it starts! 5 places good 2 places and more decimals sneek in  FIXED!
  24. x1# = .5555 'craps again  FIXED!
  25. x1# = .5555555 'WTH where are the decimals coming from? FIXED
  26.  
  27. 'try some lower and higher numbers
  28. x1# = 100000.000005 'no round?
  29. x1# = 100000.0000051 ' that should round
  30. x1# = 100000.00500001
  31. x1# = .005005005005
  32. x1# = .0505050505
  33.  
  34.  
  35. x1# = 10051.4999999999 ' 10 decimal place 15 digits total
  36.  
  37. 'comment the above line and try others
  38.  
  39.  
  40. PRINT "Rounded to 10 places: "; RoundDblX2DP$(x1#, 10)
  41. PRINT "Rounded to 5 places: "; RoundDblX2DP$(x1#, 5)
  42. PRINT "Rounded to 2 places: "; RoundDblX2DP$(x1#, 2)
  43. PRINT "Rounded to 0 places: "; RoundDblX2DP$(x1#, 0)
  44. PRINT "Rounded to Tens: "; RoundDblX2DP$(x1#, -1) 'just out of curiousity is this 0  in the 1's place ? yes!
  45. PRINT "Rounded to 100's: "; RoundDblX2DP$(x1#, -2)
  46.  
  47. FUNCTION RoundDblX2DP$ (x AS DOUBLE, DP AS LONG)
  48.     ' OK x <> .555...   .55 OK
  49.     'for DP = number of decimal places
  50.     DIM test$, p AS INTEGER, r AS DOUBLE
  51.     r = 5 * 10 ^ (-1 * (DP + 1))
  52.     'PRINT r      'that's it!
  53.     test$ = _TRIM$(STR$(INT((x + r) * 10 ^ DP) / 10 ^ DP))
  54.     p = INSTR(test$, ".")
  55.     IF p <> 0 THEN
  56.         IF DP = 0 THEN
  57.             test$ = LEFT$(test$, LEN(test$) - 1)
  58.         ELSE
  59.             IF DP > 0 THEN test$ = MID$(test$, 1, p) + MID$(test$, p + 1, DP)
  60.         END IF
  61.     END IF
  62.     RoundDblX2DP$ = test$
  63.  
  64.  

BTW the function is for displaying numbers, it doesn't work miracles with floating point math.





« Last Edit: August 10, 2020, 01:11:23 pm by bplus »

Offline krovit

  • Forum Regular
  • Posts: 179
    • View Profile
Unfortunately I don't speak English... so some communication problem is inevitable...

Meanwhile I noticed that the phrase of my previous post was translated as follows:
"See how the first three cases give an erroneous laughter."
Of course I meant "answer" and not "laughter".

I precise this to avoid thinking that I make good laughs by observing the behavior of QB64 (it would have been an unforgivable disrespect)... :)

Apart from that for the patience and examples you've provided :)

These kinds of problems that you highlighted to me were more or less known to me and I understand that it is a formal rounding because over 15 digits (15 digits before and after the comma) the system is not able to go (and all you see is garbage without numerical importance).

But the problem is strange - and it's not just our lovable and dear QB64 (lovely and dear... I'm serious!). Just the windows calculator to make calculations with numbers of more than 15 digits without any problem.

If the number comes from a calculation, the problem does not exist. Or rather, it exists but it is not relevant because I can certainly do without the accuracy over 15 digit. At worst, it can be 999 billion and three decimal places... you can stay there (for financial calculations and also for others in many cases).

The problem, however, is without a solution if the data is entered. In this case, if they are read from a file, they must already be strings to treat them. If they are numbers when you "touch" they are already out of our reach because they are automatically rounded by the system.

The same goes for a traditional INPUT: in addition to the 15 digit the result is automatically rounded.

You need a controlled and treated input to the end as a sequence of individual strings to handle the digits beyond the 15th place properly

All right... I take note of that... Though.... why does the windows calculator seem to do better?


Sorry... not if in English there is a way of saying similar to that Italian "drown in a glass of water"... to say that maybe I see a problem that doesn't exist and I struggle to come out of it.



If you don't want to answer again... understand! :)




Nothing is easy, especially when it appears simple (and nothing could be as dangerous as trying to do good to others)

Offline TempodiBasic

  • Forum Resident
  • Posts: 1792
    • View Profile
Hi Krovit
thanks to show me another way to say "Perdersi in un bicchiere d'acqua" !
I have no ever listen "Affogare in un bicchiere d'acqua". 
https://it.wikipedia.org/wiki/Affogare_in_un_bicchier_d%27acqua
There is always something to know or to understand! Good so the life is not boring.

About the issue of the fifteenth number after the point of decimal digit,
Do you think that Windows' Calculator is able to avoid the rounding of the number done by the OS (system)? And how can it do this task? Maybe it is repiclable.

Programming isn't difficult, only it's  consuming time and coffee

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
For the record, it's 14-16 digits total = digits left + digits right of the decimal.

Offline krovit

  • Forum Regular
  • Posts: 179
    • View Profile
Hi Krovit
thanks to show me another way to say "Perdersi in un bicchiere d'acqua" !
I have no ever listen "Affogare in un bicchiere d'acqua". 
https://it.wikipedia.org/wiki/Affogare_in_un_bicchier_d%27acqua
There is always something to know or to understand! Good so the life is not boring.

I'm not a kid anymore. but I still almost get emotional thinking about the diversity and wealth that can be grasped between peoples and different cultures.  And then to find out that, at the end of the day, we're all the same by staying different.



About the issue of the fifteenth number after the point of decimal digit,
Do you think that Windows' Calculator is able to avoid the rounding of the number done by the OS (system)? And how can it do this task? Maybe it is repiclable.


Code: QB64: [Select]
  1. x#=1.499999999999999999  '19 places (for example)
  2. ? x#  '--> 1.5
  3.  
  4. x#=1.499999999999999999  + .05
  5. ? x#  '--> 1.55
  6.  
  7.  

...and compares with the windows calculator: why doesn't it all seem so simple?

  [ You are not allowed to view this attachment ]    [ You are not allowed to view this attachment ]  




« Last Edit: August 13, 2020, 04:29:01 am by krovit »
Nothing is easy, especially when it appears simple (and nothing could be as dangerous as trying to do good to others)

Offline krovit

  • Forum Regular
  • Posts: 179
    • View Profile
For the record, it's 14-16 digits total = digits left + digits right of the decimal.

Ah! Ok
Nothing is easy, especially when it appears simple (and nothing could be as dangerous as trying to do good to others)

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
Yeah Windows Calculator has impressive math routines supporting it. We found that out comparing N! with _INTEGER64. As I recall we could do a deck of cards, 52!, but not 100!

Remember with calculators you need 1 calculation in a second or 2 but that would hardly do in a graphics animation.

« Last Edit: August 13, 2020, 11:55:45 am by bplus »

Offline krovit

  • Forum Regular
  • Posts: 179
    • View Profile
A contribution to the matter could come from ThinBasic (not for advertising but only to explore other possibilities).

Code: QB64: [Select]
  1. uses "Console"
  2.  
  3. CZ = 1.49999999999999  'limit: 15 places
  4. cz=cz+0.05
  5.  
  6. printl cz
  7.  
  8.  
  9. dim Cx AS ext
  10. Cx = 1.49999999999999  '15 places
  11. cx=cx+0.05
  12.  
  13. Cx = 1.49999999999999999  '18 places
  14. cx=cx+0.05
  15.  
  16. printl cx
  17.  
  18. WaitKey
  19.  

https://www.thinbasic.com/public/products/thinBasic/help/html/index.html

In ThinBasic it is possible to declare a variable type EXTENDED (EXT) width 18 digits of precision.
Okay, nothing sensational: 15 or 18, almost always wouldn't make a difference in the calculations.

However, I would like to point out a difference in behavior that appears useful for our purposes.

Then digits beyond the eighteenth are ignored (which one would expect) BUT it's interesting that the number is not automatically rounded! Only digits beyond the 18th are ignored.

The result is therefore similar to this:

1.49999999999999999999999999999999999999999999999999999 + 0.05 = 1.54999999999999999 (18 digits)

and not

1.49999999999999999999999999999999999999999999999999999 + 0.05 = 1.55


It seems to me a good compromise between precision, rounding and display!




Obviously... I have no idea how this effect can be reproduced in QB64...

« Last Edit: August 14, 2020, 10:30:51 am by krovit »
Nothing is easy, especially when it appears simple (and nothing could be as dangerous as trying to do good to others)

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
Quote
Obviously... I have no idea how this effect can be reproduced in QB64...

Make your own math subs working with strings. Lot's of work, slow running but you can have all the precision you want. 100 decimals? piece of cake*! once the math routines are worked out. ;-) 

This was already suggested by Steve, not sure you are understanding, could be lost in translation. :(

* ("Piece of cake" = expression of "easy to do"  ie, It's not hard to eat a piece of cake.)

Offline krovit

  • Forum Regular
  • Posts: 179
    • View Profile
"Nothing is easy, especially when it appears simple".

Who said it? I say it! It is my motto when it comes to IT (and not only).

I understand well that ThinBasic's result is not reproducible quickly in QB64, but it doesn't seem like a bad idea to think about it.

Ultimately it seems that "simply" the garbage beyond the 15-18th digit is ignored and deleted and the system limits itself to operating with what remains WITHOUT rounding.

In my systems I never use direct numbers but strings introduced character by character and appropriately treated, as Steve approximately says.
This way, as you say, I get total control of the input. At some point, however, these strings have to be transformed into numbers and processed. At that point - in some cases - the question I raised arises.

For example: if the result is 1.49 periodic and an IF-THEN were to deal with it, the answer would ALWAYS be incorrect because for the periodic 1.49 system (1.499999999999999999999999999999999999999...) it is ALWAYS 1.5

But that's okay ... it's my problem. excess of zeal...


...But ThinBasic seems to me to solve the problem brilliantly. And with that I don't mean to take anything away from QB64 and its programmers


Anyway thanks for your patience and for listening


_____
For the record: several other basic dialects, which I have tested, have the same behavior as QB64. Thinbasic seems like an commendable exception (I haven't tried VB and not even the C)


« Last Edit: August 14, 2020, 04:05:40 pm by krovit »
Nothing is easy, especially when it appears simple (and nothing could be as dangerous as trying to do good to others)

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
@krovit Here it is in QB64, 1.49999.... 24 nines no rounding
Code: QB64: [Select]
  1. _TITLE "Math regulator demo for krovit" ' b+ start 2020-08-18
  2. RANDOMIZE TIMER 'now that it's seems to be running silent
  3. SCREEN _NEWIMAGE(1200, 700, 32)
  4. _DELAY .25
  5. x$ = mr$("3", "/", "2") '1.5
  6. y$ = mr$(x$, "-", ".0000000000000000000000001") '1.49999... 24 nines no rounding
  7. PRINT "Differnce to 1.5 is "; mr$(y$, "-", "1.5")
  8.  
  9.  
  10. FUNCTION mr$ (a$, op$, b$) ' catchy? mr$ for math regulator
  11.     DIM ca$, cb$, aSgn$, bSgn$, postOp$, SGN$
  12.     DIM adp AS INTEGER, bdp AS INTEGER, dp AS INTEGER, lpop AS INTEGER
  13.  
  14.     op$ = _TRIM$(op$) 'save fixing each time
  15.     ca$ = _TRIM$(a$): cb$ = _TRIM$(b$) 'make copies in case we change
  16.     'strip signs and decimals
  17.     IF LEFT$(ca$, 1) = "-" THEN
  18.         aSgn$ = "-": ca$ = MID$(ca$, 2)
  19.     ELSE
  20.         aSgn$ = "": ca$ = ca$
  21.     END IF
  22.     dp = INSTR(ca$, ".")
  23.     IF dp > 0 THEN
  24.         adp = LEN(ca$) - dp
  25.         ca$ = MID$(ca$, 1, dp - 1) + MID$(ca$, dp + 1)
  26.     ELSE
  27.         adp = 0
  28.     END IF
  29.     IF LEFT$(cb$, 1) = "-" THEN
  30.         bSgn$ = "-": cb$ = MID$(cb$, 2)
  31.     ELSE
  32.         bSgn$ = "": cb$ = cb$
  33.     END IF
  34.     dp = INSTR(cb$, ".")
  35.     IF dp > 0 THEN
  36.         bdp = LEN(cb$) - dp
  37.         cb$ = MID$(cb$, 1, dp - 1) + MID$(cb$, dp + 1)
  38.     ELSE
  39.         bdp = 0
  40.     END IF
  41.  
  42.     IF op$ = "+" OR op$ = "-" OR op$ = "/" THEN 'add or subtr  even up strings on right of decimal
  43.         'even up the right sides of decimals if any
  44.         IF adp > bdp THEN dp = adp ELSE dp = bdp
  45.         IF adp < dp THEN ca$ = ca$ + STRING$(dp - adp, "0")
  46.         IF bdp < dp THEN cb$ = cb$ + STRING$(dp - bdp, "0")
  47.     ELSEIF op$ = "*" THEN
  48.         dp = adp + bdp
  49.     END IF
  50.     IF op$ = "*" OR op$ = "/" THEN
  51.         IF aSgn$ = bSgn$ THEN SGN$ = "" ELSE SGN$ = "-"
  52.     END IF
  53.     'now according to signs and op$ call add$ or subtr$
  54.     IF op$ = "+" THEN
  55.         IF aSgn$ = bSgn$ THEN 'really add
  56.             postOp$ = aSgn$ + add$(ca$, cb$)
  57.         ELSE 'have a case of subtraction
  58.             IF aSgn$ = "-" THEN postOp$ = subtr$(cb$, ca$) ELSE postOp$ = subtr$(ca$, cb$)
  59.         END IF
  60.     ELSEIF op$ = "-" THEN
  61.         IF bSgn$ = "-" THEN 'really add but switch b sign
  62.             bSgn$ = ""
  63.             IF aSgn$ = "-" THEN
  64.                 postOp$ = subtr$(cb$, ca$)
  65.             ELSE 'aSgn = ""
  66.                 postOp$ = add$(ca$, cb$)
  67.             END IF
  68.         ELSE 'bSgn$ =""
  69.             IF aSgn$ = "-" THEN
  70.                 bSgn$ = "-"
  71.                 postOp$ = aSgn$ + add$(ca$, cb$)
  72.             ELSE
  73.                 postOp$ = subtr$(ca$, cb$)
  74.             END IF
  75.         END IF
  76.     ELSEIF op$ = "*" THEN
  77.         postOp$ = SGN$ + mult$(ca$, cb$)
  78.     ELSEIF op$ = "/" THEN
  79.         postOp$ = SGN$ + divide$(ca$, cb$)
  80.     END IF ' which op
  81.     'put dp back
  82.     IF op$ <> "/" THEN
  83.         lpop = LEN(postOp$) ' put decimal back
  84.         postOp$ = MID$(postOp$, 1, lpop - dp) + "." + MID$(postOp$, lpop - dp + 1)
  85.     END IF
  86.     mr$ = trim0$(postOp$)
  87.  
  88. FUNCTION divide$ (n$, d$)
  89.     DIM di$, ndi$, nD AS INTEGER
  90.     IF trim0$(n$) = "0" THEN divide$ = "0": EXIT FUNCTION
  91.     IF trim0$(d$) = "0" THEN divide$ = "div 0": EXIT FUNCTION
  92.     IF trim0$(d$) = "1" THEN divide$ = trim0$(n$): EXIT FUNCTION '8/17 add trim0$
  93.     di$ = MID$(nInverse$(d$, 100), 2) 'chop off decimal point after
  94.     nD = LEN(di$)
  95.     ndi$ = mult$(n$, di$)
  96.     ndi$ = MID$(ndi$, 1, LEN(ndi$) - nD) + "." + RIGHT$(STRING$(nD, "0") + RIGHT$(ndi$, nD), nD)
  97.     divide$ = trim0$(ndi$)
  98.  
  99. FUNCTION nInverse$ (n$, DP AS INTEGER) 'assume decimal at very start of the string of digits returned, no rounding
  100.     DIM m$(1 TO 9), si$, r$, outstr$, d$
  101.     DIM i AS INTEGER
  102.     FOR i = 1 TO 9
  103.         si$ = _TRIM$(STR$(i))
  104.         m$(i) = mult$(si$, n$)
  105.     NEXT
  106.     outstr$ = ""
  107.     IF n$ = "0" THEN nInverse$ = "Div 0": EXIT FUNCTION
  108.     IF n$ = "1" THEN nInverse$ = "1": EXIT FUNCTION
  109.     outstr$ = "." 'everything else n > 1 is decimal 8/17
  110.     r$ = "10"
  111.     DO
  112.         WHILE LEFT$(subtr$(r$, n$), 1) = "-" '   r - n < 0
  113.             outstr$ = outstr$ + "0" '            add 0 to the  output string
  114.             IF LEN(outstr$) = DP THEN nInverse$ = outstr$: EXIT FUNCTION 'check if we've reached DP length
  115.             r$ = r$ + "0"
  116.         WEND
  117.         FOR i = 9 TO 1 STEP -1
  118.             IF LTE(m$(i), r$) THEN d$ = _TRIM$(STR$(i)): EXIT FOR
  119.         NEXT
  120.         outstr$ = outstr$ + d$
  121.         IF LEN(outstr$) = DP THEN nInverse$ = outstr$: EXIT FUNCTION
  122.         r$ = subtr$(r$, mult$(d$, n$)) 'r = r -d*n
  123.         IF r$ = "0" THEN nInverse$ = outstr$: EXIT FUNCTION 'found a perfect divisor
  124.         r$ = r$ + "0" 'add another place
  125.     LOOP
  126.  
  127. FUNCTION mult$ (a$, b$) 'assume both positive integers prechecked as all digits strings
  128.     DIM la AS INTEGER, lb AS INTEGER, m AS INTEGER, g AS INTEGER, dp AS INTEGER
  129.     DIM f18$, f1$, t$, build$, accum$
  130.  
  131.     IF trim0$(a$) = "0" THEN mult$ = "0": EXIT FUNCTION
  132.     IF trim0$(b$) = "0" THEN mult$ = "0": EXIT FUNCTION
  133.     IF trim0$(a$) = "1" THEN mult$ = trim0$(b$): EXIT FUNCTION
  134.     IF trim0$(b$) = "1" THEN mult$ = trim0$(a$): EXIT FUNCTION
  135.     'find the longer number and make it a mult of 18 to take 18 digits at a time from it
  136.     la = LEN(a$): lb = LEN(b$)
  137.     IF la > lb THEN
  138.         m = INT(la / 18) + 1
  139.         f18$ = RIGHT$(STRING$(m * 18, "0") + a$, m * 18)
  140.         f1$ = b$
  141.     ELSE
  142.         m = INT(lb / 18) + 1
  143.         f18$ = RIGHT$(STRING$(m * 18, "0") + b$, m * 18)
  144.         f1$ = a$
  145.     END IF
  146.     FOR dp = LEN(f1$) TO 1 STEP -1 'dp = digit position of the f1$
  147.         build$ = "" 'line builder
  148.         co = 0
  149.         'now taking 18 digits at a time Thanks Steve McNeill
  150.         FOR g = 1 TO m
  151.             v18 = VAL(MID$(f18$, m * 18 - g * 18 + 1, 18))
  152.             sd = VAL(MID$(f1$, dp, 1))
  153.             t$ = RIGHT$(STRING$(19, "0") + _TRIM$(STR$(v18 * sd + co)), 19)
  154.             co = VAL(MID$(t$, 1, 1))
  155.             build$ = MID$(t$, 2) + build$
  156.         NEXT g
  157.         IF co THEN build$ = _TRIM$(STR$(co)) + build$
  158.         IF dp = LEN(f1$) THEN
  159.             accum$ = build$
  160.         ELSE
  161.             accum$ = add$(accum$, build$ + STRING$(LEN(f1$) - dp, "0"))
  162.         END IF
  163.     NEXT dp
  164.     mult$ = accum$
  165.  
  166. FUNCTION subtr$ (sum$, minus$) ' assume both numbers are positive all digits
  167.     DIM m AS INTEGER, g AS INTEGER, p AS INTEGER
  168.     DIM ts$, tm$, sign$, LG$, sm$, t$, result$
  169.  
  170.     ts$ = TrimLead0$(sum$): tm$ = TrimLead0$(minus$)
  171.     IF trim0(ts$) = trim0$(tm$) THEN subtr$ = "0": EXIT FUNCTION 'OK proceed with function knowing they are not equal
  172.     tenE18 = 1000000000000000000 'yes!!! no dang E's
  173.     IF LTE(ts$, tm$) THEN ' which is bigger? minus is bigger
  174.         sign$ = "-"
  175.         m = INT(LEN(tm$) / 18) + 1
  176.         LG$ = RIGHT$(STRING$(m * 18, "0") + tm$, m * 18)
  177.         sm$ = RIGHT$(STRING$(m * 18, "0") + ts$, m * 18)
  178.     ELSE 'sum is bigger
  179.         sign$ = ""
  180.         m = INT(LEN(ts$) / 18) + 1
  181.         LG$ = RIGHT$(STRING$(m * 18, "0") + ts$, m * 18)
  182.         sm$ = RIGHT$(STRING$(m * 18, "0") + tm$, m * 18)
  183.     END IF
  184.     'now taking 18 digits at a time From Steve I learned we can do more than 1 digit at a time
  185.     FOR g = 1 TO m
  186.         VB = VAL(MID$(LG$, m * 18 - g * 18 + 1, 18))
  187.         vs = VAL(MID$(sm$, m * 18 - g * 18 + 1, 18))
  188.         IF vs > VB THEN
  189.             t$ = RIGHT$(STRING$(18, "0") + _TRIM$(STR$(tenE18 - vs + VB)), 18)
  190.             p = (m - g) * 18
  191.             WHILE p > 0 AND MID$(LG$, p, 1) = "0"
  192.                 MID$(LG$, p, 1) = "9"
  193.                 p = p - 1
  194.             WEND
  195.             IF p > 0 THEN MID$(LG$, p, 1) = _TRIM$(STR$(VAL(MID$(LG$, p, 1)) - 1))
  196.         ELSE
  197.             t$ = RIGHT$(STRING$(18, "0") + _TRIM$(STR$(VB - vs)), 18)
  198.         END IF
  199.         result$ = t$ + result$
  200.     NEXT
  201.     subtr$ = sign$ + result$
  202.  
  203. FUNCTION add$ (a$, b$) 'add 2 positive integers assume a and b are just numbers no spaces or - signs
  204.     'first thing is to set a and b numbers to same length and multiple of 18 so can take 18 digits at a time
  205.     DIM la AS INTEGER, lb AS INTEGER, m AS INTEGER, g AS INTEGER
  206.     DIM fa$, fb$, t$, new$, result$
  207.     la = LEN(a$): lb = LEN(b$)
  208.     IF la > lb THEN m = INT(la / 18) + 1 ELSE m = INT(lb / 18) + 1
  209.     fa$ = RIGHT$(STRING$(m * 18, "0") + a$, m * 18)
  210.     fb$ = RIGHT$(STRING$(m * 18, "0") + b$, m * 18)
  211.  
  212.     'now taking 18 digits at a time Thanks Steve McNeill
  213.     FOR g = 1 TO m
  214.         sa = VAL(MID$(fa$, m * 18 - g * 18 + 1, 18))
  215.         sb = VAL(MID$(fb$, m * 18 - g * 18 + 1, 18))
  216.         t$ = RIGHT$(STRING$(36, "0") + _TRIM$(STR$(sa + sb + co)), 36)
  217.         co = VAL(MID$(t$, 1, 18))
  218.         new$ = MID$(t$, 19)
  219.         result$ = new$ + result$
  220.     NEXT
  221.     IF co THEN result$ = STR$(co) + result$
  222.     add$ = result$
  223.  
  224. ' String Math Helpers -----------------------------------------------
  225.  
  226. 'this function needs TrimLead0$(s$)
  227. FUNCTION LTE (a$, b$) ' a$ Less Than or Equal b$  comparison of 2 strings
  228.     DIM ca$, cb$, la AS INTEGER, lb AS INTEGER, i AS INTEGER
  229.     ca$ = TrimLead0$(a$): cb$ = TrimLead0(b$)
  230.     la = LEN(ca$): lb = LEN(cb$)
  231.     IF ca$ = cb$ THEN
  232.         LTE = -1
  233.     ELSEIF la < lb THEN ' a is smaller
  234.         LTE = -1
  235.     ELSEIF la > lb THEN ' a is bigger
  236.         LTE = 0
  237.     ELSEIF la = lb THEN ' equal lengths
  238.         FOR i = 1 TO LEN(ca$)
  239.             IF VAL(MID$(ca$, i, 1)) > VAL(MID$(cb$, i, 1)) THEN
  240.                 LTE = 0: EXIT FUNCTION
  241.             ELSEIF VAL(MID$(ca$, i, 1)) < VAL(MID$(cb$, i, 1)) THEN
  242.                 LTE = -1: EXIT FUNCTION
  243.             END IF
  244.         NEXT
  245.     END IF
  246.  
  247. ' ------------------------------------- use these for final display
  248.  
  249. FUNCTION TrimLead0$ (s$) 'for treating strings as number (pos integers)
  250.     DIM copys$, i AS INTEGER, find AS INTEGER
  251.     copys$ = _TRIM$(s$) 'might as well remove spaces too
  252.     i = 1: find = 0
  253.     WHILE i < LEN(copys$) AND MID$(copys$, i, 1) = "0"
  254.         i = i + 1: find = 1
  255.     WEND
  256.     IF find = 1 THEN copys$ = MID$(copys$, i)
  257.     IF copys$ = "" THEN TrimLead0$ = "0" ELSE TrimLead0$ = copys$
  258.  
  259. FUNCTION TrimTail0$ (s$)
  260.     DIM copys$, dp AS INTEGER, i AS INTEGER, find AS INTEGER
  261.     copys$ = _TRIM$(s$) 'might as well remove spaces too
  262.     TrimTail0$ = copys$
  263.     dp = INSTR(copys$, ".")
  264.     IF dp > 0 THEN
  265.         i = LEN(copys$): find = 0
  266.         WHILE i > dp AND MID$(copys$, i, 1) = "0"
  267.             i = i - 1: find = 1
  268.         WEND
  269.         IF find = 1 THEN
  270.             IF i = dp THEN
  271.                 TrimTail0$ = MID$(copys$, 1, dp - 1)
  272.             ELSE
  273.                 TrimTail0$ = MID$(copys$, 1, i)
  274.             END IF
  275.         END IF
  276.     END IF
  277.  
  278. FUNCTION trim0$ (s$)
  279.     DIM cs$, si$
  280.     cs$ = s$
  281.     si$ = LEFT$(cs$, 1)
  282.     IF si$ = "-" THEN cs$ = MID$(cs$, 2)
  283.     cs$ = TrimLead0$(cs$)
  284.     cs$ = TrimTail0$(cs$)
  285.     IF RIGHT$(cs$, 1) = "." THEN cs$ = MID$(cs$, 1, LEN(cs$) - 1)
  286.     IF si$ = "-" THEN trim0$ = si$ + cs$ ELSE trim0$ = cs$
  287.  
« Last Edit: August 19, 2020, 10:53:16 am by bplus »