QB64.org Forum

Active Forums => QB64 Discussion => Topic started by: bplus on November 07, 2019, 12:18:54 pm

Title: Extract numerics from text string
Post by: bplus on November 07, 2019, 12:18:54 pm

Hello,

I am looking for easy way to extract real numbers from text strings. Got any ideas?

BTW, this is what is needed for an EVAL function parsing.

Actually, the bugger is a negative sign for a number, the negative sign for an exponent, the negative sign for a subtraction operation plus, of course, the negative sign for a hyphen.

I am guessing you have to look both ways before crossing the line with a judgement.

And what if you have negative signs to the left of you and hyphens to the right, stuck in the middle...


Must be lunch time, sugar is getting low.

Title: Re: Extract numerics from text string
Post by: SMcNeill on November 07, 2019, 12:41:22 pm

Quote from: bplus on November 07, 2019, 12:18:54 pm

Hello,

I am looking for easy way to extract real numbers from text strings. Got any ideas?

BTW, this is what is needed for an EVAL function parsing.

Actually, the bugger is a negative sign for a number, the negative sign for an exponent, the negative sign for a subtraction operation plus, of course, the negative sign for a hyphen.

And don’t forget the E and D for scientific notation.  My math evaluator does all this sort of stuff, somewhere in it, but I’d have to study back up to see where/how it does it.  It’s been years since I wrote the thing! 

Title: Re: Extract numerics from text string
Post by: STxAxTIC on November 07, 2019, 12:43:33 pm

From the same era steve mentions I offer sxript as a self consistent example. Click there to find:

<----

Title: Re: Extract numerics from text string
Post by: SMcNeill on November 07, 2019, 12:52:37 pm

Let me see if I can remember the steps to do this...

Start parsing the string from left to right.  Look for “+-0123456789.“  These can be the start of a value.

After the start, if it’s a sign, look for additional signs.  - - or + - or something odd.   For the number, only worry about the last sign you generate. 0 - -1 (zero minus negative one); all you need to worry about is the last sign.  The others are math operators.

Once you find the last sign, look for “0123456789.”  They’re the only valid input acceptable after a sign.  If the next character isn’t one, it’s not a number.

Once you have a valid digit (or period), continue to collect digits and periods, until you reach an invalid character.  Only one period is allowed.  If you get more, reject the input.

At this point, you now have most numbers.  You *could* stop here.  If you need scientific notation, look for the “E” or “D”, then the second sign, then digits...

Title: Re: Extract numerics from text string
Post by: Petr on November 07, 2019, 01:15:41 pm

My basic solution:  (integers only)

Code: QB64: [Select]

1. s$ = "I use -5 and 57, 34"
2.  
3. REDIM nrs(0) AS INTEGER
4. FOR h = 1 TO LEN(s$)
5.     oldch = ch
6.     ch = ASC(s$, h)
7.     IF oldch = 45 AND ch >= 49 AND ch <= 57 THEN num$ = "-" + CHR$(ch)
8.     IF ch >= 49 AND ch <= 57 AND oldch <> 45 THEN num$ = num$ + CHR$(ch)
9.  
10.     IF ch < 49 OR ch > 57 OR h = LEN(s$) THEN
11.         IF oldch <> 45 THEN
12.             IF LEN(num$) THEN
13.                 unrs = UBOUND(nrs) + 1
14.                 REDIM _PRESERVE nrs(unrs) AS INTEGER
15.                 nrs(unrs) = VAL(num$)
16.                 num$ = ""
17.             END IF
18.         END IF
19.     END IF
20. NEXT
21.  
22. PRINT "Found numbers:"; UBOUND(nrs)
23. PRINT
24. FOR l = 1 TO UBOUND(nrs)
25.     PRINT nrs(l)
26. NEXT
27.  

Upgrade for single numbers...

Code: QB64: [Select]

1. s$ = "I use -5 and 57, 67, -3.14"
2.  
3. REDIM nrs(0) AS SINGLE
4. FOR h = 1 TO LEN(s$)
5.     oldch = ch
6.     ch = ASC(s$, h)
7.     IF ch = 46 THEN num$ = num$ + "."
8.     IF oldch = 45 AND ch >= 49 AND ch <= 57 THEN num$ = "-" + CHR$(ch)
9.     IF ch >= 49 AND ch <= 57 AND oldch <> 45 THEN num$ = num$ + CHR$(ch)
10.  
11.     IF ch <> 46 THEN
12.         IF ch < 49 OR ch > 57 OR h = LEN(s$) THEN
13.             IF oldch <> 45 THEN
14.                 IF LEN(num$) THEN
15.  
16.                     unrs = UBOUND(nrs) + 1
17.                     REDIM _PRESERVE nrs(unrs) AS SINGLE
18.                     nrs(unrs) = VAL(num$)
19.                     num$ = ""
20.                 END IF
21.             END IF
22.         END IF
23.     END IF
24. NEXT
25.  
26. PRINT "Found numbers:"; UBOUND(nrs)
27. PRINT
28. FOR l = 1 TO UBOUND(nrs)
29.     PRINT nrs(l)
30. NEXT
31.  

Title: Re: Extract numerics from text string
Post by: bplus on November 07, 2019, 02:43:45 pm

So far Petr's ahead in my eyes, he has given me code to test maybe modify into a sub or function.

Steve, is next best (or best, after all I just asked for ideas) because he has outlined idea that seems plausible, kind of worried about part of string +--+... next to number, left of number, (if memory serves me, I think you do have to look at the right side of the character too) but this was also brought up by another great EVAL builder at JB. I think my EVAL assumes all + encountered mean add operation.

STx is third best because he is making me read, worse, making me dig up the material to read as well  ;-))
If I am lazy, it is definitely in this regard.

Yeah, the E's and D's make this thing a real chore, I guess that is partly why I am a fan of Integers and Discrete Maths.

Title: Re: Extract numerics from text string
Post by: Petr on November 08, 2019, 11:07:05 am

Hi BPlus, I combed the source code a little bit (I think SELECT CASE is more right for this use) and I made a SUB of it, which returns a number field.

Code: QB64: [Select]

1. REDIM nrs(0) AS SINGLE
2. NumFromString "Sub NumFromString return all numbers 45 from -3.14 this 000 string to 1234567890, .65, 0.001, -0.0021 array7", nrs()
3. FOR L = 1 TO UBOUND(nrs)
4.     PRINT nrs(L)
5. NEXT
6.  
7. SUB NumFromString (s AS STRING, nrs() AS SINGLE)
8.     s$ = s$ + "a" 'for case, if last character in string is number
9.     FOR R = 1 TO LEN(s$)
10.         oldCh = Ch
11.         Ch = ASC(s$, R)
12.         SELECT CASE Ch
13.             CASE 48 TO 57: num$ = num$ + CHR$(Ch)
14.             CASE 45: num$ = "-" ' -
15.             CASE 46: IF oldCh >= 48 AND oldCh <= 57 THEN num$ = num$ + "." ELSE num$ = "0."
16.             CASE IS < 48, IS > 57
17.                 IF LEN(num$) THEN
18.                     U = UBOUND(nrs)
19.                     REDIM _PRESERVE nrs(U + 1) AS SINGLE
20.                     nrs(U + 1) = VAL(num$)
21.                     num$ = ""
22.                 END IF
23.         END SELECT
24.     NEXT
25. END SUB
26.  

Basically, it is just a small modification of the program, which I made easier the work. When practicing spelling with my children it was necessary to hide the letters i, I and y, Y in the text. Basically, this is a very similar but simpler problem:

Code: QB64: [Select]

1. 'sub, which search "Y", "y", "I", "i" in strings and replace it with "_" (for kids testing)
2.  
3.  
4. Text$ = "White House, Washington, Yellowstone, Hollywood, Smokies, Permission"
5. PRINT Ifinder$(Text$)
6.  
7.  
8. FUNCTION Ifinder$ (text AS STRING)
9.     FOR T = 1 TO LEN(text$)
10.         ch = ASC(text$, T)
11.         SELECT CASE ch
12.             CASE 73, 105, 89, 121: c$ = "_"
13.             CASE ELSE
14.                 c$ = CHR$(ch)
15.         END SELECT
16.         Ifinder$ = Ifinder$ + c$
17.     NEXT
18. END FUNCTION
19.  

This is just the core of the program, of course UNICODE functions are used for Czech and the text is not fixed string, but is copied from the clipboard. Simple and very effective.

Title: Re: Extract numerics from text string
Post by: bplus on November 08, 2019, 10:19:14 pm

Here is check that should allow exponents along with normal and real but not allow date, time (different category that might be checked also by other IsDate, IsTime functions.

Code: QB64: [Select]

1. _TITLE "Find numbers in text" 'b+ 2019-11-08
2. 'a slighty different approach
3. s$(0) = "Sub NumFromString return all numbers 45 from -3.14 this 000 string to 1234567890, .65, 0.001, -0.0021 array7"
4. s$(1) = "You can reach me at +22.17 degrees latitude, -44.96 degrees"
5. s$(2) = "I use -5 and 0.0001, 34, 3.19216457E-46, -9, 00.00.00, -4.987651223847D+49, 12:32:59, 11/09/2019"
6.  
7. REDIM wrds$(1 TO 1)
8. FOR i = 0 TO 2
9.     PRINT "For Text string = "; s$(i)
10.     replace1for1 s$(i), ",", " "
11.     PRINT "Replace commas for spaces: "; s$(i)
12.     Split s$(i), " ", wrds$()
13.     PRINT: PRINT "Numbers found"
14.     FOR j = 1 TO UBOUND(wrds$)
15.         IF IsNumber(wrds$(j)) THEN PRINT wrds$(j)
16.     NEXT
17.     PRINT "press any to wakeup next..."
18.     SLEEP
19.     CLS
20. NEXT
21. PRINT "Testing is done."
22.  
23. SUB replace1for1 (s$, char1$, new1$)
24.     p = INSTR(s$, char1$)
25.     WHILE p > 0
26.         MID$(s$, p, 1) = new1$
27.         p = INSTR(p + 1, s$, char1$)
28.     WEND
29. END SUB
30.  
31. FUNCTION IsNumber% (s$)
32.     c$ = s$ 'make a copy of s$ because will tear it up
33.     'has at most 1 decimal point
34.     'neg at right  of E or D    or  + immediate right of E or D
35.     'neg at start
36.     'all other chars are digits
37.     'maybe + too at start would be OK
38.  
39.     IF _TRIM$(s$) = "" THEN EXIT FUNCTION
40.     p = INSTR(c$, ".")
41.     IF p > 0 THEN
42.         IF INSTR(p + 1, c$, ".") > 0 THEN EXIT FUNCTION 'not 2 deicmals
43.         c$ = MID$(c$, 1, p - 1) + MID$(c$, p + 1) 'get rid of decimal
44.         'PRINT "After decimal = "; c$
45.     END IF
46.     p = INSTR(c$, "E")
47.     IF p > 0 THEN
48.         IF MID$(c$, p + 1, 1) = "-" OR MID$(c$, p + 1, 1) = "+" THEN
49.             c$ = MID$(c$, 1, p - 1) + MID$(c$, p + 2) 'get rid of E+ or E-
50.             'PRINT "After E fix = "; c$
51.         ELSE
52.             EXIT FUNCTION
53.         END IF
54.     END IF
55.     p = INSTR(c$, "D")
56.     IF p > 0 THEN
57.         IF MID$(c$, p + 1, 1) = "-" OR MID$(c$, p + 1, 1) = "+" THEN
58.             c$ = MID$(c$, 1, p - 1) + MID$(c$, p + 2) 'get rid of D+ or D-
59.             'PRINT "After D fix = "; c$
60.         END IF
61.     END IF
62.     p = INSTR(c$, "-")
63.     IF p > 0 THEN
64.         IF INSTR(p + 1, c$, "-") THEN EXIT FUNCTION
65.         IF p <> 1 THEN EXIT FUNCTION
66.         c$ = MID$(c$, 1, p - 1) + MID$(c$, p + 1) 'get rid of -
67.         'PRINT "After - = "; c$
68.     END IF
69.     p = INSTR(c$, "+")
70.     IF p > 0 THEN
71.         IF INSTR(p + 1, c$, "+") THEN EXIT FUNCTION
72.         IF p <> 1 THEN EXIT FUNCTION
73.         c$ = MID$(c$, 1, p - 1) + MID$(c$, p + 1) 'get rid of -
74.         'PRINT "After + = "; c$
75.     END IF
76.     'all the rest better be digits
77.     FOR i = 1 TO LEN(c$)
78.         IF ASC(c$, i) < 48 OR ASC(c$, i) > 57 THEN EXIT FUNCTION
79.     NEXT
80.     IsNumber% = -1
81. END FUNCTION
82.  
83. SUB Split (SplitMeString AS STRING, delim AS STRING, loadMeArray() AS STRING)
84.     DIM curpos AS LONG, arrpos AS LONG, LD AS LONG, dpos AS LONG 'fix use the Lbound the array already has
85.     curpos = 1: arrpos = LBOUND(loadMeArray): LD = LEN(delim)
86.     dpos = INSTR(curpos, SplitMeString, delim)
87.     DO UNTIL dpos = 0
88.         loadMeArray(arrpos) = MID$(SplitMeString, curpos, dpos - curpos)
89.         arrpos = arrpos + 1
90.         IF arrpos > UBOUND(loadMeArray) THEN REDIM _PRESERVE loadMeArray(LBOUND(loadMeArray) TO UBOUND(loadMeArray) + 1000) AS STRING
91.         curpos = dpos + LD
92.         dpos = INSTR(curpos, SplitMeString, delim)
93.     LOOP
94.     loadMeArray(arrpos) = MID$(SplitMeString, curpos)
95.     REDIM _PRESERVE loadMeArray(LBOUND(loadMeArray) TO arrpos) AS STRING 'get the ubound correct
96. END SUB
97.  
98.  

Actually this has me thinking about finding numbers between (){} or [] after I had found problem with commas at the end of numbers.

Title: Re: Extract numerics from text string
Post by: SMcNeill on November 08, 2019, 10:55:34 pm

Quote

Actually this has me thinking about finding numbers between (){} or [] after I had found problem with commas at the end of numbers.

You’ll drive yourself crazy, unless you set a hard standard for what’s acceptable and what’s not for this type of tool.

For example: “1,234,567”

Is the above 1234567 or 1, and 234, and 567?  One number separated by commas, or 3 numbers?  What if we write it as “$12,345.67”?  Is the minus in “-$123” part of the number, or a subtraction sign?  Is “11-10-2018” a date, or math between three values?  “How about 11 - 10 - 2018”?

Once you start trying to decide IsDate, IsTime, IsMoney, and such, you’ll end up with a lot of overlap in those functions.  If you look at it properly, “12:13” is a time, but it’s also 2 numbers just like “12 + 13” is — that “:” might designate “the ratio of 12 to 13”, such as in odds.  “The bet on the football game is 12:13, favoring the home team.”

Instead of saying date, time, and such aren’t numbers, you might be better served to set flags for what they might be.

“12:13” could give 3 numbers — 12, 13, 12:13...

Or you could pop up an user box and ask them, “This value (11 - 10 - 2018) could be multiple things.  How would you like to define it?  [DATE] [MULTIPLE NUMBERS]”....

Title: Re: Extract numerics from text string
Post by: bplus on November 09, 2019, 12:28:49 am

Hi Steve,

All good points, at least I got some practice with verifying numbers with exponents, that is perfection now.

Text Only | Text with Attachments