Text Only | Text with Attachments

QB64.org Forum

Active Forums => QB64 Discussion => Topic started by: Pete on April 10, 2019, 01:20:51 pm

Title: Building a better INSTR(TRAP)?
Post by: Pete on April 10, 2019, 01:20:51 pm
All the discussion on seeding _INSTRREV got me thinking about how I'd actually like a seed function to work in INSTR(), and it isn't as is. Let's say you're searching for commas in a comma delimited string like: ",1  ,2  ,3  ,4  ,5  ,6  ,7  ,8  ,9  ," Now let's say I want everything from after the 4th comma. Well with INSTR() as is, I have to count up to where in the string that occurs, which is at position 13.  So we have rec$ = mid$(s$, instr(13, s$, ",") + 1, 1), which returns "4"

OK, wouldn't it be easier to have a method to just ask for the seed to be set to the 4th search position? In other words: rec$ = mid$(s$, instr(4, s$, ",") + 1, 1)

So the function would have to process the sub-string from the sting the number of times requested. This is something we generally have to code ourselves, but I could easily see the function doing it. It's just parsing, after all...

Code: QB64: [Select]
  1. s$ = ",1  ,2  ,3  ,4  ,5  ,6  ,7  ,8  ,9  ,"
  2. rec$ = MID$(s$, INSTR(13, s$, ",") + 1, 1)
  3. PRINT "QB64 INSTR(): "; rec$; " using 13-characters in as the seed: At position"; INSTR(13, s$, "") + 1
  4. recx$ = MID$(s$, instrx&(4, s$, ",") + 1, 1)
  5. PRINT "INSTR Function: "; recx$; " using 4(th) occurrence as the seed. At position"; instrx&(4, s$, ",") + 1
  6.  
  7. FUNCTION instrx& (seed&, s$, srch$)
  8. IF srch$ <> "" THEN
  9.     FOR instrx& = 1 TO LEN(s$)
  10.         IF RIGHT$(MID$(s$, 1, instrx&), LEN(srch$)) = srch$ THEN flag& = flag& + 1
  11.         IF flag& = seed& THEN EXIT FOR
  12.     NEXT
  14. IF flag& = 0 THEN instrx& = 0

Now that code won't stand, as considerations would have to be given for how it should act when a search term is not found, when a search term is present, but the seed is larger than the string, etc. Codeguy probably would have a faster method, anyway. So I didn't want to do a lot of extra work on this now, but I did want to use some simple code just to demonstrate the topic of discussion.

BTW - I'm not requesting INSTR() be changed. It needs to stay as is, to be QB compatible, of course. I just wanted to throw this idea out for discussion. I mean maybe I'm missing something else the seed, as it functions in INSTR() now, is needed for?

Pete     
Title: Re: Building a better INSTR(TRAP)?
Post by: Raven_Singularity on April 10, 2019, 01:35:24 pm
Unix command line tools are great for text manipulation.  I haven't looked at what text manipulation functions QB64 comes with, but would love to see more Unix functionality!



C/C++ libraries come with a ton of text manipulation functions, would be nice to see them all show up in QB64.
Title: Re: Building a better INSTR(TRAP)?
Post by: SMcNeill on April 10, 2019, 01:42:13 pm
How about this for a simple implementation:

Code: QB64: [Select]
  1. s$ = ",1  ,2  ,3  ,4  ,5  ,6  ,7  ,8  ,9  ,"
  2.  
  3. PRINT InstrOccurance(4, s$, ",") 'find the position of the 4th comma
  4. PRINT InstrOccurance(7, s$, ",") 'find the position of the 7th comma
  5.  
  6. FUNCTION InstrOccurance (occurance AS LONG, mainstring AS STRING, searchstring AS STRING)
  7.     FOR i = 1 TO occurance
  8.         o = INSTR(o + 1, mainstring, searchstring)
  9.     NEXT
  10.     IF i = occurance + 1 THEN InstrOccurance = o
Title: Re: Building a better INSTR(TRAP)?
Post by: Pete on April 10, 2019, 02:01:37 pm
My evil twin strikes again! :D

I'm grinning because I actually thought of using INSTR() with progressive seeding in my function, but then I felt bad for dis'ing it, and just went with a character by character FOR/NEXT loop, instead! You made my day!

Well, what do you think? I mean is there any point in the traditional way INSTR() is seeded that is needed? Wouldn't this be better? To me INSTR() seeding as it is seems like a half finished function. We, as coders, always have to add to it to make it useful.

Pete
Title: Re: Building a better INSTR(TRAP)?
Post by: Jack002 on April 10, 2019, 04:32:05 pm
Pete, I don't understand the usage of a "Seed" in this function

You seem to have a character separated field you want to do something with.
You seed it with 13. So what is the 13th offset of any of these strings? 1st fld? 5th? Over the end?

Back in '99 I wrote a function to parse char delimited strings. One thing I needed was the offset of the Nth field.
Lets say I want to parse field 10 out of the string.
First step, count the delimiters, if you found less than 10, return a null string, nothing found
Next, you need the offset of the 10th delimiter. I start from the far left and look for it like this

y = 1  ' find nth sepch$ char
FOR i = 1 TO fldno
  y = INSTR(y, HL7$, sepch$) + 1
NEXT

Once you've done that you can see the 11th delimiter (the system I was working within, you can always depend on a delimiter on both sides of your field) like this

' Found the nth sep, now wheres n+1?
z = INSTR(y, HL7$, sepch$)
' Fld is z - y bytes long

The rest is trivial, you do a MID$ of the incoming string and return it.

Title: Re: Building a better INSTR(TRAP)?
Post by: Pete on April 10, 2019, 06:41:19 pm
Jack, I also have ways to get records out of a string, progressive searches with the 1st parameter of INSTR() the seed parameter building so the next occurrence can be found, etc. But I always thought why does the function ask for a number, when asking for an occurrence would be better? With the current way INSTR() operates, I would have to code the routine to get that 4th record, but with the seed as an occurrence parameter, I just need one line of code...

Code: QB64: [Select]
  1. s$ = ",1  ,2  ,3  ,4  ,5  ,6  ,7  ,8  ,9  ,": srch$ = ","
  2.  
  3. INPUT "What record? 1 - 9: "; recnum&
  4. FOR i& = 1 TO recnum&
  5.     rec$ = MID$(s$, INSTR(seed&, s$, srch$) + 1, 3)
  6.     seed& = seed& + 4
  8. PRINT "Results: "; rec$; " Position #"; seed&
  9.  
  10. ' Now try it again with this one-line rec$ = statement and function.
  11. INPUT "What record? 1 - 9: "; seed&
  12. rec$ = MID$(s$, instrx&(seed&, s$, srch$) + 1, 3)
  13. PRINT "Results: "; rec$; " Record #"; seed&
  14.  
  15. FUNCTION instrx& (seed&, s$, srch$)
  16. IF srch$ <> "" THEN
  17.     FOR instrx& = 1 TO LEN(s$)
  18.         IF RIGHT$(MID$(s$, 1, instrx&), LEN(srch$)) = srch$ THEN flag& = flag& + 1
  19.         IF flag& = seed& THEN EXIT FOR
  20.     NEXT
  22. IF flag& = 0 THEN instrx& = 0


Title: Re: Building a better INSTR(TRAP)?
Post by: Jack002 on April 11, 2019, 09:00:36 am
Ok, yeah, I agree totally. We can make functions to do that, but being able to find the nth location of a char in a string is much more useful. Agreed.
I also thought we needed a direction flag. Start at the end (or some seed value) and search left or right based on your input.
Also a mirror flag as you've said. Wrap around to the other end when the end is hit (as another option)
Title: Re: Building a better INSTR(TRAP)?
Post by: codeguy on April 12, 2019, 07:50:46 am
Code: QB64: [Select]
  1. a$ = "pat: puck moonen is hot and so is emily batty IJS pat pat tom pat robert harry patty elmer bugs james felix sam patricia roger dennis sandrab donald ralph pat felicia cathy harriet albert david sam katie frank pam sarah judy pat"
  2. b$ = "pat"
  3. FOR direction% = -1 TO 1 STEP 2
  4.     SELECT CASE LEN(b$)
  5.         CASE 0
  6.             IF direction% = -1 THEN
  7.                 '* return len(a$)+1
  8.             ELSE
  9.                 '* return 1
  10.             END IF
  11.         CASE IS > LEN(a$)
  12.             '* return 0
  13.         CASE ELSE
  14.             countoccurrence& = 0
  15.             SELECT CASE direction%
  16.                 CASE -1
  17.                     x& = 0
  18.                     p& = LEN(a$) - LEN(b$) + 1 - x&
  19.                     DO
  20.                         IF MID$(a$, p& - x&, LEN(b$)) = b$ THEN
  21.                             PRINT p& - x&; MID$(a$, p& - x&, LEN(b$)); countoccurrence&
  22.                             x& = x& + LEN(b$)
  23.                             countoccurrence& = countoccurrence& + 1
  24.                         ELSE
  25.                             x& = x& - direction%
  26.                         END IF
  27.                     LOOP WHILE x& < p&
  28.                 CASE ELSE
  29.                     x& = 1
  30.                     p& = LEN(a$) - LEN(b$) + 1
  31.                     WHILE x& <= p&
  32.                         IF MID$(a$, x&, LEN(b$)) = b$ THEN
  33.                             PRINT x&; MID$(a$, x&, LEN(b$)); countoccurrence&
  34.                             x& = x& + LEN(b$)
  35.                             countoccurrence& = countoccurrence& + 1
  36.                         ELSE
  37.                             x& = x& + 1
  38.                         END IF
  39.                     WEND
  40.             END SELECT
  41.     END SELECT
  43.  
Title: Re: Building a better INSTR(TRAP)?
Post by: Pete on April 12, 2019, 12:45:13 pm
I just invented _PETEREV() where I take back every nasty thing I ever said about you. The only glitch is, it's based on the same named VBA function, so just when I'm finished ragging on somebody, it starts all over again from the top!

So when did we become bi, and do we need to add a new gender related keyword to cover this? Funnily enough, I was going to post yesterday that I was considering joining my two instr functions so the user could decide if he wanted to search forward or back.

My thought was to do this...

instrx& = (seed&, stringx$, srch$, dirx%, occ%) Where if dirx% is zero or positive, it searches forward. If negative, it searches backwards. occ% is zero, it searches like INSTR() does now, chop the string at the seed number and start searching from there. If occ% is non-zero, it searches by occurrence.

While I'm not sure if this reduction of parameters in this next example would be a good or bad coding practice, frankly, I'd prefer fewer parameters and would get that by making seed& a string. That's the same numerical length as long (&) so it should work. So...

instrx& = (seedx$, stringx$, srch$) Where the val() of seedx$ is used as the seed, but if the proper letters follow that number, they are processed as: R - reverse if present, forward searching if not. O - Search by occurrence if present. So 5RO would search in reverse for the 5th occurrence of the sub-string in the string.

Anyway, the only thing I don't like about the "O" for occurrence is the fact it looks too much like zero. Too bad it is the best word to describe what is happening, although funnily enough "happenings" also descries occurrences, as does events. Maybe "events" would be better to overcome the whole O looks too much like zero problem.

Any thoughts?

Pete

PS, Yes, I did run your code. Very nice!

Title: Re: Building a better INSTR(TRAP)?
Post by: Jack002 on April 12, 2019, 04:06:41 pm
We have all the tools we need now. I read something _Vince said in another thread. If I code to these new keywords my code is less compatible with other basics and will not work in older environments (my loose interpretation of his words as I read them)

We have instr(). Its powerful. If you work with char delimited strings, use that function, you can find anything.

I wonder if anyone knows what HL7 format is. A sample might look like "MSH|01234|b67|01-01-2019|A new world| v6.6"
It is separated by pipes. (mostly, for me, back in the day) You send a string to instr and ask where all the offsets are for all the pipes, you have all the tools to do a mid$ and print or store that field. Likewise we all know string handling enough to add or replace fields in this system.

HL7 can split one field into subfields with other separators, like this "PID|1|2|3|12-12-1911|This^is^what^subfields^look^like|moo"

I wrote a function that returns the nth field of a string, you give it the delimiter (handy if the delimiter changes) then I wrote another function that calls that function when you need a subfield. There was a function that counts all occurrences of a delimiter, these functions use it.

Also, something I call "quote mode". If you ever delve into CSV edits, you know a field can have quotes around it, so if you delimit your lines on commas and a quote comes along, you have to IGNORE all the coming commas till you see another quote. (I think in the CSV world a CR/LF can trump that, but in the way I last used it, you had to have quotes in pairs) So any function using this should look for the first offset of a quote, find the next one and kind of fence off that area as a "don't look here" area.

I wrote a family tree program and used this format. It was useful because I kept hitting walls saying "how many spaces should I make for a spouce? Children?" I just set it to take all you have, put them all in till you are done. No limit.

So to look at the new QB64 and complain and moan about the new functions makes no sense to me. Use them, don't use them. We can make do on what the old QB gave us, right? I love the mouse reading and the complex keyboard reading we can do now. I have not even started looking at all the screen options. Its far better than the old days of QB.

That's my 2c. Well, 98c

Jack
Title: Re: Building a better INSTR(TRAP)?
Post by: Pete on April 12, 2019, 04:32:24 pm
Quote from: Pete on April 10, 2019, 01:20:51 pm

BTW - I'm not requesting INSTR() be changed. It needs to stay as is, to be QB compatible, of course. I just wanted to throw this idea out for discussion. I mean maybe I'm missing something else the seed, as it functions in INSTR() now, is needed for?

Pete     

@Jack: You did see in my initial post that i'm not advocating for changing INSTR(). I'm just interested in discussion possible usable options for a better INSTR(). I love that BASIC provides building blocks to use a keyword to do a variety of things, but the older I get, the more I just want to get something done and not type all damn day. In other words, it's over being knew to me. I'm not even against drag and drop programming anymore, well, at least for those who already now how to code, but only if the building blocks are also kept available.

Pete
Title: Re: Building a better INSTR(TRAP)?
Post by: Jack002 on April 15, 2019, 11:11:48 am
Yeah Pete, I saw that. I didn't mean that I thought that instr was changing. So far it seems QB64 sure takes the old QB code and works as it did in QB back in the day.
We've been talking about a new function,  instrrev I think.
I write a function once and then use it over as needed. Once its done you just copy it in and treat it like a black box.
I would put them into a library and just call it if I was inclined to learn how to do that.
If you have a routine that works, just add it in when you write a new program. That way you have total control.
For example, you have a delimited string and you want the nth comma offset in it but you want to skip over areas with a quotes in it, you can do that, but if its in some function that the language comes with and doesn't work the way you need it to, you're looking at adding your own code with instr anyway.
Title: Re: Building a better INSTR(TRAP)?
Post by: bplus on April 15, 2019, 02:04:20 pm
Getting the nth item from a string is allot like JB's WORD$() function.

I simulated that function in QB64 for space delimited strings and added more tools that became foundation of my Evaluate sub for interpreters. Here is a demo:

Code: QB64: [Select]
  1. _TITLE "Word tools Demo" 'from: Word tools 2018-03-23.bas"  'B+ 2019-04-15
  2.  
  3. test$ = " 1   2 3  4  5  6  7  8  9  10   11 12 13 14 15   16 "
  4. PRINT "Test string: *"; test$; "*"
  5. test$ = wPrep$(test$)
  6. PRINT: PRINT "Test string run through wPrep$() to remove excess spaces:"
  7. PRINT "*"; test$; "*"
  8. PRINT: PRINT "Test string has"; wCnt(test$); "'words' in it."
  9. PRINT: PRINT "Show every third word in test string:"
  10. FOR i = 3 TO 16 STEP 3
  11.     PRINT "wrd$(test$,"; _TRIM$(STR$(i)); ") = "; Wrd$(test$, i)
  13. PRINT: PRINT "What multiples of 5 are in test string?"
  14. FOR i = 5 TO 20 STEP 5
  15.     IF wIn(test$, _TRIM$(STR$(i))) > 0 THEN PRINT i; "is in test string." ELSE PRINT i; "is NOT in test string."
  17. PRINT: PRINT "Substitute the phrase 'and all the rest...' for words after 10:"
  18. PRINT wSubst$(test$, 11, wCnt(test$), "and all the rest...")
  19.  
  20.  
  21. 'return trimmed  source string s with one space between each word
  22. FUNCTION wPrep$ (ss$)
  23.  
  24.     s$ = LTRIM$(RTRIM$(ss$))
  25.     IF LEN(s$) = 0 THEN wPrep$ = "": EXIT FUNCTION
  26.     'remove all double or more spaces
  27.     p = INSTR(s$, "  ")
  28.     WHILE p > 0
  29.         s$ = MID$(s$, 1, p) + MID$(s$, p + 2, LEN(s$) - p - 1)
  30.         p = INSTR(s$, "  ")
  31.     WEND
  32.     b$ = ""
  33.     FOR i = 1 TO LEN(s$)
  34.         c$ = MID$(s$, i, 1)
  35.         IF ASC(c$) > 31 THEN b$ = b$ + c$
  36.     NEXT
  37.     wPrep$ = b$
  39.  
  40. ' This duplicates JB word(string, wordNumber) base 1, space as default delimiter
  41. ' by returning the Nth word of source string s
  42. ' this function assumes s has been through wPrep
  43. FUNCTION Wrd$ (ss$, wNumber)
  44.     's$ = wPrep(ss$)
  45.     s$ = ss$ 'don't change ss$
  46.     IF LEN(s$) = 0 THEN Wrd$ = "": EXIT FUNCTION
  47.     w$ = "": c = 1
  48.     FOR i = 1 TO LEN(s$)
  49.         IF MID$(s$, i, 1) = " " THEN
  50.             IF c = wNumber THEN Wrd$ = w$: EXIT FUNCTION
  51.             w$ = "": c = c + 1
  52.         ELSE
  53.             w$ = w$ + MID$(s$, i, 1)
  54.         END IF
  55.     NEXT
  56.     IF c <> wNumber THEN Wrd$ = " " ELSE Wrd$ = w$
  58.  
  59. 'This function counts the words in source string s
  60. 'this function assumes s has been thru wPrep
  61. FUNCTION wCnt (s$)
  62.     DIM c AS INTEGER, p AS INTEGER, ip AS INTEGER
  63.     's = wPrep(s)
  64.     IF LEN(s$) = 0 THEN wCnt = 0: EXIT FUNCTION
  65.     c = 1: p = 1: ip = INSTR(p, s$, " ")
  66.     WHILE ip
  67.         c = c + 1: p = ip + 1: ip = INSTR(p, s$, " ")
  68.     WEND
  69.     wCnt = c
  71.  
  72. 'Where is word In source s, 0 = Not In source
  73. 'this function assumes s has been thru wPrep
  74. FUNCTION wIn (s$, wd$)
  75.     DIM wc AS INTEGER, i AS INTEGER
  76.     wc = wCnt(s$): wIn = 0
  77.     FOR i = 1 TO wc
  78.         IF Wrd$(s$, i) = wd$ THEN wIn = i: EXIT FUNCTION
  79.     NEXT
  81.  
  82. ' substitute string in s to replace section first to last words inclusive
  83. 'this function assumes s has been thru wPrep
  84. FUNCTION wSubst$ (s$, first, last, subst$)
  85.     DIM wc AS INTEGER, i AS INTEGER, subF AS INTEGER
  86.     wc = wCnt(s$): b$ = ""
  87.     FOR i = 1 TO wc
  88.         IF first <= i AND i <= last THEN 'do this only once!
  89.             IF subF = 0 THEN b$ = b$ + subst$ + " ": subF = 1
  90.         ELSE
  91.             b$ = b$ + Wrd$(s$, i) + " "
  92.         END IF
  93.     NEXT
  94.     wSubst$ = LTRIM$(RTRIM$(b$))
  96.  
  97.  

With these tools you can have array functions from strings. Getting the index item, resetting the index item, getting the number of items and you can replace whole sections of array/string.

The wrd$() is a very handy trap.

Ha! yeah I had used MID$ instead of INSTR too!
Title: Re: Building a better INSTR(TRAP)?
Post by: Jack002 on April 15, 2019, 04:12:35 pm
MID$ isn't a substitute for INSTR, it works with it. You get the offsets with INSTR, then use them in MID$.

*edit*

I see what you mean now. You tested every byte one at a time for the separator. Whoa. INSTR is your friend. :-)

Code: QB64: [Select]
  1. FUNCTION nth.field$ (HL7$, fldno, sepch$)
  2. ' this fn takes a char delimited $, a field no, and a seperation char
  3. ' and returns a string with either that fld in it or "" if the
  4. ' no of flds is under fldno                    Jack 11/13/99
  5.  
  6. IF count.char(HL7$, sepch$) < fldno THEN
  7.   nth.field$ = ""
  10.  
  11. y = 1  ' find nth sepch$ char
  12. FOR i = 1 TO fldno
  13.   y = INSTR(y, HL7$, sepch$) + 1
  15. ' Found the nth sep, now wheres n+1?
  16. z = INSTR(y, HL7$, sepch$)
  17. ' Fld is z - y bytes long
  18. IF count.char(HL7$, sepch$) = fldno THEN
  19.   nth.field$ = MID$(HL7$, y)
  21.   nth.field$ = MID$(HL7$, y, (z - y))
  23.  
  25.  
  26.  
  27.  
  28. FUNCTION count.char (stringin$, char$)
  29.     ' This fn takes a string and a char and returns the number of chars
  30.     ' in the string.                            Jack 11/13/99
  31.  
  32.     x = 0: y = 1
  33.     WHILE INSTR(y, stringin$, char$)
  34.         t1 = INSTR(y, stringin$, char$)
  35.         y = t1 + 1
  36.         x = x + 1
  37.     WEND
  38.     count.char = x
  39.  
  41.  
Title: Re: Building a better INSTR(TRAP)?
Post by: bplus on April 15, 2019, 04:39:38 pm
Yes, I was building words as I went... yes, INSTR would be faster I am sure!

Probably why I switched over to Split1000:
Code: QB64: [Select]
  1. s$ = ",1  ,2  ,3  ,4  ,5  ,6  ,7  ,8  ,9  ,"
  2. REDIM sa$(0)
  3. Split1000 s$, ",", sa$()
  4. FOR i = LBOUND(sa$) TO UBOUND(s$)
  5.     PRINT i, _TRIM$(sa$(i))
  7.  
  8.  
  9.  
  10. 'notes: REDIM the array(0) to be loaded before calling Split '<<<<<<<<<<<<<<<<<<<<<<< IMPORTANT!!!!
  11. SUB Split1000 (mystr AS STRING, delim AS STRING, arr() AS STRING)
  12.     ' bplus modifications of Galleon fix of Bulrush Split reply #13
  13.     ' http://xmaxw.[abandoned, outdated and now likely malicious qb64 dot net website - don’t go there]/forum/index.php?topic=1612.0
  14.     ' this sub further developed and tested here: \test\Strings\Split test.bas
  15.     DIM copy AS STRING, p AS LONG, curpos AS LONG, arrpos AS LONG, dpos AS LONG
  16.  
  17.     copy = mystr 'make copy since we are messing with mystr when the delimiter is a space
  18.  
  19.     'special case if delim is space, probably want to remove all excess space
  20.     IF delim = " " THEN
  21.         copy = RTRIM$(LTRIM$(copy))
  22.         p = INSTR(copy, "  ")
  23.         WHILE p > 0
  24.             copy = MID$(copy, 1, p - 1) + MID$(copy, p + 1)
  25.             p = INSTR(copy, "  ")
  26.         WEND
  27.     END IF
  28.     curpos = 1
  29.     arrpos = 0
  30.     LD = LEN(delim) 'mod
  31.     dpos = INSTR(curpos, copy, delim)
  32.     DO UNTIL dpos = 0
  33.         arr(arrpos) = MID$(copy, curpos, dpos - curpos)
  34.         arrpos = arrpos + 1
  35.         IF arrpos > UBOUND(arr) THEN REDIM _PRESERVE arr(UBOUND(arr) + 1000) AS STRING
  36.         curpos = dpos + LD
  37.         dpos = INSTR(curpos, copy, delim)
  38.     LOOP
  39.     arr(arrpos) = MID$(copy, curpos)
  40.     REDIM _PRESERVE arr(arrpos) AS STRING 'need this line? YES to get the ubound correct
  42.  
  43.  

BTW who starts and ends a string with delimiters? :D
Title: Re: Building a better INSTR(TRAP)?
Post by: Jack002 on April 15, 2019, 05:21:46 pm
Some people do end the string with a delimiter as a rule. I had cases of that in 1999. I coded for both with and without on my function.

I assume _TRIM$ is a new function in the new QB64? I don't have it on my version
Title: Re: Building a better INSTR(TRAP)?
Post by: bplus on April 15, 2019, 07:22:26 pm
Yes _TRIM$ was tested in last development update and now in stable version 1.3.
Text Only | Text with Attachments