I have a blocklist for my pihole and need to remove some lines

[smokingwheels]

I open my list then compare each line with a array of urls from my whitelist.
The problem is the longer my whitelist is the longer it takes to complete the task.

Should I try 1 url at a time then replace the list for each url?
This means I would need to read and write 200 mb of data for each url.

Currently I have to compare each line to 80 urls that I would like whitelisted or removed from my blocklist
The list is over 6 million lines of text.

Blocklist link
 https://t.co/vRyCbbpcth?amp=1

[FellippeHeitor]

Hope the idea helps. Adapt as required:

Code: QB64: [Select]

1. open "crazy long file.txt" for binary as #1
2. wholefile$ = space$(lof(1))
3. get #1, 1, wholefile$
4. close #1
5.  
6. for i=1 to 80
7.     if instr(wholefile$, arrayOfUrls$(i)) > 0 then
8.         'this url is in the file, do something with it
9.     end if
10. next
11.  

[smokingwheels]

ok will give it a go soon. Thank you

[smokingwheels]

I tried to run an old program but it seems to only work with 2 files open.
I have not converted it yet

Code: QB64: [Select]

1. Rem Updated 7-1-2018
2. Rem runs at 20 kHz
3. Rem 30-1-2018
4. Rem runs at 18.8 kHz
5. Rem About 7 meg after crunchmaster.sh
6. Rem found a space #...AND null string in the same day.
7. Rem fixed some errors 4 NOV 17
8. Timer On
9. On Timer(1) GoSub health
10.  
11. Open "/home/john/Downloads/qb64/pi-hole/Master/w10" For Input As #4
12.  
13. Print "ghjkl"
14. Rem array
15. Dim whitelist$(2000)
16.  
17. Do While Not EOF(4)
18.     i = i + 1
19.  
20.     Print "1234"
21.  
22.     Line Input #4, a$
23.     Print a$
24.     whitelist$(i) = a$
25.     Print Len(whitelist$(i))
26.  
27. Loop
28. Close #4
29.  
30. Open "/home/john/Downloads/qb64/pi-hole/Master/list5" For Input As #1
31. Print "file 1 open"
32. Open "/home/john/Downloads/qb64/pi-hole/Master/piholelist" For Output As #2
33.  
34.  
35.  
36. Rem chomp 2 try to remove null from gravity.
37.  
38. one = 1
39. stack = 578
40. Do While Not EOF(1)
41.     Line Input #1, a$
42.     a = a + 1
43.  
44.     For i = 1 To stack
45.  
46.         wline = Len(whitelist$(i))
47.         Print whitelist$(i)
48.         'PRINT RIGHT$(a$, 10), whitelistline
49.  
50.         If Right$(a$, wline) = Right$(whitelist$(i), wline) Then GoTo skipoutput
51.  
52.     Next
53.     lline = Len(a$)
54.     'IF lline > 71 THEN GOTO skipoutput
55.     Print #2, a$
56.     skipoutput:
57.     Hz = Hz + one
58. Loop
59. Close #1: Close #2: Close #3: Close #4
60. Print a, b
61. Print "Thankyou Happy Hunting..."
62. End
63.  
64. health:
65. Print Hz, a, b
66. Hz = 0
67. Return
68.  
69.