Looking for old program or help recreating it

[STxAxTIC]

Hey @random1

I'll look a little more deeply into your attempts right after I post this, but I want you let you know I'm cracking this problem pretty easily based on a previous work of mine. I want to bring you on board with this. So please consider this random string:

"010010010100100101010100101010101010101010101101101010101001111101010101010101010101001010101010"

Qualitatively, what I tried to do was just mash 1 and 0 in a sloppy left-right fashion. Sure, there are clusters of 0's and 1's, no problem. But the overall dominating pattern, particularly at the end, is alternating 0's and 1's. So, can we make a program to detect that? Absolutely yes. The model is fully general and will detect any pattern at any frequency up to n=5 characters at a time, but one can easily make this bigger. See if you can make sense of the code below. I didn't explain it much here, but the stuff I was saying above is implemented in this code. You can see that there will be numbers accumulating next to common patterns in the test string. Super fast, super systematic, will answer your question... Lemme know what you think before this code gets too far ahead. I'll quit til I hear back.

Code: QB64: [Select]

1. Screen _NewImage(800, 800, 32)
2.  
3. Type Letter
4.     Signature As String
5.     Count As Integer
6. End Type
7.  
8. Dim Shared Alphabet1(2) As Letter
9. Alphabet1(1).Signature = "0"
10. Alphabet1(2).Signature = "1"
11.  
12. Dim Shared Alphabet2(4) As Letter
13. Alphabet2(1).Signature = "00"
14. Alphabet2(2).Signature = "01"
15. Alphabet2(3).Signature = "10"
16. Alphabet2(4).Signature = "11"
17.  
18. Dim Shared Alphabet3(8) As Letter
19. Alphabet3(1).Signature = "000"
20. Alphabet3(2).Signature = "001"
21. Alphabet3(3).Signature = "010"
22. Alphabet3(4).Signature = "011"
23. Alphabet3(5).Signature = "100"
24. Alphabet3(6).Signature = "101"
25. Alphabet3(7).Signature = "110"
26. Alphabet3(8).Signature = "111"
27.  
28. Dim Shared Alphabet4(16) As Letter
29. Alphabet4(1).Signature = "0000"
30. Alphabet4(2).Signature = "0001"
31. Alphabet4(3).Signature = "0010"
32. Alphabet4(4).Signature = "0011"
33. Alphabet4(5).Signature = "0100"
34. Alphabet4(6).Signature = "0101"
35. Alphabet4(7).Signature = "0110"
36. Alphabet4(8).Signature = "0111"
37. Alphabet4(9).Signature = "1000"
38. Alphabet4(10).Signature = "1001"
39. Alphabet4(11).Signature = "1010"
40. Alphabet4(12).Signature = "1011"
41. Alphabet4(13).Signature = "1100"
42. Alphabet4(14).Signature = "1101"
43. Alphabet4(15).Signature = "1110"
44. Alphabet4(16).Signature = "1111"
45.  
46. Dim Shared Alphabet5(32) As Letter
47. Alphabet5(1).Signature = "00000"
48. Alphabet5(2).Signature = "00001"
49. Alphabet5(3).Signature = "00010"
50. Alphabet5(4).Signature = "00011"
51. Alphabet5(5).Signature = "00100"
52. Alphabet5(6).Signature = "00101"
53. Alphabet5(7).Signature = "00110"
54. Alphabet5(8).Signature = "00111"
55. Alphabet5(9).Signature = "01000"
56. Alphabet5(10).Signature = "01001"
57. Alphabet5(11).Signature = "01010"
58. Alphabet5(12).Signature = "01011"
59. Alphabet5(13).Signature = "01100"
60. Alphabet5(14).Signature = "01101"
61. Alphabet5(15).Signature = "01110"
62. Alphabet5(16).Signature = "01111"
63. Alphabet5(17).Signature = "10000"
64. Alphabet5(18).Signature = "10001"
65. Alphabet5(19).Signature = "10010"
66. Alphabet5(20).Signature = "10011"
67. Alphabet5(21).Signature = "10100"
68. Alphabet5(22).Signature = "10101"
69. Alphabet5(23).Signature = "10110"
70. Alphabet5(24).Signature = "10111"
71. Alphabet5(25).Signature = "11000"
72. Alphabet5(26).Signature = "11001"
73. Alphabet5(27).Signature = "11010"
74. Alphabet5(28).Signature = "11011"
75. Alphabet5(29).Signature = "11100"
76. Alphabet5(30).Signature = "11101"
77. Alphabet5(31).Signature = "11110"
78. Alphabet5(32).Signature = "11111"
79.  
80.  
81. Call ResetCount
82.  
83. Dim FingerPrint(6) As String
84.  
85. FingerPrint(1) = "010010010100100101010100101010101010101010101101101010101001111101010101010101010101001010101010"
86.  
87. For n = 2 To UBound(FingerPrint)
88.     FingerPrint(n) = Right$(FingerPrint(n - 1), Len(FingerPrint(n - 1)) - 1) + Left$(FingerPrint(n - 1), 1)
89. Next
90.  
91. For m = 1 To 2
92.     For n = 1 To Len(FingerPrint(m)) Step 2
93.         a$ = Mid$(FingerPrint(m), n, 2)
94.         For k = 1 To UBound(Alphabet2)
95.             If a$ = Alphabet2(k).Signature Then
96.                 Alphabet2(k).Count = Alphabet2(k).Count + 1
97.             End If
98.         Next
99.     Next
100.  
101.     For k = 1 To UBound(Alphabet2)
102.         Locate k, 1 + 12 * (m - 1)
103.         Print Alphabet2(k).Signature; Alphabet2(k).Count
104.     Next
105.  
106.     Sleep
107. Next
108.  
109. Cls
110.  
111. For m = 1 To 3
112.     For n = 1 To Len(FingerPrint(m)) Step 3
113.         a$ = Mid$(FingerPrint(m), n, 3)
114.         For k = 1 To UBound(Alphabet3)
115.             If a$ = Alphabet3(k).Signature Then
116.                 Alphabet3(k).Count = Alphabet3(k).Count + 1
117.             End If
118.         Next
119.     Next
120.  
121.     For k = 1 To UBound(Alphabet3)
122.         Locate k, 1 + 12 * (m - 1)
123.         Print Alphabet3(k).Signature; Alphabet3(k).Count
124.     Next
125.  
126.     Sleep
127. Next
128.  
129. Cls
130.  
131. For m = 1 To 4
132.     For n = 1 To Len(FingerPrint(m)) Step 4
133.         a$ = Mid$(FingerPrint(m), n, 4)
134.         For k = 1 To UBound(Alphabet4)
135.             If a$ = Alphabet4(k).Signature Then
136.                 Alphabet4(k).Count = Alphabet4(k).Count + 1
137.             End If
138.         Next
139.     Next
140.  
141.     For k = 1 To UBound(Alphabet4)
142.         Locate k, 1 + 12 * (m - 1)
143.         Print Alphabet4(k).Signature; Alphabet4(k).Count
144.     Next
145.  
146.     Sleep
147. Next
148.  
149. Cls
150.  
151. For m = 1 To 5
152.     For n = 1 To Len(FingerPrint(m)) Step 5
153.         a$ = Mid$(FingerPrint(m), n, 5)
154.         For k = 1 To UBound(Alphabet5)
155.             If a$ = Alphabet5(k).Signature Then
156.                 Alphabet5(k).Count = Alphabet5(k).Count + 1
157.             End If
158.         Next
159.     Next
160.  
161.     For k = 1 To UBound(Alphabet5)
162.         Locate k, 1 + 12 * (m - 1)
163.         Print Alphabet5(k).Signature; Alphabet5(k).Count
164.     Next
165.  
166.     Sleep
167. Next
168.  
169. End
170.  
171. Sub ResetCount
172.     For n = 1 To UBound(Alphabet1)
173.         Alphabet1(n).Count = 0
174.     Next
175.     For n = 1 To UBound(Alphabet2)
176.         Alphabet2(n).Count = 0
177.     Next
178.     For n = 1 To UBound(Alphabet3)
179.         Alphabet3(n).Count = 0
180.     Next
181.     For n = 1 To UBound(Alphabet4)
182.         Alphabet4(n).Count = 0
183.     Next
184. End Sub

[STxAxTIC]

Alright, I couldn't resist running your sample data string into my program. I can see you picked a weird sequence, but I highlighted the most indicative numbers (as of now)... see if any of that seems useful, seeing how you studied that string extensively, I bet:

Code: [Select]

0001000001011100000011000100011000010010000000000000000011000000000000000001110010000000000001111000000001001110000000000000100000000011110000000000000000000000000000010100000000000000000100000000000000110001111000010001000000000000000000011010000000000000000010001101100000100001000000110011000011010010000000000000000000100001000011000110001010000000100000000000000000000000000000000000010000000000100000000000000010100000100000000000000000000000000000000000100001000000000000000010110100000000000010010000000000000000000100000000001100000011000000000000000000000000000000110010000110000100000   
  [ You are not allowed to view this attachment ]  

Oh, and if I was to finish coding the program, I would have it predict a 0 for the next digit.

[random1]

I feel I need to elaborate a little more.  Look at the second image and pay attention
to the 0110 patterns within the results string that are highlighted in green.  Counting
the one I missed there are 9.  The data flows from right to left with the latest entries
on the left.  If we look at the results string from right to left we notice that the following
digit after each 0110 pattern is a (1).  This is what I would classify as an Expert. 

I would have no reservations predicting digit 1 as the next value in the string.  Each
string has to be analyzed to try and find it's own expert.  And each expert should be
assigned a confidence level between 0 and .99.  It's also that more than one expert
can be found through our analysis.  The search pattern should always be taken from
the left most part of the string because that's what we are looking for to make the
next prediction.   The experts if any will change day to day most of the time so it's
important to be able to find them on the fly so to say. 

The possibilities are almost limitless as to what we might use to construct a prediction
model so try to think outside the box.  Anyway, I hope the information I have presented
here helps to kick start other ideas.

R1     
           

[STxAxTIC]

I'll keep looking at what you're attempting but do you not agree that the program above assigns "0110" a somewhat high "score"? (It would be on the previous screen, not in the screenshot itself, which has the 5 digit analysis.) There are few more common strings, such as "0000", so that truth is up there too.

Let me ask, what is the end result of this? You want an Eliza-like program that tries to guess what my next number is, and then honestly reports the guess after I finally make my choice? Cause that's where I thought you were going.

[random1]

 STxAxTIC

I will have to look up the string I posted to see what the next value was as it's been
updated a couple times since I posted it.  I should have kept a note of which string I
used.  My program copies each string I look at to the clipboard automatically and I
just selected one at random.  If/when you finish your code so that it makes a 0 or 1
prediction I will run it against all 800 + strings and report back to it's overall hit rate.
Many thanks for posting.

R1

[random1]

The main goal is to produce a universal prediction tool.  The 0110 is nothing more than
the values of the last 4 entries.  The string changes day to day and when using the first
4 digits then we have 16 possibilities but each day the sequence is determined by the
events I track.  Each string is updated daily by adding a 0 or 1 to the string.  if the event
accrued then a 1 is added if not then a 0.  The reason I mentioned the old program  was
my interest in the algorithm it used.  I was wondering how it would stack up as a predictor
against my data.  Instead of me imputing one digit at a time I would feed it a string of
3k to 5k in length and see how successful it was.   It seems to me if the method it used
could predict my next choice then given enough data it might be useful to my current
project.  I am not building a game, I wanted to repurpose the code into something useful. 
 
Sorry if I was not clear in this.

R1

[random1]

P.S.  I track over 800 events each day so I have over 800 strings that I try to predict.
Standard prediction tools don't work, I've tried all sorts of methods and figure I needed
to start thinking outside the box if it's going to be successful.

Anyway, that's where I am at.   My last attempt failed and it was sometimes hitting
over 80% correct.  The problem was that the tool was not consistent enough to meet
my needs. This has been a ongoing project that I have been working on for years.

R1   

[STxAxTIC]

I may have to do a video or something on this because the details are getting a bit hard to convey. Anyway, I updated the code to be much more powerful, and I'm attaching a screenshot that outlines one possible good analysis. If we want to continue this, we need to pick a dataset that we'll both use and come up with definite questions about the data that we could use our methods to answer. (Statistical stuff can't really be trusted.) Anyway, here's where it stands on my end. I may do something less lazy than a weighted average at the end, but for now it's getting there:

Code: QB64: [Select]

1. Type Letter
2.     Signature As String
3.     Count As Integer
4. End Type
5.  
6. Dim Shared Alphabet1(2) As Letter
7. Alphabet1(1).Signature = "0"
8. Alphabet1(2).Signature = "1"
9.  
10. Dim Shared Alphabet2(4) As Letter
11. Dim Shared Alphabet3(8) As Letter
12. Dim Shared Alphabet4(16) As Letter
13. Dim Shared Alphabet5(32) As Letter
14. Dim Shared Alphabet6(64) As Letter
15. Dim Shared Alphabet7(128) As Letter
16. Dim Shared Alphabet8(256) As Letter
17.  
18. Call NewAlphabet(Alphabet1(), Alphabet2())
19. Call NewAlphabet(Alphabet2(), Alphabet3())
20. Call NewAlphabet(Alphabet3(), Alphabet4())
21. Call NewAlphabet(Alphabet4(), Alphabet5())
22. Call NewAlphabet(Alphabet5(), Alphabet6())
23. Call NewAlphabet(Alphabet6(), Alphabet7())
24. Call NewAlphabet(Alphabet7(), Alphabet8())
25.  
26. Dim Shared FingerPrint(16) As String
27.  
28. 'FingerPrint(1) = "010010010100100101010100101010101010101010101101101010101001111101010101010101010101001010101010"
29. FingerPrint(1) = "0001000001011100000011000100011000010010000000000000000011000000000000000001110010000000000001111000000001001110000000000000100000000011110000000000000000000000000000010100000000000000000100000000000000110001111000010001000000000000000000011010000000000000000010001101100000100001000000110011000011010010000000000000000000100001000011000110001010000000100000000000000000000000000000000000010000000000100000000000000010100000100000000000000000000000000000000000100001000000000000000010110100000000000010010000000000000000000100000000001100000011000000000000000000000000000000110010000110000100000"
30.  
31. For n = 2 To UBound(FingerPrint)
32.     FingerPrint(n) = Right$(FingerPrint(n - 1), Len(FingerPrint(n - 1)) - 1) + Left$(FingerPrint(n - 1), 1)
33. Next
34.  
35. Call UpdateHisto(Alphabet2(), 2)
36. Call PrintHisto(Alphabet2())
37.  
38. Call UpdateHisto(Alphabet3(), 3)
39. Call PrintHisto(Alphabet3())
40.  
41. Call UpdateHisto(Alphabet4(), 4)
42. Call PrintHisto(Alphabet4())
43.  
44. Call UpdateHisto(Alphabet5(), 5)
45. Call PrintHisto(Alphabet5())
46.  
47. Call UpdateHisto(Alphabet6(), 6)
48. Call PrintHisto(Alphabet6())
49.  
50. Call UpdateHisto(Alphabet7(), 7)
51. Call PrintHisto(Alphabet7())
52.  
53. Call UpdateHisto(Alphabet8(), 8)
54. Call PrintHisto(Alphabet8())
55.  
56. End
57.  
58. Sub UpdateHisto (arr() As Letter, w As Integer)
59.     For m = 1 To w
60.         For n = 1 To Len(FingerPrint(m)) Step w
61.             a$ = Mid$(FingerPrint(m), n, w)
62.             For k = 1 To UBound(arr)
63.                 If a$ = arr(k).Signature Then
64.                     arr(k).Count = arr(k).Count + 1
65.                 End If
66.             Next
67.         Next
68.     Next
69. End Sub
70.  
71. Sub PrintHisto (arr() As Letter)
72.     Call BubbleSort(arr())
73.     For k = UBound(arr) To 1 Step -1
74.         Print arr(k).Signature; arr(k).Count
75.     Next
76.     Sleep
77.     Cls
78. End Sub
79.  
80. Sub NewAlphabet (arrold() As Letter, arrnew() As Letter)
81.     n = 0
82.     For j = 1 To 2
83.         For k = 1 To UBound(arrold)
84.             n = n + 1
85.             arrnew(n).Signature = arrold(k).Signature
86.         Next
87.     Next
88.     For k = 1 To UBound(arrnew)
89.         If k <= UBound(arrnew) / 2 Then
90.             arrnew(k).Signature = "0" + arrnew(k).Signature
91.         Else
92.             arrnew(k).Signature = "1" + arrnew(k).Signature
93.         End If
94.     Next
95. End Sub
96.  
97. Sub BubbleSort (arr() As Letter)
98.     Dim As Integer i, j
99.     Dim As Double u, v
100.     For j = UBound(arr) To 1 Step -1
101.         For i = 2 To UBound(arr)
102.             u = arr(i - 1).Count
103.             v = arr(i).Count
104.             If (u < v) Then
105.                 Swap arr(i - 1), arr(i)
106.             End If
107.         Next
108.     Next
109. End Sub

  [ You are not allowed to view this attachment ]  

[SMcNeill]

What you're trying to do, it sounds like, is basically what this guy over at Numberphile talks about.  Take a look at the video.  I think you might like it and it might give you some insight into what your program needs to have to be a human sequence tester.

[STxAxTIC]

Yeah, this guy's ^ in the same ballpark, mostly. This whole topic does make me sigh just a little, like am I the only one who remembers doing work like this with you guys (Steve, bplus) like 2 years ago? We were able to detect ordered strings hidden in a bunch of randomness. That old problem, the video above, the present problem - all the same ballpark. In fact, the topic of the video is quite a bit simpler than the thing we're doing. (Just run the guy's string of H and T through my code and see the uneven distributions.) Anyway, thanks for bringing that up Steve. I'm gonna keep playing this this 'cause I'm not sure if I'm being thorough enough in my explanations.

[STxAxTIC]

Okay so @random1...

For the sake of setting the bar somewhere, consider this data set:

Code: [Select]

string = "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000": nextdigit = "0"
'
string = "01010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101": nextdigit = "0"
string = "10101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010": nextdigit = "1"
'
string = "00100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100": nextdigit = "1"
string = "01001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001": nextdigit = "0"
string = "10010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010": nextdigit = "0"
'
string = "00010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001": nextdigit = "0"
string = "00100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010": nextdigit = "0"
string = "01000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100": nextdigit = "0"
string = "10001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000": nextdigit = "1"
'
string = "00001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000": nextdigit = "0"
string = "00010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000": nextdigit = "1"
string = "00100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001": nextdigit = "0"
string = "01000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010": nextdigit = "0"
string = "10000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100": nextdigit = "0"
'
string = "00000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100": nextdigit = "0"
string = "00001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000": nextdigit = "0"
string = "00010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000": nextdigit = "0"
string = "00100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000": nextdigit = "1"
string = "01000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001": nextdigit = "0"
string = "10000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010": nextdigit = "0"
'
string = "00000010000001000000100000010000001000000100000010000001000000100000010000001000000100000010000001000000100000010000001000000100": nextdigit = "0"
string = "00100000010000001000000100000010000001000000100000010000001000000100000010000001000000100000010000001000000100000010000001000000": nextdigit = "1"
'
string = "00000001000000010000000100000001000000010000000100000001000000010000000100000001000000010000000100000001000000010000000100000001": nextdigit = "0"
string = "10000000100000001000000010000000100000001000000010000000100000001000000010000000100000001000000010000000100000001000000010000000": nextdigit = "1"
'
string = "00110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011": nextdigit = "0"
string = "01100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110": nextdigit = "0"
string = "11001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100": nextdigit = "1"
string = "10011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001": nextdigit = "1"
'
string = "00011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100": nextdigit = "0"
string = "00111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000": nextdigit = "1"
string = "01110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001": nextdigit = "1"
string = "11100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011": nextdigit = "1"
string = "11000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111": nextdigit = "0"
string = "10001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110": nextdigit = "0"
'
string = "01001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100": nextdigit = "0"
string = "10011000111010011000111010011000111010011000111010011000111010011000111010011000111010011000111010011000111010011000111010011000": nextdigit = "1"
string = "00110001110100110001110100110001110100110001110100110001110100110001110100110001110100110001110100110001110100110001110100110001": nextdigit = "1"
string = "01100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011": nextdigit = "1"
string = "11000111010011000111010011000111010011000111010011000111010011000111010011000111010011000111010011000111010011000111010011000111": nextdigit = "0"
string = "10001110100110001110100110001110100110001110100110001110100110001110100110001110100110001110100110001110100110001110100110001110": nextdigit = "1"
string = "00011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101": nextdigit = "0"
'
string = "01001010101010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010101010001110100101010101001": nextdigit = "0"
string = "10010101010100101010100011101001010101010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010": nextdigit = "1"
string = "00101010101001010101000111010010101010100101010100011101001010101010010101010001110100101010101001010101000111010010101010100101": nextdigit = "0"
string = "01010101010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010101010001110100101010101001010": nextdigit = "1"
string = "10101010100101010100011101001010101010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010101": nextdigit = "0"
string = "01010101001010101000111010010101010100101010100011101001010101010010101010001110100101010101001010101000111010010101010100101010": nextdigit = "1"
string = "10101010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010101010001110100101010101001010101": nextdigit = "0"
string = "01010100101010100011101001010101010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010101010": nextdigit = "0"
string = "10101001010101000111010010101010100101010100011101001010101010010101010001110100101010101001010101000111010010101010100101010100": nextdigit = "0"
string = "01010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010101010001110100101010101001010101000": nextdigit = "1"
string = "10100101010100011101001010101010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010101010001": nextdigit = "1"
What we've got is a nice bunch of patterns of zero and ones. In each case, even for the ugly later cases, there is always a pattern going on, and I specified the correct next digit in a separate string. This is the kind of data set I would need to test a method I would come up with. Come up with a guess, square it off against the known result. So before reading on, my fast question to you is: How well will your methods predict the so-called "next digit" in each of these? Will that take long? Give it a shot for me.

-----

The code below gets every single one of those examples correct, with the exception of *one* case, and that's because I had to nerf my code down to make sure it wasn't somehow giving an "always right" bug. I needed to see *one* mistake take place, so I contrived a case that I knew could trick it (it takes place right before the end). Alas, it doesn't need to be this way, there is no flaw in the method that I can see - it'll just have to run slightly slower to be 100% perfect. But anyway, here is the bar. Call it 99% "perfect" for the data being used:

Code: QB64: [Select]

1. Type Letter
2.     Signature As String
3.     Count As Integer
4. End Type
5.  
6. Dim Shared FingerPrint(16) As String
7.  
8. Dim Shared Alphabet1(2) As Letter
9. Alphabet1(1).Signature = "0"
10. Alphabet1(2).Signature = "1"
11.  
12. Dim Shared Alphabet2(4) As Letter
13. Dim Shared Alphabet3(8) As Letter
14. Dim Shared Alphabet4(16) As Letter
15. Dim Shared Alphabet5(32) As Letter
16. Dim Shared Alphabet6(64) As Letter
17. Dim Shared Alphabet7(128) As Letter
18. Dim Shared Alphabet8(256) As Letter
19. Dim Shared Alphabet9(512) As Letter
20. Dim Shared Alphabet10(1024) As Letter
21. Dim Shared Alphabet11(2048) As Letter
22. Dim Shared Alphabet12(4096) As Letter
23. 'Dim Shared Alphabet13(8192) As Letter
24.  
25. Call NewAlphabet(Alphabet1(), Alphabet2())
26. Call NewAlphabet(Alphabet2(), Alphabet3())
27. Call NewAlphabet(Alphabet3(), Alphabet4())
28. Call NewAlphabet(Alphabet4(), Alphabet5())
29. Call NewAlphabet(Alphabet5(), Alphabet6())
30. Call NewAlphabet(Alphabet6(), Alphabet7())
31. Call NewAlphabet(Alphabet7(), Alphabet8())
32. Call NewAlphabet(Alphabet8(), Alphabet9())
33. Call NewAlphabet(Alphabet9(), Alphabet10())
34. Call NewAlphabet(Alphabet10(), Alphabet11())
35. Call NewAlphabet(Alphabet11(), Alphabet12())
36. 'Call NewAlphabet(Alphabet12(), Alphabet13())
37.  
38. Dim n As Integer
39. n = 0
40. ReDim Shared TheInput(1000, 2) As String
41.  
42. n = n + 1: TheInput(n, 1) = "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000": TheInput(n, 2) = "0"
43. '
44. n = n + 1: TheInput(n, 1) = "01010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101": TheInput(n, 2) = "0"
45. n = n + 1: TheInput(n, 1) = "10101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010": TheInput(n, 2) = "1"
46. '
47. n = n + 1: TheInput(n, 1) = "00100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100100": TheInput(n, 2) = "1"
48. n = n + 1: TheInput(n, 1) = "01001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001001": TheInput(n, 2) = "0"
49. n = n + 1: TheInput(n, 1) = "10010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010010": TheInput(n, 2) = "0"
50. '
51. n = n + 1: TheInput(n, 1) = "00010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001": TheInput(n, 2) = "0"
52. n = n + 1: TheInput(n, 1) = "00100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010": TheInput(n, 2) = "0"
53. n = n + 1: TheInput(n, 1) = "01000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100": TheInput(n, 2) = "0"
54. n = n + 1: TheInput(n, 1) = "10001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000100010001000": TheInput(n, 2) = "1"
55. '
56. n = n + 1: TheInput(n, 1) = "00001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000": TheInput(n, 2) = "0"
57. n = n + 1: TheInput(n, 1) = "00010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000": TheInput(n, 2) = "1"
58. n = n + 1: TheInput(n, 1) = "00100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001": TheInput(n, 2) = "0"
59. n = n + 1: TheInput(n, 1) = "01000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010": TheInput(n, 2) = "0"
60. n = n + 1: TheInput(n, 1) = "10000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100001000010000100": TheInput(n, 2) = "0"
61. '
62. n = n + 1: TheInput(n, 1) = "00000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100": TheInput(n, 2) = "0"
63. n = n + 1: TheInput(n, 1) = "00001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000": TheInput(n, 2) = "0"
64. n = n + 1: TheInput(n, 1) = "00010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000": TheInput(n, 2) = "0"
65. n = n + 1: TheInput(n, 1) = "00100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000100000": TheInput(n, 2) = "1"
66. n = n + 1: TheInput(n, 1) = "01000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001000001": TheInput(n, 2) = "0"
67. n = n + 1: TheInput(n, 1) = "10000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010000010": TheInput(n, 2) = "0"
68. '
69. n = n + 1: TheInput(n, 1) = "00000010000001000000100000010000001000000100000010000001000000100000010000001000000100000010000001000000100000010000001000000100": TheInput(n, 2) = "0"
70. n = n + 1: TheInput(n, 1) = "00100000010000001000000100000010000001000000100000010000001000000100000010000001000000100000010000001000000100000010000001000000": TheInput(n, 2) = "1"
71. '
72. n = n + 1: TheInput(n, 1) = "00000001000000010000000100000001000000010000000100000001000000010000000100000001000000010000000100000001000000010000000100000001": TheInput(n, 2) = "0"
73. n = n + 1: TheInput(n, 1) = "10000000100000001000000010000000100000001000000010000000100000001000000010000000100000001000000010000000100000001000000010000000": TheInput(n, 2) = "1"
74. '
75. n = n + 1: TheInput(n, 1) = "00110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011": TheInput(n, 2) = "0"
76. n = n + 1: TheInput(n, 1) = "01100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110": TheInput(n, 2) = "0"
77. n = n + 1: TheInput(n, 1) = "11001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100": TheInput(n, 2) = "1"
78. n = n + 1: TheInput(n, 1) = "10011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001100110011001": TheInput(n, 2) = "1"
79. '
80. n = n + 1: TheInput(n, 1) = "00011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100": TheInput(n, 2) = "0"
81. n = n + 1: TheInput(n, 1) = "00111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000": TheInput(n, 2) = "1"
82. n = n + 1: TheInput(n, 1) = "01110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001": TheInput(n, 2) = "1"
83. n = n + 1: TheInput(n, 1) = "11100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011100011": TheInput(n, 2) = "1"
84. n = n + 1: TheInput(n, 1) = "11000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111000111": TheInput(n, 2) = "0"
85. n = n + 1: TheInput(n, 1) = "10001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110001110": TheInput(n, 2) = "0"
86. '
87. n = n + 1: TheInput(n, 1) = "01001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100": TheInput(n, 2) = "0"
88. n = n + 1: TheInput(n, 1) = "10011000111010011000111010011000111010011000111010011000111010011000111010011000111010011000111010011000111010011000111010011000": TheInput(n, 2) = "1"
89. n = n + 1: TheInput(n, 1) = "00110001110100110001110100110001110100110001110100110001110100110001110100110001110100110001110100110001110100110001110100110001": TheInput(n, 2) = "1"
90. n = n + 1: TheInput(n, 1) = "01100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011": TheInput(n, 2) = "1"
91. n = n + 1: TheInput(n, 1) = "11000111010011000111010011000111010011000111010011000111010011000111010011000111010011000111010011000111010011000111010011000111": TheInput(n, 2) = "0"
92. n = n + 1: TheInput(n, 1) = "10001110100110001110100110001110100110001110100110001110100110001110100110001110100110001110100110001110100110001110100110001110": TheInput(n, 2) = "1"
93. n = n + 1: TheInput(n, 1) = "00011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101001100011101": TheInput(n, 2) = "0"
94. '
95. n = n + 1: TheInput(n, 1) = "01001010101010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010101010001110100101010101001": TheInput(n, 2) = "0" '0
96. n = n + 1: TheInput(n, 1) = "10010101010100101010100011101001010101010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010": TheInput(n, 2) = "1" '1
97. n = n + 1: TheInput(n, 1) = "00101010101001010101000111010010101010100101010100011101001010101010010101010001110100101010101001010101000111010010101010100101": TheInput(n, 2) = "0" '0
98. n = n + 1: TheInput(n, 1) = "01010101010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010101010001110100101010101001010": TheInput(n, 2) = "1" '1
99. n = n + 1: TheInput(n, 1) = "10101010100101010100011101001010101010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010101": TheInput(n, 2) = "0" '0
100. n = n + 1: TheInput(n, 1) = "01010101001010101000111010010101010100101010100011101001010101010010101010001110100101010101001010101000111010010101010100101010": TheInput(n, 2) = "1" '1
101. n = n + 1: TheInput(n, 1) = "10101010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010101010001110100101010101001010101": TheInput(n, 2) = "0" '0
102. n = n + 1: TheInput(n, 1) = "01010100101010100011101001010101010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010101010": TheInput(n, 2) = "0" '0
103. n = n + 1: TheInput(n, 1) = "10101001010101000111010010101010100101010100011101001010101010010101010001110100101010101001010101000111010010101010100101010100": TheInput(n, 2) = "0" '0
104. n = n + 1: TheInput(n, 1) = "01010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010101010001110100101010101001010101000": TheInput(n, 2) = "1" '1
105. n = n + 1: TheInput(n, 1) = "10100101010100011101001010101010010101010001110100101010101001010101000111010010101010100101010100011101001010101010010101010001": TheInput(n, 2) = "1" '1
106. '
107. n = n + 1: TheInput(n, 1) = "0001000001011100000011000100011000010010000000000000000011000000000000000001110010000000000001111000000001001110000000000000100000000011110000000000000000000000000000010100000000000000000100000000000000110001111000010001000000000000000000011010000000000000000010001101100000100001000000110011000011010010000000000000000000100001000011000110001010000000100000000000000000000000000000000000010000000000100000000000000010100000100000000000000000000000000000000000100001000000000000000010110100000000000010010000000000000000000100000000001100000011000000000000000000000000000000110010000110000100000": TheInput(n, 2) = "?"
108. '
109.  
110. For m = 1 To n
111.  
112.     FingerPrint(1) = TheInput(m, 1)
113.  
114.     For k = 2 To UBound(FingerPrint)
115.         FingerPrint(k) = Right$(FingerPrint(k - 1), Len(FingerPrint(k - 1)) - 1) + Left$(FingerPrint(k - 1), 1)
116.     Next
117.  
118.     Call CreateHisto(Alphabet2(), 2)
119.     'Call PrintHisto(Alphabet2())
120.     p2 = MakeGuess(Alphabet2(), 2)
121.  
122.     Call CreateHisto(Alphabet3(), 3)
123.     'Call PrintHisto(Alphabet3())
124.     p3 = MakeGuess(Alphabet3(), 3)
125.  
126.     Call CreateHisto(Alphabet4(), 4)
127.     'Call PrintHisto(Alphabet4())
128.     p4 = MakeGuess(Alphabet4(), 4)
129.  
130.     Call CreateHisto(Alphabet5(), 5)
131.     'Call PrintHisto(Alphabet5())
132.     p5 = MakeGuess(Alphabet5(), 5)
133.  
134.     Call CreateHisto(Alphabet6(), 6)
135.     'Call PrintHisto(Alphabet6())
136.     p6 = MakeGuess(Alphabet6(), 6)
137.  
138.     Call CreateHisto(Alphabet7(), 7)
139.     'Call PrintHisto(Alphabet7())
140.     p7 = MakeGuess(Alphabet7(), 7)
141.  
142.     Call CreateHisto(Alphabet8(), 8)
143.     'Call PrintHisto(Alphabet8())
144.     p8 = MakeGuess(Alphabet8(), 8)
145.  
146.     Call CreateHisto(Alphabet9(), 9)
147.     'Call PrintHisto(Alphabet9())
148.     p9 = MakeGuess(Alphabet9(), 9)
149.  
150.     Call CreateHisto(Alphabet10(), 10)
151.     'Call PrintHisto(Alphabet10())
152.     p10 = MakeGuess(Alphabet10(), 10)
153.  
154.     Call CreateHisto(Alphabet11(), 11)
155.     'Call PrintHisto(Alphabet11())
156.     p11 = MakeGuess(Alphabet11(), 11)
157.  
158.     Call CreateHisto(Alphabet12(), 12)
159.     'Call PrintHisto(Alphabet12())
160.     p12 = MakeGuess(Alphabet12(), 12)
161.  
162.     'Call CreateHisto(Alphabet13(), 13)
163.     'Call PrintHisto(Alphabet13())
164.     'p13 = MakeGuess(Alphabet13(), 13)
165.  
166.     Cls
167.     Print "String:"
168.     Print FingerPrint(1)
169.     Print
170.     Print "Predicted:"; p12
171.     Print "Actual:    "; TheInput(m, 2)
172.     'Print
173.     'Print p2; p3; p4; p5; p6; p7; p8; p9; p10; p11; p12; p13
174.     Sleep
175.  
176. Next
177.  
178. End
179.  
180. Function MakeGuess (arr() As Letter, w As Integer)
181.     Dim TheReturn As Integer
182.     x$ = Right$(FingerPrint(1), w - 1)
183.     For k = 1 To UBound(arr)
184.         If Left$(arr(k).Signature, w - 1) = x$ Then
185.             'Print "..."; arr(k).Signature
186.             TheReturn = Val(Right$(arr(k).Signature, 1))
187.             Exit For
188.         End If
189.     Next
190.     MakeGuess = TheReturn
191. End Function
192.  
193. Sub CreateHisto (arr() As Letter, w As Integer)
194.     For m = 1 To w
195.         For n = 1 To Len(FingerPrint(m)) Step w
196.             a$ = Mid$(FingerPrint(m), n, w)
197.             For k = 1 To UBound(arr)
198.                 If a$ = arr(k).Signature Then
199.                     arr(k).Count = arr(k).Count + 1
200.                 End If
201.             Next
202.         Next
203.     Next
204.     Call BubbleSort(arr())
205. End Sub
206.  
207. Sub PrintHisto (arr() As Letter)
208.     For k = 1 To 3
209.         Print arr(k).Signature; arr(k).Count
210.     Next
211. End Sub
212.  
213. Sub NewAlphabet (arrold() As Letter, arrnew() As Letter)
214.     n = 0
215.     For j = 1 To 2
216.         For k = 1 To UBound(arrold)
217.             n = n + 1
218.             arrnew(n).Signature = arrold(k).Signature
219.         Next
220.     Next
221.     For k = 1 To UBound(arrnew)
222.         If k <= UBound(arrnew) / 2 Then
223.             arrnew(k).Signature = "0" + arrnew(k).Signature
224.         Else
225.             arrnew(k).Signature = "1" + arrnew(k).Signature
226.         End If
227.     Next
228. End Sub
229.  
230. Sub BubbleSort (arr() As Letter)
231.     Dim As Integer i, j
232.     Dim As Double u, v
233.     For j = UBound(arr) To 1 Step -1
234.         For i = 2 To UBound(arr)
235.             u = arr(i - 1).Count
236.             v = arr(i).Count
237.             If (u < v) Then
238.                 Swap arr(i - 1), arr(i)
239.             End If
240.         Next
241.     Next
242. End Sub

[random1]

SMcNeill

The video you posted is on target and a big goal is avoiding the gamblers fallacy like a plague.

Thanks for posting 
 



 

[random1]

STxAxTIC

The strings you show do not have any random elements except for the mentioned.
My approach at this would be a basic counter loop with a quick decimal to binary
converter.  Use an array to store the counts.  Running averages is something I use
but by itself it's not enough to make a solid prediction when the data can be some-
what random.   The video SMcNeill posted shows where I am at and also explains
where the 3rd state comes in, a wild card that skips whenever no clear prediction
can be made. 
Please don't think I am downplaying your input, I am not.  Lets say that we start
with the right most digit in the string and from that value we try to predict the
digit to the left of it.  We then check the prediction against the second leftmost
digit.  Next we add the second digit to the first so that we now have 2 digits to
use in making the next prediction and so on and on. 
If the data does not contain any random data, then as soon as the length of the
pattern is determined it's a no brain'er from that point.  The direction I am headed
is to create a self adapting algorithm to minimize the error, not exactly a neural
network but close.  It seems simple enough until one starts coding.  My thoughts
are to combine several such algorithms into a single predictor using all the outputs
to make the prediction.  The running average is already part of the project.   The
speed of the code is not that important as long as were not talking hours, my current
code takes around 45 seconds to process 800+ strings which includes 2 graphics.

Picture
Green graph shows single string process and pink shows overrall

https://i.postimg.cc/9fqHZchS/splash.png

R1   
 
 

[Dimster]

Just trying to determine if you are trying to predict an event or predict the behavior of a person?

The main goal is to produce a universal prediction tool.  The 0110 is nothing more than
the values of the last 4 entries.  The string changes day to day and when using the first
4 digits then we have 16 possibilities but each day the sequence is determined by the
events I track.  Each string is updated daily by adding a 0 or 1 to the string.  if the event
accrued then a 1 is added if not then a 0.

Events of nature are truly random but a persons behavior may be predictable.

[STxAxTIC]

Yeah, I kinda figured this would happen.

random1, my method works on random strings also - of course - we need to TEST things with regularized input before we give the algorithm true unknowns.

IF you can't run your stuff on my test data, or won't, the way it really sounds, then Im considering this issue closed and you guys have some reading to do!

Please, let's see how your attempt handles the data I provided. No speculation about it, let's see a test.