Static vs Dynamic Databases

[SMcNeill]

I was working on a little pet project for a friend this morning, and we got to discussing various types of databases.  Apparently, the definition of "database" has changed from what I used to know it as, and so has what folks call "dynamic databases" and "static databases".

Apparently, the modern definition of "static" or "dynamic" is simply determined by how we reserve memory for our database when we initially create it...   https://www.ayomaonline.com/academic/static-vs-dynamic-data-structures/

Static Data Structures

As same as the word static suggests, static data structures are designed to store static “set of data”. However, static “set of data”, doesn’t mean that we can not change the assigned values of elements. It is the memory size allocated to “data”, which is static.

Dynamic Data Structures

Dynamic data structures are designed to facilitate change of data structures in the runtime. It is possible to change the assigned values of elements, as it was with static structures. Also, in dynamic structures the initially allocated memory size is not a problem. It is possible to add new elements, remove existing elements or do any kind of operation on data set without considering about the memory space allocated initially.

*****************
*****************

Since when did the definition become that??  Apparently, if I DIM array(whatever), it's a STATIC database, but REDIM array(whatever) is DYNAMIC??

That's not how I've always used the words....

For me, a static database is one where the STRUCTURE remains static and unchanging.  We might add or remove records -- thus changing the amount of memory the database consumes -- but the elements/structure of the data doesn't change itself.

Name, Age, Sex -- this is a STATIC  data base.  We might have 5 people in the database, or we might expand it to have 500; it's still the same unchanging data structure. 

A dynamic database, on the other hand, is one where I've always felt had a structure built to be flexible and adaptable.  If you guys remember my Homework Helper which I shared while in production (once finished, the local school purchased rights to it so I couldn't share the finished product), it relied on a dynamic database.

<President><Elected In><Elected Out>            <----  It allowed the USER to create a header such as the following for their data, and it read the stuff between the brackets as individual fields, which it then used to track the actual data which followed. 

When created, I had no idea what fields a teacher might want to add to make a set of test questionnaires for students to use and practice on.  For our presidents, they might want name and years in office... Or they might want to add which president they were numerically (Who was the 14th  president?), or what party they belonged to...    I had no idea what fields the USERS might want to add to the database, so I had to create it to be flexible and dynamic to their needs.

Memory usage wasn't the least bit of a concern -- it was writing the database to the point where its STRUCTURE was fluid enough to allow field addition/editing/deletion which made me consider it to be a "dynamic database".

******************
******************

Have I just fallen out of the loop?  What do you guys think of when you hear the terms?  Is it honestly just the difference in how we allocate memory which designates the difference in "static vs dynamic" now?  If so, then how do you differentiate between the two data types I was speaking of above?  "Open vs closed data structure"?

Man, talk about something making me feel OLD....

I've worked with various databases and data styles all my life it seems, but today's the first time where it felt like I couldn't hardly even speak the same language when discussing one with a younger collage student.

Anyone else run into this conversational/generational gap?  I always thought "once a computer nerd, always a computer nerd", but apparently that doesn't hold true in today's world anymore.

I'm just old....     

[Cobalt]

If it is a definition created by a US person or entity then the rest of the world will see it corrupted and ruined. The world hates and despises anything USA based. An individual person in that world might like the USA but, as a group or a whole they despise us something fierce and will stop at nothing to ruin everything we do. We, as in USA people in the QB64 community, have some good friends around the globe but, put them in the group and they would quickly get swallowed and quashed out.  The definitions your familiar with is what I was taught too. so those definitions you found come across as quite foreign ideas to me as well.

[STxAxTIC]

Heya boys,

Appreciated your comments like crazy Cobalt. To all the USA haters out there, all I can say is sucks to be you. Honestly this isn't a matter of definitions though so let us get back to the point...

So Steve - I would've felt the same way a year or two ago til I started learning about the strangely satisfying world of alternative data structures. This delves into hashing, hash functions/tables, linked lists, etc etc etc... And once that toolset is sorta understood, you look back and say to hell with old-fashioned arrays and stuff.

So REDIM doesn't really mean it makes the array dynamic in this sense. All REDIMming does is make a new static array out of an old one. A dynamic scheme will be so flexible, you could have any behavior from each row being unequal length, shuffling fields around for some records but not others, making records point around to each other in a tree-like fashion - etc etc etc... and all of this would be MAYHEM using old-fashioned arrays. I would really Wikidive this for a night.

Meanwhile, and not to self-reference, but my up-and-coming text editor shows off exactly the principle above. Namely it's a giant doubly-linked list. To explain: you know the problems with making a general-purpose textbox in qb64. All those weird cases, all the backspacing, pasting blobs of text here and there - again this is utter mayhem for normal arrays. Implement a liked list and voila, the mechanism is tight as ever... Now if I would only get around to finishing the interface...

(And re-reading this I should disclaim that I don't really know much about databases than any other layman.)

[bplus]

Dynamic or static space allocation + dynamic or static structure, 4 combos but is dynamic structure possible with static space allocation? Static space allocation + static structure sounds super fast! Dynamic structure + dynamic space allocation sounds super interesting and more complex.

Call me, DY+NAM+IC ? ;D

[STxAxTIC]

Dynamic or static space allocation + dynamic or static structure, 4 combos but is dynamic structure possible with static space allocation? Static space allocation + static structure sounds super fast! Dynamic structure + dynamic space allocation sounds super interesting and more complex.

Holy tongue twister! It took me two passes to realize this post wasn't a joke, and actually says (at least) two things.

[bplus]

A Dynamic Do-over?

[Petr]

So, in fact, the dynamic field will be the same as the automatically redefined static field, but not just the size of the records in its field, but also the change in its type, right? So if the _UNSIGNED _BYTE field is with a value above 255 is then converted to _UNSIGNED INTEGER, and if it is some value over 65535, it will be converted again to _UNSIGNED LONG? Is  that a dynamic field? Or it is something different?

[SMcNeill]

So, in fact, the dynamic field will be the same as the automatically redefined static field, but not just the size of the records in its field, but also the change in its type, right? So if the _UNSIGNED _BYTE field is with a value above 255 is then converted to _UNSIGNED INTEGER, and if it is some value over 65535, it will be converted again to _UNSIGNED LONG? Is  that a dynamic field? Or it is something different?

When we're talking a static field, it's one that doesn't change, whereas a dynamic field may.   In QB64, we see an easy example with fixed length strings vs plain strings.

DIM Array(1 TO 100) AS STRING * 10

vs

DIM Array(1 TO 100) AS STRING

The first is going to use 1000 bytes to store the data with it, so under the "new definition" my friend was using yesterday, it's a STATIC database.  1000 bytes of memory required to hold it; nothing more, nothing less.

The second, however, is going to start out blank (all elements are ""), so uses 0 bytes to store the data, but will move and alter requirements as the elements change.  It may take 1000 bytes to hold the information, or it may take 100,000,000...   Under the new definition, it's considered a DYNAMIC database as the memory usage may change at any time.

********************

Now, the way I was taught to label databases years ago would call both examples STATIC.  The field is set -- it's a string in both cases.  One may use longer/shorter strings than the other, but it's still always a string value...

From the way I was taught, DYNAMIC databases had flexible fields that changed at the USER's desire, not the programmers.

DIM M AS _MEM
M = _MEMNEW(10000)

With the above, the user could store bytes of numbers in that memblock.  They could store string characters.  It'd be up to THEM to define what the data inside the block represented.  We'd have to code our database to allow them the flexibility to create an identifier set of headers and then enter the data that correspond to their headers.  It's this flexible, alterable, user-defined method which creates what I used to know as a DYNAMIC database --- memory usage had NO part of the definition.

Either somebody changed the jargon, I fell behind the times, or my friend is an idiot...  The way the two of us label "dynamic vs static" is just absurdly different.  To HIM, it's all about whether the memory used can be set in stone (he calls the _MEMNEW "static" as it reserves 10000 bytes and never changes it), or if it changes (as with the AS STRING array).  To ME, it's about if the fields in the database are set or variable -- which means we're not talking about the same beasts at all.

[SMcNeill]

Here's a very quick and cheesy example of what I'd consider to be a "dynamic database":

First, copy this little code and save it as "temp.txt" in your QB64 folder:

Code: QB64: [Select]

1. word 1|ST,word 2|ST,num 1|NM,num 2|NM
2. dog,food,1,2
3. hair,ball,3,4
4.  

Then copy the code here and run it in QB64:

Code: QB64: [Select]

1. CONST FieldDelimiter = ",", FieldIdentifier = "|"
2.  
3. TYPE ArrayType
4.     id AS STRING * 20
5.     tp AS STRING * 2
6. END TYPE
7. DIM Header(1 TO 100) AS ArrayType
8.  
9.  
10. OPEN "temp.txt" FOR BINARY AS #1
11. LINE INPUT #1, Header$ 'get the header
12. DO 'seperate the data fields
13.     Elements = Elements + 1
14.     l = INSTR(Header$, FieldDelimiter)
15.     IF l THEN
16.         decode$ = LEFT$(Header$, l - 1)
17.         Header$ = MID$(Header$, l + 1)
18.     ELSE
19.         decode$ = Header$
20.     END IF
21.     l1 = INSTR(decode$, FieldIdentifier)
22.     Header(Elements).id = LEFT$(decode$, l1 - 1)
23.     Header(Elements).tp = MID$(decode$, l1 + 1)
24. LOOP UNTIL l = 0
25.  
26. PRINT "Your database consists of:"
27. FOR j = 1 TO Elements
28.     PRINT Header(j).id, Header(j).tp
29. NEXT
30.  
31. DO UNTIL EOF(1)
32.     Records = Records + 1 'count lines for records
33.     LINE INPUT #1, junk$
34. LOOP
35. CLOSE
36. OPEN "temp.txt" FOR INPUT AS #1 'just so I can be lazy and use INPUT without parsing fields
37. LINE INPUT #1, junk$
38.  
39. DIM DataArray(Records, Elements) AS STRING
40. FOR i = 1 TO Records
41.     FOR j = 1 TO Elements
42.         INPUT #1, DataArray(i, j)
43.     NEXT
44. NEXT
45.  
46. 'And now to work with our data somewhat cheesily
47. FOR i = 1 TO Records
48.     FOR j = 1 TO Elements
49.         FOR k = 1 TO Elements
50.             PRINT "What do you get when you add "; DataArray(i, j); " and "; DataArray(i, k); "? ";
51.             IF Header(j).tp = "ST" AND Header(k).tp = "ST" THEN
52.                 PRINT DataArray(i, j); DataArray(i, k)
53.             ELSEIF Header(j).tp = "NM" AND Header(k).tp = "NM" THEN
54.                 PRINT VAL(DataArray(i, j)) + VAL(DataArray(i, k))
55.             ELSE
56.                 PRINT "We can't add numbers and strings!"
57.             END IF
58.         NEXT
59.     NEXT
60.     SLEEP
61.     CLS
62. NEXT
63.  

As you can see, I've established a set of minor rules for how an user can create a database to work with this program.  Name a field, then identify if it's ST or NM (string or number), then add another field by adding a comma as a separator.

Now, this is the ABSOLUTE bare bones of (what I'd call) a "dynamic database", and it has no error checking of any sort or data validation for it.  IF you follow the basic rules of creation, you can make this add your elements together (if possible), no matter how you define them.  It handles strings one way, numbers another, and refuses to mix the two....

YOU, as the user, can define the database to become anything you want it to be.  Add a set of fields.  Add records.  Delete them...   As the programmer, *I* have no idea what type of dataset you're going to throw at this little routine.  It's completely "dynamic" and responds to YOUR, the user's, data file, as you define it.

[Petr]

Thank you for the show, Steve.

[Pete]

DIM Mass AS Single
DIM U Double
LET U = Mass
' Oops. Double DIM Mass on U.

Pete :D

- Kirk from Star Trek 4

[codeguy]

The way I view static is that it's known at compile time and never changes lbound or ubound during execution. Dynamic is not fixed at compile time and CAN be altered during execution using redim. My sorting library and test harness use dynamic arrays. I prefer not to use many globals (CONSTs, STATIC, SHARED) as this can cause unpleasant conflicts.

[STxAxTIC]

EDIT: Somehow (due to the way I enter to forum page) I missed Steve's code above. I may shove my foot in my mouth after fiddling with it, but I'll keep the meandering prose below for completeness:

(I think once again we're getting lost in what "dynamic" here means. Should we try to agree that REDIM does not mean "be dynamic"?)

To elaborate on what I meant above when hinting at the fields being different length, what that really means is not just variable-length strings in types or even a generality of that. I mean each record's type being able to differ. The first record in the database may be just a person's name, and containing nothing else. Not other fields that are left blank, I really mean the name is *it*. Meanwhile, the second entry might have a user name, address, and a handful of other fields uncommon to the other records. So you gotta imagine:

Type foo
  a as string
  b as integer
end type

Type bar
  c as long
  d as double
  e as whatever
end type

coexisting in the same array... and even this example makes me cringe because it's a bit restricting. I recommend going full liberal party with how you think of data now. Scattered and unstructured, but infinitely flexible.

[SMcNeill]

I've worked with databases like you mention, the way I've seen those, they often have a leading identifier field to them:

Structure would be similar to:
Identifier
Field1
Field2
Field3
So on...

In your example, Record 1's identifier might be "F1", which says it holds the name (Field 1), as you mentioned.  Record 2's identifier might be "F1F2" to indicate it contains name, and address...

Other easy ways I've seen this done is with a termination field/code:

Field1
Field2
Field3
So on....
Terminator

So record 1 is "Steve:name" then CHR$(26) for the terminator.  Record 2 is "Bill:name;Male:sex;123 His Home:address" then CHR$(26)...

Actually lots of ways to "Link" the fields together, but as long as they're constrained internally, I'd call it a "static database".  Your example with set TYPEs is defined by the programmer and doesn't change. My definition of a "dynamic database" would be one where the user can create their own fields to suit their needs, not just add or exclude some optional preset fields.  ;)

[STxAxTIC]

Sigh. Hear that? It's the point sailing waaaay over. I'll try better with an example or something.

EDIT: Provided QB64 is even suitable. I mean - has anyone else bothered searching this out?