Monthly Archives: September 2015

data caching ideas

Hopefully the files here are all back in order , they probably were hours after i posted anyway.;D

Anyway nothing is happening the only thing i barely did lately for unreal is
start building a caching system for the db system i created for my mods.
the ini database itself was a very clever idea but after quite a bit of data( 3000 lines for me )
It got to the point where every time you shot something , It would lag due to read/search/write.

someone recommended building a hash table , but due to the Dynamic RAD design of the  database,
You cant just create references , and while for example a number compare is 5x faster then a string compare,
The database is totaly writen in stings , that can be number of types. also unreal doesnt provide a hashing fuction for string 2 md5 etc
so in general a hash table would be way more work then its worth i think.

so we think about the problem.
searching 3000 lines to do a lookup is bad.
searching 3000 lines to modify the matching value is bad.
Writing to the ini every time somone moves , also bad.

whats not bad?
read/write for small lists.

So the plan?
cache the stuff we need , mostly JIT as we need it.

Need a value? Do a lookup of the value in the database and copy it to a struct in ram.
next time you look up the value , its has been cached , so you need to only search the cache ( witch at this point is 1 row long.)
In some cases such as int , or a bit toggle , we have no reason to even get the value from the db , we can write straight to cache.

Not all data needs to be searched or  even passed over now, only relevant rows.
if a player joins the server,maybe some of there data is loaded, like login  account info ,
as things are killed relevant class data can be cached , or even the map is analyzed for killable classes and those are cached.
new unknowns a re ammended to the cache etc.

in a case like this the other 2999 rows  related to players that are not online. in the original system that data is iliterated EVERY time we do a lookup or edit.

In this system it looked up a limited number of times and cached.

at the end of the level its merged back to the ini based database when game play it irrelevant, this single write could take 3 secends maybe , but none would see it.

But hey what if the server goes down half way tho the map bro?
The cached data gets lost.yep it sucks
Writes could be written every so often , or anytime all players leave etc

for values that must change , like account info or somthing , the modify data function will provide a optional forcewrite bool.

2015-09-10 23-41-45 Screenshot

anyhow i only wrote like 4 of the  dozens of functions required to built this yet , because i don’t see the potential return of my time atm.

Leave a comment

Posted by on September 11, 2015 in Uncategorized