Hello everyone!
Stefano here and this is the 25th Project_BuildTutto's Update!
I finally got back at my usual working place on Wednesday, so I regain the productivity of this place, and the timelapses videos too!
Nothing hugely visible has been done, but a lot of "under the hood" changes to improve both performance and my quality of life maintaining and upgrading the code base.
A lot of fixes and optimizations
Last week I worked away from home on my mac machine where I have a sublime+gcc setup for coding (and a windows partition with visual studio for emergency) and with that setup I really enjoy coding a bunch of stuff without testing much and debugging the problems. So, when I work away from home I usually add new features and when I get back at my windows workstation I spend whole weeks debugging and fixing all the problems caused by rushed code on mac.
This week, sadly, was no exception to this rule. I spent a good part of Wednesday fixing small problems on the resources installation menu and improving the process of building new items without crafting them and picking the final product from the storage. While fixing that, I noticed a lot of other stuff going wrong and those lead me into a rabbit hole that ended with me analyzing code performance of the whole game and picking optimizations spot trying to improve them. Luckily, I was able to improve the performance a lot just by tweaking and changing small portion of the code, and the boost is quite noticeable.
What I *think* I learned about memory and caches
When I was optimizing stuff, I noticed a lot of the bottlenecks when the game was mass updating big portions of 2d vectors. For example, this portion of code was present into the pathfinding algorithm used by guys to move around the map:
That portion of code cycled all the vector in order to reset it back to his default value each time a guy had to find the path to some place. At first, I couldn't imagine that this could be the slowest point of all the pathfinding system since it was the simplest operation to do. But it indeed was, and by a lot.
From a sampling analysis, that portion of the code was more than the 80% of the time spent inside that function.
To give you an idea of the context this is the whole function body:
Something had to change in that snippet of code, and after a bit of thinking I remembered something about memory and caches. That's a pretty complex topic to tackle here, I think I should go straight to the point: a vector of vector isn't a contiguous memory block. This means that one of the nested vector could be anywhere inside the memory, and the next vector could be also anywhere in the memory. This creates a lot of slow down and cache misses when trying to do operations on the whole 2d vector, since the system is always busy moving each vector from ram to cache.
A contiguous data structure on the other hand has the advantage of being all compact in a big block on the main memory, this lead to huge boosts in performance since when the first part of the vector is moved to cache, a lot of other parts are moved as well, removing the needs of other movement later on.
To solve this problem, I then implemented a custom vector2d class which is simply a big 1d vector (which is contiguous) with custom operators to access to memory location simulating a 2d grid.
Replacing the old code with this new code was quick and easy and leads to an impressive boost in performance, since the old 80% of the time spent there decreased to a whopping less than 1%! (Note that I also discovered the std::fill function that is also a nice improvement instead of the manual assignment of default values like I did before)
New recipe and library manager
Phew, that was a lot of talking on the previous point. This could be another long one, but since is not really finished yet I'll talk just of the ideas here, and explain the implementation deeper next week ;)
Long story short: I had to redo the way new objects are created and discovered through the blueprint menu. The old way was just a bunch of hardcoded stuff which was a nightmare to maintain and improve. So, I redid all the system by creating a parser from external files, which I can modify to add, remove, or edit anything of the item properties like the materials needed to craft it, the pattern, the properties inheritance, value, and a lot of other stuff.
I'm also very happy with this change, but as I said before, I'll talk about it better next week.
Keep in touch
If you are curious to see how things will go feel free click any of these links to join our community:
Discord Server: Discord.gg
YouTube channel for Timelapses: Youtube.com
Twitter: Twitter.com
Changelog: Docs.google.com
