The high level idea is to create avoidance spheres.
When an ai agent enters a sphere, it avoids the center.
The deeper the agent goes, the more avoidance it has applied to it.
The avoidance removes a portion of its desired direction vector.
We create a vector from the ship towards the center of the sphere; let's call it ToCenterVector.
Then, we project the ships current movement direction onto this ToCenterVector.
This ToCenterVector is a vector that only contains the movement direction that we do not want.
So if we entirely remove this projection-derived ToCenterVector from the movement direction vector, then the ship will not move towards the center.
But just removing this vector from the movement direction would look rigid and sudden.
It also isn't useful if something (ie another ship) occupies a large of the sphere, as collision can still occur.
So what I have done is remove some percentage of this vector, based on how deep the ship is into the avoidance sphere.
At some specified depth (not at the center) the ship has 100% of the ToCenterVector removed.
With some tweaking, you can achieve really natural looking avoidance.
I created a few debug utilities to help visualize what is going on.
With this many ships, it is difficult to use traditional debugging techniques.
So having visual feedback makes it easy to determine what is happening.
I find that to be true for many things in gameplay programming.
DevBlog 14 - AI Avoidance Sphere Algorithm
An algorithm created to make AI not fly into things (it's not boids).
Posted by wafflemaker72 on