Recent Changes - Search:

Main.SideBar (edit)

PmWiki

ItemQuadTree

Idea is to keep all items in a quadtree instead of per-cell lists.

AB: I'd rather use _in addition_ to per-cell lists. We often need (see pathfinder) to access a particular cell, this would be inconvenient to implement in a quadtree

This would speed up lots of queries which need list of units/items on a certain area, such as Unit::unitsInRange().
ATM, in average game, Unit::enemyUnitsInRange() takes almost quarter of all time spent in advance calls, so it is becoming a serious performance problem.

Required operations

This is for the whole quadtree

  • SELECTION on given cell (x, y)
    • Used in pathfinder to calculate cell's status according to units on the cell
  • SELECTION in given rect (x1, y1, x2, y2)
    • Used e.g. to get list of enemies in range
  • Maybe SELECTION within given radius from given center point
    • More accurate and a bit faster unitsInRange()
    • Is this really needed? It can probably be omitted in the first version since the performance increase is probably very small (coming only from the fact that items which are not in radius wouldn't be added to the list and removed later, when it's found out that they are indeed out of radius)

SELECTION includes:

  • All items
AB: "All items" is a special case of items in rect.
  • Rendering (to get list of to-be-rendered units)
  • Collision detection?
  • All units (excludes e.g. shots)
    • BosonCanvas::explosion()
  • All land units (would probably include sea units)
    • Pathfinder, to calculate cell's status according to units on the cell
  • All air units
  • Units belonging to player x
    • BosonCanvas::canPlaceUnitAt() (although not speed critical)
  • Units not belonging to player x (or maybe units belonging to player X's enemies)
    • Unit::enemyUnitsInRange()
  • Number of units
  • Number of items
AB: I think pretty much all of these can be implemented using a simple algorithm: 1. find all items in given area 2. return all [units|air units|land units|whatever] that was returned by 1. The performance is essentially defined by 1., the 2. is a simple linear search over the result, which typically will be a low number of elements

Data structures

The main data structure of course is the quadtree.
Every node of the quadtree would contain:

  • BoItemList playerItems[MAX_PLAYERS]; - per-player item lists
  • QValueVector<Unit*> playerUnits[MAX_PLAYERS]; - per-player units? is this necessary?
  • QValueVector<Unit*> playerLandUnits[MAX_PLAYERS]; - per-player land units. necessary?
  • QValueVector<Unit*> playerAirUnits[MAX_PLAYERS]; - per-player air units. necessary?
  • int unitCount;
  • int itemCount;

Note that QValueVectors here should perhaps be replaced with BoItemLists. ATM they just emphasize that they contain only units, not e.g. shots

AB: I don't think that's a good design idea. See memory usage.

Memory usage

The quadtree would have about 1.33*n*n nodes in total for a map of size n x n cells.
Each node would store 2 ints and 4*MAXPLAYERS lists.

AB: note that the "lists" is what defines memory usage here. Think about it: on a 500x500 map with 10 players (current maxplayers) that would be > 1.33*500^2*10/(1024^2) MB per byte. That's 3.17 MB per byte in a quadtree node. Now assume every list would take 16 bytes (it takes much more!) - 50.7 MB of memory WITHOUT STORING ANYTHING! And that is only for a single (!) list per node.

Every level of the quadtree would store every unit in the game about 4 times (once in each list). As the quadtree has log n levels, the lists would store about log n * 4 * x units in total, where x is number of units in the game.

AB: note that "about" is actually "at least": a unit usually is on more than one cell and thus on more than one node per level. But the real number should be close to log n for every list. Storing it in 4 lists every time however, is pretty unelegant imho: you need to update EVERY list WHENEVER a unit moves. In total you must be careful that you don't make it slower than a version without quadtree.

Thus, total amount of memory used by the quadtree is 1.33*n*n * sizeof(node) + 4*x*sizeof(list item)*log n where sizeof(node) is about 2*sizeof(int) + 4*MAXPLAYERS*sizeof(list)
Assuming that MAXPLAYERS=10, sizeof(int)=4, sizeof(list)=16, sizeof(list item)=12 (based on memory usage of BoItemList (AB: BoItemList definitely takes more memory! also note that sizeof(list)=16 is an absolute minimum and sizeof(list item)=12 is not realistic)), amount of memory used would be 1.33*n*n * (2*4 + 4*10*16) + 4*x*12*log n = 864*n*n + 48*x*log n
For a map of size 256 with 1000 units, memory usage would be about 54mb. However, for a map of size 500 (current maximum) with 10000 units, memory usage would already be 210mb, which is a bit too much. (AB: you have an interesting definition of "a bit". also keep in mind that you have used worst-case playfield values, but about best-case values for the quadtree memory consumption)

To reduce memory usage, it might make sense to construct the quadtree so that each leaf would represent not a single cell but e.g. 4x4 cells. This would remove two bottom levels of the tree reducing memory usage to 54*n*n + 48*x*(log n - 2) which is just 16mb for map of size 500.

AB: note that such tricks only help temporarily. You don't fix the principle problem, but move it farther away only. If it goes away far enough, then that's allright, but I think the initial proposal already takes so much memory, that it's better to fix that one first. In particular remove the lists[MAXPLAYERS] thing.

The pathfinder would probably then require it's own datastructure which for each cell would store all land units on that cell. Memory usage of that would be n*n * 16 + 12*x which for the above map would be 4mb. Keeping another datastructure up to date would of course need additional time but otoh the quadtree would have less levels to keep up to date and most unit movements would affect only the bottom (i.e. removed) levels anyway.
Reduced quadtree levels would of course make the query results a bit less accurate however most queries would affect quite big area (as sight and weapon ranges of the units are quite big and will probably rather increase than decrease in the future) and would need additional post-processing anyway (e.g. to select the best target, or to remove all units not strictly within given radius) so it shouldn't be very big problem.

Operations' implementations

  • All items - would iterate through all playerItems lists, append items of those lists to a single big list and return that list
    Maybe it would be more efficient to have a big allItems list as well...
  • All units - same as above
  • All air/land units - same as above, but would use corresponding lists
  • Units belonging to given player - would use only given player's list
  • Units belonging to enemies of given player - would use only those lists which correspond to enemies
  • Add/remove unit - would need to add/remove unit to/from up to 5 lists -> possible performance problems -> lists would need to be very efficient
    • allItems/allUnits, if we choose to add them
    • playerItems
    • playerUnits
    • playerLandUnits or playerAirUnits
    • increase unitCount and itemCount

AB:
Also keep performance in mind when updating the quadtree. I have already started an implementation a while ago (but stopped due to this doc - I don't want to do duplicated work in this area) and the main problem that gave me most headache is updating the tree. Whenever an item moves you have to

  • remove it from the tree completely (beware of rounding errors! if you don't find ALL nodes, you'll have a dangling pointer)
  • add it to the tree again.

Both operations may take quite some time. In your proposal, it'd be at least log(n)*MAXPLAYERS*4 - on every quadtree level you need to update MAXPLAYERS*4 lists. (Updating a list of a large node may take O(#units) time which might be significant, too. But let's neglect this for now). If MAXPLAYERS is 10, then 4*MAXPLAYERS is already more than the time between two advanceNone() calls, which call unitsInRange() which is the performance issue we are trying to solve. So if a lot of units are moving, you may actually make the whole program significantly slower.
Of course this is only the case if lots of units are moving - if most are standing still, then we don't have a problem.

Edit - History - Print - Recent Changes - Search
Page last modified on June 16, 2006, at 20:10