Multi-threaded stats

Open
randy408 avatar @randy408
C C
Created over 2 years ago
Updated 26 days ago
4 comments
6,883

Describe the problem you are trying to solve. The stats addon and explorer is not MT aware, the values stop making sense with multiple threads.

Describe the solution you'd like Add support for per-thread stats, thread-specific stats (could indicate how well work is distributed, etc), and proper sums for time spent in systems (wall clock vs cpu time). And then support for it in explorer, of course.

Comments (2)

ZeroErrors avatar
@ZeroErrors over 1 year ago

To add a little more detail to this issue.

Currently the system_time_total is calculated only on the main thread: https://github.com/SanderMertens/flecs/blob/b5241cc76e50cba8cefe423c4478bf6fadb28462/src/addons/pipeline/pipeline.c#L632-L641

This means that the stats only knows about the amount of time the operation took to execute the main thread portion of the work, however, systems may have unbalanced work. Meaning a worker thread could have more to do than the main thread. If this happens it will cause the time interval on the main thread will not be equivalent to the amount of time the system actually took to execute.

SanderMertens avatar
@SanderMertens 26 days ago

To fix this issue, the following changes need to be made:

  • ecs_system_t::time_spent should be changed to a vec<ecs_ftime_t> that has as one element for single threaded systems, and as many elements as threads in the application for multithreaded systems
  • each thread should only increase its own element using the stage_index (thread index) to index the vector
  • the stats API must aggregate values by taking the min/avg/max of the values in the time_spent array

The following scenarios have to be tested:

  • verify time_spent is set correctly in single-threaded application
  • verify time_spent is set correctly in multi-threaded application
  • verify time_spent is set correctly after increasing the number of threads
  • verify time_spent is set correctly after decreasing the number of threads
  • verify that the stats API works with a single threaded scenario
  • verify that the stats API works with a multi threaded scenario
  • verify that single threaded systems still work the same as before
  • visually confirm that values look correct in the explorer

Bounty Available

Quick Select

Custom Amount

$
Fund this bounty to attract developers and speed up resolution. Minimum amount is $0.50.

Fund Requests

No fund requests yet. Be the first to request funding.

Please sign in to request funds for this bounty.

Paid Out

No payments have been made yet. Start working on this bounty to receive your payout

Information

Open
Current status
#872
Issue number