TBB parallel pipeline on mix of IO and compute workload

Given million query vector, for each query I want to find similar vectors among some DB of vectors and I want to parallelize this workload using TBB. The query consists of transforming the candidate vectors from DB using several functions. I wrap each function as separate tasks and use pipeline_parallel with filters as tasks. I want to ask whether parallel_pipeline would be actually good for this kind of workload where I have the mix of IO and compute.

Read more here: https://stackoverflow.com/questions/66271275/tbb-parallel-pipeline-on-mix-of-io-and-compute-workload

Content Attribution

This content was originally published by kiki at Recent Questions - Stack Overflow, and is syndicated here via their RSS feed. You can read the original post over there.

%d bloggers like this: