A very simple thread pool using pthreads in C++

I'm trying to understand some of the basics of using POSIX pthreads. The kind of thing I need to do (eventually) is parallelize some computations, using a thread pool model. At present I want to ensure I have a very basic sense for how the POSIX pthread model works. So I'm trying to create the simplest thread pool that's general enough to do the kinds of things I want to do. There will be some shared memory, an input queue, and an output queue, and there will be mutexes protecting each. I've written some code that does just that but valgrind's helgrind tool does not like what I've done. I suspect I'm missing something basic. Do you have insights into my code?

#include 
#include 
#include 
#include 
#include 

#include 
#include 
#include 

// the muticies, protectors of the shared resources
pthread_mutex_t coutLock;
pthread_mutex_t inQueueLock;
pthread_mutex_t outQueueLock;
// the shared data
std::list< std::string > inQueue;
std::list< std::string > outQueue;

struct thread_detail { // information to pass to worker threads
 unsigned long num;
};

extern "C" {
    void *workerThread(void *threadarg);
}

void *workerThread(void *threadarg) 
{
   struct thread_detail *my_data;
   my_data = (thread_detail *) threadarg;
   int taskid = my_data->num;
   std::stringstream ss; ss< threadIdList; // just the thread ids
  std::list< thread_detail > thread_table; // for keeping track of information on the various threads we'll create
  for (unsigned long i=0; i::iterator i(threadIdList.begin());
  while (i!=threadIdList.end())
   {
    if (pthread_join( *(*i), NULL)!=0) { std::cout<<"Thread join error!\n"; exit(1); }
    delete (*i);
    threadIdList.erase(i++);  
   }
  std::cout<<"\n";

   // let the user know what happened
   for (std::list< std::string >::iterator i=outQueue.begin(); i!=outQueue.end(); i++)
    {
     std::cout<<(*i)<<"\n";
    }
    // clean-up
    pthread_mutex_destroy(&coutLock);
    pthread_mutex_destroy(&inQueueLock);  
    pthread_mutex_destroy(&outQueueLock);  
    // pthread_exit(NULL);
}

Here is the helgrind output when passing the arguments 2 40 to the compiled program.

valgrind -v --tool=helgrind ./thread1 2 40
==12394== Helgrind, a thread error detector
==12394== Copyright (C) 2007-2009, and GNU GPL'd, by OpenWorks LLP et al.
==12394== Using Valgrind-3.6.0.SVN-Debian and LibVEX; rerun with -h for copyright info
==12394== Command: ./thread1 2 40
==12394== 
--12394-- Valgrind options:
--12394--    --suppressions=/usr/lib/valgrind/debian-libc6-dbg.supp
--12394--    -v
--12394--    --tool=helgrind
--12394-- Contents of /proc/version:
--12394--   Linux version 2.6.32-24-generic (buildd@yellow) (gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #38-Ubuntu SMP Mon Jul 5 09:20:59 UTC 2010
--12394-- Arch and hwcaps: AMD64, amd64-sse3-cx16
--12394-- Page sizes: currently 4096, max supported 4096
--12394-- Valgrind library directory: /usr/lib/valgrind
--12394-- Reading syms from /home/rybu/prog/regina/exercise/thread1 (0x400000)
--12394-- Reading syms from /lib/ld-2.11.1.so (0x4000000)
--12394-- Reading debug info from /lib/ld-2.11.1.so ..
--12394-- .. CRC mismatch (computed 99d13f6f wanted 0962e544)
--12394-- Reading debug info from /usr/lib/debug/lib/ld-2.11.1.so ..
--12394-- Reading syms from /usr/lib/valgrind/helgrind-amd64-linux (0x38000000)
--12394--    object doesn't have a dynamic symbol table
--12394-- Reading suppressions file: /usr/lib/valgrind/debian-libc6-dbg.supp
--12394-- Reading suppressions file: /usr/lib/valgrind/default.supp
--12394-- Reading syms from /usr/lib/valgrind/vgpreload_core-amd64-linux.so (0x4a23000)
--12394-- Reading syms from /usr/lib/valgrind/vgpreload_helgrind-amd64-linux.so (0x4c25000)
--12394-- REDIR: 0x4018310 (index) redirected to 0x4c2be59 (index)
--12394-- REDIR: 0x4018390 (strcmp) redirected to 0x4c2bf4b (strcmp)
--12394-- REDIR: 0x40184a0 (strlen) redirected to 0x4c2bec5 (strlen)
--12394-- Reading syms from /usr/lib/libregina-engine-4.6.1.so (0x4e31000)
--12394-- Reading syms from /usr/lib/libgmp.so.3.5.2 (0x52f7000)
--12394-- Reading debug info from /usr/lib/libgmp.so.3.5.2 ..
--12394-- .. CRC mismatch (computed d65050b9 wanted 1e40f6c0)
--12394--    object doesn't have a symbol table
--12394-- Reading syms from /lib/libpthread-2.11.1.so (0x5557000)
--12394-- Reading debug info from /lib/libpthread-2.11.1.so ..
--12394-- .. CRC mismatch (computed 9da7e2f6 wanted 8161fac5)
--12394-- Reading debug info from /usr/lib/debug/lib/libpthread-2.11.1.so ..
--12394-- Reading syms from /lib/librt-2.11.1.so (0x5774000)
--12394-- Reading debug info from /lib/librt-2.11.1.so ..
--12394-- .. CRC mismatch (computed 0e4f4ece wanted 920c9bed)
--12394-- Reading debug info from /usr/lib/debug/lib/librt-2.11.1.so ..
--12394-- Reading syms from /lib/libz.so.1.2.3.3 (0x597c000)
--12394-- Reading debug info from /lib/libz.so.1.2.3.3 ..
--12394-- .. CRC mismatch (computed 86f1fa27 wanted 5f1ca823)
--12394--    object doesn't have a symbol table
--12394-- Reading syms from /usr/lib/libstdc++.so.6.0.13 (0x5b93000)
--12394-- Reading debug info from /usr/lib/libstdc++.so.6.0.13 ..
--12394-- .. CRC mismatch (computed 7b5bd5a5 wanted e2f63673)
--12394--    object doesn't have a symbol table
--12394-- Reading syms from /lib/libm-2.11.1.so (0x5ea7000)
--12394-- Reading debug info from /lib/libm-2.11.1.so ..
--12394-- .. CRC mismatch (computed 043548c3 wanted a081b93d)
--12394-- Reading debug info from /usr/lib/debug/lib/libm-2.11.1.so ..
--12394-- Reading syms from /lib/libgcc_s.so.1 (0x612a000)
--12394-- Reading debug info from /lib/libgcc_s.so.1 ..
--12394-- .. CRC mismatch (computed 7c01dfc9 wanted 9d78e511)
--12394--    object doesn't have a symbol table
--12394-- Reading syms from /lib/libc-2.11.1.so (0x6341000)
--12394-- Reading debug info from /lib/libc-2.11.1.so ..
--12394-- .. CRC mismatch (computed c73d5a83 wanted 02758e3e)
--12394-- Reading debug info from /usr/lib/debug/lib/libc-2.11.1.so ..
--12394-- Reading syms from /usr/lib/libxml2.so.2.7.6 (0x66c4000)
--12394-- Reading debug info from /usr/lib/libxml2.so.2.7.6 ..
--12394-- .. CRC mismatch (computed c2590bed wanted 7aaa27a0)
--12394-- Reading debug info from /usr/lib/debug/usr/lib/libxml2.so.2.7.6 ..
--12394-- Reading syms from /lib/libdl-2.11.1.so (0x6a14000)
--12394-- Reading debug info from /lib/libdl-2.11.1.so ..
--12394-- .. CRC mismatch (computed 4a29f474 wanted e0b8d72c)
--12394-- Reading debug info from /usr/lib/debug/lib/libdl-2.11.1.so ..
--12394-- REDIR: 0x55603c0 (pthread_mutex_lock) redirected to 0x4c299fb (pthread_mutex_lock)
--12394-- REDIR: 0x5561a00 (pthread_mutex_unlock) redirected to 0x4c29e8c (pthread_mutex_unlock)
--12394-- REDIR: 0x63bd520 (malloc) redirected to 0x4c28a06 (malloc)
--12394-- REDIR: 0x63bf360 (calloc) redirected to 0x4c27cc9 (calloc)
--12394-- REDIR: 0x5c5e380 (operator new[](unsigned long)) redirected to 0x4c28e97 (operator new[](unsigned long))
--12394-- REDIR: 0x5c5e250 (operator new(unsigned long)) redirected to 0x4c2921f (operator new(unsigned long))
--12394-- REDIR: 0x5c5c380 (operator delete(void*)) redirected to 0x4c28328 (operator delete(void*))
--12394-- REDIR: 0x5c5c3c0 (operator delete[](void*)) redirected to 0x4c27fa4 (operator delete[](void*))
--12394-- REDIR: 0x63c3fe0 (strlen) redirected to 0x4a235dc (_vgnU_ifunc_wrapper)
--12394-- REDIR: 0x63c4010 (__GI_strlen) redirected to 0x4c2be91 (strlen)
--12394-- REDIR: 0x63c7c60 (memcpy) redirected to 0x4c2bfdb (memcpy)
Program will have 2 threads, working on 40 things 
--12394-- REDIR: 0x555dd60 (pthread_create@@GLIBC_2.2.5) redirected to 0x4c2d4c7 (pthread_create@*)
==12394== Thread #2 was created
==12394==    at 0x64276BE: clone (clone.S:77)
==12394==    by 0x555E172: pthread_create@@GLIBC_2.2.5 (createthread.c:75)
==12394==    by 0x4C2D42C: pthread_create_WRK (hg_intercepts.c:230)
==12394==    by 0x4C2D4CF: pthread_create@* (hg_intercepts.c:257)
==12394==    by 0x401C22: main (in /home/rybu/prog/regina/exercise/thread1)
==12394== 
==12394== Thread #1 is the program's root thread
==12394== 
==12394== Possible data race during write of size 8 at 0x7fefffcf0 by thread #2
==12394==    at 0x4C2D54C: mythread_wrapper (hg_intercepts.c:200)
==12394==  This conflicts with a previous read of size 8 by thread #1
==12394==    at 0x4C2D440: pthread_create_WRK (hg_intercepts.c:235)
==12394==    by 0x4C2D4CF: pthread_create@* (hg_intercepts.c:257)
==12394==    by 0x401C22: main (in /home/rybu/prog/regina/exercise/thread1)
==12394== 
started 21 completed 19 of 40==12394== Thread #3 was created
==12394==    at 0x64276BE: clone (clone.S:77)
==12394==    by 0x555E172: pthread_create@@GLIBC_2.2.5 (createthread.c:75)
==12394==    by 0x4C2D42C: pthread_create_WRK (hg_intercepts.c:230)
==12394==    by 0x4C2D4CF: pthread_create@* (hg_intercepts.c:257)
==12394==    by 0x401C22: main (in /home/rybu/prog/regina/exercise/thread1)
==12394== 
==12394== Possible data race during read of size 1 at 0x63401a7 by thread #3
==12394==    at 0x613A4D7: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A947: _Unwind_Resume (in /lib/libgcc_s.so.1)
==12394==    by 0x4019C9: workerThread(void*) (in /home/rybu/prog/regina/exercise/thread1)
==12394==    by 0x4C2D558: mythread_wrapper (hg_intercepts.c:202)
==12394==    by 0x555D9C9: start_thread (pthread_create.c:300)
==12394==    by 0x64276FC: clone (clone.S:112)
==12394==  This conflicts with a previous write of size 1 by thread #2
==12394==    at 0x6138331: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x5563F42: pthread_once (pthread_once.S:104)
==12394==    by 0x613A4C9: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A7B6: _Unwind_ForcedUnwind (in /lib/libgcc_s.so.1)
==12394==    by 0x556508F: __pthread_unwind (unwind.c:130)
==12394==    by 0x555EEB4: pthread_exit (pthreadP.h:265)
==12394==    by 0x40198E: workerThread(void*) (in /home/rybu/prog/regina/exercise/thread1)
==12394==    by 0x4C2D558: mythread_wrapper (hg_intercepts.c:202)
==12394== 
==12394== Possible data race during read of size 1 at 0x63401a6 by thread #3
==12394==    at 0x6139A12: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x6139AA8: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A724: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A982: _Unwind_Resume (in /lib/libgcc_s.so.1)
==12394==    by 0x4019C9: workerThread(void*) (in /home/rybu/prog/regina/exercise/thread1)
==12394==    by 0x4C2D558: mythread_wrapper (hg_intercepts.c:202)
==12394==    by 0x555D9C9: start_thread (pthread_create.c:300)
==12394==    by 0x64276FC: clone (clone.S:112)
==12394==  This conflicts with a previous write of size 1 by thread #2
==12394==    at 0x613832A: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x5563F42: pthread_once (pthread_once.S:104)
==12394==    by 0x613A4C9: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A7B6: _Unwind_ForcedUnwind (in /lib/libgcc_s.so.1)
==12394==    by 0x556508F: __pthread_unwind (unwind.c:130)
==12394==    by 0x555EEB4: pthread_exit (pthreadP.h:265)
==12394==    by 0x40198E: workerThread(void*) (in /home/rybu/prog/regina/exercise/thread1)
==12394==    by 0x4C2D558: mythread_wrapper (hg_intercepts.c:202)
==12394== 
==12394== Possible data race during read of size 1 at 0x63401b0 by thread #3
==12394==    at 0x6139AD7: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A724: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A982: _Unwind_Resume (in /lib/libgcc_s.so.1)
==12394==    by 0x4019C9: workerThread(void*) (in /home/rybu/prog/regina/exercise/thread1)
==12394==    by 0x4C2D558: mythread_wrapper (hg_intercepts.c:202)
==12394==    by 0x555D9C9: start_thread (pthread_create.c:300)
==12394==    by 0x64276FC: clone (clone.S:112)
==12394==  This conflicts with a previous write of size 1 by thread #2
==12394==    at 0x6138370: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x5563F42: pthread_once (pthread_once.S:104)
==12394==    by 0x613A4C9: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A7B6: _Unwind_ForcedUnwind (in /lib/libgcc_s.so.1)
==12394==    by 0x556508F: __pthread_unwind (unwind.c:130)
==12394==    by 0x555EEB4: pthread_exit (pthreadP.h:265)
==12394==    by 0x40198E: workerThread(void*) (in /home/rybu/prog/regina/exercise/thread1)
==12394==    by 0x4C2D558: mythread_wrapper (hg_intercepts.c:202)
==12394== 
--12394-- REDIR: 0x63bede0 (free) redirected to 0x4c28616 (free)
started 40 completed 40 of 40--12394-- REDIR: 0x555ef30 (pthread_join) redirected to 0x4c29796 (pthread_join)

thread 0 worked on task 0
thread 1 worked on task 1
thread 0 worked on task 2
thread 1 worked on task 3
thread 0 worked on task 4
thread 1 worked on task 5
thread 0 worked on task 6
thread 1 worked on task 7
thread 0 worked on task 8
thread 1 worked on task 9
thread 0 worked on task 10
thread 1 worked on task 11
thread 0 worked on task 12
thread 1 worked on task 13
thread 0 worked on task 14
thread 1 worked on task 15
thread 0 worked on task 16
thread 1 worked on task 17
thread 0 worked on task 18
thread 1 worked on task 19
thread 0 worked on task 20
thread 1 worked on task 21
thread 0 worked on task 22
thread 1 worked on task 23
thread 0 worked on task 24
thread 1 worked on task 25
thread 0 worked on task 26
thread 1 worked on task 27
thread 0 worked on task 28
thread 1 worked on task 29
thread 0 worked on task 30
thread 1 worked on task 31
thread 0 worked on task 32
thread 1 worked on task 33
thread 0 worked on task 34
thread 1 worked on task 35
thread 0 worked on task 36
thread 1 worked on task 37
thread 0 worked on task 38
thread 1 worked on task 39
==12394== 
==12394== ERROR SUMMARY: 7 errors from 4 contexts (suppressed: 804 from 64)
==12394== 
==12394== 1 errors in context 1 of 4:
==12394== Possible data race during read of size 1 at 0x63401a7 by thread #3
==12394==    at 0x613A4D7: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A947: _Unwind_Resume (in /lib/libgcc_s.so.1)
==12394==    by 0x4019C9: workerThread(void*) (in /home/rybu/prog/regina/exercise/thread1)
==12394==    by 0x4C2D558: mythread_wrapper (hg_intercepts.c:202)
==12394==    by 0x555D9C9: start_thread (pthread_create.c:300)
==12394==    by 0x64276FC: clone (clone.S:112)
==12394==  This conflicts with a previous write of size 1 by thread #2
==12394==    at 0x6138331: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x5563F42: pthread_once (pthread_once.S:104)
==12394==    by 0x613A4C9: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A7B6: _Unwind_ForcedUnwind (in /lib/libgcc_s.so.1)
==12394==    by 0x556508F: __pthread_unwind (unwind.c:130)
==12394==    by 0x555EEB4: pthread_exit (pthreadP.h:265)
==12394==    by 0x40198E: workerThread(void*) (in /home/rybu/prog/regina/exercise/thread1)
==12394==    by 0x4C2D558: mythread_wrapper (hg_intercepts.c:202)
==12394== 
==12394== 
==12394== 2 errors in context 2 of 4:
==12394== Possible data race during read of size 1 at 0x63401b0 by thread #3
==12394==    at 0x6139AD7: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A724: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A982: _Unwind_Resume (in /lib/libgcc_s.so.1)
==12394==    by 0x4019C9: workerThread(void*) (in /home/rybu/prog/regina/exercise/thread1)
==12394==    by 0x4C2D558: mythread_wrapper (hg_intercepts.c:202)
==12394==    by 0x555D9C9: start_thread (pthread_create.c:300)
==12394==    by 0x64276FC: clone (clone.S:112)
==12394==  This conflicts with a previous write of size 1 by thread #2
==12394==    at 0x6138370: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x5563F42: pthread_once (pthread_once.S:104)
==12394==    by 0x613A4C9: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A7B6: _Unwind_ForcedUnwind (in /lib/libgcc_s.so.1)
==12394==    by 0x556508F: __pthread_unwind (unwind.c:130)
==12394==    by 0x555EEB4: pthread_exit (pthreadP.h:265)
==12394==    by 0x40198E: workerThread(void*) (in /home/rybu/prog/regina/exercise/thread1)
==12394==    by 0x4C2D558: mythread_wrapper (hg_intercepts.c:202)
==12394== 
==12394== 
==12394== 2 errors in context 3 of 4:
==12394== Possible data race during read of size 1 at 0x63401a6 by thread #3
==12394==    at 0x6139A12: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x6139AA8: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A724: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A982: _Unwind_Resume (in /lib/libgcc_s.so.1)
==12394==    by 0x4019C9: workerThread(void*) (in /home/rybu/prog/regina/exercise/thread1)
==12394==    by 0x4C2D558: mythread_wrapper (hg_intercepts.c:202)
==12394==    by 0x555D9C9: start_thread (pthread_create.c:300)
==12394==    by 0x64276FC: clone (clone.S:112)
==12394==  This conflicts with a previous write of size 1 by thread #2
==12394==    at 0x613832A: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x5563F42: pthread_once (pthread_once.S:104)
==12394==    by 0x613A4C9: ??? (in /lib/libgcc_s.so.1)
==12394==    by 0x613A7B6: _Unwind_ForcedUnwind (in /lib/libgcc_s.so.1)
==12394==    by 0x556508F: __pthread_unwind (unwind.c:130)
==12394==    by 0x555EEB4: pthread_exit (pthreadP.h:265)
==12394==    by 0x40198E: workerThread(void*) (in /home/rybu/prog/regina/exercise/thread1)
==12394==    by 0x4C2D558: mythread_wrapper (hg_intercepts.c:202)
==12394== 
==12394== 
==12394== 2 errors in context 4 of 4:
==12394== Possible data race during write of size 8 at 0x7fefffcf0 by thread #2
==12394==    at 0x4C2D54C: mythread_wrapper (hg_intercepts.c:200)
==12394==  This conflicts with a previous read of size 8 by thread #1
==12394==    at 0x4C2D440: pthread_create_WRK (hg_intercepts.c:235)
==12394==    by 0x4C2D4CF: pthread_create@* (hg_intercepts.c:257)
==12394==    by 0x401C22: main (in /home/rybu/prog/regina/exercise/thread1)
==12394== 
--12394-- 
--12394-- used_suppression:    610 helgrind-glibc2X-101
--12394-- used_suppression:    192 helgrind---...-*Unwind*-*pthread_unwind*
--12394-- used_suppression:      2 helgrind-glibc2X-112
==12394== 
==12394== ERROR SUMMARY: 7 errors from 4 contexts (suppressed: 804 from 64)

在解释helgrind输出时我不太自信.

感谢您的任何见解.

您正在使用繁忙的循环:

  if (inQueue.size()==0) { somethingTodo=false; }
  else
    {
     workOnMe = inQueue.front();
     inQueue.pop_front();
    }
  pthread_mutex_unlock( &inQueueLock );

  if (!somethingTodo) continue;

查找条件变量.
这样,您的线程就不会消耗等待工作出现在队列中的资源.
看到这个问题

您标记了问题C++,但您使用的是C样式转换.
另请注意,在C++中,您不需要在此处添加struct.

my_data = (struct thread_detail *) threadarg;

从技术上讲,你应该声明你的回调函数使用C ABI(因为这是一个C库.

extern "C" void *workerThread(void *threadarg);

个人选择移动*旁边的类型(但这只是我个人的首选).

您没有使用RAII.所以你的锁定/解锁方案不是例外.

  pthread_mutex_lock( &inQueueLock );

  // Stuff that could throw.

  pthread_mutex_unlock( &inQueueLock );

即使中间的东西现在也不能扔掉.您假设有人不会添加将来不会抛出的代码.通过创建锁定对象使其安全.