def _checkpid(self):
if self.pid != os.getpid():
with self._check_lock:
if self.pid == os.getpid():
# another thread already did the work while we waited
# on the lock.
return
self.disconnect()
self.reset()
Problem:
when it get connection from pool or release to pool, it will both check pid, i don't understand why. if it run in multiprocess, it does not need the lock and it will have many identical pool. and if it run in multi thread, it will always get the same pid. any help is approciate.
Answer:
When a Unix process forks, it shares all sockets already opened in it with its children.
So if you create a connection pool, then make a request, and then fork, you will have a problem: While pools themselves get copied between the new processes, the sockets in the pool do not get copied and are shared across processes.
This can lead to a situation where one process writes to a redis client, and another waiting on the reply will get the wrong reply back.
However, newly created sockets AFTER the fork, will not be shared between the parent and children.
So by checking the pid, the pool checks if a fork has been made, and if so, resets all its sockets and creates new ones. This prevents very bad things from happening :)
PS:
So much thanks for your help! But i can not think out a situation where forks will happen, why do we need to fork, and in my opinion the checkpid method can only prevent what you said, could you give me a example?
python cannot utilize more than one CPU because of the GIL. Forking allows python programs to utilize multi-core machines. check out the Multiprocessing module in python for more details.
