多進程
- fork
Unix/Linux操作系統(tǒng)提供了一個fork()系統(tǒng)調(diào)用镜豹,它非常特殊甫匹。普通的函數(shù)調(diào)用甸鸟,調(diào)用一次,返回一次兵迅,但是fork()調(diào)用一次抢韭,返回兩次,因為操作系統(tǒng)自動把當前進程(稱為父進程)復制了一份(稱為子進程)恍箭,然后刻恭,分別在父進程和子進程內(nèi)返回。
import os
print('Process (%s) start...' % os.getpid()) #getpid()當前進程
# Only works on Unix/Linux/Mac:
pid = os.fork()
if pid == 0: #子進程返回0
print('I am child process (%s) and my parent is %s.' % (os.getpid(), os.getppid())) #getppid當前進程父進程
else: #父進程返回子進程的ID
print('I (%s) just created a child process (%s).' % (os.getpid(), pid))
子進程永遠返回0扯夭,而父進程返回子進程的ID鳍贾。這樣做的理由是,一個父進程可以fork出很多子進程交洗,所以骑科,父進程要記下每個子進程的ID,而子進程只需要調(diào)用getppid()就可以拿到父進程的ID构拳。
Process (876) start...
I (876) just created a child process (877).
I am child process (877) and my parent is 876.
- multiprocessing
由于Python是跨平臺的咆爽,自然也應該提供一個跨平臺的多進程支持梁棠。multiprocessing模塊就是跨平臺版本的多進程模塊。
下面的例子演示了啟動一個子進程并等待其結束:
from multiprocessing import Process
import os
# 子進程要執(zhí)行的代碼
def run_proc(name):
print('Run child process %s (%s)...' % (name, os.getpid()))
if __name__=='__main__':
print('Parent process %s.' % os.getpid())
p = Process(target=run_proc, args=('test',))
print('Child process will start.')
p.start()
p.join() #join()方法可以等待子進程結束后再繼續(xù)往下運行斗埂,通常用于進程間的同步符糊。
print('Child process end.')
Parent process 928.
Process will start.
Run child process test (929)...
Process end.
- Bool
如果要啟動大量的子進程,可以用進程池的方式批量創(chuàng)建子進程
from multiprocessing import Pool
import os, time, random
def long_time_task(name):
print('Run task %s (%s)...' % (name, os.getpid()))
start = time.time()
time.sleep(random.random() * 3)
end = time.time()
print('Task %s runs %0.2f seconds.' % (name, (end - start)))
if __name__=='__main__':
print('Parent process %s.' % os.getpid())
p = Pool(4)
for i in range(5):
p.apply_async(long_time_task, args=(i,))
print('Waiting for all subprocesses done...')
p.close()
p.join()
print('All subprocesses done.')
Parent process 669.
Waiting for all subprocesses done...
Run task 0 (671)...
Run task 1 (672)...
Run task 2 (673)...
Run task 3 (674)...
Task 2 runs 0.14 seconds.
Run task 4 (673)...
Task 1 runs 0.27 seconds.
Task 3 runs 0.86 seconds.
Task 0 runs 1.41 seconds.
Task 4 runs 1.91 seconds.
對Pool對象調(diào)用join()方法會等待所有子進程執(zhí)行完畢呛凶,調(diào)用join()之前必須先調(diào)用close()男娄,調(diào)用close()之后就不能繼續(xù)添加新的Process了。
- 進程間通信
from multiprocessing import Process, Queue
import os, time, random
# 寫數(shù)據(jù)進程執(zhí)行的代碼:
def write(q):
print('Process to write: %s' % os.getpid())
for value in ['A', 'B', 'C']:
print('Put %s to queue...' % value)
q.put(value)
time.sleep(random.random())
# 讀數(shù)據(jù)進程執(zhí)行的代碼:
def read(q):
print('Process to read: %s' % os.getpid())
while True:
value = q.get(True)
print('Get %s from queue.' % value)
if __name__=='__main__':
# 父進程創(chuàng)建Queue漾稀,并傳給各個子進程:
q = Queue()
pw = Process(target=write, args=(q,))
pr = Process(target=read, args=(q,))
# 啟動子進程pw模闲,寫入:
pw.start()
# 啟動子進程pr,讀取:
pr.start()
# 等待pw結束:
pw.join()
# pr進程里是死循環(huán)县好,無法等待其結束围橡,只能強行終止:
pr.terminate()
Process to write: 50563
Put A to queue...
Process to read: 50564
Get A from queue.
Put B to queue...
Get B from queue.
Put C to queue...
Get C from queue.
線程
大多數(shù)情況下使用threading模塊,啟動一個線程就是把一個函數(shù)傳入并創(chuàng)建Thread實例缕贡,然后調(diào)用start()開始執(zhí)行:
import time, threading
# 新線程執(zhí)行的代碼:
def loop():
print('thread %s is running...' % threading.current_thread().name)
n = 0
while n < 5:
n = n + 1
print('thread %s >>> %s' % (threading.current_thread().name, n))
time.sleep(1)
print('thread %s ended.' % threading.current_thread().name)
print('thread %s is running...' % threading.current_thread().name)
t = threading.Thread(target=loop, name='LoopThread')
t.start()
t.join()
print('thread %s ended.' % threading.current_thread().name)
由于任何進程默認就會啟動一個線程,我們把該線程稱為主線程拣播,主線程又可以啟動新的線程晾咪,Python的threading模塊有個current_thread()函數(shù),它永遠返回當前線程的實例贮配。