鍍金池/ 問答/Python  網(wǎng)絡(luò)安全/ 請(qǐng)問:為什么Python自帶的Multi-processing模塊需要一定時(shí)間才

請(qǐng)問:為什么Python自帶的Multi-processing模塊需要一定時(shí)間才能啟動(dòng)?

我用Python自帶的Multi-processing模塊做了一個(gè)并發(fā)的Async的系統(tǒng)。只有2個(gè)任務(wù),一快一慢,并發(fā)執(zhí)行,快的等慢的。

import sys, os, time, datetime
from timeit import timeit
import threading
import multiprocessing
from multiprocessing import Pool

def multiprocessor_hack(instance, name, args=(), kwargs=None):
    if kwargs is None:
        kwargs = {}
    return getattr(instance, name)(*args, **kwargs)

class Controller:
    request = "Whatever"
    def task1(self, request):
        print ("Now starting task1 at {}.\n".format(datetime.datetime.now()))
        time.sleep(3)
        print ("Now finished task1 at {}.\n".format(datetime.datetime.now()))
        return "111111"
    def task2(self, request):
        print ("Now starting task2 at {}.\n".format(datetime.datetime.now()))
        time.sleep(10)
        print ("Now finished task2 at {}.\n".format(datetime.datetime.now()))
        return "222222"

    def moderate(self, request):
        pool = multiprocessing.Pool(processes = 2)
        results = [pool.apply_async(multiprocessor_hack, args = (self, task1', (request,))), pool.apply_async(multiprocessor_hack, args = (self, 'task2', (request,)))]
        pool.close()
        map (multiprocessing.pool.ApplyResult.wait, results)
        response = [r.get() for r in results]
        return response


if "__main__" == __name__:
    ctrl = Controller()
    print ("\nThe pool starts at {}.\n".format(datetime.datetime.now()))
    response = ctrl.moderate("Whatever")
    print ("\nResponse emerges at {}.".format(datetime.datetime.now()))
    print ("\nThe pool ends at {}.".format(datetime.datetime.now()))

運(yùn)行結(jié)果是:

The pool starts at 2018-03-23 15:03:51.187000.
Now starting task1 at 2018-03-23 15:03:51.522000.  # 延遲?
Now starting task2 at 2018-03-23 15:03:51.522000.  # 延遲?
Now finished the task1 at 2018-03-23 15:03:54.524000.
Now finished the task2 at 2018-03-23 15:04:01.524000.
Response emerges at 2018-03-23 15:04:01.526000.
The pool ends at 2018-03-23 15:04:01.528000.

這里,pool在下午三點(diǎn) 15:03:51.187 開始運(yùn)行,但是2個(gè)任務(wù)直到 15:03:51.522 才開始執(zhí)行!… 請(qǐng)問這里為什么會(huì)延遲0.4秒呢?有沒有什么辦法減小這個(gè)延遲?

謝謝了先!

回答
編輯回答
傲寒

開多進(jìn)程要fork,開銷算是非常大的,相當(dāng)于你重新打開了一個(gè)python??梢圆婚_多進(jìn)程就不開多進(jìn)程,可以用線程代替進(jìn)程,CPython還有GIL這種東西,有時(shí)候開了多進(jìn)程或多線程CPU利用率反倒會(huì)降低(調(diào)度和規(guī)劃沒做好的話)。

2017年9月5日 17:56
編輯回答
爆扎

我搜了一下,Multi-processing模塊需要一定時(shí)間在main process和work process之間建立通信,并且由系統(tǒng)分配資源,所以需要一定時(shí)間來啟動(dòng)。

2018年7月13日 19:27
編輯回答
有你在

猜測(cè)可能是在編譯class,編譯時(shí)又要做導(dǎo)入,所以慢。你可以試試把py編譯一下看看是否有提升

2018年5月13日 12:56