Python 多线程问题

[latteczy]

https://gist.github.com/smartczy/35e692064b775dcc3e17

上面的代码分别用多线程和普通方式抓取页面信息。但是二者运行时间相差无几，大约都是0.7+秒。这里应该不是CPU密集型，所以多线程应该有效率的提高才对。不知是何原因？
最近在学习Python的多线程，请大神多多指教！

[realityone]

你在 q 里面再加一句 time.sleep(5) 然后观察下就知道问题在哪里了



`t.join()`

[RichardZhiming]

尽量使用多进程，gil是Python的历史遗留问题

[latteczy]

@RichardZhiming 嗯，只是想搞明白这个Python的多线程是怎么回事

[zhyu]

join([timeout])
Wait until the thread terminates. This blocks the calling thread until the thread whose join() method is called terminates – either normally or through an unhandled exception – or until the optional timeout occurs.

[latteczy]

@realityone 加上time.sleep(5)之后结果是21秒多，还是不太明白，麻烦能否再指点一下。是t.join()有问题么？

[latteczy]

@zhyu 额，join()是要等待该线程结束才能再起其他的线程啊。也就是说加上join()就跟单线程一样咯？

[Shazoo]

threads = []
for i in range(4):
t = threading.Thread(target=q, args=(hosts[i],))
threads.append(t)
t.start()

for t in threads:
t.join()

这样应该会快些。

[onlyice]

https://gist.github.com/smartczy/35e692064b775dcc3e17#comment-1487643

[alexapollo]

建议你用多进程 multiprocessing 模块。
python 的线程机制比较弱，一般用多进程或异步解决这个问题。

[latteczy]

@alexapollo 我知道，我只是想弄明白Python的多线程是怎么回事。而且对于这段代码，多线程应该是会有效率上的提高的，但是我并没测试出有提高。所以想弄明白是怎么回事。

[alexapollo]

@latteczy GIL，查了就懂了。活用google，SO上这种问题蛮多的

[latteczy]

@alexapollo 嗯，这段代码效率没提高就是楼上所说的t.join()位置不对。主要原因不在GIL吧

[alexapollo]

@latteczy 恩，是的

[est]

GIL 真是个活靶子。

[latteczy]

@Shazoo 感谢！确实是这样。但是又有个问题是代码改正以后貌似没法统计运行时间了。。
https://gist.github.com/smartczy/35e692064b775dcc3e17#comment-1487649

[uniquecolesmith]

gevent 用起来

[aec4d]

这TM关GIL毛事 简直躺枪
要记住 断然给你提GIL的 要么是高手 要么是SB 有GIL还有multiprocessing呢 多线程最需要注意的应该是线程安全
这是threading.join的文档
join([timeout])
Wait until the thread terminates. This blocks the calling thread until the thread whose join() method is called terminates – either normally or through an unhandled exception – or until the optional timeout occurs.

When the timeout argument is present and not None, it should be a floating point number specifying a timeout for the operation in seconds (or fractions thereof). As join() always returns None, you must call isAlive() after join() to decide whether a timeout happened – if the thread is still alive, the join() call timed out.

When the timeout argument is not present or None, the operation will block until the thread terminates.

A thread can be join()ed many times.

join() raises a RuntimeError if an attempt is made to join the current thread as that would cause a deadlock. It is also an error to join() a thread before it has been started and attempts to do so raises the same exception.

看了之后你就发现 在上面加上t.join就和下面遍历的写法实质上是一模一样的.
一般网上很多教程给你加上setDaemon 加上join 你可以试着不加看是什么样的。等你熟悉了之后你就晓得那时候该用这些 哪时候不该用

[zhyu]

@latteczy
试试不要join

[latteczy]

@aec4d 多谢大神指点！大神莫激动。。。

[fanta]

aec4d 说的对，每个连接是0.7s时，问题已不是多线程的问题了， 是网络(服务器链路)问题.