Python list comprehension 和 append

2017-05-16 留言

Python list comprehension 和 append

最近有被问list comprehension 和一个列表不停地append哪个效率高，为什么？
表示没关注过(so sad)，因为感觉list comprehension在可读性上并不好，而且也无法在遍历时候
捕获异常，下来测试学习下，记录如下：

测试1，是否list comprehension就比append快：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def test():
    """Stupid test function"""
    L = []
    for i in range(100):
        L.append(i*2)
    return L
def test2():
    return [i * 2 for i in range(100)]
if __name__ == '__main__':
    import timeit
    print(timeit.timeit("test()", setup="from __main__ import test"))
    print('------')
    print(timeit.timeit("test2()", setup="from __main__ import test2"))

结果如下：
1
2
3
16.2206947803
------
12.4496650696

看来list comprehension确实比在一个列表上不停append要快。

测试2，使用dis查看函数的字节码：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import dis
def list_c(nums):
    return [i*2 for i in nums]
print 'return [i*2 for i in nums]'
dis.dis(list_c)
print '--------------'
print '--------------'
print '--------------'
def append_list(nums):
    alist = []
    for i in nums:
        alist.append(i*2)
    return alist
dis.dis(append_list)

结果如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
return [i*2 for i in nums]
  5           0 BUILD_LIST               0
              3 LOAD_FAST                0 (nums)
              6 GET_ITER            
        >>    7 FOR_ITER                16 (to 26)
             10 STORE_FAST               1 (i)
             13 LOAD_FAST                1 (i)
             16 LOAD_CONST               1 (2)
             19 BINARY_MULTIPLY     
             20 LIST_APPEND              2
             23 JUMP_ABSOLUTE            7
        >>   26 RETURN_VALUE        
--------------
--------------
--------------
 17           0 BUILD_LIST               0
              3 STORE_FAST               1 (alist)
 18           6 SETUP_LOOP              31 (to 40)
              9 LOAD_FAST                0 (nums)
             12 GET_ITER            
        >>   13 FOR_ITER                23 (to 39)
             16 STORE_FAST               2 (i)
 19          19 LOAD_FAST                1 (alist)
             22 LOAD_ATTR                0 (append)
             25 LOAD_FAST                2 (i)
             28 LOAD_CONST               1 (2)
             31 BINARY_MULTIPLY     
             32 CALL_FUNCTION            1
             35 POP_TOP             
             36 JUMP_ABSOLUTE           13
        >>   39 POP_BLOCK           
 20     >>   40 LOAD_FAST                1 (alist)
             43 RETURN_VALUE

通过字节码，可以看出来，list comprehension的指令更少，不需要建立列表变量，同时，也就无需取变量，
更不需要调用list的append函数（调用函数需要维护stack信息），也就比append要快。

以下是StackOverflow上某大神的回答：

A list comprehension is usually a tiny bit faster than the precisely equivalent for loop (that actually builds a list), most likely because it doesn’t have to look up the list and its append method on every iteration. However, a list comprehension still does a bytecode-level loop:

dis.dis()
1 0 BUILD_LIST 0
3 LOAD_FAST 0 (.0)
6 FOR_ITER 12 (to 21)
9 STORE_FAST 1 (x)
12 LOAD_FAST 1 (x)
15 LIST_APPEND 2
18 JUMP_ABSOLUTE 6
21 RETURN_VALUE

Using a list comprehension in place of a loop that doesn’t build a list, nonsensically accumulating a list of meaningless values and then throwing the list away, is often slower because of the overhead of creating and extending the list. List comprehensions aren’t magic that is inherently faster than a good old loop.

查看Python的官方wiki的Loop页面，也有提到map和for loop中append的问题：

1
2
3
newlist = []
for word in oldlist:
    newlist.append(word.upper())

和：
1
newlist = map(str.upper, oldlist)

这里map要比for loop快，因为for loop是在解释器中执行的，而map则是编译好的C代码。

测试代码如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
def test():
    """Stupid test function"""
    oldlist = 'abcde'
    newlist = []
    for word in oldlist:
        newlist.append(word.upper())
    return newlist
def test2():
    oldlist = 'abcde'
    return map(str.upper, oldlist)
if __name__ == '__main__':
    import timeit
    print(timeit.timeit("test()", setup="from __main__ import test"))
    print('------')
    print(timeit.timeit("test2()", setup="from __main__ import test2"))

结果如下：
1
2
3
3.65194892883
------
2.49910998344

参考：
https://wiki.python.org/moin/PythonSpeed/PerformanceTips#Loops
http://stackoverflow.com/questions/22108488/are-list-comprehensions-and-functional-functions-faster-than-for-loops
http://stackoverflow.com/questions/1247486/python-list-comprehension-vs-map

#Python