python读取文件的正确方式

Python对文件的基础操作间python基础教程总结：Python 文件I/O部分；以下主要总结大文件和小文件操作过程中内存有效利用方法。

小文件

with函数(推荐使用)

The with statement handles opening and closing the file, including if an exception is raised in the inner block.

1
2
3

with open('myfile') as f:
    for line in f:
        <do something with line>

readlines/readline

1 2	for line in open('myfile','r').readlines(): do_something(line)

二者的区别是readlines读进来的是列表，而readline是字符串；

>>> import re
... with open('zsq.txt') as f:
...     lines = f.readlines()
...     print type(lines)
<type 'list'>
>>> import re
... with open('zsq.txt') as f:
...     lines = f.readline()
...     print type(lines)
<type 'str'>

大文件

fileinput

import fileinput

for line in fileinput.input(['myfile']):
    do_something(line)

fileinput.input() call reads lines sequentially, but doesn’t keep them in memory after they’ve been read or even simply so this.

结合with处理多个文件：

1
2
3

with fileinput.input(files=('spam.txt', 'eggs.txt')) as f:
    for line in f:
        process(line)

buffer

filePath = "input.txt"

buffer = "Read buffer:\n"
file = open(filePath, 'rU')
while(1):
    bytes = file.read(5)
    if bytes:
        buffer += bytes
    else:
        break

print buffer

贡献来源

http://stackoverflow.com/questions/8009882/how-to-read-large-file-line-by-line-in-python?noredirect=1&lq=1