python读取文件的正确方式


Python对文件的基础操作间python基础教程总结:Python 文件I/O部分;以下主要总结大文件和小文件操作过程中内存有效利用方法。

小文件

with函数(推荐使用)

The with statement handles opening and closing the file, including if an exception is raised in the inner block.

1
2
3
with open('myfile') as f:
for line in f:
<do something with line>

readlines/readline

1
2
for line in open('myfile','r').readlines():
do_something(line)

二者的区别是readlines读进来的是列表,而readline是字符串;

1
2
3
4
5
6
7
8
9
10
>>> import re
... with open('zsq.txt') as f:
... lines = f.readlines()
... print type(lines)
<type 'list'>
>>> import re
... with open('zsq.txt') as f:
... lines = f.readline()
... print type(lines)
<type 'str'>


大文件

fileinput

1
2
3
4
import fileinput

for line in fileinput.input(['myfile']):
do_something(line)

fileinput.input() call reads lines sequentially, but doesn’t keep them in memory after they’ve been read or even simply so this.

  • 结合with处理多个文件:
    1
    2
    3
    with fileinput.input(files=('spam.txt', 'eggs.txt')) as f:
    for line in f:
    process(line)
  • buffer

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    filePath = "input.txt"

    buffer = "Read buffer:\n"
    file = open(filePath, 'rU')
    while(1):
    bytes = file.read(5)
    if bytes:
    buffer += bytes
    else:
    break

    print buffer

    贡献来源

    http://stackoverflow.com/questions/8009882/how-to-read-large-file-line-by-line-in-python?noredirect=1&lq=1

    tiramisutes wechat
    欢迎关注