SoftwareSideHustles
How-To

How to Read a File Line by Line in Python, Node.js, Java, Go, C# and Bash

Memory-efficient ways to read a file one line at a time in six languages, plus the bugs that come with it — trailing newlines, Windows line endings, encodings, unclosed files and Go's token too long error.

By Software Side Hustles Editorial Team · Published · Updated · 4 min read

Quick answer: iterate over the file object and let the language stream it. Python: for line in open(path, encoding="utf-8") inside a with block. Node.js: readline with for await. Java: Files.lines(path) in try-with-resources. Go: bufio.Scanner. C#: File.ReadLines(path). Bash: while IFS= read -r line. These read one line at a time, so a 20 GB log file uses almost no memory.

How do I read a file line by line in Python?

python
with open("app.log", encoding="utf-8") as f:
for line in f:
line = line.rstrip("\n")
if "ERROR" in line:
print(line)

Key points:

  • Each line keeps its newline. Strip it with rstrip("\n") (or rstrip() to remove trailing whitespace too).
  • Always pass encoding. Without it Python uses the platform default; on older Windows setups that is not UTF-8, and a file that works on your Mac raises UnicodeDecodeError on a colleague's machine.
  • Use with. It closes the file even if an exception is raised.
  • f.readlines() and f.read().splitlines() load the whole file into memory. That is fine for small config files and a problem for large logs.

Want line numbers? for n, line in enumerate(f, start=1):.

How do I read a file line by line in Node.js?

javascript
import { createReadStream } from "node:fs";
import { createInterface } from "node:readline";
const rl = createInterface({ input: createReadStream("app.log"), crlfDelay: Infinity });
for await (const line of rl) {
if (line.includes("ERROR")) console.log(line);
}

crlfDelay: Infinity makes Windows \r\n endings count as a single line break. Lines are delivered without their newline characters.

For small files, reading everything at once is simpler:

javascript
import { readFileSync } from "node:fs";
const lines = readFileSync("config.txt", "utf8").split(/\r?\n/);

Note the \r? — splitting on "\n" alone leaves \r at the end of every line of a Windows file, which then breaks string comparisons in ways that are invisible when printed.

How do I read a file line by line in Java?

java
try (Stream<String> lines = Files.lines(Path.of("app.log"))) {
lines.filter(l -> l.contains("ERROR")).forEach(System.out::println);
}
try (BufferedReader reader = Files.newBufferedReader(Path.of("app.log"))) {
String line;
while ((line = reader.readLine()) != null) {
// process line
}
}

Files.lines returns a lazy stream backed by an open file, so it must be closed — hence the try-with-resources. Forgetting it leaks file handles, and long-running servers eventually fail with "Too many open files". Files.readAllLines loads everything into a list. Since Java 18 the default charset is UTF-8; on older versions pass StandardCharsets.UTF_8 explicitly.

How do I read a file line by line in Go?

go
f, err := os.Open("app.log")
if err != nil {
return err
}
defer f.Close()
scanner := bufio.NewScanner(f)
scanner.Buffer(make([]byte, 0, 64*1024), 10*1024*1024) // allow lines up to 10 MB
for scanner.Scan() {
line := scanner.Text()
_ = line
}
if err := scanner.Err(); err != nil {
return err
}

Two Go-specific traps:

  1. By default a Scanner rejects lines longer than 64 KB. Scan() simply returns false and the error — bufio.Scanner: token too long — only appears if you check scanner.Err(). Minified JSON logs hit this regularly. Raise the limit with scanner.Buffer as above, or use bufio.Reader.ReadString('\n').
  2. Always check scanner.Err() after the loop. Otherwise a read error looks exactly like the end of the file.

C# and Bash

csharp
foreach (var line in File.ReadLines("app.log")) { /* streamed */ }
var all = File.ReadAllLines("app.log"); // loads everything

ReadLines is lazy; ReadAllLines is not. The names are close enough that people swap them by accident.

bash
while IFS= read -r line || [ -n "$line" ]; do
printf '%s\n' "$line"
done < app.log
  • IFS= keeps leading and trailing whitespace.
  • -r stops backslashes being treated as escapes.
  • || [ -n "$line" ] processes a final line that has no trailing newline, which read would otherwise skip.
  • Do not write for line in $(cat file) — it splits on spaces, not lines, and expands wildcards.

Common problems when reading files

SymptomCauseFix
String comparisons fail though the text looks identicalTrailing \n or \rStrip line endings; split on \r?\n
UnicodeDecodeError / garbled charactersWrong encodingSpecify UTF-8 (or the file's real encoding)
First line has an invisible characterUTF-8 byte order markPython: encoding="utf-8-sig"; strip  in JavaScript
FileNotFoundError though the file existsRelative path resolved from a different working directoryBuild paths from the script location: Path(__file__).parent / "data.txt"
Process slows and memory climbsWhole file loaded into memoryStream line by line
"Too many open files"Files never closedwith, try-with-resources, defer f.Close()

Frequently asked questions

How do I read only the last N lines of a file?

Python: collections.deque(f, maxlen=n) streams the file and keeps the last n lines. On the command line, tail -n 100 app.log.

How do I read a file in chunks instead of lines?

For binary data use fixed-size reads: while chunk := f.read(65536): in Python, a ReadableStream in Node.js, or InputStream.read(byte[]) in Java.

How do I count lines in a large file quickly?

sum(1 for _ in f) in Python, or wc -l file on Unix (which counts newline characters, so a final line without a newline is not counted).

Why does my loop skip every other line?

You are probably calling readline() inside a for line in f loop, which consumes an extra line per iteration. Use one or the other.

Where this skill pays off

"Read a large log and report the errors" is a typical prompt in AI coding evaluations — and the typical weak answer calls read() on the whole file, forgets the encoding or ignores scanner.Err(). Spotting that is a paid skill; see how to evaluate AI-generated code or practise in the free code review lab.

About the author

Written and reviewed by the Software Side Hustles editorial team. Code examples are executed before publication, and platform facts link to the platform's own pages with the date they were checked.

Our editorial policy

Cookies on this site

We use essential cookies to run the questionnaire, practice labs and sign-in. With your permission we'd also like to use analytics to understand which pages help people, and advertising cookies to measure our Google Ads. Rejecting never limits any feature. Cookie policy