Computer-science Posts Archive

---

• Aug 12, 2018 Parallel Buffered Read Writes

  Until this summer, I’d never written a piece of code where file IO was the performance-limiting factor. It’s often tempting to call file read-write times insignificant, but this notion certainly did not carry over to terabyte-sized datasets. It got to the point that I was having to plan ahead to avoid unnecessary file copies. So I quickly started looking, thinking to myself how can I make this faster?