storing parsed csv using CsvHelper as suitable datatype [closed]

The best place to start would be with the CsvHelper Getting Started page.

What can be confusing is that csv.GetRecords<dynamic>(); returns IEnumerable<dynamic> which will yield records as you iterate over the IEnumerable object. What this means is you are not getting a list of items yet, what you are getting is an enumerator which keeps track of where you are in getting records and only returns them as you ask for them.

Often you are getting csv files from disk and this can be a costly operation. Say you just wanted to get the first 5 records from a file that had 50 million records. You could call

var records = csv.GetRecords<dynamic>().Take(5).ToList();

This would iterate through the first 5 records on disk and then return a List<dynamic> of 5 records. If you flipped ToList() and Take(5), the means of getting the data would be very different.

var records = csv.GetRecords<dymanic>().ToList().Take(5);

This would iterate through all 50 million records on disk and bring them into memory. Then it would take the first 5 of those records and return them to you as IEnumerable<dynamic>

The difference could be a fraction of a second vs minutes in getting those 5 records and considerably more memory used.

CLICK HERE to find out more related problems solutions.

Leave a Comment

Your email address will not be published.

Scroll to Top