Php Read From Text File Line by Line

If y'all work for a company that offers development services fifty-fifty for other companies of the same industry, you lot may have had this "wonderful" task of importing a huge "database" from a customer into the database engine preferred by your company. For instance, in our visitor, nosotros piece of work with MySQL and our client came upwardly with a CSV file of approximately 25GB with ~7.5M rows.

What?

Obviously, because of the logic that the projection required, we couldn't just just import the file into the database through a tool like PHPMyAdmin, every bit every row in the CSV should exist modificated to fit with our new database design.

In this article, nosotros will explain you our arroyo for reading efficiently a huge CSV file in PHP.

1. Carve up your file into smaller chunks

To get started, when we talk nigh huge files, we are not talking about files with 50K or 70K rows, we talk nearly millions of rows like in this instance, with a CSV file of 25GB. Then, the correct approach for such cases is not to piece of work with the file directly, but with smaller files.

The smaller the file, the ameliorate will it exist to have an optimal operation and control over your script, not only virtually the performance perspective but the logic as well. Nosotros wrote an article previously of how to separate huge CSV datasets into smaller chunks using CSV splitter, a tool for Windows ten. You can of grade practice the same using another approach, simply, yous become the idea right? Split the file into smaller chunks that can be easily candy by your scripts later.

two. Implementing the read and iteration script

To read the file, we will use the fopen function of PHP, this inbuilt function is used to merely open a file from a local URL, it's used to bind a resources to a steam. Information technology expects as second argument the fashion in which we'll operate, in this case, just reading with the r identifier. The method returns a file pointer as long as the file exists, otherwise it will render False in case of failure.

We will read the file using this method and will store the arrow into the $handle variable. Create equally well the variable that will shop the current line number every bit we'll iterate over the rows with an entry controlled loop (while). With a while loop, we will iterate over every single row of the file, verifying the condition that fgets ever return some content.

The fgets part of PHP returns a line from an open file with fopen and it returns imitation when there's zip left to read. Then, within the while loop, yous will exist able to parse the raw CSV string with the str_getcsv role. Having the basic stuff implemented, you will be ready to modify the script to practise whatever yous demand to practice with the all-time possible operation in PHP:

          // Read a CSV file $handle = fopen("my_huge_csv_file.csv", "r");  // Optionally, you tin keep the number of the line where // the loop its currently iterating over $lineNumber = ane;  // Iterate over every line of the file while (($raw_string = fgets($handle)) !== fake) {     // Parse the raw csv string: "one, a, b, c"     $row = str_getcsv($raw_string);      // into an array: ['one', 'a', 'b', 'c']     // And practice what you need to do with every line     var_dump($row);          // Increase the current line     $lineNumber++; }  fclose($handle);

The advantages of this arroyo are:

You are not directly reading the entire file in memory merely similar file_get_contents does, then the max amount of memory needed to run the script depends on the longest line in the input data.
Quite easy to read and sympathize.

Happy coding ❤️!

Murphy Divictlerner

Php Read From Text File Line by Line

1. Carve up your file into smaller chunks

two. Implementing the read and iteration script

Enregistrer un commentaire for "Php Read From Text File Line by Line"