Import One Million Rows to Database (PHP/Laravel)

Importing a handful of rows from a CSV is trivial. Importing a million is a different problem — the code that works fine for a thousand rows will exhaust memory, hit the execution timeout, or take hours. The naive version usually looks like this:

$rows = array_map('str_getcsv', file(storage_path('imports/records.csv')));

foreach ($rows as $row) {
    Record::create([
        'name'  => $row[0],
        'email' => $row[1],
    ]);
}

Two things go wrong here, and both get worse as the file grows. file() loads the entire CSV into memory as an array of strings. And Record::create() runs a separate INSERT per row — a million round trips to the database, each with its own query parsing, and each booting a fresh Eloquent model.

There are three separate bottlenecks to deal with: reading the file, holding it in memory, and insert throughput.

Step 1: Stream the file instead of loading it

Never read the whole file at once. PHP can walk a CSV line by line with a constant memory footprint, and a generator lets you do that without building an array:

function readCsv(string $path): Generator
{
    $handle = fopen($path, 'r');

    // Pull the header row first so we can key each record by column name.
    $header = fgetcsv($handle);

    while (($row = fgetcsv($handle)) !== false) {
        yield array_combine($header, $row);
    }

    fclose($handle);
}

Memory usage now stays flat regardless of whether the file has a thousand rows or ten million, because only one row exists at a time.

Laravel's LazyCollection wraps the same idea with the collection API you already know:

use Illuminate\Support\LazyCollection;

LazyCollection::make(function () {
    $handle = fopen(storage_path('imports/records.csv'), 'r');
    $header = fgetcsv($handle);

    while (($row = fgetcsv($handle)) !== false) {
        yield array_combine($header, $row);
    }

    fclose($handle);
})->chunk(1000)->each(function ($chunk) {
    // insert the chunk
});

Step 2: Insert in batches, not one row at a time

This is where most of the time goes. A single INSERT statement carrying a thousand rows is dramatically faster than a thousand statements carrying one row each — you pay the network round trip and query parsing cost once instead of a thousand times.

Use the query builder rather than Eloquent. Model::create() instantiates a model, fires creating/created events, and applies casts and mutators for every row. None of that helps during a bulk import:

use Illuminate\Support\Facades\DB;

DB::table('records')->insert($chunk->toArray());

A chunk size of 500 to 2,000 is a reasonable starting range. Bigger chunks mean fewer round trips but larger queries — if you push it too far you will hit MySQL's max_allowed_packet limit, which surfaces as a dropped connection rather than an obvious error. Tune it against your own data.

If the CSV may contain rows you already have, use insertOrIgnore() to skip duplicates, or upsert() to update them:

DB::table('records')->upsert(
    $chunk->toArray(),
    ['email'],           // unique column(s) to match on
    ['name', 'updated_at'] // columns to update when a match is found
);

Step 3: Stop paying per-row costs you don't need

Several defaults are helpful in normal request handling and pure overhead during an import.

Disable the query log. In a long-running process Laravel accumulates every executed query in memory. Over a million rows that is a slow memory leak:

DB::connection()->disableQueryLog();

Set timestamps yourself. The query builder does not manage created_at and updated_at, so add them to your row arrays once per chunk rather than letting Eloquent compute them per model:

$now = now();

$chunk = $chunk->map(fn ($row) => $row + [
    'created_at' => $now,
    'updated_at' => $now,
]);

Wrap chunks in a transaction. Committing once per chunk instead of once per row cuts the number of disk flushes substantially:

DB::transaction(function () use ($chunk) {
    DB::table('records')->insert($chunk->toArray());
});

Consider your indexes. Every index has to be updated on every insert. If you are importing into an empty table, it can be faster to import first and add the indexes afterwards, letting MySQL build them in one pass. This is only worth doing on an empty or new table — dropping indexes on a live table has obvious consequences.

Step 4: Move it to a queued job

A million-row import should not run inside a web request. Even from the CLI, one long-running process is fragile: if it fails at row 900,000 you have no clean way to resume.

Push the work to the queue, and split it into chunked jobs so each one is small, retryable, and independently recoverable:

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\DB;

class ImportRecordsJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public int $timeout = 600;
    public int $tries = 3;

    public function __construct(public array $rows)
    {
    }

    public function handle(): void
    {
        DB::connection()->disableQueryLog();

        DB::transaction(function () {
            DB::table('records')->insert($this->rows);
        });
    }
}

Then dispatch one job per chunk as you stream the file. Laravel's job batching is a good fit here — it gives you progress tracking and a single completion callback across all the chunks.

The fastest option: LOAD DATA INFILE

If you need raw speed and the data is already clean, MySQL can read the file directly. This skips PHP entirely for the row-by-row work and is significantly faster than anything you can do in application code:

LOAD DATA LOCAL INFILE '/path/to/records.csv'
INTO TABLE records
FIELDS TERMINATED BY ',' ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS
(name, email);

The trade-offs are real, though:

• The local_infile setting must be enabled on both the server and the client, and it is disabled by default in many builds for security reasons.
• You get no application-level validation, no events, and no transformation — the data goes in as it is in the file.
• Error handling is coarse. A malformed row produces a warning, not an exception you can catch per row.

It is an excellent tool for trusted, pre-validated data, and the wrong tool for user-uploaded files that need checking.

Measure it, don't guess

Import performance depends heavily on your hardware, your MySQL configuration, your indexes, and the shape of your data. Benchmark on your own setup rather than trusting anyone's numbers:

$start = microtime(true);

// ... run the import ...

$this->info(sprintf(
    'Imported %s rows in %.2fs, peak memory %.1f MB',
    number_format($count),
    microtime(true) - $start,
    memory_get_peak_usage(true) / 1024 / 1024
));

Track peak memory as well as elapsed time. A version that finishes quickly but peaks near your memory_limit will fail on a slightly larger file.

Things that will catch you out

• memory_limit and max_execution_time. Streaming solves the first for the CSV itself, but not for anything you accumulate in a loop.
• max_allowed_packet. Too large a chunk and MySQL rejects the query — often appearing as a lost connection rather than a clear error message.
• Character encoding. A file that is not UTF-8 will produce mangled text or insert failures. Convert it before importing, not after.
• Idempotency. Assume the import will be run twice. insertOrIgnore() or upsert() against a unique key makes that safe.

Summary

The pattern that works is consistent regardless of the framework: stream the file, batch the inserts, strip per-row overhead, and run it on the queue. Reach for LOAD DATA INFILE when the data is trusted and speed matters more than validation.

The largest gain by far comes from batching. Moving from one insert per row to one insert per thousand rows removes almost all of the round trips, and it is usually the difference between an import that takes hours and one that takes minutes.