How to Fix PGFPlots Out-of-Memory Errors with Large Data Files

A large data file can exhaust TeX’s memory when PGFPlots reads the table, converts values, and constructs the plotted paths. The right fix depends on which resource ran out, but for very large datasets the most reliable approach is usually to prepare a smaller, plot-ready file before TeX reads it.

Externalization is useful, but it is not a general solution to an out-of-memory error: PGFPlots must still process the data the first time it creates the external figure.

Identify what ran out

Check the end of the compiler log for the exact message. TeX may report a memory-capacity error, such as TeX capacity exceeded, or a different limit, such as input-stack size. These errors have different causes; increasing the wrong setting will not help. Also note which engine you use (pdflatex, xelatex, or lualatex) and whether the error occurs while compiling the plot itself or a larger document containing it.

If the plot compiles alone but not in the full document, the document may already be consuming substantial memory. Compiling the figure separately can help isolate that problem, though it will not make an intrinsically too-large plot cheap to generate.

Reduce the data before plotting

First ask what detail the final figure needs. A plot with millions of measurements may have far more points than can be distinguished at its printed size. Create a smaller file outside TeX by selecting representative points, aggregating into bins, or applying a domain-specific simplification method. Preserve important features such as peaks, discontinuities, and outliers; taking every hundredth row blindly can erase them.

This reduces work at the stage that matters: PGFPlots no longer has to read, store, and process every original row. In contrast, restricting the visible axis range or filtering points during plotting may still require PGFPlots to read the full table, so such options should not be assumed to solve a memory problem.

For example, if a measurement file has one million rows but the figure is a smooth overview, prepare a smaller file with a few thousand well-chosen points and plot that. Keep the original data for analysis; the reduced file is a visualization input, not a replacement for the source data.

Minimal example with externalization

This example reads a comma-separated file and externalizes the TikZ picture. The small sample table is included so the example compiles; replace its rows with your own data. Externalization requires compiling with shell escape enabled, for example pdflatex -shell-escape main.tex.

\documentclass{standalone}
\usepackage{pgfplots}
\pgfplotsset{compat=1.18}
\usetikzlibrary{external}
\tikzexternalize

\begin{filecontents*}{data.csv}
x,y
0,0
1,1
2,4
3,9
4,16
\end{filecontents*}

\begin{document}
\begin{tikzpicture}
  \tikzpicturedependsonfile{data.csv}
  \begin{axis}[
    xlabel={$x$},
    ylabel={$y$},
  ]
    \addplot table[
      col sep=comma,
      x=x,
      y=y,
    ] {data.csv};
  \end{axis}
\end{tikzpicture}
\end{document}

On the first build, TikZ creates a separate graphic from the plot. On later builds, it can reuse that graphic if the picture has not changed. For a table read from an external file, use \tikzpicturedependsonfile{data.csv} (as in the example) so a change to the data file triggers regeneration. This saves repeated compilation time and can keep the main document from regenerating an unchanged plot. It does not reduce the memory required to read and plot the full file during the first externalized build.

If you do not need externalization, remove \usetikzlibrary{external} and \tikzexternalize. The file can then be compiled without shell escape.

Other options and their limits

• Use a suitable TeX engine. Another engine or a newer TeX installation may have different available limits, but switching engines is not guaranteed to fix a PGFPlots table that is simply too large.
• Compile the figure separately. A small standalone document can avoid memory already used by a large thesis or presentation. It cannot overcome a limit exceeded by the plot on its own.
• Adjust TeX memory only when appropriate. TeX distributions expose engine-specific capacity settings, but changing them depends on the installation and may require rebuilding formats or using a different configuration. Follow the instructions for your distribution and the precise error; avoid copying an arbitrary memory setting from another system.
• Use a pre-rendered graphic when appropriate. If the figure does not need to remain editable as TikZ, generating it with a plotting program and including the resulting PDF can be more practical. This moves the data-processing workload out of TeX.

Common mistakes

Expecting externalization to fix the first build. It caches generated graphics for reuse; it does not skip the initial data processing.

Assuming axis limits discard input rows early. A plot may still need to read or process rows that fall outside the displayed range. Reduce the input file itself when memory is the issue.

Increasing memory before checking the error. A stack-size error, a memory-capacity error, and a problem in table parsing are not interchangeable. Use the log to identify the actual failure first.

Decimating without checking the result. A simple every-nth-row reduction can make a smooth curve look acceptable but omit short-lived peaks or other important events. Inspect the reduced plot and use a method suited to the data and the figure’s purpose.

For a large file, start by making a smaller visualization dataset and checking the plot in a standalone document. Add externalization to make repeated builds faster; investigate TeX capacity settings only if the reduced, isolated plot still fails.