‹ Back to Benchmarks

Baseline RAM and GPU-ready VRAM: OpenNN vs PyTorch vs TensorFlow

OpenNN has the lowest baseline RAM and GPU-ready VRAM in this matched footprint benchmark. It uses 240 MB of RAM versus 833 MB for PyTorch and 1,014 MB for TensorFlow, then occupies 250 MB of VRAM after GPU initialization versus 306 MB and 252 MB.

This is a framework-readiness footprint, measured from one representative process per engine. It is not peak training memory, maximum model capacity, or the memory required by a production dataset.

Contents

Introduction

A framework consumes memory before a real model or dataset becomes large. Importing and initializing the runtime, constructing the minimum training objects, creating thread pools, and preparing the GPU all establish a footprint that every application must carry.

This benchmark separates that cost into two measurements. Baseline RAM captures the resident host memory after building an empty training application. GPU-ready VRAM captures the device memory visible after one tiny fp32 matrix multiplication has initialized the CUDA math path. Keeping both results in one article shows where each framework places its readiness cost.

Benchmark application

Component Matched setup
Dataset Empty dataset with one input and one target
Model Minimal 1 -> 1 dense network
Loss Mean squared error
Optimizer Adam
RAM checkpoint After constructing dataset, model, loss, and optimizer/training strategy
GPU checkpoint After one 32×32 fp32 matrix multiplication and synchronization
GPU allocation policy Lazy module loading; TensorFlow memory growth enabled
Metrics Resident RAM and process GPU memory in MB; lower is better

The purpose is to initialize equivalent framework capabilities with negligible model and data storage. That makes runtime and context overhead the dominant measurement.

Reference computer

Component Value
CPU Intel Core i9-12900K
GPU NVIDIA GeForce RTX 4080, 16 GB
GPU operation 32×32 fp32 matrix multiplication
Frameworks OpenNN, PyTorch, TensorFlow
OpenNN 9.0.0
Samples One representative fresh process per framework

Methodology

Each framework runs in its own process. The runner first records total GPU memory in use, imports or initializes the framework, and constructs the minimum objects needed by a training application.

  • OpenNN creates an empty TabularDataset, a 1-to-1 ApproximationNetwork, and a TrainingStrategy configured with mean squared error and Adam.
  • PyTorch creates an empty TensorDataset, a 1-to-1 Linear module, MSELoss, and Adam.
  • TensorFlow creates an empty Dataset, a one-unit Keras Dense model, MeanSquaredError, and Adam.

Current resident memory is recorded at that point. On Windows this corresponds to the process working set; on Linux it corresponds to current RSS. The runner then performs and synchronizes the same tiny GPU matrix multiplication. GPU memory is read for the process through nvidia-smi; when per-process reporting is unavailable, the runner uses the before/after device-memory difference.

No real dataset, training epoch, or large activation tensor is included. One measurement is stored per framework, so small differences should not be interpreted beyond the precision justified by the process and driver reporting tools.

Results

Baseline RAM

Baseline resident RAM
Lower is better
Framework readiness · MB
OpenNN
240 MB
PyTorch
833 MB
TensorFlow
1,014 MB
Framework Baseline RAM Relative to OpenNN
OpenNN 240 MB 1.00x
PyTorch 833 MB 3.47x
TensorFlow 1,014 MB 4.23x

OpenNN uses 71.2% less baseline RAM than PyTorch and 76.3% less than TensorFlow in this setup.

GPU-ready VRAM

GPU-ready process VRAM
Lower is better
After GPU initialization · MB
OpenNN
250 MB
PyTorch
306 MB
TensorFlow
252 MB
Framework GPU-ready VRAM Relative to OpenNN
OpenNN 250 MB 1.00x
PyTorch 306 MB 1.22x
TensorFlow 252 MB 1.01x

OpenNN uses 56 MB less VRAM than PyTorch after initialization. OpenNN and TensorFlow differ by only 2 MB, or 0.8%, and should be treated as effectively tied at the granularity of this measurement.

Interpreting RAM and VRAM

The RAM result includes framework runtime and application-object overhead, not just model weights. Python-based frameworks also carry the interpreter and their native libraries inside the process. OpenNN runs as a native application, which helps keep its baseline below the two reference frameworks.

The GPU-ready result tells a different story. All three engines must initialize NVIDIA runtime and math-library state, so a common device-side floor dominates. That is why TensorFlow and OpenNN are nearly identical at about 250 MB even though their host-memory footprints differ substantially.

GPU-ready VRAM does not predict training peak memory or maximum batch size. Those depend on parameters, activations, gradients, optimizer state, workspace selection, and allocation strategy. This benchmark intentionally stops before those model-dependent costs appear.

Discussion

OpenNN’s clearest footprint advantage is host memory. The 240 MB baseline is less than one third of PyTorch’s and less than one quarter of TensorFlow’s. This matters for memory-limited containers, services that run several model processes, and desktop or edge applications that share RAM with the rest of the system.

On the GPU, OpenNN is still lowest, but the result is nuanced: its 250 MB is meaningfully below PyTorch’s 306 MB and effectively tied with TensorFlow’s 252 MB. The correct conclusion is not that one framework will always use less VRAM during a real workload, but that OpenNN reaches a ready CUDA state without adding a larger baseline than the alternatives measured here.

Conclusions

  • OpenNN records the lowest baseline RAM at 240 MB.
  • PyTorch uses 833 MB and TensorFlow 1,014 MB in the same minimal training-object setup.
  • GPU-ready VRAM is 250 MB for OpenNN, 306 MB for PyTorch, and 252 MB for TensorFlow.
  • OpenNN and TensorFlow are effectively tied on ready-state VRAM; OpenNN uses 18.3% less than PyTorch.
  • These values measure framework readiness, not peak training memory or model capacity.

Reproducing

Build the OpenNN memory target, then launch the three runners in separate fresh processes:

cmake --build build-benchmarks --target opennn_memory
cd docs/benchmarks/footprint/memory

CUDA_MODULE_LOADING=LAZY ./opennn_memory
CUDA_MODULE_LOADING=LAZY python pytorch_memory.py
CUDA_MODULE_LOADING=LAZY TF_FORCE_GPU_ALLOW_GROWTH=true python tensorflow_memory.py

Each runner prints baseline_ram_mb and gpu_ready_vram_mb.

References