GPU ResNet-50 training: OpenNN vs PyTorch vs TensorFlow

OpenNN trains ResNet-50 at 29,069 images/s in fp32 and 35,097 images/s in bf16 on an NVIDIA GeForce RTX 4080, reaching up to 2.14x the throughput of the compared framework implementations.

Contents

• Introduction
• Benchmark application
• Reference computer
• Methodology
• Results
• Discussion
• Conclusions
• Reproducing
• References

Introduction

ResNet-50 combines convolution, batch normalization, residual connections, activation functions, and a classifier head. On CIFAR-sized images, individual kernels are short and launch overhead is an important part of the measured training step. This makes the workload useful for comparing the complete optimized framework path rather than only isolated convolution kernels.

Benchmark application

Reference computer

Methodology

The canonical harness prepares one CIFAR-10 dataset and invokes each framework alone for the same model, batch, epochs, and precision. Each engine uses its fair optimized path:

• OpenNN uses GPU-resident image data and CUDA graph execution.
• PyTorch uses channels-last layout, torch.compile, TF32 policy in fp32, and bf16 autocast.
• TensorFlow uses XLA and the mixed-bfloat16 policy for bf16.
• Framework setup and graph compilation are excluded from the reported steady-state throughput.
• The artifact contains one successful run per engine and precision. A repeated-run variance study remains future work.

The result JSON does not contain a common accuracy or loss target. The comparison therefore supports claims about executed training throughput at a fixed configuration, not convergence or final predictive quality.

Results

Discussion

OpenNN leads by the widest margin in fp32: 89.6% more throughput than PyTorch and 114.3% more than TensorFlow. The bf16 comparison is closer because all engines benefit from reduced-precision tensor-core execution, but OpenNN remains 79.5% ahead of PyTorch and 90.9% ahead of TensorFlow.

OpenNN itself gains 20.7% moving from fp32 to bf16 for this model and batch. PyTorch gains 27.5% and TensorFlow 35.6%, so reduced precision narrows the relative OpenNN lead while raising absolute throughput for all three engines.

The result is specific to 32×32 CIFAR geometry. At 224×224 ImageNet geometry, convolution arithmetic has a larger share of the total step and these ratios should not be assumed. A separate ImageNet-geometry result is required before making that claim.

Conclusions

• OpenNN leads both tested precisions on CIFAR-10 geometry.
• OpenNN reaches 1.90x PyTorch and 2.14x TensorFlow throughput in fp32.
• OpenNN reaches 1.80x PyTorch and 1.91x TensorFlow throughput in bf16.
• The result is training throughput, not a time-to-accuracy comparison.

Reproducing

The canonical runner is docs/benchmarks/throughput/resnet50/run_resnet50.py:

python run_resnet50.py --dataset cifar10 --epochs 2 --batch 128 --runs 1 --precision both

The result artifact is docs/benchmarks/results/gpu-resnet50-training-speed-cifar10-20260710T120445Z.json.

References

• Deep Residual Learning for Image Recognition.
• CIFAR-10 dataset.
• OpenNN source repository.
• PyTorch compilation documentation.
• TensorFlow XLA documentation.