GPU HIGGS dense training: OpenNN vs PyTorch vs TensorFlow

OpenNN trains the canonical HIGGS dense classifier at 6.76 million samples/s in fp32 and 11.35 million samples/s in bf16 on an NVIDIA GeForce RTX 4080, leading both PyTorch and TensorFlow in the same five-epoch workload.

Contents

• Introduction
• Benchmark application
• Reference computer
• Methodology
• Results
• Held-out quality
• Discussion
• Conclusions
• Reproducing
• References

Introduction

HIGGS is a large tabular binary-classification dataset from high-energy physics. Its 10.5-million-row training split and 28 numerical features make it useful for measuring sustained dense-network throughput rather than a short synthetic kernel.

This GPU training benchmark uses the canonical 28-1024-1024-1 ReLU classifier with Adam, batch 7,000, five epochs, and fp32 and bf16 paths in OpenNN, PyTorch, and TensorFlow.

Benchmark application

Reference computer

Methodology

The three engines read the same prepared and normalized HIGGS split. Training throughput is calculated from 10.5 million training rows and the median epoch time produced inside each engine.

• OpenNN uses GPU-resident data and its CUDA graph training path.
• PyTorch uses its optimized dense path, with bf16 autocast in the bf16 cell.
• TensorFlow uses compiled graph execution and mixed bf16 in the bf16 cell.
• All models use the same layer widths, activation, parameter count, batch, and epoch count.
• Testing happens after training and is not part of the throughput numerator.
• One run is stored per cell, so the figures should be read as a controlled snapshot.

Results

Held-out quality

Throughput is only meaningful if the training loop produces a usable model. The runner reports the following held-out metrics after five epochs:

The models land in the same broad quality band, but TensorFlow has the best held-out log loss and AUC in this specific five-epoch snapshot. Because no common hard target was configured, this article does not convert throughput into a convergence-to-quality claim.

Discussion

OpenNN’s largest relative lead is fp32, where it processes 2.12x as many samples per second as PyTorch and 2.23x as many as TensorFlow. In bf16, the competitor implementations also benefit strongly from tensor-core execution; OpenNN remains first at 1.34x PyTorch and 1.68x TensorFlow.

Within OpenNN, bf16 improves throughput by 1.68x for this exact network. The absolute 11.35-million-sample/s result is the fastest cell in the training table.

The quality table is important context. OpenNN’s advantage is sustained training throughput, while the final five-epoch metrics are not identical and should not be described as a formal quality tie without repeated seeds or a fixed target.

Conclusions

• OpenNN leads both fp32 and bf16 training throughput.
• In fp32, OpenNN reaches 2.12x PyTorch and 2.23x TensorFlow.
• In bf16, OpenNN reaches 1.34x PyTorch and 1.68x TensorFlow.
• All held-out metrics are published so the throughput result can be interpreted with its quality context.

Reproducing

The canonical runner is docs/benchmarks/throughput/higgs-gpu/run_higgs_dense.py:

python run_higgs_dense.py \
  --train "$OPENNN_BENCH_DATA/higgs/higgs_train.csv" \
  --test "$OPENNN_BENCH_DATA/higgs/higgs_test.csv" \
  --epochs 5 --batch 7000 --hidden 1024 --hidden-layers 2 \
  --activation relu --shuffle shuffle --precision both --runs 1

The result artifact is docs/benchmarks/results/gpu-higgs-dense-training-speed-20260710T084732Z.json.

References

• HIGGS dataset, UCI Machine Learning Repository.
• Searching for exotic particles in high-energy physics with deep learning.
• OpenNN source repository.
• PyTorch.
• TensorFlow.