Capabilities
What each library can do: network types, ready-made models and GPU features.
10 of 12
Similar capabilities
OpenNN and PyTorch share 10 of the 12 capabilities we compared, including the four network types we benchmark. PyTorch adds image generation and distributed training.
Code
The amount of code in each library and in a small application written with it.
96%
Less code
The OpenNN library has 96% fewer source lines than PyTorch, and a small classification application takes 9 lines instead of 29.
Training
Speed, peak memory and energy while training dense and LSTM networks on CPU and GPU, and CNN and Transformer networks on GPU.
6 of 6
Faster training
OpenNN trained faster than PyTorch in all six configurations of dense, LSTM, CNN and Transformer networks.
6 of 6
Less memory for training
OpenNN used less peak memory than PyTorch while training, on both CPU and GPU.
6 of 6
Less energy for training
OpenNN used less processor or graphics-board energy than PyTorch for the same training work.
Quality
Prediction quality of the trained networks, measured on held-out data.
4 of 4
Similar prediction quality
Trained from the same initial weights, OpenNN and PyTorch reached similar test scores with dense, LSTM, CNN and Transformer networks.
Deployment
Disk space and startup time of small applications built with each library.
65–99%
Smaller deployment
An OpenNN application needs less disk space than the same application on a standard PyTorch Python installation, without counting model weights.
63–99.8%
Faster startup
OpenNN applications delivered their first prediction sooner after launch in every tested CPU and GPU configuration.
Inference
Speed, peak memory and energy while running trained dense and LSTM networks on CPU and GPU, and CNN and Transformer networks on GPU.
6 of 6
Faster inference
OpenNN made predictions faster than PyTorch in all six configurations of dense, LSTM, CNN and Transformer networks.
6 of 6
Less memory for inference
OpenNN used less peak memory than PyTorch while making predictions, on both CPU and GPU.
6 of 6
Less energy for inference
OpenNN used less processor or graphics-board energy than PyTorch for the same predictions.
How we measured
- Reference computer: Intel Core i7-14700F, 32 GB RAM, NVIDIA GeForce RTX 5070 Ti, Linux. PyTorch 2.13.0 built for CUDA 13.0, NVIDIA driver 610.43.02.
- Same work: both libraries ran the same networks on the same data, with the same batch sizes and parameter counts, checked before every timed run.
- Speed: median of three launches after a warm-up, rotating which library ran first, with GPU clocks locked.
- Memory: highest of three launches; private resident memory on the CPU, device memory above the idle baseline on the GPU.
- Energy: median of three launches; processor package on the CPU, graphics board on the GPU. Not energy at the wall socket.
- Precision: FP32 with Intel MKL on the CPU, BF16 mixed precision on the GPU, and FP32 for the prediction-quality runs.
The benchmark programs for both libraries, the data preparation scripts and the full measurement rules are in the OpenNN repository on GitHub.
Limits and next benchmarks
- The results describe these networks, this computer and these software versions. Other workloads and hardware can give different ratios.
- CPU speed, memory and energy for the CNN and Transformer are not yet measured.
- In the GPU LSTM runs, PyTorch used FP16 recurrent kernels while OpenNN used BF16, so those ratios are indicative.
- Startup times were measured without locked GPU clocks; some configurations varied by more than 3% between launches.
Each results page lists its own limits. Next, we will extend the comparison to more models, datasets and hardware, using fully matched programs and repeatable measurements.