‹ Back to Tutorials

Automated Mixed Precision

OpenNN supports BF16 mixed-precision training on CUDA devices with compute capability 8.0 or newer. It keeps the master parameters and optimizer state in FP32 while using BF16 working data on the GPU.

This reduces device memory use and can accelerate tensor operations while retaining FP32 updates for numerical stability.

Contents:

  1. Introduction
  2. Mixed-precision strategy
  3. Enable BF16 in OpenNN
  4. Check hardware requirements
  5. Query the active configuration
  6. Conclusions

1. Introduction

FP32 is the standard numeric type for neural-network training. BF16 uses fewer bits for the significand but preserves the same exponent range, making it well suited to mixed-precision workloads on recent NVIDIA GPUs.

OpenNN selects the device and numeric type through the global Configuration object before the network is compiled.

2. Mixed-precision strategy

When BF16 is selected, OpenNN combines two representations:

  • An FP32 master copy for parameters, gradients and optimizer state.
  • BF16 working data on the GPU for supported forward and backward operations.

The FP32 master copy preserves update precision. The BF16 representation lowers memory traffic and enables the BF16 execution paths available on compatible hardware.

3. Enable BF16 in OpenNN

Set the global configuration before constructing or compiling the network:

#include "opennn/core/configuration.h"

using namespace opennn;

int main()
{
    Configuration::instance().set(
        Device::CUDA,
        Type::BF16);

    // Construct and train the model after configuring OpenNN.
}

Configuration::set currently accepts two arguments: the device and the numeric type. There is no separate training and inference type.

4. Check hardware requirements

BF16 requires a CUDA build and an NVIDIA GPU with compute capability 8.0 or newer. This includes Ampere and later architectures.

If CUDA is unavailable or the selected GPU is too old, an explicit BF16 configuration throws an exception instead of silently changing precision. Use Device::Auto and Type::Auto when automatic fallback to CPU FP32 is preferred.

5. Query the active configuration

resolve returns the effective values after automatic device and type selection:

const EffectiveConfig config =
    Configuration::instance().resolve();

if(config.device == Device::CUDA
   && config.training_type == Type::BF16)
{
    // CUDA BF16 is active.
}

After compilation, neural_network.is_gpu() reports whether that network is running on CUDA. Its get_training_type() method returns the compiled numeric type.

6. Conclusions

  • Use Configuration::instance().set(Device::CUDA, Type::BF16) before compiling the network.
  • BF16 requires CUDA compute capability 8.0 or newer.
  • OpenNN retains FP32 master data for stable parameter updates.
  • Use resolve to inspect the effective configuration.

References