opennn::TransformerDecoder::SamplingConfig Struct Reference

Sampling parameters that control how the next token is drawn from the model output distribution. More...

#include <transformer_decoder.h>

Public Attributes
float	temperature = 1.0f
Index	top_k = 0
float	top_p = 1.0f
float	repetition_penalty = 1.0f
Index	maximum_tokens = 0

Detailed Description

Sampling parameters that control how the next token is drawn from the model output distribution.

Member Data Documentation

maximum_tokens

Index opennn::TransformerDecoder::SamplingConfig::maximum_tokens = 0

repetition_penalty

float opennn::TransformerDecoder::SamplingConfig::repetition_penalty = 1.0f

temperature

float opennn::TransformerDecoder::SamplingConfig::temperature = 1.0f

top_k

Index opennn::TransformerDecoder::SamplingConfig::top_k = 0

top_p

float opennn::TransformerDecoder::SamplingConfig::top_p = 1.0f