What causes "train_loss < 0.5" in d2l-ai/d2l-zh?

Use the book's standard hyperparameters for softmax regression: lr=0.1 (or 0.03 for from-scratch SGD on Fashion-MNIST), batch_size=256, num_epochs=10.

d2l-ai/d2l-zh · error · AssertionError

train_loss < 0.5

Error message

train_loss < 0.5

What it means

An AssertionError raised at the end of d2l.tensorflow.train_ch3, the chapter-3 softmax-regression training loop. After num_epochs finish, it sanity-checks the final training metrics; train_loss must be below 0.5 for Fashion-MNIST softmax regression. If the loss is >= 0.5 the model diverged or undertrained, and the assert converts that into a loud failure instead of silently returning bad weights.

Source

Thrown at d2l/tensorflow.py:319

        self.axes[0].cla()
        for x, y, fmt in zip(self.X, self.Y, self.fmts):
            self.axes[0].plot(x, y, fmt)
        self.config_axes()
        display.display(self.fig)
        display.clear_output(wait=True)

def train_ch3(net, train_iter, test_iter, loss, num_epochs, updater):
    """训练模型（定义见第3章）

    Defined in :numref:`sec_softmax_scratch`"""
    animator = Animator(xlabel='epoch', xlim=[1, num_epochs], ylim=[0.3, 0.9],
                        legend=['train loss', 'train acc', 'test acc'])
    for epoch in range(num_epochs):
        train_metrics = train_epoch_ch3(net, train_iter, loss, updater)
        test_acc = evaluate_accuracy(net, test_iter)
        animator.add(epoch + 1, train_metrics + (test_acc,))
    train_loss, train_acc = train_metrics
    assert train_loss < 0.5, train_loss
    assert train_acc <= 1 and train_acc > 0.7, train_acc
    assert test_acc <= 1 and test_acc > 0.7, test_acc

class Updater():
    """用小批量随机梯度下降法更新参数

    Defined in :numref:`sec_softmax_scratch`"""
    def __init__(self, params, lr):
        self.params = params
        self.lr = lr

    def __call__(self, batch_size, grads):
        d2l.sgd(self.params, grads, self.lr, batch_size)

def predict_ch3(net, test_iter, n=6):
    """预测标签（定义见第3章）

    Defined in :numref:`sec_softmax_scratch`"""

View on GitHub (pinned to e6b18ccea7)

Solutions

Use the book's standard hyperparameters for softmax regression: lr=0.1 (or 0.03 for from-scratch SGD on Fashion-MNIST), batch_size=256, num_epochs=10.
If you wrote your own updater, make sure gradients are averaged per batch (divide by batch_size) as the d2l.Updater does.
Verify the loss uses reduction='mean' (d2l.loss: tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True, reduction='mean')).
Increase num_epochs or lower lr if the curve in the Animator is still descending or oscillating at the end.

Example fix

# before
updater = Updater([W, b], lr=10.0)  # diverges -> AssertionError
# after
updater = Updater([W, b], lr=0.1)
train_ch3(net, train_iter, test_iter, loss, 10, updater)

Defensive patterns

Strategy: validation

Validate before calling

loss_hist = []
for epoch in range(num_epochs):
    train_metrics = train_epoch_ch3(net, train_iter, loss, updater)
    loss_hist.append(train_metrics[0])
if loss_hist[-1] >= 0.5:
    raise ValueError(f'loss did not converge below 0.5: {loss_hist[-1]:.3f}; '
                     f'check lr/updater and raise num_epochs')

Try / catch

try:
    train_ch3(net, train_iter, test_iter, loss, num_epochs, updater)
except AssertionError as e:
    raise RuntimeError(f'train_ch3 sanity check failed (loss={e}); '
                       f'verify lr=0.1, batch_size=256, mean-reduced loss') from e

Prevention

Start from the book's hyperparameters (lr=0.1, batch_size=256, num_epochs=10) before experimenting.
If writing a custom updater, verify gradients are divided by batch_size.
Use d2l.loss with reduction='mean' so the 0.5 threshold is meaningful.
Watch the Animator each epoch; fix divergence early instead of waiting for the final assert.

When it happens

Trigger: Running train_ch3 with an lr that is too large (divergence) or too small/too few epochs (undertrained); passing a fresh net whose weights were never trained because updater was a no-op; calling train_ch3 with a different loss (e.g. unaveraged sum reduction) so the reported metric scale is wrong; running on a tiny subset of Fashion-MNIST.

Common situations: Users experimenting with learning rates (e.g. lr=10) or batch sizes in the sec_softmax_scratch notebook; replacing the Updater with one that forgets to divide by batch_size, inflating the loss; TF version changes altering default loss reduction; training a deeper/worse model through the same helper.

Related errors

AI-assisted analysis of d2l-ai/d2l-zh@e6b18ccea7 (2026-08-14). Data as JSON: /api/errors/e2af2e8509a5cc31. Report an issue: GitHub.