Optimizing with Torch - optimizer$step() throws up this error

Introduction to Optimizers in R using Torch

Torch, a popular deep learning library for R, provides an efficient way to build and train neural networks. However, when working with optimizers, one of the most common errors encountered by beginners is related to the optimizer$step() function.

In this article, we will delve into the details of why optimizer$step() throws up an error in Torch, and provide solutions to resolve this issue.

Understanding Optimizers in Torch

Before diving into the solution, it’s essential to understand how optimizers work in Torch. An optimizer is a crucial component in stochastic gradient descent (SGD) that adjusts model parameters to minimize loss. In Torch, optimizers are designed to iterate over the model’s parameters and update their values based on gradients calculated during backpropagation.

The most commonly used optimizers in Torch include optim_adam, optim_sgd, and optim_rmsprop. Among these, optim_adam is a popular choice due to its ability to adapt learning rates for each parameter independently.

Error Analysis: optimizer$step() Throws an Error

When running the provided code, which uses optim_adam as the optimizer, we encounter an error message that reads:

“Error in param$state[[“exp_avg”]] : object of type ‘closure’ is not subsettable”

This error occurs because the optimizer$step() function attempts to access and update the internal state of the optimizer. Specifically, it tries to modify the exponential moving average (EMA) parameter exp_avg, which is a critical component in Adam optimizer.

The issue arises from the fact that exp_avg is not accessible as an array element using $state[["exp_avg"]]. Instead, we need to use the optimizer$state() function to access the internal state of the optimizer and then modify the EMA parameter.

Resolving the Error

To resolve this error, you can modify the code to use the correct syntax for accessing the optimizer’s state. Here’s an updated version of the training loop:

### Training Loop --------------------------------------------------------------

for (t in 1:200) {
  
 
  y_pred <- model(x)
  
  loss <- nnf_mse_loss(y_pred, y, reduction = "sum")
  
  # Accessing optimizer's state
  optim_state <- optimizer$state()
  
  # Updating the learning rate for each parameter
  optimizer$zero_grad()
  loss$backward()
  optimizer$step(optim_state)
}

In this updated version, we access the optimizer’s internal state using $state(), and then pass this object to the optimizer$step() function.

Alternative Solution: Using the optimizer Object Directly

Alternatively, you can use the optimizer object directly without accessing its internal state. To do so, simply update the learning rate for each parameter without explicitly calling $zero_grad() or passing the optimizer’s state:

### Training Loop --------------------------------------------------------------

for (t in 1:200) {
  
 
  y_pred <- model(x)
  
  loss <- nnf_mse_loss(y_pred, y, reduction = "sum")
  
  # Updating the learning rate for each parameter
  optimizer$step()
}

This approach is more concise and avoids explicit access to the optimizer’s internal state.

Additional Considerations

When working with optimizers in Torch, keep the following best practices in mind:

• Always initialize your model parameters correctly before running the optimization loop.
• Use the optimizer$zero_grad() function to zero-gradients for each parameter before updating them.
• Update the learning rate for each parameter using $step() or a custom learning rate schedule.

By understanding how optimizers work in Torch and implementing the correct syntax, you can efficiently train your neural networks using optim_adam and other optimization algorithms.

Last modified on 2025-04-28