{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Digit Recognition using Tensor Train (TT) Neural Networks\n", "\n", "This notebook demonstrates how to use **Tensor Train (TT) decomposition** in neural network layers for image classification on the MNIST dataset.\n", "\n", "## Why TT Layers?\n", "\n", "Traditional fully-connected layers have weight matrices of size `input_dim × output_dim`, which can become prohibitively large. TT layers decompose these weight matrices into a sequence of smaller 3D tensors (cores), dramatically reducing the number of parameters while maintaining expressiveness. This leads to:\n", "\n", "- **Fewer parameters** → reduced memory footprint and faster inference\n", "- **Built-in regularization** → less overfitting on small datasets\n", "- **Structured compression** → the TT-rank controls the model capacity\n", "\n", "## Imports" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import torch as tn\n", "import torch.nn as nn\n", "try: \n", " import torchtt as tntt\n", "except:\n", " print('Installing torchTT...')\n", " %pip install git+https://github.com/ion-g-ion/torchTT\n", " import torchtt as tntt\n", "from torch import optim\n", "from torchvision import datasets\n", "from torchvision.transforms import ToTensor\n", "from torch.utils.data import DataLoader\n", "\n", "device = tn.device('cuda' if tn.cuda.is_available() else 'cpu')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Load the MNIST Dataset\n", "\n", "The MNIST dataset contains 70,000 grayscale images of handwritten digits (0-9), each of size 28×28 pixels:\n", "- **Training set**: 60,000 images\n", "- **Test set**: 10,000 images\n", "\n", "We download the dataset and store it in the `downloads` folder. The `ToTensor()` transform converts PIL images to PyTorch tensors and normalizes pixel values to [0, 1]." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100.0%\n", "100.0%\n", "100.0%\n", "100.0%\n" ] } ], "source": [ "train_data = datasets.MNIST(root = 'downloads', train = True, transform = ToTensor(), download = True)\n", "test_data = datasets.MNIST(root = 'downloads', train = False, transform = ToTensor())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create DataLoaders\n", "\n", "DataLoaders handle batching, shuffling, and parallel data loading:\n", "- **Training loader**: batch size of 1000, shuffled each epoch for stochastic gradient descent\n", "- **Test loader**: batch size of 100 for evaluation\n", "\n", "Using `num_workers=10` enables parallel data loading across multiple CPU processes." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "dataloader_train = tn.utils.data.DataLoader(train_data, batch_size=1000, shuffle=True, num_workers=10)\n", "dataloader_test = tn.utils.data.DataLoader(test_data, batch_size=100, shuffle=True, num_workers=10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Define the TT Neural Network Architecture\n", "\n", "The network uses two **TT-Linear layers** followed by a standard linear output layer:\n", "\n", "### Layer 1: `LinearLayerTT`\n", "- **Input shape**: `[1, 7, 4, 7, 4]` — the 28×28 image reshaped as a 5D tensor (1×7×4×7×4 = 784)\n", "- **Output shape**: `[8, 10, 10, 10, 10]` — produces 80,000 features in TT format\n", "- **TT-ranks**: `[1, 4, 2, 2, 2, 1]` — controls the compression; lower ranks = fewer parameters\n", "\n", "### Layer 2: `LinearLayerTT`\n", "- **Input shape**: `[8, 10, 10, 10, 10]`\n", "- **Output shape**: `[8, 3, 3, 3, 3]` — produces 648 features (8×81)\n", "- **TT-ranks**: `[1, 2, 2, 2, 2, 1]`\n", "\n", "### Output Layer\n", "- Standard linear layer: 648 → 10 (one output per digit class)\n", "- **LogSoftmax** activation for numerical stability with `CrossEntropyLoss`\n", "\n", "ReLU activations are applied between the TT layers for non-linearity." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "\n", "class BasicTT(nn.Module):\n", " def __init__(self):\n", " super().__init__()\n", " self.ttl1 = tntt.nn.LinearLayerTT([1,7,4,7,4], [8,10,10,10,10], [1,4,2,2,2,1])\n", " self.ttl2 = tntt.nn.LinearLayerTT([8,10,10,10,10], [8,3,3,3,3], [1,2,2,2,2,1])\n", " self.linear = nn.Linear(81*8, 10, dtype = tn.float32)\n", " self.logsoftmax = nn.LogSoftmax(1)\n", "\n", " def forward(self, x):\n", " x = self.ttl1(x)\n", " x = tn.relu(x)\n", " x = self.ttl2(x)\n", " x = tn.relu(x)\n", " x = x.view(-1,81*8)\n", " x = self.linear(x)\n", " return self.logsoftmax(x)\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Instantiate Model, Loss, and Optimizer\n", "\n", "- **Model**: Moved to GPU if available (CUDA), otherwise runs on CPU\n", "- **Loss function**: `CrossEntropyLoss` combines LogSoftmax and NLLLoss — standard for multi-class classification\n", "- **Optimizer**: Adam with learning rate 0.001 — adaptive learning rate optimizer that works well with default hyperparameters" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "model = BasicTT().to(device)\n", "loss_function = nn.CrossEntropyLoss() \n", "optimizer = optim.Adam(model.parameters(), lr = 0.001) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Training Loop\n", "\n", "Train the model for 30 epochs. Each epoch:\n", "\n", "1. **Reshape input**: Convert the flat 28×28 images to shape `[-1, 1, 7, 4, 7, 4]` to match the TT layer's expected input format (batch dimension + 5D tensor structure)\n", "2. **Forward pass**: Compute predictions through the TT network\n", "3. **Backward pass**: Compute gradients via backpropagation\n", "4. **Update weights**: Apply Adam optimizer step\n", "\n", "Progress is printed after each batch showing epoch, iteration, and current loss value." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/tmp/ipykernel_71331/1816520187.py:15: UserWarning: Converting a tensor with requires_grad=True to a scalar may lead to unexpected behavior.\n", "Consider using tensor.detach() first. (Triggered internally at /pytorch/torch/csrc/autograd/generated/python_variable_methods.cpp:836.)\n", " print('Epoch %d/%d iteration %d/%d loss %e'%(epoch+1,n_epochs,i+1,len(dataloader_train),loss))\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1/30 iteration 1/60 loss 2.319157e+00\n", "Epoch 1/30 iteration 2/60 loss 2.281743e+00\n", "Epoch 1/30 iteration 3/60 loss 2.253771e+00\n", "Epoch 1/30 iteration 4/60 loss 2.208041e+00\n", "Epoch 1/30 iteration 5/60 loss 2.176179e+00\n", "Epoch 1/30 iteration 6/60 loss 2.147874e+00\n", "Epoch 1/30 iteration 7/60 loss 2.106683e+00\n", "Epoch 1/30 iteration 8/60 loss 2.068826e+00\n", "Epoch 1/30 iteration 9/60 loss 2.044304e+00\n", "Epoch 1/30 iteration 10/60 loss 1.979471e+00\n", "Epoch 1/30 iteration 11/60 loss 1.949439e+00\n", "Epoch 1/30 iteration 12/60 loss 1.920550e+00\n", "Epoch 1/30 iteration 13/60 loss 1.881005e+00\n", "Epoch 1/30 iteration 14/60 loss 1.838561e+00\n", "Epoch 1/30 iteration 15/60 loss 1.763517e+00\n", "Epoch 1/30 iteration 16/60 loss 1.717423e+00\n", "Epoch 1/30 iteration 17/60 loss 1.664475e+00\n", "Epoch 1/30 iteration 18/60 loss 1.613474e+00\n", "Epoch 1/30 iteration 19/60 loss 1.567955e+00\n", "Epoch 1/30 iteration 20/60 loss 1.527148e+00\n", "Epoch 1/30 iteration 21/60 loss 1.480951e+00\n", "Epoch 1/30 iteration 22/60 loss 1.393301e+00\n", "Epoch 1/30 iteration 23/60 loss 1.340357e+00\n", "Epoch 1/30 iteration 24/60 loss 1.257782e+00\n", "Epoch 1/30 iteration 25/60 loss 1.278873e+00\n", "Epoch 1/30 iteration 26/60 loss 1.199034e+00\n", "Epoch 1/30 iteration 27/60 loss 1.172296e+00\n", "Epoch 1/30 iteration 28/60 loss 1.085633e+00\n", "Epoch 1/30 iteration 29/60 loss 1.042049e+00\n", "Epoch 1/30 iteration 30/60 loss 1.032022e+00\n", "Epoch 1/30 iteration 31/60 loss 9.578884e-01\n", "Epoch 1/30 iteration 32/60 loss 9.486971e-01\n", "Epoch 1/30 iteration 33/60 loss 8.623374e-01\n", "Epoch 1/30 iteration 34/60 loss 8.256072e-01\n", "Epoch 1/30 iteration 35/60 loss 8.681351e-01\n", "Epoch 1/30 iteration 36/60 loss 7.895614e-01\n", "Epoch 1/30 iteration 37/60 loss 7.842469e-01\n", "Epoch 1/30 iteration 38/60 loss 7.534677e-01\n", "Epoch 1/30 iteration 39/60 loss 7.555861e-01\n", "Epoch 1/30 iteration 40/60 loss 7.008192e-01\n", "Epoch 1/30 iteration 41/60 loss 6.774005e-01\n", "Epoch 1/30 iteration 42/60 loss 6.680732e-01\n", "Epoch 1/30 iteration 43/60 loss 6.680810e-01\n", "Epoch 1/30 iteration 44/60 loss 5.700449e-01\n", "Epoch 1/30 iteration 45/60 loss 5.861765e-01\n", "Epoch 1/30 iteration 46/60 loss 6.326642e-01\n", "Epoch 1/30 iteration 47/60 loss 5.691887e-01\n", "Epoch 1/30 iteration 48/60 loss 5.817655e-01\n", "Epoch 1/30 iteration 49/60 loss 5.343875e-01\n", "Epoch 1/30 iteration 50/60 loss 5.776040e-01\n", "Epoch 1/30 iteration 51/60 loss 5.052800e-01\n", "Epoch 1/30 iteration 52/60 loss 5.063986e-01\n", "Epoch 1/30 iteration 53/60 loss 5.603164e-01\n", "Epoch 1/30 iteration 54/60 loss 5.035990e-01\n", "Epoch 1/30 iteration 55/60 loss 4.753400e-01\n", "Epoch 1/30 iteration 56/60 loss 4.744058e-01\n", "Epoch 1/30 iteration 57/60 loss 5.110996e-01\n", "Epoch 1/30 iteration 58/60 loss 5.367128e-01\n", "Epoch 1/30 iteration 59/60 loss 4.989136e-01\n", "Epoch 1/30 iteration 60/60 loss 5.151227e-01\n", "Epoch 2/30 iteration 1/60 loss 4.273748e-01\n", "Epoch 2/30 iteration 2/60 loss 4.963650e-01\n", "Epoch 2/30 iteration 3/60 loss 4.267454e-01\n", "Epoch 2/30 iteration 4/60 loss 4.646486e-01\n", "Epoch 2/30 iteration 5/60 loss 4.828642e-01\n", "Epoch 2/30 iteration 6/60 loss 4.411363e-01\n", "Epoch 2/30 iteration 7/60 loss 4.055088e-01\n", "Epoch 2/30 iteration 8/60 loss 3.841212e-01\n", "Epoch 2/30 iteration 9/60 loss 4.129797e-01\n", "Epoch 2/30 iteration 10/60 loss 4.507828e-01\n", "Epoch 2/30 iteration 11/60 loss 3.781027e-01\n", "Epoch 2/30 iteration 12/60 loss 4.158538e-01\n", "Epoch 2/30 iteration 13/60 loss 3.490373e-01\n", "Epoch 2/30 iteration 14/60 loss 3.628863e-01\n", "Epoch 2/30 iteration 15/60 loss 3.713616e-01\n", "Epoch 2/30 iteration 16/60 loss 3.789753e-01\n", "Epoch 2/30 iteration 17/60 loss 3.994610e-01\n", "Epoch 2/30 iteration 18/60 loss 3.801353e-01\n", "Epoch 2/30 iteration 19/60 loss 3.760593e-01\n", "Epoch 2/30 iteration 20/60 loss 3.384998e-01\n", "Epoch 2/30 iteration 21/60 loss 3.310554e-01\n", "Epoch 2/30 iteration 22/60 loss 3.413204e-01\n", "Epoch 2/30 iteration 23/60 loss 4.154034e-01\n", "Epoch 2/30 iteration 24/60 loss 3.838296e-01\n", "Epoch 2/30 iteration 25/60 loss 3.096202e-01\n", "Epoch 2/30 iteration 26/60 loss 3.491054e-01\n", "Epoch 2/30 iteration 27/60 loss 3.383074e-01\n", "Epoch 2/30 iteration 28/60 loss 3.339524e-01\n", "Epoch 2/30 iteration 29/60 loss 3.488864e-01\n", "Epoch 2/30 iteration 30/60 loss 3.163486e-01\n", "Epoch 2/30 iteration 31/60 loss 3.754135e-01\n", "Epoch 2/30 iteration 32/60 loss 3.091184e-01\n", "Epoch 2/30 iteration 33/60 loss 3.713291e-01\n", "Epoch 2/30 iteration 34/60 loss 3.024457e-01\n", "Epoch 2/30 iteration 35/60 loss 3.166254e-01\n", "Epoch 2/30 iteration 36/60 loss 3.540526e-01\n", "Epoch 2/30 iteration 37/60 loss 3.298119e-01\n", "Epoch 2/30 iteration 38/60 loss 3.246371e-01\n", "Epoch 2/30 iteration 39/60 loss 3.039279e-01\n", "Epoch 2/30 iteration 40/60 loss 3.002834e-01\n", "Epoch 2/30 iteration 41/60 loss 3.385607e-01\n", "Epoch 2/30 iteration 42/60 loss 3.077032e-01\n", "Epoch 2/30 iteration 43/60 loss 2.956550e-01\n", "Epoch 2/30 iteration 44/60 loss 2.820969e-01\n", "Epoch 2/30 iteration 45/60 loss 3.175825e-01\n", "Epoch 2/30 iteration 46/60 loss 3.030532e-01\n", "Epoch 2/30 iteration 47/60 loss 3.103965e-01\n", "Epoch 2/30 iteration 48/60 loss 3.115616e-01\n", "Epoch 2/30 iteration 49/60 loss 2.507900e-01\n", "Epoch 2/30 iteration 50/60 loss 2.761504e-01\n", "Epoch 2/30 iteration 51/60 loss 2.737729e-01\n", "Epoch 2/30 iteration 52/60 loss 2.509707e-01\n", "Epoch 2/30 iteration 53/60 loss 2.948253e-01\n", "Epoch 2/30 iteration 54/60 loss 3.163671e-01\n", "Epoch 2/30 iteration 55/60 loss 3.121327e-01\n", "Epoch 2/30 iteration 56/60 loss 2.635366e-01\n", "Epoch 2/30 iteration 57/60 loss 2.885589e-01\n", "Epoch 2/30 iteration 58/60 loss 2.569787e-01\n", "Epoch 2/30 iteration 59/60 loss 2.634967e-01\n", "Epoch 2/30 iteration 60/60 loss 3.128447e-01\n", "Epoch 3/30 iteration 1/60 loss 2.327740e-01\n", "Epoch 3/30 iteration 2/60 loss 2.707975e-01\n", "Epoch 3/30 iteration 3/60 loss 2.715830e-01\n", "Epoch 3/30 iteration 4/60 loss 2.663421e-01\n", "Epoch 3/30 iteration 5/60 loss 2.630066e-01\n", "Epoch 3/30 iteration 6/60 loss 2.569858e-01\n", "Epoch 3/30 iteration 7/60 loss 2.019530e-01\n", "Epoch 3/30 iteration 8/60 loss 2.568755e-01\n", "Epoch 3/30 iteration 9/60 loss 2.567869e-01\n", "Epoch 3/30 iteration 10/60 loss 2.022004e-01\n", "Epoch 3/30 iteration 11/60 loss 2.546880e-01\n", "Epoch 3/30 iteration 12/60 loss 2.775907e-01\n", "Epoch 3/30 iteration 13/60 loss 2.787156e-01\n", "Epoch 3/30 iteration 14/60 loss 2.649332e-01\n", "Epoch 3/30 iteration 15/60 loss 2.376900e-01\n", "Epoch 3/30 iteration 16/60 loss 2.573857e-01\n", "Epoch 3/30 iteration 17/60 loss 2.853866e-01\n", "Epoch 3/30 iteration 18/60 loss 2.780775e-01\n", "Epoch 3/30 iteration 19/60 loss 2.660152e-01\n", "Epoch 3/30 iteration 20/60 loss 2.836706e-01\n", "Epoch 3/30 iteration 21/60 loss 2.413796e-01\n", "Epoch 3/30 iteration 22/60 loss 1.937527e-01\n", "Epoch 3/30 iteration 23/60 loss 2.669611e-01\n", "Epoch 3/30 iteration 24/60 loss 2.534285e-01\n", "Epoch 3/30 iteration 25/60 loss 2.319166e-01\n", "Epoch 3/30 iteration 26/60 loss 2.319260e-01\n", "Epoch 3/30 iteration 27/60 loss 2.550859e-01\n", "Epoch 3/30 iteration 28/60 loss 2.063870e-01\n", "Epoch 3/30 iteration 29/60 loss 2.312407e-01\n", "Epoch 3/30 iteration 30/60 loss 1.962183e-01\n", "Epoch 3/30 iteration 31/60 loss 1.915111e-01\n", "Epoch 3/30 iteration 32/60 loss 2.589238e-01\n", "Epoch 3/30 iteration 33/60 loss 2.233467e-01\n", "Epoch 3/30 iteration 34/60 loss 2.693852e-01\n", "Epoch 3/30 iteration 35/60 loss 2.239518e-01\n", "Epoch 3/30 iteration 36/60 loss 2.201767e-01\n", "Epoch 3/30 iteration 37/60 loss 2.432474e-01\n", "Epoch 3/30 iteration 38/60 loss 2.095928e-01\n", "Epoch 3/30 iteration 39/60 loss 2.357772e-01\n", "Epoch 3/30 iteration 40/60 loss 2.334893e-01\n", "Epoch 3/30 iteration 41/60 loss 1.881966e-01\n", "Epoch 3/30 iteration 42/60 loss 2.294446e-01\n", "Epoch 3/30 iteration 43/60 loss 2.093347e-01\n", "Epoch 3/30 iteration 44/60 loss 2.383568e-01\n", "Epoch 3/30 iteration 45/60 loss 1.973541e-01\n", "Epoch 3/30 iteration 46/60 loss 2.225555e-01\n", "Epoch 3/30 iteration 47/60 loss 2.131327e-01\n", "Epoch 3/30 iteration 48/60 loss 2.306913e-01\n", "Epoch 3/30 iteration 49/60 loss 2.109767e-01\n", "Epoch 3/30 iteration 50/60 loss 2.495784e-01\n", "Epoch 3/30 iteration 51/60 loss 2.591890e-01\n", "Epoch 3/30 iteration 52/60 loss 2.202999e-01\n", "Epoch 3/30 iteration 53/60 loss 2.162992e-01\n", "Epoch 3/30 iteration 54/60 loss 2.068974e-01\n", "Epoch 3/30 iteration 55/60 loss 2.994433e-01\n", "Epoch 3/30 iteration 56/60 loss 2.188354e-01\n", "Epoch 3/30 iteration 57/60 loss 2.219166e-01\n", "Epoch 3/30 iteration 58/60 loss 2.092956e-01\n", "Epoch 3/30 iteration 59/60 loss 2.403654e-01\n", "Epoch 3/30 iteration 60/60 loss 2.155082e-01\n", "Epoch 4/30 iteration 1/60 loss 2.421814e-01\n", "Epoch 4/30 iteration 2/60 loss 1.839793e-01\n", "Epoch 4/30 iteration 3/60 loss 1.726423e-01\n", "Epoch 4/30 iteration 4/60 loss 1.866310e-01\n", "Epoch 4/30 iteration 5/60 loss 1.457693e-01\n", "Epoch 4/30 iteration 6/60 loss 1.426776e-01\n", "Epoch 4/30 iteration 7/60 loss 1.978460e-01\n", "Epoch 4/30 iteration 8/60 loss 2.342976e-01\n", "Epoch 4/30 iteration 9/60 loss 1.679645e-01\n", "Epoch 4/30 iteration 10/60 loss 1.816709e-01\n", "Epoch 4/30 iteration 11/60 loss 1.879822e-01\n", "Epoch 4/30 iteration 12/60 loss 1.969439e-01\n", "Epoch 4/30 iteration 13/60 loss 1.654304e-01\n", "Epoch 4/30 iteration 14/60 loss 1.619370e-01\n", "Epoch 4/30 iteration 15/60 loss 1.785001e-01\n", "Epoch 4/30 iteration 16/60 loss 2.163541e-01\n", "Epoch 4/30 iteration 17/60 loss 2.092406e-01\n", "Epoch 4/30 iteration 18/60 loss 1.483836e-01\n", "Epoch 4/30 iteration 19/60 loss 1.690139e-01\n", "Epoch 4/30 iteration 20/60 loss 2.126202e-01\n", "Epoch 4/30 iteration 21/60 loss 2.258978e-01\n", "Epoch 4/30 iteration 22/60 loss 2.205673e-01\n", "Epoch 4/30 iteration 23/60 loss 2.032424e-01\n", "Epoch 4/30 iteration 24/60 loss 1.981255e-01\n", "Epoch 4/30 iteration 25/60 loss 1.835148e-01\n", "Epoch 4/30 iteration 26/60 loss 1.803906e-01\n", "Epoch 4/30 iteration 27/60 loss 1.717636e-01\n", "Epoch 4/30 iteration 28/60 loss 2.142166e-01\n", "Epoch 4/30 iteration 29/60 loss 2.002263e-01\n", "Epoch 4/30 iteration 30/60 loss 2.052232e-01\n", "Epoch 4/30 iteration 31/60 loss 1.869922e-01\n", "Epoch 4/30 iteration 32/60 loss 2.412877e-01\n", "Epoch 4/30 iteration 33/60 loss 1.805727e-01\n", "Epoch 4/30 iteration 34/60 loss 1.851210e-01\n", "Epoch 4/30 iteration 35/60 loss 1.340946e-01\n", "Epoch 4/30 iteration 36/60 loss 1.730659e-01\n", "Epoch 4/30 iteration 37/60 loss 2.155998e-01\n", "Epoch 4/30 iteration 38/60 loss 1.950367e-01\n", "Epoch 4/30 iteration 39/60 loss 1.834176e-01\n", "Epoch 4/30 iteration 40/60 loss 1.810114e-01\n", "Epoch 4/30 iteration 41/60 loss 2.114315e-01\n", "Epoch 4/30 iteration 42/60 loss 1.955607e-01\n", "Epoch 4/30 iteration 43/60 loss 1.474129e-01\n", "Epoch 4/30 iteration 44/60 loss 1.648508e-01\n", "Epoch 4/30 iteration 45/60 loss 1.632307e-01\n", "Epoch 4/30 iteration 46/60 loss 1.960361e-01\n", "Epoch 4/30 iteration 47/60 loss 1.931790e-01\n", "Epoch 4/30 iteration 48/60 loss 2.051028e-01\n", "Epoch 4/30 iteration 49/60 loss 1.518929e-01\n", "Epoch 4/30 iteration 50/60 loss 1.666449e-01\n", "Epoch 4/30 iteration 51/60 loss 1.848719e-01\n", "Epoch 4/30 iteration 52/60 loss 2.166413e-01\n", "Epoch 4/30 iteration 53/60 loss 1.852185e-01\n", "Epoch 4/30 iteration 54/60 loss 1.946176e-01\n", "Epoch 4/30 iteration 55/60 loss 1.823148e-01\n", "Epoch 4/30 iteration 56/60 loss 1.931695e-01\n", "Epoch 4/30 iteration 57/60 loss 1.557338e-01\n", "Epoch 4/30 iteration 58/60 loss 2.037965e-01\n", "Epoch 4/30 iteration 59/60 loss 1.535442e-01\n", "Epoch 4/30 iteration 60/60 loss 1.622197e-01\n", "Epoch 5/30 iteration 1/60 loss 1.862722e-01\n", "Epoch 5/30 iteration 2/60 loss 1.411206e-01\n", "Epoch 5/30 iteration 3/60 loss 1.767921e-01\n", "Epoch 5/30 iteration 4/60 loss 1.621427e-01\n", "Epoch 5/30 iteration 5/60 loss 1.567468e-01\n", "Epoch 5/30 iteration 6/60 loss 2.073318e-01\n", "Epoch 5/30 iteration 7/60 loss 1.720709e-01\n", "Epoch 5/30 iteration 8/60 loss 1.373070e-01\n", "Epoch 5/30 iteration 9/60 loss 1.738432e-01\n", "Epoch 5/30 iteration 10/60 loss 1.362188e-01\n", "Epoch 5/30 iteration 11/60 loss 1.488026e-01\n", "Epoch 5/30 iteration 12/60 loss 1.502767e-01\n", "Epoch 5/30 iteration 13/60 loss 1.544317e-01\n", "Epoch 5/30 iteration 14/60 loss 1.682075e-01\n", "Epoch 5/30 iteration 15/60 loss 1.703126e-01\n", "Epoch 5/30 iteration 16/60 loss 1.925524e-01\n", "Epoch 5/30 iteration 17/60 loss 1.643154e-01\n", "Epoch 5/30 iteration 18/60 loss 1.226633e-01\n", "Epoch 5/30 iteration 19/60 loss 1.704757e-01\n", "Epoch 5/30 iteration 20/60 loss 1.134558e-01\n", "Epoch 5/30 iteration 21/60 loss 1.780013e-01\n", "Epoch 5/30 iteration 22/60 loss 1.558242e-01\n", "Epoch 5/30 iteration 23/60 loss 1.406226e-01\n", "Epoch 5/30 iteration 24/60 loss 1.701739e-01\n", "Epoch 5/30 iteration 25/60 loss 1.477030e-01\n", "Epoch 5/30 iteration 26/60 loss 1.696058e-01\n", "Epoch 5/30 iteration 27/60 loss 1.613795e-01\n", "Epoch 5/30 iteration 28/60 loss 1.194462e-01\n", "Epoch 5/30 iteration 29/60 loss 1.573155e-01\n", "Epoch 5/30 iteration 30/60 loss 1.462967e-01\n", "Epoch 5/30 iteration 31/60 loss 1.302588e-01\n", "Epoch 5/30 iteration 32/60 loss 1.415167e-01\n", "Epoch 5/30 iteration 33/60 loss 1.649251e-01\n", "Epoch 5/30 iteration 34/60 loss 1.735958e-01\n", "Epoch 5/30 iteration 35/60 loss 1.647125e-01\n", "Epoch 5/30 iteration 36/60 loss 1.237000e-01\n", "Epoch 5/30 iteration 37/60 loss 1.632262e-01\n", "Epoch 5/30 iteration 38/60 loss 1.471662e-01\n", "Epoch 5/30 iteration 39/60 loss 1.677307e-01\n", "Epoch 5/30 iteration 40/60 loss 1.685967e-01\n", "Epoch 5/30 iteration 41/60 loss 1.559155e-01\n", "Epoch 5/30 iteration 42/60 loss 1.439855e-01\n", "Epoch 5/30 iteration 43/60 loss 1.644606e-01\n", "Epoch 5/30 iteration 44/60 loss 1.661603e-01\n", "Epoch 5/30 iteration 45/60 loss 1.353667e-01\n", "Epoch 5/30 iteration 46/60 loss 1.881727e-01\n", "Epoch 5/30 iteration 47/60 loss 1.538214e-01\n", "Epoch 5/30 iteration 48/60 loss 1.616414e-01\n", "Epoch 5/30 iteration 49/60 loss 1.658366e-01\n", "Epoch 5/30 iteration 50/60 loss 1.631626e-01\n", "Epoch 5/30 iteration 51/60 loss 1.927939e-01\n", "Epoch 5/30 iteration 52/60 loss 1.578344e-01\n", "Epoch 5/30 iteration 53/60 loss 1.212931e-01\n", "Epoch 5/30 iteration 54/60 loss 1.629727e-01\n", "Epoch 5/30 iteration 55/60 loss 1.381435e-01\n", "Epoch 5/30 iteration 56/60 loss 1.351427e-01\n", "Epoch 5/30 iteration 57/60 loss 1.335993e-01\n", "Epoch 5/30 iteration 58/60 loss 1.174243e-01\n", "Epoch 5/30 iteration 59/60 loss 1.451000e-01\n", "Epoch 5/30 iteration 60/60 loss 1.189620e-01\n", "Epoch 6/30 iteration 1/60 loss 1.775871e-01\n", "Epoch 6/30 iteration 2/60 loss 1.434601e-01\n", "Epoch 6/30 iteration 3/60 loss 1.376940e-01\n", "Epoch 6/30 iteration 4/60 loss 1.324910e-01\n", "Epoch 6/30 iteration 5/60 loss 1.304276e-01\n", "Epoch 6/30 iteration 6/60 loss 1.172435e-01\n", "Epoch 6/30 iteration 7/60 loss 1.251949e-01\n", "Epoch 6/30 iteration 8/60 loss 1.415425e-01\n", "Epoch 6/30 iteration 9/60 loss 1.771352e-01\n", "Epoch 6/30 iteration 10/60 loss 1.554060e-01\n", "Epoch 6/30 iteration 11/60 loss 1.129687e-01\n", "Epoch 6/30 iteration 12/60 loss 1.337375e-01\n", "Epoch 6/30 iteration 13/60 loss 1.489016e-01\n", "Epoch 6/30 iteration 14/60 loss 1.538876e-01\n", "Epoch 6/30 iteration 15/60 loss 1.228016e-01\n", "Epoch 6/30 iteration 16/60 loss 1.444772e-01\n", "Epoch 6/30 iteration 17/60 loss 1.041410e-01\n", "Epoch 6/30 iteration 18/60 loss 1.288681e-01\n", "Epoch 6/30 iteration 19/60 loss 1.482636e-01\n", "Epoch 6/30 iteration 20/60 loss 1.203570e-01\n", "Epoch 6/30 iteration 21/60 loss 1.422926e-01\n", "Epoch 6/30 iteration 22/60 loss 1.103679e-01\n", "Epoch 6/30 iteration 23/60 loss 1.460140e-01\n", "Epoch 6/30 iteration 24/60 loss 1.433235e-01\n", "Epoch 6/30 iteration 25/60 loss 1.461476e-01\n", "Epoch 6/30 iteration 26/60 loss 1.782927e-01\n", "Epoch 6/30 iteration 27/60 loss 1.295701e-01\n", "Epoch 6/30 iteration 28/60 loss 1.391274e-01\n", "Epoch 6/30 iteration 29/60 loss 1.558044e-01\n", "Epoch 6/30 iteration 30/60 loss 1.073089e-01\n", "Epoch 6/30 iteration 31/60 loss 1.579653e-01\n", "Epoch 6/30 iteration 32/60 loss 1.350553e-01\n", "Epoch 6/30 iteration 33/60 loss 1.382285e-01\n", "Epoch 6/30 iteration 34/60 loss 9.566523e-02\n", "Epoch 6/30 iteration 35/60 loss 1.079037e-01\n", "Epoch 6/30 iteration 36/60 loss 1.296957e-01\n", "Epoch 6/30 iteration 37/60 loss 1.482933e-01\n", "Epoch 6/30 iteration 38/60 loss 1.364077e-01\n", "Epoch 6/30 iteration 39/60 loss 1.185526e-01\n", "Epoch 6/30 iteration 40/60 loss 1.211694e-01\n", "Epoch 6/30 iteration 41/60 loss 1.233255e-01\n", "Epoch 6/30 iteration 42/60 loss 1.106965e-01\n", "Epoch 6/30 iteration 43/60 loss 1.175879e-01\n", "Epoch 6/30 iteration 44/60 loss 1.370842e-01\n", "Epoch 6/30 iteration 45/60 loss 1.429606e-01\n", "Epoch 6/30 iteration 46/60 loss 1.331272e-01\n", "Epoch 6/30 iteration 47/60 loss 1.431700e-01\n", "Epoch 6/30 iteration 48/60 loss 1.601813e-01\n", "Epoch 6/30 iteration 49/60 loss 1.246030e-01\n", "Epoch 6/30 iteration 50/60 loss 1.215791e-01\n", "Epoch 6/30 iteration 51/60 loss 1.569257e-01\n", "Epoch 6/30 iteration 52/60 loss 1.233814e-01\n", "Epoch 6/30 iteration 53/60 loss 1.432266e-01\n", "Epoch 6/30 iteration 54/60 loss 1.250832e-01\n", "Epoch 6/30 iteration 55/60 loss 1.187075e-01\n", "Epoch 6/30 iteration 56/60 loss 1.381989e-01\n", "Epoch 6/30 iteration 57/60 loss 1.382184e-01\n", "Epoch 6/30 iteration 58/60 loss 8.949690e-02\n", "Epoch 6/30 iteration 59/60 loss 1.308110e-01\n", "Epoch 6/30 iteration 60/60 loss 1.213760e-01\n", "Epoch 7/30 iteration 1/60 loss 1.148577e-01\n", "Epoch 7/30 iteration 2/60 loss 1.437335e-01\n", "Epoch 7/30 iteration 3/60 loss 1.202633e-01\n", "Epoch 7/30 iteration 4/60 loss 1.486842e-01\n", "Epoch 7/30 iteration 5/60 loss 1.255193e-01\n", "Epoch 7/30 iteration 6/60 loss 1.217235e-01\n", "Epoch 7/30 iteration 7/60 loss 1.203821e-01\n", "Epoch 7/30 iteration 8/60 loss 1.329691e-01\n", "Epoch 7/30 iteration 9/60 loss 1.248241e-01\n", "Epoch 7/30 iteration 10/60 loss 1.190786e-01\n", "Epoch 7/30 iteration 11/60 loss 1.215561e-01\n", "Epoch 7/30 iteration 12/60 loss 1.391042e-01\n", "Epoch 7/30 iteration 13/60 loss 1.102612e-01\n", "Epoch 7/30 iteration 14/60 loss 1.224858e-01\n", "Epoch 7/30 iteration 15/60 loss 1.236300e-01\n", "Epoch 7/30 iteration 16/60 loss 1.322320e-01\n", "Epoch 7/30 iteration 17/60 loss 1.092215e-01\n", "Epoch 7/30 iteration 18/60 loss 1.387263e-01\n", "Epoch 7/30 iteration 19/60 loss 1.270421e-01\n", "Epoch 7/30 iteration 20/60 loss 8.255436e-02\n", "Epoch 7/30 iteration 21/60 loss 1.142894e-01\n", "Epoch 7/30 iteration 22/60 loss 1.153217e-01\n", "Epoch 7/30 iteration 23/60 loss 9.280580e-02\n", "Epoch 7/30 iteration 24/60 loss 1.178486e-01\n", "Epoch 7/30 iteration 25/60 loss 1.338323e-01\n", "Epoch 7/30 iteration 26/60 loss 8.668625e-02\n", "Epoch 7/30 iteration 27/60 loss 7.695022e-02\n", "Epoch 7/30 iteration 28/60 loss 1.336846e-01\n", "Epoch 7/30 iteration 29/60 loss 9.874982e-02\n", "Epoch 7/30 iteration 30/60 loss 1.273728e-01\n", "Epoch 7/30 iteration 31/60 loss 1.304254e-01\n", "Epoch 7/30 iteration 32/60 loss 1.290507e-01\n", "Epoch 7/30 iteration 33/60 loss 7.951749e-02\n", "Epoch 7/30 iteration 34/60 loss 9.418073e-02\n", "Epoch 7/30 iteration 35/60 loss 1.029518e-01\n", "Epoch 7/30 iteration 36/60 loss 1.200351e-01\n", "Epoch 7/30 iteration 37/60 loss 1.086250e-01\n", "Epoch 7/30 iteration 38/60 loss 1.081709e-01\n", "Epoch 7/30 iteration 39/60 loss 1.104232e-01\n", "Epoch 7/30 iteration 40/60 loss 9.968220e-02\n", "Epoch 7/30 iteration 41/60 loss 1.402561e-01\n", "Epoch 7/30 iteration 42/60 loss 1.444589e-01\n", "Epoch 7/30 iteration 43/60 loss 9.907074e-02\n", "Epoch 7/30 iteration 44/60 loss 1.125829e-01\n", "Epoch 7/30 iteration 45/60 loss 1.207557e-01\n", "Epoch 7/30 iteration 46/60 loss 1.107074e-01\n", "Epoch 7/30 iteration 47/60 loss 1.075932e-01\n", "Epoch 7/30 iteration 48/60 loss 1.294019e-01\n", "Epoch 7/30 iteration 49/60 loss 9.978452e-02\n", "Epoch 7/30 iteration 50/60 loss 9.640619e-02\n", "Epoch 7/30 iteration 51/60 loss 1.090421e-01\n", "Epoch 7/30 iteration 52/60 loss 1.241161e-01\n", "Epoch 7/30 iteration 53/60 loss 1.002734e-01\n", "Epoch 7/30 iteration 54/60 loss 1.570172e-01\n", "Epoch 7/30 iteration 55/60 loss 1.267823e-01\n", "Epoch 7/30 iteration 56/60 loss 1.131749e-01\n", "Epoch 7/30 iteration 57/60 loss 1.155882e-01\n", "Epoch 7/30 iteration 58/60 loss 1.192119e-01\n", "Epoch 7/30 iteration 59/60 loss 1.167645e-01\n", "Epoch 7/30 iteration 60/60 loss 1.493581e-01\n", "Epoch 8/30 iteration 1/60 loss 1.014067e-01\n", "Epoch 8/30 iteration 2/60 loss 1.057633e-01\n", "Epoch 8/30 iteration 3/60 loss 1.311273e-01\n", "Epoch 8/30 iteration 4/60 loss 1.068643e-01\n", "Epoch 8/30 iteration 5/60 loss 1.274024e-01\n", "Epoch 8/30 iteration 6/60 loss 1.088503e-01\n", "Epoch 8/30 iteration 7/60 loss 9.162042e-02\n", "Epoch 8/30 iteration 8/60 loss 1.117995e-01\n", "Epoch 8/30 iteration 9/60 loss 7.344221e-02\n", "Epoch 8/30 iteration 10/60 loss 9.957861e-02\n", "Epoch 8/30 iteration 11/60 loss 1.120609e-01\n", "Epoch 8/30 iteration 12/60 loss 1.151615e-01\n", "Epoch 8/30 iteration 13/60 loss 9.203801e-02\n", "Epoch 8/30 iteration 14/60 loss 7.613172e-02\n", "Epoch 8/30 iteration 15/60 loss 1.239789e-01\n", "Epoch 8/30 iteration 16/60 loss 9.677958e-02\n", "Epoch 8/30 iteration 17/60 loss 8.417474e-02\n", "Epoch 8/30 iteration 18/60 loss 1.005303e-01\n", "Epoch 8/30 iteration 19/60 loss 8.979536e-02\n", "Epoch 8/30 iteration 20/60 loss 1.103343e-01\n", "Epoch 8/30 iteration 21/60 loss 1.036086e-01\n", "Epoch 8/30 iteration 22/60 loss 9.481708e-02\n", "Epoch 8/30 iteration 23/60 loss 8.912113e-02\n", "Epoch 8/30 iteration 24/60 loss 1.032257e-01\n", "Epoch 8/30 iteration 25/60 loss 8.703552e-02\n", "Epoch 8/30 iteration 26/60 loss 9.632942e-02\n", "Epoch 8/30 iteration 27/60 loss 1.354027e-01\n", "Epoch 8/30 iteration 28/60 loss 1.420059e-01\n", "Epoch 8/30 iteration 29/60 loss 1.071565e-01\n", "Epoch 8/30 iteration 30/60 loss 1.008150e-01\n", "Epoch 8/30 iteration 31/60 loss 8.991566e-02\n", "Epoch 8/30 iteration 32/60 loss 8.693264e-02\n", "Epoch 8/30 iteration 33/60 loss 8.432475e-02\n", "Epoch 8/30 iteration 34/60 loss 1.197920e-01\n", "Epoch 8/30 iteration 35/60 loss 1.314195e-01\n", "Epoch 8/30 iteration 36/60 loss 1.301432e-01\n", "Epoch 8/30 iteration 37/60 loss 1.156403e-01\n", "Epoch 8/30 iteration 38/60 loss 9.067094e-02\n", "Epoch 8/30 iteration 39/60 loss 1.363443e-01\n", "Epoch 8/30 iteration 40/60 loss 1.022945e-01\n", "Epoch 8/30 iteration 41/60 loss 9.770200e-02\n", "Epoch 8/30 iteration 42/60 loss 1.142007e-01\n", "Epoch 8/30 iteration 43/60 loss 1.155107e-01\n", "Epoch 8/30 iteration 44/60 loss 9.180889e-02\n", "Epoch 8/30 iteration 45/60 loss 9.129339e-02\n", "Epoch 8/30 iteration 46/60 loss 1.087169e-01\n", "Epoch 8/30 iteration 47/60 loss 8.517482e-02\n", "Epoch 8/30 iteration 48/60 loss 1.134686e-01\n", "Epoch 8/30 iteration 49/60 loss 9.574133e-02\n", "Epoch 8/30 iteration 50/60 loss 1.274485e-01\n", "Epoch 8/30 iteration 51/60 loss 1.020887e-01\n", "Epoch 8/30 iteration 52/60 loss 1.210699e-01\n", "Epoch 8/30 iteration 53/60 loss 1.021850e-01\n", "Epoch 8/30 iteration 54/60 loss 8.099122e-02\n", "Epoch 8/30 iteration 55/60 loss 9.784356e-02\n", "Epoch 8/30 iteration 56/60 loss 1.120758e-01\n", "Epoch 8/30 iteration 57/60 loss 1.227385e-01\n", "Epoch 8/30 iteration 58/60 loss 1.281773e-01\n", "Epoch 8/30 iteration 59/60 loss 8.213913e-02\n", "Epoch 8/30 iteration 60/60 loss 1.148391e-01\n", "Epoch 9/30 iteration 1/60 loss 1.078813e-01\n", "Epoch 9/30 iteration 2/60 loss 9.083037e-02\n", "Epoch 9/30 iteration 3/60 loss 7.894283e-02\n", "Epoch 9/30 iteration 4/60 loss 1.055622e-01\n", "Epoch 9/30 iteration 5/60 loss 1.002927e-01\n", "Epoch 9/30 iteration 6/60 loss 1.138642e-01\n", "Epoch 9/30 iteration 7/60 loss 1.051978e-01\n", "Epoch 9/30 iteration 8/60 loss 1.010832e-01\n", "Epoch 9/30 iteration 9/60 loss 9.864510e-02\n", "Epoch 9/30 iteration 10/60 loss 9.089387e-02\n", "Epoch 9/30 iteration 11/60 loss 9.619322e-02\n", "Epoch 9/30 iteration 12/60 loss 1.015692e-01\n", "Epoch 9/30 iteration 13/60 loss 8.118288e-02\n", "Epoch 9/30 iteration 14/60 loss 6.347937e-02\n", "Epoch 9/30 iteration 15/60 loss 7.266455e-02\n", "Epoch 9/30 iteration 16/60 loss 8.871309e-02\n", "Epoch 9/30 iteration 17/60 loss 9.121257e-02\n", "Epoch 9/30 iteration 18/60 loss 7.608275e-02\n", "Epoch 9/30 iteration 19/60 loss 6.870835e-02\n", "Epoch 9/30 iteration 20/60 loss 8.359235e-02\n", "Epoch 9/30 iteration 21/60 loss 8.858244e-02\n", "Epoch 9/30 iteration 22/60 loss 7.850318e-02\n", "Epoch 9/30 iteration 23/60 loss 7.162030e-02\n", "Epoch 9/30 iteration 24/60 loss 1.104356e-01\n", "Epoch 9/30 iteration 25/60 loss 7.259460e-02\n", "Epoch 9/30 iteration 26/60 loss 9.818668e-02\n", "Epoch 9/30 iteration 27/60 loss 1.003015e-01\n", "Epoch 9/30 iteration 28/60 loss 8.870778e-02\n", "Epoch 9/30 iteration 29/60 loss 9.420870e-02\n", "Epoch 9/30 iteration 30/60 loss 1.133891e-01\n", "Epoch 9/30 iteration 31/60 loss 9.511310e-02\n", "Epoch 9/30 iteration 32/60 loss 1.300213e-01\n", "Epoch 9/30 iteration 33/60 loss 8.880331e-02\n", "Epoch 9/30 iteration 34/60 loss 6.875319e-02\n", "Epoch 9/30 iteration 35/60 loss 8.555046e-02\n", "Epoch 9/30 iteration 36/60 loss 1.289217e-01\n", "Epoch 9/30 iteration 37/60 loss 7.382618e-02\n", "Epoch 9/30 iteration 38/60 loss 1.014108e-01\n", "Epoch 9/30 iteration 39/60 loss 9.765117e-02\n", "Epoch 9/30 iteration 40/60 loss 7.295313e-02\n", "Epoch 9/30 iteration 41/60 loss 1.054180e-01\n", "Epoch 9/30 iteration 42/60 loss 9.589051e-02\n", "Epoch 9/30 iteration 43/60 loss 1.093192e-01\n", "Epoch 9/30 iteration 44/60 loss 6.274290e-02\n", "Epoch 9/30 iteration 45/60 loss 9.518845e-02\n", "Epoch 9/30 iteration 46/60 loss 8.818321e-02\n", "Epoch 9/30 iteration 47/60 loss 8.619930e-02\n", "Epoch 9/30 iteration 48/60 loss 8.284805e-02\n", "Epoch 9/30 iteration 49/60 loss 1.152956e-01\n", "Epoch 9/30 iteration 50/60 loss 8.678164e-02\n", "Epoch 9/30 iteration 51/60 loss 9.828892e-02\n", "Epoch 9/30 iteration 52/60 loss 7.905754e-02\n", "Epoch 9/30 iteration 53/60 loss 8.465852e-02\n", "Epoch 9/30 iteration 54/60 loss 1.025618e-01\n", "Epoch 9/30 iteration 55/60 loss 1.138767e-01\n", "Epoch 9/30 iteration 56/60 loss 1.073976e-01\n", "Epoch 9/30 iteration 57/60 loss 9.717995e-02\n", "Epoch 9/30 iteration 58/60 loss 1.199283e-01\n", "Epoch 9/30 iteration 59/60 loss 7.861901e-02\n", "Epoch 9/30 iteration 60/60 loss 6.799476e-02\n", "Epoch 10/30 iteration 1/60 loss 9.234691e-02\n", "Epoch 10/30 iteration 2/60 loss 7.636401e-02\n", "Epoch 10/30 iteration 3/60 loss 9.759475e-02\n", "Epoch 10/30 iteration 4/60 loss 8.696698e-02\n", "Epoch 10/30 iteration 5/60 loss 8.759143e-02\n", "Epoch 10/30 iteration 6/60 loss 6.513178e-02\n", "Epoch 10/30 iteration 7/60 loss 9.426711e-02\n", "Epoch 10/30 iteration 8/60 loss 9.352199e-02\n", "Epoch 10/30 iteration 9/60 loss 6.463364e-02\n", "Epoch 10/30 iteration 10/60 loss 7.432854e-02\n", "Epoch 10/30 iteration 11/60 loss 8.289580e-02\n", "Epoch 10/30 iteration 12/60 loss 9.569865e-02\n", "Epoch 10/30 iteration 13/60 loss 8.459748e-02\n", "Epoch 10/30 iteration 14/60 loss 6.935363e-02\n", "Epoch 10/30 iteration 15/60 loss 8.887809e-02\n", "Epoch 10/30 iteration 16/60 loss 8.105980e-02\n", "Epoch 10/30 iteration 17/60 loss 8.328007e-02\n", "Epoch 10/30 iteration 18/60 loss 9.008583e-02\n", "Epoch 10/30 iteration 19/60 loss 8.693419e-02\n", "Epoch 10/30 iteration 20/60 loss 8.273192e-02\n", "Epoch 10/30 iteration 21/60 loss 7.972801e-02\n", "Epoch 10/30 iteration 22/60 loss 5.491205e-02\n", "Epoch 10/30 iteration 23/60 loss 1.075932e-01\n", "Epoch 10/30 iteration 24/60 loss 9.602510e-02\n", "Epoch 10/30 iteration 25/60 loss 1.032842e-01\n", "Epoch 10/30 iteration 26/60 loss 8.502861e-02\n", "Epoch 10/30 iteration 27/60 loss 9.968244e-02\n", "Epoch 10/30 iteration 28/60 loss 8.510565e-02\n", "Epoch 10/30 iteration 29/60 loss 7.460428e-02\n", "Epoch 10/30 iteration 30/60 loss 8.980543e-02\n", "Epoch 10/30 iteration 31/60 loss 8.233826e-02\n", "Epoch 10/30 iteration 32/60 loss 8.139787e-02\n", "Epoch 10/30 iteration 33/60 loss 1.209391e-01\n", "Epoch 10/30 iteration 34/60 loss 7.084259e-02\n", "Epoch 10/30 iteration 35/60 loss 6.022143e-02\n", "Epoch 10/30 iteration 36/60 loss 9.063459e-02\n", "Epoch 10/30 iteration 37/60 loss 1.000430e-01\n", "Epoch 10/30 iteration 38/60 loss 9.755099e-02\n", "Epoch 10/30 iteration 39/60 loss 8.570157e-02\n", "Epoch 10/30 iteration 40/60 loss 7.192837e-02\n", "Epoch 10/30 iteration 41/60 loss 8.485953e-02\n", "Epoch 10/30 iteration 42/60 loss 7.557034e-02\n", "Epoch 10/30 iteration 43/60 loss 7.836923e-02\n", "Epoch 10/30 iteration 44/60 loss 7.217336e-02\n", "Epoch 10/30 iteration 45/60 loss 5.878876e-02\n", "Epoch 10/30 iteration 46/60 loss 8.656407e-02\n", "Epoch 10/30 iteration 47/60 loss 8.931419e-02\n", "Epoch 10/30 iteration 48/60 loss 7.643389e-02\n", "Epoch 10/30 iteration 49/60 loss 7.384518e-02\n", "Epoch 10/30 iteration 50/60 loss 7.872261e-02\n", "Epoch 10/30 iteration 51/60 loss 6.932097e-02\n", "Epoch 10/30 iteration 52/60 loss 6.421240e-02\n", "Epoch 10/30 iteration 53/60 loss 8.976328e-02\n", "Epoch 10/30 iteration 54/60 loss 7.473946e-02\n", "Epoch 10/30 iteration 55/60 loss 9.036523e-02\n", "Epoch 10/30 iteration 56/60 loss 1.043569e-01\n", "Epoch 10/30 iteration 57/60 loss 8.796164e-02\n", "Epoch 10/30 iteration 58/60 loss 8.668123e-02\n", "Epoch 10/30 iteration 59/60 loss 8.517113e-02\n", "Epoch 10/30 iteration 60/60 loss 7.544921e-02\n", "Epoch 11/30 iteration 1/60 loss 5.666972e-02\n", "Epoch 11/30 iteration 2/60 loss 8.701767e-02\n", "Epoch 11/30 iteration 3/60 loss 6.227640e-02\n", "Epoch 11/30 iteration 4/60 loss 7.740957e-02\n", "Epoch 11/30 iteration 5/60 loss 7.074599e-02\n", "Epoch 11/30 iteration 6/60 loss 6.795451e-02\n", "Epoch 11/30 iteration 7/60 loss 6.585341e-02\n", "Epoch 11/30 iteration 8/60 loss 6.513463e-02\n", "Epoch 11/30 iteration 9/60 loss 8.729713e-02\n", "Epoch 11/30 iteration 10/60 loss 6.561022e-02\n", "Epoch 11/30 iteration 11/60 loss 6.670790e-02\n", "Epoch 11/30 iteration 12/60 loss 7.938244e-02\n", "Epoch 11/30 iteration 13/60 loss 7.811184e-02\n", "Epoch 11/30 iteration 14/60 loss 9.588183e-02\n", "Epoch 11/30 iteration 15/60 loss 9.110650e-02\n", "Epoch 11/30 iteration 16/60 loss 5.708621e-02\n", "Epoch 11/30 iteration 17/60 loss 5.835858e-02\n", "Epoch 11/30 iteration 18/60 loss 6.938592e-02\n", "Epoch 11/30 iteration 19/60 loss 1.041567e-01\n", "Epoch 11/30 iteration 20/60 loss 6.870743e-02\n", "Epoch 11/30 iteration 21/60 loss 5.364067e-02\n", "Epoch 11/30 iteration 22/60 loss 8.212488e-02\n", "Epoch 11/30 iteration 23/60 loss 6.759892e-02\n", "Epoch 11/30 iteration 24/60 loss 8.564050e-02\n", "Epoch 11/30 iteration 25/60 loss 9.000925e-02\n", "Epoch 11/30 iteration 26/60 loss 5.222760e-02\n", "Epoch 11/30 iteration 27/60 loss 9.325029e-02\n", "Epoch 11/30 iteration 28/60 loss 6.715436e-02\n", "Epoch 11/30 iteration 29/60 loss 8.472126e-02\n", "Epoch 11/30 iteration 30/60 loss 8.448064e-02\n", "Epoch 11/30 iteration 31/60 loss 6.488612e-02\n", "Epoch 11/30 iteration 32/60 loss 9.887239e-02\n", "Epoch 11/30 iteration 33/60 loss 6.259727e-02\n", "Epoch 11/30 iteration 34/60 loss 7.816363e-02\n", "Epoch 11/30 iteration 35/60 loss 7.524212e-02\n", "Epoch 11/30 iteration 36/60 loss 4.790086e-02\n", "Epoch 11/30 iteration 37/60 loss 8.181605e-02\n", "Epoch 11/30 iteration 38/60 loss 6.131477e-02\n", "Epoch 11/30 iteration 39/60 loss 6.956294e-02\n", "Epoch 11/30 iteration 40/60 loss 8.465043e-02\n", "Epoch 11/30 iteration 41/60 loss 8.829059e-02\n", "Epoch 11/30 iteration 42/60 loss 7.262184e-02\n", "Epoch 11/30 iteration 43/60 loss 8.417833e-02\n", "Epoch 11/30 iteration 44/60 loss 6.763515e-02\n", "Epoch 11/30 iteration 45/60 loss 8.160805e-02\n", "Epoch 11/30 iteration 46/60 loss 1.013908e-01\n", "Epoch 11/30 iteration 47/60 loss 7.352680e-02\n", "Epoch 11/30 iteration 48/60 loss 8.047657e-02\n", "Epoch 11/30 iteration 49/60 loss 7.427986e-02\n", "Epoch 11/30 iteration 50/60 loss 9.308909e-02\n", "Epoch 11/30 iteration 51/60 loss 7.853285e-02\n", "Epoch 11/30 iteration 52/60 loss 8.835422e-02\n", "Epoch 11/30 iteration 53/60 loss 7.497614e-02\n", "Epoch 11/30 iteration 54/60 loss 8.462086e-02\n", "Epoch 11/30 iteration 55/60 loss 6.976774e-02\n", "Epoch 11/30 iteration 56/60 loss 9.812707e-02\n", "Epoch 11/30 iteration 57/60 loss 9.049746e-02\n", "Epoch 11/30 iteration 58/60 loss 6.435790e-02\n", "Epoch 11/30 iteration 59/60 loss 7.284597e-02\n", "Epoch 11/30 iteration 60/60 loss 6.051862e-02\n", "Epoch 12/30 iteration 1/60 loss 7.948475e-02\n", "Epoch 12/30 iteration 2/60 loss 5.191109e-02\n", "Epoch 12/30 iteration 3/60 loss 7.001609e-02\n", "Epoch 12/30 iteration 4/60 loss 7.033888e-02\n", "Epoch 12/30 iteration 5/60 loss 5.537537e-02\n", "Epoch 12/30 iteration 6/60 loss 6.998737e-02\n", "Epoch 12/30 iteration 7/60 loss 5.965875e-02\n", "Epoch 12/30 iteration 8/60 loss 5.495768e-02\n", "Epoch 12/30 iteration 9/60 loss 7.441522e-02\n", "Epoch 12/30 iteration 10/60 loss 6.867120e-02\n", "Epoch 12/30 iteration 11/60 loss 6.163524e-02\n", "Epoch 12/30 iteration 12/60 loss 4.764920e-02\n", "Epoch 12/30 iteration 13/60 loss 5.402888e-02\n", "Epoch 12/30 iteration 14/60 loss 6.213785e-02\n", "Epoch 12/30 iteration 15/60 loss 6.104750e-02\n", "Epoch 12/30 iteration 16/60 loss 1.015201e-01\n", "Epoch 12/30 iteration 17/60 loss 5.854390e-02\n", "Epoch 12/30 iteration 18/60 loss 6.398903e-02\n", "Epoch 12/30 iteration 19/60 loss 5.707650e-02\n", "Epoch 12/30 iteration 20/60 loss 7.437169e-02\n", "Epoch 12/30 iteration 21/60 loss 9.843734e-02\n", "Epoch 12/30 iteration 22/60 loss 7.510893e-02\n", "Epoch 12/30 iteration 23/60 loss 5.526597e-02\n", "Epoch 12/30 iteration 24/60 loss 5.827498e-02\n", "Epoch 12/30 iteration 25/60 loss 6.931452e-02\n", "Epoch 12/30 iteration 26/60 loss 6.727159e-02\n", "Epoch 12/30 iteration 27/60 loss 8.776272e-02\n", "Epoch 12/30 iteration 28/60 loss 8.951446e-02\n", "Epoch 12/30 iteration 29/60 loss 7.692128e-02\n", "Epoch 12/30 iteration 30/60 loss 8.929460e-02\n", "Epoch 12/30 iteration 31/60 loss 7.401639e-02\n", "Epoch 12/30 iteration 32/60 loss 6.938797e-02\n", "Epoch 12/30 iteration 33/60 loss 8.687175e-02\n", "Epoch 12/30 iteration 34/60 loss 8.753444e-02\n", "Epoch 12/30 iteration 35/60 loss 5.572415e-02\n", "Epoch 12/30 iteration 36/60 loss 6.132106e-02\n", "Epoch 12/30 iteration 37/60 loss 6.160327e-02\n", "Epoch 12/30 iteration 38/60 loss 6.829770e-02\n", "Epoch 12/30 iteration 39/60 loss 5.662048e-02\n", "Epoch 12/30 iteration 40/60 loss 6.534258e-02\n", "Epoch 12/30 iteration 41/60 loss 6.488361e-02\n", "Epoch 12/30 iteration 42/60 loss 7.071444e-02\n", "Epoch 12/30 iteration 43/60 loss 8.681848e-02\n", "Epoch 12/30 iteration 44/60 loss 7.028220e-02\n", "Epoch 12/30 iteration 45/60 loss 7.623701e-02\n", "Epoch 12/30 iteration 46/60 loss 7.630381e-02\n", "Epoch 12/30 iteration 47/60 loss 8.306305e-02\n", "Epoch 12/30 iteration 48/60 loss 7.933069e-02\n", "Epoch 12/30 iteration 49/60 loss 8.818280e-02\n", "Epoch 12/30 iteration 50/60 loss 6.708698e-02\n", "Epoch 12/30 iteration 51/60 loss 7.529664e-02\n", "Epoch 12/30 iteration 52/60 loss 6.249645e-02\n", "Epoch 12/30 iteration 53/60 loss 7.869664e-02\n", "Epoch 12/30 iteration 54/60 loss 6.843954e-02\n", "Epoch 12/30 iteration 55/60 loss 5.980313e-02\n", "Epoch 12/30 iteration 56/60 loss 6.154323e-02\n", "Epoch 12/30 iteration 57/60 loss 7.292379e-02\n", "Epoch 12/30 iteration 58/60 loss 5.717523e-02\n", "Epoch 12/30 iteration 59/60 loss 7.154105e-02\n", "Epoch 12/30 iteration 60/60 loss 6.784421e-02\n", "Epoch 13/30 iteration 1/60 loss 6.848118e-02\n", "Epoch 13/30 iteration 2/60 loss 7.052081e-02\n", "Epoch 13/30 iteration 3/60 loss 6.623928e-02\n", "Epoch 13/30 iteration 4/60 loss 6.131731e-02\n", "Epoch 13/30 iteration 5/60 loss 5.827336e-02\n", "Epoch 13/30 iteration 6/60 loss 5.771372e-02\n", "Epoch 13/30 iteration 7/60 loss 6.800959e-02\n", "Epoch 13/30 iteration 8/60 loss 6.427837e-02\n", "Epoch 13/30 iteration 9/60 loss 7.060159e-02\n", "Epoch 13/30 iteration 10/60 loss 6.474444e-02\n", "Epoch 13/30 iteration 11/60 loss 6.481043e-02\n", "Epoch 13/30 iteration 12/60 loss 6.859434e-02\n", "Epoch 13/30 iteration 13/60 loss 4.721150e-02\n", "Epoch 13/30 iteration 14/60 loss 5.642974e-02\n", "Epoch 13/30 iteration 15/60 loss 5.194378e-02\n", "Epoch 13/30 iteration 16/60 loss 5.101475e-02\n", "Epoch 13/30 iteration 17/60 loss 5.838873e-02\n", "Epoch 13/30 iteration 18/60 loss 7.329960e-02\n", "Epoch 13/30 iteration 19/60 loss 5.103143e-02\n", "Epoch 13/30 iteration 20/60 loss 6.388740e-02\n", "Epoch 13/30 iteration 21/60 loss 4.779285e-02\n", "Epoch 13/30 iteration 22/60 loss 7.268445e-02\n", "Epoch 13/30 iteration 23/60 loss 7.308382e-02\n", "Epoch 13/30 iteration 24/60 loss 4.360376e-02\n", "Epoch 13/30 iteration 25/60 loss 6.648098e-02\n", "Epoch 13/30 iteration 26/60 loss 4.195467e-02\n", "Epoch 13/30 iteration 27/60 loss 8.521536e-02\n", "Epoch 13/30 iteration 28/60 loss 8.736443e-02\n", "Epoch 13/30 iteration 29/60 loss 6.167388e-02\n", "Epoch 13/30 iteration 30/60 loss 7.774686e-02\n", "Epoch 13/30 iteration 31/60 loss 6.086308e-02\n", "Epoch 13/30 iteration 32/60 loss 7.375471e-02\n", "Epoch 13/30 iteration 33/60 loss 4.554885e-02\n", "Epoch 13/30 iteration 34/60 loss 7.122029e-02\n", "Epoch 13/30 iteration 35/60 loss 5.282730e-02\n", "Epoch 13/30 iteration 36/60 loss 7.351121e-02\n", "Epoch 13/30 iteration 37/60 loss 7.099612e-02\n", "Epoch 13/30 iteration 38/60 loss 6.369445e-02\n", "Epoch 13/30 iteration 39/60 loss 5.871558e-02\n", "Epoch 13/30 iteration 40/60 loss 6.031930e-02\n", "Epoch 13/30 iteration 41/60 loss 7.893104e-02\n", "Epoch 13/30 iteration 42/60 loss 6.605905e-02\n", "Epoch 13/30 iteration 43/60 loss 6.055351e-02\n", "Epoch 13/30 iteration 44/60 loss 5.965613e-02\n", "Epoch 13/30 iteration 45/60 loss 7.923406e-02\n", "Epoch 13/30 iteration 46/60 loss 6.098405e-02\n", "Epoch 13/30 iteration 47/60 loss 4.760082e-02\n", "Epoch 13/30 iteration 48/60 loss 5.596194e-02\n", "Epoch 13/30 iteration 49/60 loss 6.966930e-02\n", "Epoch 13/30 iteration 50/60 loss 8.535617e-02\n", "Epoch 13/30 iteration 51/60 loss 8.417282e-02\n", "Epoch 13/30 iteration 52/60 loss 7.346724e-02\n", "Epoch 13/30 iteration 53/60 loss 8.849531e-02\n", "Epoch 13/30 iteration 54/60 loss 6.532319e-02\n", "Epoch 13/30 iteration 55/60 loss 4.999448e-02\n", "Epoch 13/30 iteration 56/60 loss 5.973290e-02\n", "Epoch 13/30 iteration 57/60 loss 7.176845e-02\n", "Epoch 13/30 iteration 58/60 loss 6.128213e-02\n", "Epoch 13/30 iteration 59/60 loss 5.711015e-02\n", "Epoch 13/30 iteration 60/60 loss 6.320895e-02\n", "Epoch 14/30 iteration 1/60 loss 7.316539e-02\n", "Epoch 14/30 iteration 2/60 loss 5.628372e-02\n", "Epoch 14/30 iteration 3/60 loss 4.082441e-02\n", "Epoch 14/30 iteration 4/60 loss 7.368276e-02\n", "Epoch 14/30 iteration 5/60 loss 5.863928e-02\n", "Epoch 14/30 iteration 6/60 loss 4.294579e-02\n", "Epoch 14/30 iteration 7/60 loss 5.878832e-02\n", "Epoch 14/30 iteration 8/60 loss 5.047898e-02\n", "Epoch 14/30 iteration 9/60 loss 5.149799e-02\n", "Epoch 14/30 iteration 10/60 loss 6.301084e-02\n", "Epoch 14/30 iteration 11/60 loss 5.989907e-02\n", "Epoch 14/30 iteration 12/60 loss 5.380007e-02\n", "Epoch 14/30 iteration 13/60 loss 6.722277e-02\n", "Epoch 14/30 iteration 14/60 loss 6.606207e-02\n", "Epoch 14/30 iteration 15/60 loss 5.236108e-02\n", "Epoch 14/30 iteration 16/60 loss 5.420834e-02\n", "Epoch 14/30 iteration 17/60 loss 5.771532e-02\n", "Epoch 14/30 iteration 18/60 loss 7.784668e-02\n", "Epoch 14/30 iteration 19/60 loss 5.066681e-02\n", "Epoch 14/30 iteration 20/60 loss 5.950812e-02\n", "Epoch 14/30 iteration 21/60 loss 5.631524e-02\n", "Epoch 14/30 iteration 22/60 loss 4.687934e-02\n", "Epoch 14/30 iteration 23/60 loss 4.049658e-02\n", "Epoch 14/30 iteration 24/60 loss 6.406268e-02\n", "Epoch 14/30 iteration 25/60 loss 5.675092e-02\n", "Epoch 14/30 iteration 26/60 loss 5.036339e-02\n", "Epoch 14/30 iteration 27/60 loss 6.172265e-02\n", "Epoch 14/30 iteration 28/60 loss 7.133639e-02\n", "Epoch 14/30 iteration 29/60 loss 4.357151e-02\n", "Epoch 14/30 iteration 30/60 loss 5.090652e-02\n", "Epoch 14/30 iteration 31/60 loss 4.388752e-02\n", "Epoch 14/30 iteration 32/60 loss 4.815704e-02\n", "Epoch 14/30 iteration 33/60 loss 4.800608e-02\n", "Epoch 14/30 iteration 34/60 loss 5.402929e-02\n", "Epoch 14/30 iteration 35/60 loss 5.376553e-02\n", "Epoch 14/30 iteration 36/60 loss 4.277157e-02\n", "Epoch 14/30 iteration 37/60 loss 7.884223e-02\n", "Epoch 14/30 iteration 38/60 loss 5.620987e-02\n", "Epoch 14/30 iteration 39/60 loss 5.050397e-02\n", "Epoch 14/30 iteration 40/60 loss 5.484457e-02\n", "Epoch 14/30 iteration 41/60 loss 5.849423e-02\n", "Epoch 14/30 iteration 42/60 loss 6.274169e-02\n", "Epoch 14/30 iteration 43/60 loss 5.773868e-02\n", "Epoch 14/30 iteration 44/60 loss 4.751388e-02\n", "Epoch 14/30 iteration 45/60 loss 5.914413e-02\n", "Epoch 14/30 iteration 46/60 loss 6.055100e-02\n", "Epoch 14/30 iteration 47/60 loss 4.152948e-02\n", "Epoch 14/30 iteration 48/60 loss 7.209838e-02\n", "Epoch 14/30 iteration 49/60 loss 5.528712e-02\n", "Epoch 14/30 iteration 50/60 loss 6.141165e-02\n", "Epoch 14/30 iteration 51/60 loss 6.318222e-02\n", "Epoch 14/30 iteration 52/60 loss 1.023550e-01\n", "Epoch 14/30 iteration 53/60 loss 4.673494e-02\n", "Epoch 14/30 iteration 54/60 loss 6.360781e-02\n", "Epoch 14/30 iteration 55/60 loss 7.988529e-02\n", "Epoch 14/30 iteration 56/60 loss 5.010046e-02\n", "Epoch 14/30 iteration 57/60 loss 5.597935e-02\n", "Epoch 14/30 iteration 58/60 loss 6.506500e-02\n", "Epoch 14/30 iteration 59/60 loss 6.012530e-02\n", "Epoch 14/30 iteration 60/60 loss 5.925942e-02\n", "Epoch 15/30 iteration 1/60 loss 5.286470e-02\n", "Epoch 15/30 iteration 2/60 loss 5.223353e-02\n", "Epoch 15/30 iteration 3/60 loss 5.542358e-02\n", "Epoch 15/30 iteration 4/60 loss 5.586690e-02\n", "Epoch 15/30 iteration 5/60 loss 3.549435e-02\n", "Epoch 15/30 iteration 6/60 loss 5.686288e-02\n", "Epoch 15/30 iteration 7/60 loss 3.441252e-02\n", "Epoch 15/30 iteration 8/60 loss 5.998936e-02\n", "Epoch 15/30 iteration 9/60 loss 5.143726e-02\n", "Epoch 15/30 iteration 10/60 loss 4.199027e-02\n", "Epoch 15/30 iteration 11/60 loss 5.549772e-02\n", "Epoch 15/30 iteration 12/60 loss 5.475590e-02\n", "Epoch 15/30 iteration 13/60 loss 3.174656e-02\n", "Epoch 15/30 iteration 14/60 loss 5.367722e-02\n", "Epoch 15/30 iteration 15/60 loss 3.806247e-02\n", "Epoch 15/30 iteration 16/60 loss 4.758599e-02\n", "Epoch 15/30 iteration 17/60 loss 3.485851e-02\n", "Epoch 15/30 iteration 18/60 loss 6.174547e-02\n", "Epoch 15/30 iteration 19/60 loss 9.816795e-02\n", "Epoch 15/30 iteration 20/60 loss 2.695517e-02\n", "Epoch 15/30 iteration 21/60 loss 5.647181e-02\n", "Epoch 15/30 iteration 22/60 loss 4.026394e-02\n", "Epoch 15/30 iteration 23/60 loss 5.411763e-02\n", "Epoch 15/30 iteration 24/60 loss 5.768038e-02\n", "Epoch 15/30 iteration 25/60 loss 5.653333e-02\n", "Epoch 15/30 iteration 26/60 loss 4.139690e-02\n", "Epoch 15/30 iteration 27/60 loss 6.900817e-02\n", "Epoch 15/30 iteration 28/60 loss 5.363275e-02\n", "Epoch 15/30 iteration 29/60 loss 6.661172e-02\n", "Epoch 15/30 iteration 30/60 loss 4.744164e-02\n", "Epoch 15/30 iteration 31/60 loss 3.452324e-02\n", "Epoch 15/30 iteration 32/60 loss 6.408060e-02\n", "Epoch 15/30 iteration 33/60 loss 4.125813e-02\n", "Epoch 15/30 iteration 34/60 loss 5.623959e-02\n", "Epoch 15/30 iteration 35/60 loss 5.129208e-02\n", "Epoch 15/30 iteration 36/60 loss 5.005997e-02\n", "Epoch 15/30 iteration 37/60 loss 4.800960e-02\n", "Epoch 15/30 iteration 38/60 loss 4.398479e-02\n", "Epoch 15/30 iteration 39/60 loss 5.517742e-02\n", "Epoch 15/30 iteration 40/60 loss 5.251215e-02\n", "Epoch 15/30 iteration 41/60 loss 4.857729e-02\n", "Epoch 15/30 iteration 42/60 loss 5.914664e-02\n", "Epoch 15/30 iteration 43/60 loss 3.112919e-02\n", "Epoch 15/30 iteration 44/60 loss 4.529222e-02\n", "Epoch 15/30 iteration 45/60 loss 3.984561e-02\n", "Epoch 15/30 iteration 46/60 loss 6.773579e-02\n", "Epoch 15/30 iteration 47/60 loss 4.545084e-02\n", "Epoch 15/30 iteration 48/60 loss 4.566584e-02\n", "Epoch 15/30 iteration 49/60 loss 6.595416e-02\n", "Epoch 15/30 iteration 50/60 loss 4.924635e-02\n", "Epoch 15/30 iteration 51/60 loss 5.162095e-02\n", "Epoch 15/30 iteration 52/60 loss 7.519678e-02\n", "Epoch 15/30 iteration 53/60 loss 4.084817e-02\n", "Epoch 15/30 iteration 54/60 loss 5.184010e-02\n", "Epoch 15/30 iteration 55/60 loss 6.076666e-02\n", "Epoch 15/30 iteration 56/60 loss 5.480237e-02\n", "Epoch 15/30 iteration 57/60 loss 6.660028e-02\n", "Epoch 15/30 iteration 58/60 loss 4.802069e-02\n", "Epoch 15/30 iteration 59/60 loss 6.650259e-02\n", "Epoch 15/30 iteration 60/60 loss 5.955998e-02\n", "Epoch 16/30 iteration 1/60 loss 3.187731e-02\n", "Epoch 16/30 iteration 2/60 loss 3.689603e-02\n", "Epoch 16/30 iteration 3/60 loss 2.898584e-02\n", "Epoch 16/30 iteration 4/60 loss 5.530363e-02\n", "Epoch 16/30 iteration 5/60 loss 4.161577e-02\n", "Epoch 16/30 iteration 6/60 loss 3.837550e-02\n", "Epoch 16/30 iteration 7/60 loss 4.145394e-02\n", "Epoch 16/30 iteration 8/60 loss 3.471018e-02\n", "Epoch 16/30 iteration 9/60 loss 4.418807e-02\n", "Epoch 16/30 iteration 10/60 loss 5.863125e-02\n", "Epoch 16/30 iteration 11/60 loss 4.447248e-02\n", "Epoch 16/30 iteration 12/60 loss 5.626398e-02\n", "Epoch 16/30 iteration 13/60 loss 3.926966e-02\n", "Epoch 16/30 iteration 14/60 loss 5.029193e-02\n", "Epoch 16/30 iteration 15/60 loss 4.623499e-02\n", "Epoch 16/30 iteration 16/60 loss 6.701510e-02\n", "Epoch 16/30 iteration 17/60 loss 4.271232e-02\n", "Epoch 16/30 iteration 18/60 loss 3.939511e-02\n", "Epoch 16/30 iteration 19/60 loss 3.656479e-02\n", "Epoch 16/30 iteration 20/60 loss 3.604301e-02\n", "Epoch 16/30 iteration 21/60 loss 6.056806e-02\n", "Epoch 16/30 iteration 22/60 loss 5.367364e-02\n", "Epoch 16/30 iteration 23/60 loss 4.457333e-02\n", "Epoch 16/30 iteration 24/60 loss 4.572548e-02\n", "Epoch 16/30 iteration 25/60 loss 6.412143e-02\n", "Epoch 16/30 iteration 26/60 loss 4.883751e-02\n", "Epoch 16/30 iteration 27/60 loss 4.844224e-02\n", "Epoch 16/30 iteration 28/60 loss 3.829061e-02\n", "Epoch 16/30 iteration 29/60 loss 4.090505e-02\n", "Epoch 16/30 iteration 30/60 loss 4.200369e-02\n", "Epoch 16/30 iteration 31/60 loss 4.810308e-02\n", "Epoch 16/30 iteration 32/60 loss 5.234584e-02\n", "Epoch 16/30 iteration 33/60 loss 5.279995e-02\n", "Epoch 16/30 iteration 34/60 loss 3.614938e-02\n", "Epoch 16/30 iteration 35/60 loss 4.827019e-02\n", "Epoch 16/30 iteration 36/60 loss 4.137404e-02\n", "Epoch 16/30 iteration 37/60 loss 5.398320e-02\n", "Epoch 16/30 iteration 38/60 loss 4.553220e-02\n", "Epoch 16/30 iteration 39/60 loss 7.108425e-02\n", "Epoch 16/30 iteration 40/60 loss 6.261017e-02\n", "Epoch 16/30 iteration 41/60 loss 4.796258e-02\n", "Epoch 16/30 iteration 42/60 loss 7.559221e-02\n", "Epoch 16/30 iteration 43/60 loss 5.231358e-02\n", "Epoch 16/30 iteration 44/60 loss 4.901906e-02\n", "Epoch 16/30 iteration 45/60 loss 5.769069e-02\n", "Epoch 16/30 iteration 46/60 loss 3.135054e-02\n", "Epoch 16/30 iteration 47/60 loss 4.920128e-02\n", "Epoch 16/30 iteration 48/60 loss 3.600902e-02\n", "Epoch 16/30 iteration 49/60 loss 4.082410e-02\n", "Epoch 16/30 iteration 50/60 loss 5.707661e-02\n", "Epoch 16/30 iteration 51/60 loss 5.797957e-02\n", "Epoch 16/30 iteration 52/60 loss 5.432076e-02\n", "Epoch 16/30 iteration 53/60 loss 4.440129e-02\n", "Epoch 16/30 iteration 54/60 loss 5.737926e-02\n", "Epoch 16/30 iteration 55/60 loss 4.446498e-02\n", "Epoch 16/30 iteration 56/60 loss 5.137319e-02\n", "Epoch 16/30 iteration 57/60 loss 6.086333e-02\n", "Epoch 16/30 iteration 58/60 loss 4.694315e-02\n", "Epoch 16/30 iteration 59/60 loss 5.320931e-02\n", "Epoch 16/30 iteration 60/60 loss 5.310297e-02\n", "Epoch 17/30 iteration 1/60 loss 5.004358e-02\n", "Epoch 17/30 iteration 2/60 loss 3.222730e-02\n", "Epoch 17/30 iteration 3/60 loss 4.368144e-02\n", "Epoch 17/30 iteration 4/60 loss 4.472775e-02\n", "Epoch 17/30 iteration 5/60 loss 3.930552e-02\n", "Epoch 17/30 iteration 6/60 loss 4.144745e-02\n", "Epoch 17/30 iteration 7/60 loss 3.940764e-02\n", "Epoch 17/30 iteration 8/60 loss 5.747989e-02\n", "Epoch 17/30 iteration 9/60 loss 5.129134e-02\n", "Epoch 17/30 iteration 10/60 loss 5.169820e-02\n", "Epoch 17/30 iteration 11/60 loss 5.482836e-02\n", "Epoch 17/30 iteration 12/60 loss 4.745917e-02\n", "Epoch 17/30 iteration 13/60 loss 4.986620e-02\n", "Epoch 17/30 iteration 14/60 loss 3.878233e-02\n", "Epoch 17/30 iteration 15/60 loss 4.793305e-02\n", "Epoch 17/30 iteration 16/60 loss 3.657538e-02\n", "Epoch 17/30 iteration 17/60 loss 3.853533e-02\n", "Epoch 17/30 iteration 18/60 loss 2.991083e-02\n", "Epoch 17/30 iteration 19/60 loss 4.658032e-02\n", "Epoch 17/30 iteration 20/60 loss 4.718617e-02\n", "Epoch 17/30 iteration 21/60 loss 3.962609e-02\n", "Epoch 17/30 iteration 22/60 loss 4.161822e-02\n", "Epoch 17/30 iteration 23/60 loss 3.957597e-02\n", "Epoch 17/30 iteration 24/60 loss 4.600303e-02\n", "Epoch 17/30 iteration 25/60 loss 3.563586e-02\n", "Epoch 17/30 iteration 26/60 loss 4.640042e-02\n", "Epoch 17/30 iteration 27/60 loss 3.652013e-02\n", "Epoch 17/30 iteration 28/60 loss 3.598246e-02\n", "Epoch 17/30 iteration 29/60 loss 4.619129e-02\n", "Epoch 17/30 iteration 30/60 loss 4.311312e-02\n", "Epoch 17/30 iteration 31/60 loss 6.429487e-02\n", "Epoch 17/30 iteration 32/60 loss 5.700058e-02\n", "Epoch 17/30 iteration 33/60 loss 4.008275e-02\n", "Epoch 17/30 iteration 34/60 loss 2.942116e-02\n", "Epoch 17/30 iteration 35/60 loss 4.353973e-02\n", "Epoch 17/30 iteration 36/60 loss 2.377867e-02\n", "Epoch 17/30 iteration 37/60 loss 3.451715e-02\n", "Epoch 17/30 iteration 38/60 loss 5.648626e-02\n", "Epoch 17/30 iteration 39/60 loss 3.881289e-02\n", "Epoch 17/30 iteration 40/60 loss 6.448884e-02\n", "Epoch 17/30 iteration 41/60 loss 5.588360e-02\n", "Epoch 17/30 iteration 42/60 loss 4.821350e-02\n", "Epoch 17/30 iteration 43/60 loss 4.301345e-02\n", "Epoch 17/30 iteration 44/60 loss 4.940297e-02\n", "Epoch 17/30 iteration 45/60 loss 4.145021e-02\n", "Epoch 17/30 iteration 46/60 loss 4.583283e-02\n", "Epoch 17/30 iteration 47/60 loss 5.720795e-02\n", "Epoch 17/30 iteration 48/60 loss 3.694437e-02\n", "Epoch 17/30 iteration 49/60 loss 5.308903e-02\n", "Epoch 17/30 iteration 50/60 loss 4.953311e-02\n", "Epoch 17/30 iteration 51/60 loss 5.639818e-02\n", "Epoch 17/30 iteration 52/60 loss 5.294565e-02\n", "Epoch 17/30 iteration 53/60 loss 7.350646e-02\n", "Epoch 17/30 iteration 54/60 loss 6.001246e-02\n", "Epoch 17/30 iteration 55/60 loss 3.507512e-02\n", "Epoch 17/30 iteration 56/60 loss 3.917490e-02\n", "Epoch 17/30 iteration 57/60 loss 6.150527e-02\n", "Epoch 17/30 iteration 58/60 loss 4.160415e-02\n", "Epoch 17/30 iteration 59/60 loss 3.547834e-02\n", "Epoch 17/30 iteration 60/60 loss 4.078943e-02\n", "Epoch 18/30 iteration 1/60 loss 3.414904e-02\n", "Epoch 18/30 iteration 2/60 loss 4.758384e-02\n", "Epoch 18/30 iteration 3/60 loss 5.866758e-02\n", "Epoch 18/30 iteration 4/60 loss 4.068158e-02\n", "Epoch 18/30 iteration 5/60 loss 4.427263e-02\n", "Epoch 18/30 iteration 6/60 loss 3.922415e-02\n", "Epoch 18/30 iteration 7/60 loss 3.938901e-02\n", "Epoch 18/30 iteration 8/60 loss 3.791323e-02\n", "Epoch 18/30 iteration 9/60 loss 3.814859e-02\n", "Epoch 18/30 iteration 10/60 loss 3.350585e-02\n", "Epoch 18/30 iteration 11/60 loss 4.312675e-02\n", "Epoch 18/30 iteration 12/60 loss 5.595207e-02\n", "Epoch 18/30 iteration 13/60 loss 4.613611e-02\n", "Epoch 18/30 iteration 14/60 loss 5.472241e-02\n", "Epoch 18/30 iteration 15/60 loss 4.124759e-02\n", "Epoch 18/30 iteration 16/60 loss 3.654781e-02\n", "Epoch 18/30 iteration 17/60 loss 5.580974e-02\n", "Epoch 18/30 iteration 18/60 loss 4.451413e-02\n", "Epoch 18/30 iteration 19/60 loss 4.235303e-02\n", "Epoch 18/30 iteration 20/60 loss 4.323689e-02\n", "Epoch 18/30 iteration 21/60 loss 2.856163e-02\n", "Epoch 18/30 iteration 22/60 loss 3.447394e-02\n", "Epoch 18/30 iteration 23/60 loss 4.472925e-02\n", "Epoch 18/30 iteration 24/60 loss 4.980333e-02\n", "Epoch 18/30 iteration 25/60 loss 3.306974e-02\n", "Epoch 18/30 iteration 26/60 loss 3.467013e-02\n", "Epoch 18/30 iteration 27/60 loss 4.578506e-02\n", "Epoch 18/30 iteration 28/60 loss 5.874142e-02\n", "Epoch 18/30 iteration 29/60 loss 2.176724e-02\n", "Epoch 18/30 iteration 30/60 loss 4.483503e-02\n", "Epoch 18/30 iteration 31/60 loss 4.320022e-02\n", "Epoch 18/30 iteration 32/60 loss 4.973168e-02\n", "Epoch 18/30 iteration 33/60 loss 3.352432e-02\n", "Epoch 18/30 iteration 34/60 loss 5.135534e-02\n", "Epoch 18/30 iteration 35/60 loss 2.974908e-02\n", "Epoch 18/30 iteration 36/60 loss 5.526848e-02\n", "Epoch 18/30 iteration 37/60 loss 3.138850e-02\n", "Epoch 18/30 iteration 38/60 loss 4.292577e-02\n", "Epoch 18/30 iteration 39/60 loss 4.874125e-02\n", "Epoch 18/30 iteration 40/60 loss 6.108147e-02\n", "Epoch 18/30 iteration 41/60 loss 3.079985e-02\n", "Epoch 18/30 iteration 42/60 loss 4.284487e-02\n", "Epoch 18/30 iteration 43/60 loss 3.810488e-02\n", "Epoch 18/30 iteration 44/60 loss 4.382220e-02\n", "Epoch 18/30 iteration 45/60 loss 3.743097e-02\n", "Epoch 18/30 iteration 46/60 loss 3.987674e-02\n", "Epoch 18/30 iteration 47/60 loss 3.653007e-02\n", "Epoch 18/30 iteration 48/60 loss 4.213117e-02\n", "Epoch 18/30 iteration 49/60 loss 3.788071e-02\n", "Epoch 18/30 iteration 50/60 loss 2.376329e-02\n", "Epoch 18/30 iteration 51/60 loss 4.952196e-02\n", "Epoch 18/30 iteration 52/60 loss 4.149925e-02\n", "Epoch 18/30 iteration 53/60 loss 3.633192e-02\n", "Epoch 18/30 iteration 54/60 loss 3.480169e-02\n", "Epoch 18/30 iteration 55/60 loss 4.050085e-02\n", "Epoch 18/30 iteration 56/60 loss 4.808057e-02\n", "Epoch 18/30 iteration 57/60 loss 3.595768e-02\n", "Epoch 18/30 iteration 58/60 loss 3.919036e-02\n", "Epoch 18/30 iteration 59/60 loss 3.157930e-02\n", "Epoch 18/30 iteration 60/60 loss 4.868680e-02\n", "Epoch 19/30 iteration 1/60 loss 3.390720e-02\n", "Epoch 19/30 iteration 2/60 loss 3.913696e-02\n", "Epoch 19/30 iteration 3/60 loss 3.808791e-02\n", "Epoch 19/30 iteration 4/60 loss 3.730746e-02\n", "Epoch 19/30 iteration 5/60 loss 3.080242e-02\n", "Epoch 19/30 iteration 6/60 loss 3.767719e-02\n", "Epoch 19/30 iteration 7/60 loss 3.422583e-02\n", "Epoch 19/30 iteration 8/60 loss 4.132397e-02\n", "Epoch 19/30 iteration 9/60 loss 3.081361e-02\n", "Epoch 19/30 iteration 10/60 loss 3.540410e-02\n", "Epoch 19/30 iteration 11/60 loss 3.898554e-02\n", "Epoch 19/30 iteration 12/60 loss 3.657599e-02\n", "Epoch 19/30 iteration 13/60 loss 2.889078e-02\n", "Epoch 19/30 iteration 14/60 loss 4.370540e-02\n", "Epoch 19/30 iteration 15/60 loss 5.667674e-02\n", "Epoch 19/30 iteration 16/60 loss 3.823188e-02\n", "Epoch 19/30 iteration 17/60 loss 3.171281e-02\n", "Epoch 19/30 iteration 18/60 loss 3.469801e-02\n", "Epoch 19/30 iteration 19/60 loss 3.831056e-02\n", "Epoch 19/30 iteration 20/60 loss 4.284080e-02\n", "Epoch 19/30 iteration 21/60 loss 5.487891e-02\n", "Epoch 19/30 iteration 22/60 loss 2.406299e-02\n", "Epoch 19/30 iteration 23/60 loss 4.067361e-02\n", "Epoch 19/30 iteration 24/60 loss 3.126819e-02\n", "Epoch 19/30 iteration 25/60 loss 4.508031e-02\n", "Epoch 19/30 iteration 26/60 loss 3.996549e-02\n", "Epoch 19/30 iteration 27/60 loss 4.312384e-02\n", "Epoch 19/30 iteration 28/60 loss 4.158172e-02\n", "Epoch 19/30 iteration 29/60 loss 2.628975e-02\n", "Epoch 19/30 iteration 30/60 loss 3.814788e-02\n", "Epoch 19/30 iteration 31/60 loss 2.800366e-02\n", "Epoch 19/30 iteration 32/60 loss 3.477878e-02\n", "Epoch 19/30 iteration 33/60 loss 2.754552e-02\n", "Epoch 19/30 iteration 34/60 loss 2.913344e-02\n", "Epoch 19/30 iteration 35/60 loss 3.431461e-02\n", "Epoch 19/30 iteration 36/60 loss 2.673336e-02\n", "Epoch 19/30 iteration 37/60 loss 3.912858e-02\n", "Epoch 19/30 iteration 38/60 loss 2.395138e-02\n", "Epoch 19/30 iteration 39/60 loss 2.876132e-02\n", "Epoch 19/30 iteration 40/60 loss 4.818732e-02\n", "Epoch 19/30 iteration 41/60 loss 3.188495e-02\n", "Epoch 19/30 iteration 42/60 loss 3.706214e-02\n", "Epoch 19/30 iteration 43/60 loss 2.234084e-02\n", "Epoch 19/30 iteration 44/60 loss 3.365403e-02\n", "Epoch 19/30 iteration 45/60 loss 3.744371e-02\n", "Epoch 19/30 iteration 46/60 loss 4.017797e-02\n", "Epoch 19/30 iteration 47/60 loss 4.394723e-02\n", "Epoch 19/30 iteration 48/60 loss 3.127206e-02\n", "Epoch 19/30 iteration 49/60 loss 2.396055e-02\n", "Epoch 19/30 iteration 50/60 loss 3.577820e-02\n", "Epoch 19/30 iteration 51/60 loss 3.987161e-02\n", "Epoch 19/30 iteration 52/60 loss 2.964585e-02\n", "Epoch 19/30 iteration 53/60 loss 4.830622e-02\n", "Epoch 19/30 iteration 54/60 loss 3.068594e-02\n", "Epoch 19/30 iteration 55/60 loss 5.658322e-02\n", "Epoch 19/30 iteration 56/60 loss 2.417581e-02\n", "Epoch 19/30 iteration 57/60 loss 4.021562e-02\n", "Epoch 19/30 iteration 58/60 loss 3.868520e-02\n", "Epoch 19/30 iteration 59/60 loss 2.341410e-02\n", "Epoch 19/30 iteration 60/60 loss 4.074837e-02\n", "Epoch 20/30 iteration 1/60 loss 3.207437e-02\n", "Epoch 20/30 iteration 2/60 loss 2.985548e-02\n", "Epoch 20/30 iteration 3/60 loss 2.657185e-02\n", "Epoch 20/30 iteration 4/60 loss 2.065550e-02\n", "Epoch 20/30 iteration 5/60 loss 3.182359e-02\n", "Epoch 20/30 iteration 6/60 loss 3.030046e-02\n", "Epoch 20/30 iteration 7/60 loss 2.828691e-02\n", "Epoch 20/30 iteration 8/60 loss 3.121771e-02\n", "Epoch 20/30 iteration 9/60 loss 2.002835e-02\n", "Epoch 20/30 iteration 10/60 loss 3.829941e-02\n", "Epoch 20/30 iteration 11/60 loss 3.556832e-02\n", "Epoch 20/30 iteration 12/60 loss 3.130036e-02\n", "Epoch 20/30 iteration 13/60 loss 2.729733e-02\n", "Epoch 20/30 iteration 14/60 loss 3.137647e-02\n", "Epoch 20/30 iteration 15/60 loss 3.545750e-02\n", "Epoch 20/30 iteration 16/60 loss 3.930653e-02\n", "Epoch 20/30 iteration 17/60 loss 3.623077e-02\n", "Epoch 20/30 iteration 18/60 loss 2.063239e-02\n", "Epoch 20/30 iteration 19/60 loss 3.556792e-02\n", "Epoch 20/30 iteration 20/60 loss 3.545391e-02\n", "Epoch 20/30 iteration 21/60 loss 2.342280e-02\n", "Epoch 20/30 iteration 22/60 loss 4.842111e-02\n", "Epoch 20/30 iteration 23/60 loss 3.702892e-02\n", "Epoch 20/30 iteration 24/60 loss 2.778755e-02\n", "Epoch 20/30 iteration 25/60 loss 2.127750e-02\n", "Epoch 20/30 iteration 26/60 loss 3.431855e-02\n", "Epoch 20/30 iteration 27/60 loss 2.253204e-02\n", "Epoch 20/30 iteration 28/60 loss 2.845985e-02\n", "Epoch 20/30 iteration 29/60 loss 2.644084e-02\n", "Epoch 20/30 iteration 30/60 loss 4.179190e-02\n", "Epoch 20/30 iteration 31/60 loss 2.396011e-02\n", "Epoch 20/30 iteration 32/60 loss 3.623916e-02\n", "Epoch 20/30 iteration 33/60 loss 3.241020e-02\n", "Epoch 20/30 iteration 34/60 loss 4.213985e-02\n", "Epoch 20/30 iteration 35/60 loss 3.279527e-02\n", "Epoch 20/30 iteration 36/60 loss 2.921221e-02\n", "Epoch 20/30 iteration 37/60 loss 3.032305e-02\n", "Epoch 20/30 iteration 38/60 loss 2.345253e-02\n", "Epoch 20/30 iteration 39/60 loss 4.233387e-02\n", "Epoch 20/30 iteration 40/60 loss 2.913721e-02\n", "Epoch 20/30 iteration 41/60 loss 2.300616e-02\n", "Epoch 20/30 iteration 42/60 loss 2.439406e-02\n", "Epoch 20/30 iteration 43/60 loss 3.177070e-02\n", "Epoch 20/30 iteration 44/60 loss 4.420999e-02\n", "Epoch 20/30 iteration 45/60 loss 4.031741e-02\n", "Epoch 20/30 iteration 46/60 loss 2.712166e-02\n", "Epoch 20/30 iteration 47/60 loss 2.381432e-02\n", "Epoch 20/30 iteration 48/60 loss 2.391199e-02\n", "Epoch 20/30 iteration 49/60 loss 5.222839e-02\n", "Epoch 20/30 iteration 50/60 loss 3.818425e-02\n", "Epoch 20/30 iteration 51/60 loss 2.733395e-02\n", "Epoch 20/30 iteration 52/60 loss 3.283033e-02\n", "Epoch 20/30 iteration 53/60 loss 6.624117e-02\n", "Epoch 20/30 iteration 54/60 loss 2.219027e-02\n", "Epoch 20/30 iteration 55/60 loss 3.424757e-02\n", "Epoch 20/30 iteration 56/60 loss 5.330362e-02\n", "Epoch 20/30 iteration 57/60 loss 3.364044e-02\n", "Epoch 20/30 iteration 58/60 loss 3.381107e-02\n", "Epoch 20/30 iteration 59/60 loss 3.186830e-02\n", "Epoch 20/30 iteration 60/60 loss 5.485267e-02\n", "Epoch 21/30 iteration 1/60 loss 4.771741e-02\n", "Epoch 21/30 iteration 2/60 loss 4.621031e-02\n", "Epoch 21/30 iteration 3/60 loss 3.529404e-02\n", "Epoch 21/30 iteration 4/60 loss 2.575636e-02\n", "Epoch 21/30 iteration 5/60 loss 1.776840e-02\n", "Epoch 21/30 iteration 6/60 loss 3.520593e-02\n", "Epoch 21/30 iteration 7/60 loss 3.367742e-02\n", "Epoch 21/30 iteration 8/60 loss 1.793754e-02\n", "Epoch 21/30 iteration 9/60 loss 2.757732e-02\n", "Epoch 21/30 iteration 10/60 loss 3.645187e-02\n", "Epoch 21/30 iteration 11/60 loss 2.481675e-02\n", "Epoch 21/30 iteration 12/60 loss 3.110011e-02\n", "Epoch 21/30 iteration 13/60 loss 2.120193e-02\n", "Epoch 21/30 iteration 14/60 loss 1.980031e-02\n", "Epoch 21/30 iteration 15/60 loss 2.059595e-02\n", "Epoch 21/30 iteration 16/60 loss 2.862679e-02\n", "Epoch 21/30 iteration 17/60 loss 2.556830e-02\n", "Epoch 21/30 iteration 18/60 loss 4.055378e-02\n", "Epoch 21/30 iteration 19/60 loss 2.260055e-02\n", "Epoch 21/30 iteration 20/60 loss 2.617691e-02\n", "Epoch 21/30 iteration 21/60 loss 2.840438e-02\n", "Epoch 21/30 iteration 22/60 loss 2.463694e-02\n", "Epoch 21/30 iteration 23/60 loss 1.866104e-02\n", "Epoch 21/30 iteration 24/60 loss 3.636653e-02\n", "Epoch 21/30 iteration 25/60 loss 2.944184e-02\n", "Epoch 21/30 iteration 26/60 loss 3.986930e-02\n", "Epoch 21/30 iteration 27/60 loss 2.923150e-02\n", "Epoch 21/30 iteration 28/60 loss 1.942039e-02\n", "Epoch 21/30 iteration 29/60 loss 1.694731e-02\n", "Epoch 21/30 iteration 30/60 loss 3.506800e-02\n", "Epoch 21/30 iteration 31/60 loss 4.072146e-02\n", "Epoch 21/30 iteration 32/60 loss 3.307446e-02\n", "Epoch 21/30 iteration 33/60 loss 3.159696e-02\n", "Epoch 21/30 iteration 34/60 loss 2.170521e-02\n", "Epoch 21/30 iteration 35/60 loss 3.961062e-02\n", "Epoch 21/30 iteration 36/60 loss 2.866844e-02\n", "Epoch 21/30 iteration 37/60 loss 3.596763e-02\n", "Epoch 21/30 iteration 38/60 loss 3.094999e-02\n", "Epoch 21/30 iteration 39/60 loss 3.720694e-02\n", "Epoch 21/30 iteration 40/60 loss 2.691046e-02\n", "Epoch 21/30 iteration 41/60 loss 3.810558e-02\n", "Epoch 21/30 iteration 42/60 loss 1.732499e-02\n", "Epoch 21/30 iteration 43/60 loss 3.110371e-02\n", "Epoch 21/30 iteration 44/60 loss 3.836908e-02\n", "Epoch 21/30 iteration 45/60 loss 2.341726e-02\n", "Epoch 21/30 iteration 46/60 loss 3.193497e-02\n", "Epoch 21/30 iteration 47/60 loss 2.362278e-02\n", "Epoch 21/30 iteration 48/60 loss 3.564034e-02\n", "Epoch 21/30 iteration 49/60 loss 3.270741e-02\n", "Epoch 21/30 iteration 50/60 loss 4.594633e-02\n", "Epoch 21/30 iteration 51/60 loss 2.909661e-02\n", "Epoch 21/30 iteration 52/60 loss 2.163144e-02\n", "Epoch 21/30 iteration 53/60 loss 2.592414e-02\n", "Epoch 21/30 iteration 54/60 loss 4.281823e-02\n", "Epoch 21/30 iteration 55/60 loss 2.945587e-02\n", "Epoch 21/30 iteration 56/60 loss 1.589840e-02\n", "Epoch 21/30 iteration 57/60 loss 3.572334e-02\n", "Epoch 21/30 iteration 58/60 loss 2.670271e-02\n", "Epoch 21/30 iteration 59/60 loss 2.799757e-02\n", "Epoch 21/30 iteration 60/60 loss 2.540492e-02\n", "Epoch 22/30 iteration 1/60 loss 2.267578e-02\n", "Epoch 22/30 iteration 2/60 loss 1.570265e-02\n", "Epoch 22/30 iteration 3/60 loss 1.462274e-02\n", "Epoch 22/30 iteration 4/60 loss 2.001663e-02\n", "Epoch 22/30 iteration 5/60 loss 2.817133e-02\n", "Epoch 22/30 iteration 6/60 loss 2.584655e-02\n", "Epoch 22/30 iteration 7/60 loss 2.520597e-02\n", "Epoch 22/30 iteration 8/60 loss 1.465501e-02\n", "Epoch 22/30 iteration 9/60 loss 2.892034e-02\n", "Epoch 22/30 iteration 10/60 loss 2.469826e-02\n", "Epoch 22/30 iteration 11/60 loss 2.314330e-02\n", "Epoch 22/30 iteration 12/60 loss 2.190410e-02\n", "Epoch 22/30 iteration 13/60 loss 1.827957e-02\n", "Epoch 22/30 iteration 14/60 loss 2.544096e-02\n", "Epoch 22/30 iteration 15/60 loss 2.108983e-02\n", "Epoch 22/30 iteration 16/60 loss 1.843583e-02\n", "Epoch 22/30 iteration 17/60 loss 2.193255e-02\n", "Epoch 22/30 iteration 18/60 loss 3.459590e-02\n", "Epoch 22/30 iteration 19/60 loss 2.839163e-02\n", "Epoch 22/30 iteration 20/60 loss 1.843600e-02\n", "Epoch 22/30 iteration 21/60 loss 2.853995e-02\n", "Epoch 22/30 iteration 22/60 loss 2.071849e-02\n", "Epoch 22/30 iteration 23/60 loss 2.277375e-02\n", "Epoch 22/30 iteration 24/60 loss 1.662420e-02\n", "Epoch 22/30 iteration 25/60 loss 4.723117e-02\n", "Epoch 22/30 iteration 26/60 loss 1.752761e-02\n", "Epoch 22/30 iteration 27/60 loss 3.693568e-02\n", "Epoch 22/30 iteration 28/60 loss 2.277054e-02\n", "Epoch 22/30 iteration 29/60 loss 2.834735e-02\n", "Epoch 22/30 iteration 30/60 loss 2.571590e-02\n", "Epoch 22/30 iteration 31/60 loss 2.371014e-02\n", "Epoch 22/30 iteration 32/60 loss 1.862989e-02\n", "Epoch 22/30 iteration 33/60 loss 2.580525e-02\n", "Epoch 22/30 iteration 34/60 loss 1.820803e-02\n", "Epoch 22/30 iteration 35/60 loss 2.239054e-02\n", "Epoch 22/30 iteration 36/60 loss 1.828191e-02\n", "Epoch 22/30 iteration 37/60 loss 4.194647e-02\n", "Epoch 22/30 iteration 38/60 loss 4.204558e-02\n", "Epoch 22/30 iteration 39/60 loss 3.046080e-02\n", "Epoch 22/30 iteration 40/60 loss 1.971629e-02\n", "Epoch 22/30 iteration 41/60 loss 1.654344e-02\n", "Epoch 22/30 iteration 42/60 loss 2.748301e-02\n", "Epoch 22/30 iteration 43/60 loss 3.438049e-02\n", "Epoch 22/30 iteration 44/60 loss 1.820531e-02\n", "Epoch 22/30 iteration 45/60 loss 2.718890e-02\n", "Epoch 22/30 iteration 46/60 loss 3.403270e-02\n", "Epoch 22/30 iteration 47/60 loss 2.775526e-02\n", "Epoch 22/30 iteration 48/60 loss 2.239411e-02\n", "Epoch 22/30 iteration 49/60 loss 2.714446e-02\n", "Epoch 22/30 iteration 50/60 loss 3.034300e-02\n", "Epoch 22/30 iteration 51/60 loss 3.692735e-02\n", "Epoch 22/30 iteration 52/60 loss 2.577722e-02\n", "Epoch 22/30 iteration 53/60 loss 3.198551e-02\n", "Epoch 22/30 iteration 54/60 loss 4.374224e-02\n", "Epoch 22/30 iteration 55/60 loss 2.290322e-02\n", "Epoch 22/30 iteration 56/60 loss 2.083778e-02\n", "Epoch 22/30 iteration 57/60 loss 3.154930e-02\n", "Epoch 22/30 iteration 58/60 loss 3.081462e-02\n", "Epoch 22/30 iteration 59/60 loss 3.239931e-02\n", "Epoch 22/30 iteration 60/60 loss 3.314774e-02\n", "Epoch 23/30 iteration 1/60 loss 2.329823e-02\n", "Epoch 23/30 iteration 2/60 loss 2.569627e-02\n", "Epoch 23/30 iteration 3/60 loss 1.705502e-02\n", "Epoch 23/30 iteration 4/60 loss 1.962187e-02\n", "Epoch 23/30 iteration 5/60 loss 2.531456e-02\n", "Epoch 23/30 iteration 6/60 loss 1.785924e-02\n", "Epoch 23/30 iteration 7/60 loss 3.322075e-02\n", "Epoch 23/30 iteration 8/60 loss 2.127004e-02\n", "Epoch 23/30 iteration 9/60 loss 2.899547e-02\n", "Epoch 23/30 iteration 10/60 loss 1.989907e-02\n", "Epoch 23/30 iteration 11/60 loss 3.002794e-02\n", "Epoch 23/30 iteration 12/60 loss 2.773813e-02\n", "Epoch 23/30 iteration 13/60 loss 2.367176e-02\n", "Epoch 23/30 iteration 14/60 loss 2.940252e-02\n", "Epoch 23/30 iteration 15/60 loss 2.340090e-02\n", "Epoch 23/30 iteration 16/60 loss 2.479457e-02\n", "Epoch 23/30 iteration 17/60 loss 2.670464e-02\n", "Epoch 23/30 iteration 18/60 loss 2.151586e-02\n", "Epoch 23/30 iteration 19/60 loss 1.914508e-02\n", "Epoch 23/30 iteration 20/60 loss 1.964319e-02\n", "Epoch 23/30 iteration 21/60 loss 1.693251e-02\n", "Epoch 23/30 iteration 22/60 loss 2.780318e-02\n", "Epoch 23/30 iteration 23/60 loss 3.585224e-02\n", "Epoch 23/30 iteration 24/60 loss 2.756021e-02\n", "Epoch 23/30 iteration 25/60 loss 2.170962e-02\n", "Epoch 23/30 iteration 26/60 loss 1.587719e-02\n", "Epoch 23/30 iteration 27/60 loss 2.304654e-02\n", "Epoch 23/30 iteration 28/60 loss 1.352604e-02\n", "Epoch 23/30 iteration 29/60 loss 2.516779e-02\n", "Epoch 23/30 iteration 30/60 loss 2.614714e-02\n", "Epoch 23/30 iteration 31/60 loss 3.147797e-02\n", "Epoch 23/30 iteration 32/60 loss 2.617408e-02\n", "Epoch 23/30 iteration 33/60 loss 2.337731e-02\n", "Epoch 23/30 iteration 34/60 loss 2.039334e-02\n", "Epoch 23/30 iteration 35/60 loss 1.862754e-02\n", "Epoch 23/30 iteration 36/60 loss 2.637289e-02\n", "Epoch 23/30 iteration 37/60 loss 3.301072e-02\n", "Epoch 23/30 iteration 38/60 loss 1.391713e-02\n", "Epoch 23/30 iteration 39/60 loss 2.537156e-02\n", "Epoch 23/30 iteration 40/60 loss 1.853823e-02\n", "Epoch 23/30 iteration 41/60 loss 2.345317e-02\n", "Epoch 23/30 iteration 42/60 loss 1.874216e-02\n", "Epoch 23/30 iteration 43/60 loss 1.735976e-02\n", "Epoch 23/30 iteration 44/60 loss 3.529405e-02\n", "Epoch 23/30 iteration 45/60 loss 3.017239e-02\n", "Epoch 23/30 iteration 46/60 loss 1.560450e-02\n", "Epoch 23/30 iteration 47/60 loss 3.756632e-02\n", "Epoch 23/30 iteration 48/60 loss 3.010669e-02\n", "Epoch 23/30 iteration 49/60 loss 1.818407e-02\n", "Epoch 23/30 iteration 50/60 loss 2.803045e-02\n", "Epoch 23/30 iteration 51/60 loss 3.978969e-02\n", "Epoch 23/30 iteration 52/60 loss 2.161280e-02\n", "Epoch 23/30 iteration 53/60 loss 1.578852e-02\n", "Epoch 23/30 iteration 54/60 loss 2.279325e-02\n", "Epoch 23/30 iteration 55/60 loss 1.913052e-02\n", "Epoch 23/30 iteration 56/60 loss 2.611880e-02\n", "Epoch 23/30 iteration 57/60 loss 3.291880e-02\n", "Epoch 23/30 iteration 58/60 loss 2.271352e-02\n", "Epoch 23/30 iteration 59/60 loss 1.897492e-02\n", "Epoch 23/30 iteration 60/60 loss 2.091142e-02\n", "Epoch 24/30 iteration 1/60 loss 2.000795e-02\n", "Epoch 24/30 iteration 2/60 loss 1.707281e-02\n", "Epoch 24/30 iteration 3/60 loss 2.228511e-02\n", "Epoch 24/30 iteration 4/60 loss 1.107505e-02\n", "Epoch 24/30 iteration 5/60 loss 2.244556e-02\n", "Epoch 24/30 iteration 6/60 loss 1.639498e-02\n", "Epoch 24/30 iteration 7/60 loss 1.937288e-02\n", "Epoch 24/30 iteration 8/60 loss 1.833108e-02\n", "Epoch 24/30 iteration 9/60 loss 1.702779e-02\n", "Epoch 24/30 iteration 10/60 loss 1.917669e-02\n", "Epoch 24/30 iteration 11/60 loss 1.728105e-02\n", "Epoch 24/30 iteration 12/60 loss 2.810688e-02\n", "Epoch 24/30 iteration 13/60 loss 1.583582e-02\n", "Epoch 24/30 iteration 14/60 loss 1.685279e-02\n", "Epoch 24/30 iteration 15/60 loss 2.091303e-02\n", "Epoch 24/30 iteration 16/60 loss 2.673921e-02\n", "Epoch 24/30 iteration 17/60 loss 1.913322e-02\n", "Epoch 24/30 iteration 18/60 loss 1.553495e-02\n", "Epoch 24/30 iteration 19/60 loss 1.320947e-02\n", "Epoch 24/30 iteration 20/60 loss 1.942457e-02\n", "Epoch 24/30 iteration 21/60 loss 1.904851e-02\n", "Epoch 24/30 iteration 22/60 loss 2.832163e-02\n", "Epoch 24/30 iteration 23/60 loss 2.085733e-02\n", "Epoch 24/30 iteration 24/60 loss 2.321944e-02\n", "Epoch 24/30 iteration 25/60 loss 1.000286e-02\n", "Epoch 24/30 iteration 26/60 loss 1.905417e-02\n", "Epoch 24/30 iteration 27/60 loss 1.922051e-02\n", "Epoch 24/30 iteration 28/60 loss 2.500422e-02\n", "Epoch 24/30 iteration 29/60 loss 1.547524e-02\n", "Epoch 24/30 iteration 30/60 loss 1.648557e-02\n", "Epoch 24/30 iteration 31/60 loss 2.848950e-02\n", "Epoch 24/30 iteration 32/60 loss 1.493397e-02\n", "Epoch 24/30 iteration 33/60 loss 3.425650e-02\n", "Epoch 24/30 iteration 34/60 loss 1.022850e-02\n", "Epoch 24/30 iteration 35/60 loss 2.848590e-02\n", "Epoch 24/30 iteration 36/60 loss 2.375680e-02\n", "Epoch 24/30 iteration 37/60 loss 2.318381e-02\n", "Epoch 24/30 iteration 38/60 loss 2.463622e-02\n", "Epoch 24/30 iteration 39/60 loss 2.823709e-02\n", "Epoch 24/30 iteration 40/60 loss 2.426569e-02\n", "Epoch 24/30 iteration 41/60 loss 2.543092e-02\n", "Epoch 24/30 iteration 42/60 loss 2.631260e-02\n", "Epoch 24/30 iteration 43/60 loss 1.653041e-02\n", "Epoch 24/30 iteration 44/60 loss 1.722584e-02\n", "Epoch 24/30 iteration 45/60 loss 5.176650e-02\n", "Epoch 24/30 iteration 46/60 loss 1.861972e-02\n", "Epoch 24/30 iteration 47/60 loss 1.708845e-02\n", "Epoch 24/30 iteration 48/60 loss 2.568139e-02\n", "Epoch 24/30 iteration 49/60 loss 3.547923e-02\n", "Epoch 24/30 iteration 50/60 loss 1.972736e-02\n", "Epoch 24/30 iteration 51/60 loss 1.722540e-02\n", "Epoch 24/30 iteration 52/60 loss 2.863067e-02\n", "Epoch 24/30 iteration 53/60 loss 2.532451e-02\n", "Epoch 24/30 iteration 54/60 loss 2.623276e-02\n", "Epoch 24/30 iteration 55/60 loss 3.157199e-02\n", "Epoch 24/30 iteration 56/60 loss 2.474055e-02\n", "Epoch 24/30 iteration 57/60 loss 2.419435e-02\n", "Epoch 24/30 iteration 58/60 loss 2.845974e-02\n", "Epoch 24/30 iteration 59/60 loss 1.322265e-02\n", "Epoch 24/30 iteration 60/60 loss 1.491356e-02\n", "Epoch 25/30 iteration 1/60 loss 2.221126e-02\n", "Epoch 25/30 iteration 2/60 loss 1.566821e-02\n", "Epoch 25/30 iteration 3/60 loss 2.467163e-02\n", "Epoch 25/30 iteration 4/60 loss 2.637986e-02\n", "Epoch 25/30 iteration 5/60 loss 1.457182e-02\n", "Epoch 25/30 iteration 6/60 loss 1.152712e-02\n", "Epoch 25/30 iteration 7/60 loss 1.571925e-02\n", "Epoch 25/30 iteration 8/60 loss 1.582146e-02\n", "Epoch 25/30 iteration 9/60 loss 1.498530e-02\n", "Epoch 25/30 iteration 10/60 loss 1.823005e-02\n", "Epoch 25/30 iteration 11/60 loss 2.243865e-02\n", "Epoch 25/30 iteration 12/60 loss 1.260736e-02\n", "Epoch 25/30 iteration 13/60 loss 1.650682e-02\n", "Epoch 25/30 iteration 14/60 loss 1.514194e-02\n", "Epoch 25/30 iteration 15/60 loss 2.328467e-02\n", "Epoch 25/30 iteration 16/60 loss 1.303398e-02\n", "Epoch 25/30 iteration 17/60 loss 1.805766e-02\n", "Epoch 25/30 iteration 18/60 loss 2.708845e-02\n", "Epoch 25/30 iteration 19/60 loss 1.798976e-02\n", "Epoch 25/30 iteration 20/60 loss 1.327838e-02\n", "Epoch 25/30 iteration 21/60 loss 1.385737e-02\n", "Epoch 25/30 iteration 22/60 loss 1.256852e-02\n", "Epoch 25/30 iteration 23/60 loss 1.705007e-02\n", "Epoch 25/30 iteration 24/60 loss 1.273142e-02\n", "Epoch 25/30 iteration 25/60 loss 8.097520e-03\n", "Epoch 25/30 iteration 26/60 loss 2.301876e-02\n", "Epoch 25/30 iteration 27/60 loss 1.760321e-02\n", "Epoch 25/30 iteration 28/60 loss 1.443463e-02\n", "Epoch 25/30 iteration 29/60 loss 1.851290e-02\n", "Epoch 25/30 iteration 30/60 loss 1.284548e-02\n", "Epoch 25/30 iteration 31/60 loss 1.202810e-02\n", "Epoch 25/30 iteration 32/60 loss 1.579156e-02\n", "Epoch 25/30 iteration 33/60 loss 1.980345e-02\n", "Epoch 25/30 iteration 34/60 loss 2.132683e-02\n", "Epoch 25/30 iteration 35/60 loss 1.311738e-02\n", "Epoch 25/30 iteration 36/60 loss 1.511118e-02\n", "Epoch 25/30 iteration 37/60 loss 1.358348e-02\n", "Epoch 25/30 iteration 38/60 loss 1.623208e-02\n", "Epoch 25/30 iteration 39/60 loss 2.076954e-02\n", "Epoch 25/30 iteration 40/60 loss 2.146347e-02\n", "Epoch 25/30 iteration 41/60 loss 1.568102e-02\n", "Epoch 25/30 iteration 42/60 loss 1.502752e-02\n", "Epoch 25/30 iteration 43/60 loss 1.497519e-02\n", "Epoch 25/30 iteration 44/60 loss 2.075792e-02\n", "Epoch 25/30 iteration 45/60 loss 2.080785e-02\n", "Epoch 25/30 iteration 46/60 loss 2.962659e-02\n", "Epoch 25/30 iteration 47/60 loss 1.986332e-02\n", "Epoch 25/30 iteration 48/60 loss 2.243333e-02\n", "Epoch 25/30 iteration 49/60 loss 1.894664e-02\n", "Epoch 25/30 iteration 50/60 loss 2.469960e-02\n", "Epoch 25/30 iteration 51/60 loss 2.615363e-02\n", "Epoch 25/30 iteration 52/60 loss 3.937730e-02\n", "Epoch 25/30 iteration 53/60 loss 2.161381e-02\n", "Epoch 25/30 iteration 54/60 loss 1.083406e-02\n", "Epoch 25/30 iteration 55/60 loss 2.582054e-02\n", "Epoch 25/30 iteration 56/60 loss 1.753496e-02\n", "Epoch 25/30 iteration 57/60 loss 1.479279e-02\n", "Epoch 25/30 iteration 58/60 loss 1.396959e-02\n", "Epoch 25/30 iteration 59/60 loss 1.448226e-02\n", "Epoch 25/30 iteration 60/60 loss 1.255404e-02\n", "Epoch 26/30 iteration 1/60 loss 1.166323e-02\n", "Epoch 26/30 iteration 2/60 loss 1.230193e-02\n", "Epoch 26/30 iteration 3/60 loss 1.454022e-02\n", "Epoch 26/30 iteration 4/60 loss 3.965175e-02\n", "Epoch 26/30 iteration 5/60 loss 1.470852e-02\n", "Epoch 26/30 iteration 6/60 loss 1.236581e-02\n", "Epoch 26/30 iteration 7/60 loss 1.565552e-02\n", "Epoch 26/30 iteration 8/60 loss 1.570341e-02\n", "Epoch 26/30 iteration 9/60 loss 1.172737e-02\n", "Epoch 26/30 iteration 10/60 loss 1.180284e-02\n", "Epoch 26/30 iteration 11/60 loss 1.536539e-02\n", "Epoch 26/30 iteration 12/60 loss 1.251057e-02\n", "Epoch 26/30 iteration 13/60 loss 8.672419e-03\n", "Epoch 26/30 iteration 14/60 loss 2.067083e-02\n", "Epoch 26/30 iteration 15/60 loss 1.390245e-02\n", "Epoch 26/30 iteration 16/60 loss 1.396132e-02\n", "Epoch 26/30 iteration 17/60 loss 1.161412e-02\n", "Epoch 26/30 iteration 18/60 loss 1.797711e-02\n", "Epoch 26/30 iteration 19/60 loss 1.311021e-02\n", "Epoch 26/30 iteration 20/60 loss 3.365982e-02\n", "Epoch 26/30 iteration 21/60 loss 2.143271e-02\n", "Epoch 26/30 iteration 22/60 loss 1.063934e-02\n", "Epoch 26/30 iteration 23/60 loss 9.868468e-03\n", "Epoch 26/30 iteration 24/60 loss 1.880720e-02\n", "Epoch 26/30 iteration 25/60 loss 1.661969e-02\n", "Epoch 26/30 iteration 26/60 loss 1.060895e-02\n", "Epoch 26/30 iteration 27/60 loss 1.178522e-02\n", "Epoch 26/30 iteration 28/60 loss 1.055925e-02\n", "Epoch 26/30 iteration 29/60 loss 2.320151e-02\n", "Epoch 26/30 iteration 30/60 loss 1.456507e-02\n", "Epoch 26/30 iteration 31/60 loss 2.610074e-02\n", "Epoch 26/30 iteration 32/60 loss 1.616249e-02\n", "Epoch 26/30 iteration 33/60 loss 1.630794e-02\n", "Epoch 26/30 iteration 34/60 loss 1.149745e-02\n", "Epoch 26/30 iteration 35/60 loss 1.529864e-02\n", "Epoch 26/30 iteration 36/60 loss 1.340791e-02\n", "Epoch 26/30 iteration 37/60 loss 8.933233e-03\n", "Epoch 26/30 iteration 38/60 loss 1.885612e-02\n", "Epoch 26/30 iteration 39/60 loss 1.350928e-02\n", "Epoch 26/30 iteration 40/60 loss 2.072505e-02\n", "Epoch 26/30 iteration 41/60 loss 1.131739e-02\n", "Epoch 26/30 iteration 42/60 loss 1.577780e-02\n", "Epoch 26/30 iteration 43/60 loss 1.098552e-02\n", "Epoch 26/30 iteration 44/60 loss 1.663848e-02\n", "Epoch 26/30 iteration 45/60 loss 1.913286e-02\n", "Epoch 26/30 iteration 46/60 loss 1.715900e-02\n", "Epoch 26/30 iteration 47/60 loss 1.141995e-02\n", "Epoch 26/30 iteration 48/60 loss 1.622718e-02\n", "Epoch 26/30 iteration 49/60 loss 1.468100e-02\n", "Epoch 26/30 iteration 50/60 loss 1.589242e-02\n", "Epoch 26/30 iteration 51/60 loss 1.362159e-02\n", "Epoch 26/30 iteration 52/60 loss 2.175094e-02\n", "Epoch 26/30 iteration 53/60 loss 2.129240e-02\n", "Epoch 26/30 iteration 54/60 loss 2.280610e-02\n", "Epoch 26/30 iteration 55/60 loss 2.035093e-02\n", "Epoch 26/30 iteration 56/60 loss 2.314248e-02\n", "Epoch 26/30 iteration 57/60 loss 1.821478e-02\n", "Epoch 26/30 iteration 58/60 loss 1.314103e-02\n", "Epoch 26/30 iteration 59/60 loss 1.218902e-02\n", "Epoch 26/30 iteration 60/60 loss 1.417876e-02\n", "Epoch 27/30 iteration 1/60 loss 1.127851e-02\n", "Epoch 27/30 iteration 2/60 loss 1.146988e-02\n", "Epoch 27/30 iteration 3/60 loss 1.377964e-02\n", "Epoch 27/30 iteration 4/60 loss 6.775798e-03\n", "Epoch 27/30 iteration 5/60 loss 1.240846e-02\n", "Epoch 27/30 iteration 6/60 loss 2.101953e-02\n", "Epoch 27/30 iteration 7/60 loss 9.047457e-03\n", "Epoch 27/30 iteration 8/60 loss 2.558648e-02\n", "Epoch 27/30 iteration 9/60 loss 1.500781e-02\n", "Epoch 27/30 iteration 10/60 loss 1.172650e-02\n", "Epoch 27/30 iteration 11/60 loss 1.190380e-02\n", "Epoch 27/30 iteration 12/60 loss 1.529350e-02\n", "Epoch 27/30 iteration 13/60 loss 1.672340e-02\n", "Epoch 27/30 iteration 14/60 loss 1.123161e-02\n", "Epoch 27/30 iteration 15/60 loss 1.434248e-02\n", "Epoch 27/30 iteration 16/60 loss 1.426706e-02\n", "Epoch 27/30 iteration 17/60 loss 1.361619e-02\n", "Epoch 27/30 iteration 18/60 loss 1.187646e-02\n", "Epoch 27/30 iteration 19/60 loss 2.371498e-02\n", "Epoch 27/30 iteration 20/60 loss 1.380814e-02\n", "Epoch 27/30 iteration 21/60 loss 1.182854e-02\n", "Epoch 27/30 iteration 22/60 loss 1.008300e-02\n", "Epoch 27/30 iteration 23/60 loss 1.691967e-02\n", "Epoch 27/30 iteration 24/60 loss 1.012223e-02\n", "Epoch 27/30 iteration 25/60 loss 1.334247e-02\n", "Epoch 27/30 iteration 26/60 loss 1.636113e-02\n", "Epoch 27/30 iteration 27/60 loss 9.032222e-03\n", "Epoch 27/30 iteration 28/60 loss 2.856926e-02\n", "Epoch 27/30 iteration 29/60 loss 1.037300e-02\n", "Epoch 27/30 iteration 30/60 loss 1.420701e-02\n", "Epoch 27/30 iteration 31/60 loss 1.970352e-02\n", "Epoch 27/30 iteration 32/60 loss 1.601927e-02\n", "Epoch 27/30 iteration 33/60 loss 1.184127e-02\n", "Epoch 27/30 iteration 34/60 loss 1.668601e-02\n", "Epoch 27/30 iteration 35/60 loss 1.255630e-02\n", "Epoch 27/30 iteration 36/60 loss 1.786016e-02\n", "Epoch 27/30 iteration 37/60 loss 1.946926e-02\n", "Epoch 27/30 iteration 38/60 loss 1.126134e-02\n", "Epoch 27/30 iteration 39/60 loss 1.259185e-02\n", "Epoch 27/30 iteration 40/60 loss 1.283708e-02\n", "Epoch 27/30 iteration 41/60 loss 1.216588e-02\n", "Epoch 27/30 iteration 42/60 loss 1.857607e-02\n", "Epoch 27/30 iteration 43/60 loss 1.585112e-02\n", "Epoch 27/30 iteration 44/60 loss 1.942275e-02\n", "Epoch 27/30 iteration 45/60 loss 2.884592e-02\n", "Epoch 27/30 iteration 46/60 loss 1.204849e-02\n", "Epoch 27/30 iteration 47/60 loss 2.855255e-02\n", "Epoch 27/30 iteration 48/60 loss 1.849824e-02\n", "Epoch 27/30 iteration 49/60 loss 1.030367e-02\n", "Epoch 27/30 iteration 50/60 loss 1.234397e-02\n", "Epoch 27/30 iteration 51/60 loss 1.866140e-02\n", "Epoch 27/30 iteration 52/60 loss 1.317393e-02\n", "Epoch 27/30 iteration 53/60 loss 1.444706e-02\n", "Epoch 27/30 iteration 54/60 loss 1.907880e-02\n", "Epoch 27/30 iteration 55/60 loss 2.003645e-02\n", "Epoch 27/30 iteration 56/60 loss 1.586123e-02\n", "Epoch 27/30 iteration 57/60 loss 1.465846e-02\n", "Epoch 27/30 iteration 58/60 loss 2.369502e-02\n", "Epoch 27/30 iteration 59/60 loss 1.649183e-02\n", "Epoch 27/30 iteration 60/60 loss 1.105049e-02\n", "Epoch 28/30 iteration 1/60 loss 1.229750e-02\n", "Epoch 28/30 iteration 2/60 loss 9.370988e-03\n", "Epoch 28/30 iteration 3/60 loss 1.111893e-02\n", "Epoch 28/30 iteration 4/60 loss 1.626171e-02\n", "Epoch 28/30 iteration 5/60 loss 1.024704e-02\n", "Epoch 28/30 iteration 6/60 loss 6.149791e-03\n", "Epoch 28/30 iteration 7/60 loss 8.181250e-03\n", "Epoch 28/30 iteration 8/60 loss 2.773554e-02\n", "Epoch 28/30 iteration 9/60 loss 1.664400e-02\n", "Epoch 28/30 iteration 10/60 loss 5.504554e-03\n", "Epoch 28/30 iteration 11/60 loss 2.036426e-02\n", "Epoch 28/30 iteration 12/60 loss 1.593324e-02\n", "Epoch 28/30 iteration 13/60 loss 8.837545e-03\n", "Epoch 28/30 iteration 14/60 loss 1.047214e-02\n", "Epoch 28/30 iteration 15/60 loss 9.627848e-03\n", "Epoch 28/30 iteration 16/60 loss 8.968093e-03\n", "Epoch 28/30 iteration 17/60 loss 9.967753e-03\n", "Epoch 28/30 iteration 18/60 loss 9.333958e-03\n", "Epoch 28/30 iteration 19/60 loss 9.101396e-03\n", "Epoch 28/30 iteration 20/60 loss 1.064465e-02\n", "Epoch 28/30 iteration 21/60 loss 7.882718e-03\n", "Epoch 28/30 iteration 22/60 loss 1.101792e-02\n", "Epoch 28/30 iteration 23/60 loss 9.822614e-03\n", "Epoch 28/30 iteration 24/60 loss 1.560596e-02\n", "Epoch 28/30 iteration 25/60 loss 6.800528e-03\n", "Epoch 28/30 iteration 26/60 loss 1.083925e-02\n", "Epoch 28/30 iteration 27/60 loss 1.174902e-02\n", "Epoch 28/30 iteration 28/60 loss 8.766674e-03\n", "Epoch 28/30 iteration 29/60 loss 9.610323e-03\n", "Epoch 28/30 iteration 30/60 loss 9.060916e-03\n", "Epoch 28/30 iteration 31/60 loss 7.401435e-03\n", "Epoch 28/30 iteration 32/60 loss 6.260207e-03\n", "Epoch 28/30 iteration 33/60 loss 1.066540e-02\n", "Epoch 28/30 iteration 34/60 loss 1.470363e-02\n", "Epoch 28/30 iteration 35/60 loss 1.232584e-02\n", "Epoch 28/30 iteration 36/60 loss 1.938829e-02\n", "Epoch 28/30 iteration 37/60 loss 7.272769e-03\n", "Epoch 28/30 iteration 38/60 loss 1.485805e-02\n", "Epoch 28/30 iteration 39/60 loss 1.259716e-02\n", "Epoch 28/30 iteration 40/60 loss 1.285761e-02\n", "Epoch 28/30 iteration 41/60 loss 1.252016e-02\n", "Epoch 28/30 iteration 42/60 loss 1.484466e-02\n", "Epoch 28/30 iteration 43/60 loss 1.221592e-02\n", "Epoch 28/30 iteration 44/60 loss 1.319172e-02\n", "Epoch 28/30 iteration 45/60 loss 8.509991e-03\n", "Epoch 28/30 iteration 46/60 loss 1.061379e-02\n", "Epoch 28/30 iteration 47/60 loss 1.148551e-02\n", "Epoch 28/30 iteration 48/60 loss 7.720524e-03\n", "Epoch 28/30 iteration 49/60 loss 1.621384e-02\n", "Epoch 28/30 iteration 50/60 loss 1.255971e-02\n", "Epoch 28/30 iteration 51/60 loss 1.507710e-02\n", "Epoch 28/30 iteration 52/60 loss 2.207975e-02\n", "Epoch 28/30 iteration 53/60 loss 1.229838e-02\n", "Epoch 28/30 iteration 54/60 loss 9.431488e-03\n", "Epoch 28/30 iteration 55/60 loss 1.182416e-02\n", "Epoch 28/30 iteration 56/60 loss 9.564480e-03\n", "Epoch 28/30 iteration 57/60 loss 2.385934e-02\n", "Epoch 28/30 iteration 58/60 loss 1.872174e-02\n", "Epoch 28/30 iteration 59/60 loss 1.300564e-02\n", "Epoch 28/30 iteration 60/60 loss 1.099021e-02\n", "Epoch 29/30 iteration 1/60 loss 8.733294e-03\n", "Epoch 29/30 iteration 2/60 loss 1.181322e-02\n", "Epoch 29/30 iteration 3/60 loss 7.963166e-03\n", "Epoch 29/30 iteration 4/60 loss 6.457239e-03\n", "Epoch 29/30 iteration 5/60 loss 8.103415e-03\n", "Epoch 29/30 iteration 6/60 loss 9.664534e-03\n", "Epoch 29/30 iteration 7/60 loss 7.055564e-03\n", "Epoch 29/30 iteration 8/60 loss 1.112507e-02\n", "Epoch 29/30 iteration 9/60 loss 1.891421e-02\n", "Epoch 29/30 iteration 10/60 loss 1.019485e-02\n", "Epoch 29/30 iteration 11/60 loss 1.415514e-02\n", "Epoch 29/30 iteration 12/60 loss 7.557745e-03\n", "Epoch 29/30 iteration 13/60 loss 1.338920e-02\n", "Epoch 29/30 iteration 14/60 loss 2.131613e-02\n", "Epoch 29/30 iteration 15/60 loss 1.422145e-02\n", "Epoch 29/30 iteration 16/60 loss 9.966066e-03\n", "Epoch 29/30 iteration 17/60 loss 8.479210e-03\n", "Epoch 29/30 iteration 18/60 loss 1.073140e-02\n", "Epoch 29/30 iteration 19/60 loss 1.023581e-02\n", "Epoch 29/30 iteration 20/60 loss 1.258313e-02\n", "Epoch 29/30 iteration 21/60 loss 1.019904e-02\n", "Epoch 29/30 iteration 22/60 loss 7.194065e-03\n", "Epoch 29/30 iteration 23/60 loss 9.223606e-03\n", "Epoch 29/30 iteration 24/60 loss 9.926434e-03\n", "Epoch 29/30 iteration 25/60 loss 8.690640e-03\n", "Epoch 29/30 iteration 26/60 loss 9.453619e-03\n", "Epoch 29/30 iteration 27/60 loss 1.014985e-02\n", "Epoch 29/30 iteration 28/60 loss 1.028313e-02\n", "Epoch 29/30 iteration 29/60 loss 1.128677e-02\n", "Epoch 29/30 iteration 30/60 loss 1.228711e-02\n", "Epoch 29/30 iteration 31/60 loss 2.529289e-02\n", "Epoch 29/30 iteration 32/60 loss 1.430401e-02\n", "Epoch 29/30 iteration 33/60 loss 7.303025e-03\n", "Epoch 29/30 iteration 34/60 loss 9.759008e-03\n", "Epoch 29/30 iteration 35/60 loss 5.957085e-03\n", "Epoch 29/30 iteration 36/60 loss 6.721395e-03\n", "Epoch 29/30 iteration 37/60 loss 1.449672e-02\n", "Epoch 29/30 iteration 38/60 loss 7.030062e-03\n", "Epoch 29/30 iteration 39/60 loss 9.783615e-03\n", "Epoch 29/30 iteration 40/60 loss 1.047882e-02\n", "Epoch 29/30 iteration 41/60 loss 9.194482e-03\n", "Epoch 29/30 iteration 42/60 loss 1.072342e-02\n", "Epoch 29/30 iteration 43/60 loss 1.205567e-02\n", "Epoch 29/30 iteration 44/60 loss 7.600727e-03\n", "Epoch 29/30 iteration 45/60 loss 1.298408e-02\n", "Epoch 29/30 iteration 46/60 loss 1.128114e-02\n", "Epoch 29/30 iteration 47/60 loss 1.073928e-02\n", "Epoch 29/30 iteration 48/60 loss 9.300580e-03\n", "Epoch 29/30 iteration 49/60 loss 7.755416e-03\n", "Epoch 29/30 iteration 50/60 loss 9.142306e-03\n", "Epoch 29/30 iteration 51/60 loss 1.256518e-02\n", "Epoch 29/30 iteration 52/60 loss 1.498407e-02\n", "Epoch 29/30 iteration 53/60 loss 8.728888e-03\n", "Epoch 29/30 iteration 54/60 loss 1.540743e-02\n", "Epoch 29/30 iteration 55/60 loss 1.055407e-02\n", "Epoch 29/30 iteration 56/60 loss 9.399369e-03\n", "Epoch 29/30 iteration 57/60 loss 9.148048e-03\n", "Epoch 29/30 iteration 58/60 loss 6.360184e-03\n", "Epoch 29/30 iteration 59/60 loss 1.395655e-02\n", "Epoch 29/30 iteration 60/60 loss 1.290872e-02\n", "Epoch 30/30 iteration 1/60 loss 6.179453e-03\n", "Epoch 30/30 iteration 2/60 loss 5.771239e-03\n", "Epoch 30/30 iteration 3/60 loss 7.098936e-03\n", "Epoch 30/30 iteration 4/60 loss 1.022085e-02\n", "Epoch 30/30 iteration 5/60 loss 1.430857e-02\n", "Epoch 30/30 iteration 6/60 loss 6.720105e-03\n", "Epoch 30/30 iteration 7/60 loss 5.677706e-03\n", "Epoch 30/30 iteration 8/60 loss 1.214592e-02\n", "Epoch 30/30 iteration 9/60 loss 6.626346e-03\n", "Epoch 30/30 iteration 10/60 loss 7.727925e-03\n", "Epoch 30/30 iteration 11/60 loss 1.031202e-02\n", "Epoch 30/30 iteration 12/60 loss 5.855930e-03\n", "Epoch 30/30 iteration 13/60 loss 8.365822e-03\n", "Epoch 30/30 iteration 14/60 loss 5.262495e-03\n", "Epoch 30/30 iteration 15/60 loss 1.115127e-02\n", "Epoch 30/30 iteration 16/60 loss 7.133554e-03\n", "Epoch 30/30 iteration 17/60 loss 6.585280e-03\n", "Epoch 30/30 iteration 18/60 loss 5.692922e-03\n", "Epoch 30/30 iteration 19/60 loss 1.134837e-02\n", "Epoch 30/30 iteration 20/60 loss 9.005466e-03\n", "Epoch 30/30 iteration 21/60 loss 9.749428e-03\n", "Epoch 30/30 iteration 22/60 loss 5.590283e-03\n", "Epoch 30/30 iteration 23/60 loss 4.960632e-03\n", "Epoch 30/30 iteration 24/60 loss 8.119321e-03\n", "Epoch 30/30 iteration 25/60 loss 7.725057e-03\n", "Epoch 30/30 iteration 26/60 loss 5.392130e-03\n", "Epoch 30/30 iteration 27/60 loss 1.323371e-02\n", "Epoch 30/30 iteration 28/60 loss 7.973009e-03\n", "Epoch 30/30 iteration 29/60 loss 6.893697e-03\n", "Epoch 30/30 iteration 30/60 loss 9.262742e-03\n", "Epoch 30/30 iteration 31/60 loss 1.064317e-02\n", "Epoch 30/30 iteration 32/60 loss 7.458477e-03\n", "Epoch 30/30 iteration 33/60 loss 5.028168e-03\n", "Epoch 30/30 iteration 34/60 loss 8.583856e-03\n", "Epoch 30/30 iteration 35/60 loss 6.574021e-03\n", "Epoch 30/30 iteration 36/60 loss 9.829297e-03\n", "Epoch 30/30 iteration 37/60 loss 8.709253e-03\n", "Epoch 30/30 iteration 38/60 loss 5.577198e-03\n", "Epoch 30/30 iteration 39/60 loss 8.436301e-03\n", "Epoch 30/30 iteration 40/60 loss 1.187703e-02\n", "Epoch 30/30 iteration 41/60 loss 1.024105e-02\n", "Epoch 30/30 iteration 42/60 loss 1.710397e-02\n", "Epoch 30/30 iteration 43/60 loss 5.947308e-03\n", "Epoch 30/30 iteration 44/60 loss 8.520350e-03\n", "Epoch 30/30 iteration 45/60 loss 1.712160e-02\n", "Epoch 30/30 iteration 46/60 loss 1.143608e-02\n", "Epoch 30/30 iteration 47/60 loss 9.953085e-03\n", "Epoch 30/30 iteration 48/60 loss 8.214330e-03\n", "Epoch 30/30 iteration 49/60 loss 9.763375e-03\n", "Epoch 30/30 iteration 50/60 loss 1.290422e-02\n", "Epoch 30/30 iteration 51/60 loss 8.730859e-03\n", "Epoch 30/30 iteration 52/60 loss 2.860385e-02\n", "Epoch 30/30 iteration 53/60 loss 7.801368e-03\n", "Epoch 30/30 iteration 54/60 loss 9.244008e-03\n", "Epoch 30/30 iteration 55/60 loss 1.313851e-02\n", "Epoch 30/30 iteration 56/60 loss 1.393352e-02\n", "Epoch 30/30 iteration 57/60 loss 8.797216e-03\n", "Epoch 30/30 iteration 58/60 loss 1.182736e-02\n", "Epoch 30/30 iteration 59/60 loss 8.285982e-03\n", "Epoch 30/30 iteration 60/60 loss 4.114440e-03\n" ] } ], "source": [ "n_epochs = 30\n", " \n", "for epoch in range(n_epochs):\n", " \n", " for i,(input,label) in enumerate(dataloader_train):\n", " \n", " input = tn.reshape(input.to(device),[-1,1,7,4,7,4])\n", " label = label.to(device)\n", " \n", " optimizer.zero_grad()\n", " output = model(input)\n", " loss = loss_function(output, label)\n", " loss.backward()\n", " optimizer.step()\n", " print('Epoch %d/%d iteration %d/%d loss %e'%(epoch+1,n_epochs,i+1,len(dataloader_train),loss))\n", " \n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Evaluate on Test Set\n", "\n", "Compute the classification accuracy on the held-out test set:\n", "\n", "1. Iterate through all test batches (no gradient computation needed)\n", "2. For each sample, take the **argmax** of the output logits as the predicted class\n", "3. Count correct predictions and compute `accuracy = correct / total`\n", "\n", "A well-trained TT network should achieve **~97-98% accuracy** on MNIST, comparable to standard fully-connected networks but with significantly fewer parameters." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Test accuracy tensor(0.9735)\n" ] } ], "source": [ "n_correct = 0\n", "n_total = 0\n", "for (input,label) in dataloader_test:\n", " input = tn.reshape(input.to(device),[-1,1,7,4,7,4])\n", " \n", " output = model(input).cpu()\n", " \n", " n_correct += tn.sum(tn.max(output,1)[1] == label) \n", " \n", " n_total += input.shape[0]\n", " \n", "print('Test accuracy ',n_correct/n_total)\n" ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" }, "orig_nbformat": 4 }, "nbformat": 4, "nbformat_minor": 2 }