ML-Notes

Creating Valid Data

When creating sample data, you need at least a 2D tensor/matrix. Because machine learning models require a feature dimension. ie (n, 1) where n is some sample, and 1 is the corresponding feature.

As an example:

For a house: (Sample: n, Feature: 3)

• Sample:
  1. A specific house.

• Features:

1. Size: 1500 square feet.
2. Bedrooms: 3.
3. Location Index: 2 (e.g., urban area).

This is usually done with the unsqueeze dim=1 property for a range. ie:

X = torch.arange(0, 1, 0.02).unsqueeze(dim=1)

The torch.arange creates a matrix of 50 samples, but no features - the unsqueeze at the first dimension adds the feature column/dimension.

Creating/Inheriting Model class

When creating a model, you will need to import nn from torch, and in particular nn.Module.

Usually something like:

import torch

from torch import nn

You will have to subclass it, in a custom class, that uses the Module as a superclass.

Inside that you will need to initialize the the weights and biases, usually to random or zero, and set the forward loop. The forward loop is required.

After that is created, you will need to initialize the loss function, and the optimizer (and which paramars you are optimizing.)

Then, in the training loop, you will need to set the model to train mode, do a forward propagation, calculate the loss, set the gradient accumulation to zero, do the backward propagation, and then the step function.

Once this is done you can do a test, using model eval, and a forward pass on the test data, then calculate the loss, and see the results (on previously unseen data)