Shift error

Jose asked 2 weeks ago

Hi from Spain.

I’m teaching myself machine learning and deep learning in python, and I am now much further into it than I expected months ago, thanks in part to your online posts. So thank you very very much.

Sometimes I come across funny results that sometimes I disregard or I finally understand. This time I found a model behaviour that is haunting me.

The subject is about conv2d over a dataset I prepare using consecutive rows and a lookback l=20; this is, if xij is a row (x=value, i=feature, j=row). My “images” are like this:

x11, x21, x31,…
… (l rows)
x1(1+l), x2(1+l),x3(1+l)

for target y(1+l+1) adding 1 so I can predict the next row’s y

next frame

x1j, x2j, x3j …

x1(1+l+j), x2(1+l+j),x3(1+l+j)

for target y(1+l+j+1)

(there’s probably a notation error, but you get it)

The problem is that it doesn’t work when it should, It works when it shouldn’t: in i+l .

I mean, during the testing, the accuracy is very good, but not for the correct moment in time, but l leaps in the future:

if Y is actual event and y is prediction, the ideal situation would be Y(1+l+j+1) = y(1+l+j+1) = predict(x1(1+j),… | x1(1+j+l) )

but it is accurate l leaps later, when it is not needed anymore,  Y(1+l+j+1+l) = y(1+l+j+1) = predict(x1(1+j),… | x1(1+j+l) )

The damn thing nails it if I take the predictions with an offset 20 leaps in the future; If it was 20 leaps in the past, fine, I’d take a result 20 leaps back in the log, … but in the future? And nailing it? Bam! I know it is not by chance, and I’m messing up something.

Let's pause for a second! 👩‍💻

Blogs at MachineCurve teach Machine Learning for Developers. Sign up to MachineCurve's free Machine Learning update today! You will learn new things and better understand concepts you already know.

We send emails at least every Friday. Welcome!

By signing up, you consent that any information you receive can include services and special offers by email.

Cheers!

PS:
This is how I build the frames:

shift=0
look_back = 20

if normalize:
scalerx = MinMaxScaler(feature_range =(-1, 1))
scalerx = scalerx.fit(x1)
x_train = scalerx.transform(x1)
x_test = scalerx.transform(x2)
y_train = y1
y_test = y2
else:
x_train = x1
y_train = y1
x_test = x2
y_test = y2

def reshapexy(xp,yp, look_back, shift):
 global normalize
 if normalize:
  x = xp
 else:
  x= xp.to_numpy()

 y= yp.to_numpy()
 dataX, dataY = [], []
 offst = look_back+shift
 rng = len(x)-offst
 for i in range(rng):
  a = x[i:(i+offst)]
  dataX.append(a)
  dataY.append(y[i+offst].astype(np.float32))
 return np.array(dataX), np.array(dataY)

x_train_r, y_train_r = reshapexy(x_train,y_train,look_back,shift)
x_test_r, y_test_r = reshapexy(x_test,y_test,look_back,shift)
y_train_rh = categorize2(y_train_r,True)
y_test_rh = categorize2(y_test_r,True)
x_train_rh = tf.expand_dims(x_train_r, axis=-1)
x_test_rh = tf.expand_dims(x_test_r, axis=-1)

k4model=Sequential()
k4model.add(Conv2D(num_features, (num_features,5), activation=’relu’, data_format=’channels_last’,padding=’same’, input_shape=(look_back,num_features,1)))
k4model.add(Flatten())
k4model.add(Dense(2, activation=’softmax’))
k4model.compile(loss=’categorical_crossentropy’,optimizer=’adam’,metrics=’categorical_accuracy’)
#k4model.summary()
k4history= k4model.fit(x_train_rh,y_train_rh, epochs=epochs, batch_size=batch, shuffle=False, class_weight = cws_dict_full,verbose=0)
y_k4pred = np.argmax(k4model.predict(x_test_rh), axis=-1)

Chris Staff replied 2 weeks ago

Can it be the case that this isn’t the full code?

Your Answer

8 + 1 =