Antony A
Antony A
Nov 6 · 1 min read
  1. Can you share how you would have calculated the single vector representation of the sentence ‘I really enjoyed this movie a lot.’ I researched any many as using Bert As service to get sentence representation and there are some trouble setting up Berth as service in colab and local machine .
  2. I ran the sample code of your for sentence ‘I really enjoyed this movie a lot.’ . But the value of hidden_reps[:,0] and cls_head are not same . You said they must be same right ? why is it different ? PFB the first 5 dimension of hidden_reps[:,0] and cls_head
    hidden_reps[:,0] ->[ 3.5817e-01, -4.2569e-01, -3.4023e-02, -6.2397e-02, -2.9916e-01,
    cls_head -> [-8.4221e-01, -3.1909e-01, -6.1012e-01, 4.4884e-01, 5.2340e-01

2.

hidden_reps shape is torch.Size([1, 12, 768])
cls_head shape is torch.Size([1, 768])

This is how i tried to get vector sentences using hidden_reps .But not sure if im correct
*Sentence length is 12 tokens since we padded .
*if i do torch.mean(hidden_reps[0],1) it gives me back a vector with 12 dimension
Note:Im taking here hidden_reps[0] for calculating mean along column since first dimension is batch size .Is this correct ?

Unable to get vector sentences using cls_head
*how to represent the cls_head which is [1, 768] into a dimension of 1,12 .Since sentences must be represented in dimension (1,12) (i.e) 1 row and 12 tokens .

Antony A

Written by

Antony A