Time2Vec as positional encoding for Timeseries Transformer

External question — Data Science Stack Exchange Author: Enk9456 Original post: https://datascience.stackexchange.com/questions/122543 License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/ Adaptation: HTML converted to plain text; contact email addresses removed. Although the transformer architecture was originally designed for NLP, there exists several articles and papers that attempt to apply the same architecture for numerical timeseries classification. These are some notable examples: https://towardsdatascience.com/the-time-series-transformer-2a521a0efad3 https://towardsdatascience.com/time2vec-for-time-series-features-encoding-a03a4f3f937e https://towardsdatascience.com/stock-predictions-with-state-of-the-art-transformer-and-time-embeddings-3a4485237de6 Based on the above, it seems that the usage of time2vec for positional encoding is prevalent. However, my concern is that all these articles use timeseries values as inputs to time2vec (e.g., prices or sales) rather than the actual temporal information (i.e., a time index t=0,1,2,3..). This seems problematic as: This is not a positional encoding, but an enrichment of feature representation Even the word2vec paper (If I understood it correctly) mentions that it is designed to work on a time related feature. Several persons have asked similar questions in other sites, for example this is taken from the first article: Why is the feature being encoded in time2vec and not the actual time index? How does encoding the feature provide positional encoding? Yet, I was not able to find any explanations. So, can someone please resolve this? Is this design approach a widespread misinterpretation of positional encoding? Note: Even if time2vec is applied on a time index, it should probably be a global time index created before batching/windowing the data, because the local time index within the batch is arbitrary.