all AI news
[D] What are the embedding tokens used for in Llama index?
April 22, 2023, 3:52 p.m. | /u/ultra_mario
Machine Learning www.reddit.com
I'm trying to wrap my head around the index creation with Llama Index, mostly on the part with "embedding" the data.
As I see the embedding itself costs a number of tokens depending on the amount of data.
Does my data (e.g. file I'm indexing) is being exposed somewhere?
Thanks!
costs data embedding head index indexing llama machinelearning part tokens
More from www.reddit.com / Machine Learning
Jobs in AI, ML, Big Data
Founding AI Engineer, Agents
@ Occam AI | New York
AI Engineer Intern, Agents
@ Occam AI | US
AI Research Scientist
@ Vara | Berlin, Germany and Remote
Data Architect
@ University of Texas at Austin | Austin, TX
Data ETL Engineer
@ University of Texas at Austin | Austin, TX
Codec Avatars Research Engineer
@ Meta | Pittsburgh, PA