April 22, 2023, 3:52 p.m. | /u/ultra_mario

Machine Learning www.reddit.com

Hello everyone,

I'm trying to wrap my head around the index creation with Llama Index, mostly on the part with "embedding" the data.

As I see the embedding itself costs a number of tokens depending on the amount of data.

Does my data (e.g. file I'm indexing) is being exposed somewhere?

Thanks!

costs data embedding head index indexing llama machinelearning part tokens

Founding AI Engineer, Agents

@ Occam AI | New York

AI Engineer Intern, Agents

@ Occam AI | US

AI Research Scientist

@ Vara | Berlin, Germany and Remote

Data Architect

@ University of Texas at Austin | Austin, TX

Data ETL Engineer

@ University of Texas at Austin | Austin, TX

Codec Avatars Research Engineer

@ Meta | Pittsburgh, PA