Scraping ArXiv Papers with OpenAI's GPT 3.5 — AI Assistant #2

I'm building an AI assistant that will *actually* be useful. This video covers the ArXiv paper scraping component, using a graph approach where each paper's references become "children" in the graph. We actually use LLMs (OpenAI's text-davinci-003 in this case) to transform the messy references into a clean format that we then use in a Google search and extract paper information from.

We're currently focusing on the AI assistant's ArXiv "researcher" component. The idea here is to have a literal …

ai assistant arxiv assistant become building case children extract format google google search gpt graph ideas information llms openai paper research research assistant scraping search text video