s
Feb. 2, 2024, 2:23 a.m. |

Simon Willison's Weblog simonwillison.net

ChunkViz


Handy tool by Greg Kamradt to help understand how different text chunking mechanisms work by visualizing them. Chunking is an important part of preparing text to be embedded for semantic search, and thanks to this tool I've finally got a solid mental model of what recursive character text splitting does.


Via gkamradt/ChunkViz

ai embedded embeddings greg part recursive search semantic solid text them tool via work

AI Research Scientist

@ Vara | Berlin, Germany and Remote

Data Architect

@ University of Texas at Austin | Austin, TX

Data ETL Engineer

@ University of Texas at Austin | Austin, TX

Lead GNSS Data Scientist

@ Lurra Systems | Melbourne

Senior Machine Learning Engineer (MLOps)

@ Promaton | Remote, Europe

Senior Software Engineer, Generative AI (C++)

@ SoundHound Inc. | Toronto, Canada