New York City's official AI chatbot is hallucinating incorrect legal advice

- The MyCity chatbot, launched as a pilot program, aims to assist business owners with information from NYC Business webpages.

- However, a report revealed the chatbot's inaccuracies in providing legal advice, such as misinformation on Section 8 vouchers and worker regulations.

- The chatbot's token-based predictive models can lead to incorrect responses due to gaps in training data.

- While warnings acknowledge the potential for harmful content, the chatbot is marketed as a tool to navigate government for business …

advice ai chatbot artificial business chatbot city however information legal misinformation new york city nyc pilot predictive predictive models regulations report token