A Paradigm Shift in ML Validation: Evaluating Workflows, Not Tasks | allainews.com

c

Aug. 20, 2023, midnight | Arjun Desai

stanford-crfm-website.github.io crfm.stanford.edu

Check out this post on the Hazy Research Blog!

blog check paradigm research research blog shift tasks validation workflows

More from crfm.stanford.edu / stanford-crfm-website.github.io

st

Massive Multitask Language Understanding (MMLU) on HELM 6 days, 17 hours ago | crfm.stanford.edu

helm language language understanding massive +2

st

Acceptable Use Policies for Foundation Models 4 weeks, 1 day ago | crfm.stanford.edu

developers foundation policies

st

HELM Instruct: A Multidimensional Instruction Following Evaluation Framework with Absolute Ratings 2 months, 2 weeks ago | crfm.stanford.edu

evaluation framework helm multidimensional +1

st

Plans for v1.1 of the Foundation Model Transparency Index: Self-Assessment 2 months, 4 weeks ago | crfm.stanford.edu

allen assessment board concrete +8

st

HELM lite: Lightweight and Broad Capabilities Evaluation 4 months, 2 weeks ago | crfm.stanford.edu

capabilities chatgpt evaluation helm +3

st

Towards compromise: A concrete two-tier proposal for foundation models in the EU AI Act 5 months ago | crfm.stanford.edu

st

Drawing Lines: Tiers for Foundation Models 5 months, 2 weeks ago | crfm.stanford.edu

st

Flash-Decoding for long-context inference 6 months, 3 weeks ago | crfm.stanford.edu

context decoding flash inference

st

Observations from HALIE: A Closer Look at Human-LM Interactions in Information-Seeking Contexts 6 months, 3 weeks ago | crfm.stanford.edu

closer look human information interactions +1

Lead Developer (AI)

@ Cere Network | San Francisco, US

View on ai-jobs.net

Research Engineer

@ Allora Labs | Remote

View on ai-jobs.net

Ecosystem Manager

@ Allora Labs | Remote

View on ai-jobs.net

Founding AI Engineer, Agents

@ Occam AI | New York

View on ai-jobs.net

AI Engineer Intern, Agents

@ Occam AI | US

View on ai-jobs.net

AI Research Scientist

@ Vara | Berlin, Germany and Remote

View on ai-jobs.net