Remote

Nazneen Rajani

@nazneenrajani@mastodon.social

mastodon 4.8.0-nightly.2026-10-06

Research Lead @huggingface@mastodon.social

https://www.nazneenrajani.com/

119 Followers

67 Following

5 Posts

Joined October 28, 2022

Nazneen Rajani @nazneenrajani@mastodon.social

· 45mo ago

Did you know there are other dialog agents like ChatGPT?

And what if I told you the secret sauce is IFT, RLHF, CoT, and SFT.

Wait, what? 😱

We explain each of these terms and why they are relevant to ChatGPT by comparing with 4 other dialog agents.

We discuss what's been done and what remains to be shown as next steps for these dialog agents.

So if you are curious, check out our blog: https://huggingface.co/blog/dialog-agents

Work done with the dream team Nathan Lambert Victor Sanh and Thomas Wolf 🤗

huggingface.co

What Makes a Dialog Agent Useful?

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Nazneen Rajani @nazneenrajani@mastodon.social

· 47mo ago

Replying to

@lasha_nlp we are using in-context learning for labeling, so I'd imagine that linguistic phenomena would not be straightforward for LLMs to label. But I think maybe chain-of-thought could possibly help here? 🤔

Nazneen Rajani @nazneenrajani@mastodon.social

· 47mo ago

Replying to

3 & 4: use LLMs for generating semantic labels and visual features. 🦭Interface showing high-error groups for distilbert evaluated on the yelp dataset. (a) ex from the dataset in the high-error groups (sorted by loss), (b) tokens in high-error groups relative to the entire eval set, (c) 2d viz of model embeddings showing groups of errors in color. Paper: https://arxiv.org/abs/2210.05839 Space: https://huggingface.co/spaces/nazneen/seal Screencast: https://vimeo.com/736659216

arXiv.org

SEAL : Interactive Tool for Systematic Error Analysis and Labeling

With the advent of Transformers, large language models (LLMs) have saturated well-known NLP benchmarks and leaderboards with high aggregate performance. However, many times these models systematically fail on tail data or rare groups not obvious in aggregate evaluation. Identifying such problematic data groups is even more challenging when there are no explicit labels (e.g., ethnicity, gender, etc.) and further compounded for NLP datasets due to the lack of visual features to characterize failur

Nazneen Rajani @nazneenrajani@mastodon.social

· 47mo ago

Replying to

Joint work with @liang_weixin , Lingjiao Chen, @mmitchell_ai and @james_y_zou