[D] Decision Transformer Alignment should be better than DeepMind ReST

We've done some experiments recently,

see the tech report: [https://arxiv.org/abs/2308.12050v1](https://arxiv.org/abs/2308.12050v1)

We train an SFT model and an RM model, then align the LLM with DT/MLE with filtering (ReST) + RM /SFT datasets/SFT model-generated samples

https://preview.redd.it/195op5q636lb1.png?width=1081&format=png&auto=webp&s=a9fa862e8a9ab05819484af8619f73d918fdc26a

DT is the Decision Transformer alignment

MLE is the ReST-like alignment

https://preview.redd.it/u6x28fook5lb1.png?width=1118&format=png&auto=webp&s=4a87898129c1238c00071d43809f5daf440b26d8

alignment datasets decision deepmind filtering generated llm machinelearning mle rest transformer