RE/RS, Data Understanding - Foundations

About The Team The Data Understanding team is responsible for creating the high quality datasets and their quantized representation for OpenAI. This includes synthesizing data, building VQ representations, and processing, filtering, deduplication, quality control, and tokenization so it can be used effectively in big model training runs. About The Role We're looking to advance how OpenAI builds and understands pretraining data at scale. You'll treat data quality and curation as core research pro… Full description ↗