About opt-1.3b
OPT was predominantly pretrained with English text, but a small amount of non-English data is still present within the training corpus via CommonCrawl. The model was pretrained using a causal language modeling (CLM) objective. OPT belongs to the same family of decoder-only models like GPT-3. As such, it was pretrained using the self-supervised causal language modedling objective.
For evaluation, OPT follows GPT-3 by using their prompts and overall experimental setup. For more details, please read the official paper.
Visit Official Website
Community Posts
Nothing to display
