Posted 02/22/2020

Even More Fun with Language Models

5.0

Quality

Update 7/19/2020:
Now using GPT-3 to generate write-ups and replies. Any generated text by GPT-3 should not be taken as investment advice. This is for entertainment purposes only. With GPT-3, we have manually provided the company names.


In Fun With Language Models, we explored recent advancements in Natural Language Processing and built a few language model generators to illustrate how machines are getting better at understanding language. We also hinted at how, in the background at CrowdCent, we are using these language models in various ways to turn natural language into actionable investment decisions.

In this forum category, we’re showing how we’re having Even More Fun With Language Models. For every post in this forum category, we’re using fine-tuned language models to generate batches of long-form investment write-ups. Some of the write-ups are about existing companies, but involve imaginary events and figures about them. Some of the write-ups are about completely fabricated companies, but include plausible people and events. For all cases, we do handpick the samples we like best from the model. It should go without saying that these posts and “investment ideas” are for entertainment purposes only!

To create the write-ups, the language model first creates or picks a company name after it is fed a “special token” to indicate a company name should follow. Then, the language model sees another special token to indicate that it should generate a company ticker related to that company name. After that, the language model can begin the full length write-up.

Over time, as we build newer and better language models, we will continue to post generated results to the forum. This will hopefully include adding in components to mimic a comments section, like generators creating responses to initial write-ups.

Special thanks to OpenAI, nshepperd, and huggingface for open source contributions.