The first open source equivalent of OpenAI ChatGPT happened, but good luck running it on your laptop – or not at all.
This week, Philip Wang, the developer responsible for reverse engineering closed AI systems, including Meta Make a video, released PaLM+RLHF, a text generation model that behaves similarly to ChatGPT. The system combines Palma great language model from Google and a technique called Reinforcement Learning with Human Feedback – RLHF, for short – to create a system that can do just about any task ChatGPT can do, including writing emails. emails and computer code suggestion.
But PaLM+RLHF is not pre-trained. That is, the system has not been trained on the sample data from the web needed for it to actually work. Downloading PaLM+RLHF won’t magically install a ChatGPT-like experience – that would require compiling gigabytes of text the model can learn from and finding hardware powerful enough to handle the training workload.
Like ChatGPT, PaLM+RLHF is essentially a statistical tool for predicting words. When fed a large number of examples from training data – for example, Reddit posts, news articles and e-books – PaLM+RLHF learns the probability of words appearing based on patterns such as the semantic context of the surrounding text.
ChatGPT and PaLM+RLHF share a special sauce in reinforcement learning with human feedback, a technique that aims to better align language models with what users want them to accomplish. RLHF involves training a language model – in the case of PaLM + RLHF, PaLM – and fitting it on a dataset that includes prompts (e.g., “Explain machine learning to a six-year-old child” ) associated with what human volunteers expect from the model. say (e.g., “Machine learning is a form of AI…”). The aforementioned prompts are then passed to the refined model, which generates multiple responses, and volunteers rank all responses from best to worst. Finally, the rankings are used to form a “reward model” that takes the responses from the original model and sorts them in order of preference, filtering out the best responses to a given prompt.
It’s an expensive process, collecting training data. And the training itself is not cheap. PaLM is 540 billion parameters in size, with “parameters” referring to the parts of the language model learned from the training data. A 2020 to study pegged the development expense of a text generation model with only 1.5 billion parameters to $1.6 million. And to train the open source model Bloom, which has 176 billion parameters, it took three months with 384 Nvidia A100 GPUs; A single A100 costs thousands of dollars.
Running a trained model the size of PaLM + RLHF is not trivial either. Bloom requires a dedicated PC with approximately eight A100 GPUs. Cloud alternatives are expensive, with back-of-the-envelope calculations discovery the cost of running OpenAI’s text generation GPT-3 – which has around 175 billion parameters – on a single Amazon Web Services instance for around $87,000 per year.
AI researcher Sebastian Raschka points out in a LinkedIn Publish about PaLM+RLHF that scaling up the necessary development workflows might also prove to be a challenge. “Even if someone provides you with 500 GPUs to train this model, you still need to manage the infrastructure and have a software framework that can handle that,” he said. “It’s obviously possible, but it’s a big effort right now (of course, we’re developing frameworks to simplify this, but it’s not trivial, yet).”
That’s all to say that PaLM+ RLHF won’t replace ChatGPT today – unless a well-funded company (or person) bothers to train it and make it publicly available.
In better news, several other efforts to replicate ChatGPT are progressing at a rapid pace, including one led by a research group called CarperAI. In partnership with open AI research organization EleutherAI and startups Scale AI and Hugging Face, CarperAI plans to release the first out-of-the-box, ChatGPT-like AI model trained with human feedback.
LAION, the association that provided the initial dataset used to train Steady broadcastis also spearhead a project to replicate ChatGPT using the latest machine learning techniques. Ambitiously, LAION aims to create an “assistant of the future” – an assistant who not only writes emails and cover letters, but “does meaningful work, uses APIs, dynamically searches for information and much more “. It is in the early stages. But a GitHub page with resources for the project went live a few weeks ago.
0 Comments