OpenAI officially announced the first acquisition! Open source "Minecraft" 8-member top team joins, betting on AI agents
OpenAI's next step, run a multi-agent civilization simulation on GPT-5?
OpenAI officially announced its first acquisition ever.
Just now, OpenAI acquired Global illumination, a start-up company that produced an open source version of "Minecraft", for an unknown amount.
According to the announcement, the entire Global Illumination team will join OpenAI and focus on the development of ChatGPT in the future.
Founded by Thomas Dimson, Taylor Gordon and Joey Flynn, Global illumination is a company that uses AI to build creative tools, infrastructure and digital experiences.
What does OpenAI's acquisition imply?
In this regard, Jim Fan, a senior research scientist at Nvidia, expressed great curiosity: "What's the next step? Running a multi-agent civilization simulation on GPT-5? Maybe "My World" is all that is needed to build AGI? "
"Eight King Kong" joins OpenAI
Founded in 2021, Global Illumination is a New York-based digital products company.
Most recently, the company developed its most popular product, the open-source multiplayer online role-playing game "Biomes."
Project address: https://github.com/ill-inc/biomes-game
Historically, Global Illumination's work has focused on vision-oriented products and services.
In other words, OpenAI may be expanding more image/video functions for ChatGPT, or to build Dall-E 2 image generation services, or to launch a text-video product to compete with startup Runway's Gen- 2 Competition.
On the other hand, the company's naming is also different.
The term "Global Illumination" has been used in computer graphics for many years to describe algorithms for creating realistic lighting effects on 3D objects.
One of the original algorithms for global illumination was proposed in 1986 by Caltech researcher James Jim Kajiya.
The company consists of 8 members and 3 founders. What's interesting is that everyone's avatar on the official website uses pixel villains.
Some netizens said that this move is an excellent "acquisition" of OpenAI, through acquisitions to recruit new talents.
In the OpenAI announcement, the Global Illumination team is also listed to make contributions to famous companies such as Instagram, Facebook, YouTube, Google, Pixar, Riot Games, etc.
Let's take a look together, what is the strength of the team behind OpenAI from this acquisition?
Thomas Dimson is CEO of Global illumination.
Before starting the company, he spent most of his career on Instagram.
In 2013, Thomas joined Instagram as Chief Engineer and, 5 years later, was promoted to Director of Engineering.
He is known as the "No. 16 engineer" of Ins, "the original creator of the Instagram content ranking algorithm", and officially left in 2020.
During his 7 years at Ins, he has been instrumental in iterating on the platform's discovery algorithms, such as information ranking algorithms, exploration rankings, and story rankings.
He also invented products such as story poll stickers, Hyperlapse, emojineering, and was named one of the top 10 most creative people in business by Fast Company.
In an article written for Future last year, Thomas revealed that the team he led developed Instagram's personalized content ranking/recommendation system, replacing the default reverse chronological recommendation.
Taylor Gordon is the Chief Technology Officer of Global Illumination.
Taylor has over 20 years of programming experience, 8+ years of computer science education, and 10+ years of experience working at leading technology companies.
He has rich experience in many laboratories Facebook AI, Instagram, YouTube, Google, Microsoft.
In 2019, he was mainly doing computer vision research at Facebook AI.
Likewise, after joining Instagram in 2015, he led the development of the feed ranking, focusing on machine learning and ranking of infrastructure components.
In 2012, as a machine learning engineer at YouTube, he was mainly responsible for video recommendation and led the refactoring of the core infrastructure ranking infrastructure.
While at Microsoft, designed and built production algorithms for in-site links in Bing search results. At Google, responsible for internal core services and infrastructure development analytics, and monitoring software.
Joey Flynn is Chief Product Officer for Global Illumination.
He himself has also worked in Facebook and Ins, and is known as one of Facebook's top designers.
In 2010, he joined Facebook and led some important projects, such as Chat Heads, and Facebook's mobile software Home.
In 2014, Flynn conceived and built Slingshot, a Facebook messaging app.
Alexei Karpenko himself has worked at Facebook, Midnox, and Apple.
Andrew Top is the Technical Lead at Cobalt, the app platform behind the YouTube TV app, and has been with YouTube for 7 years.
In 2012, he joined the team as a graphics lead implementing their respective proprietary graphics API subsets of the GLES graphics API for PlayStation 3, PlayStation 4, and WiiU.
He also designed the Cobalt graphics rendering stack to enable faster animation frame rates on embedded devices than Chrome.
Andrew's other contributions include:
- Designed and implemented a cross-platform low-latency web-based solution to watch YouTube's 3D 360 spherical videos on TVs and VR headsets.
- Systems designed to monitor and analyze performance issues.
Nick Cooper has worked in Artifact, Ins, Google.
According to his homepage, Ian Silber has also worked in Artifact and Ins.
The only Chinese researcher in the team, who worked on the research of game animation at Riot Games and Pixar.
He is also very good at real-time rendering, physically based rendering, and Networked Gameplay.
Below are some of the projects designed by Brandon Wang.
The open source version of "My World" can be run with a browser
"Biomes" is an open source sandbox MMORPG (massively multiplayer online role-playing game) built for the web.
Just open the browser, you can build, gather, play mini-games and more.
As for OpenAI, in fact, as early as 2022, a brand new VPT "video pre-training model" was proposed, which allows AI to learn to build stone pickaxes from scratch in "Minecraft".
The entire process takes a hardcore player at least 20 minutes to complete, and a total of 24,000 operations are required.
Research address: https://openai.com/research/vpt
Specifically, the researchers first collected a wave of data annotation outsourcers playing games, including videos and records of keyboard and mouse operations.
Overview of the VPT method
Then, use these data to make an inverse dynamics model (inverse dynamics model, IDM), so as to infer how the keyboard and mouse move when each step in the video is in progress.
In this way, the whole task becomes much simpler, and only requires much less data than before to achieve the goal.
The influence of base model training data on fine-tuning
After 70,000 hours of training, OpenAI's behavioral cloning model was able to do things that other models couldn't.
For example, how to cut down trees to collect wood, how to use wood to make sticks, and how to use sticks to make tables. And this set of things requires a more skilled player to operate less than 50 seconds.
In addition to being a table, the model can swim, hunt, and eat.
There is even a show operation of "running, jumping and building", that is, when taking off, put a brick or wooden block under your feet, and you can build a pillar by jumping and jumping. This is a compulsory course for hardcore players.
"Run and Jump" simple version
Making a diamond pickaxe, which is more difficult, requires a long and complicated series of subtasks.
To make this task tractable, the researchers reward the agent for each item in the sequence.
They found that RL policies trained with traditional methods received almost no rewards, never learned to collect logs, and rarely collected sticks.
In stark contrast, the VPT model fine-tuned on human data can not only learn how to make a diamond pickaxe, but also achieve human-level success in collecting all items.
Make a Stone Pickaxe
In this regard, some netizens think that OpenAI Gym playgrounds is a "My World".
Before applying these systems in the real world, it's a good idea to practice them in a virtual world and see what happens. The speed of training may also be faster. Looking forward to seeing some amazing videos of Minecraft speeding up in a few months.
The engineering team of this startup comes from a high-level design team, and I think ChatGPT will be greatly upgraded soon.
Others think that from what Global Illumination has done. Their product capabilities are very much needed by OpenAI, but combined with the Stanford AI town a few days ago, this kind of sandbox game with high degree of freedom and low operating cost combined with AI will have infinite possibilities, so OpenAI may value their members The product capabilities may also be purely based on this high-degree-of-freedom game.