An AI that plays Minecraft for you? This Minecraft bot relies on AI and can improve itself
It sounds a bit like the story of a science fiction movie: An AI is created for reason X and has the ability to become smarter and smarter. In the mostly gloomy end-time flicks, such an AI is often the reason for the end of the world.
In reality, however, we are (still) spared terror AIs. Instead, an AI-controlled bot can automatically upgrade its abilities in Minecraft.
Minecraft + GPT-4
A team of Nvidia researchers led by (Linxi Jim
Fan) has developed an innovative way to integrate the GPT-4 language model into the popular brick game Minecraft.
The Minecraft bot, called Voyager, developed by Nvidia employees uses GPT-4 to solve problems in the game. The language model generates goals that help the bot explore the game, and generates code that improves the bot”s abilities over time.
What if we set GPT-4 free in Minecraft? ⛏️
I”m excited to announce Voyager, the first lifelong learning agent that plays Minecraft purely in-context. Voyager continuously improves itself by writing, refining, committing, and retrieving *code* from a skill library.
GPT-4 unlocks… pic.twitter.com/hjTxk6Qb1x
– Jim Fan (@DrJimFan) May 26, 2023
Voyager does not play Minecraft like a human, but uses an API to detect the current game state. For example, if he has a fishing rod in his inventory and discovers a river nearby, GPT-4 suggests that Voyager fish to gain experience
To accomplish this goal, GPT-4 generates the code necessary for the bot to complete the task.
You have to learn by yourself
The most innovative part of this project is the code generated by GPT-4. It adds new facets to Voyager”s behaviour. If the originally proposed code does not work optimally, Voyager refines it based on error messages, feedback from the game and a description of the code generated by GPT-4.
Over time, Voyager builds a extensive code library to continually tackle more complex tasks and further explore the game.
A graph created by the researchers shows that Voyager procures over three times as many items, explores more than twice as far and creates tools 15 times faster than other AI agents (Reflection, ReAct & AutoGPT).
According to Fan, the system could be further improved in the future by incorporating visual information from the game.
Language models and their bottomless potential
While chatbots like ChatGPT have impressed the world with their eloquent language and obvious knowledge, Voyager shows the enormous potential of language models to perform helpful actions on computers.
Applying language models in this way could potentially automate many routine office tasks, which could have a significant economic impact.
The connection between GPT-4 and Minecraft through the Voyager bot illustrates the exciting potential of language models, far beyond conducting conversations Automating tasks through language models in conjunction with games like Minecraft could open the door to a host of new applications.
From assisting with office tasks to developing intelligent software assistants that can automate tasks on PCs or phones, the possibilities are many.
What tasks would you like to have automated? Farming diamonds in Minecraft or would you rather do your taxes? Feel free to post it in the comments!
