How MuZero, AlphaZero and AlphaDev help optimize the entire computing ecosystem that powers our world of devices
Artificial intelligence (AI) algorithms are becoming more sophisticated every day, each designed to solve a problem in the best possible way. As part of our efforts to create general and increasingly capable AI systems, we are working to create AI tools with a broad understanding of the world, so that useful knowledge can be transferred between many different types of tasks.
Based on reinforcement learning, our AlphaZero and MuZero AI models have produced winning games with superhuman performance. Today, they are expanding their capabilities to help design better computer chips, optimize data centers and video compression – and more recently, our specialized version of AlphaZero, called AlphaDev, has discovered new algorithms that are already speeding up the software at the base of our digital system. Company.
While these tools drive efficiencies across the IT ecosystem, early results show the transformative potential of more general-purpose AI tools. Here we explain how these advances are shaping the future of computing and already helping billions of people and the planet.
Design better computer chips
Specialized hardware is essential to ensure that today’s AI systems are resource-efficient for large-scale users, and designing and producing new computer chips can take years of work. But today, our researchers have developed an AI-based approach to design more powerful and efficient circuits by treating a circuit like a neural network, thereby accelerating chip design and taking performance to new heights.
Neural networks are often designed to take user inputs and generate outputs, such as images, text or videos. Inside the neural network, edges connect to nodes in a graph-like structure. To create a circuit design, our team proposed “circuit neural networks,” a new type of neural network that turns edges into wires and nodes into logic gates, and learns to connect them together.
Then, we optimized the learned circuit in terms of calculation speed, energy efficiency, and size, while maintaining its functionality. We used “simulated annealing,” a classic research technique that looks one step into the future, testing different configurations in search of the most optimal. Thanks to this technique, we participated in the IWLS Programming Competition 2023 – and won – by obtaining the best solution to 82% of the competition’s circuit design problems.
Our team has also started to apply AlphaZero, which can look many steps into the future, improving the circuit design by treating the optimization challenge as a game to be solved. And so far, our research combining circuit neural networks with the reward function of reinforcement learning shows very promising results for building a future of even more advanced computer chips.
Optimization of data center resources
Data centers manage everything from delivering research results to processing datasets. Borg manages billions of tasks on Google, assigning these workloads is akin to a multidimensional game of Tetris. This system helps optimize tasks for internal infrastructure services, user-facing products such as Google Workspace and Search, and also handles batch processing.
Borg uses hand-coded rules to schedule tasks to manage this workload. At Google scale, these hand-coded rules cannot account for the diversity of ever-changing workload distributions. They are therefore designed as “one size fits all” rules. to the best suitable for everyone.” This is where machine learning technologies like AlphaZero are particularly useful: these algorithms are able to automatically create perfectly tailored individual rules that are more efficient for different workload distributions.
During the training, AlphaZero learned to recognize patterns of tasks coming into data centers and also learned to predict the best ways to manage capacity and make decisions with the best long-term results.
When we applied AlphaZero to Borg, experimental production trials showed that this approach could reduce the amount of underutilized hardware by up to 19%, thereby optimizing the use of Google’s data center resources.
Next steps for video compression
Video streaming accounts for the majority of internet traffic and consumes large amounts of data. So finding efficiencies in this process, whether big or small, will have a huge impact on the millions of people who watch videos every day.
Last year, we worked with YouTube to apply MuZero’s problem-solving capabilities to video compression and transmission. By reduce flow by 4%without compromising on visual quality, MuZero has improved the overall YouTube experience.
We initially applied MuZero to optimize the compression of each individual frame in a video. We have now extended this work to make decisions about how frames are grouped and referenced during encoding, providing further throughput savings.
Initial results from these first two steps show that MuZero has the potential to become a more generalized tool, helping to find optimal solutions for the entire video compression process.
Discover faster algorithms
More recently, AlphaDeva version of AlphaZero, has achieved a new breakthrough in computing, discovering faster sorting and hashing algorithms – two fundamental processes used billions of times a day to sort, store and retrieve data.
Sorting algorithms impact how all digital devices process and display information, from ranking online search results and social media posts to user recommendations. AlphaDev discovered an algorithm that increases the sorting efficiency of short sequences of elements by 70% and by approximately 1.7% for sequences of more than 250,000 elements, compared to algorithms in the C++ library. So when a user submits a search query, AlphaDev’s algorithm can help sort the results faster. When used on a large scale, it saves enormous amounts of time and energy.
AlphaDev also discovered a faster algorithm for hashing information, which is often used for data storage and retrieval, such as in a customer database. Hashing algorithms typically use a key (e.g. username “Jane Doe”) to generate a unique hash, which corresponds to the data values to be retrieved (e.g. “order number 164335-87”).
Like a librarian who uses a classification system to quickly find a specific book, with a hashing system, the computer already knows what it is looking for and where to find it. When applied to the range of 9 to 16 byte hash functions in data centers, AlphaDev’s algorithm improved efficiency by 30%.
Since the publication of sorting algorithms in the LLVM Standard C++ Library – replacement of subroutines used for over a decade with those generated by RL – and hashing algorithms in the callback libraryMillions of developers and businesses now use these algorithms in industries such as cloud computing, online shopping and supply chain management.
Versatile tools to power our digital future
From playing games to solving complex engineering problems at the heart of every device, our AI tools save billions of people time and energy. And that’s just the beginning.
We envision a future in which more versatile AI tools can help optimize the entire IT ecosystem that powers our digital world. But to support these tools, we will need a faster, more efficient and more sustainable digital infrastructure.
Many additional theoretical and technological advances are required to achieve fully generalized AI tools. When applied to various challenges in technology, science, and medicine, these types of general-purpose tools have the potential to be truly transformative. We’re excited about what’s on the horizon.
Learn more about sorting algorithms:
