Immediately when watching the video, the tool makes asphalt on the road a lot smoother, and cars a lot shinier and more reflective when compared to the original gameplay – not to mention causing major changes to colour grading. Of course, there’s still work to be done; as with most AI-generated imagery right now, there is visual smearing to be seen.
GTA 5 With a AI Photorealism Tool
By leveraging convolutional neural networks, Intel ISL’s photorealism tool allows it to produce images at “interactive rates” while playing the game – essentially, the tool could theoretically work in real-time as you play. It does remain theoretical, however – this is footage from an experiment using the tool, and It’s unclear when (or if) Intel ISL will release this tool for public use.
As for exactly how it all works, the Photorealism Enhancement tool selects from a database of photographic images, automatically picking those that bear similarity to the frame currently processed in-game, and using that comparison to render a more photorealistic style. Intel ISL was able to produce more life-like GTA V imagery by using a machine-learning database called Cityscapes, which features a slew of car video recordings from drivers cruising around cities in Germany.
Incidentally, that choice of database likely explains why there is a stark contrast in color compared to GTA 5’s usual look in much of the footage. Where GTA 5 features a vibrant, Los Angeles-inspired look, the Intel ISL render using Cityscapes is far more muted, presumably because the European-based images were taken in a colder, cloudier region. However, the research team does show how a different set of comparison images can generate a very different look later in the video.
This is by no means the only AI-related tool being built with gaming in mind, and some are coming sooner rather than later. During GDC last month, Intel announced Bleep, a new AI-powered tool that aims to filter out sexist, racial, and other hateful slurs you hear in voice chats while you game.