I can upscale pictures with great success.
Using AI it can fill in the gaps and make a convincing result. But what about videos?
Surely they’re just lots of pictures, put together. Wouldn’t it be great to be able
to convert all your old home videos and fraps recordings to 4K?
I’ve actually been experimenting with that since last August. And I think I might have
gone a little crazy. What started off as one or two projects has ballooned to hundreds.
Occasionally I’ll put them all in a folder somewhere, only to immediately start on something
new. Now I’ve got so many upscaled videos to show you that I don’t even know where
to begin. What I learned from all this is that upscaled
videos aren’t as convincing as upscaled pictures. When motion is a factor, it amplifies
any of the bugs and imperfections of upscaling. So while the upscaled video might look sharper,
or more detailed, it’s also so distracting to look at that you might be better off watching
the smoother, blurrier original instead. For video upscaling to work, you’d somehow
have to invent an AI that could intelligently carry details across frames to reduce the
spindly, shimmery nature of upscaled videos. And that’s exactly what the makers of Gigapixel
AI have done, with the creatively-named ‘Gigapixel AI for Video’. It was kind of separate from
the main site in a secret beta-stage for a while, but now you have to buy it, or stick
to a free 30 day trial. Alternatively, you can use this site to upload
up to 60 seconds of footage for free, and it’ll give you a side-by-side comparison
before and after. OR you can upload a full video and get it
upscaled up to 16x the resolution. You pay for this PER PIXEL! Plus 2 dollars.
Since I still have a 30 day trial, I can tell you a thing or two about it and how it performs.
For a start: you’ll want a powerful PC. I have a Geforce 2080 TI. And while that kind
of power is useful for gaming, I think I’ve actually been using it more for AI upscaling.
It can upscale low-resolution videos almost in realtime, but upscaling HD to 4K still
takes about an hour a minute. So you can only imagine how long it would take on less powerful
hardware. But any kind of automated video upscaling
is better than having to manually upscale thousands of pictures. I suggest you make
the most of the free 60 second upscaling they offer to test different resolutions to see
how the results will look before you invest the time or money into upscaling the whole
thing. I’ve been using this AI video upscaling
as an opportunity to return to some of my older videos and to see how much better I
can make them look. This is a bit of a trip down memory lane for me, and I’ll be releasing
these over the coming days in new, higher-resolution glory!
In this video I’ll quickly talk about each of them, the challenges involved with upscaling
them, and what I hoped to get out of it. I’ll also be sharing some tricks I used to improve
them in other ways, which might help you if you’re looking to restore some of your own
This is personally my favourite Going Low in CS:GO video that I’ve made. No narration
is required to tell a tale of pity and… romance? You’ll see for yourself when you
watch it. This video was already in full HD, 60 FPS. I chose to upscale this one to 4K.
My reason for choosing this one is for the intro sequence. I was only recording in 1080p
at the time, and this scene required a LOT of zooming, to the point where the end result
looked far blurrier than I’d have liked. And that’s a shame! In fact it was this
video that drove me to make future ones in 4K. Never could I have imagined that the day
would come where I’d be able to bring the details back to this video. But here we are.
The key to good upscaling with videos is the same as with pictures. You ideally want the
resolution you’re upscaling from to be as small and as crisp as possible. So because
the intro to this video was so zoomed in, I actually upscaled it several times at lower
resolutions, decided on which looked best for each scene and pieced those together back
into a full video. The result of this upscaling isn’t perfect.
Everything looks a bit painted in, and the chain link fences are still a mess. But I
feel I’ve done as well as I can, given what I had to work with.
GOING LOW IN CS:GO – CRAZY KNIFE DUDE And the same applies to this video, which
was the first to be uploaded in 1080p, 60 fps. I upscaled this one to 4K as well. The
Inferno map has brighter colours and more contrast, which I think results in a better
quality upscale even though the video itself is older, recorded way back in 2014. You can
tell because although it’s a gold nova game, everybody by today’s standards would be
low silver. Fascinating to see how things have changed, and how harshly older videos
can be judged by today’s players. In 2012, before Shadowplay was even a thing,
I filmed this trailer for de_elysium, a map for CS:GO that I made just months after the
game’s release. This was captured at 1080p, 30 FPS. Now this is a dilemma- do I keep it
at 30 FPS to retain maximum sharpness to show off the upscaling, or do I use frame-rate
conversion software to up it to 60 with nice motion blur? (Which I think will make the
video look more modern). Right now I’m thinking I’ll upload both and will let you decide
for yourselves which is best. But before any framerate conversion takes place, the upscaling
must come first. It always should. If you do the framerate conversion first then it
adds artefacts to all 60 frames, rather than to 30 which are then smoothly transitioned
between. So actually, I think framerate smoothing being applied after upscaling will help to
mask the upscaling effects and shimmer! I didn’t record this with anti-aliasing on.
I don’t know why. I knew what I was doing in 2012. I even had ambient occlusion forced
on in the Nvidia drivers! So while the result is nice and sharp, all of the jaggy edges
are upscaled to look… horrible and sharp. This presented me with a dilemma! Do I upscale
the 1080p original and remain aliased, or squish the original down to 540p, which will
make edges look smoother at the loss of some surface detail? I chose to use both- the 1080p
upscale with moving images to reduce shimmer, and 540p for the still images to cut down
on aliasing. Will any one notice? Probably not, but that’s 90% of my life in a nutshell.
The one new scene I added was just after the bomb explosion. I always regretted not having
some awesome far-out shot here so used this project as an opportunity to add one. Plus,
it’ll let you compare the rest of the video to a native 4K, 60 FPS shot. See if you can
spot the difference. Almost a decade ago I juggled up in the Lake
District. Some of it was captured at 50 FPS, others bits, 50 FPS interlaced. And this presents
another upscaling challenge! Interlaced videos only capture half the video’s
lines at once. Good software can try to join it together again, but the result is never
as good as if it had been recorded properly in the first place. Going slightly off-topic,
some time last year I upscaled all of the episodes of Peep Show and some of these suffer
badly from interlacing. I discovered that by resizing the image to half the vertical
resolution, and then upscaling that (and then squishing it back to normal shape again) it
preserved a LOT more detail than halving both dimensions, AND removed interlacing pixilation.
From all this work, I am proud owner of the finest quality Peep Show episodes in the world!
…and you can see non-copyrighted results in this upscaled juggling video. Although
the Pet Shop Boys song is still copyrighted. It’s from the era!
Now for something a little different. In 2011, an intro for Atomic Amnesia was made.
The show was going to be about naked old men floating around in space. I wrote a song for
it, my friend did some character art and we came up with some pretty amazing episodes
that we wanted to make… then life got in the way. And then Rick and Morty came along
and used such similar ideas it would have made it look like we were copying it. Whatever.
Here is the mock-up I made for the start of the first episode.
It seemed a bit desperate to upscale perfectly good video clips, so instead I just upscaled
the still images to bring the whole thing up to about the same level of quality. I just
saw this upscaling project as the best excuse I’ll ever have to release this video so,
after 9 years, I can finally get the project files off my hard-drive. If I was ever to
return to it I’d start it again from scratch any way. Now for something else a little different.
In 2008 I have what is one of the final things I did with my Counter-Strike Source clan.
We decided to make the GREATEST machinima ever. If you’d like to know, the story behind
Final Styricum Shower is that they discover a brilliant planet full of a liquid called
Styricum that people really like to shower in, so they build a huge cable going from
that planet to Earth’s and it results in world peace because everybody’s too busy
rubbing Styricum all over each other’s bodies. But for some reason, the pipe is cut! What
will happen?! I’m… not sure how the machinima was eventually going to explain this… but
that’s fine cos we only filmed this intro sequence any way.
You know how old black and white footage makes stuff look older than it actually was? I feel
the same about Counter-Strike Source videos. They were normally recorded in TERRIBLE quality,
and uploaded to youtube in 360p. But for whatever reason I did this one right by capturing it
in full 720p! …but unfortunately, I rendered it at the
wrong framerate of 25, hence all these horrible ghostly outlines around anything moving. But
I have a solution to that! How do you get rid of horrible motion blur? YOU ADD EVEN
MORE! I upped the framerate to 200 frames a second! …and then I crushed it back down
to 50 again, with resampling enabled. The result: higher framerate, and better motion
blur. I’m not saying this is the ideal solution. It’s probably far too much blur to be acceptable.
But I’m proud this botched solution, where I fought resampling with yet more resampling.
So there. Framerate conversion will result in a weird
‘morph’ when the camera suddenly changes angle- so I had to cut out those mutant frames
and stretch the clips either side. This means the action will slow down just before and
after a cut. I tried to stretch the scene with the least action to make it less noticeable,
and even at points for cinematic effect- but it’s amazing how your brain filters the
slow-down out. I guess it’s a bit like how you don’t see yourself blinking.
With all this work done, I then upscaled again to 4K. Remember how I said you should always
upscale first before upping the framerate? Yeah, didn’t take long for me to ignore
my own advice, did it. But in this case I did it to save time- that framerate conversion
took over 4 hours at 1080p- I just couldn’t be bothered with the time that 4K would have
taken. Now back to 2007. Back then everybody on the
messaging software, xfire, was addicted to Solitaire. A dark time indeed. In a cry for
help, I made this video. I’ll warn you now- there’s only so much you can do with something
of such limited quality. My room was dark. My camera, BAD. And when I compared the upscaled
result with the original, I was horrified to discover that it looked no better, other
than for the colour and contrast adjustments that I had made to it. But with the upscaling,
to get it to work effectively I had to crush the video down to such a small resolution
that it no longer had much information to work with! I was close to giving up with this
video completely, but by happy chance discovered that my upscaled my face looked terrifying-
though this is only when using the picture upscaling software. And I had to upscale each
frame of the video individually. But I’m proud of this video because it’s an example
of using upscaling to achieve something OTHER than better quality- I’m using it here to
make it look trippy and sinister. Every time my face was on show, I manually adjusted the
resolution to get the software to upscale my face in the most unflattering way possible.
Suddenly, this video has a new purpose to exist, and releasing it as a new stand-alone
video is justified! Now back even further, to Morrowind. I’m
not sure when exactly I made this video. The Bloodmoon expansion I used came out in 2003
and I was heavily inspired by The Two Towers, so I was probably 12. I hope I was. But I
might have been 13. Or 14. Or 16 I don’t know. Any way, this is an absolute worst-case
example for video upscaling. Low resolution, pixelated, dark, 10 FPS! It’s like upscaling
on HARD mode. I think, per-pixel, I have obsessed more over this video than any other. It’s
not even a good video! I just see it as a challenge to restore to the best quality I
can. I have run every frame through a jpeg-artefact-remover. I have tried blending several different upscales
together. I’ve tried anything that could possibly have helped- and more besides.
But after all that, I eventually decided just to run it through the upscaler and to be done
with it. In the future this video should be used to benchmark how good video restoration
software is. It’s got everything! It’s got snowy scenes- which gigapixel fills
in with lots of snowflakes which weren’t really there.
It’s got dark scenes, where it has about a dozen blocky squares to make a picture from.
It’s got high contrast outdoor scenes, but without enough detail to generate believable
textures from. And it’s all at 10 FPS. Here’s the problem
with framerate enhancing. It’s most needed on low framerate videos, but it’s low framerate
videos that it performs worst with. 25 FPS is bad enough, but the jumps between each
frame of a 10 fps video are so BIG that it has no choice but to do weird transformations
that look utterly nauseating, to the point where, although I see it as an overall improvement,
I know already that uploading it to Youtube at 60 fps will result in dozens of comments
from people saying it looks weird and that I should have kept it at 10 fps. And now I’ve
said this, it’ll lead to dozens of comments from people ironically saying it looks weird
and that I should have kept it at 10 fps. I know you so well. I was even thinking of
making it 60 FPS in the centre of the screen and having it fade to the 10 FPS original
around the sides, which is where the framerate conversion bugs are normally the most obvious.
But the contrast between a smooth 60 in the centre and a juddery 10 around the outside
was equally distracting. This video is hell to upscale. But that’s
also its appeal. I’d be better off just loading up Morrowind
and re-recording this entire project. But that would be cheating!
And that’s it! That’s as far back as my videos go. But in secret, another upscale
was made. No, not Lord of the Rings! I’m talking about the true Lord of the Rings.
A decade ago I worked with the Fart Master on a movie, but production stopped after some
disagreements about the plot. Because at some point you need something other than farts.
But I’ve been keeping tabs on this legend. He had a video scheduled to be released every
month, spanning centuries into the future! Clickbait channels had their fun with him.
They showcased his works and laughed at him. Here’s Chocolate Rain dude appreciating
his craft! But while others mocked him, to me he was
a misunderstood genius, ahead of his time. Unfortunately, in a bid to make Youtube more
child-friendly last year they sent him a hundred strikes within a matter of seconds, wiping
him from the site. Which is a shame, because he managed a 2 and
a half minute-long fart. The longest on record- more than twice as long as Mr Methane’s
best! So, out of respect for this legend… this rainman of the gaseous elements, I hand
upscaled each and every one of this video’s frames to 8K. And I think this video perfectly encapsulates
the beauty of upscaling. It’s to respect the past. To acknowledge videos which deserve
better than whatever format they find themselves bound by. It is for timeless content that
you want future generations to appreciate, long after those who recorded it are gone
from this world. It’s truly remarkable, and a testament to mankind’s drive to better
themselves. It takes my breath away. Thank you for your service, AI upscaling.