Reviving the Past: Deep Learning Transforms Old Videos into Vibrant Colors

Have you ever come across old black and white videos and wondered what they would look like if they were in color? With the advancement of deep learning, it is now possible to transform black and white videos into colored ones with stunning accuracy.

My new project is about colorizing black and white evergreen songs. The colors may not be perfect since this is not a studio level film roll remastering, but we can get the look and feel of how the video will be if shot with a color camera.

The process of colorizing old videos involves training deep neural networks to predict the colors of each pixel in a grayscale image. The neural network is trained on a large dataset of images to learn the relationship between the grayscale and color images. Once trained, the neural network can be used to colorize new grayscale images or videos.

One of the most popular methods for colorizing videos is to use a technique called "frame interpolation." This technique involves using a neural network to predict the colors of intermediate frames between two existing frames. This results in smoother transitions between frames and a more natural-looking colorization.

Another approach is to use a technique called "temporal coherence." This technique involves ensuring that the colorization of one frame is consistent with the colorization of the surrounding frames. This helps to prevent color flickering and produces a more visually pleasing result.

One of the benefits of using deep learning to colorize old videos is that it can be done automatically and in real-time. This makes it possible to colorize old movies and TV shows, bringing them to life in a way that was never before possible. It also allows us to see historical events and footage in a new light, with colors that were previously hidden.

However, there are also some limitations to this technology. One of the challenges is that the neural network may make errors in predicting the colors of certain objects or regions in the video. For example, if there are no reference images of a particular car model, the neural network may not be able to accurately predict its color. Another challenge is that the colorization process may introduce artifacts or noise into the video.

Despite these challenges, the technology for colorizing old videos using deep learning is rapidly advancing. It is now possible to produce colorized videos that are visually stunning and incredibly realistic. With further improvements to the technology, we may soon be able to see the world in a whole new way, with old videos and images transformed into vibrant and colorful representations of the past.

This is my first try,

Song: Kalyani Kalavani…
Movie: Anubhavangal Paalichakal

Here is the final out,

Movie: Shree 420 (1955)

Song: Mera Joota Hai Japani

Song: Saranamayyappa

Artist: K. J. Yesudas, Chorus
Album: Chembarathi

More videos will be available in below playlist,
Remastered Videos

Breaking Language Barriers: How Deep Learning Automates Language Conversion of Comics

My next project is about converting comics to my native language. Since manual work is a time consuming, we are discussing about automating this. So took my first comic, Doctor Strange(English), and lets talk about the steps involved in this.

Comics are a form of visual storytelling that have gained immense popularity over the years. However, one of the challenges in reading comics is that they are often published in a single language, making them inaccessible to people who don't understand that language. But with the help of deep learning, it is now possible to automate the language conversion of comics, making them accessible to a wider audience.

The process of automating language conversion of comics involves training deep neural networks to recognize and translate text from one language to another. The neural network is trained on a large dataset of comics and their translations to learn the relationship between the text and the images. Once trained, the neural network can be used to automatically translate the text in new comics.

One of the challenges in automating language conversion of comics is that the text is often integrated with the images. This means that the neural network needs to be able to recognize and extract the text from the images. One approach to addressing this challenge is to use Optical Character Recognition (OCR) technology, which can recognize and extract text from images.

Another challenge is that different languages may have different sentence structures and word orders, making it difficult for the neural network to accurately translate the text. To address this, the neural network can be trained on a larger dataset of translations to improve its accuracy.

One of the benefits of automating language conversion of comics is that it can be done automatically and in real-time. This makes it possible for publishers to translate their comics into multiple languages without the need for manual translation. It also makes comics more accessible to people who may not have access to translations, such as those living in remote areas or those with visual impairments.

However, there are also some limitations to this technology. One of the challenges is that the neural network may make errors in translating certain words or phrases, especially those with multiple meanings. Another challenge is that the translated text may not always fit seamlessly with the images, which can be distracting for readers.

Despite these challenges, the technology for automating language conversion of comics using deep learning is rapidly advancing. It is now possible to produce translated comics that are visually stunning and accurate in their translations. With further improvements to the technology, we may soon be able to enjoy comics in multiple languages, bringing new audiences to this beloved form of storytelling.

In conclusion, automating language conversion of comics using deep learning is a promising technology that has the potential to revolutionize the comic industry. By making comics more accessible to a wider audience, we can foster a greater appreciation for this unique form of storytelling and bring people together across language barriers.

This is the page we are going to translate,

Step 1: Detection of text, conversation: This involves passing the image through a deep learning model to identify all the balloons in it.

Step 2: OCRing, So we have the location of all text boxes with us. So we will take each of them and pass through it via an OCR. The quality of OCR depends on the quality of the source image, we can use Tesseract, Abby, Google or any OCR as needed.

So we will get something like this,

Step 3: Translation is the next step. We can use any kind of language translators available for translation.

At this point, the translation may not be perfect. So we cannot fully depend on a translator and s we need to tune it or we should create our own translation model for each type of comic and then do the translation. Different comics use different own kind of dialogues/phrases and content delivery, like for Amar Chitra Katha the English’s content will be different and for Marvel comics it will be exactly different. So we cannot convert them based on any available conversational models for a professional output, but automated translation works for the time being.

Step 4: Text Masking: So now we know where are out texts, so the next step is to mask all the texts in it. This uses AI models to detect text in the balloons that we detected, and remove them. OR just clear all the balloons which may not be perfect since balloons can be of any shape.

Step 5: Replace original text, we can now paste our translated contents over this bubbles.

So yes we have the comic ready.

Here I’m adding the converted comic (Just a couple of pages only.)

Original: Dr. Strange from Marvel

Translated: Dr. Strange From Marvel – Malayalam Translated

Here is one more,

This I did in a hurry (ignore the imperfect mask removal), but it does the job.

Original: Arjun Unicrystal
Translated: Arjun – Unicrystal – Malayalam Translated

So if you want to convert any comics can P.M me, all we need is a person (manual translator) who can verify the automatic translated content and do the correction.

Thanks for reading.

Reviving History: Sree Padmanabha Swamy Temple During 1895 Comes to Life in HD Color with Deep Learning Technology

One of the most amazing things about artificial intelligence is its ability to learn and improve through experience. This is especially true for deep learning, a subset of AI that uses neural networks to analyze and learn from data. Colorization and enhancement of images and videos is one area where deep learning has made significant strides, and it's all thanks to the power of AI.

Colorization involves adding color to grayscale images or videos, while enhancement refers to improving the overall quality of an image or video, such as sharpening or denoising. These tasks were traditionally done manually, requiring a lot of time and effort from professionals. But with the advent of deep learning, these tasks can now be done quickly and accurately.

One of the most popular methods for colorization and enhancement is using deep neural networks, which are trained on large datasets of images and videos. The neural networks learn to recognize patterns and relationships between the grayscale and color or degraded and enhanced versions of the same image. Once the neural network has been trained, it can be used to apply color or enhancement to new images or videos.

Another technique used for colorization is to use reference images, where a neural network is trained to learn the color distribution of similar images. This allows the neural network to predict the colors of the pixels in a grayscale image more accurately.

Deep learning has also made it possible to enhance images and videos in real-time, which is useful for applications such as video conferencing or live streaming. This involves using deep neural networks to remove noise, blur, or other imperfections from an image or video in real-time, producing a clearer and more visually pleasing result.

One of the benefits of using deep learning for colorization and enhancement is that it can be done automatically and at scale. This means that large amounts of images and videos can be colorized or enhanced quickly and with a high degree of accuracy. This also saves professionals a significant amount of time and effort, allowing them to focus on other tasks.

However, there are also some limitations to this technology. For example, if the neural network has not been trained on a specific object or scene, it may not be able to accurately colorize or enhance it. Additionally, the colorization or enhancement process may introduce artifacts or errors into the image or video.

Despite these challenges, deep learning AI has revolutionized the way we approach colorization and enhancement of images and videos. It has made it easier and more accessible for professionals to produce high-quality images and videos, while also offering new possibilities for real-time applications. As deep learning technology continues to improve, we can expect to see even more exciting developments in this area.

Colorization and enhancing is done by deep learning AI.

Original

After colorization

Source Credits:

Photograph of the gopura of the Sree Padmanabha Swamy Temple at Trivandrum, taken by Zachariah D’Cruz in the 1890s from the ‘Album of South Indian Views’ of the Curzon Collection.