Artificial intelligence (AI) technologies have been deployed by the Rijksmuseum in the Netherlands to help restore one of Rembrandt’s best-known works of art some 300 years after it was cut down in size. The Night Watch was painted in 1642 by the Dutch master and it depicts a group scene of a militia. Often referred to as a masterpiece of the Dutch Golden Age of painting, the image is officially known as Militia Company of District II under the Command of Captain Frans Banninck Cocq. It was commissioned by Cocq as well as seventeen members of his militia group, known as Kloveniers. However, it has not been seen as it was originally intended by the commissioning group or the artist for over three centuries until recently.
By making extensive use of high-resolution photography of the image that remains in public ownership at the Rijksmuseum and matching it with similar photographs of a 17th-century copy of the full painting that hangs in the National Gallery in London, the Amsterdam-based museum had enough to go on to restore the work of art in all its former glory. Computer learning techniques were used to match Rembrandt’s brushstroke style in the original with the parts of the image that are possible to see in the copy. By using AI in this way, the four cut-down sides of the original have now been faithfully reproduced. Crucially, what the restoration team at the Rijksmuseum have done is to make anew what had been removed rather than to restore what was there but tarnished. This makes the project a truly remarkable one for AI technology as applied to old and often fragile works of art.
Cut Down to Size
After Rembrandt completed The Night Watch, it was initially displayed in the Groote Zaal, or Great Hall, part of Amsterdam’s Kloveniersdoelen complex of buildings. However, in 1715, the decision was made to shift the painting to a new location at Amsterdam’s Town Hall. Because the image, which is well over four metres wide, was too large to fit through a doorway on its journey, it was astonishingly cut down in order to fit. The painting was trimmed on all four sides with the most sizeable chunk of the image disappearing from its left-hand side including the faces of two militiamen.
One of the problems that needed to be overcome with the AI project was that the copy that was used to help put the picture back together again was on a much smaller scale than Rembrandt’s original. This meant much more than simply scaling up the copy, which was painted by Gerrit Lundens, but matching Rembrandt’s style in the original to the composition that Lundens had left behind. There was a danger that some of Lundens’ style would work its way into the original if the project had not been assisted by machine learning and incredibly finely detailed digital photography.
The director of the Rijksmuseum, Taco Dibbits, said that he thought the missing scenes that were taken from the almost contemporaneous copy of the original added to the historical perspective of The Night Watch. He said that as the image has traditionally been hung in the Rijksmuseum, it has become seared into the public’s collective memories. However, Dibbits went on to add that he thought it is now possible to understand that the composition, as it was originally intended by Rembrandt, is even more dynamic thanks to the reconstruction project.
“It is wonderful to now be able to view the composition as Rembrandt had intended to paint it,” he said. Thanks to the use of AI to mimic Rembrandt’s painterly style, Dibbits said that, in his view, the left and the bottom portions of the painting being restored created an empty space in the composition of the painting. This gives the militiamen depicted a space which they are marching towards and, consequently a greater sense of purpose.
“When the painting was cut,” Dibbits said, “[the lieutenants in it]… were repositioned in the centre of the composition.” However, according to the museum director, Rembrandt had chosen them to be off-centre quite deliberately so they appear to be moving. The loss of dynamism when the work of art was downsized is one of the biggest problems with the image as it has been seen for the last three centuries. According to Dibbits, it is the compositional genius of Rembrandt that had been most let down by the process of cutting it rather than the elements which had been edited out. With the AI process, all of these compositional shortcomings are no longer in evidence. Much of what has been added to the canvas may be darker tones and devoid of details but now they are added back in again, the highlighted sections of the image are much more full of movement and dynamism.
A Long History
Although the image is often thought to be that of a nighttime scene, this appears to have been caused by a thick varnish that was used to coat the painting sometime after it was completed. This varnish may have altered the tonal qualities of the painting but it helped to protect it in 1911 against an act of vandalism when someone tried to slash it. The varnish has long since been removed, however, allowing its subtle colours to be enjoyed much more.
In 2011, the Rijksmuseum used technology to help the image be seen better by deploying LED lighting. A restoration project began in 2019 which was conducted in public behind a glass screen. The following year, a 44.8 gigapixel image of the painting was put together from over 500 different still digital photographs of the canvas. This led to the reconstruction phase of the project that has just been completed.
How are museums using artificial intelligence, and is AI the future of museums?
About the author – Manuel Charr
Manuel Charr is a journalist working in the arts and cultural sectors. With a background in marketing, Manuel is drawn to arts organizations which are prepared to try inventive ways to reach new audiences.